[gpz]

 cjakeman, on 12 December 2021 - 01:52 AM, said:

Seems a very straightforward rule, but I hope there will be an information message in the log (or perhaps there is already).

Currently no, but I can add it, good idea. For the vfs I've implemented a two-level logging mechanism, the default is that it logs only its initialization information, but a verbose, general file access logging can also be switched on. While this virtual-overwrite happens at mount time, yet it falls into the verbose category in my opinion, since several hundred such files might be logged because of this reason. For example I have a route that includes most of the original MSTS shape and texture files it references in its distribution package, and I would not necessarily like to see these flooding my log when I don't specifically need this information.

[conductorchris]

I am following this with a lot of interest and excitement. Thank you for your work on this.

Question: Might it be possible to combine more than one zip file for TRAINSET (or other folders). For example I might point toward a folder of circa 1972 freight cars and another folder of locomotives specific to a route. Rolling stock from both folders ends up as part of the profile, usable in the root.

Christopher

[gpz]

Yes, of course is possible, just add the folder containing the zips along with the desired mount point as one line into the vfs config file, and the system will enumerate all the zips it finds there, and auto-mounts them, combining everything into the common virtual file system the game will run in.

[Laci1959]

Hello.

Does this speed up loading as well? Especially with the hundreds of train Time Table traffic? Or is it another thread?

Sincerely, Laci1959

[gpz]

I would also be curious about the loading time differences of reading from archives vs. the pure filesystem, I haven't done any measurements yet. In this vfs uncompressing is done on demand at file loading time, and theoretically it needs some CPU work. However the bottleneck is always the disk access time there, so uncompressing a chunk maybe done while the CPU would wait for the disk anyway, so uncompressing the data on-the-fly is probably mostly free, it will not slow down anything. And there is an advantage, that less data must be transferred between the disk and the memory. And also for loading many files we must access the windows filesystem only once per archive, which is by the way theoretically an expensive operation. So loading from a zip might even speed up things. I don't know, measurements need to be done.

However I think the data transfer from the disk to memory is not the most important factor at timetables loading. The major part there is the pre-run phase, and that doesn't do any data loading, so I don't think this would largely affect the startup time of that particular game mode.

[Laci1959]

Thank you very much. I guessed, I was just hoping he would bring something to the kitchen.

[cjakeman]

 conductorchris, on 12 December 2021 - 07:43 AM, said:

I am following this with a lot of interest and excitement. Thank you for your work on this.

You can try it for yourself as this feature is now included in the Unstable Version. Just update your installation.

If you find any problems, please report them of course.

[Genma Saotome]

 gpz, on 12 December 2021 - 03:13 AM, said:

Currently no, but I can add it, good idea. For the vfs I've implemented a two-level logging mechanism, the default is that it logs only its initialization information, but a verbose, general file access logging can also be switched on. While this virtual-overwrite happens at mount time, yet it falls into the verbose category in my opinion, since several hundred such files might be logged because of this reason. For example I have a route that includes most of the original MSTS shape and texture files it references in its distribution package, and I would not necessarily like to see these flooding my log when I don't specifically need this information.

Peter, perhaps the answer is a second log file dedicated to only reporting the occurrence of duplicate files. It only needs to be looked at after adding a new zip to the stew and with that data being dumping into a second file the first log remains properly terse.

[Genma Saotome]

Another thought of file duplicates. I suspect the only place where this might become a real problem is within \trainset. There, so many of us fiddle with values that having a Johnny-come-lately overwrite our work w/o telling us would be problematic for many.

Aside from the suggestion I made in the previous post there is something else that might help in this situation: Adding the name of whomever created or last edited the .wag or .eng to the path. I suggested that once before for use in the common.inc\ tree. It might be too burdonsome to manage on the receiving end as those files are referenced in .con and activity files but the problem of junk vales superseding those made by hard work will be a real one.

p.s. perhaps allowing a user to add a parameter to certain files to indicate always use this one and ignore any duplicates would be the easiest solution. Sort of like sticking a tack in it so it stays put.

[gpz]

I'd rather extend the number of log levels to 3 intead of introducing a second logfile. All right, I will make the change that the default fist logging level will output only the mounting information and the possible problems (warnings) it faces, the second log level will output the mount-time virtual file operations such as the overwrites, and the third level will also output the load- and runtime file operations such as reads and writes.

"Sticking" a file is actually a valid file operation, that could be supported in the virtual file system. It is similar to the windows read-only attribute on a file. When you set it, the file cannot be ovewritten. (At least not without a warning.) However reading into the files at mount-time would slow down the operation by an order of magnitude, so I could rather imagine this as properly utilizing the individual files read-only attribute inside the zip. When a file is set read-only, subsequent zips would be prevented to "overwrite" this file. But in this case an opposite operation needed to be defined, that does delete this read-only flag in a subsequent zip, to allow the original author to amend his own file. For example, I set a file to read-only in my coolpackage_v1.0.zip, and I would yet like to overwrite it in coolpackage_v1.1.zip, so I need to delete this read-only flag. How to do that without modifying the v1.0 package, any ideas?

All in all, I would outrule anything that involves reading into the files generally at mount-time, so all we have, to work with, is the filename and the flag. But I am not outruling to read into exactly one file per zip, which file could contain some metadata about the package. But at this time I don't know what would be the best dataset and format to use. This leads us to the swampy soil of defining the OR package format, that can properly handle the OR file formats, which don't yet exist.