[gpz]

Chris, Thank you for trying this out! Yes, I relabelled the "Choose" button to "Dir" to save space, as you guessed. :-) I'm actually not sure whether the current UI change can remain permanently or some better alignment should be implemented. I'm also unsure about the mount/config file format. I was reluctant to implement a json format file for this simple purpose of storing simple string pairs. It must be decided if this is all right.

The name of the Zip archive need not match the folder it contains, at least not by my intention. I have unmatching Zip files as a testbed, and they work fine on my computer.

You might also be interested in trying the other mounting feature out I find quite comfortable during my tests: Add a line that mounts only an almost emply directory, like this:

C:\_tmp\TrainSimDir\ /MSTS/

And just copy a zip there to its appropriate directory to let it auto-mounted. For example copy your Trains.zip to C:\_tmp\TrainSimDir\TRAINS\ and see as it getting auto-mounted without explicitly mentioning it in the mount config file.

Cheers,

[Genma Saotome]

Q: Will it be possible to separate the activity files from the rest of a route package while sharing the route package into several locations? Something like this:

/ one moniroute <--- miniroute specific activity folders

route package

\ a different miniroute <--- other miniroute specific activity folders

Hanging a loaded zip of .con files and another one of .wags and .engs into each of those mini-routes would make the idea complete.

I've long wanted to have era-specific equipment referenced in the activity files w/o having to physically duplicate the route itself.



A potential problem to look at: What happens when two different .zips provide an identically named file (or folder) to a location where the duplicated file came from two people unaware of the duplication. This could very easily happen in \global\shapes and anywhere in \]trains.

[gpz]

Genma Saotome, on 11 December 2021 - 03:08 PM, said:

Q: Will it be possible to separate the activity files from the rest of a route package while sharing the route package into several locations? Something like this:

/ one moniroute <--- miniroute specific activity folders

route package

\ a different miniroute <--- other miniroute specific activity folders

Hanging a loaded zip of .con files and another one of .wags and .engs into each of those mini-routes would make the idea complete. I've long wanted to have era-specific equipment referenced in the activity files w/o having to physically duplicate the route itself.

It is possible with two different vfs-config files, e.g.:

C:\packages\MSTS1.2.zip /MSTS/
C:\packages\route-package.zip /MSTS/ROUTES/
C:\packages\first-activity-package.zip /MSTS/ROUTES/ROUTE-IN-QUESTION/ACTIVITIES/

C:\packages\MSTS1.2.zip /MSTS/
C:\packages\route-package.zip /MSTS/ROUTES/
C:\packages\second-activity-package.zip /MSTS/ROUTES/ROUTE-IN-QUESTION/ACTIVITIES/

And both of these files can be added as "profiles" in the options, so they are selectable (one or the other). This can also be done with the con, wag, eng packages. One has to create as many vfs-config files as many different combinations of them are needed, and all of them may reference the same commonly needed zip files, while referencing only their own specific ones. The vfs subsystem will merge the contents of all the zip files mentioned in a config file to one common virtual directory structure where the game will run as if they were residing on a disk in uncompressed form.

Genma Saotome, on 11 December 2021 - 03:08 PM, said:

A potential problem to look at: What happens when two different .zips provide an identically named file (or folder) to a location where the duplicated file came from two people unaware of the duplication. This could very easily happen in \global\shapes and anywhere in \]trains.

The order of the rows in the vfs-config file does matter: the latter wins. If a zip contains a file, and another zip contains the same file with different contents, then the later standing zip in the vfs-config file will "overwrite" the prior's corresponding files. Exactly as if they were unzipped in the written order with overwrite-all option.

[cjakeman]

gpz, on 12 December 2021 - 12:50 AM, said:

The order of the rows in the vfs-config file does matter: the latter wins. If a zip contains a file, and another zip contains the same file with different contents, then the later standing zip in the vfs-config file will "overwrite" the prior's corresponding files. Exactly as if they were unzipped in the written order with overwrite-all option.

Seems a very straightforward rule, but I hope there will be an information message in the log (or perhaps there is already).

[cjakeman]

gpz, on 11 December 2021 - 12:15 PM, said:

The name of the Zip archive need not match the folder it contains, at least not by my intention. I have unmatching Zip files as a testbed, and they work fine on my computer.

I've done a little more testing of the names and you were right, of course. I must have been confusing myself.

Anyway, proceeding now to try out some of the more exciting elements of this new feature.

[gpz]

cjakeman, on 12 December 2021 - 01:52 AM, said:

Seems a very straightforward rule, but I hope there will be an information message in the log (or perhaps there is already).

Currently no, but I can add it, good idea. For the vfs I've implemented a two-level logging mechanism, the default is that it logs only its initialization information, but a verbose, general file access logging can also be switched on. While this virtual-overwrite happens at mount time, yet it falls into the verbose category in my opinion, since several hundred such files might be logged because of this reason. For example I have a route that includes most of the original MSTS shape and texture files it references in its distribution package, and I would not necessarily like to see these flooding my log when I don't specifically need this information.

[conductorchris]

I am following this with a lot of interest and excitement. Thank you for your work on this.

Question: Might it be possible to combine more than one zip file for TRAINSET (or other folders). For example I might point toward a folder of circa 1972 freight cars and another folder of locomotives specific to a route. Rolling stock from both folders ends up as part of the profile, usable in the root.

Christopher

[gpz]

Yes, of course is possible, just add the folder containing the zips along with the desired mount point as one line into the vfs config file, and the system will enumerate all the zips it finds there, and auto-mounts them, combining everything into the common virtual file system the game will run in.

[Laci1959]

Hello.

Does this speed up loading as well? Especially with the hundreds of train Time Table traffic? Or is it another thread?

Sincerely, Laci1959

[gpz]

I would also be curious about the loading time differences of reading from archives vs. the pure filesystem, I haven't done any measurements yet. In this vfs uncompressing is done on demand at file loading time, and theoretically it needs some CPU work. However the bottleneck is always the disk access time there, so uncompressing a chunk maybe done while the CPU would wait for the disk anyway, so uncompressing the data on-the-fly is probably mostly free, it will not slow down anything. And there is an advantage, that less data must be transferred between the disk and the memory. And also for loading many files we must access the windows filesystem only once per archive, which is by the way theoretically an expensive operation. So loading from a zip might even speed up things. I don't know, measurements need to be done.

However I think the data transfer from the disk to memory is not the most important factor at timetables loading. The major part there is the pre-run phase, and that doesn't do any data loading, so I don't think this would largely affect the startup time of that particular game mode.

[Laci1959]

Thank you very much. I guessed, I was just hoping he would bring something to the kitchen.

[cjakeman]

conductorchris, on 12 December 2021 - 07:43 AM, said:

I am following this with a lot of interest and excitement. Thank you for your work on this.

You can try it for yourself as this feature is now included in the Unstable Version. Just update your installation.

If you find any problems, please report them of course.

[Genma Saotome]

gpz, on 12 December 2021 - 03:13 AM, said:

Currently no, but I can add it, good idea. For the vfs I've implemented a two-level logging mechanism, the default is that it logs only its initialization information, but a verbose, general file access logging can also be switched on. While this virtual-overwrite happens at mount time, yet it falls into the verbose category in my opinion, since several hundred such files might be logged because of this reason. For example I have a route that includes most of the original MSTS shape and texture files it references in its distribution package, and I would not necessarily like to see these flooding my log when I don't specifically need this information.

Peter, perhaps the answer is a second log file dedicated to only reporting the occurrence of duplicate files. It only needs to be looked at after adding a new zip to the stew and with that data being dumping into a second file the first log remains properly terse.

[Genma Saotome]

Another thought of file duplicates. I suspect the only place where this might become a real problem is within \trainset. There, so many of us fiddle with values that having a Johnny-come-lately overwrite our work w/o telling us would be problematic for many.

Aside from the suggestion I made in the previous post there is something else that might help in this situation: Adding the name of whomever created or last edited the .wag or .eng to the path. I suggested that once before for use in the common.inc\ tree. It might be too burdonsome to manage on the receiving end as those files are referenced in .con and activity files but the problem of junk vales superseding those made by hard work will be a real one.

p.s. perhaps allowing a user to add a parameter to certain files to indicate always use this one and ignore any duplicates would be the easiest solution. Sort of like sticking a tack in it so it stays put.

[gpz]

I'd rather extend the number of log levels to 3 intead of introducing a second logfile. All right, I will make the change that the default fist logging level will output only the mounting information and the possible problems (warnings) it faces, the second log level will output the mount-time virtual file operations such as the overwrites, and the third level will also output the load- and runtime file operations such as reads and writes.

"Sticking" a file is actually a valid file operation, that could be supported in the virtual file system. It is similar to the windows read-only attribute on a file. When you set it, the file cannot be ovewritten. (At least not without a warning.) However reading into the files at mount-time would slow down the operation by an order of magnitude, so I could rather imagine this as properly utilizing the individual files read-only attribute inside the zip. When a file is set read-only, subsequent zips would be prevented to "overwrite" this file. But in this case an opposite operation needed to be defined, that does delete this read-only flag in a subsequent zip, to allow the original author to amend his own file. For example, I set a file to read-only in my coolpackage_v1.0.zip, and I would yet like to overwrite it in coolpackage_v1.1.zip, so I need to delete this read-only flag. How to do that without modifying the v1.0 package, any ideas?

All in all, I would outrule anything that involves reading into the files generally at mount-time, so all we have, to work with, is the filename and the flag. But I am not outruling to read into exactly one file per zip, which file could contain some metadata about the package. But at this time I don't know what would be the best dataset and format to use. This leads us to the swampy soil of defining the OR package format, that can properly handle the OR file formats, which don't yet exist.