Elvas Tower: Overall File System and Data Structure - Elvas Tower

Jump to content

Posting Rules

All new threads will be started by members of the Open Rails team, Staff, and/or Admins. Existing threads started in other forums may get moved here when it makes sense to do so.

Once a thread is started any member may post replies to it.
  • 17 Pages +
  • « First
  • 12
  • 13
  • 14
  • 15
  • 16
  • Last »
  • You cannot start a new topic
  • You cannot reply to this topic

Overall File System and Data Structure Rate Topic: ***** 1 Votes

#131 User is offline   gpz 

  • Superintendant
  • Group: Status: Elite Member
  • Posts: 1,772
  • Joined: 27-October 12
  • Gender:Male
  • Location:Budapest
  • Simulator:OpenRails
  • Country:

Posted 05 December 2021 - 08:28 AM

Two basic file operations are missing from this PR yet: DELETE and LINK inside the VFS.

Let's take an example of the CoolPackage_1.0.zip having the following entries:
/ROUTES/USA85/TEXTURES/test.ace
/ROUTES/USA85/TEXTURES/reallycool.ace


It comes to the package creator's mind, that he made a mistake, because the test.ace causes problems there for some reason, and he also needs the "boring.ace" from the "boring" route, but with name "notsoboring.ace", and he doesn't want to redistribute that file, only link it from the original place.

A possible solution is that the VFS would allow two special entry types in an archive, like to following:
CoolPackage_1.1.zip would contain the following entries:
/ROUTES/USA85/TEXTURES/test.ace.or-delete
/ROUTES/USA85/TEXTURES/notsoboring.ace.or-link

with the latter one containing the following text:
/MSTS/ROUTES/BORING/BORING.ACE


Then if both packages were mounted to the /MSTS/ mountpoint, the latter package could make the necessary changes:
"C:\downloadedpackages\BoringRoute.zip\Train Simulator\" /MSTS/
C:\downloadedpackages\CoolPackage_1.0.zip /MSTS/
C:\downloadedpackages\CoolPackage_1.1.zip /MSTS/


But the format of these special files needs to be decided by the OR community, (If such features are necessary at all, of course.) I haven't implemented any of these "proposed" features yet in my PR.

#132 User is offline   Genma Saotome 

  • Owner Emeritus and Admin
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • Group: ET Admin
  • Posts: 15,350
  • Joined: 11-January 04
  • Gender:Male
  • Location:United States
  • Simulator:Open Rails
  • Country:

Posted 05 December 2021 - 12:59 PM

A couple of real world practices:

What happens if a route name has been edited to something else?
What happens when you decide the boring route really is boring and so you delete it to save disk space?
What happens when you install something in a check-it-out-first area before moving it to where it will be used?
What does one do with multiple occurrences of the same route? This could be \development, \testing, and \release or it could be \1930-52, \1953-65, and \1966-99?

I expect many of the answers will be procedural but to whatever degree that's not all cases will be an issue.

#133 User is offline   gpz 

  • Superintendant
  • Group: Status: Elite Member
  • Posts: 1,772
  • Joined: 27-October 12
  • Gender:Male
  • Location:Budapest
  • Simulator:OpenRails
  • Country:

Posted 05 December 2021 - 11:25 PM

 Genma Saotome, on 05 December 2021 - 12:59 PM, said:

What happens if a route name has been edited to something else?

A player user doesn't edit route names (I assume you are mentioning directory names, as that is the only thing matters), but if the downloaded newer package has the route directory renamed, they can still mount the route to its original directory name by modifying the mount configuration file, without altering the downloaded archive.

 Genma Saotome, on 05 December 2021 - 12:59 PM, said:

What happens when you decide the boring route really is boring and so you delete it to save disk space?

Obviously, it is the scope of your backup software. :-) But answering your real question: A good Package Manager Software's role is to reliably handle dependecies. However the VFS is not a package manager, it is just providing the background for developing one.

 Genma Saotome, on 05 December 2021 - 12:59 PM, said:

What happens when you install something in a check-it-out-first area before moving it to where it will be used?

Just add a line to your VFS mounting configuration file, like
C:\check-it-out-first\just-downloaded.zip /MSTS/

and try it out.

 Genma Saotome, on 05 December 2021 - 12:59 PM, said:

What does one do with multiple occurrences of the same route? This could be \development, \testing, and \release or it could be \1930-52, \1953-65, and \1966-99?

You may have multiple mounting configuration files as "profiles", with each one referencing a different directory as your route base dir, while having the rest of the references the same. Only alter the following line in the 3 configuration files while leaving everything else intact:
C:\MSTS-Route-Development\MyRoute\1930-52\ /MSTS/ROUTES/MYROUTE/


#134 User is offline   Genma Saotome 

  • Owner Emeritus and Admin
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • Group: ET Admin
  • Posts: 15,350
  • Joined: 11-January 04
  • Gender:Male
  • Location:United States
  • Simulator:Open Rails
  • Country:

Posted 05 December 2021 - 11:49 PM

Thank you for the answers. This is going to be interesting.

#135 User is offline   cjakeman 

  • Vice President
  • PipPipPipPipPipPipPipPip
  • Group: ET Admin
  • Posts: 2,867
  • Joined: 03-May 11
  • Gender:Male
  • Location:Peterborough, UK
  • Simulator:Open Rails
  • Country:

Posted 11 December 2021 - 09:09 AM

 gpz, on 04 December 2021 - 06:54 AM, said:

Since I am not a software architect, and I wrote this whole thing from the top of my head, a deep review would be very welcome.

Anyone who can write this whole thing from the top of his head is very talented. (76 files changed because it's pretty fundamental.)

I took your code diff today for a look-see. Here's what I did, in case anyone else wants to do the same:


Attached Image: 2021-12-11 17_08_24-MS Excel with extensions - dev gpz.xlsx  Compatibility Mode.jpg

I spotted the new button "File". (The old Change ... is now labelled Dir, presumably to save space).

I created a configuration file for this File button to read. My file was at "D:\OR\VFS\MSTS\VFS.ini" but I think the path and filename could be anything at all.


I added 3 statements to the VFS.ini file which mounted Zip archives to the virtual filesystem:

Attached Image: 2021-12-11 16_46_14-MS Excel with extensions - dev gpz.xlsx  Compatibility Mode.jpg
(I've added comments on the right as I don't expect comments directly in the file are supported yet.)

I used Peter Newell's CTN Test Route as an example because it's small.I split it into 3 Zip archives just to see if that would work - and it did, perfectly.

It was very nice to be using Zip archives instead of uncompressed folders. So much simpler (and smaller).

Note that the names of the Zip archives must match the folder they contain. Not sure if this is a permanent restriction.

After this set up, I have just scratched the surface, so will be exploring the functionality (and the code) much further. But I hope I've "broken the ice" and encouraged a few other people to explore the possibilities.


P.S. Peter's work is not yet included in the Unstable Version of Open Rails, so if anyone wants to know how to get that code running on their PC, please send me a PR.

#136 User is offline   gpz 

  • Superintendant
  • Group: Status: Elite Member
  • Posts: 1,772
  • Joined: 27-October 12
  • Gender:Male
  • Location:Budapest
  • Simulator:OpenRails
  • Country:

Posted 11 December 2021 - 12:15 PM

Chris, Thank you for trying this out! Yes, I relabelled the "Choose" button to "Dir" to save space, as you guessed. :-) I'm actually not sure whether the current UI change can remain permanently or some better alignment should be implemented. I'm also unsure about the mount/config file format. I was reluctant to implement a json format file for this simple purpose of storing simple string pairs. It must be decided if this is all right.

The name of the Zip archive need not match the folder it contains, at least not by my intention. I have unmatching Zip files as a testbed, and they work fine on my computer.

You might also be interested in trying the other mounting feature out I find quite comfortable during my tests: Add a line that mounts only an almost emply directory, like this:
C:\_tmp\TrainSimDir\ /MSTS/

And just copy a zip there to its appropriate directory to let it auto-mounted. For example copy your Trains.zip to C:\_tmp\TrainSimDir\TRAINS\ and see as it getting auto-mounted without explicitly mentioning it in the mount config file.

Cheers,

#137 User is offline   Genma Saotome 

  • Owner Emeritus and Admin
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • Group: ET Admin
  • Posts: 15,350
  • Joined: 11-January 04
  • Gender:Male
  • Location:United States
  • Simulator:Open Rails
  • Country:

Posted 11 December 2021 - 03:08 PM

Q: Will it be possible to separate the activity files from the rest of a route package while sharing the route package into several locations? Something like this:


/ one moniroute <--- miniroute specific activity folders

route package
\ a different miniroute <--- other miniroute specific activity folders


Hanging a loaded zip of .con files and another one of .wags and .engs into each of those mini-routes would make the idea complete.

I've long wanted to have era-specific equipment referenced in the activity files w/o having to physically duplicate the route itself.



A potential problem to look at: What happens when two different .zips provide an identically named file (or folder) to a location where the duplicated file came from two people unaware of the duplication. This could very easily happen in \global\shapes and anywhere in \]trains.

#138 User is offline   gpz 

  • Superintendant
  • Group: Status: Elite Member
  • Posts: 1,772
  • Joined: 27-October 12
  • Gender:Male
  • Location:Budapest
  • Simulator:OpenRails
  • Country:

Posted 12 December 2021 - 12:50 AM

 Genma Saotome, on 11 December 2021 - 03:08 PM, said:

Q: Will it be possible to separate the activity files from the rest of a route package while sharing the route package into several locations? Something like this:

/ one moniroute <--- miniroute specific activity folders
route package
\ a different miniroute <--- other miniroute specific activity folders

Hanging a loaded zip of .con files and another one of .wags and .engs into each of those mini-routes would make the idea complete. I've long wanted to have era-specific equipment referenced in the activity files w/o having to physically duplicate the route itself.

It is possible with two different vfs-config files, e.g.:
C:\packages\MSTS1.2.zip /MSTS/
C:\packages\route-package.zip /MSTS/ROUTES/
C:\packages\first-activity-package.zip /MSTS/ROUTES/ROUTE-IN-QUESTION/ACTIVITIES/

C:\packages\MSTS1.2.zip /MSTS/
C:\packages\route-package.zip /MSTS/ROUTES/
C:\packages\second-activity-package.zip /MSTS/ROUTES/ROUTE-IN-QUESTION/ACTIVITIES/

And both of these files can be added as "profiles" in the options, so they are selectable (one or the other). This can also be done with the con, wag, eng packages. One has to create as many vfs-config files as many different combinations of them are needed, and all of them may reference the same commonly needed zip files, while referencing only their own specific ones. The vfs subsystem will merge the contents of all the zip files mentioned in a config file to one common virtual directory structure where the game will run as if they were residing on a disk in uncompressed form.

 Genma Saotome, on 11 December 2021 - 03:08 PM, said:

A potential problem to look at: What happens when two different .zips provide an identically named file (or folder) to a location where the duplicated file came from two people unaware of the duplication. This could very easily happen in \global\shapes and anywhere in \]trains.

The order of the rows in the vfs-config file does matter: the latter wins. If a zip contains a file, and another zip contains the same file with different contents, then the later standing zip in the vfs-config file will "overwrite" the prior's corresponding files. Exactly as if they were unzipped in the written order with overwrite-all option.

#139 User is offline   cjakeman 

  • Vice President
  • PipPipPipPipPipPipPipPip
  • Group: ET Admin
  • Posts: 2,867
  • Joined: 03-May 11
  • Gender:Male
  • Location:Peterborough, UK
  • Simulator:Open Rails
  • Country:

Posted 12 December 2021 - 01:52 AM

 gpz, on 12 December 2021 - 12:50 AM, said:

The order of the rows in the vfs-config file does matter: the latter wins. If a zip contains a file, and another zip contains the same file with different contents, then the later standing zip in the vfs-config file will "overwrite" the prior's corresponding files. Exactly as if they were unzipped in the written order with overwrite-all option.

Seems a very straightforward rule, but I hope there will be an information message in the log (or perhaps there is already).

#140 User is offline   cjakeman 

  • Vice President
  • PipPipPipPipPipPipPipPip
  • Group: ET Admin
  • Posts: 2,867
  • Joined: 03-May 11
  • Gender:Male
  • Location:Peterborough, UK
  • Simulator:Open Rails
  • Country:

Posted 12 December 2021 - 03:02 AM

 gpz, on 11 December 2021 - 12:15 PM, said:

The name of the Zip archive need not match the folder it contains, at least not by my intention. I have unmatching Zip files as a testbed, and they work fine on my computer.

Attached Image: 2021-12-11 16_46_14-MS Excel with extensions - dev gpz.xlsx  Compatibility Mode.jpg

I've done a little more testing of the names and you were right, of course. I must have been confusing myself.

Anyway, proceeding now to try out some of the more exciting elements of this new feature.

  • 17 Pages +
  • « First
  • 12
  • 13
  • 14
  • 15
  • 16
  • Last »
  • You cannot start a new topic
  • You cannot reply to this topic

1 User(s) are reading this topic
0 members, 1 guests, 0 anonymous users