[Genma Saotome]

 YoRyan, on 08 May 2021 - 06:09 PM, said:

In general, we wish to separate related content into packages, rather than smashing it all together in a single MSTS directory.
SharpFileSystem creates a virtual filesystem that maps virtual file paths to physical files or files in archives.

So instead of having a file in one place that is accessible to all sorts of things you want to make copies of it for every place it is used? If yes, how do you keep them identical after an edit? Or do you mean all of those instance will simply be a file reference to one package? I think you mean the later but I'm not sure.

Seems to me you could try the same thing by inserting a full path before every file name recorded in these game files. Could be a very simple way to find issues w/o having to do much, if any, code.

Your example forgot \traffic.

[YoRyan]

 Genma Saotome, on 08 May 2021 - 07:40 PM, said:

Or do you mean all of those instance will simply be a file reference to one package?

Yes, that is the intent.

 Genma Saotome, on 08 May 2021 - 07:40 PM, said:

Seems to me you could try the same thing by inserting a full path before every file name recorded in these game files. Could be a very simple way to find issues w/o having to do much, if any, code.

Well, that algorithm wouldn't be compatible with the virtual filesystem, which exists entirely in memory.

None of the reference and folder structure stuff is relevant until we actually have new OR file formats and new OR editors, anyway. (So, the cynicists around here would say it will never be relevant.) Ergo, that implies new code.

 Genma Saotome, on 08 May 2021 - 07:40 PM, said:

Your example forgot \traffic.

This concept is a carryover from Kuju and it would probably not make sense in the context of OR. But that's a call for whoever designs a new activity editor.

[Amtrak115]

So let's see if I understand what this entire thread and James' summation on Git means to the "average" Openrails user and content providers such as route designers, activity designers, and a variety of shape designers, etc.

The Virtual File System (VFS) will be an abstract layer between the physical storage of OR content and the simulation itself. Take for instance a typical MSTS installation which typically look something like this in a simplified format:

/Global
/Routes
/Sound
/Trains

the OR VFS would understand that structure and it's physical location on your hard drive.

So if I as a content developer created a route say...ATSF's Raton Subdivision. so all the folders necessary for that route could be placed in a zip file. Then either through some sort of structured file that describes the content of that zip zip contained in the zip or through some sort of user interface in OR that describes the structure of that zip file, I could add that route to my OR instantiation. I could leave the zip file "zipped" and without having to copy all the required files from the original MSTS install to make my route work. Since OR would know where all the required shapes are stored on the original MSTS installation. This would prevent excessive file duplication that we do today.

Say I'm an Activity content developer. I would define only the "new" content necessary for that activity and I could "reference" all the existing content without have to copy or duplicate files.

Engines/wagons/structures/sound...etc would also be handled in the same way.

Given if my understanding is correct....several question come up....

1. will users still be able to "organize" their files as they wish (assumption on my part is yes as long as the content description "documents" where everything is)

2. Will embedded editors such as trackviewer, potentially activity editors, and new consist editor that "blocks" cars and engines be able to utilize this VFS?

3. Since VFS is an abstract layer, I as a content developer, say Activities, could post a link to my Github
repository of my developed activity, then OR could utilize that Github repository as a virtual node of the
VFS without necessarily having to download the repository to my local storage.

[Amtrak115]

I meant Chris' summation, not James. sorry to both James and Chris.

Barry

[YoRyan]

You have good a point that up until now, we haven't really discussed what the virtual filesystem would look like from the POV of the end user. So, I'll make an attempt:

We're all familiar with the age-old MSTS data directory:

/Microsoft Train Simulator
  /Global
  /Routes
  /Sound
  /Trains

What the virtual filesystem will do is allow you to compose your MSTS data from multiple overlapping sources. Such sources can be directories or archive files. (Network sources like HTTP or GitHub would also be possible, but would be severely constrained by the performance of your Internet connection.)

It will be entirely up to you as to how you want to organize your sources. One could imagine a system like this: Start with the base MSTS data, then add XTracks, routes, and rolling stock. (Alternatively, you could choose not to bother with the VFS at all, in which case your "Microsoft Train Simulator" directory would itself be a single source.)

/MSTS1.2
  /Global
  /Routes
  /Sound
  /Trains
/XTracks.zip
  /Global
/RatonSubdivision.zip
  /Routes
  /Trains
/SouthwestChief.zip
  /Trains
/ORPathEditor
  /Routes

Open Rails will then combine all five sources into a single virtual data directory (with Global, Routes, Sound, and Trains folders) that only exists in memory. If there are conflicting paths, then sources that are later in the load order will take precedence. As a user, you will also have the ability to change your source load order.

The motivation is not trying to avoid file duplication, or even attempting to improve on Kuju's directory structure (you'll notice that that does not change as part of this proposal)—it's that you'll have an additional abstraction layer with which you can organize your content. If you want to uninstall an activity, you can just delete its package! There's no need to hunt down its individual traffic, service, path, and consist files. As a content creator, you could package up your stuff into a ZIP file that could be loaded directly by Open Rails as a VFS source. This would make it much easier for users to install it.

Ignore the stuff about the /OR namespace and the file path reference system. It's not relevant until we design new OR editors and file formats.

[Genma Saotome]

Ryan, it might help everyone if you explain what you mean by virtual data directories that only exist in memory (people do tend to to think in terms of some sort of organizing principle on their disks).

AFAIK there are three approaches to consider when storing all of these files:

• Exactly as KUJU defines things -- everything in one place but problems with both duplicate names and too many files in any non-route branch.
  
• Minroutes -- many installations, but problems with shape, texture, and rolling stock duplication.
  
• Master library -- everything in one place but not using KUJU's tree, with problems "expressing" these master files into those places where the game software looks for files.

For myself, I use #3 where I can (but I rather doubt more than handful of other people do the same) in combination with #2. If I understand where you guys are trying to go it seems to look more like #3 than anything else I can figure out except that unlike a library w/ books sorted onto shelves it sounds more like a storage locker filled w/ unmarked boxes containing who knows what.

[YoRyan]

All of the above, and none of the above. Using the VFS is like Ctrl-C'ing, Ctrl-V'ing the contents of five different directories into a single MSTS directory, except the game will do this for you. Does that make things more clear?

(This should arguably be its own topic, because the virtual filesystem has little or no relation to the eventual Open Rails directory structure. That is a separate thing.)

[Amtrak115]

Ryan,

I was going to jump in again....but I hadn't read the earlier posts....I'll back out now..or until I have another question....

Barry

[Csantucci]

It's the moment also for me to try to understand better.
What about mini-routes installations? How can a trainset be made common to more MSTS-like installations (e.g. base installation, and a mini-route installation)?
Are there actual content structures that will be no more permitted with this new arrangement?

[YoRyan]

By sharing the same VFS source among multiple installation profiles, you can avoid duplicating stuff that is common to multiple installations.

Source: XTracks.zip
  /Global

Source: DekosoftTrains
  /Trains

Source: Surfliner2
  /Routes/Surfliner2

Source: NECv4
  /Routes/USA1A

Profile: Surfliner
  Load order: [XTracks.zip, Surfliner2, DekosoftTrains]

Profile: Northeast Corridor
  Load order: [XTracks.zip, USA1A, DekosoftTrains]

If you do nothing, all of your existing installation profiles will continue to work as normal. So, no, the VFS will not constrain any existing configurations.