[RR1]

gpz, on 28 August 2020 - 07:29 AM, said:

By reading the directories of the packages first, it is straightforward to select the newest files only to load.

Thanks, that answers 1 question. But what about all those updated files? Are they retained forever or is there a way to purge them. I suspect that would not be easy?

I realize they are compressed so maybe this really isn't an issue. But regardless of that I'm simply seeking a better understanding of the new process.

[cjakeman]

YoRyan, on 27 August 2020 - 10:27 PM, said:

As a content creator releasing your first version, you would release a package that includes the entire route and all of its files. When you release an update, you would have two options: either issue a package that completely replaces the original, or issue a much smaller net-change package that depends on the original and must come after it in the load order.

An alternative, which I think is compatible with these mechanisms, is to publish the route as a Git repository (e.g. on GitHub) and let users download it into the correct folder.

Then corrections are published to the remote repository and pulled down using Git. Different version as branches (e.g. steam-era and modern-era) can be pulled down as required. You would use Git to check-out a different version from your local repo.

As I have mentioned before, this is working very well for Rick Loader's extensive New Forest route.

[YoRyan]

RR1, on 28 August 2020 - 09:29 AM, said:

But what about all those updated files? Are they retained forever or is there a way to purge them. I suspect that would not be easy?

If their files are no longer required, you would be able to delete the old content packages, and OR would no longer load them into the virtual filesystem. Just how many you could delete would depend on how the author structured his or her updates. Perhaps you would need to retain all 16 updates, or perhaps you would only need to retain #14, #15, and #16, or maybe the author didn't use change-type packages at all and you would only need to retain the very latest package. Any methodology is possible.

cjakeman, on 28 August 2020 - 10:02 AM, said:

An alternative, which I think is compatible with these mechanisms, is to publish the route as a Git repository (e.g. on GitHub) and let users download it into the correct folder.

Yes, this is also quite possible - just use the Git repository itself as a directory-type package.

[cjakeman]

This thread is an amazing 9 years long, refers back to previous discussions, sets out a detailed proposal and adopts some decisions as the thread progresses.

Now that work has begun on this (thanks, Ryan), I thought it might be helpful to write down the design of the filesystem for easier reference than trawling through all the posts. I think I have succeeded in recording the final state of things accurately, but let me know if I've missed anything.

The design is now stored here in the repository.

[Amtrak115]

Chris, I've read through the entire chain as well as what you condensed down on Git.....I'm still confused....

"Files in the MSTS-organised packages can continue to use legacy references such as:

..\..\SomePackage\Shapes\bungalow.s

but files in the OR-organised packages must start from the /OR root and may not use ".."
For OR assets that are inextricably linked with other assets (paths and activities vis-a-vis routes)

An activity must be linked to a route. In OR packages, this will be achieved with a more restrictive reference:

"Route": "/OR/SomePackage/Routes/SomeRoute.route-or"

"

So today I package an activity for say LaJunta Sub. I create a zip that has the structure shown in the attached jpg

how would this look as a OR package??

Attached thumbnail(s)

• 

[Genma Saotome]

In this example:

Quote

/OR/SomePackage/Shapes/bungalow.s

refers to file

C:\Users\Ryan\Open Rails\SomePackage.zip\SomePackage\Shapes\bungalow.s

How does this reference occur?

[cjakeman]

Amtrak115, on 07 May 2021 - 12:53 PM, said:

I've read through the entire chain as well as what you condensed down on Git.....I'm still confused....

Sorry it's confusing. I'll do my best to clarify and maybe tweak the document where it helps.

The point of packages is to avoid duplication and make it easier to mix and match content. For example, to run your activities but using my enhanced consists instead of yours and without touching the files in your package.

As I understand it, the package mechanism will work for MSTS without any changes to the contents of MSTS files.

How would this look as an OR package using *-or files is still to be determined, but MSTS content will be able to reference OR content and vice-versa.

We may need to tweak the folder structure for OR packages because we want to support real-life complications such as blocks of railcars, so the Consist may let you nest (contain other consists) and it may not even be called a consist.

[cjakeman]

Genma Saotome, on 07 May 2021 - 05:21 PM, said:

In this example:

/OR/SomePackage/Shapes/bungalow.s

refers to file

C:\Users\Ryan\Open Rails\SomePackage.zip\SomePackage\Shapes\bungalow.s

How does this reference occur?

I prepared a response to your query, but then I found it might raise an issue.

I'll check and post again.

[Amtrak115]

Thanks in trying to clarify this situation...I have two reasons for asking...one I've been making content (Activities. basically Amtrak's Southwest Chief from Chicago to LA, since all the routes exists via trainsim.com) The second reason is in digging into trackviewer and am trying to basically add all the features and functions for an Activity editor, the structure of Activities and how all the related files are referenced including consists or blocks or however we are going to refer to them plays into my design. anyway thanks again in advance...

Barry

[YoRyan]

Amtrak115, on 07 May 2021 - 12:53 PM, said:

how would this look as a OR package??

Until there is an OR activity editor (if there is going to be an OR activity editor...), it's impossible to say what the OR structure would look like.

Personally, I would spitball something like this:

/OR
  /MyActivityPackage
    /Activities
      something.activity-or
    /Paths
      player.path-or
    /Consists
      player.consist-or

In general, we wish to separate related content into packages, rather than smashing it all together in a single MSTS directory.

In the near term, the virtual filesystem will allow users, players, and content creators to compose their existing MSTS data directories in neater and simpler ways.

Genma Saotome, on 07 May 2021 - 05:21 PM, said:

How does this reference occur?

SharpFileSystem creates a virtual filesystem that maps virtual file paths to physical files or files in archives.

[Genma Saotome]

YoRyan, on 08 May 2021 - 06:09 PM, said:

In general, we wish to separate related content into packages, rather than smashing it all together in a single MSTS directory.
SharpFileSystem creates a virtual filesystem that maps virtual file paths to physical files or files in archives.

So instead of having a file in one place that is accessible to all sorts of things you want to make copies of it for every place it is used? If yes, how do you keep them identical after an edit? Or do you mean all of those instance will simply be a file reference to one package? I think you mean the later but I'm not sure.

Seems to me you could try the same thing by inserting a full path before every file name recorded in these game files. Could be a very simple way to find issues w/o having to do much, if any, code.

Your example forgot \traffic.

[YoRyan]

Genma Saotome, on 08 May 2021 - 07:40 PM, said:

Or do you mean all of those instance will simply be a file reference to one package?

Yes, that is the intent.

Genma Saotome, on 08 May 2021 - 07:40 PM, said:

Seems to me you could try the same thing by inserting a full path before every file name recorded in these game files. Could be a very simple way to find issues w/o having to do much, if any, code.

Well, that algorithm wouldn't be compatible with the virtual filesystem, which exists entirely in memory.

None of the reference and folder structure stuff is relevant until we actually have new OR file formats and new OR editors, anyway. (So, the cynicists around here would say it will never be relevant.) Ergo, that implies new code.

Genma Saotome, on 08 May 2021 - 07:40 PM, said:

Your example forgot \traffic.

This concept is a carryover from Kuju and it would probably not make sense in the context of OR. But that's a call for whoever designs a new activity editor.

[Amtrak115]

So let's see if I understand what this entire thread and James' summation on Git means to the "average" Openrails user and content providers such as route designers, activity designers, and a variety of shape designers, etc.

The Virtual File System (VFS) will be an abstract layer between the physical storage of OR content and the simulation itself. Take for instance a typical MSTS installation which typically look something like this in a simplified format:

/Global
/Routes
/Sound
/Trains

the OR VFS would understand that structure and it's physical location on your hard drive.

So if I as a content developer created a route say...ATSF's Raton Subdivision. so all the folders necessary for that route could be placed in a zip file. Then either through some sort of structured file that describes the content of that zip zip contained in the zip or through some sort of user interface in OR that describes the structure of that zip file, I could add that route to my OR instantiation. I could leave the zip file "zipped" and without having to copy all the required files from the original MSTS install to make my route work. Since OR would know where all the required shapes are stored on the original MSTS installation. This would prevent excessive file duplication that we do today.

Say I'm an Activity content developer. I would define only the "new" content necessary for that activity and I could "reference" all the existing content without have to copy or duplicate files.

Engines/wagons/structures/sound...etc would also be handled in the same way.

Given if my understanding is correct....several question come up....

1. will users still be able to "organize" their files as they wish (assumption on my part is yes as long as the content description "documents" where everything is)

2. Will embedded editors such as trackviewer, potentially activity editors, and new consist editor that "blocks" cars and engines be able to utilize this VFS?

3. Since VFS is an abstract layer, I as a content developer, say Activities, could post a link to my Github
repository of my developed activity, then OR could utilize that Github repository as a virtual node of the
VFS without necessarily having to download the repository to my local storage.

[Amtrak115]

I meant Chris' summation, not James. sorry to both James and Chris.

Barry

[YoRyan]

You have good a point that up until now, we haven't really discussed what the virtual filesystem would look like from the POV of the end user. So, I'll make an attempt:

We're all familiar with the age-old MSTS data directory:

/Microsoft Train Simulator
  /Global
  /Routes
  /Sound
  /Trains

What the virtual filesystem will do is allow you to compose your MSTS data from multiple overlapping sources. Such sources can be directories or archive files. (Network sources like HTTP or GitHub would also be possible, but would be severely constrained by the performance of your Internet connection.)

It will be entirely up to you as to how you want to organize your sources. One could imagine a system like this: Start with the base MSTS data, then add XTracks, routes, and rolling stock. (Alternatively, you could choose not to bother with the VFS at all, in which case your "Microsoft Train Simulator" directory would itself be a single source.)

/MSTS1.2
  /Global
  /Routes
  /Sound
  /Trains
/XTracks.zip
  /Global
/RatonSubdivision.zip
  /Routes
  /Trains
/SouthwestChief.zip
  /Trains
/ORPathEditor
  /Routes

Open Rails will then combine all five sources into a single virtual data directory (with Global, Routes, Sound, and Trains folders) that only exists in memory. If there are conflicting paths, then sources that are later in the load order will take precedence. As a user, you will also have the ability to change your source load order.

The motivation is not trying to avoid file duplication, or even attempting to improve on Kuju's directory structure (you'll notice that that does not change as part of this proposal)—it's that you'll have an additional abstraction layer with which you can organize your content. If you want to uninstall an activity, you can just delete its package! There's no need to hunt down its individual traffic, service, path, and consist files. As a content creator, you could package up your stuff into a ZIP file that could be loaded directly by Open Rails as a VFS source. This would make it much easier for users to install it.

Ignore the stuff about the /OR namespace and the file path reference system. It's not relevant until we design new OR editors and file formats.