[Jack@Elvas]

Siebren recently added a cool new feature to openrails - the ability to download and install routes with a single click!

The 1st pass/proof of concept has Openrails download a json file with all the known routes and their download URLS.
This design allows for other routes to be added later without needing to re-compile and update the openrails.exe (or in this case Menu.exe)

This is a proposal to extend this design with a "Provider" interface.

What is a provider interface?

Right now the route installer pulls exactly one routes.json file - this file is maintained by the openrails team.

A provider architecture would allow the installer to pull multiple routes.json files from a variety of locations.

A community website like trainsim.com might want to manage their own routes.json file - adding routes over time as members verify the routes are compatible with the route installer.

The provider architecture then creates a fair playing field where any forum owner can offer up the compatible routes on their forum with the route downloader.

It also has the added benefit of offloading much of the route cataloging work - for example, trainsim.com has approx. 2600 routes in its download library, and it's staff and members could do the work of validating which routes work with the installer, and adding those to its own local routes.json file.


The rough proposal for the provider data structure would be something like:
[
{
"name": "Official openrails Download registry",
"URI": "https://raw.githubusercontent.com/openrails/content/main/routes.json",
"image": "https://www.openrails.org/images/logo.png",
"description": "The official openrails download registry",
"userid": "",
"password": ""
},
{
"name": "Awesome website with forums about openrails",
"URI": "https://www.awesome.com/routes.json",
"image": "https://www.awesome.com/images/logo.png",
"description": "Awesome website serves the OpenRails ...",
"userid": "fred@slateconstructionco.com",
"password": "password"
}
]


The proposal is for a UI to maintain this data structure (which I can help with) and for code changes to loop though each provider and add that provider's routes to the current list of routes.

I've chatted with Siebren individually about this and he suggested we discuss it publicly here so others could get involved.

[Jack@Elvas]

Weter, on 24 February 2024 - 01:10 AM, said:

Hello.
Can You allow me to ask one silly quesion:
How good will it be secured against malware spreading and who would be responsible/in charge about each download item's safety?
Sorry, that's almost 1,5 silly questions :)

Right now it's still in the early phases, so it doesn't run any executables or scripts - it just downloads a route to a folder.
The way Siebren's got it coded, it can download a Git repo, or a single zip file.

In its current state, the risk of running malware is the same as the risk when downloading a route manually.

It may actually help with security - since it makes route installation very easy - there's less of a need for a route builder to create an installer.

- Jack

[Weter]

Thanks.

Quote

since it makes route installation very easy - there's less of a need for a route builder to create an installer

It definitely will safely help to ones, who is not ready to mess with manual installation, indeed.

Quote

the risk of running malware is the same as the risk when downloading a route manually

Technically. But trusting in publisher, the vigilance would be decreased, against doing on one's own.

[Jack@Elvas]

Weter, on 24 February 2024 - 06:24 AM, said:

Thanks.

Technically. But trusting in publisher, the vigilance would be decreased, against doing on one's own.

I don't know the right answer here.

Being open, and letting anyone work with this system seems to fit best with the sprit of openrails.
Being closed might help with security, but then we risk alienating people.

Apple's got a closed system, where you must go through apple and it works well for them.
but it's not perfect and we don't have the resources apple does to moderate submissions.

This will require some further introspection.

[Weter]

I see, You are understanding the problem very well.
I agree - some balance is needed. The possible solution is not just a single "point", but the "field".
I feel myself calm now - as You know, what You do.

[eric from trainsim]

Seems like there's a piece missing... the JSON above seems to be listing out repositories, but what about the routes themselves? Is this intended to return a JSON or other response that would list out the routes?


I know this is a mock-up, but I'm really not a fan of user ID's and passwords being stored in the open within a JSON file.

I've already tested this -- as long as the user had cookies enabled and had previously logged into the site via their default web browser, a download can be done.

Yes, I know there are a few folks who refuse to allow cookies to be saved out of fear or other concerns, but they're a necessary evil when it comes to authentication.

Could this approach of retrieving individual zips from a website also work against a folder containing zips? That could accommodate those who don't want to remain logged in.

I know of one content provider who uses password protected zips. Hopefully that will be accommodated?

[Siebren]

eolesen, on 24 February 2024 - 11:05 AM, said:

I know of one content provider who uses password protected zips. Hopefully that will be accommodated?

Would be easy to add, popup window during the download where the OR user has to fill in the password.

[cjakeman]

Jack@Elvas, on 23 February 2024 - 09:43 PM, said:

Siebren recently added a cool new feature to openrails - the ability to download and install routes with a single click!

The 1st pass/proof of concept has Openrails download a json file with all the known routes and their download URLS.
This design allows for other routes to be added later without needing to re-compile and update the openrails.exe (or in this case Menu.exe)

Right now the route installer pulls exactly one routes.json file - this file is maintained by the openrails team.

The file is kept in a separate repo from the source code but is to be maintained in the same way as code files - through peer review. This gives us a curated list where the entries are described in similar ways and are easy to search and compare.

This hasn't been rolled out to Testing yet, so preparing to expand it to every stand-alone route compatible with Open Rails is surely premature.

Some issues we want to consider once Siebren's work has settled in include:

• how does that fit with the virtual file system (VFS)?
• how can we install routes that are not stand-alone (no doubt taking advantage of VFS capability)?
• how we install other content such as trains and consists (again making use of VFS)

Another issue is copyright. If Open Rails can pull in content from anywhere (including content which is illegally copied from the creator), does that make Open Rails responsible for breaking copyright?

James and I are both excited about this project which Siebren, Looky and Henk have all worked on. But it's too early to widen it out to all providers.

[Jack@Elvas]

eolesen, on 24 February 2024 - 11:05 AM, said:

Seems like there's a piece missing... the JSON above seems to be listing out repositories, but what about the routes themselves? Is this intended to return a JSON or other response that would list out the routes?

I didn't explain this well enough, let me back up a bit...

routes.json is a file Siebren is already using for the route downloader tool- that file format is already set, the json describes each route.

You can see the current routes.json file here:
https://github.com/o...ain/routes.json

As it stands today, the openrails route downloader uses just this one routes.json file shown above.

This means the OpenRails team would be a bottleneck to getting new routes added.

Additionally there's no mechanism to have the routes stored behind any form of password system - so the current system is not compatible with protected archive sites such as trainsim.com.

Also in case anyone reading this isn't aware, sites like trainsim.com don't allow anonymous, unauthenticated content. Users must have an ID and password.


So if we level set here, Siebren's built an awesome one click route downloader - this is a good thing and will help Openrails retain new users.
At the same time this new downloader makes it less likely that a user will download routes from traditional sources.

This conversation was started to offer a solution to that later part.

eolesen, on 24 February 2024 - 11:05 AM, said:

I know this is a mock-up, but I'm really not a fan of user ID's and passwords being stored in the open within a JSON file.

Yep just a mockup with some suggested fields - no reasons some basic encryption couldn't be used on the values. Main goal right now is to get us thinking of what we'd need for this to work.

eolesen, on 24 February 2024 - 11:05 AM, said:

I've already tested this -- as long as the user had cookies enabled and had previously logged into the site via their default web browser, a download can be done.

You're saying if a user has logged in to your website with their browser, then a C# program can fetch content from that same website? I've not seen this work in the past so I'd be interested in seeing sample code that achieves this.

eolesen, on 24 February 2024 - 11:05 AM, said:

Could this approach of retrieving individual zips from a website also work against a folder containing zips?

No I wouldn't expect this system to handle that use case.

[Jack@Elvas]

cjakeman, on 24 February 2024 - 11:48 AM, said:

• how can we install routes that are not stand-alone (no doubt taking advantage of VFS capability)?
  
• how we install other content such as trains and consists (again making use of VFS)

I don't think this needs to deal with non-standalone routes, nor other content to be of value.
I'd assume some of those factors such as rolling stock, would be addressed in future iterations.

cjakeman, on 24 February 2024 - 11:48 AM, said:

Another issue is copyright. If Open Rails can pull in content from anywhere (including content which is illegally copied from the creator), does that make Open Rails responsible for breaking copyright?

I'm not sure on the legality - no one holds Google responsible for illegal downloads done using chrome, but years ago, I believe the Napster was held accountable for facilitating piracy back in the day.

If we're just pulling routes from Github, then github will handle things like copywrite infringement claims and will take down content as requested.
I believe Eric does the same over at trainsim.com - if he's made aware of asset re-use I've seen him go to great lengths to ensure content is ok before being posted.

cjakeman, on 24 February 2024 - 11:48 AM, said:

...it's too early to widen it out to all providers.

I appreciate the feedback - I think there is value to considering how other providers would be included in the initial design.

A few thoughts come to mind:

• This functionality will be easier to add now.
  
• The last OpenRails release was roughly 15 months ago - That may be an anomaly, but it would be a shame to introduce this new downloader and the exclude communities like trainsim.com for over a year if it takes that long between releases.