[James Ross]

wacampbell, on 12 December 2011 - 07:40 AM, said:

1. We have an import process where they are brought in, validated and converted to the OR file structure

Unless you're proposing we convert the files to the OR file formats, I don't really see how you can sensibly deal with file links from MSTS content in the new structure - it certainly wasn't meant in any way to operate with MSTS content.

[wacampbell]

James Ross, on 12 December 2011 - 08:03 AM, said:

Unless you're proposing we convert the files to the OR file formats, I don't really see how you can sensibly deal with file links from MSTS content in the new structure - it certainly wasn't meant in any way to operate with MSTS content.

So are you thinking we don't support MSTS content?

[James Ross]

wacampbell, on 12 December 2011 - 08:11 AM, said:

So are you thinking we don't support MSTS content?

Existing content, with this system, yes. There's nothing to stop it being used for new content that happens to use the MSTS file formats, if that's useful, but one of the important points is how it arranges the files - that poses problems to importing some MSTS content, at least, because of the relative paths involved. Maybe the majority of things would work, but it wasn't designed with that in mind.

[Genma Saotome]

WRT Waynes's comments: Being able to name the level attached / right below \Content is a very god thing because then then each end user can self-organize (and re-organize, as the need arises). When such re-organization occurs, the string that forms the path name changes, or the file name itself changes, or both. A change applied directly to those strings will work of course... and then the question occurs of where else do the original strings occur? A dbms could handle that, tho the devil is always in the details. An alternative that might also work is the tilde / %name% idea that is, partially, not just a short cut but a set of foreign keys between master (the string in the tilde / %name% path record and the dependent records, which simply holds the shortcut value. It covers renaming the paths but not the files themselves.

WRT James's comments: One may hope the person providing the patch / revisions includes the original name and one may hope it's still known by that name on each PC, but I wouldn't count on that. As for everything you have in mind as a predefined directory, (1) why not simple use that?, and (2) What does that list of predefined names contain?

What it looks like is you've taken MSTS \routes, put two levels above that, and shuffled some directories that are in MSTS \routes up a level effectively making them \global\shapes, while leaving others alone. All in all, it's a bit like a series of MSTS mini-routes.

Anyway, at this point the concern I'm trying to express is first about clarity... how is thing going to be managed long term, and second to ask how is it going to be better than any other idea that pops up? I'm not trying to discourage the proposal but instead am probing for understanding. Hope you can bear with me as I ask questions.

[wacampbell]

Genma Saotome, on 12 December 2011 - 09:24 AM, said:

such re-organization occurs, the string that forms the path name changes,

Lets say we retain 'wag' type configuration files at least for the purposes of this explanation.
In that file there may be a link to a cab view file.
With the proposed file structure, that file reference would never contain a full path to the cab view file. It would reference the package by name, and the filename.
The actual location of the package can be in any content directory anywhere on any disk. And in a different location on different users computers. And in fact be freely moved by the end user should he want to reorganize his content folders. The only thing that must remain constant is the package name.

We leave it to OpenRails to search for the referenced package and complete the link at run time.

[James Ross]

Genma Saotome, on 12 December 2011 - 09:24 AM, said:

As for everything you have in mind as a predefined directory, (1) why not simple use that?, and (2) What does that list of predefined names contain?

For (1), I have no idea what you're asking.

For (2), the list isn't finalised but the idea is that every major file/object type has its own pre-defined directory, with exceptions for groups like the route example. It's hard to finalise the list without knowing what we're going to be having as OR file formats, so I've tried to use MSTS names when I can, but that's not set in stone or anything.

Genma Saotome, on 12 December 2011 - 09:24 AM, said:

What it looks like is you've taken MSTS \routes, put two levels above that, and shuffled some directories that are in MSTS \routes up a level effectively making them \global\shapes, while leaving others alone. All in all, it's a bit like a series of MSTS mini-routes.

That's basically what I said in an earlier post. It's an attempt to improve upon the structure found in MSTS; shapes and textures, for example, don't need to be locked inside a route, so they're moved up. It also equalises the level of things, so there's no "global" involved - just what type of thing it is. You should certainly be able to include everything needed for a route (or multiple routes) in a single package - but it's not limited to that either.

[longiron]

Sorry for so many questions :>)

Does Open Rails require ENG files to have the two sections ENGINE & WAGON?
Does it mean anything to Open Rails where a parameter is located in the file, or section of file?
Is there an ideal order that parameters should be in?
Does the software just look for the PARAMETER NAME no matter where it resides in the file and loop to find the next one?

thanks

[James Ross]

longiron, on 19 December 2011 - 11:07 AM, said:

Does Open Rails require ENG files to have the two sections ENGINE & WAGON?
Does it mean anything to Open Rails where a parameter is located in the file, or section of file?
Is there an ideal order that parameters should be in?
Does the software just look for the PARAMETER NAME no matter where it resides in the file and loop to find the next one?

The order of the parameters should not matter, but they must be nested correctly; this means they need to be in the expected section (engine vs wagon) and not inside any random other blocks.

[longiron]

James Ross, on 19 December 2011 - 12:30 PM, said:

The order of the parameters should not matter, but they must be nested correctly; this means they need to be in the expected section (engine vs wagon) and not inside any random other blocks.

Thank you. So they DO need to be in the expected section, that's an "ah ha". Do you believe there's any performance improvement by having an established order of parameters?

[James Ross]

longiron, on 19 December 2011 - 12:51 PM, said:

Do you believe there's any performance improvement by having an established order of parameters?

No; the parser has to munch through the same number of characters and the same processing gets called for each no matter the order so it should not make any difference.

[Genma Saotome]

May I add that one thing I learned from study of what MSTS does is that there are registry & system calls every time the value of a path changes. I've since sorted my world files so anything that is in the \global\shapes directory is read first and when that's done there is one switch over to \routename\shapes and all the remaining files for the tile are read from there. The de-fragger I have can operate on directories too so I also defrag the same directories one at a time. I've seen no evidence that sorting by filename adds value.

I'm assuming this applies to OR as well.

[Genma Saotome]

One of the other threads got me thinking so I went and searched on *.wav in the Train Simulator directory tree... 0.67gb in > 5000 files. Now I don't have that many routes and I think by most considerations a rather small \trainset tree and so for me these numbers are a surprise.

I havn't bothered to determine exactly how many dups are present... just eyeballin the list I see a lot of two's and three's of stuff, but there are also 9 of these and 13 of those in there too.

I mention this WRT James's proposal for wholly self-contained packages of content. It seems to me that the sound files in the MSTS \trainset directory tree would serve as a very good indicator of the kind of duplication that would exist were we to adopt that proposal as-is -- a lot of unnecessary duplication. The same duplication would occur with cars or locomotives distributed as part of a new activity.

In thinking on the issue, it strikes me that James's ideas are good for distribution purposes as you get everything you need, probably easier for the programmer, and not that good for the purpose of reuse, not good for the task of locating and replacing files with something better.

What we have here is a long list of content -- mesh, texture, and sound files that can be used in many places -- a simple one to many relationship. The dilemma to solve is how to ensure only one instance of content and having done so to put it into master list that is known "globally" to all other content. In business systems the term I've used to describe that task is merge/purge: a comparison of candidates to existing master data that takes in new content as new stuff and on matches performs a reconciliation between master data and the in-coming data (the merge part) so it can get rid of the duplicate candidate (the purge part).

Were we to use something like that, the objective would be to put commonly used content would be located in "global" directories and unique content left with the downloaded stuff. A sophisticated process might also differentiate global freeware from vender-specific "global" content. Prior to the merge/purge step distribution package content would be as James has suggested -- you get it all, altho we might also see proactive content creators might pre-segregate files into candidates for "global" consideration and everything else. As part of the merge / purge process our code would have to replace the original references to candidate file with the correct values of the master data. It might sound knarly but it might not be much more than doing a string search on all installed files and looping thru the list changing the string.

For OR the reconciliation step might be based on certain file attributes (e.g., name, size, date, etc.), values of tags, and so on. To be on the safe side, it might also ask for the end-users agreement to accept each recommended merge/purge. It might also be wise to limit what types of files that go thru this process -- sound, mesh, and texture files come to mind as good choices for master data but not consist, activity, or world files. I've no opinion on whether this step occurs on install... or as a result of optimization for Open Rails.

The end result would be some directories holding single instances of "global" master content that were referenced many times elsewhere.

[James Ross]

Genma Saotome, on 27 December 2011 - 06:09 PM, said:

I mention this WRT James's proposal for wholly self-contained packages of content. It seems to me that the sound files in the MSTS \trainset directory tree would serve as a very good indicator of the kind of duplication that would exist were we to adopt that proposal as-is -- a lot of unnecessary duplication. The same duplication would occur with cars or locomotives distributed as part of a new activity.

In thinking on the issue, it strikes me that James's ideas are good for distribution purposes as you get everything you need, probably easier for the programmer, and not that good for the purpose of reuse, not good for the task of locating and replacing files with something better.

Were we to use something like that, the objective would be to put commonly used content would be located in "global" directories and unique content left with the downloaded stuff.

The entire point of the organised packages is that commonly used content would be available in a package of its own, which others would then reference, to avoid duplication. A bit like how things use XTracks or NewRoads currently, but actually designed for that.

[longiron]

I think these proposals are excellent. On the duplication issue, I suggest we have to be careful. There are many wave, ace, s files with the same file name, but are completely different content. The community has already moved in the "shared" direction with common.snd and common.cab folders instead of individual sets within each train folder.

Whatever the proposal we adopt, I do not want to get into the content management business - i.e. writing a program to manage the users files. I suggest this is better left to the community to do.

[wacampbell]

Jame's proposal does give us the flexibility to set up shared content.

There may still be cases where a builder improperly distributes his work with certain duplicate files contained in his package instead of referencing the proper external shared pakage. If the distributor were worried that the end user might not have the required external package, he could, with permission, include the necessary parts of that external package in his distribution as a separate package. In other words his distribution contains his package, and the necessary parts of the common package. We'll have to set up some recommended guidelines for builders to use. And it would be easy for others in the community to make some tools to allow removal of duplicates and substitutions.