[wacampbell]

I think we've openned this discussion before with lots of good points made, but the mass of the task being too much to settle on an approach.

I'll copy over some posts from the filenaming thread to kick off the discussion.


EDIT ... on second thought, it turns out I don't know how to copy topics over from another thread . Dave, if you can bring over the last two or three posts from the file extension thread it would kick us off.

[wacampbell]

Just to frame this discussion a bit, lets limit it to file and folder structure.

The files and folders are really just one element of the overall storage plan for the data that openrails needs. For example, it is technically possible to have all the data for a route in a single file. But MSTS choose not to for various reasons which may or may not be valid. Hopefully this discussion will disect and challenge that rationale along with the rest of the folders etc that are defined.

Exactly where the data set breaks out into separate files is a core design decision and how those files are organized is a core design decision.

Lets make the following out of scope for this discussion to try to keep it manageable.
- the exact internal format of the datafiles
- the exact names and extensions used for the various files and folders

[captain_bazza]

Let me pose this basic question:

Is having individual OR vehicle assets* in a single folder structure (akin to MSTS) a bad thing?

(*Vehicle = MPU and rollingstock.)

Let me put my hand up and say, "NO, it's a good thing."

Why?

Because it's self-contained, in most cases, and it's easy to package and redistribute the asset.

Cheers Bazza

Edited.

[wacampbell]

I'll share a couple of pain points with MSTS.

1. Major Pain - Information needed for activities is spread all over - consist files, traffic files, services, etc. It makes it hard to distribute activities. And hard to remove unwanted activities.

2. Minor Pain - Scenery shape files have their textures and mesh files split over two folders - a minor pain for adding and removing scenery items to the route

[captain_bazza]

Allied to this subject, becaused it may have some effect on file structure, the identification and naming of models to properly differentiate them from similar, or identical subjects.

We need to consider an asset author UID system, with the UID prefixing the main file names associated with an asset. The identifying UID could be alphabetical or numerical, or a combination of both.

As you know, I use the prefix BZZ to identify my assets, so if I produce an asset, such as a Mikado, even in it's most simple form, my UID differentiates my asset from any other, ie BZZ_Mikado. This is also a case for assets from individual authors be placed in a folder structure under the author's UID, thence all future assets go into that author's main folder.

Trainsets>
UID_BZZA>sub-folders>>

>>DRGWMikado486>sub-folders>>>

>>RGSMikado384>sub-folders>>>

>>NZRJA1271>sub-folders>>>

Cheers Bazza

[captain_bazza]

Quote

1. Major Pain - Information needed for activities is spread all over - consist files, traffic files, services, etc. It makes it hard to distribute activities. And hard to remove unwanted activities.

Whole heartedly agree....

Major point....don't let this mish-mash approach affect OR, whether it's activities, routes or assets.

Cheers Bazza

[longiron]

wacampbell, on 09 December 2011 - 11:38 AM, said:

Just to frame this discussion a bit, lets limit it to file and folder structure.

The files and folders are really just one element of the overall storage plan for the data that openrails needs. For example, it is technically possible to have all the data for a route in a single file. But MSTS choose not to for various reasons which may or may not be valid. Hopefully this discussion will disect and challenge that rationale along with the rest of the folders etc that are defined.

Exactly where the data set breaks out into separate files is a core design decision and how those files are organized is a core design decision.

There is a lot to like and dislike about the storage plan implemented for MSTS. The core issue is the MSTS failed to adequately anticipate the impact of scale. Everything works very well if you have a few routes and a limited number of trainset/consists/etc. It breaks down as these scale. Kuji clearly didn't anticipate scales as MSTS fails to load because of too much active data (routes/trainsets/etc),

TrainStore, among others, attempted to solve this problem by creating a passive secondary store for MSTS stuff, thereby reducing the amount of active data (route/trainsets/etc), In essence it was a data management solution to a design shortcoming. TrainStore introduces its own issues, such as keeping everything in synch and leaving things in unplayable states because of user error.

RW attempted to solve the same problem by using Steam to "verify" the basic software content & folders, but introduced an annoying side effect. Customization gets overwritten unless special efforts were made to segregate the customizations.

What I Like about MSTS storage plan
- easy to understand, its just folders & files like on any Windows PC
- can customize to my individual preferences
- can use generally available tools to edit (Notepad, etc)
- segregation allows for problems/mistakes to be isolated
- easy to add & delete stuff (no fancy installers/uninstallers necessary, just copy & paste or delete)


What I DISLike about MSTS storage plan
- difficult to manage at scale, lots of time & effort
- must use other's conventions (folder name, file name, etc) to share stuff (Trainset/Activities/Paths)
- lots of repetitious stuff (wav, ace, etc) that needs to be managed
- clean up (delete) is impossible unless you understand the dependencies

[roeter]

Storage rules suffer from the age-old conflict between automation and freedom.
If you allow freedom to choose any kind of name for wagons, engines etc., there is no way you can get automatic tools to sort out what you've got.
A good example is sound and cab-files. Obviously, if you download an engine you want it complete with cab and sound. That means all these files have to be included with all engines, but that causes an awfull amount of duplication. Cross-reference to files from other engines, or central stored files, can only be done if there is a strict naming convention to which EVERYONE has to stick - almost impossible. No easy way out there.

One possible solution could be to have 'personal' fields in files which can be set by the user using special tooling. For instance, when downloading some rolling-stock the user can set these fields to what he (or she!) thinks is should be called, and these names will then be shown in consists editors etc., without the need to change the actual filenames. It will require some special 'explorer' windows to actually show these names rather than the actual filenames, but that can be done. If not set, the actual filename will be used. These fields should not be set in download items.
Consist editors would use the personal name, but would have an 'export' option which converts this to the original filenames, and can then easy select these files for transfer.
On receiving, an 'import' function would do the reverse. There would be no need to change the filenames and yet anyone can use his own naming conventions.
The same could be done for sound-files, cab-files and the like. It would not be very useful for normal static objects.

As for another point made earlier re. consists / activities / rolling stock : the problems of sharing and deleting etc. have little or nothing to do with the fact that the files are stored in different directories. Put them all in one directory and you will have the same problem - it is the fact that you cannot 'see' what's in them which is the root of all problems.

There is a lot to say for keeping different files in different directories. At least when looking in the activities directory, for instance, you can immediately see which activities you have. Put them in the same directory as your rolling stock and you 'loose' them immediately.

Regards,

Rob Roeterdink

[longiron]

Just thought I would give the team a comparison to a simple exercise in RW3. I want to get to a specific train car to modify something.

Step 1 - open steam folder and find "railworks" folder which i buried under the Steam\steam apps\common


Step 2 - go to the railworks\assets folder


Step 3 - Know I need to go to the railworks\assets\kuji folder


Step 4 - Know I need to go to the railworks\assets\kuji\SimulatorUS folder


Step 4 - Know I need to go to the railworks\assets\kuji\SimulatorUS\RailVehicles folder


Step 5 - Know I need to go to the railworks\assets\kuji\SimulatorUS\RailVehicles\Freight folder


Step 5 - Know I need to go to the railworks\assets\kuji\SimulatorUS\RailVehicles\Freight\2-Bay folder


Step 6 - I finally get to where I want to be :>)


So, a simple file that I want is buried 6 levels deep without much guidance as to where


And now for the route side of the house

Step 1 - Go to the Content folder instead of the Assets folder and open the Routes folder

Step 2 - choose the route I want to work on ?????



Here's the route folder structure underneath


And here's your activity folder, Fun huh?

[Genma Saotome]

There are times when I wondered if the problem in any tree based directory scheme is the inability to simply cut to the chase and say this one here. Chris's examples are great in this regard. It's been a long time now but I seem to recall there was something in a unix environment that helped in that regard.

Anyway, one thing I want to speak to is the unfortunate need to occasionally rely on the old ms-dos ..\..\ scheme, something I always thought was highly prone to both error and confusion. Without stepping into the operating system itself, would it be feasible for our data files to support the concept of the percent-name-percent convention (e.g.,%routename%)? If we add something like PathShortcut ( %DonnerPass% = "c:\program files\microsoft games\train simulator\routes\3dtsDonner1" ) then somewhere further down in the file we could do something like %DonnerPass%\Shapes\3dtsnBarn4.s; It reads much better, it's not likely to be as prone to error as is the MS-dos convention, and if the original definition of %DonnerPass% is done outside of the many files, it probably could be in one location and defined once. Whether that once is "global" to the route or "global" to the installation, or "global" to the machine is something I havn't considered but having to define it once is, IMO, a big, big plus. I don't think we should be using junctions via the OS... I don't want to be messing with somebodies PC that way, but if we could do that on our own, I think it would be very well received.