[RR1]

FWIW, there is an alternative to the proliferation of eng and wag files (and many other ancillary files used by MSTS/OR) caused by trying to manage data via text files and that is through the use of a database. This is probably of academic interest at this point in OR's life but I have built a database to store all of the eng and wag data. I have been picking away at this for a while now as I don't have a lot of time to spend on OR and I've had to learn C# but it works and has, I think, considerable advantages in managing the data over text files. At a minimum it completely eliminates worrying about files at all. The downside is that this approach would be a major change to most people's paradigm and would involve a bit of work to actually get the data into the database in the first place. Once there though, the process of how the data gets from the DB to OR can be completely transparent to the user. In a nutshell I'll indicate some of what I have done and what I see are advantages to this approach.

⦁ I used the OR code which reads and parses ALL eng and wag data to load the database. (it takes a few seconds to load an eng or wag file into the database. I can then add any metadata before saving or I can add the metadata to a group of records later)
⦁ Each variable is stored in exactly the same format as in the eng or wag file so when I recreate the eng or wag from the database, from OR's perspective, there is no difference from an existing eng or wag file.
⦁ The original source directory is stored in the DB along with a ton of metadata such as Country, RR name, era, type of eng or wag, etc. etc. etc.
⦁ Parameters are grouped by function, via a table for easy changes, (eg all parameters related to brakes) so I can easily get all GP9s from RR X and change a brake parameter in seconds.
⦁ The selected data is displayed in a spreadsheet format (datagridview) so changing a whole bunch of the same parameter is a simple copy/paste. I get a nice big view of the data.
⦁ I have no files to manage.
⦁ I have tried to eliminate eng and wag files completely by having OR read the data directly from the database but so far have had only limited success. (I still have much to learn about how to replace the STFReader in some places)
⦁ So the "simple" approach is to simply write the eng or wag file back to it's original location as soon as an edit is made to any variable in the DB and the changes are saved. This has absolutely no effect on how OR runs today.

But the real advantage of what I have done (again, in my view at least) is in writing the eng and wag files needed for the activity when the activity loads

⦁ I have inserted a module into OR when the activity is chosen which lets me "configure" the activity and eng/wag at run time (at least its supposed to)
⦁ This is still early days but I can, for example, specify that the train load is X%. 0% would be every car empty, 100% every car full, X% anything in between. This percentage of load is applied to the appropriate wag parameters (including updating friction parameters and anything else relevant) when the eng/wag files are written at the completion of this module.
⦁ I can also calculate the resulting load after applying my load percentage and then add (or delete) engines based on ruling grades, weather etc.
⦁ Any changes made at run time can be saved so that I can re-run a certain configuration any time.
⦁ My view is that con files and activity files (any OR file really) are also potential fodder for the database which would allow changing ANYTHING at load time. But that's way in the future. In the meantime I looking into also changing some activity parameters during load. But this depends on how OR works and I'm still trying to sort that out.
I haven’t figured everything out yet but for now at least the bottom line for me is that I no longer have to care about eng or wag files at all. Right now simply writing a new eng or wag file after making database updates changes nothing in how OR works. I can change parameters on 1 car or 100 - takes almost the same amount of time. Eventually, once I get more of OR figured out, they will go away completely (except for the variables that consist editors currently need).

[R H Steele]

I find this very interesting...isn't this what other simulators use? Like RUN8 for instance.
The only thing I wonder about, is making this method compatible with ORTS native parameters...like the ORTSDieselEngines block, and any other parameter prefaced by "ORTS"Thanks for posting...

[Genma Saotome]

As a professional data architect I thought of this many years ago but chose not to push the idea forward because I was concerned end users would not be able to administer their database properly (low odds of something going wrong but life happens) combined with nobody charged to help them. The other block of course is the need to write an app for the front and back ends.

In lieu of that I proposed the Include() statement which would allow normalizing the data in .inc files, something end users could easily understand and administer. But the idea of using a DBMS is still very interesting.

What DBMS are you using?

Have a data model you can share?

[RR1]

Genma Saotome, on 29 June 2018 - 10:31 AM, said:

As a professional data architect I thought of this many years ago but chose not to push the idea forward because I was concerned end users would not be able to administer their database properly (low odds of something going wrong but life happens) combined with nobody charged to help them. The other block of course is the need to write an app for the front and back ends.

In lieu of that I proposed the Include() statement which would allow normalizing the data in .inc files, something end users could easily understand and administer. But the idea of using a DBMS is still very interesting.

What DBMS are you using?

Have a data model you can share?

I very much share your concerns about the complexities, both in the code and with support, in actively pursuing a DBMS. It isn't trivial. But for me, it's a learning experience (always like learning new stuff) and a way for me to better enjoy using OR without the "chore" of organizing files. I'm just a bit lazy in that respect!

Anyway, I'm using SQL Server mainly because when I started this exercise I really had no idea where to start and I found lots of examples through Google in creating/attaching databases to C# projects which used SQL Server.

I'm not in the least concerned with optimizing the data storage since I'll only ever have a few thousand? records at most, not millions, so 'brute force 101' is the order of the day.

I don't have a graphical data model so I'll try to quickly describe the direction I took.

There are currently 18 tables.

- 4 tables are used to store all wag data variables, all diesel engine variables, all steam engine variables and all electric engine variables. These are essentially flat tables which, when loaded, display in a grid view every variable in for every wagon or engine type unless filtered. Yes, a number of fields are duplicated but again, I'm not concerned with performance and I found it easier to separate the engine types and focus on them individually when trying to sort out all of the variables in the first place. I can edit data just like in excel. I can duplicate wagons and engines. I'm using a composite key in these tables comprised of the ident which is the current name of the wag or eng plus its full path. This lets me have duplicate idents to cater for miniroutes although I will consolidate in the future.
- 3 tables are used for lights, light states and ORTS Freight Animation data. These are separate because the number of lights, states and animations can vary greatly.
- 4 tables are used for the parameter group filters. For example, parameter groups include general data, physical characteristics, brakes, couplers, files, ORTS, etc. There is one for each of the 4 main tables. These are easy to adjust.
- 6 are used in the dropdowns in the datagridviews for entering meta data such as country, car description, engine description RRnames and so forth. Basically anywhere I need consistent data (spelling) since these fields are used to filter what it is I want to edit. These tables can be updated with new information entered on the datagridviews as I go.
- 1 table contains the image data for the engine or wagon whose line in the gridview has focus. I can click on the image and call up the shape file viewer.

The data fields for the wag and engine variables are driven by keeping an identical format for the data as is found in the eng and wag files. It's not the best way to store some fields but it certainly simplifies the re-creation of the eng and wag files when I write them back. If I ever figure out how to have OR read the tables directly I can improve the formats of some of the more complicated fields. Or not as the case may be.

To me this is worth the effort because of the ability to dynamically create the eng and wag files at run time. I'm just getting into this phase and this is the exciting part.

[RR1]

R H Steele, on 29 June 2018 - 09:41 AM, said:

I find this very interesting...isn't this what other simulators use? Like RUN8 for instance.
The only thing I wonder about, is making this method compatible with ORTS native parameters...like the ORTSDieselEngines block, and any other parameter prefaced by "ORTS"Thanks for posting...

ALL ORTS parameters are supported. Every time I think I'm done I find a few more! The new ORTS Freight Animations were a bit of a challenge.

[Genma Saotome]

I use this for data modeling: https://sourceforge....bdesigner-fork/

I'm pretty sure it can handle sql server (I use Mariadb).

[RR1]

Thanks for the link. I'll give this a try.

[Genma Saotome]

RR1, on 30 June 2018 - 05:06 AM, said:

Thanks for the link. I'll give this a try.

Any luck w/ that software?

[RR1]

Genma Saotome, on 18 July 2018 - 04:27 PM, said:

Any luck w/ that software?

Sorry, been distracted with summer! I thought retirement would give me more time to devote to this project but not so. To many other (good) distractions to keep me away from the confuser. Anyway, I did try that s/w but couldn't connect to my DB. I didn't spend much time trying to troubleshoot though.

But it did get me thinking and I've probably given the wrong impression when I implied I was "using a DBMS". I also don't have a data model since none of my tables are related!

To clarify, the database file (collection of tables) is an SQL Server file but created entirely from within Visual Studio/.net. I don't actually have a full version SQL Server installed. It's just what Visual Studio needs to muck about with tables. So no DBA skills needed. My C# program either uses or can use C#/.net to query, update and even manage (add/delete tables, table columns, add/delete data to tables, tables used for dropdowns, etc., etc.) all aspects of the "database". So it's really just some C#/.net reading and writing to some tables. SQL queries fill a datatable with data which is then viewed and edited in a datagridview. Saving the changes is handled entirely by .net so I have no SQL Update queries. I pull with an SQL query and C#/.net does all the pushing. (For me, this is KISS). So at this point no DBMS skills needed to administer the database (everything not coded is easily accomplished through Visual Studio).

I have also realized that I can implement this database approach in parallel with the current OR setup. Basically, if the database file exists then, just prior to actually starting to load data (read eng/wag files) call the module that can "configure" certain eng/wag variables before writing an updated eng/wag file, then continue. If no database, simply skip this module and carry on just like today. This will make it easy for me to keep a current version of OR by simply dropping my module into the updated source when needed.

Hope this clarifies what I'm doing . . .

John