[JeroenP]

I started to investigate/make a prototype of the command-line-tool to load all supported file formats. This is in the roadmap for version 1.3: https://trello.com/c...ed-file-formats (blueprint is https://blueprints.l...ine-file-loader)
The longer term vision is mentioned in: https://trello.com/c...format-checking (blueprint https://blueprints.l...mat-diagnostics).

This topic is to discuss various aspects of this new tool. This includes your wishes and needs, and at a later stage also your feedback.

Initially it will, as said in the blueprint, accept any file format known to Open Rails and attempts to load it. The output would be all of the normal loading messages from the game (e.g. warnings from the STF parser) and a simple "OK" if it successfully reaches the end of loading. The important point here is that there should not be any new loading code: the already available code should be used since that already gives the proper warnings.
I think it should either accept a single file or a set of files using a search pattern. It should be possible to run it over a complete route and perhaps even over the complete set of files in TRAINS (at this point in time still considering mostly a single file at a time).

At a later stage it would then have an option not only to load a single file but also load, subsequently, all the files referenced in the initial file (e.g. loading an .act file would also load the referred .pat and .srv files, and I guess loading a worldfile would also lead to loading hazards, shapes, textures, ...). In the end, perhaps, it would be possible to load all files needed for a route, e.g. starting from a .trk file.

I am still looking for a good name for this tool. Do note that it is a command-line tool. So you would need to type it, possibly often. Furthemore, the long term vision is that also cross-checks between different files will be taken into account. So it is not just loading files. ContentChecker or CheckContent? FileChecker? CheckFiles? Just Check? Perhaps without capitals like check.exe or checkcontents.exe?

Any other thoughts people might already have?

Jeroen.

[James Ross]

JeroenP, on 16 February 2018 - 12:41 PM, said:

I am still looking for a good name for this tool. Do note that it is a command-line tool. So you would need to type it, possibly often. Furthemore, the long term vision is that also cross-checks between different files will be taken into account. So it is not just loading files. ContentChecker or CheckContent? FileChecker? CheckFiles? Just Check? Perhaps without capitals like check.exe or checkcontents.exe?

I personally favour "FileChecker" but it's not easy to name. Perhaps something like "FileValidator" or "FileVerifier" would work, but I worry that we'd be implying that our loading code is perfect when I am certain it is not.

[mrmosky]

As a potential user, I would like to see a gui for such a tool eventually, rather than a command line. That way, you can easily search for files to load, get any warning messages, see a progress bar, read a report of what has been done, all that useful stuff that anyone other than a programmer likes to see.
Geoff

[Eldorado.Railroad]

I like the command line idea where redirecting the output to a flat text file (ascii I hope) would be possible. This would dovetail with any scripts a user might create to use this detailed info. As for an executable name, well it is checking/crosschecking a bunch files so:

omnicheck.exe
omnichk.exe (shorter)
ORomnicheck.exe
ORomnichk.exe (shorter)
ORXcheck.exe
ORXchk.exe (shorter)
ORfileXcheck.exe X = for cross, as in cross check.
ORfileXchk.exe (shorter)
ORFXcheck.exe
ORFXchk.exe (shorter), but FX looks a bit too much like effects as in D3D FX...???!!!!
ORFileIntegrityChk.exe

I would like to see a non-cryptic but explicit name used so that a casual user will not be left scratching their head as to what this executable might do. For those of us who type at the commandline, we like super short exe names, but that is so 1980s DOS, 8.3. Either way nobody will really be happy. I suppose somebody could rename it to "orfc.exe" or "orxfc.exe".

Thank you for undertaking this task!

Steve

[JeroenP]

I have a first version, attached as a patch. Obviously this is not yet complete. However, I want to share this so that we have something to discuss more concretely. I added it is a ContentChecker project (the name is open for change) with as command line for now ORfileXcheck.exe, even though at the moment there is no real cross-checking going on.
Some .dat files are not supported yet. And I guess some openrails sub-directories are also not yet supported. Feedback is appreciated. For instance on how and where this needs to go into the Source directory. And of course whether it does what was intended.
I also added some initial documentation.

Attached File(s)

•  contentchecker.zip (22.44K)
  Number of downloads: 530
•  Content checker.pdf (103.89K)
  Number of downloads: 580

[cjakeman]

JeroenP, on 16 February 2018 - 12:41 PM, said:

I am still looking for a good name for this tool. Do note that it is a command-line tool. So you would need to type it, possibly often. Furthemore, the long term vision is that also cross-checks between different files will be taken into account. So it is not just loading files. ContentChecker or CheckContent? FileChecker? CheckFiles? Just Check? Perhaps without capitals like check.exe or checkcontents.exe?

How about "lint"?

Wikipedia writes:
"A linter or lint refers to tools that analyze source code to flag programming errors, bugs, stylistic errors, and suspicious constructs. The term is originated from a Unix utility that examined C language source code." It comes "from the name of the undesirable bits of fiber and fluff found in sheep's wool."

[Simon E]

cjakeman, on 20 February 2018 - 11:28 AM, said:

How about "lint"?

Wikipedia writes:
"A linter or lint refers to tools that analyze source code to flag programming errors, bugs, stylistic errors, and suspicious constructs. The term is originated from a Unix utility that examined C language source code." It comes "from the name of the undesirable bits of fiber and fluff found in sheep's wool."

Yeah, I'm very Familiar with "lint" on Unix. It Sounds Like a "Lint" for OR. It would be good to search for UN-closed brackets and other dumb things like that. Or Misspelt Items. Also Probably other stupid mistakes that are sometimes hard to notice.

[Eldorado.Railroad]

At what point will you have an alpha executable to test?

Thanks,
Steve

[JeroenP]

Simon E, on 21 February 2018 - 04:37 AM, said:

Yeah, I'm very Familiar with "lint" on Unix. It Sounds Like a "Lint" for OR. It would be good to search for UN-closed brackets and other dumb things like that. Or Misspelt Items. Also Probably other stupid mistakes that are sometimes hard to notice.

If I am not mistaken the current warnings you already get are things like un-closed brackets and other real syntax errors. So for sure, these errors are then also present in the command-line tool. But I think misspelt items are currently silently disregarded. Obviously it could be a future improvement to catch these as well. But that would then be in the second stage (and after 1.3?) because it impacts current parsing.
Furthermore, I do think that the future vision (see the Trello card for instance) contains more than just checking single-file constructs. It would check also consistency between different files (e.g. are the locations in a .pat file really present in the .tdb). Therefore, I think the name "lint" is too restrictive

Eldorado.Railroad, on 22 February 2018 - 06:33 PM, said:

At what point will you have an alpha executable to test?

I added alpha code to my post of Februari 20th. To generate code I need to check things in into SVN. I wanted to wait for some feedback, also again on the name of the project. Some people have downloaded the code. I got not feedback telling me it should be done differently, so I guess I can start checking in.

[JeroenP]

I had a look at the loading of various .dat files. And I am getting slowly convinced that files like speedpost.dat, spotter.dat, ssource.dat, and telepole.dat are not in fact used by ORTS. I could not find any code loading them. And I ran some routes through the debugger triggering on loading of .dat files and they did not pop up. So it seems to me that perhaps these files are needed in MSTS perhaps to do some initial loading or maybe even only for editing. But that they are not really needed in ORTS because ORTS can find the same information in for instance the world files.
The manual does say they are loaded. I suspect the manual is wrong. I did find an old post, http://www.elvastowe...rogramming-team, asking confirmation on whether these files are in fact loaded, but that confirmation never came.

Currently in the command line tool I am flagging them as 'Not used by ORTS''.

So, does anyone know for a fact that these files are loaded? Otherwise we should perhaps correct the manual.

[James Ross]

JeroenP, on 20 February 2018 - 06:05 AM, said:

I have a first version, attached as a patch. Obviously this is not yet complete. However, I want to share this so that we have something to discuss more concretely. I added it is a ContentChecker project (the name is open for change) with as command line for now ORfileXcheck.exe, even though at the moment there is no real cross-checking going on.
Some .dat files are not supported yet. And I guess some openrails sub-directories are also not yet supported. Feedback is appreciated.

Thanks! The executable name should match the project name, so it should make ContentChecker.exe. All the files are inside the Open Rails directory and will be set up with the appropriate metadata so it's clear they're part of OR, so that's not necessary in the filename. If there's any part of the metadata you're not sure about please ask - and I can easily fix anything that hasn't been filled in afterwards. :)

JeroenP, on 20 February 2018 - 06:05 AM, said:

For instance on how and where this needs to go into the Source directory. And of course whether it does what was intended.

I've been incredibly busy recently so haven't had a chance to look at the changes yet, but I would say that all the code goes in Source\ContentChecker and references the necessary other projects.

JeroenP, on 20 February 2018 - 06:05 AM, said:

I also added some initial documentation.

Brilliant!

JeroenP, on 24 February 2018 - 11:37 PM, said:

I had a look at the loading of various .dat files. And I am getting slowly convinced that files like speedpost.dat, spotter.dat, ssource.dat, and telepole.dat are not in fact used by ORTS. I could not find any code loading them. And I ran some routes through the debugger triggering on loading of .dat files and they did not pop up. So it seems to me that perhaps these files are needed in MSTS perhaps to do some initial loading or maybe even only for editing. But that they are not really needed in ORTS because ORTS can find the same information in for instance the world files.

I suspect most of these are only for the editor - although MSTS may still load them in-game.

JeroenP, on 20 February 2018 - 06:05 AM, said:

Currently in the command line tool I am flagging them as 'Not used by ORTS''.

This is perfect.

JeroenP, on 20 February 2018 - 06:05 AM, said:

So, does anyone know for a fact that these files are loaded? Otherwise we should perhaps correct the manual.

I've unfortunately not got time to double-check any of these specifically, but I would say that:

• If there's no mention of the file name in the OR source code
  
• And there's no mention of any variables pointing to the file name in other data files (for example, the way the level crossing sound file is loaded from DefaultCrossingSMS)

Then you can safely assume for now that we don't use it.

If anything does turn out to be used, we can correct the tool/manual/etc. at that time. Don't worry about being perfect right now!

[JeroenP]

I did a first SVN commit. Note the documentation is in Source/Documentation.