[jonas]

Just for my understanding - If this is the correct json file:

[
  {
    "Name": "Foo.s",
    "Type": "tree"
  }
]

And this would be the same json file with a typo:

[
  {
    "Name": "Foo.s",
    "Type": "tree"\
  }
]

Would the json parsers in general be robust enough to read the latter file flawless?

[jonas]

Meanwhile I've test it with "animated.clocks-or" and get this box:


To be fair: At least an error box pops up in OR with a hint where the error comes from.
But no error box with the OR-STF-Parser when reading the SD40-2, but then the SD40-2 isn't visible ...and one does not know why.

[YoRyan]

As you can see by running your snippet through JSONLint, it's not considered valid JSON:

Error: Parse error on line 3:
....s",	"Type": "tree"\}]
----------------------^
Expecting 'EOF', '}', ':', ',', ']', got 'undefined'

The point is not so much that the parser is "robust" - it's that you can expect every JSON reader ever written to follow the exact same logic. If it doesn't pass this validator, it's not JSON. Period.

Is the SD40-2 shape valid STF? Kuju thinks so. MSTS thinks so. Open Rails does not. But Open Rails has become the de-facto STF reader.

[Goku]

 YoRyan, on 28 April 2021 - 08:57 AM, said:

The SD40-2. Here's the file that doesn't load; it's straight from the MSTS 1.2 update. I have just tested it on master and on NewYear and it doesn't work on either. It's possible you've already repaired your own copy.

SD402.zip

It's the syntax that's badly defined about STF, not the names of the properties. In the case of the SD40-2, there's a backslash at the end of one of the lines. What is that supposed to mean to the parser? Kuju apparently just ignores it, but Open Rails doesn't. Should we change the behavior to match MSTS's? If we do, does this break any existing OR files that assume our current behavior?

But the case with this file may be just an user made bug.

 YoRyan, on 28 April 2021 - 08:57 AM, said:

You don't have these kinds of problems with JSON. If your data doesn't load correctly in your web browser, then it's not valid JSON.

So, maybe the SD40-2 file is not a valid file and MSTS parser just ignores this bug?

 YoRyan, on 28 April 2021 - 08:57 AM, said:

Actually, they don't. There's one and only one way to parse JSON. This information does not exist for STF.

No programmer is perfect and many of these fast libs don't have to follow the specs 100% correctly. Try opening advanced ODF file id different apps and the result may be completely different. It's common.

 YoRyan, on 28 April 2021 - 08:57 AM, said:

But it's a file produced by Kuju themselves. It's not a bug - it was intentional. The bug is in Open Rails itself, a consequence of the extreme difficulty of writing a consistent STF parser.

How do you know that it was intentional? It doesn't mean anything.

I think that you are looking for problems where there are none. If you don't know what to do with "\" character in STF file when parsing it, just don't allow using it in OR files.

[Goku]

Look, TSRE parser doesn't know nothing about "\" character, but this file works fine here:

https://i.imgur.com/WcSYOhq.png

I'm just a noob programmer and TSRE doesn't care about looking for errors. In the time of MSTS development it was common, handling errors makes things slower.

[YoRyan]

 Goku, on 28 April 2021 - 12:01 PM, said:

No programmer is perfect and many of these fast libs don't have to follow the specs 100% correctly. Try opening advanced ODF file id different apps and the result may be completely different. It's common.

The fact remains that there are JSON and ODF specs and there is no such spec for STF.

---

Let's flip the script: Is there a reason why this community is coming out of the woodwork to defend STF? Is there an argument being made here to change OR's new formats policy? Because if not, all of this talk really amounts to nothing more than a waste of everyone's time.

[jonas]

 YoRyan, on 28 April 2021 - 11:56 AM, said:

As you can see by running your snippet through JSONLint, it's not considered valid JSON: ...The point is not so much that the parser is "robust" - it's that you can expect every JSON reader ever written to follow the exact same logic. If it doesn't pass this validator, it's not JSON. ...

Yes, I see. Can we (OR) then make the external json files pass a "JSONLint test" so to speak first before reading them in to prevent the error box above?
Would certainly take too much loading time while the game is running with the many shapes and world files, but maybe with the smaller files that only read once, like the animated.clocks-or similar, that the end user "likes" to manipulate?

[YoRyan]

That error box provides the exact same information that a JSON linter would. How do you think OR should behave differently?

[jonas]

 YoRyan, on 28 April 2021 - 12:13 PM, said:

That error box provides the exact same information that a JSON linter would. How do you think OR should behave differently?

Oh, ok, I probably have a misconception of the hierarchical flows in OR code.
My way of thinking was assuming to avoid the parsing causing the error box and to test beforehand if it is correct json. Now I understand that the error box itself already comes from the parser. Is that correct?

[YoRyan]

That is correct, yes.