[disc]

gpz, on 25 November 2015 - 02:22 PM, said:

I don't really know how to proceed with such changes. Certainly I will not commit it to svn.

Just create a blueprint on launchpad, with the draft of the changes, and wait for approval :)

[gpz]

Traditionally, it is not kind of a thing that can be handled by a simple blueprint. Introducing a new world file format is a thing that needs wide agreement among the community members. And, also traditionally, noone likes to talk about it, beyond saying that "Hey, it's great, but it is too early. Maybe a year later." :)

[Genma Saotome]

gpz, on 26 November 2015 - 05:02 AM, said:

Traditionally, it is not kind of a thing that can be handled by a simple blueprint. Introducing a new world file format is a thing that needs wide agreement among the community members. And, also traditionally, noone likes to talk about it, beyond saying that "Hey, it's great, but it is too early. Maybe a year later." :)

IMO the choice of .json vs. something else is almost 100% a technical issue and as such should be left to the OR team. There should be a few user requirements attached, such as (1) editable and (2) not excessively verbose. AFAIK, this has already been addressed and IIRC .json was chosen (but my recollection may be off).

Beyond that lies a second set of issues: What data should be held in said file? AFAIK that has not been discussed, at least not beyond tossing a few ideas around (e.g., moving signaling data to a different file). Also within this area are things like how large an area should each tile cover (e.g., Kuju's 4km^2 or something else, such as 1km^2?). In this discussion care must be given to a matter that KUJU handled poorly and that is the referential integrity between their .w file and all related files -- interactives plus the rdb/tdb. This discussion must be done ASAP. That said, again I am of the opinion this is almost entirely a OR Team decision w/ little to necessary input from the larger community.

Last thing that comes to mind is the question of whether there will be some utility provided by the OR team that translates and moves existing w. file data to their go-forward files. I don't see a need to provide that right away but eventually it is necessary and on that count I'm sure the community would declare it as an essential requirement..


Did you have something else in mind?

[gpz]

Genma Saotome, on 26 November 2015 - 01:27 PM, said:

Last thing that comes to mind is the question of whether there will be some utility provided by the OR team that translates and moves existing w. file data to their go-forward files. I don't see a need to provide that right away but eventually it is necessary and on that count I'm sure the community would declare it as an essential requirement..

This, fortunately, comes almost automatically with json de/serializer. When the underlying class is built, that handles the json format deserialization (a.k.a. reading), it can do the serialization (writing) too, without extra programming overhead.

Currently, everywhere in the simulator code the MSTS object is used directly. STF reader creates the object, and the resulting object is used directly elsewhere. This must be changed: Whenever the STF reader has finished its job, the resulting object must be converted to the new, custom object, that directly represents the underlying json structure, and this new object must be used in the simulator code. This object can be written to disk in json format at any time then. I did this conversion easily with forest and carspawner objects, however I haven't yet tried to output converted world files to disk on the fly, at reading time. But should be possible, with a single command.

I believe Wayne mentioned earlier, that he already made this work for the whole MSTS data structure, but maybe I misunderstood something. And also he is not around here these days. :(

[Genma Saotome]

gpz, on 26 November 2015 - 02:12 PM, said:

This must be changed: Whenever the STF reader has finished its job, the resulting object must be converted to the new, custom object, that directly represents the underlying json structure, and this new object must be used in the simulator code.

I'm not so sure that is so.

Obviously a greenfield project would do so... or something very much like that, but it isn't obvious to me why ripping out the existing object classes to replace them w/ something that, in most cases are going to be remarkably similar, if not identical, is called for as opposed to simply adjusting the existing class as needed and changing (or ultimately replacing) the loader program.

To add a unpleasant wrinkle, what about that period of time when both old and new are in use, perhaps even files of the same purpose? Either new inputs need to be placed into the existing classes or old inputs into the new ones. Awkward to say the least.

[gpz]

Remembering, that you were the founder and biggest supporter of idea of dropping MSTS data format support in future versions, it is interesting for me, that now you are arguing pro bundling the formats unbreakably once and forever. :)

BTW, I am not against any alternative, but for me (not being a true programmer my opinion might not be the most relevant one) the separated code looks way much nicer by looking on it, than a smashed thing. (In my mech. engineering practice, we always say that something that doesn't look nice, usually is also proved to be wrong later.)

[roeter]

Changes and alterations to world-files have been discussed before, but the consensus at the time was that, while we still depend on the MSTS Route Editor, changes or enhancements to files directly created by the RE could cause all kind of problems and should therefor be avoided.
I have been looking through the book of tricks to see if there's something in there that could help out here, but as not everybode likes tricks I have kept this idea under the hat. But with the discussion on the world files getting stuck a little, perhaps it is time to see if there is indeed a little trick which could get this thing rolling. And there is - not even a bad trick at that, I think.

Earlier, I suggested to use an alternative forest definition in the OpenRail subdirectory to define these multi-tree forests. It was correctly pointed out that this could not be done just like that, as only the RE uses the forest definition, the world files only refer to the used ace files.
But - if you can't have what you want, why not use what is there?

So here is how my little trick would work. Rather than simply using the ace-file directly and only as ace-file, why not use this as a reference to something else?

Suppose you want a forest consisting of beech, birch and oak. You then create a forest in the forest.dat file using an ace-file named BeechBirchOak.ace. And you create that ace file, and never mind what it looks like. The MSTS RE will now create that forest, and set the world entry to BeechBirchOak.ace.

The alternative forest definition in the OpenRail subdirectory would now look like this :

Forest ( "Diversified NE Forest" ( 3
                  Reference ( "BeechBirchOak" )
                  Forest ( "Oak.ace" 41.0f 30.0f 0.9f 1.1f 60)
                  Forest ( "Beech.ace" 16.0f 18.0f 0.9f 1.1f 30)
                  Forest ( "Birch.ace" 12.0f 22.0f 0.9f 1.1f 10 )
                                )
       )

OR will read the alternative forest definition at the start of the program, and create a cross-reference list.
When reading a forest entry from a world-file, it will first check if the ace is used as reference for an alternative forest, and if so, use that alternative definition, otherwise it simply uses it as an ace file. When the alternative definition is used, it combines the world forest entry with the alternative definition to create the required new forest entry which it then can process normally.

The required program change would be quite simple, and the setup for users is quite straightforward as well. The little reference trick can be dropped as soon as we no longer rely on the MSTS RE.

Regards,
Rob Roeterdink

[gpz]

You are right in saying, that for forest redefinition actually there is no world file extension is needed. The texture name there might function as a key in an OR custom dictionary. My only concern is that for really new OR files it would be better to drop this STF format in favour of JSON. Our STF reader has many small unreliabilities and bugs, and the format itself is also obsolete and non-standardized. E.g. just look at that number "3": that is nor used, and neither is necessary.

Unfortunately this trick cannot be done for car spawners, since in MSTS world file they are unnamed. Only an other unused-but-available property could be used for referencing, like "Position" or "StaticFlags", but this more and more looks like a really ugly hack.

[Genma Saotome]

gpz, on 26 November 2015 - 11:30 PM, said:

Remembering, that you were the founder and biggest supporter of idea of dropping MSTS data format support in future versions, it is interesting for me, that now you are arguing pro bundling the formats unbreakably once and forever. :)

Is this comment for me? If so, I don't understand. Other than requirements for editablity / readablity I'm actually agnostic on the matter of which physical format... any reference I've made to .json is simply with respect to what the OR team has told me is a pretty good choice. As for the second half of your comment, more below....

gpz, on 26 November 2015 - 11:30 PM, said:

BTW, I am not against any alternative, but for me (not being a true programmer my opinion might not be the most relevant one) the separated code looks way much nicer by looking on it, than a smashed thing. (In my mech. engineering practice, we always say that something that doesn't look nice, usually is also proved to be wrong later.)

By and large I agree. When I did code I often "improved" a few things outside of what I was supposed to do and always argued for getting the data relationships right the first time around because it would be so hard to change them later (e.g., should signalling and other inter actives be in or out of the .w file: Logically they are their own objects (there is an associated relationship between a section/instance of track and an placed instance of signal) but is it necessary to physically set them out in their own files? -- That is a very important question.

The point I was trying to make in my previous post is it is not obvious to me that the existing objects are sufficiently different from any future-state file content as the natural state of the data really isn't changing much (just a few attributes being added somewhere)... nor am I aware of any serious design flaws w/ the existing objects. OTOH, if it is generally understood that some of the objects and their methods would be improved by re-thinking then yes, a time like this when future features are under discussion would be a good time to look at the object designs. The answer to that question is entirely up to the programmers.

[Genma Saotome]

roeter, on 27 November 2015 - 02:31 AM, said:

The required program change would be quite simple, and the setup for users is quite straightforward as well. The little reference trick can be dropped as soon as we no longer rely on the MSTS RE.

Regards,
Rob Roeterdink

An off the wall thought: What about using the data you wrote out as an input to a program that creates an .s file (also using the existing logic to determine all xyz values). That way the only change inside of the .w file is to replace via a text editor the lines for the forest with lines for the static object just created. Such text editing is trivial and (for now) there is no need for any new file structures. The new program could even spit out the .w lines describing the static item and if it was rather bold, do the editing in the .w file too.