Elvas Tower: OR consist format - Elvas Tower

Jump to content

Posting Rules

All new threads will be started by members of the Open Rails team, Staff, and/or Admins. Existing threads started in other forums may get moved here when it makes sense to do so.

Once a thread is started any member may post replies to it.
  • 24 Pages +
  • 1
  • 2
  • 3
  • Last »
  • You cannot start a new topic
  • You cannot reply to this topic

OR consist format Let's talk details Rate Topic: -----

#1 User is offline   YoRyan 

  • Conductor
  • Group: Status: Active Member
  • Posts: 391
  • Joined: 19-February 20
  • Gender:Male
  • Location:California, United States
  • Simulator:Open Rails/unstable
  • Country:

Posted 06 July 2020 - 04:33 PM

Revised 2020-08-19

This is an attempt to take a clean-sheet look at the problem of spawning railcars for use in the simulator.

Definitions:

  • "Session": A gameplay context, be it an activity, timetable, or a route/path/consist combination selected for explore the route mode. Sessions define one or more trains.
  • "Train": An entity in the Open Rails world that consists of one or more railcars connected in sequence. Trains can divide and combine with other trains. A string of wagons parked in a siding (for the purposes of this specification) make up a train. When you couple to them, you are combining their "train" with your own. All attributes of a train are defined by the loaded activity or timetable session.
  • "Car": Any rail vehicle, which is at all times attached to a single train - trains cannot share cars. While MSTS required that all articulated rail vehicles be modeled as separate .eng and .wag files, and therefore separate cars, we can expect this limitation to be abolished for the eventual Open Rails rolling stock format. Some attributes of a car are facts that are defined in its rolling stock file; others will be overlaid by the active session.

Objectives:

  • We want to design a mechanism to spawn trains and their attached cars, each with their desired attributes.
  • "Modability": We want to make it easy for players to customize activities and timetables to use their own equipment.
  • "Variability": We want mechanisms that can select among alternative sequences of railcars, so as to vary the gameplay and make it more interesting.
  • "Reusability": It should be possible to share sequences of railcars and attributes among multiple trains, so as to reduce the input burden on the session designer.

Assumptions:

  • We are dealing only with the problem of assigning attributes to cars. Speculation on how Open Rails would make use of them is welcome, but largely outside of the scope of this proposal.
  • We will not explicitly link or group cars together, except by assigning common attribute-value combinations to them. It is up to the simulator's other subsystems to interpret this data.
  • We will share mechanisms and file formats, but not data, between the various types of sessions (activity, timetable, explorer), as I suspect few players think it important to share consists between the different modes. In particular, we will be breaking Kuju's link (the "consist" format and directory) between player train definitions in activity mode and player trains in explorer mode.

By shedding Kuju's shared consist database, we are essentially moving toward the RailWorks model, where train and car information are strictly part of the session format. However, we will be adding our own mechanisms to spice up our sessions with some unpredictability.

With that said, here is my proposed data model. Attributes are on a "including but (where they make sense) not limited to" basis.

Sessions contain:
  • A starting date (time is determined by the first active train)
  • The state of the weather
  • A collection of one or more trains to spawn immediately, or at some future point

Trains contain:
  • A dispatcher symbol (if active when spawned)
  • A priority class (if active when spawned)
  • A maximum authorized speed (if active when spawned)
  • A start time or trigger
  • A spawn location
  • Instructions to perform, such as passenger stops to make, locations to travel to, and pick ups and drop offs to make (unless in explorer mode)
  • A sequence of one or more cars and their attributes

Cars contain:
  • A reference to the rolling stock definition (.eng, .wag, or future format) to load into the simulator
  • A starting orientation (forward or reversed)
  • Initial states for pertinent controls, such as the reverser and pantograph
  • A starting fuel or water level (for locomotives)
  • A starting coal level (for tenders)
  • A starting load (for freight wagons)
  • An acceleration tolerance (durability) (for freight and passenger wagons)
  • Data tag(s) used by by the current active session to specify and track objectives
    • For example, activity sessions might tag certain cars for drop off at a particular siding.
    • Timetable sessions would tag cars for use in $attach and $detach commands, succeeding the current method of associating cars with their originating consists.

Anticipated structures:

Native activities should store their train information in a new "train" file to be distributed as part of the activity package. Likewise for timetables, although encoding this information into one of the csv rows is also an option. Explore the route "trains" will continue to be referred to as "consists" and the sole train attribute will be the display name in the menu. In any case, all train formats contain the same information: a list of cars plus their corresponding initial state attributes.

There will be another format, call it a "car list," that fulfills the variability and reusability objectives. It will resemble a stripped-down train file without any of the train attributes. Car lists can apply any attributes to their items, which they string together, and they can contain cars or references to other car lists. In addition, "car list randomizers" can select from their mutually exclusive items on probabilistic basis, while "car list pools" can select one from a depleting pool of items. Through nesting, any conceivable combination of randomized cars, plus their attributes, is possible.

Car lists can be shared among multiple trains, and they, too, should be distributed as part of the activity or timetable package. Attributes that a train applies to a car list will apply to all items generated by the car list, on an additive basis - if a car list applies attribute "X=1" to its cars while its parent train applies attribute "Y=2" to the car list, the cars generated by the car list will have attributes "X=1;Y=2". Trains will not be permitted to reference other trains.

Consists (trains for explore the route mode) will be created by the player or can be distributed along with rolling stock packages. Consists will also support car lists.

Regardless of how a train is constructed, the end result made available to the rest of the simulator is a flat linear list of engines or wagons and their corresponding attributes.

How current data formats fit into this model:

  • Activities (.act) and timetables (.timetable-or) define sessions. So does the player when he or she selects a route, path, and consist for explore the route mode.
  • These sessions define collections of trains.
  • The consist file (.con) defines:
    • A maximum speed, which applies to all trains spawned using this consist.
    • A durability value, which applies to every car of all trains spawned using this consist.
    • A list of cars, minus any attributes, that is duplicated for every train spawned using this consist.

JSON formats:

Now, let's move on to the data formats. In general, all of the structures are going to involve list of items, where items may be references to rolling stock (engines and wagons) or "segments," the proper name I've decided upon for the "car list" concept described earlier.

At this time, I will not define train formats for use by activities and timetables, but their construction should be fairly self-explanatory.

Consist:

A consist represents a train "template" for use in explore the route mode. Its only properties are a display name for use in the main menu and a list of items.

// myexplorer.consist-or
{
    "DisplayName": "My Explorer Consist",
    "Items": [
        ...
    ]
}

Item:

An item is a JSON block that represents a reference to a rolling stock wagon or a segment. Items can be repeated, can be flipped, and can introduce session-specific attributes in an additive relationship.

{
    "Item": "/MSTS/TRAINS/TRAINSET/dash9/dash9.eng",
    "Count": 2,
    "Flip": true,
    "Session": {
        "Attribute1": "value1",
        "Attribute2": "value2",
        ...
    }
}

Segment:

A segment is a reusable block of wagons or other segments that can be referenced by multiple trains. (I picked the "segment" name because its meaning has not been overloaded by real-world railways, and it conveys a sense of "something that's part of something larger.") Segments are self-contained files, which permits nesting.

The "list" segment type combines all of its items in a linear order:

// mylist.segment-or
{
    "Type": "list",
    "Items": [
        ...
    ]
}

The "random" segment type selects one of its items given an accompanying list of probabilistic weights:

// myrandom.segment-or
{
    "Type": "random",
    "Items": [
        ...
    ],
    "Probabilities": [
        /* The first weight corresponds to the first item, the second weight the second, etc. */
        0.2,
        0.3,
        ...
    ]
}

The "fallback" segment type is intended for activity designers to gracefully handle missing equipment. It attempts to load the first item, and if it contains an unknown path it falls back to the second item, and if that item contains an unknown path it falls back to the third item, and so on.

// myfallback.segment-or
{
    "Type": "fallback",
    "Items": [
        ...
    ]
}

The "pool" segment type simulates a fixed equipment pool. When first loaded, it randomly selects an item, and then when loaded again, it selects another distinct item, and then yet another item, and so on. When its "pool" of items is exhausted, it restarts from the first selected item.

// mypool.segment-or
{
    "Type": "pool",
    "Items": [
        ...
    ]
}

Conclusion:

In my opinion, this data model puts Open Rails on a sound footing to introduce new features and deeper models of rail operations, including native replacements for the consist and activity formats inherited from MSTS.

______________________________________________

Original post follows:

I'd like to revive the topic of a native consist format for OR. There was a big discussion created about this some years ago, but it ended inconclusively, and also kind of got side-tracked by a content manager proposal. Here, I'd like to focus on a concrete proposal for a greenfield, JSON-based consist format with additional OR-exclusive goodies.

It would be our very first native data format - but by my reading of the TrainCfg parser code, it shouldn't be too difficult to pour some virtual concrete, so to speak, once we've settled on the particulars. And it also wouldn't touch any of the low-level graphics subsystems that are currently up in the air due to the ongoing Monogame migration.

As a reminder, the MSTS .con format consists of the following information:

  • Consist name
  • Consist durability
  • Maximum permitted speed
  • Serial number (unused)
  • Next free wagon UID (unused)
  • A list of wagons, each with:
    • A UID
    • The TRAINSET folder name
    • The .eng or .wag filename (without the extension) in an "EngineData" or "WagonData" block, respectively
    • A flag indicating a forward or reverse orientation

And here is a sample:

SIMISA@@@@@@@@@@JINX0D0t______

Train (
	TrainCfg ( "GP38-2 lumber"
		Serial ( 1 )
		MaxVelocity ( 36.65728 1.00000 )
		NextWagonUID ( 4 )
		Durability ( 1.00000 )
		Engine (
			UiD ( 0 )
			EngineData ( GP38 GP38 )
		)
		Wagon (
			WagonData ( US2WoodChipper US2WoodChipper )
			UiD ( 2 )
		)
		Wagon (
			WagonData ( US2FCarYE2 US2FCarYE2 )
			UiD ( 3 )
		)
		Wagon (
			WagonData ( US2EmpLoggerCar US2EmpLoggerCar )
			UiD ( 1 )
		)
	)
)

My proposal would largely mirror Kuju's data fields, with some minor alterations:

  • Drop the unused "Serial" and "NextWagonUID" fields
  • Drop the "UID" field for each individual wagon; have Open Rails auto-generate these if they are truly necessary
  • Consolidate folder and wagon name into a single "path" field - but continue to omit the file extension, because in the future, native "engine-or" and "wagon-or" formats should supersede their MSTS counterparts

The new format would use the "consist-or" extension and it would supersede any .con file with an identical base name, in all contexts (activity, timetable, explorer).

{
    "name": "GP38-2 lumber",
    "maxVelocityMpS": 36.65728,
    "durability": 1.0,
    "formation": [
        {
            "type": "engine",
            "path": "GP38/GP38",
            "flip": false
        },
        {
            "type": "wagon",
            "path": "US2WoodChipper/US2WoodChipper",
            "flip": false
        },
        {
            "type": "wagon",
            "path": "US2FCarYE2/US2FCarYE2",
            "flip": false
        },
        {
            "type": "wagon",
            "path": "US2EmpLoggerCar/US2EmpLoggerCar",
            "flip": false
        }
    ]
}

Going beyond a 1:1 MSTS replacement, I would also like to propose the following additions:

Consists should be able to refer to other consists.

{
    "type": "consist",
    "path": "gp38",
    "flip": false
}

There should be a special "formation" type that instructs Open Rails to select from a number of alternatives on a probabilistic basis.

{
    "type": "formation",
    "formation": [
        {
            "type": "consist",
            "path": "gp38",
            "flip": false,
            "probability": 0.7
        },
        {
            "type": "consist",
            "path": "dash9",
            "flip": false,
            "probability": 0.3
        }
    ]
}

(Formation types should be able to be recursively defined, although perhaps we wouldn't expose this in the eventual consist editor.)

As a stepping-stone to an eventual content manager, it should be possible to refer to content in another installation profile. Rather than specifying an absolute filesystem path, I suggest using the "Name" field as entered by the user as the identifier:

{
    "type": "consist",
    "path": "Senator Vincent",
    "profile": "PRR Eastern Region",
    "flip": false
}

Any and all feedback is welcome!

#2 User is offline   eolesen 

  • Superintendant
  • Group: Private - Open Rails Developer
  • Posts: 1,546
  • Joined: 30-October 10
  • Gender:Male
  • Simulator:ORTS
  • Country:

Posted 06 July 2020 - 05:49 PM

Reasonable... can we set some defaults and avoid declaring in each and every object?


1) Presume Flip = false

2) Presume type = wagon unless declared as engine or consist


Behavior question if we drop file extensions... what happens if I declare an engine as a wagon, and there's only an .eng or engine-or file for that object? Does it show up as unpowered (desirable outcome) or will it light up and smoke based on the leading engine?

#3 User is offline   YoRyan 

  • Conductor
  • Group: Status: Active Member
  • Posts: 391
  • Joined: 19-February 20
  • Gender:Male
  • Location:California, United States
  • Simulator:Open Rails/unstable
  • Country:

Posted 06 July 2020 - 06:07 PM

Of course - "flip" as false is a reasonable default.

Defaulting the formation type, I am not so comfortable with. The "path" could refer to one of an engine, wagon, or consist, and I think that's just a little too ambiguous. If I'm not mistaken, Open Rails throws up an error if you swap an "EngineData" block for a "WagonData" block in a .con file, so we would do likewise in your scenario. For now.

#4 User is offline   conductorchris 

  • Vice President
  • Group: Status: First Class
  • Posts: 2,339
  • Joined: 24-March 10
  • Gender:Male
  • Simulator:Open Rails - MSTS
  • Country:

Posted 06 July 2020 - 06:47 PM

This seems like a good little project that shouldn't be too hard.

While in other contexts, "Path" matches how you are using it, in Open Rails is opens the possibility of confusion with the path along tracks that is in activities. Can we think of another name? FileLocation ? Or something better.

I don't know anything about how other simulators do consist files, but perhaps we should borrow ideas from them. (Or even use their formats?)

I would think it is very important to check in with Goku and get his input and participation.

The UID is used in game when you press F9 and open the window to uncouple or put on handbrakes and so on. This always annoyed me because I've always wanted that to correspond to the number that matches the actual rolling stock. Of course that information is not a standard field in the .wag and .eng file, but maybe it should be. In the American context, you'd want reporting marks (railroad initials) and number, which could be one field. Perhaps the same is true in Europe?

I love the ability to refer to other consists. I wonder if that should be in the format of include files the same way that .eng and .wag files now do?

I really really love the formation idea with a probability. I would suggest that probability be optionally defined (ie, the default probability would be 1) for each .eng or .wag regardless of it it is in a "formation" or not. Thus the length of the consist could end up varying from day to day if this option was used. I think that formations themselves should have the ability to have a probability defined. Formations should return either 1 unit (default) or multiple units or the probability based resul. Example of how this would be. A USA local freight. Formation of 2 .eng from a pool of a dozen possibilities. 3 .wag,predetermined. Then a formation for the largest customer of 6-10 .wag (depending on probability) out of a pool of 40 possibilities. Then several individual .wag with probabilities of less than 1 for several small customers. And so on.
I'm not so sure "formation" is the right term for this concept (in USA we don't use the term; the Brits use "formation" to essentially mean "consist" I believe.)

Christopher

#5 User is offline   conductorchris 

  • Vice President
  • Group: Status: First Class
  • Posts: 2,339
  • Joined: 24-March 10
  • Gender:Male
  • Simulator:Open Rails - MSTS
  • Country:

Posted 06 July 2020 - 06:53 PM

I just read the thread on the pool concept in timetables.
Perhaps it makes sense to include that in consist files.
That is, an activity file calls a consist but gets a file defining a pool made up of any number of other consists which can be used based on a probability. (that is, not like an include file where consists are added together but where one or a different one is used randomly.

#6 User is offline   YoRyan 

  • Conductor
  • Group: Status: Active Member
  • Posts: 391
  • Joined: 19-February 20
  • Gender:Male
  • Location:California, United States
  • Simulator:Open Rails/unstable
  • Country:

Posted 06 July 2020 - 08:03 PM

View Postconductorchris, on 06 July 2020 - 06:47 PM, said:

While in other contexts, "Path" matches how you are using it, in Open Rails is opens the possibility of confusion with the path along tracks that is in activities. Can we think of another name? FileLocation ? Or something better.

View Postconductorchris, on 06 July 2020 - 06:47 PM, said:

I'm not so sure "formation" is the right term for this concept (in USA we don't use the term; the Brits use "formation" to essentially mean "consist" I believe.)

Good points, and I am absolutely open to clearer and less ambiguous terminology.

View Postconductorchris, on 06 July 2020 - 06:47 PM, said:

I don't know anything about how other simulators do consist files, but perhaps we should borrow ideas from them. (Or even use their formats?)

Somebody has to ask Dovetail Games or 3DTrainStuff very nicely, because the consist formats of Train Sim World and Run 8 are not public information. :D

(But in all seriousness, the probabilistic substitution feature is an innovation by Train Sim World.)

View Postconductorchris, on 06 July 2020 - 06:47 PM, said:

I would think it is very important to check in with Goku and get his input and participation.

As always, but note that Open Rails has always planned to develop its own data formats and editors, regardless of the intentions of the TSRE project.

View Postconductorchris, on 06 July 2020 - 06:47 PM, said:

The UID is used in game when you press F9 and open the window to uncouple or put on handbrakes and so on. This always annoyed me because I've always wanted that to correspond to the number that matches the actual rolling stock. Of course that information is not a standard field in the .wag and .eng file, but maybe it should be.

I agree that this is a .wag/.eng (or .wagon-or/.engine-or) problem and doesn't need to be under the purview of the consist format.

View Postconductorchris, on 06 July 2020 - 06:47 PM, said:

I love the ability to refer to other consists. I wonder if that should be in the format of include files the same way that .eng and .wag files now do?

Include directives are a hack on top of the old MSTS formats. Explicitly referencing other consists accomplishes the same thing, but much more elegantly.

View Postconductorchris, on 06 July 2020 - 06:53 PM, said:

I just read the thread on the pool concept in timetables. Perhaps it makes sense to include that in consist files.

This is exactly what the probabilistic system would allow. The activity would reference a consist that, in turn, could reference any number of alternative sub-consists with corresponding probabilities.

#7 User is offline   pschlik 

  • Conductor
  • Group: Status: Active Member
  • Posts: 311
  • Joined: 04-March 15
  • Gender:Male
  • Simulator:OpenRails - Unstable
  • Country:

Posted 06 July 2020 - 08:09 PM

I feel this approach is quite nice. Gives us what MSTS can already do, while providing an excuse to fit in some new features, and I can't argue with that.

The possibility of extra variety through randomization is especially important but can be quite complicated when it comes to how information is presented to the player. Currently, many parts of the interface rely on the assumption that each consist has only one possible lead locomotive. Adding many possible lead locomotives could cause trouble, as the same consist would then appear multiple times for each possible lead locomotive. There's also the information screen for an activity, which includes the name and description of the lead engine that is in the player consist. If randomization occurs, that text could easily become wrong [eg: activity menu tells me "Locomotive: GP38-2" but I subsequently find myself in a Dash 9] (unless the randomization occurs before the activity is loaded, but I suspect that wouldn't be the case.)

There will likely need to be some changes to the way the main menu works to handle this appropriately.

#8 User is offline   YoRyan 

  • Conductor
  • Group: Status: Active Member
  • Posts: 391
  • Joined: 19-February 20
  • Gender:Male
  • Location:California, United States
  • Simulator:Open Rails/unstable
  • Country:

Posted 06 July 2020 - 08:34 PM

Yes, that is very important to keep in mind, too - and we also need to think about the command-line interface necessary to select one of a number of possible consists.

Perhaps - in the spirit of the "player-drivable" flag for paths - we could designate certain consists as selectable in the main menu, and others as AI-only.

#9 User is offline   steamer_ctn 

  • Open Rails Developer
  • Group: Status: Elite Member
  • Posts: 1,879
  • Joined: 24-June 11
  • Gender:Male
  • Country:

Posted 06 July 2020 - 09:52 PM

This sounds like a really good initiative.

Can I suggest that if this is potentially going to be the first step for an OR only simulator, rather then a tweak of MSTS stuff, that a full "literature survey" be undertaken (perhaps with the help of some community members) to allow any appropriate needs that have been suggested over the years to be considered, for example like this one.

This will help develop a list of all the identified needs and desires? Then the question becomes what items are relevant, and finally a specification can be developed as a result of this?

How would these changes be captured in any OR consist editing tools?

#10 User is offline   Genma Saotome 

  • Owner Emeritus and Admin
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • Group: ET Admin
  • Posts: 15,308
  • Joined: 11-January 04
  • Gender:Male
  • Location:United States
  • Simulator:Open Rails
  • Country:

Posted 06 July 2020 - 10:40 PM

I'll start with what I see as the real objects in railroading: Train, Blocks, Cars, for which we need files. This defines the classes. Instances of these classes are part of the Activity process which, am ong other things, assigns values to certain attribytes of the individual instances mentioned above. I have no opinion on where the act of assigning these values should occur; I'm quite sure programmers will have a much better idea of the answer than I.


Starting with Block: In North American practice the concept of Blocks is perhaps 70 years old with most roads adopting it maybe 50 years ago. The basic idea is this: when a train leaves a yard you don't have an apparently random sequence of cars in your train; you have one to n blocks of cars with each block having a specific destination, usually somewhere well across the country but a block could also be the just-before-midnight yard transfer used to avoid per diem payments.. All cars are part of a block, all trains are collections of blocks. Conceptually cars that are handled in-route are, effectively, in an in-route block, or if you will, a loose consist block.

A simple example is a thru train leaving Oakland on the Western Pacific. It'll have at least two blocks, one for the D&GRW in Salt Lake City and the other for the UP, also Salt Lake city. There may be another block that is left at Keddie for the journey up to the GN in Oregon.

It has long been my opinion that the .con file that KUJU gave us as consists could be morphed into this block concept if there was a replacement for consist in other areas -- IMO we need a new object of Train.

The Block file has an ordered list of cars, similar to what we see today in the .con file and would need to gain attributes of DestUltiamte() and DestNextYard(). These are largely intended for end user viewing. If for programatic management things like an ID() and /or name() are necessary then that's up to the programmer to choose.
++++++++++
Train then takes on the attributes you would expect for a real train as well as an ordered list of blocks. All data present for the train file is for an instance of an object independant of any activity in which it appears, which is to say independant of a real train until it is called by the Yardmaster and sent out into the world. THAT data belongs in the activity object. That allows the same Train file to be used in multiuple places obtaining any activity based attribyte values from the activity object.

The last issue is more problematic; Individual cars (as seen in the ordered list in each block), if they are not empty then they have a lading, lading weight, consignee, and lading name. All of the values for those could be assigned in the block file, or in some cases in the individual .wags but it would be far more useful for those values to be assigned by the activity. In that way any car is, essentially an empty car with (MassEmpty()), gaining attributes and values only in relation to other objects. In this way the end user would need only one .wag file per skinned model. Everything else that varies -- lading, weight, consignee get picked up as part of the activity in which they are used.

In the end we'd see this:
* Cars whose physical models include a visible lading would need attributes and default values for LadnimeName() and LadingMass(); others could have the same -- defaults -- but in the ideal these attribute values would be assigned by the activity.
* Blocks are an ordered list of cars usually going from point A to B, plus any blocks of cars to handle in-route.
* Trains are, well, trains, made up of one or more locomotives, 1 or more blocks.
* Activities define the actions of the train, including car Consignee(), etc. that are relevant to that specific activity. In-game the end user can toggle train information, block information, and specific car information as desired. New methods are needed for the program's internal objects for car, block, and train to pick up the data contained in the activity file.

And of course it would probably require a real Activity Editor to collect the data to pass into those objects.

Oh, one last thing, please get rid of loose cars in the Activity file; each individual listing should be replaced by a block specification.

  • 24 Pages +
  • 1
  • 2
  • 3
  • Last »
  • You cannot start a new topic
  • You cannot reply to this topic

1 User(s) are reading this topic
0 members, 1 guests, 0 anonymous users