[YoRyan]

steamer_ctn, on 06 July 2020 - 09:29 PM, said:

Currently it appears that initaialisation code, and runtime code is distributed across multiple modules. A better OR code structure might be to have discrete combined functionality where perhaps all data is read first, then simulator is initilised, and lastly simulator runs. Similarly a structured function for save and restore might also be good, again rather then distributing this function across various modules.

Keep in mind that streaming will be necessary for some kinds of data. For example, you probably don't want to load all of the scenery for an entire 200-mile route in one shot. Internet resources are another case.

Nevertheless, your point about improving OR's code structure rings very true. When I was writing the cab pillarboxing feature, I think I found at least 7 locations where the current display resolution is stored. We could spend time refactoring things - for example, by making use of the ISerializable interface instead of sticking Save() and Restore() methods on every class on an ad-hoc basis - but this has to be weighed against the risk of breaking something that slipped through testing.

There is a very good document on Open Rails' architecture floating around on this forum, which, unfortunately, I cannot link as it is in the private section.

steamer_ctn, on 06 July 2020 - 09:29 PM, said:

iv) Some guidance for new developers in terms of naming conventions, other coding related principles would be helpful to ensure consistency of code development.

We have this in the form of the CONTRIBUTING.md document, albeit one that perhaps could be better emphasized.

steamer_ctn, on 06 July 2020 - 09:29 PM, said:

I suspect that if we embark upon this, we should really be creating a separate version of software (ie OR only) that is not constrained by the current architecture, and code structures, etc.

Note that it has always been the plan for Open Rails to implement replacement data formats, even while retaining the ability to read legacy MSTS ones. There are specific hooks in the code that allow for multiple possible input sources. So to write off the current codebase completely, even as messy as it can get, is, I think, a little premature.

steamer_ctn, on 06 July 2020 - 09:29 PM, said:

So lets set realistic goals and achievable outcomes rather then just rearranging the "deck chairs". What do we have the resource to realistically achieve?

With respect to my new consists proposal, I believe that the bulk of the work would involve debating the new format, whereas writing the actual loader would be a trivial task. A new editor would, undoubtedly, involve more work, but nothing that a few months of spare time couldn't accomplish. Are you in?

[roeter]

steamer_ctn, on 06 July 2020 - 09:29 PM, said:

ii) Where in OR should data be read (there appear to be a couple of instances where similar data is read in multiple different locations depending upon features selected)? How is (should) it shared such that it is available for use in any feature?

YoRyan, on 07 July 2020 - 01:30 PM, said:

Keep in mind that streaming will be necessary for some kinds of data. For example, you probably don't want to load all of the scenery for an entire 200-mile route in one shot. Internet resources are another case.

A proper example of the problems faced in this aspect is the processing and reading of world files.
When initializing and configuring the full signal database, it is necessary to read through all world files to collect the signal information as defined in these files. This information is required during the "prerun" phase (processing of train movements before player train start time), as this part of the process needs to use the same signal logic as the actual running part.
But it's just not possible at that moment in processing to collect and store all scenery details also held in those world files. So, the world files must be read and processed again during normal running, but then only those files within the display range of the scenery.
So, yes, there is duplication there, but with the present datastructure it can not be avoided in any way.

Such problems can only be fully sorted by setting up completely new datastructures, which take proper account of when the data is needed and split the information over multiple files as required.

Regards,
Rob Roeterdink

[steamer_ctn]

YoRyan, on 07 July 2020 - 01:30 PM, said:

Keep in mind that streaming will be necessary for some kinds of data. For example, you probably don't want to load all of the scenery for an entire 200-mile route in one shot. Internet resources are another case.

Nevertheless, your point about improving OR's code structure rings very true. When I was writing the cab pillarboxing feature, I think I found at least 7 locations where the current display resolution is stored. We could spend time refactoring things - for example, by making use of the ISerializable interface instead of sticking Save() and Restore() methods on every class on an ad-hoc basis - but this has to be weighed against the risk of breaking something that slipped through testing.

Agreed, but if we never try to set a direction it will continue head in the direction that it is currently heading.

So the question is, "is the project at risk by doing nothing"? Will it break of its own accord? If the answer is no, then we don't need to worry about it, and we can continue as we are.

To change direction always takes a huge amount of effort, and it is often easier if it is done incrementally, rather then in one huge effort.

Perhaps we could do this in a number of stages, such as:

i) Stage 1 - Set the desired direction and structures for OR to reach its full potential.

ii) Stage 2 - encourage developers to develop any new code to align with the directions identified in i) above.

iii) Stage 3 - encourage developers to rework older code that they are working on to align with the desired direction

iv) Stage 4 - when the amount of rework reaches a point that the effort is not too daunting, start a program of code refactoring.

YoRyan, on 07 July 2020 - 01:30 PM, said:

There is a very good document on Open Rails' architecture floating around on this forum, which, unfortunately, I cannot link as it is in the private section.

Apart form a basic architectural block diagram, I am unaware of any architectural documents. If you find something then I would be interested.

Certainly it is not available via the link in CONTRIBUTING.md .

YoRyan, on 07 July 2020 - 01:30 PM, said:

So to write off the current codebase completely, even as messy as it can get, is, I think, a little premature.

It would not be a case of writing off the existing code base, but developing an appropriate transition approach.

Ideally if somebody wants MSTS compatibility, then a lot of that is probably already existing. The issue comes when OR features are added over the top, so the struggle is then to keep some form of legacy compatibility. Perhaps a way to do this would be to internally parallel stream the two simulators (the user starts it in either MSTS compatibility mode or OR mode)? MSTS features could remain as they are and OR uses a "new" parallel version that uses any relevant features from the MSTS stream, but also adds its own new features without needing to worry about legacy issues.

YoRyan, on 07 July 2020 - 01:30 PM, said:

With respect to my new consists proposal, I believe that the bulk of the work would involve debating the new format, whereas writing the actual loader would be a trivial task. A new editor would, undoubtedly, involve more work, but nothing that a few months of spare time couldn't accomplish. Are you in?

So does it need to be a "full blown" editor? Given the fact that the files need to be read by OR anyway, we are possibly half way there with an editor(?), so could a simple text type screen be provided to a user which displays the current values, and allows them to overwrite any values as desired, and then save them? This approach may not be as good as a purpose built editor, but it might potentially provide an "acceptable" alternative until somebody has the time and desire to develop a fully featured editor?

[Genma Saotome]

steamer_ctn, on 07 July 2020 - 03:19 PM, said:

Agreed, but if we never try to set a direction it will continue head in the direction that it is currently heading.

So the question is, "is the project at risk by doing nothing"? Will it break of its own accord? If the answer is no, then we don't need to worry about it, and we can continue as we are.\

Speaking as a route builder and content creator, in my eyes the project has been dead in the water from whenever the decision was in year 1 to not build a route editor. FWIW, Wayne told me a year or so ago he came to realize I had been right in advocating that and not doing it was a major mistake. The editors are the means whereby new file structures can be introduced, either as extensions to the old ones or as clean replacements.

The ONLY feature add for the route builder over all of these years is Carlo's car spawner camera (which I objected to so long as there wasn't a route editor that could fully replace KUJU's world files.

Goku came and went (and yes, I made a terrible mistake of losing my temper with him; It looks now that perhaps he was very near the end of either his interest or his ability to continue and while he deserves full marks for many things he accomplished his work is not really part of the OR project, it is not a panacea to fix this projects problems, and as is it remains with lots of room for improvement, esp. in the UI.).

Which is to say "to continue as we are" is to continue to hold the funeral services for several more years while the competition rolls merrily along into the future. The Monogame effort, with ReShade, has helped hide the ashen gray pallor of the ancient, dead looking content we use; moving along to DirectX 10 (or beyond) will help too (perhaps a lot) but it can't really answer the question of "What has anyone done for end users that is NEW since 2008?". A handful of routes and content that could have been done in 2003.

So my advice is yes, do something new with at least one file, so long as new means more than a literal translation of SIMIS to .json. Whatever is is, do it with one eye fixed on what it should be when at some future date so everybody can agree, yes, it is both good and complete. Which probably will mean some sort of editor to facilitate and control data entry and modification.

[YoRyan]

steamer_ctn, on 07 July 2020 - 03:19 PM, said:

Apart form a basic architectural block diagram, I am unaware of any architectural documents. If you find something then I would be interested.

This is a short, 6-page interpretation (not necessarily correct) of the Open Rails architecture written by WaltN and shared by BillC on one of the new team member threads. I don't think it would be a problem if I redistributed it.

Attached File(s)

•  OR_Design_Overview.pdf (146.86K)
  Number of downloads: 291

[YoRyan]

steamer_ctn, on 07 July 2020 - 03:19 PM, said:

The issue comes when OR features are added over the top, so the struggle is then to keep some form of legacy compatibility. Perhaps a way to do this would be to internally parallel stream the two simulators (the user starts it in either MSTS compatibility mode or OR mode)? MSTS features could remain as they are and OR uses a "new" parallel version that uses any relevant features from the MSTS stream, but also adds its own new features without needing to worry about legacy issues.

To an extent, we already have an "OR mode" and an "MSTS mode" with advanced physics settings that are incompatible with the behavior of MSTS - much to the frustration of activity designers. As for native data formats - nobody is suggesting we remove loaders for legacy MSTS content; the principle has always been to have new and old coexist. Just because Open Rails can load DDS textures does not mean we removed the ACE textures loader.

steamer_ctn, on 07 July 2020 - 03:19 PM, said:

So does it need to be a "full blown" editor? Given the fact that the files need to be read by OR anyway, we are possibly half way there with an editor(?), so could a simple text type screen be provided to a user which displays the current values, and allows them to overwrite any values as desired, and then save them? This approach may not be as good as a purpose built editor, but it might potentially provide an "acceptable" alternative until somebody has the time and desire to develop a fully featured editor?

I'm not convinced an "intermediate editor" would be any faster or simpler to create compared to just prototyping a fully capable one. In the meantime, nothing would stop players from creating their JSON consists in Notepad, just as they currently edit their .wag and .eng files.

[perpetualKid]

let's face reality:
there is a pretty small set of active developers only, mostly working in dedicated areas which they are most comfortable with, or feel the biggest need for improvement to their personal benefit (nothing wrong, just the way open source contributions work). They do a great job adding features, providing bug fixes etc, but this is mostly scattered work with limited focus on cross-cutting efforts or architectural groundwork. Given capacity if this group, there are limits how much progress can be done.

What seems missing though is active discussion and/or guidance from a governance entity providing future directions and architecture/design work beyond isolated features. This is not something limited to individual developers, but should be joint efforts across a broader community, and also needs to be actively supported/encouraged by the steering entity.
Most such discussions are grounded either with reference to decisions done in the inception days of OR (such as the assumed MSTS compatibility), or consecutively killed with compatibity concerns to status quo, blocking off more progress. And looking through the forums here, this is the status since about 2014...

It's been brought up few times, maybe it's really worth to split off an forwarded-looking effort, from the backward-compatible, stable state which OR is now.

Such "New OR" could explore new directions, but may break compatiblity or leave older decisions behind. Not only allowing for new user experience (such as long discussed route/activity editors as integral part, based on new data format), but also attracting developers with a revamped experience of more current frameworks/tools/design&development paradigms.
The last official OR release is still based on XNA 3.1, .NET 3.5 to ensure Windows XP compatibility, and there's been no progress towards MG in a year. The various MG-offsprings already aim to enable new features at a faster pace rate, yet trying to be compatible (and were even backported to the offical version also).

If there's a group of enthusiasts not only interested, but actively want to contribute/drive to modernize OR (or whatever it may be called than), let's be open to discuss options. And this is not just about developers!

[NickonWheels]

perpetualKid, on 08 July 2020 - 12:45 AM, said:

What seems missing though is active discussion and/or guidance from a governance entity providing future directions and architecture/design work beyond isolated features. This is not something limited to individual developers, but should be joint efforts across a broader community, and also needs to be actively supported/encouraged by the steering entity.
Most such discussions are grounded either with reference to decisions done in the inception days of OR (such as the assumed MSTS compatibility), or consecutively killed with compatibity concerns to status quo, blocking off more progress. And looking through the forums here, this is the status since about 2014...

It's been brought up few times, maybe it's really worth to split off an forwarded-looking effort, from the backward-compatible, stable state which OR is now.

Such "New OR" could explore new directions, but may break compatiblity or leave older decisions behind. Not only allowing for new user experience (such as long discussed route/activity editors as integral part, based on new data format), but also attracting developers with a revamped experience of more current frameworks/tools/design&development paradigms.
The last official OR release is still based on XNA 3.1, .NET 3.5 to ensure Windows XP compatibility, and there's been no progress towards MG in a year. The various MG-offsprings already aim to enable new features at a faster pace rate, yet trying to be compatible (and were even backported to the offical version also).

Wise words...
The thing about TSRE is that even Goku could only work with ORTS functions already esthablished, though newer formats esp. for route building would make a huge step forward for all those folks who like making their own content, even if that means getting rid of many functions there since MSTS days but this is what ORTS is all about.

[cjakeman]

perpetualKid, on 08 July 2020 - 12:45 AM, said:

The last official OR release is still based on XNA 3.1, .NET 3.5 to ensure Windows XP compatibility, and there's been no progress towards MG in a year. The various MG-offsprings already aim to enable new features at a faster pace rate, yet trying to be compatible (and were even back-ported to the official version also).

As I have mentioned a few times recently, James is actively working on merging the MonoGame version to replace the XNA. The graphics are now done but the merge also has lots of other features loaded on top of the graphics which are being resolved right now, so that the merge can be completed.

it will happen!

[Genma Saotome]

Chris, is there a path to DirectX10 and beyond? If so, what is it?