Elvas Tower: MGfx...why?...when (nov 2022)..what does a user do now? - Elvas Tower

Jump to content

  • 6 Pages +
  • 1
  • 2
  • 3
  • Last »
  • You cannot start a new topic
  • You cannot reply to this topic

MGfx...why?...when (nov 2022)..what does a user do now? I have looked everywhere, here, gtihub, blueprint...no info!! Rate Topic: -----

#1 User is offline   Eldorado.Railroad 

  • Foreman Of Engines
  • Group: Status: Contributing Member
  • Posts: 977
  • Joined: 31-May 10
  • Gender:Male
  • Country:

Posted 10 March 2023 - 09:22 AM

TLDR;
We went from .fx shaders in Content folder to .mgfx for Open Rails.We are now
back to the .xnb files, that cannot be edited to the users tastes/requirements
or experimental updates. Why? No public consultation on this?


Long version:
The last experimental version : Open Rails U2022.10.23-1821 2022-10-23 19-28-11.zip
and last official release: OpenRails-1.5.1.zip
They have .fx shaders. The user can edit/modify these files. eg: CabShader.fx

Any release after (eg:Open Rails U2023.03.10-1451 2023-03-10 14-58-19.zip)
that has the .fx files replaced with .mgfx, eg: CabShader.mgfx

The .mgfx files CANNOT be edited, they are compiled binary. The user must go
elsewhere to obtain the compiler "mgfxc"...and it requires that ".net 6" be installed
beforehand. I looked at the ".nuget" file that has the compiler code "somewhere" as there is no
clear .exe file in that ".nuget" container. BTW, you can open a ".nuget" if you rename it ".zip"
without any "special" tools.

In the early days of OpenRails .fx files were locked up in .xnb file. These were also
compiled binary, that could not be edited, easily. Those days were rather unfriendly
for open source. Binary blobs are not what we should be aiming for.

I went over to Monogame to find out what benefits if any could be had by locking up
.fx into .mgfx binaries. It seems to be more convenience than usefulness. I cannot
find any clear link either here, github, blueprint that put this into motion for Open Rails.
I do not see any public discussion on whether this was a good idea or not.

On another note: ".net 6" requires Win 7 SP1 ESU as a platform for installation. I have
two Win 11 licenses, but I am in no hurry to install them on any machine. I do not need to
be lectured on EOL O/S packages. Thanks.

At the moment, if a user/developer wishes to change ANY parameters of a .fx file he/she
must download the source code from github and have the mgfxc compiler on hand to add
them back in to the Content directory of their Open Rails installation.

If we are going to insist on using .mgfx then it should be clear where the user
HAS to proceed to, to make any changes to .fx files. It also would be convenient
that the user not be required to install ".net 6" to use the mgfxc compiler. Are
any of the developers going to offer their services for compiling experimental
changes to .fx files??

If you read the blurb on the Monogame site regarding .mgfx files the writer
of the documentation there seems to go on at length about the "inefficiency"
.png files. Yet in our Content directory we are replete with .png files, so
why on earth use the .mgfx file format if the purpose of .mgfx is to increase
"efficiency"? The user can edit/replace .png files easily. This is NOT the case
with .mgfx files! The reading of .fx files is a one off affair, so how much
time are we really gaining at runtime to have .fx locked up in .mgfx?

Since I am the messenger of this of this "news" thank you for another "you are out of luck"
situation. Given that the 64 bit Z fighting code fix/change occurred AFTER the usage
of .mgfx was instigated I suffer a double misgiving. Hint: for years I have been using a
modified SceneryShader.fx that had to be modified on each installation to suit my needs/
experiments. Not to mention testing, a lot of testing.

Steve

#2 User is offline   perpetualKid 

  • Fireman
  • Group: Status: Active Member
  • Posts: 190
  • Joined: 10-June 18
  • Gender:Male
  • Simulator:OR
  • Country:

Posted 10 March 2023 - 11:35 PM

I can not talk about the specific OR implementation and decisions, but you basically hit the dependencies leading to such change in your post already.

The changes were introduced with the update of Monogame to 3.8 version.
With that Monogame version, the effect compiler became a separate tool (mgfxc), only available for .NET 6.0, while other Monogame runtime libraries still run with older .NET framework version such as the ones OR is using (.NET FX 4.7.2). Although slowly, but Monogame is moving forward as well https://docs.monogam...na.html#effects

Not only this brings generally incompatibility with the previous OR approach to compile shaders directly from .fx at runtime (at OR startup), with the different framework versions there is no other/simple way to work around. And right now, OR can't even update to latest Monogame version (3.8.1), because the newest Monogame version is no longer supporting older .NET versions at all.

So if OR eventually would move to .NET 6, it may be possible to restore previous behavior by either calling the external tool at startup, or link the compiler with the OR sources again.
But I doubt the usefulness in such, why would you need to add the overhead of a compiler for the few cases tweaking effects? Sources are still available and you can make changes yourself like you did, maybe it should be documented also.

Quote

At the moment, if a user/developer wishes to change ANY parameters of a .fx file he/she
must download the source code from github and have the mgfxc compiler on hand to add
them back in to the Content directory of their Open Rails installation.

Same for anyone who wants to make changes to other functionality of OR... you don't want to compile the OR sources next time yourself as well and rather take the binaries versions, don't you? Or should OR be distributed as package of all source files only, and a C# compiler, so everyone has to compile themselves, to align your understanding of Open Source? ;)
If you'd assume .NET 6 as a baseline (ie. if OR was on that version), using mgfxc as a separate tool is a simple command line call to use, anyone who's tweaking .fx effects should be able to handle this.

Again its basically one of those discussions to balance legacy vs moving forward. If OR at least wants to keep up with newer environments and dependencies incl. Monogame, it will have to accept other constraints coming with that. Same discussions certainly for .NET framework versions, or the whole OS discussions leaving XP, Vista etc behind...

#3 User is offline   Eldorado.Railroad 

  • Foreman Of Engines
  • Group: Status: Contributing Member
  • Posts: 977
  • Joined: 31-May 10
  • Gender:Male
  • Country:

Posted 11 March 2023 - 08:38 AM

First, thank you taking the time to write this up. I have read it carefully, a few times. I am going to try to be objective here with responses to your quotes. Any objection or offence taken to my replies are purely accidental and unintentional.



Quote

The changes were introduced with the update of Monogame to 3.8 version.
With that Monogame version, the effect compiler became a separate tool (mgfxc), only available for .NET 6.0, while other Monogame runtime libraries still run with older .NET framework version such as the ones OR is using (.NET FX 4.7.2).

This is a FULL STOP moment. Maybe more a Monogame management issue than an Open Rail implementation. This should have been a red flag to strongly discuss and consider the negatives of having a tool that is NOT aligned to a specific set of dependencies. Why would we at Open Rails want to accept/implement a new version of Monogame that breaks things? The fact that ".net 6" is required to compile .fx implies that Open Rails is moving away from ".net 4.7.2" without consultation. From a developers point of view I can see the allure of moving on to the next version. From a users point of view, things that get broken without an alternate path to have the same functionality or BETTER is a non starter.

Quote

Not only this brings generally incompatibility with the previous OR approach to compile shaders directly from .fx at runtime (at OR startup), with the different framework versions there is no other/simple way to work around. And right now, OR can't even update to latest Monogame version (3.8.1), because the newest Monogame version is no longer supporting older .NET versions at all.

You have answered your own question/statement here. It was implied in Monogame 3.8 that compatibility with ".net 4.7.2" was going to be broken. This was given by the requirement to install ".net 6" to run the mgfxc compiler for the shaders. So why not wait until we are all "ready" to break things and move on to Monogame 3.8.x? I have looked at the update documentation (Changelog, https://docs.monogam.../CHANGELOG.html) and I am loathe to find the need to install ".net 6" in that documentation for the release 3.8 (not 3.8.1) specifically.

Quote

So if OR eventually would move to .NET 6, it may be possible to restore previous behavior by either calling the external tool at startup, or link the compiler with the OR sources again.
But I doubt the usefulness in such, why would you need to add the overhead of a compiler for the few cases tweaking effects? Sources are still available and you can make changes yourself like you did, maybe it should be documented also.

It is not eventually, we have NOW moved onto ".net 6", how else are .fx files compiled? Something that developers lose track of is that, that things that can be hacked will be hacked and often need to be hacked. If you look at various youtube channels that show Open Rails, a trained eye will notice that .fx shaders have been hacked! Some of those videos are not using "stock" .fx settings. I know, because I have done the same. The whole purpose of Open Rails is to modify/hack/explore/change etc.
I have yet to run Open Rails on two different machines with different hardware and not notice that white and black levels are not consistent. Other sims allow us to adjust those levels (not just gamma) at runtime, this is not the case with Open Rails. Clearly, what we have with Monogame is not ideal in all circumstances and hence having open .fx files allows the user to overcome those impediments. If I told you that all I was doing was modifying white and black levels in a .fx file I would be telling you a lie. I think it should not be peculiar that people such as myself are constantly experimenting with the internals of Open Rails without the knowledge or "consent" of the Open Rails developers/principles. I do not think that the direction of this project has ever been fully transparent to all parties. Developers least of all are given to "compiling by committee", having worked in such environments, I get it. We are going to step on each other toes over and over again.

Quote

Same for anyone who wants to make changes to other functionality of OR... you don't want to compile the OR sources next time yourself as well and rather take the binaries versions, don't you? Or should OR be distributed as package of all source files only, and a C# compiler, so everyone has to compile themselves, to align your understanding of Open Source? ;)
If you'd assume .NET 6 as a baseline (ie. if OR was on that version), using mgfxc as a separate tool is a simple command line call to use, anyone who's tweaking .fx effects should be able to handle this.

This is a bit of a "straw man" argument. Historically, certain tools were adhered to that required anybody trying to make changes to Open Rails to stick to those tools and the merry-go-round of upgrades dictated by the originator of those tools. A .fx file can be edited by ANY simple text editor, on any platform or O/S. Ditto for .eng/.wag, MSTS shape files. No special tools or versions of ".net" are required. Open Rails implies "Open"...so this is not uniquely "my" point of view. If everything gets "hidden" behind platform specific compilers, you may as well call this project "Closed Rails". Is that view "aligning" with your understanding of Open Source?

Quote

Again its basically one of those discussions to balance legacy vs moving forward. If OR at least wants to keep up with newer environments and dependencies incl. Monogame, it will have to accept other constraints coming with that. Same discussions certainly for .NET framework versions, or the whole OS discussions leaving XP, Vista etc behind...

I am sure that Open Rails will want to keep up with the newer environments. Does this mean that we will also discard MSTS shapefiles etc because the latest Monogame version will not allow its use? Some upgrades need to be looked beyond the "newness". I personally do not want to see Open Rails become another embrace/extend/extinguish project.

#4 User is offline   alteo80 

  • Hostler
  • Group: Status: Active Member
  • Posts: 65
  • Joined: 07-January 18
  • Gender:Male
  • Location:Paris 13 (France)
  • Simulator:open rails ny
  • Country:

Posted 12 March 2023 - 08:04 AM

good evening, I totally approve of the words of Eldorado.Railroad, this change without notice and many others is unacceptable from my point of view.I'm sticking with older versions as long as possible.

#5 User is offline   Weter 

  • Member, Board of Directors
  • PipPipPipPipPipPipPipPipPipPip
  • Group: ET Admin
  • Posts: 6,806
  • Joined: 01-June 20
  • Gender:Not Telling
  • Simulator:ORTS
  • Country:

Posted 12 March 2023 - 09:49 AM

For me too:
That way, everyone should buy brand new PC after every major upgrade of Monogame to occur.
I hope to keep my for 10-15 years.

#6 User is offline   Laci1959 

  • Foreman Of Engines
  • Group: Status: Contributing Member
  • Posts: 910
  • Joined: 01-March 15
  • Gender:Male
  • Simulator:Alföld
  • Country:

Posted 12 March 2023 - 11:14 AM

Hello.

It is already very difficult to persuade the average user to change. Many people already reject version 1.5. This will increase the number of those who stay with version 1.3 or maybe 1.4. It will increase the number of people leaving Open Rails for something else.
I know it's unbelievable, but many people are stuck with version 4334.

Sincerely, Laci1959

#7 User is offline   gpz 

  • Superintendant
  • Group: Status: Elite Member
  • Posts: 1,771
  • Joined: 27-October 12
  • Gender:Male
  • Location:Budapest
  • Simulator:OpenRails
  • Country:

Posted 13 March 2023 - 11:44 AM

I'm dropping my brainstorming with myself here, because this thread turns out to be a good single thread for everyone freaking out.

1. Now that this partial .NET 6 affair happened, (what I have nothing to do with btw), I would note to everyone that my development environments are ready for going full .NET 6. We have been struggling with this upgrade plan for more than a year now. Just do it. Let anyone, who feels like that, rage a little, then the dust will settle.

There are talented programmers that at first seemed like they wanted to join OpenRails, but then thought that it was moving "too slow", forked out, and mostly working on their own version. (I see  at least 3 people around here on this forum. I find this kind of forking kind of ineffective, but since the code is free, everyone is free to do with that what he wants to. And I am also free to think anything about it.) But for me one of the reasons to proceed with the newest versions is: simply fun. For me part of the reasons why I am doing this is experimenting with the interesting programming technologies, exciting language features. Dealing with outdated technologies is: no fun. It is not my work for a living, so I don't want to do things with no fun. And to me looks like most of us still doing the programming think similarly, so sorry for the users want to remain. Anyway, you may stay with any existing OR version sympathetic for you forever, by your choice. I don't really understand the complaints.

2. Actually it would be interesting to work with some recent 3D API like Vulkan on DX12. I always hoped that at one time MonoGame would bring some advanced structures for being able to finally step up from the current DX10 level. Its maintainers have been promising various things for ages, but the reality is that MonoGame is going nowhere, at least I don't see a bright future there. Actually the MonoGame "customers" are mostly using that for 2D, so there is no real push from anyone to advance further. So sooner or later OR has to make its move. Of course, given that there is still interest in new things, and we don't want to be stuck at the current technological level.

Looks like the MS dotnet foundation made its vote for its supported C# 3D library: it took Silk.Net as a member, now it also residing under the dotnet umbrella on GitHub: https://github.com/dotnet/Silk.NET

Silk.Net differs from MonoGame in that it provides only (generated, uptodate and reliable) bindings to the corresponding C++ library, while MonoGame is a more abstracted game engine. But since OR haven't used almost any of the framework features, mostly just the pure bindings anyway, it would not be a huge step going one level down to really pure bindings. But certainly it would be another MonoGame "migration", but this time not "to", but "from". John Carmack could finish with such a migration overnight. A real programmer could do this in a couple of weeks. For someone like me it would probably take a couple of months. I don't know if it would be worth the effort, I am just wondering about this. But even if it would, with Silk it needed to be decided in advance, what exact API the migration is done to. It is not something like migrate the code, and decide at runtime if to go with Vulkan or DX. The bindings are different, so it needed to be decided what to migrate to.

Just some thoughts...

#8 User is offline   scottb613 

  • Vice President
  • Group: Status: First Class
  • Posts: 2,973
  • Joined: 06-July 09
  • Gender:Male
  • Location:Downeast Maine (soon)
  • Simulator:ORTS
  • Country:

Posted 14 March 2023 - 08:12 AM

Hi Folks,

I run Carlo's MG ORTS so I guess that puts me on the bleeding edge.

While not a programmer - I have spent a truly MASSIVE amount of time developing content for ORTS.

While I may not understand all of what we've lost with the migration from "fx" to "mgx" and the associated ramifications - I've always been a strong advocate of always moving forward.

The thing that makes me cringe is when I hear we're forgoing a new feature or enhancement because we would lose compatibility with Windows XP or Windows 7 or Windows 8.

When - MS drops support (End of Life) for an OS - at that point - we shouldn't be tied to constraints of discontinued OS's to prevent us from moving forward.

If I need to upgrade content to support enhancements - that's fine with me.


As for anything controversial or subject to debate - as we see here - the ORTS Management Team should have the final decision. We have a board of sorts - no?

The ORTS Management Team should consider all arguments, but I would suggest prioritizing the wishes of those doing the heavy lifting on programming over those of the general community. If the programmers lose interest - we're dead in the water.

Just my two cents - take it for what it's worth.

Regards,
Scott

#9 User is offline   steved 

  • Superintendant
  • Group: Status: Elite Member
  • Posts: 1,827
  • Joined: 19-December 09
  • Gender:Male
  • Location:South of here
  • Simulator:ORMG
  • Country:

Posted 14 March 2023 - 08:21 AM

As far as I can tell the only thing that affects me is the ability to lighten the background of the F4 and F5 windows.
Since I've started using the web server to show this information in a separate window that need has gone away.
The mitigation of Z fighting in the latest ORMG makes most of it a moot point.

Onward and upward.
Randy



#10 User is offline   Eldorado.Railroad 

  • Foreman Of Engines
  • Group: Status: Contributing Member
  • Posts: 977
  • Joined: 31-May 10
  • Gender:Male
  • Country:

Posted 14 March 2023 - 09:18 AM

Quote

While I may not understand all of what we've lost with the migration from "fx" to "mgx" and the associated ramifications - I've always been a strong advocate of always moving forward.

Go back a few years, and discover .xnb, then you will know. Then spend some time reading about users who have posted fixes/changes to .fx files, which later became incorporated into Open Rails.

Quote

The thing that makes me cringe is when I hear we're forgoing a new feature or enhancement because we would lose compatibility with Windows XP or Windows 7 or Windows 8.

By all means, lets dump Win 10 as well. While were at it, embrace Win 11 exclusively. You require TPM 2.0, and new motherboard/cpu, and Nvidia wants me to spend $2000 on a new video card.

Quote

When - MS drops support for an OS - at that point - we shouldn't be tied to constraints of discontinued OS's to prevent us from moving forward.

Certainly, I guess we should drop Open Rails as well since....MS dropped MSTS how many years ago? Obsolete, right? While we are at it, stop using any of the MSTS stuff that made/makes Open Rails even remotely possible.

Quote

The ORTS Management Team should consider all arguments, but I would suggest prioritizing the wishes of those doing the heavy lifting on programming over those of the general community. If the programmers lose interest - we're dead in the water.

Conversely, when users lose interest you are also VERY "dead in the water". How much time/money has MS spent to hopefully not have each NEW version of Windows not break every single program written for prior versions of Windows? Upgrades should provide substantial benefit, not the other way around. If each new version of Open Rails casts off those who have worked as hard as you have, what cost/benefit analysis is at play here? Is the purpose of each new version of Open Rails to attract more users than it loses? Where is the unbiased statistics that type of thinking actually works?

Thus far, no response from the ORTS Management Team or open public discussion with the principles. Users are an annoyance to be avoided?

Steve

  • 6 Pages +
  • 1
  • 2
  • 3
  • Last »
  • You cannot start a new topic
  • You cannot reply to this topic

1 User(s) are reading this topic
0 members, 1 guests, 0 anonymous users