Elvas Tower: Going beyond the 4 GB of memory - Elvas Tower

Jump to content

  • 22 Pages +
  • « First
  • 19
  • 20
  • 21
  • 22
  • You cannot start a new topic
  • You cannot reply to this topic

Going beyond the 4 GB of memory Rate Topic: -----

#291 User is offline   Csantucci 

  • Member, Board of Directors
  • Group: Posts: Elite Member
  • Posts: 7,443
  • Joined: 31-December 11
  • Gender:Male
  • Country:

Posted 24 August 2018 - 12:09 PM

View PostperpetualKid, on 24 August 2018 - 03:36 AM, said:

As this is not Monogame specific, we may split that into a separate thread?

Yes, you could open a separate thread in the "Maybe it's a bug" subforum.
Re this
http://www.elvastowe...post__p__240204
I'd prefer that, before I commit, you make a test that the fix at least does not harm the official OR XNA version.

#292 User is offline   James Ross 

  • Open Rails Developer
  • Group: Posts: Elite Member
  • Posts: 5,512
  • Joined: 30-June 10
  • Gender:Not Telling
  • Simulator:Open Rails
  • Country:

Posted 27 August 2018 - 03:38 AM

View PostperpetualKid, on 24 August 2018 - 03:36 AM, said:

James, based on that statement I did some more digging. I think the issue comes through AddAutoPrimitive from RenderFrame.cs - actually seems with Dynamic Shadows the materials on shadow map are not grouped but just added in AddShadowPrimitive, whereas other materials are properly grouped in AddPrimitive.
So sorting and grouping the RenderItem collections for shadowmap either while inserting or afterwards should give result how you describe. The alternative approach how I fixed that equally ensures the correct index is used.
Your approach will save the GPU from changing the IndexBuffer (it's only a pointer which buffer to be used since they are already setup before), at the cost of sorting the Materials.

Ah right, the problem is with the shadow map rendering not the main rendering. It should be fixable with a new group for shadow map primitives (using the shared terrain material) so I don't think it's actually a real sorting cost - and although it seems like it's "just" setting a pointer for the index buffer, in practice we've found that many GPU-related things are much more costly than they appear, but I haven't looked at index buffers specifically.

#293 User is offline   HansJansen 

  • Apprentice
  • Group: Posts: Dispatcher
  • Posts: 25
  • Joined: 28-November 17
  • Gender:Male
  • Location:Stiphout (Helmond), The Netherlands
  • Simulator:Open Rails; Zusi3
  • Country:

Posted 03 September 2018 - 12:00 PM

View PostCsantucci, on 05 July 2018 - 07:40 AM, said:

With the end activity evaluation a major feature has been added to OR. Therefore I've decided to generate the corresponding OR MG version. It includes also a patch by dennisat which removes the obstacles that didn't allow to use the TrackViewer within the OR MG pack.
Here the runtime kit
Attachment OR4157_MG.zip
and here the dev kit
Attachment OR_MG_DevKit_x4157.zip


Hello all,

I have been reading this thread during the last few days, and downloaded the above-mentioned package. Just to look how far I could get, I installed it in my Linux system (XUbuntu 18.04) and tried to run OpenRails.exe using Wine. The result is shown in the attachment. Does anybody have an inkling as to why it insists on using libpng.so.12, instead of the current version libpng.so.16?
The programs seems to continue running, but without any visible output. The last two lines appear after I break it with ctrl-C.

Regards,

Hans.

Attached File(s)



#294 User is offline   perpetualKid 

  • Fireman
  • Group: Posts: Active Member
  • Posts: 190
  • Joined: 10-June 18
  • Gender:Male
  • Simulator:OR
  • Country:

Posted 04 September 2018 - 12:47 PM

View PostHansJansen, on 03 September 2018 - 12:00 PM, said:

Hello all,

I have been reading this thread during the last few days, and downloaded the above-mentioned package. Just to look how far I could get, I installed it in my Linux system (XUbuntu 18.04) and tried to run OpenRails.exe using Wine. The result is shown in the attachment. Does anybody have an inkling as to why it insists on using libpng.so.12, instead of the current version libpng.so.16?
The programs seems to continue running, but without any visible output. The last two lines appear after I break it with ctrl-C.

Regards,

Hans.



I've no idea of Wine implementation, but the LockWindowUpdate call in second last line is a call to native Win32 function outside of .NET to suspend or resume drawing a window or control, in that case it's the main window of the Menu.exe, and null (nil) parameter means this unlocks the drawing.
My starting point would therefore be such native Win32 calls which may cause issues, the libpng may be negligible at this point.

#295 User is offline   HansJansen 

  • Apprentice
  • Group: Posts: Dispatcher
  • Posts: 25
  • Joined: 28-November 17
  • Gender:Male
  • Location:Stiphout (Helmond), The Netherlands
  • Simulator:Open Rails; Zusi3
  • Country:

Posted 05 September 2018 - 01:25 AM

Hi PerpetualKid,

Thanks for your answer. So it seems I have to get in at the deep end...
I'm only just beginning to learn C# etc., and never have done any Windows programming (I come from the Unix world, but have been retired some 15 years now. So it's all a bit rusty anyway).
I will have to study things quite a bit more before I can produce anything significant, I am afraid...

Regards,
Hans

#296 User is offline   edwardk 

  • Open Rails Developer
  • Group: Posts: Elite Member
  • Posts: 1,354
  • Joined: 11-December 09
  • Gender:Male
  • Location:Chula Vista, CA
  • Simulator:MSTS
  • Country:

Posted 15 September 2018 - 03:57 PM

This post is for Monogame, but the question I have could still apply to the regular version. On my own, I would modify the width box size to 2000 (The default size is 1500) to create a better snowfall area in case the tracks happen to turn to far in one direction. This would naturally require more use of memory, but I would not have problems. Both the regular version using LAA and monogame. Its been a while since I have done this, but I am now experiencing out of memory issues for both the regular version using LAA and monogame. Monogame seems the most sensitive.

This leads to the question, is OR is using more memory now than lets say a year ago? I know the route being loaded is one variable, but I do not recall experiencing out of memory issues this much before. I do not normally shut down background apps, but even doing that, monogame still has issues and the regular version at times. Is the experimental version that is downloaded compiled in release mode or debug mode?

Edit: Just did a quick experiment compiling in release. This did not help the memory issue and there are over 1000 warnings based on proper naming conventions and object creation.



Edward K.

#297 User is offline   perpetualKid 

  • Fireman
  • Group: Posts: Active Member
  • Posts: 190
  • Joined: 10-June 18
  • Gender:Male
  • Simulator:OR
  • Country:

Posted 15 September 2018 - 11:08 PM

View Postedwardk, on 15 September 2018 - 03:57 PM, said:

This post is for Monogame, but the question I have could still apply to the regular version. On my own, I would modify the width box size to 2000 (The default size is 1500) to create a better snowfall area in case the tracks happen to turn to far in one direction. This would naturally require more use of memory, but I would not have problems. Both the regular version using LAA and monogame. Its been a while since I have done this, but I am now experiencing out of memory issues for both the regular version using LAA and monogame. Monogame seems the most sensitive.

This leads to the question, is OR is using more memory now than lets say a year ago? I know the route being loaded is one variable, but I do not recall experiencing out of memory issues this much before. I do not normally shut down background apps, but even doing that, monogame still has issues and the regular version at times. Is the experimental version that is downloaded compiled in release mode or debug mode?

Edit: Just did a quick experiment compiling in release. This did not help the memory issue and there are over 1000 warnings based on proper naming conventions and object creation.



Edward K.



would be interesting to see if that also happens in the 64bit MG version I've build (mentioned earlier in this thread) which goes beyond 4GB.
How much memory does your setup need?
If you're interested, my code clone is kept (almost) in sync with ORTS repo, but running MG.
I'd be interested seeing your changes, but most likely don't have the same tracks and materials to create the same condition.

#298 User is offline   edwardk 

  • Open Rails Developer
  • Group: Posts: Elite Member
  • Posts: 1,354
  • Joined: 11-December 09
  • Gender:Male
  • Location:Chula Vista, CA
  • Simulator:MSTS
  • Country:

Posted 16 September 2018 - 05:36 AM

View PostperpetualKid, on 15 September 2018 - 11:08 PM, said:

would be interesting to see if that also happens in the 64bit MG version I've build (mentioned earlier in this thread) which goes beyond 4GB.
How much memory does your setup need?
If you're interested, my code clone is kept (almost) in sync with ORTS repo, but running MG.
I'd be interested seeing your changes, but most likely don't have the same tracks and materials to create the same condition.


Will your code compile and run on Windows 7?

#299 User is offline   perpetualKid 

  • Fireman
  • Group: Posts: Active Member
  • Posts: 190
  • Joined: 10-June 18
  • Gender:Male
  • Simulator:OR
  • Country:

Posted 24 September 2018 - 10:26 PM

View Postedwardk, on 16 September 2018 - 05:36 AM, said:

Will your code compile and run on Windows 7?

working with Edward offline on that, we'll report back on findings.


on a side note, Monogame community has released version 3.7 http://community.mon...7-release/10971,
so expect the ORTS MG clones will move forward also.

#300 User is offline   HansJansen 

  • Apprentice
  • Group: Posts: Dispatcher
  • Posts: 25
  • Joined: 28-November 17
  • Gender:Male
  • Location:Stiphout (Helmond), The Netherlands
  • Simulator:Open Rails; Zusi3
  • Country:

Posted 04 October 2018 - 04:39 AM

View PostperpetualKid, on 24 September 2018 - 10:26 PM, said:

...
on a side note, Monogame community has released version 3.7 http://community.mon...7-release/10971,
so expect the ORTS MG clones will move forward also.

Hi perpetualKid,
I have tried to run your latest ORTS-MG with MG3.7 on my Linux system, but without much success, alas.
The binary package starts running with Wine, up to the point of pressing the Start button; then it vanishes without a trace (:(). If I start it with the mono command (I have installed the latest MonoDevelopment, including the MonoGame V3,7), it comes up with a small window stating it needs .NET Framework 3.5 SP1 and XNA Framework 3.1, and shuts off when I hit the OK button.

So I downloaded the source package from GitHub and tried to compile it. Needless to say I got a number of errors and warnings. Concentrating on the (8) Error messages, I found the following solution for three of them:

- In Contrib/ActivityEditor/ActivityEditor/ActivityEditor.csproj, changed line
<Compile Include="ActionProperties\ControlStartProperties.designer.cs">
into
<Compile Include="ActionProperties\ControlStartProperties.Designer.cs">


- In Menu.csproj, changed line
<Compile Include="ImportExportSaveForm.designer.cs">
into
<Compile Include="ImportExportSaveForm.Designer.cs">


– Renamed file Orts.Formats.Msts/ACEFile.cs to Orts.Formats.Msts/AceFile.cs

So three occurrences of the case-sensitive nature of Linux, as opposed to Windows.
Maybe you can put these changes into your sources?

After that, when built individually, the following 5 projects produce this error:
The command “echo $Revision: 000 $>Revision.txt
	date /t>>Revision.txt
	time /t>>Revision.txt exited with code 127. (MSB3073)

– ContentChecker
– Launcher
– Menu
– RunActivity
– TrackViewer

(“Build All” results in the same error for only Launcher, Menu and RunActivity.)

I have no idea how to tackle these errors. Could you point me to possible cause(s)?
It would be nice to at least finish the building of the solution (not to mention running the result!)
I am quite willing to delve further into that problem, should you wish me to do so.

Regards,
P.S. please also check my next message!

#301 User is offline   HansJansen 

  • Apprentice
  • Group: Posts: Dispatcher
  • Posts: 25
  • Joined: 28-November 17
  • Gender:Male
  • Location:Stiphout (Helmond), The Netherlands
  • Simulator:Open Rails; Zusi3
  • Country:

Posted 04 October 2018 - 06:52 AM

@perpetualKid:

For what it's worth, I include the log file and the list of errors and warnings the "Build All" command produced.

(I had to rename the Ide.log file as the system refuses to upload "this kind of file".)

Attached File(s)



#302 User is offline   James Ross 

  • Open Rails Developer
  • Group: Posts: Elite Member
  • Posts: 5,512
  • Joined: 30-June 10
  • Gender:Not Telling
  • Simulator:Open Rails
  • Country:

Posted 04 October 2018 - 11:47 AM

View PostHansJansen, on 04 October 2018 - 04:39 AM, said:

I have tried to run your latest ORTS-MG with MG3.7 on my Linux system, but without much success, alas.
The binary package starts running with Wine, up to the point of pressing the Start button; then it vanishes without a trace (:(). If I start it with the mono command (I have installed the latest MonoDevelopment, including the MonoGame V3,7), it comes up with a small window stating it needs .NET Framework 3.5 SP1 and XNA Framework 3.1, and shuts off when I hit the OK button.

The .NET and XNA framework checks are all OpenRails.exe does before starting Menu.exe, so you can skip that and try running Menu.exe directly.

View PostHansJansen, on 04 October 2018 - 04:39 AM, said:

- In Contrib/ActivityEditor/ActivityEditor/ActivityEditor.csproj, changed line
<Compile Include="ActionProperties\ControlStartProperties.designer.cs">
into
<Compile Include="ActionProperties\ControlStartProperties.Designer.cs">


- In Menu.csproj, changed line
<Compile Include="ImportExportSaveForm.designer.cs">
into
<Compile Include="ImportExportSaveForm.Designer.cs">


– Renamed file Orts.Formats.Msts/ACEFile.cs to Orts.Formats.Msts/AceFile.cs

All fixed in X4248, although I am unsure what'll happen for people on Windows with regards "AceFile.cs". Subversion will probably leave it alone, but I believe the rename is correctly in the repo so should propagate to git okay:

https://james-ross.co.uk/temp/orts_159.png

View PostHansJansen, on 04 October 2018 - 04:39 AM, said:

So three occurrences of the case-sensitive nature of Linux, as opposed to Windows.
Maybe you can put these changes into your sources?

They're in the official sources now, so when the next update of the MonoGame version is made they should be included.

View PostHansJansen, on 04 October 2018 - 04:39 AM, said:

After that, when built individually, the following 5 projects produce this error:
The command “echo $Revision: 000 $>Revision.txt
	date /t>>Revision.txt
	time /t>>Revision.txt exited with code 127. (MSB3073)

– ContentChecker
– Launcher
– Menu
– RunActivity
– TrackViewer

(“Build All” results in the same error for only Launcher, Menu and RunActivity.)

I have no idea how to tackle these errors. Could you point me to possible cause(s)?
It would be nice to at least finish the building of the solution (not to mention running the result!)
I am quite willing to delve further into that problem, should you wish me to do so.

These errors are all caused by code which is included in the project build configuration to ensure the version numbers are reset to 0, to avoid confusion with any official versions.

Inside each project file (.csproj), the code is inside "<PostBuildEvent>". You can just remove it, but be aware that Open Rails may report official version numbers when it shouldn't do. I believe the equivalent code for Linux would be (with the XML escaping needed for the project files):

echo "$Revision: 000 $"&gt;Revision.txt
date +%x&gt;&gt;Revision.txt
date +%R&gt;&gt;Revision.txt


#303 User is offline   HansJansen 

  • Apprentice
  • Group: Posts: Dispatcher
  • Posts: 25
  • Joined: 28-November 17
  • Gender:Male
  • Location:Stiphout (Helmond), The Netherlands
  • Simulator:Open Rails; Zusi3
  • Country:

Posted 04 October 2018 - 12:28 PM

James, thank you very much for your reply. Tomorrow, I will try to change things as you have pointed out, and see where that takes me. Of course, I will report failure or success here!

Regards,

#304 User is offline   dajones 

  • Open Rails Developer
  • Group: Posts: Contributing Member
  • Posts: 433
  • Joined: 27-February 08
  • Gender:Male
  • Location:Durango, CO
  • Country:

Posted 04 October 2018 - 02:22 PM

Assuming that you are able to get the MG version compiled and running on Linux, you will then start running into file name case problems in the content files. There are hundreds or thousands of them and often the names aren't used consistently, so renaming the files won't always fix the problems. I have a private sim that can use MSTS content on Linux and I ended up adding code to automatically fix the case when an open attempt fails. You will probably need to do something similar.

I didn't add code to automatically fix directory names, but there are problems there also. I usually use symbolic links to fix those. For example I have a symbolic link from Routes to ROUTES, so that both work.

Doug

#305 User is offline   HansJansen 

  • Apprentice
  • Group: Posts: Dispatcher
  • Posts: 25
  • Joined: 28-November 17
  • Gender:Male
  • Location:Stiphout (Helmond), The Netherlands
  • Simulator:Open Rails; Zusi3
  • Country:

Posted 05 October 2018 - 06:50 AM

View Postdajones, on 04 October 2018 - 02:22 PM, said:

Assuming that you are able to get the MG version compiled and running on Linux, you will then start running into file name case problems in the content files. There are hundreds or thousands of them and often the names aren't used consistently, so renaming the files won't always fix the problems. I have a private sim that can use MSTS content on Linux and I ended up adding code to automatically fix the case when an open attempt fails. You will probably need to do something similar.

I didn't add code to automatically fix directory names, but there are problems there also. I usually use symbolic links to fix those. For example I have a symbolic link from Routes to ROUTES, so that both work.

Doug


Doug, thank you for these thoughts! I wouldn't be surprised by that kind of problems anyway - they are only to be expected in Windows programs that never have to bother with case problems.
For the moment I am not anywhere near a compilation success, but I will keep this in mind just in case I get lucky somewhere/when...

Thanks, and regards,

  • 22 Pages +
  • « First
  • 19
  • 20
  • 21
  • 22
  • You cannot start a new topic
  • You cannot reply to this topic

1 User(s) are reading this topic
0 members, 1 guests, 0 anonymous users