[charland]

Hi,

As I've been working along on my route I've started to notice some kind of memory problem while test running in OR. If I run a test train of 2 RDCs through the new part of the route I'm fine, I can make it from one end that completed to the other end. If I run a 50 car freight, I get so far then OR crashes to desktop, sometime with an error message saying it's out of system memory, sometimes nothing.

As I build further south of Bellows Falls, running the same freight causes OR to crash further south. I'm just finishing up another eight miles south and now the freight crashes OR about eight miles further south then the last area it crashed in. The RDCs still make it through to the north end that's complete and if I run the same freight in MSTS it gets pretty rough loading the Bellows Falls tile, but it does smooth out and makes it to the north end of completed scenery.

I tried adding the -mem:1024 to the target line of the short cut and ran another test but it still crashed in the same location.

Here's the log... I hope the fact that it's 666 KB doesn't mean anything nasty!

 OpenRailsLog.txt (666.49K)
Number of downloads: 193

Paul :-)

[disc]

-mem does nothing, as OR isn't MSTS :)

Didn't you disabled the page file/virtual memory? Was the system memory really full? Else the application memory was full. Did you checked the "Use large address aware binaries..." setting at experimental tab?

[James Ross]

charland, on 09 September 2014 - 04:59 AM, said:

I tried adding the -mem:1024 to the target line of the short cut and ran another test but it still crashed in the same location.

We don't support the MSTS command-line options, but nice try. :) If you're running a 64bit version of Windows (very likely if it is Windows 7 or later) you can select the experimental "Large address aware" option to double the amount of usable space for Open Rails.

Whatever you're running, press Shift-F5 until you see "DEBUG INFORMATION" on the HUD/F5 display - this includes a textual "Memory" line and a graph (bottom right in orange) of Open Rails' memory usage relative to its own limits. Note that this limit is a limit on OR and has no relation to the operating system or your system's amount of RAM.

You can probably save and resume when the memory starts getting risky (around 1.5GB of 2.0GB) to side-step the problem for now.

It's very likely we have some memory leaks in OR, given the size and complexity of the code, but it's tricky to pin some of them down. Is this route freely available anywhere or have you experienced the same problem on another route that we might be able to try?

[charland]

Morning James and disc,

OK, yes for the Windows 7-64.

"Large Address Aware"... is this somewhere in OR or Windows? I'll press F5 and see what I can see. I'll try the save and see if I can resume but it's usually crashing to desktop and it's not like there is much notice before hand. I will try saving it just before the area it's been crashing and see if I can resume it and pass the area though.

The route is what I'm currently working on and can upload a beta if you'd like to see what's happening yourself.

Paul :-)

[disc]

James Ross, on 09 September 2014 - 06:30 AM, said:

You can probably save and resume when the memory starts getting risky (around 1.5GB of 2.0GB) to side-step the problem for now.

Now that's a problem that i want to ask about. Is this limit can be lifted by using newer .net or 64 bit compilation (if that's possible). I notice that when OR memory reaching 900 mbyte then the game starts with light stuttering, at 1500 mbyte it does heavy stuttering and sometimes stops for seconds. Of course save and resume solves the problem(until reaching another object heavy part of the route), as after resume the memory usage is much lower.

charland, on 09 September 2014 - 06:39 AM, said:

"Large Address Aware"... is this somewhere in OR or Windows?

As i wrote before, it's in openrails options on experimental tab.

[James Ross]

disc, on 09 September 2014 - 06:46 AM, said:

Now that's a problem that i want to ask about. Is this limit can be lifted by using newer .net or 64 bit compilation (if that's possible). I notice that when OR memory reaching 900 mbyte then the game starts with light stuttering, at 1500 mbyte it does heavy stuttering and sometimes stops for seconds. Of course save and resume solves the problem(until reaching another object heavy part of the route), as after resume the memory usage is much lower.

XNA is 32bit-only, so OR must stay in 32bit for now - that means we're limited to 2GB or 4GB (using large address aware). New versions of .NET won't help that. However, they could help the stuttering you see (note that I do not see this almost at all) due to improvements in the memory profile and GC. There's no guarantees though, and we can't go up to .NET 4.5 without losing Windows XP support - which is still important, but probably not for much longer.

[copperpen]

I recently upgraded my cpu from a core2duo at 2.9Ghz to a Haswell I5 o/clocked to 4.2Ghz and retained my Nvidia GTX650Ti card with 1Gb of ddr5 memory. This has almost eliminated stutters, in fact reduced them to micro-stutters in most places except the Mid East Plus route at the Kings Cross end. If I limit FPS the stutters are much worse.

It therefore seems that OR in its present state requires some serious horsepower under the bonnet to totally eliminate stuttering.

[James Ross]

copperpen, on 09 September 2014 - 07:47 AM, said:

I recently upgraded my cpu from a core2duo at 2.9Ghz to a Haswell I5 o/clocked to 4.2Ghz and retained my Nvidia GTX650Ti card with 1Gb of ddr5 memory. This has almost eliminated stutters, in fact reduced them to micro-stutters in most places except the Mid East Plus route at the Kings Cross end.

I have an Core i7 at stock 2.80GHz so I think it may depend more on the cores/threading than raw power.

[charland]

Hi James and disc,

OK, I did find the checkbox for Large Adress Aware and that cured the problem, bumping the available RAM to 4 GB. I did notice something interesting using the graphs, so I took some pictures along the way. After loading the heaviest tiles and then traveling through what should have been easy rural scenery, the game still held on to the heavy Bellows Falls tiles for seven miles before deleting them from the memory.

First shot is from opening the activity, just terrain, track, toggled water, and telephone poles... using about 1 GB:

http://i479.photobucket.com/albums/rr160/paul_charland/ORgraphA_zps1f0c0ab7.jpg

Second shot, this is where it was crashing, loads that big tile and then crashed, now using about 2 GB.

http://i479.photobucket.com/albums/rr160/paul_charland/ORgraphB_zpsa3acc4c7.jpg

Third shot, this is about 7 miles north of Bellows Falls, the heaviest tile on the route so far, it has 2250 objects. The train is now on a rural tile at the end of the completed scenery and the usage has dropped from about 2 GB to about 1.5 GB.

http://i479.photobucket.com/albums/rr160/paul_charland/ORgraphC_zps3e4e8c7d.jpg

Forth shot, three miles north of the last shot, that's three miles of just terrain and track, so why hasn't the usage dropped to 1 GB like the activity started with as there is even less to draw with no telephone poles and no water?

http://i479.photobucket.com/albums/rr160/paul_charland/ORgraphD_zps44db7661.jpg

Paul :-)

[disc]

Yes, or needs power for render thread (directx 12/mantle to solve it? :) but OpenGL currently have low overhead support ). But the stuttering isn't connected to the object count, which is causing constant lower fps when too high. Stuttering doesn't happen even with 9000 primitive count, if the memory usage is lower than 900 mbyte, but can happen with 2000 primitive count if the memory usage is above 900 mb.

[disc]

charland, on 09 September 2014 - 08:34 AM, said:

Hi James and disc,

OK, I did find the checkbox for Large Adress Aware and that cured the problem, bumping the available RAM to 4 GB. I did notice something interesting using the graphs, so I took some pictures along the way. After loading the heaviest tiles and then traveling through what should have been easy rural scenery, the game still held on to the heavy Bellows Falls tiles for seven miles before deleting them from the memory.
Paul :-)

2000 object and 8000+ draw calls? that means average 4 draw call per object, which isn't too optimal :) i see nearly the same on every msts routes.

However you should check "Use model instancing" to at experimental, it may lower the drawcall count, and the cpu usage.

[charland]

Hi disc,

Have no idea what Use model instancing means but I'll give it a try!

There is just the one tile with 2250 objects, the rest are 1000 objects or less. Just happened that all of Bellows Falls and North Walpole ended up on one tile.

Paul :-)

[RTP]

Reduce the amount of smoke in your loco.

Is a memory killer.

Regards.

[James Ross]

charland, on 09 September 2014 - 08:34 AM, said:

OK, I did find the checkbox for Large Adress Aware and that cured the problem, bumping the available RAM to 4 GB. I did notice something interesting using the graphs, so I took some pictures along the way. After loading the heaviest tiles and then traveling through what should have been easy rural scenery, the game still held on to the heavy Bellows Falls tiles for seven miles before deleting them from the memory.

Thanks for trying things; the tiles are kept for as long as they're needed for the set view distance and only discarded at the next tile boundary. Each time you cross a boundary, the game checks what new tiles need to be loaded and what old tiles to discard. Your viewing distance is set to 5.3km so it's quite possible to still have them loaded 7 miles (~11km) down the track, depending on how much the track is not a straight line, where the boundaries are exactly, etc..

There are a variety of things we do poorly with respect to memory usage of scenery, such as keeping all LODs loaded (including GPU memory), which would probably help a lot for higher viewing distances.

[charland]

Hi James,

Thanks, I'm just figuring that out as I go and it's making a lot of sense. I took disc' suggestion and checked the 'use model instance' and the memory usage went up from a max of 1976 to 2042 but where it was loading the heaviest tile and causing the earlier crashes that started this thread, it was actually smoother.

RTP... not my locomotives, just using them for testing while building the route!

Paul :-)