Elvas Tower: ORTS new shape format??? - Elvas Tower

Jump to content

  • 37 Pages +
  • « First
  • 10
  • 11
  • 12
  • 13
  • 14
  • Last »
  • You cannot start a new topic
  • You cannot reply to this topic

ORTS new shape format??? Rate Topic: ****- 3 Votes

#111 User is offline   ErickC 

  • Foreman Of Engines
  • Group: Status: Contributing Member
  • Posts: 997
  • Joined: 18-July 17
  • Gender:Male
  • Location:Hastings, MN, US
  • Simulator:ORTS
  • Country:

Posted 26 January 2018 - 02:37 PM

No, the process of drawcall batching is combining all non-animated parts that share the same material into a single node, which reduces the drawcall count, which is an important performance factor (though I find that it's secondary to the number of vertices). Let's say I have a model, and this model uses a single material, and has ten parts: a cab, two hoods, a frame, two trucks, and two wheels on each truck. If I leave it as-is and export it, this is ten drawcalls. But the only parts that really need to be separate are the trucks and wheels, because those are animated. So I can join all of the non-animated parts together, and now we are left with seven drawcalls. This assumes a fairly basic material. If you start adding more maps (bump, reflection, et c), that single material now multiplies the drawcall count by the number of additional maps. So let's say I added a specular map to this model, now it's the difference between 20 and 14 drawcalls. So a model that has been optimized into the fewest possible drawcalls is optimal, because things will get out of hand very quickly once you start dealing with modern materials (which might have a reflection map, specular map, normal map, and Fresnel map).

The glory of the MSFS compilers is that they do the drawcall batching automatically, so you can keep your parts separate in the source files, which makes building and editing models easier. With the TS exporters, we have to do this manually, and the only way to go back is to have a separate set of source files for export - which means joining parts over and over again whwne you modify something and create a new export version. This is tedious and time-consuming. If we had a compiler that did this automatically... no problem.

Note that joining parts doesn't necessarily create a single drawcall. Let's say, for example, I used a pair of small textures, one for the hoods, and one for everything else. Well, I join the hoods, cab, and frame together, and the entire assembly is still two drawcalls - again multiplied by the number of maps in the material (so it becomes four if each material has a diffuse map and a specular map). This is why I cringe when I see a model that uses a bunch of tiny images instead of a few or one big image.

Having couplers and trucks and whatnot as separate meshes probably wouldn't affect drawcall counts in that each is an animated part that will always be an additional drawcall anyway. If you're putting handrails and exhaust stacks and other static parts in a separate mesh, though, you're screwing the user out of some degree of performance. This becomes very important when there are a lot of objects on-screen.

As far as I know, a texture atlas really only helps you when everything that uses it can somehow be combined into a single drawcall (such as with shrubbery and buildings, that is, static scenery). If the objects can't, then it can actually be even worse if the platform treats each instance of the same file as a separate file. Let's say you have four boxcars, and you decide to combine the four 2048 x 2048 pixel textures (it's 2018) into a single 4096 x 4096 pixel texture. Well, those four boxcars are always going to be four sets of n drawcalls, so you didn't gain anything performance-wise. Let's say the program treats the same texture as a separate file for each car, though. Now you're still loading four images into memory... but they're significantly larger and taking up much more memory. I think that this is rare, however. I'd hope that OR only loads one instance of each particular file into memory... does it? The answer could very well determine how I build models in the future (combining several models into one texture may not save on drawcalls, but it can mean a more efficient use of space within the textures themselves, especially with flatcars).

"Interchangeable parts" was a poor word choice. I'm referring to building a fleet of similar pieces of rolling stock with parts that might be the same, but might need to be moved from model to model. As an example, I am presently building the Soo Line's entire 7-locomotive GP7 fleet (MNTX 559 belonged to the CRIPs and doesn't count). I can't use the same model for all of them, because each locomotive is different. The final step in the export process is joining together all of the static parts (I am using a single material for the entire locomotive). Thus each final GP7 model should only be 7 drawcalls. Having to do this 7 times is really annoying, especially when LODs enter the equation.

I could build a basic model and add secondary freight shapes - but, again, I'd be adding additional drawcalls. This will become very significant when more advanced shaders and more complex materials enter the equation (that said, I have considered this process as a possibility for "build your own locomotive" packages where I'd provide a generic basic model and one particular railroad's detail parts as a freight shape, allowing the user to create their own detail parts for a particular road and repaint).

#112 User is offline   Hamza97 

  • Engineer
  • Group: Status: Contributing Member
  • Posts: 606
  • Joined: 01-March 15
  • Gender:Male
  • Simulator:Open Rails
  • Country:

Posted 26 January 2018 - 10:45 PM

Quote

decide to combine the four 2048 x 2048 pixel textures (it's 2018) into a single 4096 x 4096 pixel texture.


I am thinking of using the same technique above in my project, 4 2K textures in a single 4K one, solely from performance point of view, should i use it or try something else....?? :mellow: :search:

#113 User is online   wacampbell 

  • Member since Nov. 2003
  • Group: Fan: Traction Nuts
  • Posts: 2,343
  • Joined: 22-November 03
  • Gender:Male
  • Location:British Columbia, Canada
  • Country:

Posted 27 January 2018 - 06:55 AM

View PostErickC, on 26 January 2018 - 02:37 PM, said:

The glory of the MSFS compilers is that they do the drawcall batching automatically, so you can keep your parts separate in the source files. With the TS exporters, we have to do this manually,


Just an FYI, the Blender to MSTS exporter does this drawcall batching automatically on export. It combines meshes everywhere possible to use the minimum number of drawcalls in the .s shape file it creates.

I still support the idea of using compilers as suggested. The concept of compiling geometry to an internal format has many advantages:

- this internal format can be optimized for fast loading

- it can support whatever source formats are convenient for the artist

- drawcall batching can be done according to the capabilities of the player's hardware. For example if we introduce texture arrays into Open Rails, these require a different strategy for drawcall batching than the current graphic algorithm. Adopting the Vulkan graphics API would use different batching and formats again. All of this can be made transparent to the end user by using compilers.

- We don't have to have artists trying to second guess what works best on the hardware. Artists build their content in the way that makes sense to them, and let the compiler figure out what optimizations make sense for the graphics code.

- having a compiler makes batching across a full scene possible. ie the compile operation isn't done model by model, its done tile by tile, merging multiple models that share the same material settings across a tile or section of a tile. And it can choose to use instancing, or mesh consolidation which ever is best for the end users hardware and software.


Wayne

#114 User is offline   James Ross 

  • Open Rails Developer
  • Group: Status: Elite Member
  • Posts: 5,488
  • Joined: 30-June 10
  • Gender:Not Telling
  • Simulator:Open Rails
  • Country:

Posted 29 January 2018 - 01:51 PM

View PostErickC, on 26 January 2018 - 02:37 PM, said:

I'd hope that OR only loads one instance of each particular file into memory... does it?

You are correct; Open Rails will load only one copy of each texture, shape (mesh), tile, etc. into memory no matter who is referring to it. It will even deduplicate materials (textures + render options) across all loaded shapes, so if multiple shapes use the same texture with the same options (e.g. TexBlendA), there will only be one material for it - and the draw calls for the whole scene get batched by material.

View Postwacampbell, on 27 January 2018 - 06:55 AM, said:

- having a compiler makes batching across a full scene possible. ie the compile operation isn't done model by model, its done tile by tile, merging multiple models that share the same material settings across a tile or section of a tile. And it can choose to use instancing, or mesh consolidation which ever is best for the end users hardware and software.

I've toyed with the idea of merging non-animated parts of tiles and auto-generating texture atlases, which I know isn't going to be perfect but ought to work in our favour in many cases.

The only thing about a compiler is where does it run: on the content-creators computer, or the end-user? Maybe it's both. I ask because some of the items listed mention the player's hardware but most commonly the compilation only happens at the content-creator end AIUI.

#115 User is offline   Genma Saotome 

  • Owner Emeritus and Admin
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • Group: ET Admin
  • Posts: 15,308
  • Joined: 11-January 04
  • Gender:Male
  • Location:United States
  • Simulator:Open Rails
  • Country:

Posted 29 January 2018 - 06:22 PM

View PostJames Ross, on 29 January 2018 - 01:51 PM, said:

You are correct; Open Rails will load only one copy of each texture, shape (mesh), tile, etc. into memory no matter who is referring to it.


James, does the above include identical textures loaded from multiple folders in \trainsets?



Quote

It will even deduplicate materials (textures + render options) across all loaded shapes, so if multiple shapes use the same texture with the same options (e.g. TexBlendA), there will only be one material for it - and the draw calls for the whole scene get batched by material.


Ohhh, I did not know this...interesting... it sounds sort of like instancing by texture. What happens if the texture is applied to faces with different LOD's? I'm curious because I do A LOT of texture sharing across many different models (e.g., brick, concrete, windows, door textures are commonly shared in all of my models).

#116 User is offline   James Ross 

  • Open Rails Developer
  • Group: Status: Elite Member
  • Posts: 5,488
  • Joined: 30-June 10
  • Gender:Not Telling
  • Simulator:Open Rails
  • Country:

Posted 30 January 2018 - 12:38 PM

View PostGenma Saotome, on 29 January 2018 - 06:22 PM, said:

James, does the above include identical textures loaded from multiple folders in \trainsets?

No, we make the decision based on the whole path and filename (but I believe it is case-insensitive). To use the contents would mean loading the file in some form and then discarding it if it matches an existing texture, which wouldn't be great for performance.

View PostGenma Saotome, on 29 January 2018 - 06:22 PM, said:

Ohhh, I did not know this...interesting... it sounds sort of like instancing by texture. What happens if the texture is applied to faces with different LOD's? I'm curious because I do A LOT of texture sharing across many different models (e.g., brick, concrete, windows, door textures are commonly shared in all of my models).

LODs have no impact on the rendering optimisations, i.e. they only control which of many possible things get drawn. If the furthest LOD of one shape is visible and the nearest LOD of another, and they both use the same texture file and options, they'll be grouped. To be clear: it's still going to be two draw calls, but the shaders are set up just once.

Your texture strategy sounds like it is good for reducing shader set up, which is a costly thing we don't have a good display of in the HUD. :(

#117 User is offline   Genma Saotome 

  • Owner Emeritus and Admin
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • Group: ET Admin
  • Posts: 15,308
  • Joined: 11-January 04
  • Gender:Male
  • Location:United States
  • Simulator:Open Rails
  • Country:

Posted 30 January 2018 - 03:02 PM

This has been a very interesting discussion so far, thanks everyone!

ErickC: Clearly you model locomotives. I model city blocks and so our relative objectives have some significant differences. Consider what I'm presently working on: one entire city block in downtown Chicago, all w/ some 3d detail instead of painted features:
Attached Image: ft a.jpg

FWIW the tall building is the old Fashion Trades building, 337 W Adams. It's still there tho the front is now completely different. The stone face one to the right is also standing and is now a heritage protected site.

The footprint for this model is 390ft x 320ft with heights varying from 100 ft to 275 ft. As a city block the model presently uses 20 textures and 6000 polys. All of the textures are reused in other models, many of which will be adjacent to this one. Even tho the numbers are large, on a volume basis it is vastly less expensive than a set of locomotives and their train; ignoring volume I'd guess it costs about 4 locos and and 10-20 cars.

Eventually the flat-black you see will be replaced by an 8bit alpha set to high gloss to represent windows (the only variant material type) and I'll move the black behind the walls and merge them into a large dark quad representing the interior. There are no named parts.

I build models like this (w/ 3d details) in part because I can gain a huge amount of visual clarity by keeping the pixel to surface area as small as possible, and in part a lot of my models like this one are actually fairly close to the tracks so I want it to look real, not a blotchy, blurred mess like so many MSTS buildings are.

Having a rendering environment where most of the textures in this model are also used in models nearby would mean I can construct this as one model and then for placement in the route explode it all on a building by building basis. That would let me set LOD values specific to the size of each building rather than the size of the whole city block. IOW like-purposed faces, such as the tops of parapets, could all be assigned the same LOD value (based on a 12in width, rather than figuring out how far each of those polys are from the center of the block and adding that distance to unique LOD value for each face acting as the top of a parapet. That actually is quite a time saver.

M<y point is the needs of a model made w/ animation and those that are all static are not necessarily different in complexity but only on the presence or absence of animation and whatever tools and file formats are in our future need to keep that in mind.

#118 User is offline   SP 0-6-0 

  • Foreman Of Engines
  • Group: Status: Contributing Member
  • Posts: 985
  • Joined: 12-November 05
  • Gender:Not Telling
  • Location:Another planet.
  • Simulator:MSTS/ORTS
  • Country:

Posted 30 January 2018 - 03:53 PM

I still am interested in Dave's idea of expanding the .SD file and making it a way of assembling and rendering in game multiple part locomotives like the Milw electrics. In a way such an idea would eliminate so many problems with these electrics and some steam and diesel locomotives. I believe Trainz does something similar?

Robert

#119 User is offline   Genma Saotome 

  • Owner Emeritus and Admin
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • Group: ET Admin
  • Posts: 15,308
  • Joined: 11-January 04
  • Gender:Male
  • Location:United States
  • Simulator:Open Rails
  • Country:

Posted 30 January 2018 - 05:03 PM

View PostSP 0-6-0, on 30 January 2018 - 03:53 PM, said:

I still am interested in Dave's idea of expanding the .SD file and making it a way of assembling and rendering in game multiple part locomotives like the Milw electrics. In a way such an idea would eliminate so many problems with these electrics and some steam and diesel locomotives. I believe Trainz does something similar?

Robert
AFAIK there is no law that says a game object must be a single mesh file, whther it is placed via the game editor or presented in-game via the loader, esp. since we have an example with .fa parts. I think such assembly could be done in the loader software but perhaps shuffling multiple .s file together like playing cars is trickier than I'm guessing it might be.

#120 User is offline   Hamza97 

  • Engineer
  • Group: Status: Contributing Member
  • Posts: 606
  • Joined: 01-March 15
  • Gender:Male
  • Simulator:Open Rails
  • Country:

Posted 30 January 2018 - 09:40 PM

Very helpful and interesting discussion folks..... :sign_rockon:

  • 37 Pages +
  • « First
  • 10
  • 11
  • 12
  • 13
  • 14
  • Last »
  • You cannot start a new topic
  • You cannot reply to this topic

1 User(s) are reading this topic
0 members, 1 guests, 0 anonymous users