Elvas Tower: New Texture format and the memory problems - Elvas Tower

Jump to content

  • 6 Pages +
  • 1
  • 2
  • 3
  • 4
  • Last »
  • You cannot start a new topic
  • You cannot reply to this topic

New Texture format and the memory problems Rate Topic: -----

#16 User is offline   Csantucci 

  • Member, Board of Directors
  • Group: Posts: Elite Member
  • Posts: 7,450
  • Joined: 31-December 11
  • Gender:Male
  • Country:

Posted 04 February 2014 - 12:27 AM

View PostGenma Saotome, on 03 February 2014 - 04:21 PM, said:

I'm rather surprised no one has suggested to have the code first look for a .dds file and if not found look for .ace instead.

There are two possibilities: first look for .dds file or first look for .ace file. Until .dds contents does not overwhelm .ace contents (I think some water will flow under the bridge) first checking for .ace file is the better choice. When in the future .dds contents is more widespread a small change can be done in the software so that first a .dds file is chosen.

#17 User is offline   gpz 

  • Superintendant
  • Group: Posts: Elite Member
  • Posts: 1,846
  • Joined: 27-October 12
  • Gender:Male
  • Location:Budapest
  • Simulator:OpenRails
  • Country:

Posted 04 February 2014 - 12:48 AM

Carlo, I think the case might be exactly the opposite. While there wouldn't be dedicated dds-textured models, everyone needed to make his own conversions. And it would be a pain modifying all .s files for containing the correct references for dds textures. In case dds was searched for first, there would be no need to modify .s files. Later, when ORTS-only content will be available, that will already contain the correct reference to the dds texture, and no fallback is needed for ace, since that content would not work in MSTS anyway. So in case an automatic replacement was made, it would be in the opposite direction than you wrote.

#18 User is offline   disc 

  • Foreman Of Engines
  • Group: Private - Open Rails Developer
  • Posts: 818
  • Joined: 07-October 12
  • Gender:Male
  • Simulator:OpenRails
  • Country:

Posted 04 February 2014 - 04:17 AM

I think a lot of people use MSTS, so looking ACE files first, then for DDS file won't work, as in this case ACE files should be deleted to make the game load DDS. The opposite is much better, look for DDS, if not found then look for ACE. .s models not need any change, the path is the same the file extensions can be replaced on the fly.

#19 User is offline   James Ross 

  • Open Rails Developer
  • Group: Posts: Elite Member
  • Posts: 5,514
  • Joined: 30-June 10
  • Gender:Not Telling
  • Simulator:Open Rails
  • Country:

Posted 04 February 2014 - 04:35 AM

All the comments that DDS should be checked first are right. We could start off by adding an experimental option in OR to "Load DDS textures in preference to ACE" so that we don't impact anyone unexpectedly, but can at least try out the ideas of batch conversions for those who want to.

If that goes well, and we can recommend (or easily provide ourselves) a tool to do the batch conversion, then option can be removed (and we'd always load DDS first, ACE second).

#20 User is offline   copperpen 

  • Executive Vice President
  • Group: Posts: Elite Member
  • Posts: 3,192
  • Joined: 08-August 05
  • Gender:Male
  • Simulator:MSTS & OR
  • Country:

Posted 04 February 2014 - 07:59 AM

I see a problem. MSTS needs ace files. OR would like to use DDS. The s file for a model contains the names of the files required for the model. OR also reads this s file.

The nature of the problem is that you cannot have a model looking for ace files where there are only dds, and you cannot name the dds files with the ace extension because they will not work in MSTS. That means you need a second shape file to call the dds files, but that will break activities. Unless OR reads the s file and substitutes the dds for the ace.

#21 User is offline   James Ross 

  • Open Rails Developer
  • Group: Posts: Elite Member
  • Posts: 5,514
  • Joined: 30-June 10
  • Gender:Not Telling
  • Simulator:Open Rails
  • Country:

Posted 04 February 2014 - 08:00 AM

View Postcopperpen, on 04 February 2014 - 07:59 AM, said:

I see a problem. MSTS needs ace files. OR would like to use DDS. The s file for a model contains the names of the files required for the model. OR also reads this s file.

The nature of the problem is that you cannot have a model looking for ace files where there are only dds, and you cannot name the dds files with the ace extension because they will not work in MSTS. That means you need a second shape file to call the dds files, but that will break activities. Unless OR reads the s file and substitutes the dds for the ace.


That's the plan - have OR check for "foo.dds" every time it sees "foo.ace". That's why I am suggesting we have it controlled by a setting to start with, as it isn't clear if this would cause problems (performance or otherwise).

#22 User is offline   Eldorado.Railroad 

  • Superintendant
  • Group: Posts: Elite Member
  • Posts: 1,021
  • Joined: 31-May 10
  • Gender:Male
  • Country:

Posted 04 February 2014 - 08:22 AM

View PostFrom 04 February 2014 - 07:59 AM:

The nature of the problem is that you cannot have a model looking for ace files where there are only dds, and you cannot name the dds files with the ace extension because they will not work in MSTS. That means you need a second shape file to call the dds files, but that will break activities. Unless OR reads the s file and substitutes the dds for the ace.


Not really...

eg (before):
..\Mytrain
foobar.eng
foobar.s
one.ace
two.ace
three.ace

eg (after):
..\Mytrain
foobar.eng
foobar.s
one.ace
two.ace
three.ace
one.dds
two.dds
three.dds

Note: THE ONLY THING THAT CHANGED is the addition of one.dds, two.dds, and three.dds. ALL of the other files remain EXACTLY the same.
Yes, we will have to access just how much of a penalty it is to load and process a .dds file instead of an .ace file. The tradeoff is that there are many tools/formats to create .dds files and the creation from .ace files could be automated in simple batch scripts (you only need to do this conversion ONCE). Nobody will be FORCED to convert anything if they choose not to. The price paid for 100% compatibility with existing MSTS tools, etc is more hard disk space. With 4TB drives on the market, how expensive is that progress?

And, we move OR into the mainstream, simulation graphics wise.

#23 User is offline   Genma Saotome 

  • Owner Emeritus and Admin
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • Group: ET Admin Group
  • Posts: 15,661
  • Joined: 11-January 04
  • Gender:Male
  • Location:United States
  • Simulator:Open Rails
  • Country:

Posted 04 February 2014 - 09:35 AM

Put it into the .sd file. ALL shapes have a .sd file. I'm pretty sure KUJU meant .sd to mean shape description. You want to use .dds? just record the replacement in the .sd file. OR looks there... if it says use def.dds for .abc.ace... no problems.

TextureSubsitution (
	Replace ( oldfilename, newfilename )
	etc.
)


It's so simple and it only comes into effect when somebody adds the parameters with the data.

Stick it on the .sd for cars and locomotives and you have a simple reskin. No need for a new folder, copying all the other files that remain shared; Just drop in the new texture, write a new .sd and .eng. Done. Consider... there are hundreds of re-skins of a PS-1 boxcar model. They could all go into one folder called PS-1 40ft Boxcar instead of the hundreds of different folders there are now. Ditto for all those American Locomotive models... get rids of dozens, if not hundreds of folders and have F-7, SD-40, etc., etc. Or SLI_SD_90's. Whatever. Vastly better organization -- IF someone wants to do it that way.

FWIW the exact same technique can be use to replace ancient LOD settings... 100m becomes 500. No big deal:
MeshFileAlterations (
	TextureSubsitution (
		ReplaceTexture ( oldfilename, newfilename )
		etc.
	)
	LODSubsitution (
		ReplaceLOD ( oldvalue, newvalue )
	)
)

It's a simple way to help people update old content to more modern expectations and setting it up as MeshAlterations() let the OR team put anything that sometime down the road into the same general area... stuff like bias, maybe even shaders. It can't change anything more or less than what's already defined in the mesh but it will allow some changes.

Down the road new features, like time of day to turn on/off night textures go into a different section -- they're not changing the mesh, they're doing something new.


Bottom Line: Using the .sd as I suggest not only address migrating to .dds but also addresses other similar issues about the shape when it is being loaded, and doing it all the same way, which, IMO is better than addressing them all ad-hoc.

#24 User is offline   disc 

  • Foreman Of Engines
  • Group: Private - Open Rails Developer
  • Posts: 818
  • Joined: 07-October 12
  • Gender:Male
  • Simulator:OpenRails
  • Country:

Posted 04 February 2014 - 02:50 PM

I don't understand why are you overcomplicate this texture thing. Trying to load DDS first (no needed for any modification), if no dds found then load ace method will have immeasurably small performance penalty, while the properly compressed files in DDS will surely increase the performance of the game.

As you can see in the other topic: here
I've made a small app to list ACE files with specific format, and this list can be used for batch dds conversion(BGR565 and BGRA5551 -> DXT1 dds, BGRA4444 to DXT5). Now the only thing is needed, a command line application that can convert and compress ACE files to DDS, for the batch script. Is there any application for this task?

#25 User is offline   Genma Saotome 

  • Owner Emeritus and Admin
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • Group: ET Admin Group
  • Posts: 15,661
  • Joined: 11-January 04
  • Gender:Male
  • Location:United States
  • Simulator:Open Rails
  • Country:

Posted 04 February 2014 - 09:51 PM

View Postdisc, on 04 February 2014 - 02:50 PM, said:

I don't understand why are you overcomplicate this texture thing.


I don't know to whom you are asking the question but if perchance it's me the answer is there are many opportunities to do more w/ textures than what KUJU gave us in MSTS. One is to use some other format besides .ace; Another is to make it easy to replace texture file names that are recorded in the mesh file; Another is to reduce the amount of unnecessary duplication of files when someone reskins a car.

You can solve each of those one by one... probably different people at different times, taking a year or two to get them all done, or you can get them all done by enhancing the text file that rides along with every shape -- the .sd file. ANYTHING done in the .sd file to add features is in plain sight, can be clearly structured, and is completely up to the end user to apply. IMO fixing several problems at once is a far better solution than a simple if then else tweek to fix only one, letting the other feature suggestions twist in the wind for several more years beyond the several years they've already been hanging out to dry.

#26 User is offline   gpz 

  • Superintendant
  • Group: Posts: Elite Member
  • Posts: 1,846
  • Joined: 27-October 12
  • Gender:Male
  • Location:Budapest
  • Simulator:OpenRails
  • Country:

Posted 04 February 2014 - 10:59 PM

DDS format support has been committed. It is available in recent versions, that can be downloaded from the usual place.

#27 User is offline   cjakeman 

  • Executive Vice President
  • PipPipPipPipPipPipPipPipPip
  • Group: ET Admin Group
  • Posts: 3,031
  • Joined: 03-May 11
  • Gender:Male
  • Location:Peterborough, UK
  • Simulator:Open Rails
  • Country:

Posted 05 February 2014 - 12:05 AM

Hi Dave,

View PostGenma Saotome, on 04 February 2014 - 09:51 PM, said:

I don't know to whom you are asking the question but if perchance it's me the answer is there are many opportunities to do more w/ textures than what KUJU gave us in MSTS.

I don't know either, but I did want to say that I'm glad you look ahead for issues and put so much effort into nudging OR in a good direction..

#28 User is offline   gpz 

  • Superintendant
  • Group: Posts: Elite Member
  • Posts: 1,846
  • Joined: 27-October 12
  • Gender:Male
  • Location:Budapest
  • Simulator:OpenRails
  • Country:

Posted 05 February 2014 - 03:33 AM

Dave,

I'm just wondering why we should mess with these old .sd files at all. The modifications you propose mean a complete restructuring of the .eng format, which would mean we move away from it completely. Rather than trying to fit everything into the old format, a new tech data format specification could be made.

While we are stuck with current eng files, until our own AE is ready, we have to keep the current eng files anyway. So there is not much point in partly restructuring it. The only benefit of keeping the formal eng file would be that MSTS would not crash while reading it, but it isn't a real benefit, since MSTS could not use the restructured form anyway.

But otherwise you could also make a new complete reference example configuration of a widely available locomotive, how it supposed to look like. Then we could start talking about it in detail, so that when a coder will be assigned to this task, there would be a package, that could be passed to him, saying "Here it is, implement this!" For a coder sometimes takes more time to think about what the code supposed to do, than actually do the coding. Making a reference implementation is not necessarily a "coder" task, and there would be something to "peddle" with. ;)

#29 User is offline   scottb613 

  • Executive Vice President
  • Group: Status: First Class
  • Posts: 3,187
  • Joined: 06-July 09
  • Gender:Male
  • Location:Downeast Maine (soon)
  • Simulator:ORTS
  • Country:

Posted 05 February 2014 - 04:07 AM

Morn'n Folks,

Just curious - what is DDS actually buying us here ? It's a more efficient so we should see better performance over all ? It's a better quality of compression graphic ? Is this leading into supporting bump maps ?

Thanks,
Scott

#30 User is offline   gpz 

  • Superintendant
  • Group: Posts: Elite Member
  • Posts: 1,846
  • Joined: 27-October 12
  • Gender:Male
  • Location:Budapest
  • Simulator:OpenRails
  • Country:

Posted 05 February 2014 - 04:15 AM

All of the above.

  • 6 Pages +
  • 1
  • 2
  • 3
  • 4
  • Last »
  • You cannot start a new topic
  • You cannot reply to this topic

1 User(s) are reading this topic
0 members, 1 guests, 0 anonymous users