L3DT users' community
Large 3D terrain generator

Happy customer & a few questions

Any and all chit-chat regarding L3DT.

Postby Q-dad » Mon May 14, 2007 10:36 pm

I can now make the following with L3DT (click on the links to see detailed descriptions)...:

- The Color Map (1024x1024, 24-bit/16 mill colors, 72 ppi, .JPG)
- The Elevation Map (1024x1024, 8-bit/256 gray scale, 72 ppi, PCX)

Next step is to be able to make the following with L3DT (click on the links to see detailed descriptions)...:

- The Texture Map (64x16384, 24-bit/16 mill colors, 200* ppi, .TGA w/alpha)
- The Texture Elevation Map (64x16384, 8-bit/256 grey scale, 200* ppi, .PCX)
- The Detail Map (1024x1024, 8-bit/256 colors, 200* ppi, .PCX file)
- The Character Data Map (256 lines, ascii, .CAL)

*) These are sometimes 72 ppi instead of 200 ppi, don't know why, or if it matters

aaron wrote:I'll look at adding another plugin for handling the complete compositing of the maps and writing the text files (this will require some improvements to the plugin API too, which I am more than happy to make).

I see now that you wrote that just after I had described the Color Map and the Elevation Map, and after I had sent you some sample DF2 terrain files, but before I had started describing the rest of the terrain files. After I had described the rest of the terrain files, you wrote...:
aaron wrote:Making a plugin to compile everything together is almost possible at the moment, but I need to add a few more features to the plugin API.

and...:
aaron wrote:Also, to compile the texture map and the texture elevation map I'll need to add some more functions to the plugin API for handling climates. Nothing too serious though.

and...:
aaron wrote:the DF2 detail map and texture elevation maps will come after a bit more work is done.

Now, I didn't put those quotes there to make you feel guilty of not following up or something, lol, but I'd like to know if you still think it would be possible to include features into L3DT that would make it possible to create those last 4 types of terrain files. I don't feel like I've given you enough information yet, so I'll write a little essay tomorrow about how I'm guessing Novalogic made those terrain files. If you could tell me something about how you believe L3DT can handle this, we could compare notes afterwards... :wink:
TXP (Terrain Xpansion Pack) for DF2: txp.df2.org

Image
User avatar
Q-dad
Luminary
 
Posts: 97
Joined: Tue Nov 22, 2005 11:54 pm
Location: Jessheim, Norway

Postby Aaron » Tue May 15, 2007 12:39 am

Hi Q-dad,

From a quick re-perusal of those posts, I think I've done everything required to allow a plugin to implement these features, but I'll have to check whether RGBA is supported in the TGA plugin, and whether there is a convenient or easy way to get the required texture data (there is a way, but it may not be convenient).

I vaguely recall thinking that I'd have to add displacement maps to the climates in L3DT to make the texture elevation map. I don't think that's required now, as they can be handled in the plugin.

Hmmm...more thoughts: rather than making a new plugin, I think this could all be implemented in a suitably advanced python script, running through Carl's python plugin. More thinkering required though.

Cheers,
Aaron.
User avatar
Aaron
Site Admin
 
Posts: 3696
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia

Postby Aaron » Wed May 16, 2007 12:50 pm

Hello Q-dad,

Just a quick follow-up:

I wrote:I'll have to check whether RGBA is supported in the TGA plugin...


It wasn't, but it is in the next release.

I wrote:...and whether there is a convenient or easy way to get the required texture data


Yes, it appears I did this already.

So, it appears that L3DT is ready. As for the scripting, do you know python by any chance?

Cheers,
Aaron.
User avatar
Aaron
Site Admin
 
Posts: 3696
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia

Postby Q-dad » Fri May 18, 2007 12:02 pm

Hi Aaron,

This sounds promising! :) However, I still don't fully understand how you're planning to implement the necessary features in L3DT (or in plugins or scripts) in order to produce those remaining DF2 terrain files... I'll add some more info below in a new post tonight (had started writing it 2 days ago at work, but forgot to email it home, and we had our national celebration day yesterday)...

I haven't used python before, but I believe I have got access to some tutorials and ebooks, and if there are already similar scripts made for L3DT, those could be of help as well...
TXP (Terrain Xpansion Pack) for DF2: txp.df2.org

Image
User avatar
Q-dad
Luminary
 
Posts: 97
Joined: Tue Nov 22, 2005 11:54 pm
Location: Jessheim, Norway

Postby Q-dad » Sat May 19, 2007 11:46 pm

Below are some more thoughts about the process I'm guessing Novalogic used in order to create these DF2 terrain files, as well as which connections there are between them... I'll be using the DFD2 terrain files as examples below. In order to make them show up inside these forums, I have converted all of them to .JPG, and I've also resized them if needed.

Color Map and Elevation Map:
DF2 was released in 1999, and I believe I've read somewhere that they used an early version of Bryce to make the elevation maps. The color maps can also have been made with Bryce, but they might also have used other graphics tools or even painted them it manually.

DF2 includes 18 different terrains, and they consist of a limited set of main terrain types when it comes to colors, i.e. brown (desert), green (meadows), and white (snow). Some of them also contains blue (water) and grey (mountain). The 18 color maps can be seen here...: http://txp.df2.org/gallery/thumbnails.php?album=5

The main things to remember about the color map and the elevation map, is that both are 1024x1024 and that they tile seamlessly in all directions. I have earlier said that 1 pixel looks to represent ca 0.5 meters inside the game, but for purposes becoming obvious below, we should instead make that 0.64 meters, or 64 cm.

The color map is the skin of the elevation map, but the detailed look of it is only visible in-game when viewing the commander's map, or outside the game, when making maps for DF2 with the included map editor.

The elevation map contains 1024x1024=1048576 height values to render for the voxel terrain engine. The DF2 voxel terrain engine doesn't simply render the flat top of all the columns for those height values, though, but it also uses some sort of algorithm to level out the sharp vertical borders between columns of different heights. Thus, inside the game, the player will walk on a surface made out of the column heights (from elevation map) + delta heights (created by leveling algorithm).

There are no very high mountains in DF2, due to the limited resolution of the elevation map. Most of the terrains consist of medium sized hills and valleys, or plains and river beds, lakes, etc. There are a few cliffs here and there, which the players can't climb, but they're never very steep, since steep hills are poorly rendered with a voxel terrain engine.

The DFD2 Color Map...:
Image

The DFD2 Elevation Map...:
Image

Texture Map and Texture Elevation Map:
I'm not exactly sure how Novalogic created the texture maps and the texture elevation maps. I'll present a theory further down, but first I'll describe these maps a bit more detailed.

Each of them are 64x16384, and they consist of 256 64x64 tiles. The first tile is always black, and at the end there are also a number of black tiles, or which are used for special non-terrain purposes. The number of those tiles at the end seems to be different for each terrain.

The purpose of the texture map and the texture elevation map is to add more detail to the terrain as viewed in-game by the players. The texture map includes detailed texture tiles, and the texture elevation map includes the detailed elevation information for those detailed texture tiles.

Each 64x64 tile of the texture map is blended together with the color of one of the pixels of the color map. I believe the alpha channel of the texture map decides the level of blending.

Each 64x64 block of the texture elevation map adds additional height to the resulting texture from blending one 64x64 texture map tile with one pixel of the color map. This additional height isn't solid, i.e. the player will be able to walk through it, be it grass, small rocks, soil, dirt, etc... Because of this, the texture elevation maps usually don't contain large height values.

I have studied the content of the texture map and the texture elevation map a bit more, and below is shown the 7 first tiles (0 - 6) of them. The 4 columns are (from the left)...:

A) Texture Map (DFD2_CM.TGA)
B) Alpha Channel of Texture Map
C) Texture Elevation Map (DFD2_DM.PCX)
D) Texture Elevation Map (with values adjusted so that one can see that it's not all black)

.......A...............B...............C..............D.......
Image Image Image Image

It's no coincident that I've showed only the first seven 64x64 tiles here, since they (except for the first, black one) represent the six main texture patterns (or climate types in L3DT...?) for the DFD2 terrain. Most of the remaining tiles below are tiles which contains different variants of transitions between those six main texture patterns. I'll get to those below... I have checked with some of the other DF2 terrains, and the number of main texture patterns is different from terrain to terrain...

Now, if we look closer on tile 3 of the texture map, we can see that it tiles seamlessly in all directions...:

Image

The same thing is valid for tile 3 of the texture elevation map as well (doesn't look 100%, but this could be due to the adjustments I did to it in order to see any content) ...:

Image

It even looks to be valid for tile 3 of the alpha channel of the texture map...:

Image

So, what comes below these 6 main texture patterns...? Well, before the transition tiles, there are some more tiles for the main texture patterns (but not necessarily for all of them). There are some 3 - 7 tiles from each texture pattern, tiling seamlessly in the vertical direction. For DFD2, the texture patterns of tile 1, 2, 4, 5 and 6 (but not 3) are represented...:

Image Image Image Image Image

As far as I can see, the texture tiles 1 - 6 are not used inside any of those vertical texture strips... Now, will these texture strips tile seamlessly in both horizontal & vertical directions...?

Image Image

Yes, they will... (only showing the results for the first of them, but I've checked for the others as well).

Just to satisfy my curiosity, I tested to see if perhaps tiles of the same texture pattern would tile seamlessly no matter which way you put them together, and it seems that they do (at least regarding colors - haven't tested with height value tiles in the texture elevation map yet, but it would surprise me if the same wasn't true for those)...:

Image

This would mean that for the DFD2 terrain we have 6 different main texture patterns, and for each of those there are between 1 and 7 different tiles which can be placed next to each other and tile seamlessly no matter which way you put them together. This would make it possible to make a large area of the same main texture pattern seem more natural, both with regard to the variation of colors, as well as variation of height.

The only thing missing now, is the transition tiles, i.e. tiles for adding a transition in both color and height, between those 6 main texture patterns, and that's exactly what the next 180 or so tiles are...

I've tried to discover some sort of pattern to the order they've been added to the texture map, but there doesn't seem to be any, except that transition tiles between each pair of texture patterns usually are grouped together...

From what I can see, it looks like only the single tiles at the top of the file (i.e. tiles 1 - 6) have been used to create the transition tiles. Most of them include transitions between two of the main texture patterns, but a few of them include transitions between three of the main texture patterns. Here's an example of the latter, a transition between the main texture pattern tiles 2, 3 and 5...:

Image

After the transition tiles, there are usually around 40 more tiles at the end of the file, reserved for special purposes. For the DFD2 terrain, this includes some railroad track and concrete tiles...:

Image Image

However, these tiles are not used for the DFD2 terrain at all, so they must be left-overs from another terrain. Also, the tiles 178 - 143 are black, i.e. not used, so if needed, we could have had 77 more texture tiles inside the texture map for DFD2. Anyway, here's a screen shot from another terrain, to show how the rail road track appears inside the game...:

Image

Character Data Map:
If looking at the character data map (http://txp.df2.org/q/L3DT/DFD2_CM.TXT), we see that the null lines correspond with the black tiles in the texture map, i.e. the ones not used, in that line 1 and lines 179 - 144 are null lines. If we look at the lines at the beginning of the file, we see that 3 different descriptions are used for the 6 main texture patterns, and this will thus describe 3 different footstep sounds when walking on the different parts of the terrain, and also what type of animation (if any) should be shown when shooting at the ground...:

Code: Select all
null,0
Dt2,0
Rk2,0
Sd3,0
Rk2,0
Dt2,0
Sd3,0


I guess that they've only got a limited number of different footstep sounds included for the game, and that they decided already when creating the main texture patterns which footstep sound should be used for each of them. This could easily be changed later on by editing the character data map.

When it comes to the lines further down, corresponding to the transition tiles, I can't see any pattern to which description is used if the texture tiles are represented by different terrain descriptions in the lines at the beginning of the file, except that one of those descriptions is used.

Detail Map:
The detail map is simply an index over which texture (and texture elevation) maps are to be used at which pixel coordinate of the color map inside the game. ...:

The DFD2 Detail Map (and its palette)...:
Image Image

I believe this file must have been generated automatically, since it would be an enormous task to create it manually. I haven't studied the palette (index) thoroughly yet, but one can see that only the first and last ones have been given colors, the rest showing as blank/white. Among the first ones are the primary texture pattern tiles, and among the last ones are the special purpose tiles... This means for instance that it would be relatively easy to add a railroad track to the map afterwards.

Now, if we pick 4 pixels from each of the corners of the detail map, and add the palette numbers to them, we get...:
Image Image
Image Image

Then, if we put them together the way they would be aligned when the terrain tiles seamlessly in all directions, we get:
Image

If we align the corresponding texture map tiles together in the same way, we'll get:
Image

Hmm, when showing the corresponding color map pixels, we get:
Image

That doesn't seem correct... I'll have to ponder on this for a while...



Theories of how Novalogic did it:
Now, back to my theory about how the texture map and texture elevation map were created by Novalogic... Remember that in order to cover all of the 1024x1024 pixels in the color map, you'll need 1048576 texture map tiles, but since you've only got around 220 tiles to pick from inside the texture map, they'll have to be used many times each, i.e. around 4770 times each on average... :shock: Since the number of main texture patterns vary between each DF2 terrain, the number of transition tiles between two (or more) texture patterns might also vary. The latter might also vary depending on the layout of the map, i.e. for instance which angles there are between the different texture patterns, as well as how complex the paths of the borderlines between them are.

One theory is that they made a special utility for calculating all of this. The algorithm of this utility would...:

- Analyze the color patterns and palette of the color map, and define how many different (and which) main texture patterns (and variants of each of them) should be used for the terrain. Alternatively, this information could already be defined manually and used as an input to the utility. Anyway, Novalogic probably had pre-made a large set of main texture pattern tiles, and variants of them.

- Calculate the minimum number of transition tiles needed (but within the limit of 220 or thereabouts) in order to be able to render the transitions between the defined main texture patterns as detailed as possible. This calculation would also need to include the way (angle, direction, size) two (or more) main texture patterns should be blended together. The more tiles available, the more detailed the rendering could become, for instance for complex borders between two (or more) main texture patterns. Thus, if there are no special texture tiles planned for this terrain, the last 40 or so tiles could be used as well, bringing the number of available tiles up to 255.

- Produce the transition tiles by blending the first variants of texture pattern tiles together according to the calculations above. Alternatively, these transition tiles could also have been pre-made, but I don't believe so, since the number of possible variants of transition tiles would be very large.

- Create the detail map and the character data map, based on the above calculations and definitions.

(This theory doesn't explain the process of creating the accompanying texture elevation map, though, and I'm a bit clueless about this... If you put all of texture map tiles together for the complete 1024x1024 terrain and pretend the terrain is flat (since the elevation map doesn't matter), then the texture elevation map tiles would form a mini elevation map for this detailed terrain surface, defining the height differences for the crust of the terrain, like dirt, mud, pebbles, etc., but also for grass, which in general is much higher. I believe they must have created each of the texture elevation map tiles at the same time as they created each of the texture map tiles, and I can't see that they can have used a completely automatic utility for this, but that they must have had some sort of manual control, for instance in order to be able to decide the placement and height of grass and pebbles, etc... I believe they must have had some sort of visual GUI available for this purpose, but I'm not sure. They could have had a special utility for adding grass later on, since the height values for the grass pixels are quite extreme compared to the rest of the texture elevation map content.)

An alternative theory is that instead of creating the color map on the basis of the elevation map (like you can do in Bryce, L3DT etc.) and then creating the texture map from there, they could instead have created the texture map directly on the basis of the elevation map. The color map could then have been created by putting all the 1024x1024=1048576 texture map based tiles together, according to the detail map, and finally resized/saved to a 1024x1024 JPG file.
Last edited by Q-dad on Mon May 28, 2007 2:35 pm, edited 1 time in total.
TXP (Terrain Xpansion Pack) for DF2: txp.df2.org

Image
User avatar
Q-dad
Luminary
 
Posts: 97
Joined: Tue Nov 22, 2005 11:54 pm
Location: Jessheim, Norway

Postby Q-dad » Mon May 28, 2007 2:32 pm

Hi, Aaron

In my latest long post above, I have tried to display all that I have figured out so far about the DF2 terrain files. This is new stuff to me, so you might already have figured out most of it yourself. There are still things that I'm not 100% sure about, for instance...:

- How exactly they produced the texture maps, what kind of utility/GUI they used to have control over calculating and adding enough main texture pattern & transition tiles, and the same thing for adding the corresponding texture elevation maps, including the adding of grass
- Why the color map pixels don't seem to fit the detail map (see example above)

Anyway, what I would like to know, is if you still believe L3DT (as is, or with the use of plug-ins/scripts) would be able to create the remaining of those DF2 terrain files (I know how to create the elevation map and the color map already)...? If so, which of them would be possible to create with L3DT as is, and which of them would require plug-ins or scripts...?

Would L3DT be able to support the creation of several main texture patterns, with variant tiles for each of them, and where only the first one of each of them, placed at the beginning of the file, would be used for creating the transition tiles...? (I haven't yet studied the documentation for the design map or the climates, though, so the answer might be there somewhere...)

Would it be possible to have some control over the process, either at creation time or afterwards, including some visual editing interface, or would it all be automatically created with no easy way of editing the result later on.

Would it be possible to load pre-made main texture patterns and their variants (for instance the original ones from the 18 DF2 texture maps) and use them instead of having L3DT create new ones...?

Again, there's no rush to solving these issues... I'll be around for a while... :wink:
TXP (Terrain Xpansion Pack) for DF2: txp.df2.org

Image
User avatar
Q-dad
Luminary
 
Posts: 97
Joined: Tue Nov 22, 2005 11:54 pm
Location: Jessheim, Norway

Postby DeathTwister » Mon May 28, 2007 11:16 pm

Wow is this a huge post and I was wondering if we could make this a resource/tutorial on the tut page? Q-Dad has done so much work I would love to see this so we can go back to it here and there, WOW very well done Q.

I am still reading and digesting all this, and am just now geting into all these issues and thoughts, and this is a heads up it looks like with raods and train tracks KEWL /smiles.

Ahh, I see the post:
To use this, unzip the file and copy the 'L3DTio_FI.zeo' file to the following directory:

C:\Program files\Bundysoft\[L3DT version you're using]\Extensions

Then, you need to open the extension manager in L3DT, and remove the 'L3DTio_FI' plugin, then load it again. This process is required to nuke some of the file formation options associated with the plugin, which are reset when it's reloaded.

Okay, now when you're exporting the texture, you'll notice that when you select the jpeg/bmp/etc image format and click on the 'options' button, there is an entry called 'PixelsPerMetre'. To set this value, double-click on it, and enter the desired resolution.
OK good to klnow and I am on it, can we move the info though to a resource page? he has done such hard work and would love to see this as a resource/tutorial for reeference, and I need to re-read all this again as some of it went way over my little head. Great going guys.
DT
User avatar
DeathTwister
Dearly missed
 
Posts: 562
Joined: Thu Dec 15, 2005 12:30 pm
Location: Klamath, CA.

Postby Q-dad » Tue May 29, 2007 8:28 am

Thanks, DeathTwister

Not sure if any of this would be useful for other people than those who try using L3DT to create new terrains for Delta Force 2, though, unless Aaron should decide that some of the methods involved here could be of interest and added to L3DT for general purposes. The methods used for creating terrains for an 8 year old game are probably somewhat outdated, though, and in this case, very specific for this game only.
TXP (Terrain Xpansion Pack) for DF2: txp.df2.org

Image
User avatar
Q-dad
Luminary
 
Posts: 97
Joined: Tue Nov 22, 2005 11:54 pm
Location: Jessheim, Norway

Postby DeathTwister » Tue May 29, 2007 12:53 pm

Ah, well it looked like with a bit of a tweak that it could work for all of us, I liked your thinking on it. I like the way the tracks got layed out and while I have not gotten yet into the road feature Aaron just implimented, I was thinking this coul;d be a good resource since you put so much time and care into all this layed to explain what you are doing. Very fine work by the way.

Maybe talk you into doing one like this for Aaron and a tut for us with l3dt? /smioes...well done in any case.

DT
User avatar
DeathTwister
Dearly missed
 
Posts: 562
Joined: Thu Dec 15, 2005 12:30 pm
Location: Klamath, CA.

Postby Tomkat » Tue Feb 05, 2008 3:19 am

I can't believe I just read this entire thread! :D

Has any progress been made on this?
Tomkat
New member
 
Posts: 4
Joined: Tue Feb 05, 2008 3:08 am

Postby Aaron » Tue Feb 05, 2008 2:01 pm

Hi Tomkat,

Tomkat wrote:Has any progress been made on this?


Not as far as I know. From Q-dad's most excellent and thorough description, it's apparent that some components of the DF2 maps are extremely specialised, and would require the development of new calculation algorithms and user interface controls just for the DF2 exporter. I think it's possible, on a technical level, to make a plugin to do all of this. However, the real challenge will be to find a developer (or dev team) with the skills, time, interest and commitment to take up the task. I'm prepared to help out with modifications to L3DT that may assist such development (within reason), but I'm not prepared to take the lead; it's too big a project and too far off-topic for me.

Cheerio,
Aaron.
User avatar
Aaron
Site Admin
 
Posts: 3696
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia

Postby Q-dad » Tue Feb 05, 2008 3:00 pm

I'm going to return to this project after my current project is finished and released, i.e. some time after April 11th...
TXP (Terrain Xpansion Pack) for DF2: txp.df2.org

Image
User avatar
Q-dad
Luminary
 
Posts: 97
Joined: Tue Nov 22, 2005 11:54 pm
Location: Jessheim, Norway

Previous

Return to General discussion

Who is online

Users browsing this forum: No registered users and 10 guests