L3DT users' community
Large 3D terrain generator

My data on TGEA atlas map import

Got a problem or need advice?

My data on TGEA atlas map import

Postby Madcowthomas » Fri May 18, 2007 4:10 am

This is a collection of data that I have been collecting since I start trying to learn this TGEA pipeline. Like to say this is information from several people that help me during this process.

Please let me know if something is misspelled or wrong information and I'll correct it asap.

Thanks to everyone that help me and thanks in advanced for any help in the future. This is collection of data and not be regard as set instructions.

Please note at this time the L3DT Torque Exported only works with files 2048 and under. Recommend running the commands through the console in the engine.


TGEA Tutorial on Atlas


PART 1 - Converting the .raw file into a geometry chunk file
Code: Select all
atlasOldGenerateChunkFileFromRaw16(
"[data/terrains/heightmap.raw",
1024,
1.0,
0.0018,
"data/terrains/deleteme.geometry.chu",
1,
2);


Above is the basic file you will run to convert the .raw file to a geometry chunk file. You will need to substitute the values above with your values. Note the tree leaf size of 2, in the example above in Part 1. Which I also believe is also the Tree Depth

Break down of the ATLAS variables


atlasOldGenerateChunkFileFromRaw16(
"YOUR_MOD_FOLDER/data/terrains/heightmap.raw",
HEIGHTMAP_SIZE,
METERS_PER_PIXEL,
HEIGHT_SCALE,
"YOUR_MOD_FOLDER/data/terrains/deleteme.geometry.chu",
HEIGHT_ERROR,
TREE_DEPTH );



The code above in red is what you focusing on changing here. Also want to insure your paths to the files are correct.



Tree depth for Terrains

2 - for a 1024x1024 pixel texture
3 - for a 2048x2048 pixel texture
4 - for a 4096x4096 pixel texture
5 - for a 8192x8192 pixel texture
6 - for a 16384x16384 pixel texture
7 - for a 32768x32768 pixel texture



Information on your .raw files

It's a raw file, so right click on your .RAW file and select 'Properties'.
The 'Size:' should be 2,101,250 bytes which is 1025 x 1025 x 2.

If the size is 1,050,625 then it is 1025 x 1025 (the right size) but only 8 bits.
If the size is 2,097,152 then it is 1024 x 1024 (the wrong size) but 16 bits(right bit size).
If the size is 1,048,576 then it is 1024 x 1024 (the wrong size) and only 8 bits.

TREE DEPTH
is used to balance the number of vertices per LOD mesh in the terrain. It is recommended that you
have between 1000 and 32000 vertices per level, but you will have to play with this to get good results.
Atlas2 will give you some suggestions, but mostly HEIGHT_SCALE and TREE_DEPTH are adjusted through trial and error.




The tree depth for the generated terrain quadtree is another parameter needed during import. This value is chiefly used to balance the number of vertices per LOD mesh in the terrain. When you export you'll see a graph like this one:

Code: Select all
[formatted]============ Chunk Statistics ============
      Level    Count    Avg. Size
         0      256      761.562500
         1       64     2978.140625
         2       16     8176.562500
         3        4    22972.250000
         4        1    73725.000000
==========================================
                    chunks:      341
               input verts:      4194304
             output verts:       58817
        avg verts/chunk:         172
            output bytes:        12645
        bytes/input vert:        0.00
      bytes/output vert:         0.21
             real triangles:     682000
===========================================[/formatted]

In this example we specified a tree depth of 4. You can see the Avg. Size (actually average triangles) in each terrain LOD level. Ideally you want more than 1000 vertices per level (since the triangles are rendered as strips you can estimate verts = triangle count * 2), but it isn't mandatory. Also you want to keep each level under 32K vertices. Many times this isn't possible with the top levels of LOD like in this example without increasing the error tolerance. As long as you never get far enough away from the terrain to show the top level LOD block this doesn't matter.

So basically this is a trial and error process. I suggest starting with a value of 4 to 6 and doing a few imports before you settle on one value. If you wanted to be really through you could go as far as measuring frame rate with different tree levels, but in general this isn't necessary.


New Notes of Vertical Scale = HEIGHT_ERROR

Fixing map height is done with the “vertical scaling” parameter on the “generateChunkFile” part. By expressing that parameter as just a decimal (e.g. 0.0856). Its easier to pass in my vertical scale divided by 65,536 here. So, if your terrain has a vertical scale of 1000 (minimum height 300 and maximum 1300 for example), then pass in “1000/65536” for that parameter. Torque will evaluate the math equation when it performs the function. To make your terrain features more dramatic, increase the first number before the “/” there. These are usually big jumps, so bump it up a few hundred at a time.

Vertical Scale for L3DT = HEIGHT_ERROR

HEIGHT_SCALE is found using the L3DT viewer. If HEIGHT_ERROR is too large,
you will get a terrain that does not closely follow the heightmap and may have LOD problems.
If HEIGHT_ERROR is too small, the terrain size will be huge and framerates will be terrible.

For Vertical Scale in L3DT which is know as the height scale in the script.

To get this value from L3DT, go to
'operations->heightfield->change vert. scale' menu option.
In that dialog there will be an 'alt range' field. Take note of this number.

To calc this number use the follow calculation

Then, open calc.exe (or equiv.) and calculate the following:

65536 / AltRange



HEIGHT_ERROR Notes

This value is used to decide how much error, error being deviation from the heightmap, to allow in the mesh generated from the heightmap. Too large an error tolerance and you will get significant visual popping of LODs and the mesh won't follow the heightmap closely. Too small and your performance is terrible and your final atlas file will be huge.

For the numbers for the height error is best always to start with a high number. For example for a 4096 map size start off with a number 20, you end up working this number down.

Example of a map work on other night, I started with 20 and end up with 0.5 when I was done.


Here is a break down of reducing the numbers in half.

"[20, 10, 5, | 0.5, 0.25]"




Size And Height Scale - For FreeWorld3D

Two of the parameters needed when converting the heightmap into geometry is its size and height scale. In the 16bit RAW file each pixel is a value between 0 and 65536... 0 being the lowest terrain point and 65536 being the highest. The height scale converts the value in the height map into a meter scale unit. The size is simply the dimensions of the RAW file.

Freeworld3D provides these values for you in the world.cfg

<Terrain Size="1024" Step="1.000000">
<Heightmap File="/terrain/heightmap16bit.raw" Format="16bit RAW" Scale="0.002777"/>
</Terrain>

The Size="1024" and Scale="0.002777" bits are the parts your interested in here. These values will be needed later in the command to generate the Atlas chunk file.



HEIGHTMAP_SIZE and METERS_PER_PIXEL were determined when we created the design map.
If you recall, we set them equal to 1024 x 1024 and 1 (respectively) so that we get a 1 km x 1 km terrain.


Meters Per Pixel

Another value you'll need for chunk file generation is how many meters of terrain to generate per heightmap pixel. Freeworld units are pretty arbitrary and it's up to you to decide on the final Torque size. Consider that a 1024 heightmap at 1 meter per pixel is 1 sq km (or a little under 2/3 of a mile). It may just take a few imports and play tests before you decide. Remember that this value can also be fractional.


Meters Per Pixel for L3DT

This is simple and was the number you put in when first starting building your map. In the Design map size in the area called "Heightfield Resolution (m)" this is the number you be using for your meter per pixel. For torque recommend using a 1 or a 2 for this area.

------------------------------------------------------------------------------------------------------------------

Now we need to generate the actual Atlas2 geometry. Enter the following command in your console window.

PART 2

Code: Select all
importOldAtlasCHU(
"data/terrains/deleteme.geometry.chu",
"data/terrains/geometry.atlas");



Again, include the full path to your chunk and Atlas geometry files.
The variables are pretty self-explanatory here


importOldAtlasCHU(
"YOUR_MOD_FOLDER/data/terrains/deleteme.geometry.chu",
"YOUR_MOD_FOLDER/data/terrains/deleteme.geometry.atlas" )

------------------------------------------------------------------------------------------------------------------

Now we will generate our Atlas2 texture map. Enter the following command in your console window:

PART 3

Code: Select all
atlasGenerateTextureTOCFromLargeJPEG(
"data/terrains/colormap.jpg",
2,
"data/terrains/texture.atlas");


As always, include full paths. The middle number, 2 in the example above, is the same TREE_DEPTH


At last, we can generate our final Atlas2 terrain. We are generating a "unique terrain"
which uses a single texture file overlaid on the terrain. This tends to be the most accurate type of
textured terrain, but it usually takes up very large amounts of disk space. A "blended terrain" is somewhat
more efficient space-wise, but that will be the subject of another tutorial..

------------------------------------------------------------------------------------------------------------------


atlasGenerateTextureTOCFromTiles

usage: (leafCount, tileMask, outFile, outFormat)

Generate a texture TOC from a set of tiles.
leafCount is the size of the grid of tiles on a side.

tileMask is the path for the tiles (no extension) with %d for x and y, ie,'demo/alpha/Alpha1_x%dy%d'.

outFile is the file to output a new .atlas file to.

"Example YOUR_MOD_FOLDER/data/terrains/colormap_x%dy%d."

Note: Your file name will look like this for example "colormap_y0d1.png" any number are replaced with this "%d"
the %d is a place holder for the script.


In the code there is no extension needed.

The following formats are allowed "PNG, JPG " there is other one don't recall it at this time.


The final field in that command defines the input file format. If you are using jpeg use "0" while if you are using png use "1".

Code: Select all
atlasGenerateTextureTOCFromTiles(   4,
                                       "YOUR_MOD_FOLDER/data/terrains/colormap_y%dx%d",
                                       "YOUR_MOD_FOLDER/data/terrains/deleteme.lightmap.atlas",
                                       0 );



-------------------------------------------------------------------------------------------------------------------

Enter the following command in your console window:

PART 4

Code: Select all
atlasGenerateUniqueTerrain(
"data/terrains/terrain.atlas",
"data/terrains/geometry.atlas",
"data/terrains/texture.atlas");



At this point you are done creating your files needed for Torque. The next step is too remove the unwanted files and edit you .mis file to tell it the file name for the .atlas


Recommend running all these commands in a script, also doing each part separate. By running the commands in separate scripts you see the errors or any problems that might pop up.

Part 1 of this scripting process is the most important, this script you have run over a few times till your able get the numbers down to the correct parameters

For exec a script in the console type the following
Code: Select all
exec ("FOLDER/FILENAME.cs")


Myself i have the four scripts named out as part1.cs and so forth, this allows me to up arrow and change the number only to call the next script.
-----------------------------------------------------------------------------------------------------
Map Sizes and comparison

Map Size 1
8192 x 8192 pixels (64 Mpxls) 16 x 16 kilometres

16 x 16 kilometers = 256 kilometers = 159.071025 miles
Which is the size of the SWG maps!

Map Size 2
32768 x 32768 pixels (1.0 Gpxls) 66 x 66 kilometres

66 x 66 kilometres = 4 356 kilometers = 2 706.69291 miles

Map Size 2
131072 x 131072 pixels (16 Gpxls) 262 x 262 kilometres

262 x 262 kilometres = 68 644 kilometers = 42 653.4041 miles
------------------------------------------------------------------------------------------------------------------

Heightfields Extra Information from GG

Until a more intuitive tool comes along, you will work with the generateChunkFileFromRaw16 script function to create chunk files for Atlas. These are the arguments that it uses:

* srcFile
Filename of the source raw 16 bit heightfield we wish to generate data for. Example: "demo/data/terrains/test_16_4097.raw".


* size
Size of the heightfield, which is assumed to be square. The heightfield will be expected to be ONE MORE THAN THIS. Integer. Size must be a power of 2. Example: 4096.


* squareSize
Spacing between sample points, in meters. The default in legacy was 8 meters. This is a floating point number. There are no limitations on this value, although high polygon density may result in reduced frame rates. Example: 2.0. In L3DT terms this is the heightfield resolution.


* vertScale
Vertical scale factor. Incoming raw data comes in as a whole number between -32,765 and +32,765, and is scaled by this factor to convert to meters. Factors are commonly of the form 1.0 / 2.0n, n often being between 1 and 8. 1.0 / 256.0 gives you a workable range of about 256 meters in the vertical range. To get an accurate size find the highest point of your terrain and the lowest point. If the lowest is negative add them together. If the lowest is positive then subtract. With your total divide by 65535. Take the first 4 decimal places over and you have your vertical scale. Example: 0.02098


* destFile
File to which generated data is written. This file should end in .chu for TGEA's resource manager to properly load it. Example: "demo/data/terrains/large.chu".


* error
Floating point number indicating the acceptable amount of error for each LOD to possess compared to the previous. This directly and dramatically impacts the number of polygons that will be rendered. Example: 2.0.


* treeDepth
Integer indicating how deep the quadtree of chunks is to be. Example: 6.
Last edited by Madcowthomas on Wed Jun 06, 2007 9:49 am, edited 1 time in total.
From the mad cows desk to yours,
Peace
Madcowthomas

Yo...Reggie vacation mom!

"Without imagination we would cease to exist"
William Thomas
(1970-)
Warning!
Burping Babies are the Cause of Global Warming!
User avatar
Madcowthomas
Doyen
 
Posts: 159
Joined: Mon Feb 19, 2007 3:40 am
Location: Seattle Washington - Salmon Fishing

Postby Aaron » Wed May 30, 2007 12:39 am

Hi Madcowthomas,

This is great. Would you mind if I uploaded this to the tutorials wiki?

Cheers,
Aaron.
User avatar
Aaron
Site Admin
 
Posts: 3696
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia

Postby DeathTwister » Wed May 30, 2007 12:51 am

Wow Madcowthomas,
awesome work dude.
Yes please I think that is a very good idea Aaron. this is one awesome tutorial, thanks so much for the foot work you did for us dumb artists like me that have been having trouble with the math and steps. /bows.
DT
User avatar
DeathTwister
Dearly missed
 
Posts: 562
Joined: Thu Dec 15, 2005 12:30 pm
Location: Klamath, CA.

Postby Madcowthomas » Wed May 30, 2007 2:30 am

Yeah welcome to upload to the Wiki.

Hoping we all find stuff to add to it and make this knowledge a lot easer for the new people

Have to thank all the people who took the time to help me.


DeathTwister now like to pick everyone brain here in L3DT write up some tutorials on creating some awesome terrains, myself still lost using this software.
From the mad cows desk to yours,
Peace
Madcowthomas

Yo...Reggie vacation mom!

"Without imagination we would cease to exist"
William Thomas
(1970-)
Warning!
Burping Babies are the Cause of Global Warming!
User avatar
Madcowthomas
Doyen
 
Posts: 159
Joined: Mon Feb 19, 2007 3:40 am
Location: Seattle Washington - Salmon Fishing

Postby Aaron » Wed May 30, 2007 4:15 am

Hi Madcowthomas,

I've uploaded the info to:

http://www.bundysoft.com/wiki/doku.php?id=tutorials:torque:madcowthomas

I've reorganised it a little bit, added some extra info where it might help, and also removed/consolidated some repeated info.

Thank-you for compiling such a thorough guide! Based on that information I think I can write a new Atlas export plugin to export the map files, automatically write the script files, and (maybe) call TGEA to execute the scripts. I'll put that on the to-do list.

Edit: Here is the PDF:

http://www.bundysoft.com/wiki/lib/exe/fetch.php?id=tutorials%3Atorque&cache=cache&media=tutorials:torque:tgea_madcowthomas.pdf

Best regards,
Aaron.
User avatar
Aaron
Site Admin
 
Posts: 3696
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia

Postby Madcowthomas » Wed May 30, 2007 6:08 am

:) Thanks for cleaning that up Arron, looks awesome!
From the mad cows desk to yours,
Peace
Madcowthomas

Yo...Reggie vacation mom!

"Without imagination we would cease to exist"
William Thomas
(1970-)
Warning!
Burping Babies are the Cause of Global Warming!
User avatar
Madcowthomas
Doyen
 
Posts: 159
Joined: Mon Feb 19, 2007 3:40 am
Location: Seattle Washington - Salmon Fishing

Postby Aaron » Wed May 30, 2007 1:36 pm

Hi Madcowthomas,

I've finished my first draft of the new plugin to save atlas files using the console scripts, based on the info in your tutorial. Would you care to test it for me? (this invitation is also open to any other interested readers).

Firstly, this plugin requires the latest dev build of L3DT Pro (v2.5a build 2), which you can find on the pro downloads page.

Once v2.5a build 2 is installed, download the following zip file:

http://www.bundysoft.com/L3DT/downloads/plugins/bin/L3DTio_Atlas2-bin.zip

Extract the plugin (L3DTio_Atlas2.zeo) file from the zip to some place sensible, like:

C:\Program files\Bundysoft\L3DT Professional 2.5.1.2\Extensions\

Double-click on the zeo file. This will launch L3DT, which will install the plugin.

Now, to use the plugin, go to 'Extensions->L3DTio_Atlas2->Export to TGEA Atlas (take two)' in the menu.

Enter the desired atlas file name, tree depth, and max error (same as height error). The default values for tree depth and max error (4 and 1.0) should be reasonably OK anyway.

Click OK.

The plugin will then go away and save the heightfield as a RAW, save the texture as a JPEG (if required), and create the '.cs' script to run in the console. Note it does not create the atlas file. However, it does create the complete script file to create the atlas file. If you run that script file in the TGEA console, it will create the atlas file.

...at least, that's the theory. I don't have TGEA installed on this computer, so I haven't been able to test it yet. Feedback is most welcome.

Cheers,
Aaron.

PS: If it works, I'll add the extra code to call the console application with the script, plus add some nice stuff like blended terrain support.
User avatar
Aaron
Site Admin
 
Posts: 3696
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia

Postby Madcowthomas » Wed May 30, 2007 3:41 pm

Oh wow! Reading that post just woke me up. :) Downloading the new dev and plugin and start testing right away. You work fast
From the mad cows desk to yours,
Peace
Madcowthomas

Yo...Reggie vacation mom!

"Without imagination we would cease to exist"
William Thomas
(1970-)
Warning!
Burping Babies are the Cause of Global Warming!
User avatar
Madcowthomas
Doyen
 
Posts: 159
Joined: Mon Feb 19, 2007 3:40 am
Location: Seattle Washington - Salmon Fishing

Postby DeathTwister » Thu May 31, 2007 12:57 am

Madcowthomas hay there,

DeathTwister now like to pick everyone brain here in L3DT write up some tutorials on creating some awesome terrains, myself still lost using this software.


haha You gussed it /winks Ok I will work on some Tuts for yaz, but I am kinda busy atm, in a week or so will be good? and what kind of stuff you like to see on Terrains?

Oh wow! Reading that post just woke me up. Smile Downloading the new dev and plugin and start testing right away. You work fast


Haha Yes he does, and I just installed the last one DOh /smiles... So I guess I need to go do the same thing again /smiles....worth it every time.

DT
User avatar
DeathTwister
Dearly missed
 
Posts: 562
Joined: Thu Dec 15, 2005 12:30 pm
Location: Klamath, CA.

Postby Aaron » Thu May 31, 2007 1:01 am

Hi DT,

Just a quick note: With William's help I've found a few mistakes in the new plugin, which I'll fix later today. I'll let you know when the update is ready for testing.

Cheers,
Aaron.
User avatar
Aaron
Site Admin
 
Posts: 3696
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia

Postby Madcowthomas » Thu May 31, 2007 5:41 am

Hey there DeathTwister,

Or better yet my neighbor down below me, I sure miss living in Ca. I used to live in Southern Ca in Thousand Oaks.

Tutorials, well right now like to pick everyones brain and ask very newbie questions to start with first. Then have a better clue on where to go from there.

Right now trying to understand the edit tool and how to create my own maps from scratch. What I'm doing is create a map in Terragen and imported into Leveller and make the changes that I looking for and flatten the edges of the map out.

Then export as heightmap into L3DT , one problem I had with the heightmaps until I went into the , Design map and told it to generate a HF map for it.

Not sure how to explain what it was doing , so I'll run a map and show it to explain it better. Best I describe it was burning the heightmap lines into the lightmap and texture map.
From the mad cows desk to yours,
Peace
Madcowthomas

Yo...Reggie vacation mom!

"Without imagination we would cease to exist"
William Thomas
(1970-)
Warning!
Burping Babies are the Cause of Global Warming!
User avatar
Madcowthomas
Doyen
 
Posts: 159
Joined: Mon Feb 19, 2007 3:40 am
Location: Seattle Washington - Salmon Fishing

Postby Aaron » Thu May 31, 2007 12:30 pm

Hi William and Maylock,

Would you care to try the updated plugin:

http://www.bundysoft.com/L3DT/downloads/plugins/bin/L3DTio_Atlas2-bin.zip

This should, in theory, launch TGEA and generate your atlas file. The first time you use it you will be asked to locate TGEA.exe, but after that, you shouldn't be bothered with that again. Please let me know how it goes.

Cheers,
Aaron.
User avatar
Aaron
Site Admin
 
Posts: 3696
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia

Postby DeathTwister » Thu May 31, 2007 1:13 pm

hay terra nut brothers,

Wow your from Seattle? Cool, if you come down this way and pass by? I live almost on hwy 101 2 miles into the redwood forest, along the coast at the Mouth of the Klamath River. Bring a tent as I have a camping area like in the parks for friends to stay and a motor home when not being used as an art studio I let my friends stay who do not like to camp out, but like to camp in /winks. 37 miles south of the Oregon border, 4 miles from the world famous Trees of Mystery. all who terrain are welcome /bows.

Oh very cool Aaron,
I will try and do it later today after my Programmer gets here from Eugene, OR. or tomorrow first thing, and now I have to decide what to try it on hahahahaha. Been working in TGE for now as it had/has a few less bugs still then TGEA, but been wanted to switch out the farm so to speak, and probably can after this gets smoothed /smiles out.

Going to grab the latest update as well to make sure I mess it all up ROTFL just kidding, good test is when even a dumb artist like me can use it easy /smiles. If I can do it? so can a cave man hahahahahahaha I know dumb joke Oh well, KK downloading.

DT
User avatar
DeathTwister
Dearly missed
 
Posts: 562
Joined: Thu Dec 15, 2005 12:30 pm
Location: Klamath, CA.

Postby Madcowthomas » Thu May 31, 2007 4:21 pm

Sand in my eyes burning, waking up ahhh!

Oh now that sound like good spot you have there Deathtwister.

Downloading the new plugin and start testing
From the mad cows desk to yours,
Peace
Madcowthomas

Yo...Reggie vacation mom!

"Without imagination we would cease to exist"
William Thomas
(1970-)
Warning!
Burping Babies are the Cause of Global Warming!
User avatar
Madcowthomas
Doyen
 
Posts: 159
Joined: Mon Feb 19, 2007 3:40 am
Location: Seattle Washington - Salmon Fishing

Postby Madcowthomas » Thu May 31, 2007 4:43 pm

Okay it launches my TGEA but as far as generating the atlas files is a no go there. Open up the console window and nothing there indicates its processing the files.

It creates the .raw file and the script in the directory where tell it.
From the mad cows desk to yours,
Peace
Madcowthomas

Yo...Reggie vacation mom!

"Without imagination we would cease to exist"
William Thomas
(1970-)
Warning!
Burping Babies are the Cause of Global Warming!
User avatar
Madcowthomas
Doyen
 
Posts: 159
Joined: Mon Feb 19, 2007 3:40 am
Location: Seattle Washington - Salmon Fishing

Next

Return to Help and support

Who is online

Users browsing this forum: No registered users and 8 guests

cron