L3DT the right choice?

[h4voc]

Greetings,

while searching the internet for a solution for a problem i came here.
Now my problem and if L3DT can solve it.

I have created a huge map file in Photoshop. It covers borders and a raw sketch of a heightmap, colormap, splatmap etc. . But to be able to paint in Photoshop in this huge file I was forced to slice the file to 16 tiles. (Original size was 53k*38k pixels.). Now this slicing causes seams between this tiles.
I thought of a way to stitch then together again, map-type by map-type and a way to bring detail to the map-types.
I saw that L3DT can stitch tiles.
Now my question is:
-Can L3DT import heightmap files as input?
-Can enhance the imported tiles?
-Then stitch them to my final map?
-Does stitching remove the seams well?
-Can L3DT handle this huge files?

Any other ideas or solution based on L3DT?

Thanks everyone.

[Aaron]

Hi H4voc,

Welcome to the forum. I think L3DT should be able to help here, but you'll need the Professional version to cope with maps of that size. You can download a free 90 day trial from here.

-Can L3DT import heightmap files as input?

Yes. What file format are you using, and are the tiles all exactly the same size? Does the file contain only height data (in greyscale, if an image), or is there other annotation in the file? Can you post an image of a small sub-region so that I can see what we're dealing with?

-Can enhance the imported tiles?

Yes. The recommended way to do this would be to import as a 'design map', and then have L3DT generate a heightfield that follows the same general shape but with additional details and effects. Importing a design map is covered here.

-Then stitch them to my final map?
-Does stitching remove the seams well?

Yes, please see the stitching heightmaps tutorial.

I would strongly recommend that you stitch all your heightmaps together into one heightmap before you import as a design map, as this will ensure that the seams will be fixed in the most seamless way.

-Can L3DT handle this huge files?

Yes, should do. I've made larger heightmaps in L3DT, though not when following this exact workflow. Please let me know if you find any problems.

Cheers,
Aaron.

[h4voc]

Hi H4voc,

Welcome to the forum. I think L3DT should be able to help here, but you'll need the Professional version to cope with maps of that size. You can download a free 90 day trial from here.

-Can L3DT import heightmap files as input?

Yes. What file format are you using, and are the tiles all exactly the same size? Does the file contain only height data (in greyscale, if an image), or is there other annotation in the file? Can you post an image of a small sub-region so that I can see what we're dealing with?

-Can enhance the imported tiles?

Yes. The recommended way to do this would be to import as a 'design map', and then have L3DT generate a heightfield that follows the same general shape but with additional details and effects. Importing a design map is covered here.

-Then stitch them to my final map?
-Does stitching remove the seams well?

Yes, please see the stitching heightmaps tutorial.

I would strongly recommend that you stitch all your heightmaps together into one heightmap before you import as a design map, as this will ensure that the seams will be fixed in the most seamless way.

-Can L3DT handle this huge files?

Yes, should do. I've made larger heightmaps in L3DT, though not when following this exact workflow. Please let me know if you find any problems.

Cheers,
Aaron.

Hi Aaron, thank for the reply.
After your answere and further investigation i think L3DT is the way to go for me.
I am really happy that i found this software and this forum with all the tutorials and support.

Allready downloaded the trial. Now i will start playing around with all the features.

So for my Project the next step would be to Import the hole sketchmap/heightmap as design map?

Is there any tutorial for understanding design maps? When i look at this tileview i dont get it. How can i "design" something?

[Aaron]

Hi H4voc,

Thinking again about it, the first thing you might like to try is to just import one tile or sub-region of your map. It should be much faster and easier to learn the tools and workflow with only a small map area.

So, I'd recommend you start by importing just one heightmap tile as a design map. If you like, you can also import your sketch as an image drape to help with editing the design map (see user guide). Once you're happy that you can edit the design map and heightmap to get the results you want, you might then move on to stitching the whole heightmap together and processing it as a single huge design map.

Is there any tutorial for understanding design maps?

Yes indeed. The fjord tutorial is recommended reading. The ski run tutorial may also be a useful example of using the design map to get a specific terrain shape. You can also find out more info about the different parameters in a the design map from here. If you have any questions, please feel free to ask.

Cheerio,
Aaron.

[h4voc]

Hi Aaron, thanks for the fast reply.

Hi H4voc,
Thinking again about it, the first thing you might like to try is to just import one tile or sub-region of your map. It should be much faster and easier to learn the tools and workflow with only a small map area.
So, I'd recommend you start by importing just one heightmap tile as a design map. If you like, you can also import your sketch as an image drape to help with editing the design map (see user guide). Once you're happy that you can edit the design map and heightmap to get the results you want, you might then move on to stitching the whole heightmap together and processing it as a single huge design map.

Yes that is what is thought too. I just started with a small Piece of my big heightmap and started designing the map and understanding how things work.

Is there any tutorial for understanding design maps?

Yes indeed. The fjord tutorial is recommended reading. The ski run tutorial may also be a useful example of using the design map to get a specific terrain shape. You can also find out more info about the different parameters in a the design map from here. If you have any questions, please feel free to ask.

I just went to all tutorials i could find. Seems i got it! Thank you.


After finishing a small map i will try to get a huge map working.
Now i came to the point where i try to import the huge sketch file and use the design map to get things working.
In my first attempt i created a new project with a design map and set the size to 64k x 64k.

My first question here is:
When i try to set the HF/DM Ration i got an errormsg. (cannot remeber which one). I was tryn to set the ration to the highest resolution. After the error i set it to 64. Any way or chance to increase my Resolution there? I will try to go further in this problem next days. So I can post the specific errormsg.

Next question is about performance.
After I got a design map with a size of 64k x 64k and a HF/DM Ration of 64 I just “painted” some mountains on the map and started the heightmap generation process.
Well it took pretty long to generate this huge size of a map. Which is ok for my. But then I checked my computer ressources : CPU usage was 9% ram about 20%, no significant scratchdisc usage (<5%).
Now I was checking the multithreading settings. I try both, manual and auto mode. Both did not increase cpu usage.
Anything I can setup to increase the performance? I got 6 physical core with multithreading, so 12 virtual cores.

Thanks for your great support here Aaron!

Greetings h4voc

[Aaron]

Hi H4voc,

When i try to set the HF/DM Ration i got an errormsg. (cannot remeber which one).

Hmm...128 seems to work fine for me. Can you please send me the error message next time it occurs?

I was tryn to set the ration to the highest resolution.

Just curious: was this to reduce your design map size to minimise design work, or did you have some other reason?

Any way or chance to increase my Resolution there?

Not at present. I did tinker with even larger HF/DM ratio algorithms, but I found that it was increasingly difficult to tune all the parameters in the subroutines to produce a nice output. At present, the best quality seems to be the algorithm for the HF/DM ratio of 64. Of course, if you want higher expansion ratios, you can always use the 64x algorithm to generate the heightfield, then import that heightfield as a design map with no sub-sampling, and give it another 64x inflation to the final heightfield. This would give a final effective HF/DM ratio of 4096.

Well it took pretty long to generate this huge size of a map. Which is ok for my. But then I checked my computer ressources : CPU usage was 9% ram about 20%, no significant scratchdisc usage (<5%).

The heightfield generation process is not multithreaded yet. Multithreading has so far only been applied to the really CPU-intensive high-resolution algorithms; namely generating the attributes map, the normal map, the light map, and the texture map. These maps are usually generated with 4 to 16 times the resolution of the heightfield (i.e. 16 to 256 times the number of pixels), so they would be prohibitively slow without multithreading. Multithreading some of the heightfield generation filters is certainly on my to-do list, but it's far from straightforward for some of these algorithms. Rest assured that I will definitely let you and everyone know when I have made progress on this.

Best regards,
Aaron.

[h4voc]

Hi Aaron,

gettin into it more and more.
First thanks for your great support.

I was not able to recreate the error i got last time. Works fine now.

My workflow will be like this:
Take my sketch and reduce the resolution from 54kx38k to 2014x~1300 to be able to import for image drape (i did try to use 54k x 38k res for this, but got an error msg)
What i found out is that even if my target mapresolution is 54k x 38k the image drape scales the imported image and stretches it over the design map. (very useful)
Now i can create my whole map with the designmap editor.

After understanding how designmapping works i have to say i like the way it works. It speeds up my peviously planed workflow by over 1500%. Thats awesome.

One thing i am unsure how it works is the HF/DM Ratio:

What i understood is: the higher the ration is the smaller is the area visualized by one designmap tile right?
So when my map resolution is really high the area i can control with a ratio of 128 is relative huge compared to a small map with a ration of 128? (that is why it is a ration i know )
So now i have to test if i have enough control over the map details with my resolution of 54k x 38k and a ration 128?
Or does it work the complete opposite way?
Any chance to see a ratio of 256 comming?

Looking forward to multithreaded heightmap calculations.
Another performance question, i found a thread about 64x bit Version. Any news on that?

regards, h4voc

[Aaron]

Hi H4voc,

So now i have to test if i have enough control over the map details with my resolution of 54k x 38k and a ration 128? Or does it work the complete opposite way?

Yes, I think you've got that right. Higher ratios give less detailed control over larger areas.

Any chance to see a ratio of 256 comming?

It's not very likely to be any time soon, I'm sorry to say. Debugging the next release is the current (and only) priority. Once complete, there's still a big pipeline of feature requests to handle.

Another performance question, i found a thread about 64x bit Version. Any news on that?

The 64-bit version won't be available until I'm perfectly happy that the 32-bit version is bug-free (which is not yet). The 64-bit version boosts performance only over a narrow range of map sizes and only for some calculations. If you're making maps larger than will fit in the installed amount of RAM (and I think you are), then there's no huge benefit to going 64-bit.

Cheerio,
Aaron.

[h4voc]

Thanks again for your support Aaron.

Now i encountered an error:

Workflow:
Created a DM with a Resolution of: 53248 x 38912
Loaded an Image drape Image with a Resolution of: 2048x1497
Traced the Image. Painted some mountains on the map.
Clicked Create HM:

Now the following error appears.
L3DT is still runnning since 12hours.


This is an extract of the error msg i am getting. The first value (in this case 3108) counts very fast and exceeds the 3328 while the seconds value (in this case the 2470) Counts slowly from 1 to a final value. After this the seconds value starts again from 0.

Code: Select all

CMapWrap::SetPixel error:

- invalid coordinates (3108, 2470) in 3328x2432 map, type 1, name ''

CMapWrap::SetPixel error:

- invalid coordinates (3109, 2470) in 3328x2432 map, type 1, name ''

CMapWrap::SetPixel error:

- invalid coordinates (3110, 2470) in 3328x2432 map, type 1, name ''

CMapWrap::SetPixel error:

- invalid coordinates (3111, 2470) in 3328x2432 map, type 1, name ''

CMapWrap::SetPixel error:

- invalid coordinates (3112, 2470) in 3328x2432 map, type 1, name ''

CMapWrap::SetPixel error:

- invalid coordinates (3113, 2470) in 3328x2432 map, type 1, name ''


Any idea?
It seams that the count keeps repeating.

- invalid coordinates (35xx, 25xx) in 3328x2432 map, type 1, name ''

both coordinates were exceeded.
I dont know if there is any progress on calculating the HM.

[Aaron]

Hi H4voc,

Thank you for the bug report, and sorry for this fault. I'm trying to reproduce it now, and I'll let you know what I find when the run finishes.

Best regards,
Aaron.

[Aaron]

Hi H4voc,

Hmmm...this build is going to take quite a long time. In the mean-time, could I ask you for the complete error log? You can get to it by clicking on the 'All programs->Bundysoft->L3DT [version]->L3DT logs' shortcut in the Windows Start menu. That shortcut will open a directory, in which you will find a file called 'log.txt'. Please send it to aaron@bundysoft.com. With that file I should be able to work out what is going wrong. Please collect the log file after the fault occurs in L3DT, and before re-opening the program, as the log file is overwritten at the start of each session.

Kindest regards,
Aaron.

Edit: Err, I just had a closer look at what you've posted above, and it looks like the error might just be coming from the progress map, so the calculation might go through OK. The log would really help me find which exact calculation is botching up the progress map.

[h4voc]

Hi Aaron,
thanks again for the great support.
Ill check if i can find the log when i am back from work.

But the log might not help you. I had to push the clear log button several times as it slowed my machine.
The error msg is generated 1 to 10 times a seconds depending on Speed.
When i hit the clear log button it runs much faster for ~10min.
Does "clearlog" button only reset the active shown log or the file it self?

but maybe i can confirm your guess that the calculation is going through. From time to time i can see the progress window and the background msg changes from "peak calculations" to "inflating". Seems to take a while. What i dont know is, if it is slower because of the error log?

many thanks and greetings
h4voc

[Aaron]

Hi H4voc,

The clear button only clears the window. The log file is unaffected.

Writing to the log definitely slows down some calculations.

Best regards,
Aaron.

[h4voc]

Hey Aaron,

the calculation is still in progress but i sent you a mail with the log from the current progress.

best regards
h4voc

[Aaron]

Hi H4voc,

Thanks for the log. I think this is related to using the HF/DM ratio of 16. Will look into it more after work tonight.

Cheerio,
Aaron,

PS: For the next build I've also modified the program to make smaller log files by default.