L3DT users' community
Large 3D terrain generator

ME-DEM requests

It doesn't hurt to ask...

ME-DEM requests

Postby monks » Sun Jul 02, 2006 8:46 pm

Hi Aaron, I'm currently looking at the ME-DEM workflow and I'm attempting to work through the entire process of importing the base DEM and then replacing it in the modelling apps. Consequently I've had a few insights as to what would be useful specifically for ME-DEM, but I also think for a lot of other terrain modellers out there, looking to build large terrains that 'make sense'.
Among other thngs, the upcoming article will compare the relative speeds and terrains of L3DT, Wilbur and Leveller. GeoControl will also be thrown into the mix, though not as a complete solution yet.
I'm not going to be able to build the terrain in each app- (1 4K terrain is quite enough!) so I've opted to time-test the building of the mountain range only.
Apart from time considerations I'm not able to use L3DT or Leveller to illustrate base dem import and remodelling because I don't feel there are really the heightmap tools in place yet. It's possible to use these apps but there are currently other more efficient methods (as covered in the article).

The base DEM is extremely useful and I'm now very glad that we chose to go down that path. It basically avoids the problem that a lot of large hand-modelled terrains have: of an essentially flat landscape interspersed with hills and mountains. In ME-DEM's workflow, it's possible to make every pixel contribute to a coherent terrain over continent scale.

I've divided the terrain modelling into 3 parts based on scale: macro, meso, micro.

Macro is really where all the important stuff happens (for large terrains) and which most terrain apps are not really good at. The general flow of the land, governed mainly by the larger rivers. How to use the base DEM? I think the only way to work with it is to completely replace it during our phase 2.
There's only one real solution to this imo and that's contours and contour interpolation or the equivalents. If there was any way to introduce this into L3DT it would triple it’s usefulness at a stroke.

In L3DT we have the DM generated from the imported base DEM and contours in the heightmap generation or the heightmap prev. One immediate help would be to have those contours visible over the Design Map. With this it would be possible to quickly and accurately re-model large areas through the DM. Perhaps only the contour pixels too, rather than a complete overlay if you see what I mean.

Another way to achieve contours is simply by select area> set to height, but contour drawing- ie a pencil rather than arbitrary select, etc, are more intuitive imo. Perhaps a Heightmap pencil? or a DM pencil with sub-pixel res for drawing contours which snapped to the DM pixels on inflation to the heightmap?
You could introduce the abilty to cycle between the DM and heightmap within L3DT:import heightmap> generate DM> draw contours on heightmap> generate DM > alter the DM with the DM pencil > generate heightmap, etc. The abilty to define how the new info is incorporated into the existing info would also be handy; replace, add especially.

When altering large areas, if arbitrary selections are not possible, better visibilty for the pixels being altered by the DM pencil in the current pass: maybe set those red until another operation is preformed.

Coupled with the ability to draw/use contours is the real advantage of having a tool which can select from a heightmap by height range: ie select between 899-900m would give a selection on screen defining the 900m contour. With this selection still in place, it would be great to be able to draw a contour which developed this ‘proto-contour' to add more detail or make corrections, etc. For ME-DEM, this avoids the complication of having to import yet another layer: ie dem and contour info. Simply extract the contour info from the dem. The operation has to allow the simultaneous selection and contour or DM drawing though to be useful.


Imo, the two most useful things would be 3D prev and the contours (visibilty/draw/select by height). The pencil current edit visibilty is another must as well, try keeping track of a sea of DM pixels altered by 100m with an image overlay in place...ouch!

I realise you're busy so just some thoughts for the future thrown in. If I have any more I'll pipe up :)


Cheers,
monks
monks
Oracle
 
Posts: 292
Joined: Tue Nov 22, 2005 10:38 pm
Location: Middle Earth

Re: ME-DEM requests

Postby Aaron » Mon Jul 03, 2006 1:17 am

Hi Monks,

Whoa, lots to think about here.

monks wrote:There's only one real solution to this imo and that's contours and contour interpolation or the equivalents. If there was any way to introduce this into L3DT it would triple it’s usefulness at a stroke.


Are there not other (free) programs that can convert contours to DEMs? I could certainly add this to L3DT, but I would probably do a much worse job than the existing solutions out there.

monks wrote:One immediate help would be to have those contours visible over the Design Map. Perhaps only the contour pixels too, rather than a complete overlay if you see what I mean.


Okie dokie, I think that can be arranged.

monks wrote:Perhaps a Heightmap pencil?


That's on the to-do list for v2.4a.

monks wrote:...or a DM pencil with sub-pixel res for drawing contours which snapped to the DM pixels on inflation to the heightmap?


No can do. Too messy.

monks wrote:You could introduce the abilty to cycle between the DM and heightmap within L3DT:import heightmap> generate DM> draw contours on heightmap> generate DM > alter the DM with the DM pencil > generate heightmap, etc.


Yup, that shouldn't be a problem.

monks wrote:The abilty to define how the new info is incorporated into the existing info would also be handy; replace, add especially.


Possible, but highly unlikely I'll do this any time soon. It makes things a whole lot more complex.

monks wrote:When altering large areas, if arbitrary selections are not possible, better visibilty for the pixels being altered by the DM pencil in the current pass: maybe set those red until another operation is preformed.


I like it. Added to the to-do list.

monks wrote: Coupled with the ability to draw/use contours is the real advantage of having a tool which can select from a heightmap by height range: ie select between 899-900m would give a selection on screen defining the 900m contour. With this selection still in place, it would be great to be able to draw a contour which developed this ‘proto-contour' to add more detail or make corrections, etc. For ME-DEM, this avoids the complication of having to import yet another layer: ie dem and contour info. Simply extract the contour info from the dem. The operation has to allow the simultaneous selection and contour or DM drawing though to be useful.


I'll think about this after I make the above changes to contour drawing.

Cheers,
Aaron.
User avatar
Aaron
Site Admin
 
Posts: 3696
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia

Postby monks » Mon Jul 03, 2006 11:12 am

Whoa, lots to think about here.


:)


There's only one real solution to this imo and that's contours and contour interpolation or the equivalents. If there was any way to introduce this into L3DT it would triple it’s usefulness at a stroke.


Are there not other (free) programs that can convert contours to DEMs? I could certainly add this to L3DT, but I would probably do a much worse job than the existing solutions out there.


There are, but if L3DT is establishing itself as a LARGE terrain modeller, and I think it is (!), then contours is THE feature for this job imo.
The way I see it, is you can do one of 3 things for the big stuff (in terms of the more user-defined, hand modelled terrains):
Work with a brush at a smaller res, then upscale. This is one traditional method. L3DT basically implements this, albeit in a cool, novel (and improved) way: DM-> HM.
Use contours: draw and interpolate.
Use selections and set those to uniform alts, building up the terrain like a layer cake.

The great thing about contours is that they are so efficient in terms of information. The way I see it is that if you have a heightfield described as contours it's like having a compressed heightfield which you can unzip upon interpolation. Wilbur has a tesselation tool which gives the abiltity to define (lofted) Strokes. Set a Stroke with all points at equal alt and you have a contour. Very convenient. I'm not saying that that is necessarily the only way to go, but the general process of Stroke/contour definition (design) -> interpolation (inflate) is pretty much unbeatable for the large scale stuff. This is what L3DT has in place already right? 8)

As for an implementation of contour interpolation, at the scale that L3DT is capable of working at, the contour description is never going to provide anything more than a base onto which to build. I also think you are being modest :lol: Beyond a certain resolution, contours are inefficient and difficult anyway, they're only part of the picture. It's that base though that is all important if you want to get a logically sound *large* heightmap, because it's all about the rivers at that scale. Define contours that define the river courses and essentially you have a good foundation in place, ie, you are not adding rivers as an aferthought.
The way I've approached the terrain is to split it into the base, the mountains and hills,(meso) and the final noise (micro) I built them as seperate components and combined them in Leveller. Levellers MDI (and GeoControl's layers- though I've yet to use them) are imo, indispensable right now. I understand Wilbur has MDI (unreleased) and layers is in development.

Imo your implementation needn't be super-powerful (eg Global Mapper, VNS), indeed Wilbur's is kind of 'ropey' (I don't think contours was even in the tess design), but it's still the best tool for the job currently out there.
Also, ME-DEM has had something of a re-think regards the topo transcription that we are using. We're really not going to want to be that exact/literal for a number of reasons: accuracy is less of an issue now.
So (ideally) you get your base down with contours, interpolate, then add your hills and mountains, then the finishing touches.

Another advantage of contours is that it becomes visually very clear how tiles fit together at the edges. There are still other problems to be ironed out there though in all the apps in terms of importing adjacent tiles, etc.



One immediate help would be to have those contours visible over the Design Map. Perhaps only the contour pixels too, rather than a complete overlay if you see what I mean.


Okie dokie, I think that can be arranged.



Great! I'll be very interested to see what can be done with that.


Perhaps a Heightmap pencil?


That's on the to-do list for v2.4a.



Cool. That opens up a completely new chapter for L3DT doesn't it? If you had a contour pencil, imagine the power if you could work over the existing contours in the Heightmap pane, basically fine tuning them.



You could introduce the abilty to cycle between the DM and heightmap within L3DT:import heightmap> generate DM> draw contours on heightmap> generate DM > alter the DM with the DM pencil > generate heightmap, etc.



Yup, that shouldn't be a problem.


Cool. It can already be done, but you need to export and reimport the heightmap. This is similar to the setup that Wilbur has in place. L3DT's current setup already has the possibility of having two 'pseudo' layers, because once you can interact with the heightmap you then have two distinct heightfields: DM and HM.
This is why I mentioned the combination options between DM and HM: Add, Replace, etc. In Wilbur you only have the combination options going one way, tess->heightmap. This translates in L3DT as DM->Heightmap. Would it be possible to eventualy provide blending options when going DM->HM only?
The options this workflow gives are such that you can design a design map, convert it to a heightfield, then go back into the design map, wipe everything and set to 0 alt and build yourself some mountains, convert to a heightmap using an add operation say, adding the mountains to the existing heightfield. It's the next best thing to having layers anyway. In Wilbur, the heightmap is visible and the height info is accessable from the tess. Having the contours visible over the DM would go some way towards this. I found that I only needed broad visual cues, rather than access to heghts at the pixel level.
I think long term all of the terrain apps are going to layers. Leveller 3.0 seeks to use PS layer technology, GeoControl has stole a march here. I still think there is a good argument for keeping some distinction between the DM and HM. I think Wilbur may have multiple tess layers eventually. Multiple DM layers would be extremely cool. Having some kind of design stage capability really extends map size. Compress->unpack, and can deal with the oft neglected macro-scales.




When altering large areas, if arbitrary selections are not possible, better visibilty for the pixels being altered by the DM pencil in the current pass: maybe set those red until another operation is preformed.



I like it. Added to the to-do list.


Lovely- running out of words for approbation :D



Coupled with the ability to draw/use contours is the real advantage of having a tool which can select from a heightmap by height range: ie select between 899-900m would give a selection on screen defining the 900m contour. With this selection still in place, it would be great to be able to draw a contour which developed this ‘proto-contour' to add more detail or make corrections, etc. For ME-DEM, this avoids the complication of having to import yet another layer: ie dem and contour info. Simply extract the contour info from the dem. The operation has to allow the simultaneous selection and contour or DM drawing though to be useful.



I'll think about this after I make the above changes to contour drawing.


Perhaps this is not as important for L3DT as you already have a contour generator. In fact a HM pencil would put the emphasis back with the full heightmap generation again, as that would provide the most reliable contours out of the two. Simply generate the full heightmap, then make contour alterations in the heightmap pane. I still think it is a useful tool though because you can use it for selecting areas, not just contours.

Cheers,
monks
monks
Oracle
 
Posts: 292
Joined: Tue Nov 22, 2005 10:38 pm
Location: Middle Earth

Postby monks » Mon Jul 03, 2006 9:05 pm

Hey Aaron- never noticed the Heightfield clip tool before- very nice. It's helped with what I'm doing. When did you add that?

//spoke too soon- :lol: not sure if it actually alters the terrain- runs off to read some docs.
// Yes it does, but it only allows it once- I think it's because you are only allowed 1 DM generation from the heightfield. I initially tried it after generating the DM from the HM.

monks
monks
Oracle
 
Posts: 292
Joined: Tue Nov 22, 2005 10:38 pm
Location: Middle Earth

Postby monks » Mon Jan 08, 2007 8:44 pm

I'm even more convinced now that contours are so damn useful. I have pretty much settled on a workflow for building these suckers.
I recently made a terrain at 50m res covering 204 Km. There is simply no way it could have been done with so much clarity and ease (even with Wilbur's somewhat clunky tessellation tool: it was never designed for contours). Yes, it could have been done with L3DT's DM, but with no where near the accuracy, and it could have been done with greyscale gradient maps in a paint prog but you have lots of limitations: not least the whole 8bit thing. Contours are LOTS easier to use and are very elegant.
Here's a link to a shot of the terrain:
http://www.me-dem.org/images/zoom/HQAJC ... rosion.png
That shot is *before* erosion (and perlins) have been applied.

What's more, I think L3DT's Design Map would be a perfect home for a contours feature. It sits so well with it. Are you planning on implementing these at some point Aaron?

I've not actually shelled out for L3DT yet :oops: I did ask you in an email for a link to the purchase point but you never replied to that part of the message.

monks
monks
Oracle
 
Posts: 292
Joined: Tue Nov 22, 2005 10:38 pm
Location: Middle Earth

Postby Aaron » Mon Jan 08, 2007 10:46 pm

Hi Monks,

I agree this would be pretty cool. The plugin API is sufficiently advanced that an arbitrarily skilled C++ programmer could write a plugin for contour interpolation. If however no-one wants to take this on I can add it to my to-do list, but the list is currently quite long, and before I get to an advanced/complex heightfield feature like this, I have to first do the basic heightfield editing stuff (mouse sculpt tool, etc.)

I've not actually shelled out for L3DT yet I did ask you in an email for a link to the purchase point but you never replied to that part of the message.


Oops, I'm very sorry Monks. The payment options (Kagi or PayPal) are available on the downloads page.

Cheers,
Aaron.
User avatar
Aaron
Site Admin
 
Posts: 3696
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia

Postby monks » Tue Jan 09, 2007 10:15 am

I agree this would be pretty cool. The plugin API is sufficiently advanced that an arbitrarily skilled C++ programmer could write a plugin for contour interpolation. If however no-one wants to take this on I can add it to my to-do list, but the list is currently quite long, and before I get to an advanced/complex heightfield feature like this, I have to first do the basic heightfield editing stuff (mouse sculpt tool, etc.)


Hi Aaron, yeah, you're definitely doing things in the right order imo: plugin architecture, multi -threading. I think multi-threading is way more important than some people think.
I'd be chuffed if you did take it on but I understand the backlog of work you have. I just think L3DT would especially benefit from it because it has an emphasis on large terrains: contours are even more useful for large terrains. It also has an explicit Design mode, blah, blah, blah.

Aw- so no freebie then? :D Keep up the great work dude, I'm happy to offer any support where/when I can.

btw, would running L3DT on a quaddie require a a multi-user license?

monks
monks
Oracle
 
Posts: 292
Joined: Tue Nov 22, 2005 10:38 pm
Location: Middle Earth

Postby Aaron » Thu Jan 11, 2007 7:15 am

Hi Monks,

I think multi-threading is way more important than some people think.


You're right; multi-threading is essential for any problem that can be parallelised, if only for the reason that most users have or will soon have multi-core computers. However, we should be careful to remember that 'multithreading = good' is not always true, and probably never will be. Some algorithms and procedures can't be split into parallel operations, and more still can be split but won't perform any better. Take loading/saving files; there's only one read head on the disk, so even if you multithread your file I/O code it won't run any faster. Another example is my water table flooding algorithm; it has several optimisations that will only work when the calculation is run block-at-a-time, and multithreading that calculation (if it is even possible), would then probably run slower than the single-thread version. That said, there are many cases in which multithreading is applicable. At the moment I've got multithreading working on the attributes map, normals, lighting, and texture calculations, and I can probably extend that to include erosion, alphas, and certain HF filters. I don't expect to multithread water table, heightfield inflation, terraces, plateaux, volcanos, mountains, and lake/sea flooding, but I may be able to get some of these going.

There are also other benefits in splitting calculations into threads, such as then being able to split the same calculations across different computers (render farms). I still have to plug-in some socket and management code for that, but we're getting there.

btw, would running L3DT on a quaddie require a a multi-user license?


No, I find those per-core or per-computer licenses a bit impersonal. Here at Bundysoft, all licenses are per-human.

Cheers,
Aaron.
User avatar
Aaron
Site Admin
 
Posts: 3696
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia


Return to Feature requests

Who is online

Users browsing this forum: No registered users and 4 guests

cron