News for June 2007

Hi All,

I've cobbled together the beginnings of the Spring import/export plugin for L3DT. Below are two screenshots of its work; the first is a Spring map exported directly from L3DT, and the second is a Spring map ('Small Divide') that I've imported into L3DT via the plugin, and rendered in Sapphire.

L3DT map in Spring!
Spring map in L3DT!

At this stage, the plugin can read and write SMT and SMF files. However it doesn't support SMD yet (you have to write these manually), and nor does it properly support features or vegetation. Furthermore, the main window display needs to be modified to allow the use of indexed colours when editing the vegetation/type maps.

Anyhoo, I should have a dev build ready in the next few days.

Cheers, Aaron.

Hi All,

Since release 2.1 back in 2003, L3DT has been able to generate so-called 'Megatextures'. These textures are rather large (32k x 32k pixels or greater) and rather detailed (32x texture to heightfield res or greater), and require both dynamic paging and texture mip/clip-mapping to render on current hardware. Up until now, however, we L3DT users haven't had a 3D renderer that could efficiently handle that much texture data. L3DTVi2, the 'old' renderer for L3DT, handled the paging part but not the mipmapping. This meant that whilst it could render large textures (32k x 32k were OK), it choked on the highly detailed textures (e.g. 32x res) because it ran out of texture memory. Sapphire, the 'new' renderer for L3DT, now supports both dynamic paging and mipmapping, which means that it can now render megatextures with framerates comparable to our normal 'tiny' maps.

Here are some screenshots of Sapphire rendering a 1024×1024 heightfield with a 32k x 32k pixel (3GB) texture.

Firstly, and aerial shot (~20fps)
Now, a bit lower down at the same corner (~5fps)
Standing on the rock in the middle of the image above, looking to the left (~5fps)
Moved to the left, looking to the right (~20fps)
Back at the corner, but with camera zoomed in 8x (<1fps)
Zooming in further (still sharp, but «1fps )
A random shot from the opposite corner of the map (~20fps)

If you're still not sure just how detailed a megatexture can be, consider this 2048x2048 pixel texture tile, which covers a single 64×64 pixel tile of the heightfield. That's a lot of detail, and I'm pretty happy that I managed to render all this on an elderly 256Mb 128MB¹⁾ graphics card (did I mention the texture was 3GB uncompressed?).

Note also that I'm not yet using any shaders here, and nor am I cheating by using a detail map. This is all fixed-function pipeline OpenGL 1.2.

Now, you may have noticed that the frame-rates of the zoomed-in images above are just dreadful (<1 fps). What has happened here is that I've zoomed in to the point where I've run out of video memory. Clearly I have to further optimise the texture caching routine to prevent this from occurring, but I have some ideas on how to do this. I hope to have a dev build ready sometime next week, so do stay tuned.

Cheerio, Aaron.

Update: I've fixed the texture caching problem, and uploaded an updated version of Sapphire and L3DT that includes megatexture rendering support. It's on the pro downloads page under 'developmental builds' as L3DT 2.5.1.10, dated the 24th of June. Note also that there are some tips and rules for rendering megatextures in Sapphire, which are explained over here. — aaron 2007/06/24 02:22

Hi All,

I've added mipmapped texture rendering to Sapphire to speed-up rendering of extremely large texture maps. I'll post some screenshots tomorrow, and explain a little bit more about the technology and benefits thereof. Anyway, to enable efficient mipmap texture rendering, I have had to modify the mipmap generation scheme in L3DT. Previously, mipmaps were generated at 4x resolution steps, meaning the 1st mipmap was 1/4th the resolution of the full map, the second was 1/16th, the third 1/64th, etc. This was fine for the 2D renderer in L3DT, and didn't use up much disk space. However, for efficient 3D texture rendering I've had to change to 2x resolution (1/2, 1/4, 1/8, etc). The upside is that Sapphire will now render megatextures efficiently, but the down-side is that L3DT will now take a bit longer to generate mipmaps, and that more space on disk will be taken up by those mipmaps (~1/3 the total map size). Given that the mipmaps now let me fly around over a terrain with a 3 gigabyte texture using only a 256Mb graphics card, I consider a little extra storage space to be a pretty good trade-off.

Also, the next release of the Zeolite plugin API will include functions to manipulate these mipmaps, just in case clever plugin developers can find another clever use for them.

Cheers, Aaron.

Hi All,

I've just started work on the 'undo' plugin for L3DT, which is planned to be one of the key features in release 2.5b. In the first instance, L3DT's undo system will work a little bit like the Windows 'system restore' feature, insofar as you'll be able to save/load restore points for whole projects, or maybe even specific maps in a project. Using 'undo' in the menu or hitting CTRL+Z will revert to the previous restore point, and there will probably also be a history listbox (like in Paint.NET) that allows you to revert further back to specific restore points.

However, what I suspect everyone wants is the nice 'undo' behaviour of applications such as word processors and spreadsheets, where you don't have to manually save restore points. For this to happen, each calculation/tool will have to automatically save a mini restore point before it modifies a map. That's a non-trivial amount of work, and it's going to be particularly tricky with some of the brush tools (not impossible, just tricky). Another consideration is ensuring your hard-disk drive isn't eaten by all the restore points. For this there will need to be a limit on the maximum number of undos permitted (easy), and/or the maximum disk space allocated to undo (harder). The undo history cache will also need to be deleted whenever the project is saved or closed, just like it is in MS Office.

Finally, once all that work is done, it should be possible to add a crash-recovery feature to pick up the pieces when it all goes to pot (i.e. crash of L3DT/Windows/hardware/power grid/etc).

Anyway, work is in progress, and I'll let y'all know when there's something ready for testing.

Cheers, Aaron.

Update:

I've just uploaded a dev build of L3DT Pro (2.5a build 7) that includes the beginnings of the undo system for L3DT. In this version, you can backup the active map or areas thereof using the 'Extensions→L3DTio_Backup→Backup active map' and 'Extensions→L3DTio_Backup→Backup selected area' menu options. If you wish to revert to the last backups, use the 'Extensions→L3DTio_Backup→Restore to last backup'. The trick now, of course, will be to make L3DT do the backups automatically. aaron 2007/06/14 05:20

Hi All,

Last week I was messing around once more with 'megatextures'²⁾ to see how Sapphire performed when rendering such vast quantities of texture data (just fine, as it happened). When reloading the map between builds, it struck me that L3DT was being very inefficient with its use of mipmaps. Mipmaps, for those who don't know, are low-resolution copies of a map, which are often used in level-of-detail algorithms to reduce memory usage and/or accelerate rendering. When viewing a really large map (say, 32k x 32k), L3DT knows that it doesn't need to display the map at full resolution, because you don't have that many pixels on the screen. Instead, L3DT will generate one or more mipmaps, typically 1/4th, 1/16th (etc.) resolution, and render the appropriate mipmap instead. With this trick, a 32k x 32k map can be rendered in the same time it takes to show a 2k x 2k map.

However, mipmap generation adds an overhead to the first time the map is drawn, which can be several minutes for huge maps. The inefficient part about all of this is that L3DT wasn't saving the mipmaps with the project, which meant that when you close and reload the project, L3DT had to regenerate all the mipmaps again. That has been fixed, so in future releases you will enjoy significantly faster load times for really large mosaic maps.

Cheers, Aaron.

Hello all,

On the downloads page you will now find L3DT release 2.5a. This update includes a number of improvements, such as multi-core support, anti-aliased texture blending, an updated 3D renderer, better debugging features, and an assortment of bugfixes. Additionally, TGEA users may be interested in the updated L3DT_Atlas plugin, which fixes the crashing bug that was affecting some users (kudos to Ben for the fix!). This updated plugin is also available from the Pro downloads page.

The full change log is over here.

Best regards,
Aaron.

Hi All,

With some help from a big red book, the Sapphire rendering plugin now supports mosaic terrain textures. This was the last remaining feature that needed to be added to Sapphire to bring it to parity with L3DTVi2. Consequently, following the next release (v2.5a, this Friday), Sapphire will replace L3DTVi2 as the 'official' renderer for L3DT. L3DTVi2 will still be available on the 3rd party downloads page, but it won't be included in the L3DT installation package after L3DT v2.5a.

The main benefit of this change is an improvement in stability, as L3DTVi2 tended to crash when unsupported formats were encountered (L3DTVi2 only supported 16-bit HFF). Sapphire does all file I/O through the plugin API, and therefore supports every file format that L3DT does. This also means that L3DT can now 'safely' use formats other than HFF without having to worry about crashing L3DTVi2. Thus, as of L3DT v2.5b the default heightfield format will be the new compressed heightfield format 'HFZ'. This format should save users somewhere between 30 and 60 percent of the space required to save the heightfield, relative to the uncompressed HFF/TER/BT formats.

This change also allows me to dispense with some hacks in L3DT that were required to maintain legacy support for L3DTVi2. These hacks include:

• The saving of the 'map group file' (.mgf), which was deprecated last year.
• The “tx” and “lm” map types in the mosaic master files, which should both read “24bit”.

If you wish to continue using L3DTVi2 after L3DT v2.5a, you will need to re-enable these hacks by selecting the 'Settings→Local settings' menu option, then double-clicking on the 'File→Legacy→MosaicTypeHack' and 'File→Legacy→SaveMGF' entries to enable those flags.

If you'd like to familiarise yourself with the newly updated Sapphire renderer, please download the latest developmental build of L3DT. If you have any problems, please let me know, and I will attempt to fix them forthwith.

Best regards, Aaron.

Edit:

I ran a few more tests and noticed that the scene often freezes when Sapphire has to load new texture tiles. This is particularly noticeable when you fly around on terrain with high-res mosaic textures. Ideally I should have the texture loader in another thread (and one day I will), but in the mean-time I'll limit the impact of this problem my adding clipmap support for textures in Sapphire. The 2D display in L3DT already uses clipmaps to display large maps, so this shouldn't take too long to get going in Sapphire.