News for August 2008

Hi All,

Yesterday I posted some benchmark results comparing the speed of texture generation in with the current release (v2.5c) against the forthcoming release (v2.6). Today, I'm going to compare the speed of lightmap generation.

The conditions for this test were the same as those used yesterday, except that instead of generating the texture map, I was generating the light map.

The graph below shows the performance of the previous release of L3DT Professional, version 2.5c, for both mosaic and non-mosaic light map generation.

As appalling as the benchmarks were yesterday for texture generation with L3DT 2.5c, these results for light-mapping are worse. As before, you can see that the mosaic map calculation in v2.5c is much slower than the non-mosaic calculation. What makes these results worse than yesterday's was that the mosaic light mapping algorithm in v2.5c threw errors and aborted when run with three or four active cores, and thus the three and four core mosaic results are missing. These errors in the mosaic light map system were related to the mosaic map cache manager, which as I described yesterday, was very poor at handling concurrent requests from multiple threads. This has been fixed in v2.6.

Now fore the results from the forthcoming release, version 2.6. The graph below shows the performance v2.6 for both mosaic and non-mosaic light map generation:

Here you can see that the light map performance scale up linearly with the number of cores (as it should), and that the mosaic map and non-mosaic light map calculations are now about the same speed (as they should be). So, whilst version 2.5c couldn't handle multi-core generation of mosaic light maps, version 2.6 has no such problems.

I apologise in advance if this is a little gratuitous, but as I did yesterday, I'm now going to directly compare the results for the old release (v2.5c) with the forthcoming release (v2.6).

First up is the comparison of non-mosaic light map generation:

There wasn't much wrong with the non-mosaic calculations in v2.5c, so the improvements here are fairly modest. These results show that version 2.6 (light green) is about 10% faster than v2.5c (dark green) for non-mosaic lgiht map generation. This improvement is partly due to a compiler upgrade (now using MSVC 2008), and partly due to some minor optimisations in the light mapping algorithm.

As with yesterday, the big news remains the improvements in mosaic light map generation:

The results show that for single-core mosaic light maps, version 2.6 is ~700% faster than v2.5c, and for dual-core, it's 1500% faster.

No direct comparison can be made for triple and quad-core calculations, because release 2.5c simply didn't work with more than two cores. However, if you take the fastest speed achieved with version 2.6 (which was for quad core) and compare it with the fastest working speed from v2.5c (which was for single core), you get a speed increase of two thousand four hundred percent.

If you like to make big maps, upgrading to L3DT v2.6 will save you a lot of time. It your computer happens to be multi-core, the time savings could be tremendous*.

The release date for L3DT 2.6 is mid-September.

* Your mileage may vary. The actual time savings depends on how you're using L3DT, your choice of settings, and how your computer is configured.

Hi All,

With L3DT release 2.6 only a few weeks away, I thought this might be a good time to run some head-to-head comparisons of the new release (v2.6) against the previous release (v2.5c). In particular, I'm going to show you just how far we have come in optimising the multi-core, mosaic-mapped calculations in L3DT Professional Edition.

Before we get to the numbers, I should describe how the tests were conducted. Basically, I generated a complete map (heightmap, lightmap, etc.), and then I ran the texture generation algorithm repeatedly with 1, 2, 3, and 4 cores enabled, using mosaic and non-mosaic texture generation.

The results will be presented in terms of the texture pixel throughput, measured in pixels per millisecond. Larger values are better, and imply faster texture generation.

All of the benchmark tests were conducted on the same map, with the following settings:

For the mosaic tests, the attributes, normal, light map and texture map were all mosaics, with a tile size of 512×512 pixels. For the non-mosaic tests, all these maps were in-RAM, with mosaic mapping disabled.

The benchmarking system was setup as follows:

The graph below shows the performance of the previous release of L3DT Professional, version 2.5c, for both mosaic and non-mosaic texture generation.

These results show two very bad things about the mosaic system in v2.5c:

1. The first problem was that the speed of the mosaic map calculation (blue bars) simply did not increase when more cores were added. This indicates that the the mosaic map cache manager was very inefficient at handling concurrent requests from multiple threads, and was making threads wait for one-another far too often.
2. The second problem was that even for a single-core calculation, the mosaic map calculation was only ~35% as fast as the non-mosaic (RAM only) texture calculation. This implies that there was a huge overhead in reading/writing pixels in a mosaic map, even when the tiles were already loaded into RAM.

If you described these results for the mosaic map system “appalling”, I would be inclined to agree. Fortunately, I'm very pleased to say that these two problems have now been solved in version 2.6, as shown by the next section.

The graph below shows the performance of the forthcoming release of L3DT Professional, version 2.6, for both mosaic and non-mosaic texture generation:

Here you can see that the mosaic map results (blue bars) scale up with the number of cores. In fact, the mosaic map calculation is now faster than the non-mosaic calculation (red bars), by about 10% for any number of cores (up to four, anyway). This huge improvement in the mosaic system was brought about by a smarter mosaic cache manager with far fewer thread locks, and by optimising the texture calculation to bypass some of the overheads in the mosaic system.

Now I'm going to directly compare the results for the old release (v2.5c) with the forthcoming release (v2.6).

First up is the comparison of non-mosaic texture generation:

Across the board, v2.6 (light green) is about 25% faster than v2.5c (dark green) for non-mosaic texture generation. Not bad.

However, the big news is the improvements in mosaic map texture generation:

Version 2.6 (light green) is monumentally faster than v2.5c (dark green) for mosaic texture generation, ranging from an impressive 275% improvement for single-core calculations, to an absurd 1100% improvement for quad core calculations (yes, one thousand and one hundred percent faster).

The mosaic map system in L3DT 2.6 is a colossal improvement over that of v2.5c, providing a speed-up of better than a thousand percent for texture generation on quad-core processors. Speaks for itself, really.

The release date for L3DT 2.6 is mid-September.

Disclaimer: These benchmark results apply only to the texture generation algorithm. You should not expect the heightfield, water-mapping or attributes map algorithms to improve by comparable amounts. Similar improvements may be occur with the normal map and light map calculations, as these algorithms are structured identically to the texture algorithm. However, this is not guaranteed, as the benchmark tests have not yet been conducted for those algorithms.

Hi All,

I spent a few hours tonight working on the texture generation algorithm. More specifically, I've been looking at reducing the 'overheads' from the mosaic system that make mosaic texture generation slower than generating non-mosaic textures. I'm pleased to say that I've had some good luck: In my latest test, the mosaic overheads were negligible. This means that mosaic textures will generate up to 50% faster than before*. This update will be included in the final release of L3DT version 2.6.

Time permitting, I will publish more rigorous benchmark figures in the next week or so, comparing new with old, and comparing mosaic with non-mosaic.

Cheerio,
Aaron.

* Disclaimer: this was only tested on one map, and your mileage may vary. Please wait for the benchmark figures before believing any of this.

Hi Everyone,

The beta release of L3DT version 2.6 is now available for download, in both Standard and Professional flavours. The expected release date for the final build of L3DT version 2.6 is nominally mid-September.

If you're not sure how to download updates, please see the updates FAQ.

The main changes from the previous stable release (v2.5c) are listed on the development plan, but I've included some of the highlights below:

There have been major changes to the multi-threaded calculations to fix stability issues with mosaic-map builds. I've now logged hundreds of hours of testing of L3DT's multi-threaded mosaic-map calculations on a quad-core Intel Q6600, and I'm completely satisfied that the cache bugs have all been fixed. For multi-core performance benchmarks, please see this page.

All versions of L3DT now include the Atlas2 exporter plugin, which supports both blended and unique textured Atlas generation. The exporter is now accessible using the 'Atlas' button on the toolbar. Please refer to the TGEA tutorial for more info.

There are lots of new goodies for Spring mappers in the new Professional Edition, including:

• Import and export support for SD7/SMF/SMT, and export-only support for SMD.
• A new Spring Mapping wizard, which organises and automates the process of making maps for Spring (see 'Spring' button on toolbar). The tutorial for using this new wizard is currently being re-written to refelect these changes.
• Countless bug-fixes for the SMD/SMF export plugins.

The required changes have been made to allow L3DT to run properly under Vista's User Access Control (UAC) system, including moving volatile settings from protected folders and registry keys to non-protected ones (i.e. HKLM→HKCU, PROGRAMFILES→APPDATA, etc.) Extensive testing has been performed under 32- and 64-bit versions of Vista Business and Enterprise, and no further issues were identified.

Changes have also been made to improve support for L3DT in Linux using the WINE compatibility layer, including the disabling by default of a plugin that is not available in non-NT operating systems (ZeoPerfMonNT). I and others have successfully installed and run L3DT Standard and Professional in Ubuntu 8.04. However, some users have reported (easily fixable) problems with file paths under Ubuntu 8.04 and Slackware 12.1, as as seen in this forum thread. For more information, please refer to the L3DT in Linux page.

• Added brushes for painting the texture and attributes maps, for both L3DT (2D) and Sapphire (3D) windows.
• Added 'leveller' and 'level at' heightfield brushes to Sapphire, for building platforms, walls, etc. See here for pics.
• Made the mouse scrollwheel change the brush radius in L3DT, as it already did in Sapphire.

• Added 'tabbed' map view, for switching maps in main window. See here for screenshots.
• Added toolbar buttons for image drape, TGEA export, Spring wizard.
• Added user interface for georefererencing settings. See here for more info.
• Added 'heightfield size' wizard, and revised 'design map size' wizard.
• Added 'spin' (up/down) buttons to many dialog boxes, to make it easier to increase/decrease values.
• Added 'Edit→Screen capture→to clipboard' and 'Edit→Screen capture→to file' options to menus in both L3DT and Sapphire.

Please refer to the development plan for the full list of changes.

To the best of my knowledge and testing, this beta release is more stable, usable and bug-free than the previous stable release of L3DT 2.5c. However, I can't be sure of that until more users have downloaded it and tried it out. Thus, I would greatly appreciate it if you would do so, and report any bugs that you find. I would also appreciate any feedback that you'd care to give, either good or bad.

Best regards, Aaron.

Hi Everyone,

In case you were wondering, I have indeed been rather busy with new developments for the upcoming L3DT release 2.6. The latest developmental build (L3DT 2.5.4.14, 3rd of Aug 2008) includes several such improvements, which I'll be describing and documenting over the coming days and weeks. The first cab off the rank - regarding features to be described - is the new tabbed map view. Put simply, you can now switch between maps in the main window using the new tab control that is above the map display:

The new map tab control, highlighted by the red rectangle.

At the moment the tab control uses the ancient 'button' style tabs. I will update this to fancy XP-style tabs as soon as I can work out how to prevent the screen from flickering when changing tabs using the fancy tab style.

Cheerio, Aaron.