Huge maps in Sapphire

Hi All,

There are some nice improvements to the Sapphire 3D renderer in the latest developmental build of L3DT Professional (v12.03 build 2), which incidentally is available on the downloads page now. These improvements include faster startup, higher frame rates, lower memory usage, and larger maximum map sizes. For more information, read on…

Sapphire has, from its inception, had a bug that effectively limited the maximum heightfield size that could be displayed on most systems to 32k × 32k pixels or smaller. For large maps Sapphire would exhaust memory during startup when allocating the array of ‘patches’, which are objects that each represent an area of the heightfield equal to 64×64 pixels (typically)¹⁾. These patches were not particularly huge objects; each was 4432 bytes, which included space for variance trees, plus patch coordinates, rendering mode info, flags, etc. However, these smallish objects still added up to a very large amount of memory on ‘big’ maps. For example, with a 32k × 32k pixel heightfield, there are 512×512 patches (or 262144 in total), which add up to 1.1GB in memory. That figure accounts for ~30% of the address space of L3DT v11.11, or a whopping ~65% for versions 11.08 and earlier (which weren’t large address aware). This huge patch array didn’t leave a lot of leftover space for the actual height data, the terrain mesh geometry, the textures, resources, and what not. Clearly, that design was always going to hit problems with memory exhaustion, and so it did.

Obviously, the size of the patch objects had to be reduced to fix this problem. Discarding some unnecessary flags and settings brought the patch size down to 4208 bytes. Of those remaining 4208 bytes, 4096 are occupied by the variance tree (i.e. 97%). Alas, simply reducing the size of the variance tree would result in more ‘brute force’ triangles being rendered, which would slow the frame rate. Instead, the new version only allocates the variance tree when the patch is visible, which means for the vast majority of patches that are outside the view distance, the memory occupied is only 112 bytes per patch. This makes for a massive reduction in memory usage on large maps.

In addition to size optimisations, there have also been speed optimisations. Most significantly, the heightfield tile objects now use the direct access interface to bypass the overheads of the map functions in the Zeolite plugin API. This resulted in a dramatic speed increase for both the startup tile and the frame rate.

The net effect of these changes were:

• Faster startup. The start times were reduced by 85%.
• Higher frame rates. The frame rate increased by ~50%.
• Lower memory. The application’s memory consumption during 3D rendering was reduced by ~92%²⁾.
• Larger maps. Sapphire can now render heightfields up to L3DT’s current maximum heightfield size of 128k × 128k pixels.

As far as I can divine, there are no additional restrictions imposed by these changes, nor any other downsides. If you do find any problems, or would otherwise like to post questions or feedback, please use this forum thread.

Anyway, that is all for today.

Cheerio, Aaron.