L3DT documentation
Large 3D terrain generator

Mosaic maps and dynamic caching

L3DT allows the creation and manipulation of very large maps (up to a theoretical 4 billion gigapixels) using mosaic maps. A mosaic map is essentially an array of sub-map tiles that are wrapped-up to behave like one coherent map. These tiles may be loaded and saved independently, allowing the mosaic map to minimise RAM-load by saving unused parts of the map to the hard-disk. This 'paging' is managed automatically using a schedule based on the pixel-access histories of the map tiles1). The net result is that a monster-sized map can be generated with only minimal RAM-usage (<100Mb).

An example of a mosaic map, taken from the alpha maps tutorial, is shown below. This map is a 4096×2048 pixel bitmap image that was split into 8×4 tiles of 5122 pixels. It has been resized to 642 pixel tiles for display here.

How big can a mosaic map be?

Theoretical limit

The theoretical map size limit for the mosaic system is 262 pixels, or 4 billion gigapixels, or (just to be silly) 2,147,483,648 × 2,147,483,648 pixels. This limit is imposed by the use of signed 32-bit integers for x and y coordinates.

Practical limit

A practical limit has been set to 16 terapixels (4 million × 4 million). This is to restrict the mosaic state table to a size of 8192 × 8192 tiles, as this will itself start to hog-up memory on larger maps. This may be increased in the future, if required.

Current record-holder

The largest mosaic map made to date was 256k × 128k (32 gigapixels), which is large enough to prove that there are no 32-bit index limitations in the code. Perhaps there is another unexpected limitation that sits between 16 billion and 16 trillion pixels, but I certainly don't have a big enough disk-drive to find out. Please feel free to try.

Performance

As shown in these benchmark results (and these), there is a negligible calculation overhead for using mosaic maps compared to in-RAM maps for well-behaved calculations (e.g. normals, light, texture map, etc). However, for some calculations with unpredictable access operations (e.g. fractal, erosion, shadow casting), the slow-down due to the extra HDD/RAM caching may be noticeable.

File names and tile ordering

Each mosaic map includes a mosaic master file, which is an ASCII file that describes the size and number of map files within the mosaic. Details of the MMF format may be found on the file formats page.

The file-names for map tiles in a mosaic are given by the name of the mosaic master file (sans .mmf extension), appended with the tile location string (something like “_x0y5”) followed by a dot and the file extension. The tile location string is determined by the x and y coordinates of the tile.

In L3DT, the origin (x = 0, y = 0) is in the bottom-left corner. Thus x increases from left to right, and y increases from bottom to top. A simple 4×4 example is shown below:

x0y3 x1y3 x2y3 x3y3
x0y2 x1y2 x2y2 x3y2
x0y1 x1y1 x2y1 x3y1
x0y0 x1y0 x2y0 x3y0
1) You can monitor the mosaic map allocated/paged memory using the Ram chart window.
 
l3dt/algorithms/mosaics.txt · Last modified: 2017/08/31 05:56 (external edit)
 
Except where otherwise noted, content on this wiki is licensed under the following license:CC Attribution-Share Alike 3.0 Unported
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki