Heightfield step artefacts

From time to time, users encounter ugly ‘steps’ in their heightfields when rendered in 3D, such as those shown below. This page explains what causes step artefacts, what you can do to avoid creating them, and what you can do to fix them once they’re there.

Terrain with step artefacts	What it should've looked like

These steps are usually the result of quantisation caused by using low bit-depths to store the height data, such as using 8-bit file formats. I’ll explain more about programs and image formats in a moment, but for now it suffices to say that many programs and file formats store image and heightfield data using 8-bit integers for each pixel. These 8-bit integers can store 2⁸ (or 256) different values, from 0 to 255 in whole-number steps. Integers don’t store fractional values; instead they are truncated to the lower whole-number value (i.e. 12.8 becomes 12).

To demonstrate the effect, somewhat exaggerated, I have plotted below a slope showing the steps caused by severe quantisation from a hypothetical 2-bit file format. The ‘raw’ slope is shown in green, and the 2-bit integer slope with steps is shown in red. Two bits can store 4 values (2² = 4), so you see four distinct levels.

For every extra bit we use to encode the height values, we double the number of possible levels, which increases the number of steps but decreases their height. Below I have plotted the same slope showing the effect of increasing the bit depth from 3 bits (8 levels) to 6 bits (64 levels).

Clearly, the more bits you use, the ‘smoother’ the line becomes, and the less noticeable the steps become. This brings us to the obvious question…

Let’s have a look, shall we? Below I have plotted the same heightmap in 3D after it had been encoded with different levels of quantisation from 8 bit (256 levels) to 12 bit (4096 levels).

8-bit (256 levels)	9-bit (512 levels)
10-bit (1024 levels)	12-bit (4096 levels)

From this analysis, it would seem that a bit depth of about 12 is required to avoid causing noticeable star step artefacts. For very sensible architectural reasons, programs and file formats usually work in multiples of 8 bits, so this means you should make sure your height data is stored with 16-bit depth or greater.

The following file formats store data with 8-bit colour depth, and will always cause stair step artefacts when used to store height data:

• JPEG ¹⁾
• BMP
• TGA

Furthermore, the following image formats can cause step artefacts when saved in certain modes:

• PNG in 8-bit, or 24/32-bit true colour modes.
• TIFF in 8-bit, or 24/32-bit true colour modes.
• PGM in 8-bit mode.
• RAW in 8-bit mode.

If you use these formats in any part of your heightfield tool chain, you will introduce step artefacts to the height data, which cannot be removed by simply expanding the height data back out to 16 or 32 bits.

You may think that it’s OK to store height data in true colour images like 24/32-bit BMPs or JPEGs because they’re using more than 8-bits, right? Sorry, it’s not that easy. True colour images use 8 bits to store each of the red, green, blue values at each pixel. By convention, height data is stored with the same red, green and blue values in any given pixel, which makes the image appear greyscale. This means that, rather than storing your height data in glorious 24- or 32-bit precision, you’re actually just storing three or four copies of the same 8-bit precision data, which will of course have step artefacts.

In short, 24- or 32-bit true colour images have the same heightfield precision as 8-bit images, and will still cause stair stepping artefacts.

The following image file formats are generally safe and will not cause stair step artefacts:

• PNG, in 16-bit greyscale mode.
• PGM, in 16-bit greyscale mode.
• RAW, in 16-bit integer or 32-bit float modes.
• TIFF, in 32-bit GeoTIFF mode.
• G16

Additionally, the following common terrain formats are also safe from step artefacts:

• TER (Terragen or Torque)
• HFF, HF2/HFZ ²⁾
• BT

Mesh formats like OBJ, X, 3DS, DAE, etc typically use 16 or 32-bit precision, so are also safe from step artefacts.

Step artefacts aren’t just caused by using low bit-depth file formats; they’re also caused by programs that store the image data in memory in 8-bit precision. Programs that store height data in 8-bit precision are usually image editors like MS Paint, Paint.NET, the GIMP, and Adobe PhotoShop. Whilst some of these programs do have 16-bit image modes (i.e. PhotoShop), this is not the default mode, and in some cases common image tools and filters won’t work on 16-bit image data.

Users should also note that even if a program loads the data from a 16-bit file, and saves it back to a 16-bit file, it can still cause step artefacts in the data if it has loaded the data into memory as 8-bits greyscale or 24/32-bit true colour data.

Consequently, users are recommended to avoid loading or saving height data in image editor programs if at all possible.

Generally speaking, any program that was purposefully designed to handle heightfields will store the data in 16- or 32-bit (greyscale, not true colour) precision, and will therefore be safe from step artefacts. L3DT uses 32-bit floating point numbers to represent height data, and this is quite common for similarly purposed applications like World-Machine, Grome, Leveller, GeoControl, Wilbur, VTP Enviro/Builder, etc.

Nope. The memory address bit depth used by a program or operating system (i.e. 32- or 64-bit) has absolutely nothing to do with the bit depth that the program uses to store height data. Old 16-bit programs could use 32-bit depth for images, and new 64-bit programs can still use 8-bit depth.

If I put a steam roller on my sofa, and then remove the steam roller, my sofa will still be flattened afterwards. The same is true with quantisation errors. If you crunch your data down to 8 bits, and then expand it out to 16 bits, it will still show the effects of quantisation from the lower bit depth. As an example, below I show some images that I’ve crunched down to various low bit depths before saving as 16-bit PNGs. Note that even though the data is all 16-bit in the images, you can still see the artefacts caused by the lowest bit depth used in the history of the data:

All images below are 16 bit greyscale
Was once 2 bit (4 levels)	Was once 3 bit (8 levels)	Was once 4 bit (16 levels)

So, users should be mindful that the step artefacts may not be caused by, or fixed by, the final file format or bit depth used to store the height data. Step artefacts in the output could have been introduced at any stage of the generation pipeline.

Since you’re reading this page, it is reasonable to assume that you have some height data that contains step artefacts, and you’d like them removed. As I’ve explained above, simply re-saving the height data in a higher bit depth file like a 16-bit image won’t do any good by itself. What you need to do is somehow filter the data to turn this…

…back into this…

Some approaches you can take to remove the steps are listed below.

The ‘ideal’ way to fix steppy height data may be to use some sort of edge detection followed by contour interpolation. Unfortunately, I don’t know of a program that will do this for you. If you do know of such a program, please let us know in this forum thread.

An easy approach to removing step artefacts is to apply a smoothing filter to your height data. You can do this in L3DT using the ‘Operations→Heightfield→Smooth’ menu option. However, users should note that to fully remove step artefacts you do have to do rather a lot of smoothing, and this will make your terrain look unnaturally smooth. Smooth is better than stepped, perhaps, but it’s still not ideal.

Below I have included before-and-after screenshots to show the effect of smoothing on a heightmap that contains 8-bit step artefacts.

Original	8-bit steps	After smoothing

It should go without saying that smoothing a heightmap image in an image editor will probably not remove the step artefacts, because the image editor is probably storing the height data in 8-bit precision. To perform smoothing, you need to use a program that represents the data in 16- or 32-bit depth precision, such as L3DT.

One approach that has helped some users is to load the steppy heightmap into L3DT as a design map layer (see user guide), and then re-generate the heightfield from the design map. This process will produce a heightfield with the same basic shape and characteristics as your original, but will include new details. Before and after shots are provided below:

Original	8-bit steps	Re-generated

If you find any inaccuracies or contentious points in this guide, or would like to suggest improvements, please feel free to recommend changes in this forum thread.