Table of Contents
HF2 is a compressed heightfield format designed to replace uncompressed terrain formats such as HFF, TER and BT. In addition to smaller file sizes thanks to difference and gzip encoding, the HF2 format also includes an extended header section for storing additional metadata such as georeferencing coordinates, comments, and the like. Thus, HF2 was designed with extensibility in mind.
The compression in the basic HF2 files is difference encoding of lines. The HF2 format is seldom much smaller than the equivalent uncompressed data files. However, the HF2 files are efficiently compressed by the gzip algorithm, and so the HF2.GZ files (or HFZ, equivalently) may be significantly smaller than the uncompressed data (see compression rates). By convention, the HF2 file extension refers to files that are compressed by the difference method only, whereas the HFZ or HF2.GZ extensions identifiy files compressed with both difference and gzip. HFZ or HF2.GZ files may be decompressed to recover the diff-encoded HF2 file using standard archiving software (e.g. 7-zip).
An important feature of the HF2/HFZ format is that the vertical precision of the data may be user-defined (e.g. 1mm). This allows users to achieve very high accuracy (with large file sizes) or very small files (with lower accuracy). HF2 has a maximum vertical precision of 1/4294967296 of the height range (32 bits). For comparison, the precision of the nominally 'lossless' 16-bit TER/BT/HFF/PNG files is 1/65536 of the height range (16 bits). However, small variations may not be meaningful, and may therefore be discarded without data loss. For instance, a heightfield generated by the SRTM survey has an average error of 0.1-1.8m (depending on region)1), and so the SRTM data may be stored in a HF2 with a precision of 0.1m without causing the loss of meaningful data. Increasing the precision value in this way decreases the file size. Such variable precision and compression recommends the HF2 format over the uncompressed fixed bit-depth formats when greater accuracy or smaller file sizes are required.
The compression achieved by HF2/HFZ is a function of the randomness of the heightfield and the precision chosen by the user. The figures below represent the compression rates achieved for HF2/HFZ for a very noisy artificial heightfield generated by a diamond-square fractal algorithm (download as 32-bit HFF, 16-bit TER or 16-bit HF2.GZ). This heightfield had dimensions of 1024×1024 pixels, at 10m horizontal spacing, and a vertical range of 640m.
If you don't want to read all the numbers, the take-home messages are:
Some more technical notes:
The canonical implementation of the HF2/HFZ formats is the LibHFZ library, the source code of which is freely provided under the terms of the LGPL. LibHFZ is written in the C programming language.
The HF2 file contains a fixed-length header, an optional variable-length extended header section, and finally the height data stored as difference-encoded tiles. The HFZ is merely a gzip of a HF2 file.
The header of a HF2 file consists of 28 bytes of the following structure:
The extended header may contain additional application-specific data, stored in tagged blocks of variable length. The total length of the extended header is defined in the core header, described above.
Within the extended header, data blocks have the following byte structure:
If a program does not “understand” a block, it should either store the block and re-write upon save, or else warn the user that data may have been lost.
Commonly used extended header blocks are listed on the extended header block page.
Map data is also stored in square tiles, with a side-length given by 'tile size' in the header. The tile order is in rows from west-to-east, with successive rows going from south-to-north. The tile ordering for a 4×4 example is shown below:
Each tile has an 8-byte header of the following structure:
These values are used when reading a HF2 to calculate the floating-point altitude from the integer values stored in the file, as shall be described below. When writing a HF2 file, these values are calculated from the min/max altitude in each block and the user-defined vertical precision, using the following equations:
float IntRange = (max - min) / precis + 1; // number of integer height values required float VertScale = (max - min) / range; float VertOffset = min;
Following the tile header, the map data is stored in lines (rows) of pixels, again going west-to-east, with lines ordered south-to-north.
Each line has a 5-byte header of the following structure:
The line data is stored as signed character, short or long integers, depending on the line byte depth specified in the line header. The line does not contain the first value, which is already specified in the line header. Hence the number of values in the line is TileSize-1 (See caveat on map/tile sizes).
To calculate the second and subsequent pixel heights in the line, use the following equation:
long IntValue = FileValue + LastValue; float h = (float)IntValue * VertScale + VertOffset;
…where FileValue is the integer value read from the file, LastValue is the integer value from the preceeding pixel. In the case of the second pixel in the line, LastValue is the Start value read from the line header. To read successive pixels, store IntValue in LastValue and iterate with the new FileValue. Please refer to LibHFZ for an example implementation.
Height values are written by first converting the floating-point height values into integers using the scale and offset values from the tile header, as given in the formula below, and then using difference encoding to record the height change from pixel to pixel.
To calculate the height as an integer, use the following formula:
long HeightInt= (long)( (HeightFloat- VertOffset) / VertScale );
…where HeightInt is the output integer value, HeightFloat is the input height of the given map pixel as a floating point number, and VertOffset and VertScale are the scaling values for the current tile.
To write a line, you must first calculate the required bit depth for difference encoding. To do this, parse through the pixels in a line and use the equation above to calculate the HeightInt values. If the largest difference in integer values between adjacent pixels in the line is within the range of -128…127, the line may be encoded using a byte depth of 1. The maximum difference range for the supported byte depths are provided below:
In the line header, the scaled height value of the first pixel in the line is written in full 32-bit integer precision, as calculated by the above equation. Following the block header, the height values are written using difference encoding, where the delta between successive pixel heights is written to the file. The delta is calculated from taking the HeightInt value for a pixel (formula given above), and subtracting from this the HeightInt value from the previous pixel in the line.
The tile size used for internal tiling in HF2 does not, and will not in general, be a whole number divisor of the map size. Maps of size 513×513 can, for instance, use an internal tiling of 64, even though the edge tiles will themselves only contain one pixel. These tiles are written with truncated size; no additional empty values are inserted to 'fill out' the tile.
Please refer to LibHFZ for an example implementation.
Except where otherwise noted, content on this wiki is licensed under the following license:CC Attribution-Share Alike 3.0 Unported