Please see The HDF Group's new Support Portal for the latest information.
NASA Data Product Examples
| HDF4 File | Map File | Note |
|---|---|---|
| AIRS L3 | AIRS L3 | SDS with gzip compression |
| AMSR_E L2 | AMSR_E L2 | Vdata |
| AVHRR #1 | AVHRR #1 | SDS with chunking, compression, and dimensions |
| AVHRR #2 | AVHRR #2 | Palette and Raster |
Additional NASA File Examples
h4mapwriter was run with -f -m -t to produce these maps.| HDF4 File | Map File | Notes |
|---|---|---|
| CT01.hdf | CT01.map.xml |
Original filename: AIRS.2010.07.29.091.L2.RetSup.v5.2.2.0.G10211121151.hdf
This file has many objects of different types and sizes: (1) File Attributes that can be combined (2) File Attributes with many null characters, then a non-null at the end (3) SDSes with up to six dimensions (4) SDSes with named Dimensions (5) Vdatas (6) Vgroups h4mapwriter release 1.0.0 did not handle unsigned integers correctly in this file. The problem was fixed in release 1.0.1. |
| CT02.hdf | CT02.map.xml |
Original filename: MYD14.A2007273.0255.005.2007275112449.hdf
This file has several SDS objects that do not have data values, such as one named FP_line. |
| CT03.hdf | CT03.map.xml |
Original filename: MISR_ELLIPSOID_GM_P017_O036105_DF_F03_0024_GOM_b64-72.hdf
This file has SDSes and Vdatas that have Fill Values. |
| CT04.hdf | CT04.map.xml |
Original filename: olslit1995.aug_digital_12.hdf
This file has two Vdatas, /Strip/strip and /Strip/Description/coverage, that have no data. |
| CT05.hdf | CT05.map.xml |
Original filename: S1999001.L3m_DAY_CHL_chlor_a_9km.hdf
This file has a Palette with an Annotation but no image. |
| CT06.hdf | CT06.map.xml |
Original filename: sd2001003.hdf
This file has SDSes that contain only Fill Values. Also has Rasters & Palettes. Starting with h4mapwriter release 1.0.2, value(s) for verification are not included for arrayData elements that have only fill values. |
| CT07.hdf | CT07.map.xml |
Original filename: TRMM_LIS_SC.04.1_1998.002.00570.hdf
This file has 2 palettes, only one of which is associated with a Raster. The Raster uses RLE compression. There are also Tables where there are multiple entries per column (for some columns). |
| CT08.hdf | CT08.map.xml |
Original filename: MCD43B2.A2007257.h35v10.005.2007277185619.hdf
This file includes four SDSes with three different datatypes (uint8, uint16, and uint32); all have associated fill values. h4mapwriter releases prior to 1.0.1 did not format the uint32 values correctly in the Map. |
| CT09.hdf | CT09.map.xml |
Original filename: O1996307.L3m_DAY_L565
This OCTS Level 3 file has a palette with tag 201 and a palette with tag 301 that have the same ref numbers and different data elements (the data for the two palettes is at different offsets in the HDF4 file.) h4mapwriter releases prior to 1.0.4 did not map this file correctly because the HDF4 library prior to release 4.2.8 essentially hid the second data element from view, expecting that a DD with tag 201 and a DD with tag 301 with the same ref would have the same data element. |
Simple HDF4 Examples
| HDF4 File | Map File | Note |
|---|---|---|
| File Attribute | File Attribute | Various file attributes including annotations |
| Raster | Raster | Raster with RLE compreesion and attribute |
| SDS | SDS | Single contiguous storage |
| Vdata | Vdata | Vdata with 20 rows and 4 columns |
| Vgroup | Vgroup | Vgroup with long name |
Additional HDF4 File Examples
These HDF4 files were created as part of the Mapping project to test the capabilities of the HDF4 File Content Map Schema and of the HDF4 Map Writer. Several of the files cannot be mapped by h4mapwriter. Some can be partially mapped when the -c option is used.| h4mapwriter Options | HDF4 File | External File | h4mapwriter Map File | Hand-edited Map File | Notes |
|---|---|---|---|---|---|
| -f -m -t | T00.hdf | n/a | T00.map.xml | n/a | The file has 2 Groups, 1 Array, and 2 Dimension Scales. The Array is a member of both Groups. |
| -c -m -t | T01.hdf | n/a | T01.map-c.xml | T01.handmap.xml | The file has 2 Groups, 1 Array, and 2 Dimension Scales. The Array is a member of both Groups. The data in the Array and in the Dimension Scales is all stored in Little-Endian format. h4mapwriter will warn that little-endian data can't be mapped and no map file will be produced. When run with the -c option, h4mapwriter will map this file, but will still produce warnings because little-endian data has not been thoroughly tested. |
| -f -m -t | T02.hdf | n/a | T02.map.xml | n/a | The file has 2 Arrays. The first Array has no cells populated - all use the fill value. The second Array has cells populated except for first row and first column - those use fill value. |
| -c -f -m -t | T03.hdf | n/a | T03.map-c.xml | T03.handmap.xml | The file has 2 Arrays. The first Array has no cells populated - all use the fill value. The second Array has cells populated except for first row and first column - those use fill value. All data is stored Little-Endian (fill values and arrays) h4mapwriter will warn that little-endian data can't be mapped and no map file will be produced. When run with the -c option, h4mapwriter will map this file, but will still produce warnings because little-endian data has not been thoroughly tested. |
| -f -m -t | T04.hdf | n/a | T04.map.xml | n/a | The file has one DFR8-created raster in 2 groups. Raster has associated palette; no compression. |
| -f -m -t | T05.hdf | n/a | T05.map.xml | n/a | The file has two Rasters created with the GR interface. The first Raster has no cells populated - all use the fill value. The second Raster has cells populated except for first row and first column - those use fill value. Neither Raster has an associated Palette. Parameters for both Rasters: No compression; Pixel interlace; Number color planes (components) = 1; UINT8 datatype |
| -f -m -t | T06.hdf | n/a | T06.map.xml | n/a |
The file has a Raster created with the GR interface.
Raster has an attribute and uses RLE compression.
Raster does not have an associated Palette.
Parameters for Raster: RLE compression; Pixel interlace; Number color planes (components) = 1; UINT8 datatype
Note that while RLE compression is used, the data in each cell of the raster is unique, so no compression will occur and the number of bytes needed to store the data is more that if no compression were used. |
| -c -f -m -t | T07.hdf | n/a | T07.map-c.xml | Szip compression not supported in current version of Schema, so no complete HDF4 File Content Map for this HDF4 file. | The file has a Raster created with the GR interface. Raster uses SZIP compression. Raster does not have an associated Palette. Parameters for Raster: SZIP compression; Pixel interlace; Number color planes (components) = 1; UINT8 datatype When run with the -c option, h4mapwriter will produce a map file, but it won't have any Raster elements as the only Raster uses szip compression which is currently not supported by the writer. |
| -f -m -t | T08.hdf | n/a | T08.map.xml | n/a | The file has a Raster created with the GR interface. Raster uses RLE compression. Raster has an associated Palette. Parameters for Raster: RLE compression; Pixel interlace; Number color planes (components) = 1; UINT8 datatype. Parameters for Palette: Pixel interlace; Number color planes (components) = 1; Number entries = 256; UINT8 datatype. The cell values on the left half of the image should lend themselves to RLE compression. The palette has been tuned to the values in the image so that variations can be readily noticed. |
| -f -m -t | T09.hdf | n/a | T09.map.xml | n/a | The file has a single vdata (Table) in 2 vgroups (Groups). |
| -c -f -m -t | T10.hdf | n/a | T10.map-c.xml | T10.handmap.xml | This file has a variety of File (Global) Attributes and Annotations. SD-created Attributes have both Character and Integer types. One uses little-endian storage for the Integers. The Attributes have similar names. Some have embedded and/or trailing nulls. One has "&" character in the middle of the attribute value. GR-created Attributes all have Characters types. They have similar names. One has trailing nulls. One has "&" character in the middle of the attribute value. There is one File Label Annotation and one File Description Annotation. h4mapwriter will warn that little-endian data can't be mapped and no map file will be produced. When run with the -c option, h4mapwriter will map this file, but will still produce warnings because little-endian data has not been thoroughly tested. |
| -f -m -t | T11.hdf | n/a | T11.map.xml | n/a | The file has 8 vdata (Tables). All use big-endian data representation and FULL_INTERLACE (row-order) storage. One table uses linked storage. Tables vary in number of rows, columns, entries per column, and types of data. Last table has an Attribute, and one column in the last table has an attribute. |
| -c -f -m -t | T12.hdf | n/a | T12.map-c.xml | T12.handmap.xml | This file has 2 vdata (Tables). The first uses big-endian data representation and FULL_INTERLACE (row-order) storage. The second uses little-endian data represtnation and FULL_INTERLACE (row-order) storage. h4mapwriter will warn that little-endian data can't be mapped and no map file will be produced. When run with the -c option, h4mapwriter will map this file, but will still produce warnings because little-endian data has not been thoroughly tested. |
| -c -f -m -t | T13.hdf | n/a | T13.map-c.xml | T13.handmap.xml | This file has 8 vdata (Tables). All use big-endian data representation storage. The second table uses NO_INTERLACE (Column) storage order One table uses linked storage. Tables vary in number of rows, columns, entries per column, and types of data. Last table has an Attribute, and one column in the last table has an attribute. Compare to T11_TablesAttributes, which is identical except for the Column storage order. h4mapwriter will warn that column storage order can't be mapped and no map file will be produced. When run with the -c option, h4mapwriter will not map Table 2. Other tables will be mapped correctly. The map file will have a warning message at the end. |
| -c -f -m -t | T14.hdf | n/a | T14.map-c.xml | T14.handmap.xml | This file has 8 vdata (Tables). All use big-endian data representation storage and FULL_INTERLACE (row order) storage. One table uses linked storage. Tables vary in number of rows, columns, entries per column, and types of data. Last table has an Attribute, and one column in the last table has an attribute. Last table also has 2 Annotations, a Label and a Description. Compare to T11_TablesAttributes, which is identical except for the Annotations. h4mapwriter will warn that Annotations can't be mapped and no map file will be produced. When run with the -c option, h4mapwriter will not map the Annotations. Other content will be mapped correctly. The map file will have a warning message at the end. |
| -c -f -m -t | T15.hdf | T15_External_File | n/a | T15.handmap.xml | This file has 2 vdata (Tables). Both use big-endian data representation and FULL_INTERLACE (row-order) storage and have the same values. The second uses an external file to store the data, with data storage beginning at offset 10. h4mapwriter will warn that external files aren't supported and no map file will be produced. When run with the -c option, h4mapwriter will produce warnings and will not create a map file because it is unable to tell which object(s) use external storage and therefore could write bogus offset/lengths to the map file if one were created. |
| -c -f -m -t | T16.hdf | T16_External_File Put this file in /tmp directory. | n/a | T16.handmap.xml | This file has 2 vdata (Tables). Both use big-endian data representation and FULL_INTERLACE (row-order) storage and have the same values. The second uses an external file to store the data, with data storage beginning at offset 10. The external file is in the /tmp directory. h4mapwriter will warn that external files aren't supported and no map file will be produced. When run with the -c option, h4mapwriter will produce warnings and will not create a map file because it is unable to tell which object(s) use external storage and therefore could write bogus offset/lengths to the map file if one were created. |
| -c -f -m -t | T17.hdf | T17_External_File Put this file in /tmp directory. | n/a | T17.handmap.xml | This file has 2 vdata (Tables). Both use big-endian data representation and FULL_INTERLACE (row-order) storage and have the same values. The second uses an external file to store the data, with data storage beginning at offset 10. The external file is in the /tmp directory and the full path was specified when the file was created. h4mapwriter will warn that external files aren't supported and no map file will be produced. When run with the -c option, h4mapwriter will produce warnings and will not create a map file because it is unable to tell which object(s) use external storage and therefore could write bogus offset/lengths to the map file if one were created. |
| -c -f -m -t | T18.hdf | T18_External_File | n/a | T18.handmap.xml | This file has 8 Vdata (Tables). The Vdata exercise many possible features including big-endian and little-endian datatypes, FULL_INTERLACE (row-order) and NO_INTERLACE (column-order) storage, Annotations, and Storage in external file. The HDF4 File Content Map Schema can handle all of these, but some are not yet supported by the h4mapwriter program. Compare to T11_TablesAttributes, which is similar but which does not have any of the features that h4mapwriter doesn't support. h4mapwriter will warn that little endian, column-order storage, annotations, and external files aren't supported and no map file will be produced. When run with the -c option, h4mapwriter will produce warnings and will not create a map file because it is unable to tell which object(s) use external storage and therefore could write bogus offset/lengths to the map file if one were created. |
| -f -m -t | T19.hdf | n/a | T19.map.xml | n/a | The file has one SDS (Array). It uses chunked storage layout and has ghost cells. |
| -f -m -t | T20.hdf | n/a | T20.map.xml | n/a | The file has one SDS (Array). It uses chunked storage layout, deflate compression, and has ghost cells. Compare to T19_ArrayChunksGhostCells, which has the same array data values and ghost cells, but is not compressed. |
| -f -m -t | T21.hdf | n/a | T21.map.xml | n/a | The file has one SDS (Array). It uses chunked storage layout, deflate compression, has ghost cells, and uses linked blocks for storing the first chunk.. Compare to T20_ArrayChunksGhostCellsDeflate, which has the same data values for all the chunks but the first. |
| -c -f -m -t | T22.hdf | n/a | T22.map-c.xml | T22.handmap.xml | This file has Vgroups (Groups) in a range of different hierarchies, including cycles. It also has Group Annotations and a Group Attribute with little-endian storage order. Groups and features include: (1) Strict hierarcy: Group One / Group Two / Group Three; (2) Cycle under a single parent group: Group Four / Group Five / Group Six / Group Five; (3) Parent group with member in another part of the hierarchy: Group Seven / Group Five / Group Six / Group Five; (4) Group Eight with two annotations (a label and a description) and one attribute with little-endian storage; (5) An all-to-all cycle (all groups members of all other groups) with three Groups: Square, Circle, Line; h4mapwriter will warn about cycles and little-endian data. When run with the -c option, h4mapwriter will map the little-endian data correctly, but will still produce warnings because little-endian data has not been thoroughly tested. The Group cycles won't appear in the HDF4 Content Map. |
| -c -f -m -t | T23.hdf | n/a | T23.map-c.xml |
IMCOMP and JPEG compression not supported in current version of Schema, so no complete HDF4 File Content Map file for this HDF4 file. |
This file has 12 Rasters all created using the DFR8 APIs. It also has one Palette.
All Rasters were created with the same data, but some lose precision due to compression.
They have a range of compression types and not all have a palette associated.
Rasters in file:
(1) with no compression and no palette;
(2) with no compression and no palette;
(3) with no compression and palette;
(4) with no compression and palette;
(5) with RLE compression and palette;
(6) with RLE compression and palette;
(7) with IMCOMP compression and palette;
(8) with IMCOMP compression and palette;
(9) with JPEG compression and palette;
(10) with JPEG compression and palette;
(11) with RLE compression and palette;
(12) with RLE compression and palette.
The HDF4 File Content Map schema has the IMCOMP and JPEG compression types commented out. h4mapwriter will warn that not all images are 8-bit with RLE compression and no map file will be produced. Prior to release 1.0.4, h4mapwriter would not produce a map file when run with the -c option, because the HDF4 library prior to release 4.2.8 returned errors for Rasters with IMCOMP compression. Those errors have been corrected in the HDF4 library and the -c option now produces a map file. Issues remain with palette values for rasters with IMCOMP compression, but the issues are in the HDF4 file creation, not in h4mapwriter. Since IMCOMP is not seen in NASA files, these are low-priority HDF4 issues. |
| -f -m -t | T24.hdf | n/a | T24.map.xml | n/a |
The file has one SDS (Array) and it has 2 dimensions. The first dimension is unlimited, with a current size of 600.
Both dimensions have associated dimension scales. The file uses linked blocks to store both the SDS (Array) and the first
dimension scale (Dimension).
Starting with h4mapwriter release 1.0.3, adjacent byteStream elements are combined for dimensionData. |
| -c -f -m -t | T25.hdf | T25_External_File | n/a |
T25.handmap.xml
Corrected 6/12 to add arrayData element with fillValues for SDStemplate; this matches behavior introducted in h4mapwriter in release 1.0.2. |
This file includes many possible features of HDF4 that are related to SDSes (Arrays) and Dimension Scales (Dimensions). The HDF4 File Content Map schema can handle all of these, but some are not supported by version 1.0.1 of the h4mapwriter program. h4mapwriter will warn that little endian data and external files aren't supported and no map file will be produced. When run with the -c option, h4mapwriter will produce warnings and will not create a map file because it is unable to tell which object(s) use external storage and therefore could write bogus offset/lengths to the map file if one were created. |
| -f -m -t | T26.hdf | n/a | T26.map.xml | n/a |
This file includes many possible features of HDF4 that are related to SDSes (Arrays) and Dimension Scales (Dimensions).
But, not as many as T25_ArraysDimensions_LotsOfFeatures. In particular, there is no little-endian data or external files.
Starting with h4mapwriter release 1.0.2, default fill values that are supplied by the HDF4 library (but not stored in the binary HDF4 file), are written to the map file so they will be available without reliance on the library. |
| -c -f -m -t | T27.hdf | n/a | T27.map-c.xml | JPEG compression not supported in current version of Schema, so no complete HDF4 File Content Map file for this HDF4 file. | This file has 10 Rasters all created using the DFR8 APIs. It also has one Palette. It is very similar to T23, but doesn't have any images with IMCOMP compression -- the one that caused the HDF4 library to crash prior to Release 4.2.8. All Rasters were created with the same data, but some lose precision due to compression. They have a range of compression types and not all have a palette associated. Rasters in file: (1) with no compression and no palette; (2) with no compression and no palette; (3) with no compression and palette; (4) with no compression and palette; (5) with RLE compression and palette; (6) with RLE compression and palette; (7) with JPEG compression and palette; (8) with JPEG compression and palette; (9) with RLE compression and palette; (10) with RLE compression and palette. The HDF4 File Content Map schema has the JPEG compression type commented out. The hdf4mapwriter does not support JPEG compression type for images. h4mapwriter will warn that not all images are 8-bit with RLE compression and no map file will be produced. When run with the -c option, warnings will still be displayed, but h4mapwriter will produce an HDF4 File Content Map with 8 imanges -- those with JPEG compression will be left out. Note that there are no image names for images created with the DFR8 API, hence none appear in the HDF4 Content Map. The output of "hdp dumpgr" does show Image Names, but those are created by hdp are are not actually part of the data that is stored in the file. |
| -f -m -t | T28.hdf | n/a | T28.map.xml | n/a |
This file includes 13 SDSes (Arrays) of rank 0, 1, and 2 and was designed to demonstrate how rank 0,
unpopulated, and empty SDSes (those with no cells) are mapped.
Only one of SDSes (a rank 0) was populated with data.
Some of the SDSes have user-defined fill values.
Some of the rank 1 and 2 SDSes have an unlimited dimension (but no data).
h4mapwriter releases prior to 1.0.2 did not map this file. As of release 1.0.3, h4mapwriter no longer writes an arrayData element for Array 8 because it does not have any raw data (it has a zero-length dimension size). |
- - Last modified: 15 September 2016