This web site is no longer maintained (but will remain online).
Please see The HDF Group's new Support Portal for the latest information.

HOME > HDF5 > TUTOR

HDF5 Tutorial: Advanced Topics
File Drivers

What is a File Driver ?

In HDF5, a file driver is a mapping between the HDF5 format address space and storage. By default, HDF5 simply maps the format address space directly onto a single file.

However, users may want the ability to map the format address space onto different types of storage with various types of maps. With HDF5 we provide a small set of pre-defined file drivers, and we also provide the Virtual File Layer API to enable users to implement their own mappings.

Detailed information on file drivers can be found under VFL Technical Notes in the Documentation.

File Drivers Defined in HDF5

Following are the file drivers that HDF5 provides.

H5FD_SEC2: This is the default driver which uses Posix file-system functions like read and write to perform I/O to a single file.

H5FD_STDIO: This driver uses functions from 'stdio.h' to perform buffered I/O to a single file.

H5FD_CORE: This driver performs I/O directly to memory and can be used to create small temporary files that never exist on permanent storage.

H5FD_MPIIO: This driver is used with Parallel HDF5, and is only pre-defined if the library is compiled with parallel I/O support. Refer to the Parallel HDF5 Tutorial for more information on using Parallel HDF5.

H5FD_MULTI: This driver enables different types of HDF5 data and metadata to be written to separate files. The H5FD_SPLIT driver is an example of what the H5FD_MULTI driver can do.

H5FD_FAMILY: This driver partitions a large format address space into smaller chunks (separate storage of a user's choice).
H5FD_SPLIT: This driver splits the meta data and raw data into separate storage of a user's choice.

Programming Model for Using a Pre-Defined File Driver

Create a copy or instance of the File Access property list:
```
   fapl = H5Pcreate (H5P_FILE_ACCESS);
```
Initialize the file driver. Each pre-defined file driver has it's own initialization function, whose name is H5Pset_fapl_ followed by the driver name and which takes a file access property list as the first argument, followed by additional driver-dependent arguments. For example:
```
  size_t member_size = 100*1024*1024;  /* 100 MB */
  status = H5Pset_fapl_family (fapl, member_size, H5P_DEFAULT);
```
An alternative to using the driver initialization function is to set the driver directly using H5Pset_driver, which is not covered here.
Call H5Fcreate, passing in the identifier of the property list just modified.
```
   file_id = H5Fcreate (HDF5FILE, H5F_ACC_TRUNC, H5P_DEFAULT, fapl);
```
Close the File Access Property List:
```
   status = H5Pclose (fapl);
```
Perform I/O on the file, if need be. To do so, a Data Access/Transfer property must be copied, modified, and passed in to H5Dread or H5Dwrite.
For example, the following sets the MPI-IO driver to use independent access for I/O operations:
```
  dxpl = H5Pcreate (H5P_DATA_XFER);
  status = H5Pset_dxpl_mpio (dxpl, H5FD_MPIO_INDEPENDENT);
  status = H5Dread (dataset_id, type, mspace, fspace, buffer, dxpl);
```

User Designed File Drivers

These are out of the scope of this tutorial. Refer to the Technical Notes documentation on the Virtual File Layer.

How Does a General Application Open an HDF5 File ?

A general application does not know what drivers were used to create a file. It would have to try different file drivers until it succeeds. An example of a general application is the h5dump tool that we provide with HDF5.

- - Last modified: 21 December 2016

HDF5 Tutorial: Advanced TopicsFile Drivers