• SWMR
• About
• Team Members
• Sponsors

About the Project

The problem.
Data acquisition and computer modeling systems often need to analyze and visualize data while it is being written. It is not unusual, for example, for an application to produce results in a middle of a run that suggest some basic parameters be changed, sensors be adjusted, or the run be scrapped entirely.

HDF5 is routinely used in such acquisition and modeling systems in areas such as materials science, flight testing, and finance. Unfortunately, the current implementation of the HDF5 Library (1.8.x) does not easily support concurrent reading from an HDF5 file while data is being written to the file. This limitation makes inspecting data while it is being collected at best very difficult and often impossible.

For more information on how explicit coordination among processes accessing an HDF5 file currently works, see the HDF5 FAQ.

The solution: concurrent access.
The HDF Group is developing a file access pattern known as single-writer/multiple-reader or SWMR for short. The SWMR access pattern in its current state of development allows a writer process to append data to the datasets in an HDF5 file while several reader processes can concurrently read the new data from the file. No communications between the processes and no file locking are required. The processes can run on the same or on different platforms.

For more information on the current development of SWMR capabilities, see the “Documentation for HDF5 New Features” page.

For more information, click the links on the left.