Please, help us to better serve our user community by answering the following short survey:
HDF5  1.15.0
API Reference
No Matches


Tracking Free Space in HDF5 Files

You sometimes delete objects in HDF5 files and don't have new content to use the free space, but would like to reuse it in the future.
At file creation time, set the file space management strategy and persistence of free space tracking information via H5Pset_file_space_strategy().
This feature is only supported in HDF5 1.10.1+.
15 {
16 __label__ fail_fcpl, fail_fapl, fail_file;
17 hid_t fcpl, fapl, file;
19 if ((fcpl = H5Pcreate(H5P_FILE_CREATE)) < 0) {
20 ret_val = EXIT_FAILURE;
21 goto fail_fcpl;
22 }
23#if H5_VERSION_GE(1, 10, 1)
26#error HDF5 1.10.1+ required
28 ret_val = EXIT_FAILURE;
29 goto fail_fapl;
30 }
32 if ((fapl = H5Pcreate(H5P_FILE_ACCESS)) < 0) {
33 ret_val = EXIT_FAILURE;
34 goto fail_fapl;
35 }
36#if H5_VERSION_GE(1, 10, 1)
39#error HDF5 1.10.x+ required
41 ret_val = EXIT_FAILURE;
42 goto fail_file;
43 }
45 if ((file = H5Fcreate("free_space.h5", H5F_ACC_TRUNC, fcpl, fapl)) < 0) {
46 ret_val = EXIT_FAILURE;
47 goto fail_file;
48 }
49 H5Fclose(file);
52 H5Pclose(fapl);
54 H5Pclose(fcpl);
56 }
Definition H5Fpublic.h:188
Definition H5Fpublic.h:195
#define H5F_ACC_TRUNC
Definition H5Fpublic.h:50
Definition H5Fpublic.h:202
int64_t hid_t
Definition H5Ipublic.h:60
Definition H5Ppublic.h:51
Definition H5Ppublic.h:52
herr_t H5Pset_libver_bounds(hid_t plist_id, H5F_libver_t low, H5F_libver_t high)
Controls the range of library release versions used when creating objects in a file.
herr_t H5Pset_file_space_strategy(hid_t plist_id, H5F_fspace_strategy_t strategy, hbool_t persist, hsize_t threshold)
Sets the file space handling strategy and persisting free-space values for a file creation property l...
herr_t H5Fclose(hid_t file_id)
Terminates access to an HDF5 file.
hid_t H5Fcreate(const char *filename, unsigned flags, hid_t fcpl_id, hid_t fapl_id)
Creates an HDF5 file.
herr_t H5Pclose(hid_t plist_id)
Terminates access to a property list.
hid_t H5Pcreate(hid_t cls_id)
Creates a new property list as an instance of a property list class.
Free space tracking is supported only in HDF5 versions 1.10.x and higher. This has implications for the accessibility of your HDF5 files and should be considered carefully. If compatibility with previous versions of HDF5 must be maintained, space reclamation via h5repack might be an option.
The file space strategy H5F_FSPACE_STRATEGY_FSM_AGGR is not the only option that supports free-space tracking. H5F_FSPACE_STRATEGY_PAGE is another option, which adds paged allocation and is used most effectively with page buffering.
For an in-depth account of HDF5 file space management, paged-allocation, and page buffering, see the following documents:
See Also
See Maintaining Compatibility with other HDF5 Library Versions for HDF5 compatibility implications.

Removing Unused Space from HDF5 Files

Based on estimates or h5stat output you know that a large portion of an HDF5 file consists of free or unaccounted space, and you would like to remove it.

Creating an HDF5 File User Block

You would like to include certain ancillary, non-HDF5 content in an HDF5 file such that it can be accessed without the HDF5 library.
Use a file creation property list in which the user block size is set via H5Pset_userblock(). In the example below, we create an 8 MiB user block.
60 {
61 __label__ fail_fcpl, fail_file;
62 hid_t fcpl, file;
64 if ((fcpl = H5Pcreate(H5P_FILE_CREATE)) < 0) {
65 ret_val = EXIT_FAILURE;
66 goto fail_fcpl;
67 }
68 if (H5Pset_userblock(fcpl, 8192 * 1024) < 0) {
69 ret_val = EXIT_FAILURE;
70 goto fail_file;
71 }
72 if ((file = H5Fcreate("userblock.h5", H5F_ACC_TRUNC, fcpl, H5P_DEFAULT)) < 0) {
73 ret_val = EXIT_FAILURE;
74 goto fail_file;
75 }
76 H5Fclose(file);
79 H5Pclose(fcpl);
81 }
#define H5P_DEFAULT
Definition H5Ppublic.h:102
herr_t H5Pset_userblock(hid_t plist_id, hsize_t size)
Sets user block size.
The user block begins at offset 0 and must be at least 512 bytes and a power of 2. The HDF5 library ignores any content between the beginning of the file and the end of the user block.
You can add or strip a user block to/from an existing HDF5 file with the h5jam/h5unjam tool, respectively.
If you try to embed content into the user block for use by other applications, pay close attention to how they handle space beyond the last used byte in the user block or the user block in general. In the worst case, applications might try to truncate the rest of the file and destroy the HDF5 portion of the file.
See Also
References to related recipes