Detailed Description

Collaboration diagram for Metadata Cache:

Functions
herr_t	H5Fget_mdc_config (hid_t file_id, H5AC_cache_config_t *config_ptr)
	Obtains current metadata cache configuration for target file.

herr_t	H5Fset_mdc_config (hid_t file_id, const H5AC_cache_config_t *config_ptr)
	Attempts to configure metadata cache of target file.

herr_t	H5Fget_mdc_hit_rate (hid_t file_id, double *hit_rate_ptr)
	Obtains target file's metadata cache hit rate.

herr_t	H5Fget_mdc_size (hid_t file_id, size_t max_size_ptr, size_t min_clean_size_ptr, size_t cur_size_ptr, int cur_num_entries_ptr)
	Obtains current metadata cache size data for specified file.

herr_t	H5Freset_mdc_hit_rate_stats (hid_t file_id)
	Resets hit rate statistics counters for the target file.

herr_t	H5Fstart_mdc_logging (hid_t file_id)
	Starts logging metadata cache events if logging was previously enabled.

herr_t	H5Fstop_mdc_logging (hid_t file_id)
	Stops logging metadata cache events if logging was previously enabled and is currently ongoing.

herr_t	H5Fget_mdc_logging_status (hid_t file_id, hbool_t is_enabled, hbool_t is_currently_logging)
	Gets the current metadata cache logging status.

herr_t	H5Fget_mdc_image_info (hid_t file_id, haddr_t image_addr, hsize_t image_size)
	Obtains information about a cache image if it exists.

Function Documentation

◆ H5Fget_mdc_config()

herr_t H5Fget_mdc_config	(	hid_t	file_id,
		H5AC_cache_config_t *	config_ptr
	)

Obtains current metadata cache configuration for target file.

Parameters

[in]	file_id	File identifier
[in,out]	config_ptr	Pointer to the H5AC_cache_config_t instance in which the current metadata cache configuration is to be reported. The fields of this structure are discussed here.

Returns: Returns a non-negative value if successful; otherwise, returns a negative value.

Note: The in direction applies only to the H5AC_cache_config_t::version field. All other fields are out parameters.

H5Fget_mdc_config() loads the current metadata cache configuration into the instance of H5AC_cache_config_t pointed to by the config_ptr parameter.
The fields of the H5AC_cache_config_t structure are shown below:

typedef struct H5AC_cache_config_t {
    /* general configuration fields: */
    int version;
    hbool_t rpt_fcn_enabled;
    hbool_t open_trace_file;
    hbool_t close_trace_file;
    char trace_file_name[H5AC__MAX_TRACE_FILE_NAME_LEN + 1];
    hbool_t evictions_enabled;
    hbool_t set_initial_size;
    size_t initial_size;
    double min_clean_fraction;
    size_t max_size;
    size_t min_size;
    long int epoch_length;
 
    /* size increase control fields: */
    enum H5C_cache_incr_mode incr_mode;
    double lower_hr_threshold;
    double increment;
    hbool_t apply_max_increment;
    size_t max_increment;
    enum H5C_cache_flash_incr_mode flash_incr_mode;
    double flash_multiple;
    double flash_threshold;
 
    /* size decrease control fields: */
    enum H5C_cache_decr_mode decr_mode;
    double upper_hr_threshold;
    double decrement;
    hbool_t apply_max_decrement;
    size_t max_decrement;
    int epochs_before_eviction;
    hbool_t apply_empty_reserve;
    double empty_reserve;
 
    /* parallel configuration fields: */
    size_t dirty_bytes_threshold;
    int metadata_write_strategy;
} H5AC_cache_config_t;

(Click on a enumerator, field, or type for more information.)

Since: 1.8.0

◆ H5Fget_mdc_hit_rate()

herr_t H5Fget_mdc_hit_rate	(	hid_t	file_id,
		double *	hit_rate_ptr
	)

Obtains target file's metadata cache hit rate.

Parameters

[in]	file_id	File identifier
[out]	hit_rate_ptr	Pointer to the double in which the hit rate is returned. Note that `hit_rate_ptr` is undefined if the API call fails

Returns: Returns a non-negative value if successful; otherwise, returns a negative value.

H5Fget_mdc_hit_rate() queries the metadata cache of the target file to obtain its hit rate (cache hits / (cache hits + cache misses)) since the last time hit rate statistics were reset. If the cache has not been accessed since the last time the hit rate stats were reset, the hit rate is defined to be 0.0.

The hit rate stats can be reset either manually (via H5Freset_mdc_hit_rate_stats()), or automatically. If the cache's adaptive resize code is enabled, the hit rate stats will be reset once per epoch. If they are reset manually as well, the cache may behave oddly.

See the overview of the metadata cache in the special topics section of the user manual for details on the metadata cache and its adaptive resize algorithms.

Since: 1.8.0

◆ H5Fget_mdc_image_info()

herr_t H5Fget_mdc_image_info	(	hid_t	file_id,
		haddr_t *	image_addr,
		hsize_t *	image_size
	)

Obtains information about a cache image if it exists.

Parameters

[in]	file_id	File identifier
[out]	image_addr	Offset of the cache image if it exists, or HADDR_UNDEF if it does not
[out]	image_size	Length of the cache image if it exists, or 0 if it does not

Returns: Returns a non-negative value if successful; otherwise, returns a negative value.

H5Fget_mdc_image_info() returns information about a cache image if it exists.

When an HDF5 file is opened in Read/Write mode, any metadata cache image will be read and deleted from the file on the first metadata cache access (or, if persistent free space managers are enabled, on the first file space allocation / deallocation, or read of free space manager status, whichever comes first).

Thus, if the file is opened Read/Write, H5Fget_mdc_image_info() should be called immediately after file open and before any other operation. If H5Fget_mdc_image_info() is called after the cache image is loaded, it will correctly report that no cache image exists, as the image will have already been read and deleted from the file. In the Read Only case, the function may be called at any time, as any cache image will not be deleted from the file.

Since: 1.10.1

◆ H5Fget_mdc_logging_status()

herr_t H5Fget_mdc_logging_status	(	hid_t	file_id,
		hbool_t *	is_enabled,
		hbool_t *	is_currently_logging
	)

Gets the current metadata cache logging status.

Parameters

[in]	file_id	File identifier
[out]	is_enabled	Whether logging is enabled
[out]	is_currently_logging	Whether events are currently being logged

Returns: Returns a non-negative value if successful; otherwise, returns a negative value.

The metadata cache is a central part of the HDF5 library through which all file metadata reads and writes take place. File metadata is normally invisible to the user and is used by the library for purposes such as locating and indexing data. File metadata should not be confused with user metadata, which consists of attributes created by users and attached to HDF5 objects such as datasets via H5A API calls.

Due to the complexity of the cache, a trace/logging feature has been created that can be used by HDF5 developers for debugging and performance analysis. The functions that control this functionality will normally be of use to a very limited number of developers outside of The HDF Group. The functions have been documented to help users create logs that can be sent with bug reports.

Control of the log functionality is straightforward. Logging is enabled via the H5Pset_mdc_log_options() function, which will modify the file access property list used to open or create a file. This function has a flag that determines whether logging begins at file open or starts in a paused state. Log messages can then be controlled via the H5Fstart_mdc_logging() and H5Fstop_mdc_logging() functions. H5Pget_mdc_log_options() can be used to examine a file access property list, and H5Fget_mdc_logging_status() will return the current state of the logging flags.

The log format is described in the Metadata Cache Logging document.

Note: Unlike H5Fstart_mdc_logging() and H5Fstop_mdc_logging(), this function can be called on any open file identifier.

Since: 1.10.0

◆ H5Fget_mdc_size()

herr_t H5Fget_mdc_size	(	hid_t	file_id,
		size_t *	max_size_ptr,
		size_t *	min_clean_size_ptr,
		size_t *	cur_size_ptr,
		int *	cur_num_entries_ptr
	)

Obtains current metadata cache size data for specified file.

Parameters

[in]	file_id	File identifier
[out]	max_size_ptr	Pointer to the location in which the current cache maximum size is to be returned, or NULL if this datum is not desired
[out]	min_clean_size_ptr	Pointer to the location in which the current cache minimum clean size is to be returned, or NULL if that datum is not desired
[out]	cur_size_ptr	Pointer to the location in which the current cache size is to be returned, or NULL if that datum is not desired
[out]	cur_num_entries_ptr	Pointer to the location in which the current number of entries in the cache is to be returned, or NULL if that datum is not desired

Returns: Returns a non-negative value if successful; otherwise, returns a negative value.

H5Fget_mdc_size() queries the metadata cache of the target file for the desired size information, and returns this information in the locations indicated by the pointer parameters. If any pointer parameter is NULL, the associated data is not returned.

If the API call fails, the values returned via the pointer parameters are undefined.

If adaptive cache resizing is enabled, the cache maximum size and minimum clean size may change at the end of each epoch. Current size and current number of entries can change on each cache access.

Current size can exceed maximum size under certain conditions. See the overview of the metadata cache in the special topics section of the user manual for a discussion of this.

Since: 1.8.0

◆ H5Freset_mdc_hit_rate_stats()

herr_t H5Freset_mdc_hit_rate_stats ( hid_t file_id )

Resets hit rate statistics counters for the target file.

Parameters

[in] file_id File identifier

Returns: Returns a non-negative value if successful; otherwise, returns a negative value.

H5Freset_mdc_hit_rate_stats() resets the hit rate statistics counters in the metadata cache associated with the specified file.

If the adaptive cache resizing code is enabled, the hit rate statistics are reset at the beginning of each epoch. This API call allows you to do the same thing from your program.

The adaptive cache resizing code may behave oddly if you use this call when adaptive cache resizing is enabled. However, the call should be useful if you choose to control metadata cache size from your program.

See Metadata Caching in HDF5 for details about the metadata cache and the adaptive cache resizing algorithms. If you have not read, understood, and thought about the material covered in that documentation, you should not be using this API call.

Since: 1.8.0

◆ H5Fset_mdc_config()

herr_t H5Fset_mdc_config	(	hid_t	file_id,
		const H5AC_cache_config_t *	config_ptr
	)

Attempts to configure metadata cache of target file.

Parameters

[in]	file_id	File identifier
[in,out]	config_ptr	Pointer to the H5AC_cache_config_t instance containing the desired configuration. The fields of this structure are discussed here.

Returns: Returns a non-negative value if successful; otherwise, returns a negative value.

H5Fset_mdc_config() attempts to configure the file's metadata cache according configuration supplied in config_ptr.

typedef struct H5AC_cache_config_t {
    /* general configuration fields: */
    int version;
    hbool_t rpt_fcn_enabled;
    hbool_t open_trace_file;
    hbool_t close_trace_file;
    char trace_file_name[H5AC__MAX_TRACE_FILE_NAME_LEN + 1];
    hbool_t evictions_enabled;
    hbool_t set_initial_size;
    size_t initial_size;
    double min_clean_fraction;
    size_t max_size;
    size_t min_size;
    long int epoch_length;
 
    /* size increase control fields: */
    enum H5C_cache_incr_mode incr_mode;
    double lower_hr_threshold;
    double increment;
    hbool_t apply_max_increment;
    size_t max_increment;
    enum H5C_cache_flash_incr_mode flash_incr_mode;
    double flash_multiple;
    double flash_threshold;
 
    /* size decrease control fields: */
    enum H5C_cache_decr_mode decr_mode;
    double upper_hr_threshold;
    double decrement;
    hbool_t apply_max_decrement;
    size_t max_decrement;
    int epochs_before_eviction;
    hbool_t apply_empty_reserve;
    double empty_reserve;
 
    /* parallel configuration fields: */
    size_t dirty_bytes_threshold;
    int metadata_write_strategy;
} H5AC_cache_config_t;

(Click on a enumerator, field, or type for more information.)

Since: 1.8.0

◆ H5Fstart_mdc_logging()

herr_t H5Fstart_mdc_logging ( hid_t file_id )

Starts logging metadata cache events if logging was previously enabled.

Parameters

[in] file_id File identifier

Returns: Returns a non-negative value if successful; otherwise, returns a negative value.

The metadata cache is a central part of the HDF5 library through which all file metadata reads and writes take place. File metadata is normally invisible to the user and is used by the library for purposes such as locating and indexing data. File metadata should not be confused with user metadata, which consists of attributes created by users and attached to HDF5 objects such as datasets via H5A API calls.

Due to the complexity of the cache, a trace/logging feature has been created that can be used by HDF5 developers for debugging and performance analysis. The functions that control this functionality will normally be of use to a very limited number of developers outside of The HDF Group. The functions have been documented to help users create logs that can be sent with bug reports.

Control of the log functionality is straightforward. Logging is enabled via the H5Pset_mdc_log_options() function, which will modify the file access property list used to open or create a file. This function has a flag that determines whether logging begins at file open or starts in a paused state. Log messages can then be controlled via the H5Fstart_mdc_logging() and H5Fstop_mdc_logging() functions. H5Pget_mdc_log_options() can be used to examine a file access property list, and H5Fget_mdc_logging_status() will return the current state of the logging flags.

The log format is described in the Metadata Cache Logging document.

Note: Logging can only be started or stopped if metadata cache logging was enabled via H5Pset_mdc_log_options().
When enabled and currently logging, the overhead of the logging feature will almost certainly be significant.
The log file is opened when the HDF5 file is opened or created and not when this function is called for the first time.
This function opens the log file and starts logging metadata cache operations for a particular file. Calling this function when logging has already been enabled will be considered an error.

Since: 1.10.0

◆ H5Fstop_mdc_logging()

herr_t H5Fstop_mdc_logging ( hid_t file_id )

Stops logging metadata cache events if logging was previously enabled and is currently ongoing.

Parameters

[in] file_id File identifier

Returns: Returns a non-negative value if successful; otherwise, returns a negative value.

The metadata cache is a central part of the HDF5 library through which all file metadata reads and writes take place. File metadata is normally invisible to the user and is used by the library for purposes such as locating and indexing data. File metadata should not be confused with user metadata, which consists of attributes created by users and attached to HDF5 objects such as datasets via H5A API calls.

Due to the complexity of the cache, a trace/logging feature has been created that can be used by HDF5 developers for debugging and performance analysis. The functions that control this functionality will normally be of use to a very limited number of developers outside of The HDF Group. The functions have been documented to help users create logs that can be sent with bug reports.

Control of the log functionality is straightforward. Logging is enabled via the H5Pset_mdc_log_options() function, which will modify the file access property list used to open or create a file. This function has a flag that determines whether logging begins at file open or starts in a paused state. Log messages can then be controlled via the H5Fstart_mdc_logging() and H5Fstop_mdc_logging() functions. H5Pget_mdc_log_options() can be used to examine a file access property list, and H5Fget_mdc_logging_status() will return the current state of the logging flags.

The log format is described in the Metadata Cache Logging document.

Note: Logging can only be started or stopped if metadata cache logging was enabled via H5Pset_mdc_log_options().
This function only suspends the logging operations. The log file will remain open and will not be closed until the HDF5 file is closed.

Since: 1.10.0

Detailed Description

Functions

Function Documentation

◆ H5Fget_mdc_config()

◆ H5Fget_mdc_hit_rate()

◆ H5Fget_mdc_image_info()

◆ H5Fget_mdc_logging_status()

◆ H5Fget_mdc_size()

◆ H5Freset_mdc_hit_rate_stats()

◆ H5Fset_mdc_config()

◆ H5Fstart_mdc_logging()

◆ H5Fstop_mdc_logging()