The HDF Group

Other HDF Links HDF5 1.10 Documentation New Features in HDF5 1.10 Additional New APIs Collective Metadata I/O Fine-tuning the Metadata Cache File Space Management Page Buffering Partial Edge Chunks Reference SWMR Virtual Datasets (VDS)
		Fine-tuning the Metadata Cache Documentation The purpose of this page is to list and briefly describe the documentation available to those who want to fine-tune how the metadata cache behaves. The following are the sections on this page: HDF5 Library APIs Fine-tuning the Metadata Cache Design Documents and Information Tools   HDF5 Library APIs   H5Pget_evict_on_close Retrieves the property list setting that determines whether an HDF5 object will be evicted from the library's metadata cache when it is closed.   H5Pset_evict_on_close Controls the library's behavior of evicting metadata associated with a closed object.   H5Pget_mdc_image_config Retrieves the metadata cache image option setting from a file access property list.   H5Pset_mdc_image_config Sets the metadata cache image option for a file access property list.   H5Fget_mdc_image_info Gets information about a metadata cache image for a file if it exists.   H5Fget_metadata_read_retry_info   Retrieves the collection of read retries for metadata items with checksum.   H5Pget_metadata_read_attempts Retrieves the number of read attempts from a file access property list.   H5Pset_metadata_read_attempts Sets the number of read attempts in a file access property list.   H5Dflush Causes all buffers associated with a dataset to be immediately written to disk without removing the data from the cache.   H5Drefresh Causes all buffers associated with a dataset to be cleared and immediately re-loaded with updated contents from disk storage.   H5Gflush Causes all buffers associated with a group to be immediately flushed to disk without removing the data from the cache.   H5Grefresh Causes all buffers associated with a group to be cleared and immediately re-loaded with updated contents from disk storage.   H5Oflush Causes all buffers associated with an object to be immediately flushed to disk without removing the data from the cache.   H5Orefresh Causes all buffers associated with an object to be cleared and immediately re-loaded with updated contents from disk storage.   H5Tflush Causes all buffers associated with a committed datatype to be immediately flushed to disk without removing the data from the cache.   H5Trefresh Causes all buffers associated with a committed datatype to be cleared and immediately re-loaded with updated contents from disk storage.   H5Fget_intent Determines the read/write or read-only status of a file.     Logging APIs   H5Pset_mdc_log_options Sets metadata cache logging options.   H5Pget_mdc_log_options Gets metadata cache logging options.   H5Fstart_mdc_logging Starts logging metadata cache events if logging was previously enabled.   H5Fstop_mdc_logging Stops logging metadata cache events if logging was previously enabled and is currently ongoing.   H5Fget_mdc_logging_status   Gets the current metadata cache logging status.   Fine-tuning the Metadata Cache Design Documents and Information The design documents listed below provide background information for the functions that have been developed to provide better control of the metadata cache and why better control might be useful.   Fine-grained Control of Metadata Cache Flushes This document describes some new functions which allow dynamic, fine-grained flush control of the entire metadata cache and the caches related to individual HDF5 objects.   Read Attempts for Metadata with Checksum The HDF5 Library encounters intermittent checksum failures from reading metadata of an HDF5 file opened with single-writer/multiple-reader (SWMR) access. The failure is from a checksum check that does not reflect the data read. This document describes the modifications to the library that address this problem.   Metadata Cache Image This document describes the metadata cache image feature and how it improves performance by writing the metadata cache in a single block on file close, and then populating the cache with the contents of this block on file open. This avoids the many small I/O operations that would otherwise be required on file open and close.   Metadata Cache Evict on Close This feature will cause all metadata for an object to be evicted from the cache as long as metadata is not referenced from any other open object. This can relieve memory pressure on an application or system caused by a large amount of metadata in the metadata cache. See the H5Pget_evict_on_close and H5Pset_evict_on_close APIs for more details.   Metadata Cache Logging This document describes the improvements made to the library’s metadata cache logging facility. The changes include APIs that can be used to control logging operations and the format of log messages.   Flush Dependency Testing This document considers how to design tests for flush dependencies of metadata cache items for single-writer/multiple-reader (SWMR) access.   Tools The following tool was modified to support the Metadata Cache Image feature. h5clear The "-m" and "--image" flags were added to h5clear to close a cache image if it exists.

---

The HDF Group Help Desk: Last modified: April 2017

Copyright by The HDF Group and the Board of Trustees of the University of Illinois. All rights reserved.