Dataset Transfer Properties

Detailed Description

Use dataset transfer properties to customize certain aspects of reading and writing datasets such as transformations, MPI-IO I/O mode, error detection, etc. These properties are not persisted with datasets, and can be adjusted at runtime before a dataset is read or written.

Functions
H5_DLL herr_t	H5Pset_dxpl_mpio (hid_t dxpl_id, H5FD_mpio_xfer_t xfer_mode)
	Sets data transfer mode. More...
H5_DLL herr_t	H5Pget_dxpl_mpio (hid_t dxpl_id, H5FD_mpio_xfer_t *xfer_mode)
	Returns the data transfer mode. More...
H5_DLL herr_t	H5Pset_dxpl_mpio_collective_opt (hid_t dxpl_id, H5FD_mpio_collective_opt_t opt_mode)
	Sets data transfer mode. More...
H5_DLL herr_t	H5Pset_dxpl_mpio_chunk_opt (hid_t dxpl_id, H5FD_mpio_chunk_opt_t opt_mode)
	Sets a flag specifying linked-chunk I/O or multi-chunk I/O. More...
H5_DLL herr_t	H5Pset_dxpl_mpio_chunk_opt_num (hid_t dxpl_id, unsigned num_chunk_per_proc)
	Sets a numeric threshold for linked-chunk I/O. More...
H5_DLL herr_t	H5Pset_dxpl_mpio_chunk_opt_ratio (hid_t dxpl_id, unsigned percent_num_proc_per_chunk)
	Sets a ratio threshold for collective I/O. More...
herr_t	H5Pget_btree_ratios (hid_t plist_id, double *left, double *middle, double *right)
	Gets B-tree split ratios for a dataset transfer property list. More...
size_t	H5Pget_buffer (hid_t plist_id, void **tconv, void **bkg)
	Reads buffer settings. More...
ssize_t	H5Pget_data_transform (hid_t plist_id, char *expression, size_t size)
	Retrieves a data transform expression. More...
H5Z_EDC_t	H5Pget_edc_check (hid_t plist_id)
	Determines whether error-detection is enabled for dataset reads. More...
herr_t	H5Pget_hyper_vector_size (hid_t fapl_id, size_t *size)
	Retrieves number of I/O vectors to be read/written in hyperslab I/O. More...
int	H5Pget_preserve (hid_t plist_id)
	Checks status of the dataset transfer property list (DEPRECATED) More...
herr_t	H5Pget_type_conv_cb (hid_t dxpl_id, H5T_conv_except_func_t *op, void **operate_data)
	Gets user-defined datatype conversion callback function. More...
herr_t	H5Pget_vlen_mem_manager (hid_t plist_id, H5MM_allocate_t *alloc_func, void **alloc_info, H5MM_free_t *free_func, void **free_info)
	Gets the memory manager for variable-length datatype allocation in H5Dread() and H5Dvlen_reclaim() More...
herr_t	H5Pset_btree_ratios (hid_t plist_id, double left, double middle, double right)
	Sets B-tree split ratios for a dataset transfer property list. More...
herr_t	H5Pset_buffer (hid_t plist_id, size_t size, void *tconv, void *bkg)
	Sets type conversion and background buffers. More...
herr_t	H5Pset_data_transform (hid_t plist_id, const char *expression)
	Sets a data transform expression. More...
herr_t	H5Pset_edc_check (hid_t plist_id, H5Z_EDC_t check)
	Sets the dataset transfer property list to enable or disable error detection when reading data. More...
herr_t	H5Pset_filter_callback (hid_t plist_id, H5Z_filter_func_t func, void *op_data)
	Sets user-defined filter callback function. More...
herr_t	H5Pset_hyper_vector_size (hid_t plist_id, size_t size)
	Sets number of I/O vectors to be read/written in hyperslab I/O. More...
herr_t	H5Pset_preserve (hid_t plist_id, hbool_t status)
	Sets the dataset transfer property list status. More...
herr_t	H5Pset_type_conv_cb (hid_t dxpl_id, H5T_conv_except_func_t op, void *operate_data)
	Sets user-defined datatype conversion callback function. More...
herr_t	H5Pset_vlen_mem_manager (hid_t plist_id, H5MM_allocate_t alloc_func, void *alloc_info, H5MM_free_t free_func, void *free_info)
	Sets the memory manager for variable-length datatype allocation in H5Dread() and H5Dvlen_reclaim() More...
herr_t	H5Pget_mpio_actual_chunk_opt_mode (hid_t plist_id, H5D_mpio_actual_chunk_opt_mode_t *actual_chunk_opt_mode)
	Retrieves the type of chunk optimization that HDF5 actually performed on the last parallel I/O call (not necessarily the type requested) More...
herr_t	H5Pget_mpio_actual_io_mode (hid_t plist_id, H5D_mpio_actual_io_mode_t *actual_io_mode)
	Retrieves the type of I/O that HDF5 actually performed on the last parallel I/O call (not necessarily the type requested) More...
herr_t	H5Pget_mpio_no_collective_cause (hid_t plist_id, uint32_t *local_no_collective_cause, uint32_t *global_no_collective_cause)
	Retrieves local and global causes that broke collective I/O on the last parallel I/O call. More...

Function Documentation

H5Pget_btree_ratios()

herr_t H5Pget_btree_ratios	(	hid_t	plist_id,
		double *	left,
		double *	middle,
		double *	right
	)

Gets B-tree split ratios for a dataset transfer property list.

Parameters

[in]	plist_id	Dataset transfer property list identifier
[out]	left	The B-tree split ratio for left-most nodes
[out]	middle	The B-tree split ratio for right-most nodes and lone nodes
[out]	right	The B-tree split ratio for all other nodes

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

H5Pget_btree_ratios() returns the B-tree split ratios for a dataset transfer property list.

The B-tree split ratios are returned through the non-NULL arguments left, middle, and right, as set by the H5Pset_btree_ratios() function.

H5Pget_buffer()

size_t H5Pget_buffer	(	hid_t	plist_id,
		void **	tconv,
		void **	bkg
	)

Reads buffer settings.

Parameters

[in]	plist_id	Identifier for the dataset transfer property list
[out]	tconv	Address of the pointer to application-allocated type conversion buffer
[out]	bkg	Address of the pointer to application-allocated background buffer

Returns

Returns buffer size, in bytes, if successful; otherwise 0 on failure.

H5Pget_buffer() reads values previously set with H5Pset_buffer().

Version

1.6.0 The return type changed from hsize_t to size_t.

1.4.0 The return type changed to hsize_t.

H5Pget_data_transform()

ssize_t H5Pget_data_transform	(	hid_t	plist_id,
		char *	expression,
		size_t	size
	)

Retrieves a data transform expression.

Parameters

[in]	plist_id	Identifier of the property list or class
[out]	expression	Pointer to memory where the transform expression will be copied
[in]	size	Number of bytes of the transform expression to copy to

Returns

Success: the size of the transform expression. Failure: a negative value.

H5Pget_data_transform() retrieves the data transform expression previously set in the dataset transfer property list plist_id by H5Pset_data_transform().

H5Pget_data_transform() can be used to both retrieve the transform expression and query its size.

If expression is non-NULL, up to size bytes of the data transform expression are written to the buffer. If expression is NULL, size is ignored, and the function does not write anything to the buffer. The function always returns the size of the data transform expression.

If 0 is returned for the size of the expression, no data transform expression exists for the property list.

If an error occurs, the buffer pointed to by expression is unchanged, and the function returns a negative value.

Since

1.8.0

H5Pget_dxpl_mpio()

H5_DLL herr_t H5Pget_dxpl_mpio	(	hid_t	dxpl_id,
		H5FD_mpio_xfer_t *	xfer_mode
	)

Returns the data transfer mode.

Parameters

[in]	dxpl_id	Dataset transfer property list identifier
[out]	xfer_mode	Transfer mode

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

H5Pget_dxpl_mpio() queries the data transfer mode currently set in the data transfer property list dxpl_id.

Upon return, xfer_mode contains the data transfer mode, if it is non-null.

H5Pget_dxpl_mpio() is not a collective function.

Since

1.4.0

H5Pget_edc_check()

H5Z_EDC_t H5Pget_edc_check

(

hid_t

plist_id

)

Determines whether error-detection is enabled for dataset reads.

Parameters

[in]

plist_id

Dataset transfer property list identifier

Returns

Returns H5Z_ENABLE_EDC or H5Z_DISABLE_EDC if successful; otherwise returns a negative value.

H5Pget_edc_check() queries the dataset transfer property list plist to determine whether error detection is enabled for data read operations.

Since

1.6.0

H5Pget_hyper_vector_size()

herr_t H5Pget_hyper_vector_size	(	hid_t	fapl_id,
		size_t *	size
	)

Retrieves number of I/O vectors to be read/written in hyperslab I/O.

Parameters

[in]	fapl_id	Dataset transfer property list identifier
[out]	size	Number of I/O vectors to accumulate in memory for I/O operations

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

H5Pget_hyper_vector_size() retrieves the number of I/O vectors to be accumulated in memory before being issued to the lower levels of the HDF5 library for reading or writing the actual data.

The number of I/O vectors set in the dataset transfer property list fapl_id is returned in size. Unless the default value is in use, size was previously set with a call to H5Pset_hyper_vector_size().

Since

1.6.0

H5Pget_mpio_actual_chunk_opt_mode()

herr_t H5Pget_mpio_actual_chunk_opt_mode	(	hid_t	plist_id,
		H5D_mpio_actual_chunk_opt_mode_t *	actual_chunk_opt_mode
	)

Retrieves the type of chunk optimization that HDF5 actually performed on the last parallel I/O call (not necessarily the type requested)

Parameters

[in]	plist_id	Dataset transfer property list identifier
[out]	actual_chunk_opt_mode	The type of chunk optimization performed by HDF5

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

Motivation:

A user can request collective I/O via a data transfer property list (DXPL) that has been suitably modified with H5Pset_dxpl_mpio(). However, HDF5 will sometimes ignore this request and perform independent I/O instead. This property allows the user to see what kind of I/O HDF5 actually performed. Used in conjunction with H5Pget_mpio_actual_io_mode(), this property allows the user to determine exactly what HDF5 did when attempting collective I/O.

H5Pget_mpio_actual_chunk_opt_mode() retrieves the type of chunk optimization performed when collective I/O was requested. This property is set before I/O takes place, and will be set even if I/O fails.

Valid values returned in actual_chunk_opt_mode:

 
typedef enum H5D_mpio_actual_chunk_opt_mode_t {
    H5D_MPIO_NO_CHUNK_OPTIMIZATION = 0,
    H5D_MPIO_LINK_CHUNK,
    H5D_MPIO_MULTI_CHUNK
} H5D_mpio_actual_chunk_opt_mode_t;

(Click on a enumerator, field, or type for more information.)

Since

1.8.8

H5Pget_mpio_actual_io_mode()

herr_t H5Pget_mpio_actual_io_mode	(	hid_t	plist_id,
		H5D_mpio_actual_io_mode_t *	actual_io_mode
	)

Retrieves the type of I/O that HDF5 actually performed on the last parallel I/O call (not necessarily the type requested)

Parameters

[in]	plist_id	Dataset transfer property list identifier
[out]	actual_io_mode	The type of I/O performed by this process

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

Motivation:

A user can request collective I/O via a data transfer property list (DXPL) that has been suitably modified with H5Pset_dxpl_mpio(). However, HDF5 will sometimes ignore this request and perform independent I/O instead. This property allows the user to see what kind of I/O HDF5 actually performed. Used in conjunction with H5Pget_mpio_actual_chunk_opt_mode(), this property allows the user to determine exactly HDF5 did when attempting collective I/O.

H5Pget_mpio_actual_io_mode() retrieves the type of I/O performed on the selection of the current process. This property is set after all I/O is completed; if I/O fails, it will not be set.

Valid values returned in actual_io_mode:

 
typedef enum H5D_mpio_actual_io_mode_t {
    H5D_MPIO_NO_COLLECTIVE = 0x0,
    H5D_MPIO_CHUNK_INDEPENDENT = 0x1,
    H5D_MPIO_CHUNK_COLLECTIVE = 0x2,
    H5D_MPIO_CHUNK_MIXED = 0x1 | 0x2,
    H5D_MPIO_CONTIGUOUS_COLLECTIVE = 0x4
} H5D_mpio_actual_io_mode_t;

(Click on a enumerator, field, or type for more information.)

Attention

All processes do not need to have the same value. For example, if I/O is being performed using the multi chunk optimization scheme, one process's selection may include only chunks accessed collectively, while another may include chunks accessed independently. In this case, the first process will report H5D_MPIO_CHUNK_COLLECTIVE while the second will report H5D_MPIO_CHUNK_INDEPENDENT.

See also

H5Pget_mpio_no_collective_cause(), H5Pget_mpio_actual_chunk_opt_mode()

Since

1.8.8

H5Pget_mpio_no_collective_cause()

herr_t H5Pget_mpio_no_collective_cause	(	hid_t	plist_id,
		uint32_t *	local_no_collective_cause,
		uint32_t *	global_no_collective_cause
	)

Retrieves local and global causes that broke collective I/O on the last parallel I/O call.

Parameters

[in]	plist_id	Dataset transfer property list identifier
[out]	local_no_collective_cause	An enumerated set value indicating the causes that prevented collective I/O in the local process
[out]	global_no_collective_cause	An enumerated set value indicating the causes across all processes that prevented collective I/O

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

Motivation:

A user can request collective I/O via a data transfer property list (DXPL) that has been suitably modified with H5P_SET_DXPL_MPIO. However, there are conditions that can cause HDF5 to forgo collective I/O and perform independent I/O. Such causes can be different across the processes of a parallel application. This function allows the user to determine what caused the HDF5 library to skip collective I/O locally, that is in the local process, and globally, across all processes.

H5Pget_mpio_no_collective_cause() serves two purposes. It can be used to determine whether collective I/O was used for the last preceding parallel I/O call. If collective I/O was not used, the function retrieves the local and global causes that broke collective I/O on that parallel I/O call. The properties retrieved by this function are set before I/O takes place and are retained even when I/O fails.

Valid values returned in local_no_collective_cause and global_no_collective_cause are as follows or, if there are multiple causes, a bitwise OR of the relevant causes; the numbers in the center column are the bitmask values:

 
typedef enum H5D_mpio_no_collective_cause_t {
    H5D_MPIO_COLLECTIVE = 0x00,
    H5D_MPIO_SET_INDEPENDENT = 0x01,
    H5D_MPIO_DATATYPE_CONVERSION = 0x02,
    H5D_MPIO_DATA_TRANSFORMS = 0x04,
    H5D_MPIO_MPI_OPT_TYPES_ENV_VAR_DISABLED = 0x08,
    H5D_MPIO_NOT_SIMPLE_OR_SCALAR_DATASPACES = 0x10,
    H5D_MPIO_NOT_CONTIGUOUS_OR_CHUNKED_DATASET = 0x20,
    H5D_MPIO_FILTERS = 0x40
} H5D_mpio_no_collective_cause_t;

(Click on a enumerator, field, or type for more information.)

Attention

Each process determines whether it can perform collective I/O and broadcasts the result. Those results are combined to make a collective decision; collective I/O will be performed only if all processes can perform collective I/O.
If collective I/O was not used, the causes that prevented it are reported by individual process by means of an enumerated set. The causes may differ among processes, so H5Pget_mpio_no_collective_cause() returns two property values. The first value is the one produced by the local process to report local causes. This local information is encoded in an enumeration, the H5D_mpio_no_collective_cause_t described above, with all individual causes combined into a single enumeration value by means of a bitwise OR operation. The second value reports global causes; this global value is the result of a bitwise-OR operation across the values returned by all the processes.

Since

1.8.10

H5Pget_preserve()

int H5Pget_preserve

(

hid_t

plist_id

)

Checks status of the dataset transfer property list (DEPRECATED)

Deprecated:

{H5Pget_preserve() is deprecated as it is no longer useful; compound datatype field preservation is now core functionality in the HDF5 library.}

Parameters

[in]

plist_id

Identifier for the dataset transfer property list

Returns

Returns 1 or 0 if successful; otherwise returns a negative value.

H5Pget_preserve() checks the status of the dataset transfer property list.

Since

1.0.0

Version

1.6.0 The flag parameter was changed from INTEGER to LOGICAL to better match the C API. (Fortran 90)

1.8.2 Deprecated.

H5Pget_type_conv_cb()

herr_t H5Pget_type_conv_cb	(	hid_t	dxpl_id,
		H5T_conv_except_func_t *	op,
		void **	operate_data
	)

Gets user-defined datatype conversion callback function.

Parameters

[in]	dxpl_id	Dataset transfer property list identifier
[out]	op	User-defined type conversion callback function
[out]	operate_data	User-defined input data for the callback function

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

H5Pget_type_conv_cb() gets the user-defined datatype conversion callback function op in the dataset transfer property list dxpl_id.

The parameter operate_data is a pointer to user-defined input data for the callback function.

The callback function op defines the actions an application is to take when there is an exception during datatype conversion.

Please refer to the function H5Pset_type_conv_cb() for more details.

Since

1.8.0

H5Pget_vlen_mem_manager()

herr_t H5Pget_vlen_mem_manager	(	hid_t	plist_id,
		H5MM_allocate_t *	alloc_func,
		void **	alloc_info,
		H5MM_free_t *	free_func,
		void **	free_info
	)

Gets the memory manager for variable-length datatype allocation in H5Dread() and H5Dvlen_reclaim()

Parameters

[in]	plist_id	Identifier for the dataset transfer property list
[out]	alloc_func	User's allocate routine, or NULL for system malloc
[out]	alloc_info	Extra parameter for user’s allocation routine. Contents are ignored if preceding parameter is NULL
[out]	free_func	User's free routine, or NULL for system free
[out]	free_info	Extra parameter for user’s free routine. Contents are ignored if preceding parameter is NULL

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

H5Pget_vlen_mem_manager() is the companion function to H5Pset_vlen_mem_manager(), returning the parameters set by that function.

Since

1.0.0

H5Pset_btree_ratios()

herr_t H5Pset_btree_ratios	(	hid_t	plist_id,
		double	left,
		double	middle,
		double	right
	)

Sets B-tree split ratios for a dataset transfer property list.

Parameters

[in]	plist_id	The dataset transfer property list identifier
[in]	left	The B-tree split ratio for left-most nodes
[in]	middle	The B-tree split ratio for all other nodes
[in]	right	The B-tree split ratio for right-most nodes and lone nodes

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

H5Pset_btree_ratios() sets the B-tree split ratios for a dataset transfer property list. The split ratios determine what percent of children go in the first node when a node splits.

The ratio left is used when the splitting node is the left-most node at its level in the tree; the ratio right is used when the splitting node is the right-most node at its level; and the ratio middle is used for all other cases.

A node that is the only node at its level in the tree uses the ratio right when it splits.

All ratios are real numbers between 0 and 1, inclusive.

H5Pset_buffer()

herr_t H5Pset_buffer	(	hid_t	plist_id,
		size_t	size,
		void *	tconv,
		void *	bkg
	)

Sets type conversion and background buffers.

Parameters

[in]	plist_id	Dataset transfer property list identifier
[in]	size	Size, in bytes, of the type conversion and background buffers
[in]	tconv	Pointer to application-allocated type conversion buffer
[in]	bkg	Pointer to application-allocated background buffer

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

Given a dataset transfer property list, H5Pset_buffer() sets the maximum size for the type conversion buffer and background buffer and optionally supplies pointers to application-allocated buffers. If the buffer size is smaller than the entire amount of data being transferred between the application and the file, and a type conversion buffer or background buffer is required, then strip mining will be used.

Note that there are minimum size requirements for the buffer. Strip mining can only break the data up along the first dimension, so the buffer must be large enough to accommodate a complete slice that encompasses all of the remaining dimensions. For example, when strip mining a 100x200x300 hyperslab of a simple data space, the buffer must be large enough to hold 1x200x300 data elements. When strip mining a 100x200x300x150 hyperslab of a simple data space, the buffer must be large enough to hold 1x200x300x150 data elements.

If tconv and/or bkg are null pointers, then buffers will be allocated and freed during the data transfer.

The default value for the maximum buffer is 1 MiB.

Version

1.6.0 The size parameter has changed from type hsize_t to size_t.

1.4.0 The size parameter has changed to type hsize_t.

H5Pset_data_transform()

herr_t H5Pset_data_transform	(	hid_t	plist_id,
		const char *	expression
	)

Sets a data transform expression.

Parameters

[in]	plist_id	Dataset transfer property list identifier
[in]	expression	Pointer to the null-terminated data transform expression

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

H5Pset_data_transform() sets the data transform to be used for reading and writing data. This function operates on the dataset transfer property list plist_id.

The expression parameter is a string containing an algebraic expression, such as (5/9.0)*(x-32) or x*(x-5). When a dataset is read or written with this property list, the transform expression is applied with the x being replaced by the values in the dataset. When reading data, the values in the file are not changed and the transformed data is returned to the user.

Data transforms can only be applied to integer or floating-point datasets. Order of operations is obeyed and the only supported operations are +, -, *, and /. Parentheses can be nested arbitrarily and can be used to change precedence. When writing data back to the dataset, the transformed data is written to the file and there is no way to recover the original values to which the transform was applied.

Since

1.8.0

H5Pset_dxpl_mpio()

H5_DLL herr_t H5Pset_dxpl_mpio	(	hid_t	dxpl_id,
		H5FD_mpio_xfer_t	xfer_mode
	)

Sets data transfer mode.

Parameters

[in]	dxpl_id	Dataset transfer property list identifier
[in]	xfer_mode	Transfer mode

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

H5Pset_dxpl_mpio() sets the data transfer property list dxpl_id to use transfer mode xfer_mode. The property list can then be used to control the I/O transfer mode during data I/O operations.

Valid transfer modes are H5FD_MPIO_INDEPENDENT (default) and H5FD_MPIO_COLLECTIVE.

Since

1.4.0

H5Pset_dxpl_mpio_chunk_opt()

H5_DLL herr_t H5Pset_dxpl_mpio_chunk_opt	(	hid_t	dxpl_id,
		H5FD_mpio_chunk_opt_t	opt_mode
	)

Sets a flag specifying linked-chunk I/O or multi-chunk I/O.

Parameters

[in]	dxpl_id	Dataset transfer property list identifier
[in]	opt_mode	Transfer mode

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

H5Pset_dxpl_mpio_chunk_opt() specifies whether I/O is to be performed as linked-chunk I/O or as multi-chunk I/O. This function overrides the HDF5 library's internal algorithm for determining which mechanism to use.

When an application uses collective I/O with chunked storage, the HDF5 library normally uses an internal algorithm to determine whether that I/O activity should be conducted as one linked-chunk I/O or as multi-chunk I/O. H5Pset_dxpl_mpio_chunk_opt() is provided so that an application can override the library's algorithm in circumstances where the library might lack the information needed to make an optimal decision.

H5Pset_dxpl_mpio_chunk_opt() works by setting one of the following flags in the parameter opt_mode:

• H5FD_MPIO_CHUNK_ONE_IO - Do one-link chunked I/O
• H5FD_MPIO_CHUNK_MULTI_IO - Do multi-chunked I/O

This function works by setting a corresponding property in the dataset transfer property list dxpl_id.

The library performs I/O in the specified manner unless it determines that the low-level MPI IO package does not support the requested behavior; in such cases, the HDF5 library will internally use independent I/O.

Use of this function is optional.

Since

1.8.0

H5Pset_dxpl_mpio_chunk_opt_num()

H5_DLL herr_t H5Pset_dxpl_mpio_chunk_opt_num	(	hid_t	dxpl_id,
		unsigned	num_chunk_per_proc
	)

Sets a numeric threshold for linked-chunk I/O.

Parameters

[in]	dxpl_id	Dataset transfer property list identifier
[in]	num_chunk_per_proc

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

H5Pset_dxpl_mpio_chunk_opt_num() sets a numeric threshold for the use of linked-chunk I/O.

The library will calculate the average number of chunks selected by each process when doing collective access with chunked storage. If the number is greater than the threshold set in num_chunk_per_proc, the library will use linked-chunk I/O; otherwise, a separate I/O process will be invoked for each chunk (multi-chunk I/O).

Since

1.8.0

H5Pset_dxpl_mpio_chunk_opt_ratio()

H5_DLL herr_t H5Pset_dxpl_mpio_chunk_opt_ratio	(	hid_t	dxpl_id,
		unsigned	percent_num_proc_per_chunk
	)

Sets a ratio threshold for collective I/O.

Parameters

[in]	dxpl_id	Dataset transfer property list identifier
[in]	percent_num_proc_per_chunk

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

H5Pset_dxpl_mpio_chunk_opt_ratio() sets a threshold for the use of collective I/O based on the ratio of processes with collective access to a dataset with chunked storage. The decision whether to use collective I/O is made on a per-chunk basis.

The library will calculate the percentage of the total number of processes, the ratio, that hold selections in each chunk. If that percentage is greater than the threshold set in percent_proc_per_chunk, the library will do collective I/O for this chunk; otherwise, independent I/O will be done for the chunk.

Since

1.8.0

H5Pset_dxpl_mpio_collective_opt()

H5_DLL herr_t H5Pset_dxpl_mpio_collective_opt	(	hid_t	dxpl_id,
		H5FD_mpio_collective_opt_t	opt_mode
	)

Sets data transfer mode.

Parameters

[in]	dxpl_id	Dataset transfer property list identifier
[in]	opt_mode	Transfer mode

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

H5Pset_dxpl_mpio() sets the data transfer property list dxpl_id to use transfer mode xfer_mode. The property list can then be used to control the I/O transfer mode during data I/O operations.

Valid transfer modes are H5FD_MPIO_INDEPENDENT (default) and H5FD_MPIO_COLLECTIVE.

Since

1.4.0

H5Pset_edc_check()

herr_t H5Pset_edc_check	(	hid_t	plist_id,
		H5Z_EDC_t	check
	)

Sets the dataset transfer property list to enable or disable error detection when reading data.

Parameters

[in]	plist_id	Dataset transfer property list identifier
[in]	check	Specifies whether error checking is enabled or disabled for dataset read operations

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

H5Pset_edc_check() sets the dataset transfer property list plist to enable or disable error detection when reading data.

Whether error detection is enabled or disabled is specified in the check parameter. Valid values are H5Z_ENABLE_EDC (default) and H5Z_DISABLE_EDC.

Note

The initial error detection implementation, Fletcher32 checksum, supports error detection for chunked datasets only.

Attention

The Fletcher32 EDC checksum filter, set with H5Pset_fletcher32(), was added in HDF5 Release 1.6.0. In the original implementation, however, the checksum value was calculated incorrectly on little-endian systems. The error was fixed in HDF5 Release 1.6.3.
As a result of this fix, an HDF5 library of Release 1.6.0 through Release 1.6.2 cannot read a dataset created or written with Release 1.6.3 or later if the dataset was created with the checksum filter and the filter is enabled in the reading library. (Libraries of Release 1.6.3 and later understand the earlier error and compensate appropriately.)
Work-around: An HDF5 library of Release 1.6.2 or earlier will be able to read a dataset created or written with the checksum filter by an HDF5 library of Release 1.6.3 or later if the checksum filter is disabled for the read operation. This can be accomplished via an H5Pset_edc_check() call with the value H5Z_DISABLE_EDC in the second parameter. This has the obvious drawback that the application will be unable to verify the checksum, but the data does remain accessible.

Version

1.6.3 Error in checksum calculation on little-endian systems corrected in this release.

Since

1.6.0

H5Pset_filter_callback()

herr_t H5Pset_filter_callback	(	hid_t	plist_id,
		H5Z_filter_func_t	func,
		void *	op_data
	)

Sets user-defined filter callback function.

Parameters

[in]	plist_id	Dataset transfer property list identifier
[in]	func	User-defined filter callback function
[in]	op_data	User-defined input data for the callback function

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

H5Pset_filter_callback() sets the user-defined filter callback function func in the dataset transfer property list plist_id.

The parameter op_data is a pointer to user-defined input data for the callback function and will be passed through to the callback function.

The callback function func defines the actions an application is to take when a filter fails. The function prototype is as follows:

 
typedef H5Z_cb_return_t (*H5Z_filter_func_t)(H5Z_filter_t filter, void *buf, size_t buf_size, void *op_data);

where filter indicates which filter has failed, buf and buf_size are used to pass in the failed data, and op_data is the required input data for this callback function.

Valid callback function return values are H5Z_CB_FAIL and H5Z_CB_CONT.

Since

1.6.0

H5Pset_hyper_vector_size()

herr_t H5Pset_hyper_vector_size	(	hid_t	plist_id,
		size_t	size
	)

Sets number of I/O vectors to be read/written in hyperslab I/O.

Parameters

[in]	plist_id	Dataset transfer property list identifier
[in]	size	Number of I/O vectors to accumulate in memory for I/O operations Must be greater than 1 (one) Default value: 1024

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

H5Pset_hyper_vector_size() sets the number of I/O vectors to be accumulated in memory before being issued to the lower levels of the HDF5 library for reading or writing the actual data.

The I/O vectors are hyperslab offset and length pairs and are generated during hyperslab I/O.

The number of I/O vectors is passed in size to be set in the dataset transfer property list plist_id. size must be greater than 1 (one).

H5Pset_hyper_vector_size() is an I/O optimization function; increasing vector_size should provide better performance, but the library will use more memory during hyperslab I/O. The default value of size is 1024.

Since

1.6.0

H5Pset_preserve()

herr_t H5Pset_preserve	(	hid_t	plist_id,
		hbool_t	status
	)

Sets the dataset transfer property list status.

Parameters

[in]	plist_id	Dataset transfer property list identifier
[in]	status	Status toggle of the dataset transfer property list

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

Deprecated:

This function is deprecated as it no longer has any effect; compound datatype field preservation is now core functionality in the HDF5 library.

H5Pset_preserve() sets the dataset transfer property list status to 1 or 0.

When reading or writing compound datatypes and the destination is partially initialized and the read/write is intended to initialize the other members, one must set this property to 1. Otherwise the I/O pipeline treats the destination datapoints as completely uninitialized.

Since

1.0.0

Version

1.8.2 Deprecated.

H5Pset_type_conv_cb()

herr_t H5Pset_type_conv_cb	(	hid_t	dxpl_id,
		H5T_conv_except_func_t	op,
		void *	operate_data
	)

Sets user-defined datatype conversion callback function.

Parameters

[in]	dxpl_id	Dataset transfer property list identifier
[in]	op	User-defined type conversion callback function
[in]	operate_data	User-defined input data for the callback function

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

H5Pset_type_conv_cb() sets the user-defined datatype conversion callback function op in the dataset transfer property list dxpl_id

The parameter operate_data is a pointer to user-defined input data for the callback function and will be passed through to the callback function.

The callback function op defines the actions an application is to take when there is an exception during datatype conversion. The function prototype is as follows:

 
typedef H5T_conv_ret_t (*H5T_conv_except_func_t)(H5T_conv_except_t except_type, hid_t src_id, hid_t dst_id,
                                                 void *src_buf, void *dst_buf, void *user_data);

Since

1.8.0

H5Pset_vlen_mem_manager()

herr_t H5Pset_vlen_mem_manager	(	hid_t	plist_id,
		H5MM_allocate_t	alloc_func,
		void *	alloc_info,
		H5MM_free_t	free_func,
		void *	free_info
	)

Sets the memory manager for variable-length datatype allocation in H5Dread() and H5Dvlen_reclaim()

Parameters

[in]	plist_id	Dataset transfer property list identifier
[in]	alloc_func	User's allocate routine, or NULL for system malloc
[in]	alloc_info	Extra parameter for user's allocation routine. Contents are ignored if preceding parameter is NULL.
[in]	free_func	User's free routine, or NULL for system free
[in]	free_info	Extra parameter for user's free routine. Contents are ignored if preceding parameter is NULL

Returns

Returns a non-negative value if successful; otherwise returns a negative value.

H5Pset_vlen_mem_manager() sets the memory manager for variable-length datatype allocation in H5Dread() and free in H5Dvlen_reclaim().

The alloc_func and free_func parameters identify the memory management routines to be used. If the user has defined custom memory management routines, alloc_func and/or free_func should be set to make those routine calls (i.e., the name of the routine is used as the value of the parameter); if the user prefers to use the system's malloc and/or free, the alloc_func and free_func parameters, respectively, should be set to NULL

The prototypes for these user-defined functions are as follows:

typedef void *(*H5MM_allocate_t)(size_t size, void *alloc_info);

typedef void (*H5MM_free_t)(void *mem, void *free_info);

The alloc_info and free_info parameters can be used to pass along any required information to the user's memory management routines.

In summary, if the user has defined custom memory management routines, the name(s) of the routines are passed in the alloc_func and free_func parameters and the custom routines' parameters are passed in the alloc_info and free_info parameters. If the user wishes to use the system malloc and free functions, the alloc_func and/or free_func parameters are set to NULL and the alloc_info and free_info parameters are ignored.

Since

1.0.0