To write to an existing SDS, the calling program must contain the following sequence of routine calls:
C: sds_id = SDselect(sd_id, sds_index);
status = SDwritedata(sds_id, start, stride, edges, data);
FORTRAN: sds_id = sfselect(sd_id, sds_index)
status = sfwdata(sds_id, start, stride, edges, data)
OR status = sfwcdata(sds_id, start, stride, edges, data)
To write to a new SDS, simply replace the call SDselect with the call SDcreate, which is described in Section 3.4.1 on page 25.
SDwritedata takes five arguments:
sds_id
, start
, stride
, edges
, and data
. The argument sds_id
is the data set identifier returned by SDcreate or SDselect. start
is a one-dimensional array specifying the location in the SDS array at which the write operation will begin. The values of each element of the array start
are relative to 0 in both the C and FORTRAN-77 interfaces. The size of start
must be the same as the number of dimensions in the SDS array. In addition, each value in start
must be smaller than its corresponding SDS array dimension unless the dimension is unlimited. Violating any of these conditions causes SDwritedata to return FAIL
.stride
is a one-dimensional array specifying, for each dimension, the interval between values to be written. For example, setting the first element of the array stride
equal to 1 writes data to every location along the first dimension. Setting the first element of the array stride
to 2 writes data to every other location along the first dimension. Figure 3b illustrates this example, where the shading elements are written and the white elements are skipped. If the argument stride
is set to NULL
in C (or either 0
or 1
in FORTRAN-77), SDwritedata operates as if every element of stride
contains a value of 1, and a contiguous write is performed. For better performance, it is recommended that the value of stride
be defined as NULL
(i.e., 0
or 1
in FORTRAN-77) rather than being set to 1.
The size of the array
stride
must be the same as the number of dimensions in the SDS array. Also, each value in stride
must be smaller than or equal to its corresponding SDS array dimension unless the dimension is unlimited. Violating any of these conditions causes SDwritedata to return FAIL
.FIGURE 3b - An Example of Access Pattern ("Strides")
edges
is a one-dimensional array specifying the length of each dimension of the slab to be written. If the slab has fewer dimensions than the SDS data set has, the size of edges
must still be equal to the number of dimensions in the SDS array and all the elements corresponding to the additional dimensions must be set to 1.edges
must not be larger than the length of the corresponding dimension in the SDS data set unless the dimension is unlimited. Attempting to write slabs larger than the size of the SDS data set will result in an error condition. edges
and the corresponding value in the start
array must be smaller than or equal to its corresponding SDS array dimension unless the dimension is unlimited. Violating any of these conditions causes SDwritedata to return FAIL
.data
contains the SDS data to be written. If the SDS array is smaller than the buffer data
, the amount of data written will be limited to the maximum size of the SDS array.SUCCEED
(or 0
) or FAIL
(or -1
). The parameters of this routine are described in Table 3D.
TABLE 3D - SDwritedata Parameter List
start
are set to 0
, the argument stride
is set equal to NULL
in C or each element of the array stride
is set to 1
in both C and FORTRAN-77, and each element of the array edges
is equal to the length of each dimension.
C version
3.5.1.2 Writing Slabs to an SDS Array
To allow preexisting data to be modified, the HDF library does not prevent SDwritedata from overwriting one slab with another. As a result, the calling program is responsible for managing any overlap when writing slabs. The HDF library will issue an error if a slab extends past the valid boundaries of the SDS array. However, appending data along an unlimited dimension is allowed.EXAMPLE 3. Writing a Slab of Data to an SDS.
This example shows how to fill a 3-dimensional SDS array with data by writing series of 2-dimensional slabs to it.
EXAMPLE 4. Altering Values within an SDS Array.
This example demonstrates how the routine SDwritedata can be used to alter the values of the elements in the 10th and 11th rows, at the 2nd column, in the SDS array created in the Example 1 and written in Example 2. FORTRAN-77 routine sfwdata is used to alter the elements in the 2nd row, 10th and 11th columns, to reflect the difference between C and Fortran internal storage.
3.5.1.3 Appending Data to an SDS Array along an Unlimited Dimension
An SDS array can be made appendable, however, only along one dimension. This dimension must be specified as an appendable dimension when it is created.dim_sizes
(i.e., the dimension of the lowest rank or the slowest-changing dimension) can be assigned the value SD_UNLIMITED
(or 0
) to make the first dimension unlimited. In FORTRAN-77, only the last
dimension (i.e., the dimension of the highest rank or the slowest-changing dimension) can be unlimited. In other words, in FORTRAN-77 dim_sizes(rank)
must be set to the value SD_UNLIMITED
to make the last dimension appendable. start
parameter of the SDwritedata routine. For example, if 15 data elements have been written to an unlimited dimension, appending data to the array requires a start
coordinate of 15. Specifying a starting coordinate less than the current number of elements written to the unlimited dimension will result in data being overwritten. In either case, all of the coordinates in the array except the one corresponding to the unlimited dimension must be equal to or less than the lengths of their corresponding dimensions.
3.5.1.4 Determining whether an SDS Array is Appendable: SDisrecord
SDisrecord determines whether the data set identified by the parameter sds_id
is appendable, which means that the slowest-changing dimension of the SDS array is declared unlimited when the data set is created. The syntax of SDisrecord is as follows:C: status = SDisrecord(sds_id);
FORTRAN: status = sfisrcrd(sds_id)
SDisrecord returns TRUE
(or 1
) when the data set specified by sds_id
is appendable and FALSE
(or 0
) otherwise. The parameter of this routine is defined in Table 3E.
TABLE 3E - SDisrecord Parameter List
|
|
|
| |
|
| |||
|
|
|
Data set identifier
|
C: status = SDsetblocksize(sds_id, block_size);
FORTRAN: status = sfsblsz(sds_id, block_size)SDsetblocksize must be called after SDcreate or SDselect and before SDwritedata. The parameter
block_size
should be set to a multiple of the desired buffer size.SDsetblocksize returns a value of
SUCCEED
(or 0
) or FAIL
(or -1
). Its parameters are further described in Table 3F.
TABLE 3F - SDsetblocksize Parameter List
|
|
|
| |
|
| |||
|
|
|
Data set identifier
| |
|
|
|
Block size
|
C version
3.5.2 Compressing SDS Data: SDsetcompress
The SDsetcompress routine compresses an existing data set or creates a new compressed data set. It is a simplified interface to the HCcreate routine, and should be used instead of HCcreate unless the user is familiar with the lower-level routines.
In the future, the following algorithms may be included: Lempel/Ziv-78 dictionary coding, an arithmetic coder, and a faster Huffman algorithm.
C: status = SDsetcompress(sds_id, comp_type, &c_info);
FORTRAN: status = sfscompress(sds_id, comp_type, comp_prm)
The parameter comp_type
specifies the compression type definition and is set to COMP_CODE_RLE
(or 1
) for run-length encoding (RLE), COMP_CODE_SKPHUFF
(or 3
) for Skipping Huffman, COMP_CODE_DEFLATE
(or 4
) for GZIP compression, or COMP_CODE_NONE
(or 0
) for no compression.c_info
in C, and by the parameter comp_prm
in FORTRAN-77. The parameter c_info
is a pointer to a union structure of type comp_info.
(Refer to the SDsetcompress entry in the HDF Reference Manual for the description of the comp_info
structure.) If comp_type
is set to COMP_CODE_NONE
or COMP_CODE_RLE
, the parameters c_info
and comp_prm
are not used; c_info
can be set to NULL
and comp_prm
can be undefined. If comp_type
is set to COMP_CODE_SKPHUFF
, then the structure skphuff
in the union comp_info
in C (comp_prm(1)
in FORTRAN-77) must be provided with the size, in bytes, of the data elements. If it is set to COMP_CODE_DEFLATE
, the deflate structure in the union comp_info
in C (comp_prm(1)
in FORTRAN-77) must be provided with the information about the compression effort.C: comp_info c_info;
c_info.skphuff.skp_size = sizeof(int16);
status = SDsetcompress(sds_id, COMP_CODE_SKPHUFF, &c_info);
FORTRAN: comp_prm(1) = 2
COMP_CODE_SKPHUFF = 3
status = sfscompress(sds_id, COMP_CODE_SKPHUFF, comp_prm)
To compress a data set using the gzip deflation algorithm with the maximum effort specified, the following definition and SDsetcompress call are used.C: comp_info c_info;
c_info.deflate.level = 9;
status = SDsetcompress(sds_id, COMP_CODE_DEFLATE, &c_info);
FORTRAN: comp_prm(1) = 9
COMP_CODE_DEFLATE = 4
status = sfscompress(sds_id, COMP_CODE_DEFLATE, comp_prm)
SDsetcompress functionality is currently limited to the following:
The existing compression algorithms supported by HDF do not allow partial modification to a compressed datastream. Overwriting the contents of existing data sets may be supported in the future. Note also that SDsetcompress performs the compression of the data, not SDwritedata.
SUCCEED
(or 0
) or FAIL
(or -1
). The C version parameters are further described in Table 3G and the FORTRAN-77 version parameters are further described in Table 3H.
TABLE 3G - SDsetcompress Parameter List
|
|
|
|
| |||
|
|
Data set identifier
| |
|
|
Compression method
| |
|
|
Pointer to compression information structure
|
|
|
|
|
| |||
|
|
Data set identifier
| |
|
|
Compression method
| |
|
|
Compression parameters array
| |
EXAMPLE 6. Compressing SDS Data.
This example uses the routine SDsetcompress/sfscompress to compress SDS data with the GZIP compression method. See comments in the program regarding the use of the Skipping Huffman or RLE compression methods.
3.5.3 External File Operations
The HDF library provides routines to store SDS arrays in an external file that is separate from the primary file containing the metadata for the array. Such an SDS array is called an external SDS array. With external arrays, it is possible to link data sets in the same HDF file to multiple external files or data sets in different HDF files to the same external file. 3.5.3.1 Specifying the Directory Search Path of an External File: HXsetdir
There are three filesystem locations the HDF external file routines check when determining the location of an external file. They are, in order of search precedence:
The syntax of HXsetdir is as follows:
C: status = HXsetdir(dir_list);
FORTRAN: status = hxisdir(dir_list, dir_length)
HXsetdir has one argument, a string specifying the directory list to be searched. This list can consist of one directory name or a set of directory names separated by colons. The FORTRAN-77 version of this routine takes an additional argument, dir_length
, which specifies the length of the directory list string.SUCCEED
(or 0
) or FAIL
(or -1
). The parameters of HXsetdir are described in Table 3I on page 35.
3.5.3.2 Specifying the Location of the Next External File to be Created: HXsetcreatedir
HXsetcreatedir specifies the directory location of the next external file to be created. It overrides the directory location specified by $HDFEXTCREATEDIR and the locations searched by the open(3) call in the same manner as HXsetdir. Specifically, the search precedence is:
The syntax of HXsetcreatedir is as follows:
C: status = HXsetcreatedir(dir);
FORTRAN: status = hxiscdir(dir, dir_length)
HXsetcreatedir has one argument, the directory location of the next external file to be created. The FORTRAN-77 version of this routine takes an additional argument, dir_length
, which specifies the length of the directory list string. If an error is encountered, the directory location is left unchanged.SUCCEED
(or 0
) or FAIL
(or -1
). The parameters of HXsetcreatedir are described in Table 3I.
TABLE 3I - HXsetdir and HXsetcreatedir Parameter Lists
To create a data set with data stored in an external file, the calling program must make the following calls.
C: sds_id = SDcreate(sd_id, name, data_type, rank, dim_sizes);
status = SDsetexternalfile(sds_id, filename, offset);
status = SDwritedata(sds_id, start, stride, edges, data);
status = SDendaccess(sds_id);
FORTRAN: sds_id = sfcreate(sd_id, name, data_type, rank, dim_sizes)
status = sfsextf(sds_id, filename, offset)
status = sfwdata(sds_id, start, stride, edges, data)
OR status = sfwcdata(sds_id, start, stride, edges, data)
status = sfendacc(sds_id)
For a newly-created data set, SDsetexternalfile marks the SDS identified by sds_id
as one whose data is to be written to an external file. It does not actually write data to an external file; it marks the data set as an external data set for all subsequent SDwritedata operations.filename
is the name of the external data file and offset
is the number of bytes from the beginning of the external file to the location where the first byte of data should be written. If a file with the name specified by filename
exists in the current directory search path, HDF will access it as the external file. If the file does not exist, HDF will create one in the directory named in the last call to HXsetcreatefile. If an absolute pathname is specified, the external file will be created at the location specified by the pathname, overriding the location specified by the last call to HXsetcreatefile. Use caution when writing to existing external or primary files since the HDF library starts the write operation at the specified offset without determining whether data is being overwritten.SUCCEED
(or 0
) or FAIL
(or -1
). The parameters of SDsetexternalfile are described in Table 3J.
TABLE 3J - SDsetexternalfile Parameter List
To move data set data to an external file, the calling program must make the following calls:
C: sds_id = SDselect(sd_id, sds_index);
status = SDsetexternalfile(sds_id, filename, offset);
status = SDendaccess(sds_id);
FORTRAN: sds_id = sfselect(sd_id, sds_index)
status = sfsextf(sds_id, filename, offset)
status = sfendacc(sds_id)
For an existing data set, SDsetexternalfile moves the data to the external file. Any data in the external file that occupies the space reserved for the external array will be overwritten as a result of this operation. Data of an existing data set in the primary file can only be moved to the external file once. During the operation, the data is written to the external file as a contiguous stream regardless of how it is stored in the primary file. Because data is moved as is, any unwritten locations in the data set are preserved in the external file. Subsequent read and write operations performed on the data set will access the external file.EXAMPLE 7. Moving Data to the External File.
This example illustrates the use of the routine SDsetexternalfile/sfsextf to move the SDS data written in Example 2 to the external file.