As described in Progressive Retrieval and Hierarchical Visualization of Large Remote Data we identified two HDF5 high-level operations to be executed on the data server. We use a development version of the striped GridFTP server developed in the Globus project for which we wrote a hdf5 plugin. You cand find the server source code at the bottom of the page and install instructions in this README.

Even if the number of use cases for the remote HDF5 library is limited (the main use case is the HDF5 GridFTP I/O driver) we provide the source code (download at the bottom of the page) and a documentation for this library.

We modified the HDF5 I/O pipeline in order to allow the efficient execution of remote H5Dread in the low-level I/O driver. In order to reduce overall latency, we execute the data gather operation in the low-level driver. Otherwise the operation is split in many (potentially small) low-level read operation, resulting in a very large execution time. We also added a couple of useful methods in HDF5. An overview is included in this automatically generated documentation.

The changes described above make the implementation of an H5Dread iterator in the I/O driver possible. Also, we combine all the other read operations in a single meta-operation. More details in the documentation

All downloads are signed with Steffen Prohaska's GPG key (0xDA749299). After downloading and importing the key or retreiving it from a keyserver you can verify the uncompressed archive. Run

  $ [gpg --import prohaska.pgp | gpg --search-keys 0xDA749299]
  $ gunzip thearchive.tar.gz
  $ gpg --verify thearchive.tar.asc
  gpg: Signature made ... using DSA key ID DA749299
  gpg: Good signature from "Steffen Prohaska ..."

The following downloads are available:

• Modified GridFTP server (20040506 snapshot) -- gpg signature
• Client library (20040506 snapshot) -- gpg signature
• HDF5 patch (for HDF5 version 1.6.2-post3) (20040506 snapshot) -- gpg signature
• HDF5 GridFTP I/O driver (including client library) (20040506 snapshot) -- gpg signature
• A Patched HDF5 including GridFTP I/O driver is not yet available.
• Old versions might still be available in the download directory

• Hans-Christian Hege, Andrei Hutanu, Ralf Kähler, André Merzky, Thomas Radke, Edward Seidel, Brygg Ullmer. Progressive Retrieval and Hierarchical Visualization of Large Remote Data. Proceedings of the 2003 Workshop on Adaptive Grid Middleware, p. 60–72, 2003.