Extending the HDF Library to Support
Intelligent I/O Buffering for Deep Memory and Storage Hierarchy Systems
(NSF OCI-1835764)
https://github.com/hdfgroup/hermes
This project proposes the development of Hermes, a heterogeneous aware, multi-tiered, dynamic, and distributed I/O buffering system that will significantly accelerate I/O performance.
This project proposes to extend HDF technologies with the Hermes design. Hermes is new, and the enhancement of HDF5 is new. We believe that the combination of DMSH I/O buffering and HDF technologies is a reachable practical solution that can efficiently support scientific discovery.
Principal Investigator
Illinois Tech
Technical Lead
Illinois Tech
Co-Principal Investigator
HDF Group
PhD Student SCS Lab
Illinois Tech
PhD Student SCS Lab
Illinois Tech
PhD Student SCS Lab
Illinois Tech
Software Engineer
HDF Group
That is true. We suggest using profiling tools before hand to learn about the application’s behavior and tune Hermes. Default policy works great.
As of now, applications link to Hermes (re-compile or dynamic linking). We envision a system scheduler that also incorporates buffering resources.
Hermes’ Application Orchestrator was designed for multi-tenant environments. This work is described in Vidya: Performing Code-Block I/O Characterization for Data Access Optimization.
It can be severe but in scenarios where there is some computation in between I/O then it can work nicely to our advantage.
In our evaluation, for 1 million user files, the metadata created were 1.1GB.
Hermes’ System Profiler provides the current status of the system (i.e., remaining capacity, etc) and DPE is aware of this before it places data in the DMSH.
Horizontal data movement can be in the way of the normal compute traffic. RDMA capable machines can help. We also suggest using the “service class” of the Infiniband network to apply priorities in the network.
Configurable by the user. Typical trade-off. More RAM to Hermes can lead to higher performance. No RAM means skip the layer.
Hermes captures existing I/O calls. Our own API is really simple consisting of hermes::read(…, flags) and hermes::write(…,flags). Flag system implements active buffering semantics (currently only for the burst buffer nodes).
We expose a configuration_manager class which is used to pass several Hermes’ configuration parameters.
