I/O Acceleration via Multi-Tiered Data Buffering and Prefetching
Abstract
Modern High-Performance Computing (HPC) systems are adding extra layers to the memory and storage hierarchy, named deep memory and storage hierarchy (DMSH), to increase I/O performance. New hardware technologies, such as NVMe and SSD, have been introduced in burst buffer installations to reduce the pressure for external storage and boost the burstiness of modern I/O systems. The DMSH has demonstrated its strength and potential in practice. However, each layer of DMSH is an independent heterogeneous system and data movement among more layers is significantly more complex even without considering heterogeneity. How to efficiently utilize the DMSH is a subject of research facing the HPC community. Further, accessing data with a high-throughput and low-latency is more imperative than ever. Data prefetching is a well-known technique for hiding read latency by requesting data before it is needed to move it from a high-latency medium (e.g., disk) to a low-latency one (e.g., main memory). However, existing solutions do not consider the new deep memory and storage hierarchy and also suffer from under-utilization of prefetching resources and unnecessary evictions. Additionally, existing approaches implement a client-pull model where understanding the application’s I/O behavior drives prefetching decisions. Moving towards exascale, where machines run multiple applications concurrently by accessing files in a workflow, a more data-centric approach resolves challenges such as cache pollution and redundancy. In this paper, we present the design and implementation of Hermes: a new, heterogeneous-aware, multi-tiered, dynamic, and distributed I/O buffering system. Hermes enables, manages, supervises, and, in some sense, extends I/O buffering to fully integrate into the DMSH. We introduce three novel data placement policies to efficiently utilize all layers and we present three novel techniques to perform memory, metadata, and communication management in hierarchical buffering systems. Additionally, we demonstrate the benefits of a truly hierarchical data prefetcher that adopts a server-push approach to data prefetching. Our evaluation shows that, in addition to automatic data movement through the hierarchy, Hermes can significantly accelerate I/O and outperforms by more than 2x state-of-the-art buffering platforms. Lastly, results show 10% to 35% performance gains over existing prefetchers and over 50% when compared to systems with no prefetching.
Finding
Hermes outperforms state-of-the-art buffering platforms by more than 2x, and its server-push prefetcher delivers 10% to 35% gains over existing prefetchers.
System
- 2018–2024
Hermes
The multi-tier I/O buffering system I created during my PhD, carried through five years of NSF funding into versioned middleware released under The HDF Group, and forward into IOWarp.
More papers on Hermes
Browse all papers on Hermes- 2026
Benchmarking the Performance of Semantic Search Techniques for Agent Discovery
Rajni Pawar, Luke Logan, Xian-He Sun, Anthony Kougkas. 13th Greater Chicago Area Systems Research Workshop (GCASR 2026)
- 2024
MegaMmap: Blurring the Boundary Between Memory and Storage for Data-Intensive Workloads
Luke Logan, Anthony Kougkas, Xian-He Sun. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis SC
- 2024
DaYu: Optimizing Distributed Scientific Workflows by Decoding Dataflow Semantics and Dynamics
Meng Tang, Jaime Cernuda, Jie Ye, Luanzheng Guo, Nathan R. Tallent, Anthony Kougkas et al. Proceedings of the IEEE International Conference on Cluster Computing
- 2024
HStream: A hierarchical data streaming engine for high-throughput scientific applications
Jaime Cernuda, Jie Ye, Anthony Kougkas, Xian-He Sun. Proceedings of the 53rd International Conference on Parallel Processing
Citation
@article{kougkas2020acceleration,
author = {Kougkas, Anthony and Devarajan, Hariharan and Sun, Xian-He},
journal = {Journal of Computer Science and Technology},
title = {I/O Acceleration via Multi-Tiered Data Buffering and Prefetching},
year = {2020},
month = jan,
publisher = {Springer},
volume = {35},
number = {1},
pages = {92--120},
doi = {10.1007/s11390-020-9781-1},
url = {https://link.springer.com/article/10.1007/s11390-020-9781-1},
} Related notes
- 9 min
One destination, N tiers
Every tier supplies the application at once, and the application sees one destination. The thesis from a November 2026 talk to my group at the Gnosis Research Center, the six lines of I/O work behind it since 2016, and my bet that they are the foundation agents need for context.
- 3 min
CLIO Core 2.3 and 2.4: when a tier fills or a node dies
Four CLIO Core releases between October 9 and November 2, 2026 added a device-paged GPU vector and Intel GPU support, then taught the context store to recover a full tier and keep data and directories readable when a node goes down.
- 4 min
Two years of IOWarp
Two years after the NSF award, what IOWarp promised, what shipped as public releases, what changed course, and what is still a plan.