IRIS: I/O Redirection via Integrated Storage
Abstract
There is an ocean of available storage solutions in modern high-performance and distributed systems. These solutions consist of Parallel File Systems (PFS) for the more traditional high-performance computing (HPC) systems and of Object Stores for emerging cloud environments. More of ten than not, these storage solutions are tied to specific APIs and data models and thus, bind developers, applications, and entire computing facilities to using certain interfaces. Each storage system is designed and optimized for certain applications but does not perform well for others. Furthermore, modern applications have become more and more complex consisting of a collection of phases with different computation and I/O requirements. In this paper, we propose a unified storage access system, called IRIS (i.e., I/O Redirection via Integrated Storage). IRIS enables unified data access and seamlessly bridges the semantic gap between file systems and object stores. With IRIS, emerging High-Performance Data Analytics software has capable and diverse I/O support. IRIS can bring us closer to the convergence of HPC and Cloud environments by combining the best storage subsystems from both worlds. Experimental results show that IRIS can grant more than 7x improvement in performance than existing solutions.
Finding
IRIS bridges the semantic gap between file systems and object stores and can grant more than 7x better performance than existing solutions.
System
-
IRIS
A unified I/O layer over POSIX files and object stores, from my doctoral work.
- 2023–2026
More papers on IRIS
Browse all papers on IRIS- 2022
LuxIO: Intelligent Resource Provisioning and Auto-Configuration for Storage Services
Keith Bateman, Neeraj Rajesh, Jaime Cernuda Garcia, Luke Logan, Jie Ye, … Anthony Kougkas et al. Proceedings of the 29th International Conference on High Performance Computing, Data, and Analytics
- 2022
LabStor: A modular and extensible platform for developing high-performance, customized I/O stacks in userspace
Luke Logan, Jaime Cernuda Garcia, Jay Lofstead, Xian–He Sun, Anthony Kougkas. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis
- 2022
Stimulus: Accelerate Data Management for Scientific AI applications in HPC
Hariharan Devarajan, Anthony Kougkas, Huihuo Zheng, Venkatram Vishwanath, Xian-He Sun. Proceedings of the 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing
- 2021
HFlow: A Dynamic and Elastic Multi-Layered I/O Forwarder
Jaime Cernuda, Hariharan Devarajan, Luke Logan, Keith Bateman, Neeraj Rajesh, … Anthony Kougkas et al. Proceedings of the International Conference on Cluster Computing
Citation
@inproceedings{kougkas2018iris,
author = {Kougkas, Anthony and Devarajan, Hariharan and Sun, Xian-He},
booktitle = {Proceedings of the International Conference on Supercomputing},
title = {{IRIS}: {I/O} Redirection via Integrated Storage},
year = {2018},
month = jun,
pages = {33--42},
doi = {10.1145/3205289.3205322},
url = {https://dl.acm.org/doi/abs/10.1145/3205289.3205322},
} Related notes
- 9 min
One destination, N tiers
Every tier supplies the application at once, and the application sees one destination. The thesis from a November 2026 talk to my group at the Gnosis Research Center, the six lines of I/O work behind it since 2016, and my bet that they are the foundation agents need for context.