akougkas.io

IOMax: Maximizing Out-of-Core I/O Analysis Performance on HPC Systems

Izzet Yildirim, Hariharan Devarajan, Anthony Kougkas, Xian-He Sun, Kathryn Mohror

Proceedings of the SC’23 Workshops of The International Conference on High Performance Computing, Network, Storage, and AnalysisACM

Abstract

I/O analysis is an essential task for improving the performance of scientific applications on high-performance computing (HPC) systems. However, current analysis tools, which often use data drilling techniques (iterative exploration for deeper insights), treat every query independently and do not optimize column data for data-slicing (extracting specific data subsets), resulting in subpar querying performance. In this paper, we designed IOMax, a tool for efficient data drilling analysis on large-scale I/O traces. IOMax utilizes a novel query optimization technique to improve the query performance by 8.6x while reducing the memory footprint required for analysis by 11x. Additionally, it employs data transformation techniques to improve data-slicing performance by up to 11.4x. In conclusion, IOMax optimizes I/O analysis for scientific workflows on the Lassen supercomputer, resulting in up to 7x improvement.

System

  1. 2024–present
    active

    IOWarp

  2. experimental

    WisIO

Browse all papers on IOWarp
  1. 2026
    conference

    Trigger, Render, Reason: Multimodal LLM Agents for Science-Aware Scientific Simulation Monitoring

    Hua Xu, Jaime Cernuda, Ana Gainaru, Scott Klasky, Anthony Kougkas, Xian-He Sun. International Conference for High Performance Computing, Networking, Storage, and Analysis (SC 2026)

  2. 2026
    workshop

    GPUH5: A GPU-first I/O Library for Scientific Data

    J. Fernandes, Luke Logan, Xian-He Sun, Anthony Kougkas. 11th International Parallel Data Systems Workshop (PDSW 2026), held in conjunction with SC26

  3. 2026
    workshop

    Trigger-Gated Multimodal LLM Agents for In-situ Scientific Simulation Monitoring

    Hua Xu, Jaime Cernuda, Anthony Kougkas, Ana Gainaru, Scott Klasky, Xian-He Sun et al. 11th International Parallel Data Systems Workshop (PDSW 2026), held in conjunction with SC26

  4. 2026
    workshop

    Type-Safe, Dynamically Defined Metadata Support for Scientific Applications

    J. Teves, C. Brinker, Anthony Kougkas, Ana Gainaru, Scott Klasky, Xian-He Sun et al. 11th International Parallel Data Systems Workshop (PDSW 2026), held in conjunction with SC26

Citation

@inproceedings{yildirim2023iomax,
  author = {Yildirim, Izzet and Devarajan, Hariharan and Kougkas, Anthony and Sun, Xian-He and Mohror, Kathryn},
  booktitle = {Proceedings of the SC'23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis},
  title = {IOMax: Maximizing Out-of-Core I/O Analysis Performance on HPC Systems},
  year = {2023},
  month = nov,
  publisher = {ACM},
  pages = {1209--1215},
  doi = {10.1145/3624062.3624191},
  url = {https://dl.acm.org/doi/abs/10.1145/3624062.3624191},
}

Related notes

  1. essay

    One destination, N tiers

    Every tier supplies the application at once, and the application sees one destination. The thesis from a November 2026 talk to my group at the Gnosis Research Center, the six lines of I/O work behind it since 2016, and my bet that they are the foundation agents need for context.

    9 min
  2. field note

    SC26 in Chicago

    What the group brings to SC26 in Chicago, the pressure from GPUs and agent fleets that runs through all of it, and an invitation to Thursday's Birds of a Feather on HDF5 in the age of AI-driven science.

    4 min
  3. essay

    CLIO as the entry point

    CLIO should become the single place a developer starts with IOWarp, built as an application SDK with deployment profiles rather than an umbrella package or a gateway service. This is the direction for 2027; the API and the scope are still open.

    6 min