IOWarp
Context management infrastructure for scientific AI.
The NSF-funded context-management program I proposed and architect across GRC, The HDF Group, and the University of Utah. IOWarp connects the HPC, storage, I/O, and data-systems lineage to scientific AI through the CLIO architecture.
The question
Scientific computing spent decades getting better at moving bytes. The newer problem is deciding which bytes, which metadata, which tools, which history, and which constraints matter right now, especially once software agents take part in the workflow.
IOWarp is the program built around that problem: a context layer for scientific work, with a runtime and tools that move data across storage tiers, preserve useful state, connect capabilities, and keep the boundary between automation and human approval explicit.
Where it came from
I wrote the IOWarp proposal in 2023, at the end of Hermes. Its core was a self-describing unit called Content, a transfer engine that was Hermes grown up, an assimilation engine for reshaping data across formats, and a thrust I named WarpGPT: a natural-language interface that would answer questions about scientific data by querying a content catalog and calling operators as tools. That was before the Model Context Protocol existed. The proposal marks the transition from moving and organizing scientific data to managing the full context around scientific computation.
NSF funded it in August 2024 as a collaborative Frameworks award among Illinois Tech, The HDF Group, and the University of Utah. The work runs across the three institutions with Gerd Heber, Jacob Hochhalter, Vivek Srikumar, Xian-He Sun, and their teams.
IOWarp and CLIO
IOWarp is the research program, platform, and institutional collaboration. CLIO, Context Layer for Input/Output, is the architecture I created for it. CLIO defines the component boundaries and shared contracts through which the program becomes one software ecosystem.
What shipped
IOWarp is research infrastructure under active development. The first integrated beta shipped in September 2025, and each component has released on its own cadence since. CLIO Core is the runtime that keeps scientific data usable across GPU memory, DRAM, NVMe, and parallel file systems; its transfer engine is Hermes grown up. CLIO Kit gathers the scientific tools and Model Context Protocol servers that agents call. CLIO Agent is a workspace where a scientist and an agent work on data together, and CLIO Relay carries that work to remote clusters. CLIO Coder, at coder.iowarp.ai, is the coding agent for the software science runs on.
The team has built CLIO Core on Ares at Illinois Tech, DeltaAI at NCSA, and Polaris and Aurora at ALCF, and at the INCITE 2026 GPU hackathon showed it scaling to 512 Aurora nodes.
Shazzadul Islam’s eScience 2025 poster measured what the tools are worth: EarthScope retrieval and visualization in about two minutes with an agent that had IOWarp context, where the comparison agent did not finish; LAMMPS trajectory analysis in about 2.5 minutes against roughly 12 by hand. Rajni Pawar’s Acropolis and Izzet Yildirim’s AgentStage, both accepted at eScience 2026, are the next pieces of agent-facing storage.
In August 2026 GRC began two first-round DOE Genesis Mission projects that build on CLIO: SPOTTER-AI with Argonne, Oak Ridge, and HPE, which traces threats to scientific data through workflow provenance, and PORTUS with Pacific Northwest, which guides coding agents that port and tune scientific workflows.
My part, and the team around it
I framed the context problem, wrote the proposal, shaped the partnerships, defined the architecture and component boundaries, built the team, and remain responsible for integration, milestones, coauthored research, releases, and reporting across the program. I also write code across the software. The implementation is collective. Luke Logan leads day-to-day Core engineering and built Chimaera, the coroutine-based distributed runtime underneath it; Jaime Cernuda leads Agent and Relay; the students Isa Muradli and Shazzadul build Kit; Jie Ye, Keith Bateman, Neeraj Rajesh, Rajni, and Hua Xu develop components and research directions, as Meng Tang did until August 2026; Gerd’s engineers at The HDF Group lead format integration; Jacob and Vivek bring the materials and machine-learning domains from Utah.
What carried forward
The idea that a request should carry its intent comes from LABIOS. The task pipeline comes from DTIO, whose mechanisms were reimplemented inside CLIO Core in 2025. Time-ordered provenance comes from ChronoLog. The measurement discipline for scientific AI comes through DeepIO. IOWarp is where those lineages become a context system.
What I am asking next
- What is the smallest context contract that stays useful across scientific domains?
- Which provenance must be captured at runtime, and which can be reconstructed later?
- How do we make an agent’s authority legible before it acts, not only after something goes wrong?
- Can the same infrastructure feel natural to a scientist, a systems developer, and an automated tool?
Code, releases, papers, and records
- documentation
iowarp.ai platform hub
The CLIO stack, product portals, deployment guides, and SC26 program.
- release
CLIO Core v2.4.1, November 11, 2026
The current runtime release at core.iowarp.ai.
- release
CLIO Kit v2.12.3, November 11, 2026
The current tools release at toolkit.iowarp.ai, on PyPI as clio-kit.
- code
IOWarp organization on GitHub
The runtime, every CLIO component, documentation, and the development history behind each of them.
- paper
MegaMmap at SC 2024
The tiered-memory design that informs the Context Transfer Engine.
- record
NSF award OAC-2411318
The public award record. PI Xian-He Sun; co-PIs Kougkas, Heber, Hochhalter, and Srikumar.
- press
Illinois Tech news, September 2024
The university report on the $4M NSF grant to explore next-generation data management.
- documentation
The CLIO platform on iowarp.ai
How CLIO Agent, Coder, Kit, and Core fit together.
- record
IOWarp results and milestones
Where the program stands and where each result is published.
- record
IOWarp research and publications
The papers and posters behind the program since 2022.
- documentation
IOWarp applications in science
Materials science, federated catalogs, the DOE Genesis Mission projects, and national research systems.
- record
GRC project record
The program's page at the Gnosis Research Center.
Releases
-
CLIO Agent v0.9.6.6
lets the agent work on attached folders of any size where they already are, reads files with unusual names from connected sources, and fixes install problems on Linux and Windows.
-
CLIO Kit v2.12.3
adds offline MCP server environment preparation, self-healing server environments, the public National Data Platform catalog endpoint, and sandboxed Windows web-tool launches.
-
CLIO Core v2.4.1
stops the Unix-socket client receive thread from spinning a CPU core while idle, following the v2.4.0 replication and failover release.
-
CLIO Coder v0.7.0
adds the data tool for HDF5, NetCDF, and Parquet, sealed reproduce checks judged against a tolerance, a per-project claims ledger, and the research and arena commands.
Papers from this line of work
Browse these papers- 2026
Trigger, Render, Reason: Multimodal LLM Agents for Science-Aware Scientific Simulation Monitoring
Hua Xu, Jaime Cernuda, Ana Gainaru, Scott Klasky, Anthony Kougkas, Xian-He Sun. International Conference for High Performance Computing, Networking, Storage, and Analysis (SC 2026)
- 2026
GPUH5: A GPU-first I/O Library for Scientific Data
J. Fernandes, Luke Logan, Xian-He Sun, Anthony Kougkas. 11th International Parallel Data Systems Workshop (PDSW 2026), held in conjunction with SC26
- 2026
Trigger-Gated Multimodal LLM Agents for In-situ Scientific Simulation Monitoring
Hua Xu, Jaime Cernuda, Anthony Kougkas, Ana Gainaru, Scott Klasky, Xian-He Sun et al. 11th International Parallel Data Systems Workshop (PDSW 2026), held in conjunction with SC26
- 2026
Type-Safe, Dynamically Defined Metadata Support for Scientific Applications
J. Teves, C. Brinker, Anthony Kougkas, Ana Gainaru, Scott Klasky, Xian-He Sun et al. 11th International Parallel Data Systems Workshop (PDSW 2026), held in conjunction with SC26
- 2026
StageOnce: Speculative Cross-Job Data Staging for LLM Agent Fleets
Izzet Yildirim, Xian-He Sun, Anthony Kougkas. International Conference for High Performance Computing, Networking, Storage, and Analysis (SC 2026)
- 2026
ToolFuse: Provenance-Aware In-Flight Coalescing of Redundant Agent Tool Calls
Izzet Yildirim, Xian-He Sun, Anthony Kougkas. International Conference for High Performance Computing, Networking, Storage, and Analysis (SC 2026)
- 2026
GNNs Beyond HBM: Compressed, Bit-Exact Feature Streaming for Out-of-Core Training
Rajni Pawar, Luke Logan, Xian-He Sun, Anthony Kougkas. International Conference for High Performance Computing, Networking, Storage, and Analysis (SC 2026)
- 2026
Acropolis: A Portable, AI-Driven Semantic Search System for HPC
Rajni Pawar, Luke Logan, Jaime Cernuda, Xian-He Sun, Anthony Kougkas. 22nd IEEE International Conference on eScience (eScience 2026)
- 2026
Rich I/O-centric Derived Quantity Operators for Advanced In Situ Processing
Hua Xu, Jaime Cernuda, Xian-He Sun, Michela Taufer, Nigel Tan, Anthony Kougkas et al. 22nd IEEE International Conference on eScience (eScience 2026)
- 2026
AgentStage: Exploiting LLM Thinking for Data Staging in Scientific Agents
Izzet Yildirim, Xian-He Sun, Anthony Kougkas. 22nd IEEE International Conference on eScience (eScience 2026)
- 2026
Pre-RoPE versus Post-RoPE: Trading Per-Step Compute for KVCache Survivability
Zia Uddin Chowdhury, Jaime Cernuda, Jie Ye, Anthony Kougkas, Xian-He Sun. 13th Greater Chicago Area Systems Research Workshop (GCASR 2026)
- 2026
Agentic Search Efficiency for Scientific Data Discovery
Shazzadul Islam, Jaime Cernuda, Xian-He Sun, Anthony Kougkas. 13th Greater Chicago Area Systems Research Workshop (GCASR 2026)
- 2026
Accelerating Multi-Agent Orchestration with Speculative Dispatching
Jie Ye, Shazzadul Islam, Jaime Cernuda, Xian-He Sun, Anthony Kougkas. 13th Greater Chicago Area Systems Research Workshop (GCASR 2026)
- 2026
Correct Is Not I/O Efficient: Characterizing I/O Behavior of LLM Agents on Scientific Data
Izzet Yildirim, Jie Ye, Jaime Cernuda, Xian-He Sun, Anthony Kougkas. 13th Greater Chicago Area Systems Research Workshop (GCASR 2026)
- 2026
AgentRecall: Provenance and Rewind for Multi-Agent Workflows
Keith Bateman, Xian-He Sun, Anthony Kougkas. 13th Greater Chicago Area Systems Research Workshop (GCASR 2026)
- 2026
Benchmarking the Performance of Semantic Search Techniques for Agent Discovery
Rajni Pawar, Luke Logan, Xian-He Sun, Anthony Kougkas. 13th Greater Chicago Area Systems Research Workshop (GCASR 2026)
- 2026
PagedEviction: Structured Block-wise KV Cache Pruning for Efficient Large Language Model Inference
Krishna Teja Chitty-Venkata, Jie Ye, Siddhisanket Raskar, Xian-He Sun, Anthony Kougkas, Murali Emani et al. Findings of the Association for Computational Linguistics (EACL 2026)
- 2025
Quantifying AWS S3 I/O Performance Boundaries Using the Roofline Model
Meng Tang, Zhaobin Zhu, Luanzheng Guo, James G. Bandy, Tim Carlson, … Anthony Kougkas et al. The 10th International Parallel Data Systems Workshop
- 2025
Optimizing I/O for an Exascale Implicit Kinetic Plasma Simulation using the Rabbit Storage System
Ian Lumsden, Hariharan Devarajan, Izzet Yildirim, Stefano Markidis, Andong Hu, … Anthony Kougkas et al. REX-IO 2025 in conjunction with IEEE Cluster 2025
- 2025
Towards an AI-driven scientific workflow
Shazzadul Islam, Jaime Cernuda, Isa Muradli, Anthony Kougkas, Xian-He Sun. 21st IEEE International Conference on eScience (eScience ’25), September 15-18, 2025
- 2025
Insights into GPUDirect Data Transfer through NIXL Benchmarking
Isa Muradli, Jie Ye, Luke Logan, Anthony Kougkas, Xian-He Sun. 21st IEEE International Conference on eScience (eScience ’25), September 15-18, 2025
- 2025
Accelerate LLM inference with Asynchronous model offload
Jie Ye, Bogdan Nicolae, Xian-He Sun, Anthony Kougkas. 37th International Conference on Scalable Scientific Data Management (SSDBM 2025)
- 2024
MegaMmap: Blurring the Boundary Between Memory and Storage for Data-Intensive Workloads
Luke Logan, Anthony Kougkas, Xian-He Sun. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis SC
- 2024
DFTracer: An Analysis-Friendly Data Flow Tracer for AI-Driven Workflows
Hariharan Devarajan, Loïc Pottier, Kaushik Velusamy, Huihuo Zheng, Izzet Yildirim, … Anthony Kougkas et al. Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis
- 2024
To Derive or Not to Derive: I/O Libraries Take Charge of Derived Quantities Computation
Ana Gainaru, Norbert Podhorszki, Liz Dulac, Qian Gong, Scott Klasky, … Antonios Kougkas et al. 2024 IEEE 36th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)
- 2024
Jarvis: Towards a Shared, User-Friendly, and Reproducible, I/O Infrastructure.
Jaime Cernuda, Luke Logan, N. Lewis, Suren Byna, Xian-He Sun, Anthony Kougkas. The International Parallel Data Systems Workshop (PDSW’24)
- 2024
DaYu: Optimizing Distributed Scientific Workflows by Decoding Dataflow Semantics and Dynamics
Meng Tang, Jaime Cernuda, Jie Ye, Luanzheng Guo, Nathan R. Tallent, Anthony Kougkas et al. Proceedings of the IEEE International Conference on Cluster Computing
- 2024
An Evaluation of DAOS for Simulation and Deep Learning HPC Workloads
Luke Logan, Jay Lofstead, Xian-He Sun, Anthony Kougkas. ACM SIGOPS Operating Systems Review
- 2024
Viper: A High-Performance I/O Framework for Transparently Updating, Storing, and Transferring Deep Neural Network Models
Jie Ye, Jaime Cernuda, Neeraj Rajesh, Keith Bateman, Orcun Yildiz, … Anthony Kougkas et al. Proceedings of the 53rd International Conference on Parallel Processing
- 2024
HStream: A hierarchical data streaming engine for high-throughput scientific applications
Jaime Cernuda, Jie Ye, Anthony Kougkas, Xian-He Sun. Proceedings of the 53rd International Conference on Parallel Processing
- 2024
TunIO: An AI-powered Framework for Optimizing HPC I/O
Neeraj Rajesh, Keith Bateman, Jean Luca Bez, Suren Byna, Anthony Kougkas, Xian-He Sun. Proceedings of the International Parallel and Distributed Processing Symposium
- 2023
Data Flow Lifecycles for Optimizing Workflow Coordination
Hyungro Lee, Luanzheng Guo, Meng Tang, Jesun Firoz, Nathan Tallent, Anthony Kougkas et al. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis
- 2023
IOMax: Maximizing Out-of-Core I/O Analysis Performance on HPC Systems
Izzet Yildirim, Hariharan Devarajan, Anthony Kougkas, Xian-He Sun, Kathryn Mohror. Proceedings of the SC’23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis
- 2022
LabStor: A modular and extensible platform for developing high-performance, customized I/O stacks in userspace
Luke Logan, Jaime Cernuda Garcia, Jay Lofstead, Xian–He Sun, Anthony Kougkas. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis
- 2022
Stimulus: Accelerate Data Management for Scientific AI applications in HPC
Hariharan Devarajan, Anthony Kougkas, Huihuo Zheng, Venkatram Vishwanath, Xian-He Sun. Proceedings of the 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing
People
- Xian-He Sun Long-standing collaborator
- Gerd Heber Co-PI, The HDF Group
- Jacob Hochhalter Co-PI, University of Utah; materials science
- Vivek Srikumar Co-PI, University of Utah; machine learning
- Luke Logan Engineering lead; CLIO Core
- Jaime Cernuda CLIO Agent, Relay, and Jarvis
- Isa Muradli CLIO Kit
- Shazzadul Islam CLIO Kit; eScience'25 agent workflows
- Jie Ye Researcher
- Keith Bateman Researcher
- Neeraj Rajesh Researcher
- Meng Tang Researcher
- Rajni Pawar Researcher; Acropolis
- Hua Xu Researcher
- Bogdan Nicolae Argonne; DOE Genesis lead
- Nathan Tallent Pacific Northwest; DOE Genesis lead
Related notes
- 4 min
SC26 in Chicago
What the group brings to SC26 in Chicago, the pressure from GPUs and agent fleets that runs through all of it, and an invitation to Thursday's Birds of a Feather on HDF5 in the age of AI-driven science.
- 2 min
CLIO in two Genesis Mission projects
Two first-round DOE Genesis Mission projects, PORTUS with Pacific Northwest and SPOTTER-AI with Argonne, build on CLIO. What each asks of it, and why context and provenance matter when agents act on shared infrastructure.
- 3 min
CLIO Core 2.3 and 2.4: when a tier fills or a node dies
Four CLIO Core releases between October 9 and November 2, 2026 added a device-paged GPU vector and Intel GPU support, then taught the context store to recover a full tier and keep data and directories readable when a node goes down.
- 5 min
Context is part of the system
A capable model still fails when the system hands it the wrong slice of reality. CLIO's answer is to treat context the way a database treats state, and to seal a receipt for every action.
- 7 min
When is research software shipped?
A working definition of "shipped" for research software, built around a named state someone else can run, evidence tied to that state, explicit limits, and a continuity signal, with Hermes and DLIO as the worked examples.
- 2 min
NSF funds the next five years of IOWarp
A five-year NSF award for IOWarp, the collaboration with The HDF Group and the University of Utah that grew out of Hermes.
- 7 min
The label that became a patent
US Patent 11,630,834 was granted today for label-based I/O. What a label is, how the idea went from a dissertation chapter to a best-paper award to a patent, and what a patent does not mean for research software.
- 5 min
Hermes is complete
Hermes v1.2.1 is the final release of a project that started as the center of my PhD and ran for five years with The HDF Group. What the multi-tier buffering bet proved, what it did not, and what moves into IOWarp.