akougkas.io

Uncover the Overhead and Resource Usage for Handling KV Cache Overflow in LLM Inference

Jie Ye, Bogdan Nicolae, Anthony Kougkas, Xian-He Sun

The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC’24)

System

  1. 2020–present
    active

    DeepIO

    A line of subcontracts with Argonne, Pacific Northwest, and Lawrence Livermore that funds students working where I/O meets AI, from Viper and KV-cache scheduling for LLM inference to WisIO's I/O bottleneck analysis.

Browse all papers on DeepIO
  1. 2026
    poster

    Extending I/O Observability to Explainability for Targeted Optimization

    Neeraj Rajesh, S. Neuwirth, Jay Lofstead, Anthony Kougkas, Xian-He Sun, Nikoli Dryden et al. International Conference for High Performance Computing, Networking, Storage, and Analysis (SC 2026)

  2. 2026
    conference

    PKAS: Predictive KVCache-Aware Scheduling for Faster LLM and Transformer Inferences

    Jie Ye, A. Maurya, K. Chitty-Venkata, Bogdan Nicolae, Anthony Kougkas, Xian-He Sun. 35th International Symposium on High-Performance Parallel and Distributed Computing (HPDC 2026)

  3. 2026
    poster

    Pre-RoPE versus Post-RoPE: Trading Per-Step Compute for KVCache Survivability

    Zia Uddin Chowdhury, Jaime Cernuda, Jie Ye, Anthony Kougkas, Xian-He Sun. 13th Greater Chicago Area Systems Research Workshop (GCASR 2026)

  4. 2026
    conference

    PagedEviction: Structured Block-wise KV Cache Pruning for Efficient Large Language Model Inference

    Krishna Teja Chitty-Venkata, Jie Ye, Siddhisanket Raskar, Xian-He Sun, Anthony Kougkas, Murali Emani et al. Findings of the Association for Computational Linguistics (EACL 2026)

Citation

@misc{ye2024uncoveroverhead,
  author = {Ye, J. and Nicolae, B. and Kougkas, A. and Sun, X.-H.},
  title = {{Uncover the Overhead and Resource Usage for Handling KV Cache Overflow in LLM Inference}},
  howpublished = {The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC'24)},
  year = {2024},
  month = nov,
  url = {http://cs.iit.edu/%7Escs/assets/files/ye2024kvcache_poster.pdf},
}

Related notes

  1. essay

    When is research software shipped?

    A working definition of "shipped" for research software, built around a named state someone else can run, evidence tied to that state, explicit limits, and a continuity signal, with Hermes and DLIO as the worked examples.

    7 min
  2. essay

    We need an Agent500

    Agent leaderboards score answers. Science needs a ranking that runs an agent through a real workflow with real tools and data and measures correctness, cost, I/O efficiency, and provenance together.

    7 min
  3. field note

    Reflecting on SC25

    What I noticed at SC25 in St. Louis, what the group brought, what a year of running the PDSW program committee taught me about how a workshop gets made, and a few notes for people attending their first SC.

    8 min