ChronoLog
A distributed shared log ordered by physical time.
A distributed shared log that orders activity data by physical time instead of consensus. My MSST 2020 design, funded by NSF for six years, and now at version 3 with SQL, pub/sub, and an MCP server.
The question
Beyond storing data, computing systems increasingly need to store activity: things that happen rather than things that are. Instruments, workflows, schedulers, and now agents all produce streams of events that someone later needs to replay in order. Every distributed log I could find at the time paid for that order with a sequencer or a consensus protocol, and the sequencer became the bottleneck.
ChronoLog asked a blunt question: what if the clock did the ordering?
Where it came from
I designed ChronoLog in 2019 and 2020, while Hermes was maturing and I was watching how much of the metadata traffic in a storage system was activity data. The MSST 2020 paper with Hariharan Devarajan, Keith Bateman, Jaime Cernuda, Neeraj Rajesh, and Xian-He Sun laid out the design and measured a prototype at more than one million tail operations per second, an order of magnitude past the distributed logs we compared against.
In May 2021, NSF funded ChronoLog as a CSSI Frameworks award, collaborative with Kyle Chard at the University of Chicago.
The architectural bet
Use synchronized physical clocks to establish total order, and remove the sequencer from the append path entirely. New entries become visible the moment they land. Many writers and many readers can work at once. The cost is that ordering is bounded by clock skew rather than exact, which is a fair trade for activity data: a log ordered within microseconds is enough for provenance, monitoring, and replay.
The second bet is that a log should scale in three dimensions at once. Horizontally across nodes for throughput, vertically across storage tiers for capacity, and temporally by timestamp so a range query knows where to look. Records enter close to compute, get assembled into time-ordered stories, and migrate from memory to NVMe to the parallel file system to the archive without the application noticing.
The system
ChronoLog is five services with deliberately narrow jobs. ChronoVisor coordinates clients, metadata, and time. ChronoKeeper ingests events over RDMA into a distributed journal and serves tail reads. ChronoGrapher collects events into stories and writes them down through an elastic pipeline. ChronoStore manages the intermediate and archival tiers. ChronoPlayer merges parallel reads across tiers into one ordered stream for replay().
1.0.0 in June 2024 deployed the service pipeline. 2.0.0 in February 2025 added ChronoPlayer and historical replay, the first end-to-end bidirectional release. v3.0.0 in May 2026 cleaned up the client API, added ChronoSQL and a ChronoPubSub plugin, and moved the MCP server into the main repository so an agent can create chronicles, record, replay, and run time-range queries through CLIO Kit. The team ran a half-day ChronoLog workshop at PEARC25 in July 2025.
The people who built it
Kun Feng and Eneko Gonzalez wrote most of the current code and shipped the 2.x and 3.0 releases. Inna Brodkin at the University of Chicago engineered the UChicago side. Keith, Jaime, and Neeraj carried the design from the paper into the first services, and Luke Logan connected it to the runtime lineage that became CLIO Core.
What carried forward
ChronoLog’s physical-time ordering and tiered retrieval now shape how IOWarp handles provenance and activity history. Through CLIO, the broader context program can query and replay that history while ChronoLog remains a high-rate distributed log across storage tiers. The team is designing AgentLog on top of it with typed event records, causal views, and policy-controlled sharing for multi-agent science in the National Data Platform’s federated environment with the University of Utah. AgentLog remains design-stage work.
What I am still asking
- How much activity data needs to be sealed at write time, and how much can be reconstructed from a replayable log?
- What is the smallest useful query surface over a log: time range, causal chain, or both?
- Can the same log serve an instrument at a million events per second and an interactive scientific workflow at ten?
Code, releases, papers, and records
- paper
ChronoLog at MSST 2020
The physical-time ordering model, the three-dimensional distribution, and the first evaluation. Over one million tail operations per second.
- code
grc-iit/chronolog
The services, client library, plugins, MCP server, tests, and release history.
- release
ChronoLog v3.4.0, November 5, 2026
Durable writes. A write made durable returns only after ChronoKeeper has synced it to a write-ahead log, and a restarted keeper recovers those events and ships them to the archive.
- release
ChronoLog v3.0.0, May 29, 2026
ChronoSQL, a ChronoPubSub plugin, the MCP server in-tree, and a client API cleanup.
- documentation
chronolog.dev
Installation, the deployment guide, the API, and the public integrations roadmap.
- record
NSF award OAC-2104013
The public award record. PI Xian-He Sun; co-PIs include me and Kyle Chard at the University of Chicago.
- press
Illinois Tech news, June 2021
The university announcement of the multi-million dollar NSF CSSI award for ChronoLog.
- record
GRC project record
The project's page at the Gnosis Research Center.
- documentation
ChronoLog MCP server in CLIO Kit
The documentation for the MCP server an agent uses to record and replay through ChronoLog.
Papers from this line of work
Browse these papers- 2026
ChronoLog: Distributed Shared Tiered Log Store
Eneko Gonzalez, Kun Feng, I. Brodkin, Kyle Chard, Anthony Kougkas, Xian-He Sun. 13th Greater Chicago Area Systems Research Workshop (GCASR 2026)
- 2025
ChronoLog & AI: A Scalable, Collaborative Solution for LLM Conversation Logging & Retrieval
S. Sonar, Jaime Cernuda, Anthony Kougkas, Xian-He Sun. 37th International Conference on Scalable Scientific Data Management (SSDBM 2025)
- 2024
Jarvis: Towards a Shared, User-Friendly, and Reproducible, I/O Infrastructure.
Jaime Cernuda, Luke Logan, N. Lewis, Suren Byna, Xian-He Sun, Anthony Kougkas. The International Parallel Data Systems Workshop (PDSW’24)
- 2024
HStream: A hierarchical data streaming engine for high-throughput scientific applications
Jaime Cernuda, Jie Ye, Anthony Kougkas, Xian-He Sun. Proceedings of the 53rd International Conference on Parallel Processing
- 2022
LuxIO: Intelligent Resource Provisioning and Auto-Configuration for Storage Services
Keith Bateman, Neeraj Rajesh, Jaime Cernuda Garcia, Luke Logan, Jie Ye, … Anthony Kougkas et al. Proceedings of the 29th International Conference on High Performance Computing, Data, and Analytics
- 2022
LabStor: A modular and extensible platform for developing high-performance, customized I/O stacks in userspace
Luke Logan, Jaime Cernuda Garcia, Jay Lofstead, Xian–He Sun, Anthony Kougkas. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis
- 2021
Apollo: An ML-assisted real-time storage resource observer
Neeraj Rajesh, Hariharan Devarajan, Jaime Cernuda Garcia, Keith Bateman, Luke Logan, … Anthony Kougkas et al. Proceedings of the 30th International Symposium on High-Performance Parallel and Distributed Computing
- 2020
ChronoLog: A Distributed Shared Tiered Log Store with Time-based Data Ordering
Anthony Kougkas, Hariharan Devarajan, Keith Bateman, Jaime Cernuda, Neeraj Rajesh, Xian-He Sun. Proceedings of the 36th International Conference on Massive Storage Systems and Technology
- 2020
HCL: Distributing Parallel Data Structures in Extreme Scales
Hariharan Devarajan, Anthony Kougkas, Keith Bateman, Xian-He Sun. Proceedings of the International Conference on Cluster Computing
- 2019
An intelligent, adaptive, and flexible data compression framework
Hariharan Devarajan, Anthony Kougkas, Xian-He Sun. Proceedings of the 19th International Symposium on Cluster, Cloud and Grid Computing
People
- Xian-He Sun MSST'20 co-author
- Kyle Chard Co-PI, University of Chicago
- Keith Bateman MSST'20 co-author; researcher
- Jaime Cernuda MSST'20 co-author; researcher
- Neeraj Rajesh MSST'20 co-author; researcher
- Hariharan Devarajan MSST'20 co-author
- Kun Feng Engineer; the largest share of the current code
- Eneko Gonzalez Engineer; releases 2.x and 3.0
- Luke Logan Researcher
Related notes
- 7 min
Physical time as a sequencer
ChronoLog 1.0.0 is out. A distributed log can order by synchronized physical clocks and drop the central sequencer. Why that trade fits activity data, what 1.0 delivers, and what worries me about clocks.
- 9 min
MCP for science: what sixteen servers taught us
CLIO Kit ships sixteen MCP servers over scientific formats, schedulers, and a shared log. What they taught us about tool shapes, what a summarize tool must refuse, and where the protocol runs out.
- 8 min
Agent memory is a storage problem
Memory for agents decomposes into tiers with different access patterns: episodic, semantic, procedural, working. Storage people already know how to build tiers, and a time-ordered shared log is the episodic one.
- 4 min
ChronoLog 3.0: SQL, pub/sub, and an MCP server
ChronoLog 3.0.0 adds ChronoSQL, a pub/sub plugin, and an in-tree MCP server, so an agent can record and replay time-ordered activity through the same log a scientific workflow writes to.
- 9 min
One destination, N tiers
Every tier supplies the application at once, and the application sees one destination. The thesis from a November 2026 talk to my group at the Gnosis Research Center, the six lines of I/O work behind it since 2016, and my bet that they are the foundation agents need for context.