akougkas.io

Why CLIO Coder, part 5 of 5: decisions without prose

A coding harness makes hundreds of small decisions per turn, and a chat model is the wrong instrument for them. CLIO Coder v0.5.3 binds four decision sites to a System One model that returns a calibrated distribution over a closed answer shape, and adds a diffusion runtime for the latency-sensitive slot beside it.

Published
Reading
10 min read
Topics
agents, systems, building, research practice

September 27, 2026: v0.5.4 expanded decision-model sites, retired legacy routing, and removed the old evaluation engine and baselines. This essay describes the v0.5.3 design. See the current product guides and release history.

A coding turn produces one visible answer and a few hundred invisible decisions. Which worker takes this dispatch. Which installed skills the listing surfaces first. Which memory records are worth the prompt budget. How much a parked command could destroy. Every one is a closed question with a small answer set, and the only instrument on hand was a model trained to write English: slow, unparseable, unable to state how sure it is. v0.5.3 of CLIO Coder gives the harness a second kind of model for them.

A closed question deserves a closed answer

A System One model does not generate prose. Every question names its answer shape up front and the model returns a calibrated distribution over that shape in one round trip. decide() on the runtime descriptor answers a batch of independent typed questions against one body of evidence. Three primitives: noul returns a truth probability, choice returns one option from a declared set with the distribution over the whole set, and score returns a position on a criteria ladder. The question’s criteria says what each rung means, so the caller defines the scale instead of hoping a prompt implies it.

Confidence is a separate axis from the answer, because they are different questions. A noul of 0.5 at high confidence is a decided coin-flip; at low confidence it is an abstention, and a caller gating on the result has to tell them apart. The readers in decisions.ts return null rather than false below a confidence floor. A missing or unrecognised answer throws instead: callers index by the ids they submitted, so a dropped answer is a contract break, and a caller gating dispatch must never receive a decision the model did not make.

Four sites, off until you bind them

The runtime is typesafe-jev, hidden and declaring chat: false, so it never appears as a conversational target and the configure wizard does not offer it. It binds through fleet.decisionProfiles, a map from a decision site to an existing fleet.profiles entry, because a decision model is a provider rather than an agent and the profile machinery already validates a target and a model. Four sites name moments in a turn: routing picks a worker at dispatch, skills and memory narrow what the prompt carries, toolRisk rates a parked command’s blast radius at the approval prompt.

A turn begins and fans out to four decision sites: routing, skills, memory, and toolRisk. The four sites converge into one decide call carrying their questions together, which returns typed answers with their own confidence, and those answers return to the sites that asked. turn begins routing skills memory toolRisk one decide call typed answers
A turn fanning out to its four decision sites, the sites batching their questions into one decide call, and the typed answers returning to the sites that asked.

A site with no entry resolves to nothing and its caller keeps the behavior it had before the site existed. It is opt-in by absence, and a misbehaving site can be unbound without giving up the other three. Abstention below the confidence floor and provider failure take that same path, so an unbound site, an outage, and an uncertain answer are indistinguishable from each other and from the behavior that preceded the site.

That matters most at routing. Unbound, classifyAgentTask reads a task with an ordered regex list and reports a confidence of 0.3 or 0.7 depending only on whether any task-type rule matched. On “Review the auth middleware for timing attacks, then fix anything you find and add regression tests”, the word-count ladder returns simple and the conjunction rule returns indivisible. Both are wrong. The live provider returns complexity 1.86 of 3 and decomposable 0.81 for that task, using 718 input and 188 output tokens in 249 ms, pinned in a contract test. Bound, the site asks four independent questions in one call, and each field falls back to the regex value on its own. routeValidationProjection strips task features on the same terms as the reservation, so a model cannot author the features its own routing reads.

The other three carry rules of their own. The skills listing is ranked and never filtered, because a skill the model cannot see is a capability it has no way to know was withheld, and a row that abstains keeps its position, since an unscored skill is not a low-scored one. toolRisk is advisory and never changes what is permitted, so a decision model can neither widen nor narrow what an operator approves. Memory scores resolve once at the turn boundary and ride into the prompt-cache key, because reusing the previous turn’s cached section after a new pass would be a correctness bug. A four-candidate fan-out cost 593 input and 81 output tokens in 274 ms, and swapping the text behind an id moved that id’s score from 0.25 to 0.07, confirming that the model resolves an id against the state rather than reading the id.

A runtime that denoises instead of predicting

inception serves Inception’s Mercury models, which denoise whole blocks of tokens in parallel instead of emitting them left to right, and that is what makes them fast enough for a latency-sensitive slot a frontier model cannot fill. Chat runs through the ordinary transport with tool calling and json-schema structured outputs. Fill-in-the-middle runs through the existing infill() verb, so callers reach Mercury as they reach llama.cpp’s, and a target naming no defaultModel infills with the edit-tuned mercury-edit-2.

reasoning_effort is pinned to instant and reasoning is declared false. Above instant the response carries content: null with reasoning_summary: null, so there is nothing to show an operator. Omitting the field is worse: Mercury then reasons by default and returns an empty completion with finish_reason: "length". /v1/edit/completions is left out on purpose, because its next-edit prediction needs a contract verb of its own and its context-tag request format is not covered by the published reference.

One quirk earned a general mechanism. Inception rejects the developer role outright, so every request carrying a system prompt in it is a 400. Catalog-backed model synthesis gained compat and samplingParams passthrough, so a runtime can declare the wire quirks it knows about its own endpoint over whatever the shared catalog says. A provider can be OpenAI-compatible in shape without being compatible in vocabulary, and a descriptor can now say so.

The discipline that lets any of this be judged

Two new surfaces in one release is when a harness starts moving underneath you, so v0.5.3’s third piece is measurement. A suite now declares baseline: {file, pin}, and eval baseline record|check writes and verifies a committed per-task reference. A threshold bounds a metric across a matrix; a baseline says one named scenario reproduces one exact value. eval run runs the check inline, because a reference only a later command consults leaves the run that broke it exiting zero.

Recording refuses an unstable value. A pinned metric absent from a task or disagreeing across repeats is not a baseline, and recording one makes the next check fail for a reason nobody can act on. Pins stay limited to properties of the harness, so wall time, memory, and CPU are excluded. The file is sorted by task and by metric with one value per line, which makes its git diff the behavioral changelog: a change that moves one tool’s result shape appears as that scenario’s custom.digest.behavior line changing, and the review question becomes whether it was intended. The corpus is 261 pinned scenarios: 230 across the tool bench’s six tools and two splits, 31 across five offline model-free suites for dispatch admission, prompt layering, context budgets, memory selection, and continuity handoff.

Discovery got cheaper too. gateway(op="find") and describe answer from a recorded catalog instead of launching every trusted MCP server first; only call and an explicit refresh connect.

Try it

CLIO Coder is pre-1.0 and the System One surface is alpha within it. A site answers only once a credential, a target, a profile, and a binding all exist. The configure wizard does not offer a hidden runtime, so the target goes into settings.yaml by hand.

targets:
  - id: jev
    runtime: typesafe-jev
    defaultModel: jev-latest
    auth:
      apiKeyEnvVar: TYPESAFE_API_KEY

fleet:
  profiles:
    system-one:
      target: jev
      model: jev-latest
  decisionProfiles:
    routing: system-one
export TYPESAFE_API_KEY='...'
clio-coder auth status jev
clio-coder targets --probe
clio-coder models --target jev

A listing returning jev-latest and jev-preview proves the key, the URL, and the model id together. The binding itself has no verification command yet. inspectDecisionSite answers that question and no CLI surface calls it, so what you rely on is settings validation rejecting an unknown site name and an undefined profile. A Mercury target for fill-in-the-middle is four lines.

targets:
  - id: mercury
    runtime: inception
    auth:
      apiKeyEnvVar: INCEPTION_API_KEY
clio-coder doctor
clio-coder eval baseline check --suite evals/machinery/dispatch-admission.yaml

The machinery suite needs no model: it runs the pinned dispatch-admission scenarios and names every value that moved, disappeared, or became unstable.

What is left for you

Nobody knows how a decision model’s calibration drifts as a repository’s tasks move away from the distribution it was measured on, and a site that slowly becomes overconfident degrades quietly. Nobody knows which other moments in a turn deserve a closed question: context eviction, verifier selection, and continuity checkpoint placement are candidates, each an experiment rather than a plan. A non-autoregressive runtime raises the same question from the other side, which is where in an agent loop a fast model belongs when it is not the one holding the conversation.

Four surfaces accept contributions without a pull request against the harness. An extension declares command tools in a clio-coder-extension.yaml Clio validates without importing package code. A verifier turns a scientific claim into an exact argv vector with a declared tolerance, and a domain whose tolerance Clio cannot express yet is the most useful bug report here. An eval turns an opinion into a row. A provider runtime adds a model, and institutional gateways, national-laboratory endpoints, and inference servers nobody upstream has heard of are what a research group can contribute and nobody else can.

Five parts, one argument: a coding agent you can trust with a repository is one whose every decision you can name, bound, measure, and turn off. The rest is prose. The repository is where a field report changes what happens next. The install path is in the GPU tutorial; the launch essay covers what “shipped” had to mean first.

Sources, code, and records

  1. source

    CLIO Coder v0.5.3 release

    The 0.5.3 section, dated 2026-09-22, records the decision-model contract, the Inception runtime, committed eval baselines, and cached MCP discovery.

  2. source

    CLIO Coder repository

    Source, documentation, roadmap, and the pinned evaluation suites under evals/.

  3. source

    Can a small decision model improve an agent's work?

    The open questions on System One, with a first experiment.

  • 2023–present
    active

    CLIO Coder

    A local-first coding agent with typed tools, bounded workers, and receipts

  • 2025–present
    active

    CLIO

    A context layer for scientific AI agents and workflows

More on CLIO Coder

  1. essay

    Reliable work from unreliable agents

    A coding agent is a system built around one brilliant, persuasive, unreliable component. The argument of my November 9, 2026 lecture on agentic engineering: five ideas for building trustworthy work around that component, what they could do for science, and the problems still open.

    14 min
  2. essay

    CLIO as the entry point

    CLIO should become the single place a developer starts with IOWarp, built as an application SDK with deployment profiles rather than an umbrella package or a gateway service. This is the direction for 2027; the API and the scope are still open.

    6 min
  3. essay

    A capable model in a capable harness

    Claude 3.5 Sonnet brought me back to writing code. A capable model still needs a harness that makes its progress checkable and its cost visible, and CLIO Coder is the one I built around models that many people can afford to use.

    4 min