<?xml version="1.0" encoding="utf-8"?> <feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en"> <title>Tutorials · Anthony Kougkas</title> <subtitle>Tutorials by Anthony Kougkas.</subtitle> <id>https://akougkas.io/feed/tutorial.xml</id> <link rel="self" type="application/atom+xml" href="https://akougkas.io/feed/tutorial.xml"/> <link rel="alternate" type="text/html" href="https://akougkas.io/journal/?kind=tutorial"/> <updated>2026-09-17T00:00:00-05:00</updated> <author> <name>Anthony Kougkas</name> <uri>https://akougkas.io/about/</uri> </author> <entry> <title type="html">A closed loop for AI-assisted research writing</title> <link rel="alternate" type="text/html" href="https://akougkas.io/journal/2026/a-closed-loop-for-ai-assisted-research-writing/"/> <id>https://akougkas.io/journal/2026/a-closed-loop-for-ai-assisted-research-writing/</id> <published>2026-09-17T00:00:00-05:00</published> <updated>2026-09-17T00:00:00-05:00</updated> <summary type="html">Install CLIO Coder, WTF-P, and CiteNexus on Linux and run a first supervised writing session: the agent keeps the bookkeeping, every citation stays traceable to its source, and the author keeps the judgment.</summary> <content type="html" xml:base="https://akougkas.io/journal/2026/a-closed-loop-for-ai-assisted-research-writing/">&lt;p class=&quot;entry-note&quot;&gt;&lt;strong&gt;September 27, 2026:&lt;/strong&gt; This walkthrough retains its original tested CLIO Coder v0.4.9 instructions. Starting with &lt;a href=&quot;https://github.com/iowarp/clio-coder/releases/tag/v0.5.6&quot;&gt;v0.5.6&lt;/a&gt;, the old autonomy names are rejected; &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;default&lt;/code&gt; and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;yolo&lt;/code&gt; are the supported levels, and read-only is a separate dispatch restriction. Use the &lt;a href=&quot;https://coder.iowarp.ai/docs.html&quot;&gt;current product guides&lt;/a&gt; for a new installation. The full workflow below has not been retested against v0.5.7.&lt;/p&gt; &lt;p&gt;Every coding agent I have tried can write a fluent related-work paragraph in ten seconds. Most will also cite a paper that does not exist, forget by Thursday what we decided on Monday, and leave me unable to say which sentences carry my judgment. The model is capable. What it lacks is a research method.&lt;/p&gt; &lt;p&gt;So the lab built one, in three open pieces:&lt;/p&gt; &lt;ul&gt; &lt;li&gt;&lt;a href=&quot;/work/clio-coder/&quot;&gt;CLIO Coder&lt;/a&gt; is the agent harness: it runs a model you choose, reads your folder, and decides what waits for your approval. It is the hands-on component I maintain inside &lt;a href=&quot;/work/clio/&quot;&gt;CLIO&lt;/a&gt;, part of &lt;a href=&quot;/work/iowarp/&quot;&gt;IOWarp&lt;/a&gt;.&lt;/li&gt; &lt;li&gt;&lt;a href=&quot;/work/wtf-p/&quot;&gt;WTF-P&lt;/a&gt; (Write The F***ing Paper, where P also means proposal, presentation, and poster) is the method. It gives the agent &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/wtfp:*&lt;/code&gt; actions that interview you, map your materials, outline with a word budget, then plan, draft, and review one section at a time. Every decision is saved in a &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;.planning/&lt;/code&gt; directory next to your manuscript, so next week’s session resumes from disk. Each stage stops at a gate, a point only you can cross.&lt;/li&gt; &lt;li&gt;&lt;a href=&quot;https://github.com/akougkas/cite-nexus-mcp&quot;&gt;CiteNexus&lt;/a&gt; is the evidence layer: a scholarly search server that returns each record as a candidate, meaning a pointer with its source URL, retrieval time, and the provider behind every field.&lt;/li&gt; &lt;/ul&gt; &lt;figure class=&quot;figure--diagram&quot;&gt; &lt;svg viewBox=&quot;104 8 432 374&quot; role=&quot;img&quot; aria-labelledby=&quot;fig-writing-loop&quot;&gt; &lt;title id=&quot;fig-writing-loop&quot;&gt;A loop in four layers. The author sends intent down to CLIO Coder, which passes a bounded action to WTF-P, which sends a query to CiteNexus. Candidates return to WTF-P, a gated preview returns through CLIO Coder, and the author&apos;s approval closes the loop.&lt;/title&gt; &lt;rect class=&quot;d-box&quot; x=&quot;120&quot; y=&quot;16&quot; width=&quot;400&quot; height=&quot;64&quot; rx=&quot;4&quot; /&gt; &lt;text class=&quot;d-text&quot; x=&quot;320&quot; y=&quot;43&quot; text-anchor=&quot;middle&quot;&gt;Author&lt;/text&gt; &lt;text class=&quot;d-label&quot; x=&quot;320&quot; y=&quot;66&quot; text-anchor=&quot;middle&quot;&gt;intent, judgment, approval&lt;/text&gt; &lt;path class=&quot;d-line&quot; d=&quot;M200 84 V100&quot; /&gt; &lt;polygon class=&quot;d-accent-fill&quot; points=&quot;194,100 200,110 206,100&quot; /&gt; &lt;text class=&quot;d-label&quot; x=&quot;214&quot; y=&quot;102&quot;&gt;intent&lt;/text&gt; &lt;path class=&quot;d-line&quot; d=&quot;M440 110 V94&quot; /&gt; &lt;polygon class=&quot;d-accent-fill&quot; points=&quot;434,94 440,84 446,94&quot; /&gt; &lt;text class=&quot;d-label&quot; x=&quot;426&quot; y=&quot;102&quot; text-anchor=&quot;end&quot;&gt;approval&lt;/text&gt; &lt;rect class=&quot;d-soft&quot; x=&quot;120&quot; y=&quot;114&quot; width=&quot;400&quot; height=&quot;64&quot; rx=&quot;4&quot; /&gt; &lt;text class=&quot;d-text&quot; x=&quot;320&quot; y=&quot;141&quot; text-anchor=&quot;middle&quot;&gt;CLIO Coder&lt;/text&gt; &lt;text class=&quot;d-label&quot; x=&quot;320&quot; y=&quot;164&quot; text-anchor=&quot;middle&quot;&gt;harness: model, tools, autonomy&lt;/text&gt; &lt;path class=&quot;d-line&quot; d=&quot;M200 182 V198&quot; /&gt; &lt;polygon class=&quot;d-accent-fill&quot; points=&quot;194,198 200,208 206,198&quot; /&gt; &lt;text class=&quot;d-label&quot; x=&quot;214&quot; y=&quot;200&quot;&gt;action&lt;/text&gt; &lt;path class=&quot;d-line&quot; d=&quot;M440 208 V192&quot; /&gt; &lt;polygon class=&quot;d-accent-fill&quot; points=&quot;434,192 440,182 446,192&quot; /&gt; &lt;text class=&quot;d-label&quot; x=&quot;426&quot; y=&quot;200&quot; text-anchor=&quot;end&quot;&gt;preview&lt;/text&gt; &lt;rect class=&quot;d-soft&quot; x=&quot;120&quot; y=&quot;212&quot; width=&quot;400&quot; height=&quot;64&quot; rx=&quot;4&quot; /&gt; &lt;text class=&quot;d-text&quot; x=&quot;320&quot; y=&quot;239&quot; text-anchor=&quot;middle&quot;&gt;WTF-P&lt;/text&gt; &lt;text class=&quot;d-label&quot; x=&quot;320&quot; y=&quot;262&quot; text-anchor=&quot;middle&quot;&gt;method, gates, .planning/ state&lt;/text&gt; &lt;path class=&quot;d-line&quot; d=&quot;M200 280 V296&quot; /&gt; &lt;polygon class=&quot;d-accent-fill&quot; points=&quot;194,296 200,306 206,296&quot; /&gt; &lt;text class=&quot;d-label&quot; x=&quot;214&quot; y=&quot;298&quot;&gt;query&lt;/text&gt; &lt;path class=&quot;d-line&quot; d=&quot;M440 306 V290&quot; /&gt; &lt;polygon class=&quot;d-accent-fill&quot; points=&quot;434,290 440,280 446,290&quot; /&gt; &lt;text class=&quot;d-label&quot; x=&quot;426&quot; y=&quot;298&quot; text-anchor=&quot;end&quot;&gt;candidates&lt;/text&gt; &lt;rect class=&quot;d-soft&quot; x=&quot;120&quot; y=&quot;310&quot; width=&quot;400&quot; height=&quot;64&quot; rx=&quot;4&quot; /&gt; &lt;text class=&quot;d-text&quot; x=&quot;320&quot; y=&quot;337&quot; text-anchor=&quot;middle&quot;&gt;CiteNexus&lt;/text&gt; &lt;text class=&quot;d-label&quot; x=&quot;320&quot; y=&quot;360&quot; text-anchor=&quot;middle&quot;&gt;source-backed scholarly search&lt;/text&gt; &lt;/svg&gt; &lt;figcaption&gt;The loop, with the author approval gate between the agent&apos;s proposal and any write, and approved state landing in &lt;code&gt;.planning/&lt;/code&gt;.&lt;/figcaption&gt; &lt;/figure&gt; &lt;p&gt;The pieces are independent: WTF-P also installs on Claude Code and five other coding agents, and CiteNexus works with any Model Context Protocol client. I describe this stack because every layer is open and runs on a model you choose.&lt;/p&gt; &lt;h2 id=&quot;prerequisites&quot;&gt;Prerequisites&lt;/h2&gt; &lt;p&gt;This path is tested on Linux. Other platforms are not yet covered here.&lt;/p&gt; &lt;ul&gt; &lt;li&gt;Node.js 22.19 or newer with npm. CLIO Coder sets that floor; WTF-P alone needs Node.js 20.&lt;/li&gt; &lt;li&gt;Python 3.11 or newer, and &lt;a href=&quot;https://docs.astral.sh/uv/&quot;&gt;uv&lt;/a&gt; for a one-line CiteNexus install.&lt;/li&gt; &lt;li&gt;A model for CLIO Coder to talk to: a local server such as LM Studio, Ollama, or llama.cpp, a lab gateway, or a cloud API. For one GPU, see &lt;a href=&quot;/journal/2026/clio-coder-on-your-own-gpu/&quot;&gt;Run CLIO Coder against a model on your own GPU&lt;/a&gt;.&lt;/li&gt; &lt;li&gt;A real folder for what you are writing, holding your notes, drafts, the call for papers, and any &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;.bib&lt;/code&gt; file.&lt;/li&gt; &lt;/ul&gt; &lt;p&gt;I tested everything below on September 17, 2026 with CLIO Coder 0.4.9, WTF-P 0.7.3, and CiteNexus 0.2.2. My CLIO Coder was a build of 0.4.9 ahead of its npm release. WTF-P requires CLIO Coder 0.4.7 or newer, which the version on npm already satisfies, and the research step needs WTF-P 0.7.2 or newer.&lt;/p&gt; &lt;h2 id=&quot;1-install-clio-coder-and-connect-a-model&quot;&gt;1. Install CLIO Coder and connect a model&lt;/h2&gt; &lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;npm &lt;span class=&quot;nb&quot;&gt;install&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-g&lt;/span&gt; @iowarp/clio-coder clio-coder configure &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;Choose &lt;strong&gt;Quick Connect&lt;/strong&gt;, paste your endpoint URL (&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;localhost:1234&lt;/code&gt; for LM Studio, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;localhost:11434&lt;/code&gt; for Ollama, or your gateway’s address), supply a key and pick a model if asked, then review and connect. Clio calls a configured endpoint a target, and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;clio-coder targets --probe&lt;/code&gt; confirms it answers.&lt;/p&gt; &lt;h2 id=&quot;2-install-wtf-p-into-clio&quot;&gt;2. Install WTF-P into Clio&lt;/h2&gt; &lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;npx &lt;span class=&quot;nt&quot;&gt;--yes&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;--package&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;wtf-p@latest &lt;span class=&quot;nt&quot;&gt;--&lt;/span&gt; wtf-p &lt;span class=&quot;nb&quot;&gt;install &lt;/span&gt;clio clio-coder library inspect wtfp &lt;span class=&quot;nt&quot;&gt;--user&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;--json&lt;/span&gt; &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;Keep that exact &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;npx&lt;/code&gt; form, and pin a version such as &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;wtf-p@0.7.3&lt;/code&gt; when a paper artifact must be reproducible. The lines to check in the second command’s output, trimmed from my machine:&lt;/p&gt; &lt;div class=&quot;language-json highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;id&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;wtfp&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;version&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;0.7.3&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;enabled&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;kc&quot;&gt;true&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;valid&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;kc&quot;&gt;true&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;compatible&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;kc&quot;&gt;true&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;loadable&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;kc&quot;&gt;true&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;h2 id=&quot;3-install-citenexus&quot;&gt;3. Install CiteNexus&lt;/h2&gt; &lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;uv tool &lt;span class=&quot;nb&quot;&gt;install &lt;/span&gt;cite-nexus-mcp cite-nexus-wtfp &lt;span class=&quot;nt&quot;&gt;--check&lt;/span&gt; &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;WTF-P never installs CiteNexus for you. It looks for &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;cite-nexus-wtfp&lt;/code&gt; on your &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;PATH&lt;/code&gt;. The check starts the real server and lists providers without querying them. Trimmed from my run:&lt;/p&gt; &lt;div class=&quot;language-json highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;providers&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;providers&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;id&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;crossref&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;available&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;kc&quot;&gt;true&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;default&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;kc&quot;&gt;true&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;},&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;id&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;arxiv&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;available&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;kc&quot;&gt;true&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;default&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;kc&quot;&gt;false&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;},&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;metadata&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;backend&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;cite-nexus&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;version&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;0.2.2&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;transport&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;mcp-stdio&quot;&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;Crossref, DataCite, and Europe PMC are the defaults and need no key. arXiv and Semantic Scholar are free but join a search only when you name them, and so does any keyed index such as Scopus.&lt;/p&gt; &lt;h2 id=&quot;4-a-first-session&quot;&gt;4. A first session&lt;/h2&gt; &lt;p&gt;The example is a four-page workshop paper. Start Clio in your folder in supervised mode, where the agent proposes and you approve:&lt;/p&gt; &lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;nb&quot;&gt;cd&lt;/span&gt; ~/papers/my-workshop-paper clio-coder &lt;span class=&quot;nt&quot;&gt;--autonomy&lt;/span&gt; suggest &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;Enter one action at a time and finish each before the next. The method works because you are present at every decision.&lt;/p&gt; &lt;!-- prettier-ignore-start --&gt; &lt;ol class=&quot;entry-steps&quot;&gt; &lt;li&gt;&lt;strong&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/wtfp:help&lt;/code&gt;&lt;/strong&gt; prints the action list without calling the model, which proves the plugin loaded. Its first line on my machine reads &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;WTF-P 0.7.3 on Clio Coder: evidence-grounded academic writing&lt;/code&gt;.&lt;/li&gt; &lt;li&gt;&lt;strong&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/wtfp:new-paper&lt;/code&gt;&lt;/strong&gt; with a brief, such as: &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;I am preparing a four-page workshop paper on our storage benchmark results. Interview me before deciding scope or claims, and mark unknowns instead of inventing citations.&lt;/code&gt; The agent interviews you, then previews the initial project records. You decide the question, the contribution, and the venue limits. You approve the write that creates &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;.planning/&lt;/code&gt;. Nothing is on disk before that.&lt;/li&gt; &lt;li&gt;&lt;strong&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/wtfp:map-project&lt;/code&gt;&lt;/strong&gt; inventories the drafts, figures, data notes, and references already in the folder. You decide what is in scope and correct anything it misread.&lt;/li&gt; &lt;li&gt;&lt;strong&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/wtfp:create-outline&lt;/code&gt;&lt;/strong&gt; proposes sections, the argument, and a word budget, and gives each section a stable identifier. You approve the outline, then use those identifiers from here on.&lt;/li&gt; &lt;li&gt;&lt;strong&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/wtfp:research-gap related-work&lt;/code&gt;&lt;/strong&gt; researches the literature one section needs. Name the search and the providers in the same request, because that list is what you approve before anything touches the network: &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;Use citation-search with --backend=cite-nexus and --providers=crossref,arxiv for hierarchical storage and I/O buffering in HPC.&lt;/code&gt; You decide which candidates matter. You approve what enters the evidence table and the bibliography.&lt;/li&gt; &lt;li&gt;&lt;strong&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/wtfp:discuss-section&lt;/code&gt;&lt;/strong&gt;, &lt;strong&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/wtfp:plan-section&lt;/code&gt;&lt;/strong&gt;, &lt;strong&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/wtfp:write-section&lt;/code&gt;&lt;/strong&gt;, and &lt;strong&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/wtfp:review-section&lt;/code&gt;&lt;/strong&gt; repeat per section. The agent captures your intent, produces a checked plan, drafts only from the approved plan and evidence, and records review findings without rewriting. You approve each plan and each draft.&lt;/li&gt; &lt;/ol&gt; &lt;!-- prettier-ignore-end --&gt; &lt;p&gt;You can run the same search the agent runs. I ran this:&lt;/p&gt; &lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;node ~/.config/clio-coder/plugins/wtfp/tools/wtfp-tool.js citation-search &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;--backend&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;cite-nexus &lt;span class=&quot;nt&quot;&gt;--query&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;I/O buffering hierarchical storage HPC&quot;&lt;/span&gt; &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;--limit&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;3 &lt;span class=&quot;nt&quot;&gt;--timeout&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;40 &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;The first result and the search summary, trimmed from the real output:&lt;/p&gt; &lt;div class=&quot;language-json highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;results&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;id&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;doi:10.1109/cluster.2018.00025&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;title&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;UniviStor: Integrated Hierarchical and Distributed Storage for HPC&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;verification&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;candidate&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;citeNexus&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;is_retracted&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;kc&quot;&gt;null&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;sources&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[{&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;provider&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;crossref&quot;&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;}]&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;],&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;metadata&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;version&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;0.2.2&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;total&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;8&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;returned&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;3&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;nl&quot;&gt;&quot;errors&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;:&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[]&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;Read &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;&quot;verification&quot;: &quot;candidate&quot;&lt;/code&gt; literally. CiteNexus established that a record with this DOI exists at Crossref. It did not establish that the paper supports your sentence, and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;&quot;is_retracted&quot;: null&lt;/code&gt; means unknown, not clean. &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/wtfp:check-refs&lt;/code&gt; audits a finished bibliography through the same search. CiteNexus is also an ordinary MCP server, so you can register it in Clio’s &lt;a href=&quot;https://github.com/iowarp/clio-coder/blob/main/docs/guide/configuration-reference.md&quot;&gt;MCP configuration&lt;/a&gt; for quick lookups outside a section’s research record.&lt;/p&gt; &lt;p&gt;To stop for the day, run &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/wtfp:pause-writing&lt;/code&gt;, which writes a handoff into &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;.planning/&lt;/code&gt;, and quit Clio. Tomorrow, from the same folder, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/wtfp:resume-writing&lt;/code&gt; picks up from disk. &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/wtfp:create-poster&lt;/code&gt; turns the same argument and evidence into a laid-out poster with a claim ledger. &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/wtfp:create-slides&lt;/code&gt; does the same for a talk. For a grant, follow the &lt;a href=&quot;https://github.com/akougkas/wtf-p/blob/main/docs/PROPOSAL_WORKFLOW.md&quot;&gt;proposal workflow&lt;/a&gt;.&lt;/p&gt; &lt;h2 id=&quot;what-you-decide-and-what-you-approve&quot;&gt;What you decide and what you approve&lt;/h2&gt; &lt;p&gt;A gate is not a courtesy prompt. When I recorded a session to start a conference poster, I told the agent to use defaults and not to interview me further. It still stopped, because no instruction in a prompt can cross an author gate.&lt;/p&gt; &lt;figure class=&quot;figure--paper&quot;&gt; &lt;img src=&quot;/assets/img/wtfp-clio-approval-gate.webp&quot; alt=&quot;A CLIO Coder terminal session. The agent explains that project initialization is the one gate this workflow requires, then opens a dialog titled Answer a question with three tabs: Initialize WTF-P poster project, Word budget, and Workflow mode. The highlighted choice reads Approve, create project.&quot; width=&quot;1037&quot; height=&quot;530&quot; loading=&quot;lazy&quot; /&gt; &lt;figcaption&gt;The gate, from a session I recorded on September 17, 2026 while starting a conference poster with CLIO Coder 0.4.9 and WTF-P 0.7.1, two patch releases before the one this tutorial installs. The gate behaves the same today.&lt;/figcaption&gt; &lt;/figure&gt; &lt;figure class=&quot;figure--paper&quot;&gt; &lt;img src=&quot;/assets/img/wtfp-clio-planning-write.webp&quot; alt=&quot;The same session after approval. The recorded answers appear as Approve, create project; 1500 words; and standard workflow mode. The agent allocates the 1500 words across five poster sections, then writes .planning/project.json and .planning/config.json, each shown as a diff of added lines.&quot; width=&quot;1037&quot; height=&quot;882&quot; loading=&quot;lazy&quot; /&gt; &lt;figcaption&gt;The state written after approval: my answers on the record, a word budget summing to 1,500, and each write to &lt;code&gt;.planning/&lt;/code&gt; shown as a diff.&lt;/figcaption&gt; &lt;/figure&gt; &lt;h2 id=&quot;upgrade-and-removal&quot;&gt;Upgrade and removal&lt;/h2&gt; &lt;p&gt;To upgrade WTF-P, run the install command again. Since 0.7.3 it replaces the files it owns and keeps any you edited. Each layer comes out independently:&lt;/p&gt; &lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;npx &lt;span class=&quot;nt&quot;&gt;--yes&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;--package&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;wtf-p@latest &lt;span class=&quot;nt&quot;&gt;--&lt;/span&gt; wtf-p uninstall clio &lt;span class=&quot;nt&quot;&gt;--yes&lt;/span&gt; uv tool uninstall cite-nexus-mcp npm uninstall &lt;span class=&quot;nt&quot;&gt;-g&lt;/span&gt; @iowarp/clio-coder &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;Use &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;--dry-run&lt;/code&gt; in place of &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;--yes&lt;/code&gt; to preview the first one, and prefer it over &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;clio-coder library remove wtfp&lt;/code&gt;, which leaves a version receipt behind. None of these touch your project’s &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;.planning/&lt;/code&gt; or &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;paper/&lt;/code&gt; directories.&lt;/p&gt; &lt;h2 id=&quot;troubleshooting&quot;&gt;Troubleshooting&lt;/h2&gt; &lt;p&gt;&lt;strong&gt;A search returns fewer results than expected, or none.&lt;/strong&gt; Look at &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;metadata.errors&lt;/code&gt; first. One provider can time out or rate-limit while the others answer, and the partial failure is recorded there. An empty result with errors means a source was unavailable, not that the literature is absent.&lt;/p&gt; &lt;p&gt;&lt;strong&gt;WTF-P cannot find CiteNexus.&lt;/strong&gt; Confirm &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;cite-nexus-wtfp --check&lt;/code&gt; works in the shell that launches Clio. If you installed into a virtual environment, export the absolute path first:&lt;/p&gt; &lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;nb&quot;&gt;export &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;WTFP_CITE_NEXUS_COMMAND&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$HOME&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/.local/share/cite-nexus/bin/cite-nexus-wtfp&quot;&lt;/span&gt; &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;&lt;strong&gt;An action returns &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;WTFP_ACTION_UNAVAILABLE&lt;/code&gt;.&lt;/strong&gt; That is a deliberate refusal. On CLIO Coder 31 of the 36 actions are available. The other five (&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;contribute&lt;/code&gt;, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;report-bug&lt;/code&gt;, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;request-feature&lt;/code&gt;, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;remove-section&lt;/code&gt;, and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;update&lt;/code&gt;) fail closed, so do not ask the model to improvise around them.&lt;/p&gt; &lt;h2 id=&quot;limitations&quot;&gt;Limitations&lt;/h2&gt; &lt;ul&gt; &lt;li&gt;I ran the install checks, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/wtfp:help&lt;/code&gt;, and the citation search above, and the two images are frames from my own session. The session prompts are illustrative.&lt;/li&gt; &lt;li&gt;All three projects are early software below 1.0. Interfaces can change, and a session is only as good as the model you point it at.&lt;/li&gt; &lt;li&gt;Linux only for now. The CiteNexus companion does not forward proxy or CA-bundle variables, so networks that require them are unsupported.&lt;/li&gt; &lt;li&gt;The research actions reach CiteNexus on CLIO Coder and Claude Code only.&lt;/li&gt; &lt;li&gt;WTF-P does not render, compile, submit, or touch Git. Poster, slide, and LaTeX actions emit source and hand you the build step.&lt;/li&gt; &lt;li&gt;Nothing here verifies a scientific claim. A candidate citation is a pointer, not proof, and a passed review is a checkpoint. Reading the paper and owning the argument remain your job, by design.&lt;/li&gt; &lt;/ul&gt; &lt;h2 id=&quot;try-it-break-it-tell-us&quot;&gt;Try it, break it, tell us&lt;/h2&gt; &lt;p&gt;I built this for my own students first. A first-year PhD student should be able to open a folder on Monday, be interviewed about what the paper claims, and on Friday still know why every sentence is there and where every citation came from.&lt;/p&gt; &lt;ul&gt; &lt;li&gt;&lt;strong&gt;CLIO Coder&lt;/strong&gt;: &lt;a href=&quot;https://github.com/iowarp/clio-coder&quot;&gt;star or try it&lt;/a&gt;, &lt;a href=&quot;https://github.com/iowarp/clio-coder/issues&quot;&gt;file an issue&lt;/a&gt;, &lt;a href=&quot;https://github.com/iowarp/clio-coder/blob/main/CONTRIBUTING.md&quot;&gt;contribute&lt;/a&gt;.&lt;/li&gt; &lt;li&gt;&lt;strong&gt;WTF-P&lt;/strong&gt;: &lt;a href=&quot;https://github.com/akougkas/wtf-p&quot;&gt;star or try it&lt;/a&gt;, &lt;a href=&quot;https://github.com/akougkas/wtf-p/issues&quot;&gt;file an issue&lt;/a&gt;, &lt;a href=&quot;https://github.com/akougkas/wtf-p/blob/main/CONTRIBUTING.md&quot;&gt;contribute&lt;/a&gt;.&lt;/li&gt; &lt;li&gt;&lt;strong&gt;CiteNexus&lt;/strong&gt;: &lt;a href=&quot;https://github.com/akougkas/cite-nexus-mcp&quot;&gt;star or try it&lt;/a&gt;, &lt;a href=&quot;https://github.com/akougkas/cite-nexus-mcp/issues&quot;&gt;file an issue&lt;/a&gt;, &lt;a href=&quot;https://github.com/akougkas/cite-nexus-mcp/blob/master/CONTRIBUTING.md&quot;&gt;contribute&lt;/a&gt;.&lt;/li&gt; &lt;/ul&gt; &lt;p&gt;The most useful report names the action you ran, what you expected, what happened, and the versions, with unpublished text removed. Students in my lab: start your next workshop paper this way and bring me what annoyed you. Everyone else, the invitation is the same, and a provider for your field or a venue template is a good first contribution.&lt;/p&gt; </content> </entry> <entry> <title type="html">Serving Qwen3.8-27B on a Radeon AI PRO R9700 is not a one-liner</title> <link rel="alternate" type="text/html" href="https://akougkas.io/journal/2026/serving-qwen38-27b-on-a-radeon-ai-pro-r9700/"/> <id>https://akougkas.io/journal/2026/serving-qwen38-27b-on-a-radeon-ai-pro-r9700/</id> <published>2026-09-08T00:00:00-05:00</published> <updated>2026-09-08T00:00:00-05:00</updated> <summary type="html">The same Qwen3.8-27B GGUF on the same 32 GB Radeon decodes at 27 or 46 tokens per second, and the difference is speculative decoding. The backend is a prefill and decode trade, KV precision is a memory choice, and this is the llama.cpp preset I run under CLIO Coder.</summary> <content type="html" xml:base="https://akougkas.io/journal/2026/serving-qwen38-27b-on-a-radeon-ai-pro-r9700/">&lt;p&gt;The same GGUF file, on the same card, decodes at 27 tokens per second or at 46. Nothing about the model changed between those two numbers. What changed were a handful of server flags that most “run a local model” guides never mention, and the right values depend on who is on the other end of the socket.&lt;/p&gt; &lt;p&gt;This note is about &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;Qwen/Qwen3.8-27B&lt;/code&gt; in Unsloth’s IQ4_NL quantization, served by llama.cpp on an AMD Radeon AI PRO R9700 with 32 GB of VRAM. I run it under &lt;a href=&quot;/work/clio-coder/&quot;&gt;CLIO Coder&lt;/a&gt;, and a coding agent is a demanding client. It ships 10k to 100k tokens of files per turn, so prefill matters as much as decode, and it runs several workers at once, so slots and how they share the KV cache matter. A person chatting needs one slot, short prompts, and fast decode. Several of the choices below come out differently for the two, and one that I expected to differ did not.&lt;/p&gt; &lt;p&gt;Everything here was measured on 2026-09-08 unless I say otherwise.&lt;/p&gt; &lt;h2 id=&quot;prerequisites&quot;&gt;Prerequisites&lt;/h2&gt; &lt;ul&gt; &lt;li&gt;An RDNA4 card with 32 GB. Mine is a Radeon AI PRO R9700 (Navi 48, gfx1201, 32624 MiB usable, 640 GB/s) in a Ryzen 7 5700X3D box with 8 cores, Ubuntu 24.04.4 and the 6.17 OEM kernel. A 24 GB card works with the recipe near the end.&lt;/li&gt; &lt;li&gt;ROCm 7.2.4 from the legacy &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;amdgpu-install&lt;/code&gt; line, with amdgpu 30.20.1 DKMS. For the Vulkan comparison, AMDVLK 2025.Q2.1, which is end of life and held in apt.&lt;/li&gt; &lt;li&gt;llama.cpp tag b10687 (commit c841aee, 2026-08-29), built twice as single-backend binaries: one HIP build with &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;-DGGML_HIP=ON -DGPU_TARGETS=gfx1201 -DGGML_HIP_NO_VMM=ON&lt;/code&gt;, and one Vulkan build. The server runs in router mode from a preset file, one router process with a worker binary per model.&lt;/li&gt; &lt;li&gt;The model files from &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;unsloth/Qwen3.8-27B-GGUF&lt;/code&gt;: &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;Qwen3.8-27B-IQ4_NL.gguf&lt;/code&gt; (15.21 GiB, 288 IQ4_NL and 113 Q5_K tensors) with the MTP &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;nextn&lt;/code&gt; layer included as blk.64 (mostly Q4_K, with &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;nextn.eh_proj&lt;/code&gt; in Q8_0), and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;mmproj-F16.gguf&lt;/code&gt; (0.9 GiB) for vision. The quality alternate is UD-Q6_K at 20.46 GiB.&lt;/li&gt; &lt;li&gt;Sampling from the official model card’s thinking mode: temperature 1.0, top-p 0.95, top-k 20, min-p 0, presence penalty 0, repeat penalty 1.0. Every measurement here uses those values, including the production preset, which keeps them and only turns thinking off by default. The card’s non-thinking values (temperature 0.7, top-p 0.8, presence penalty 1.5) are an alternative I did not run.&lt;/li&gt; &lt;li&gt;For the laptop section: a Strix Halo machine (Ryzen AI Max+ PRO 395, Radeon 8060S, gfx1151, 128 GB unified memory) on Windows 11 with Lemonade 11.9.0, which bundles a ROCm llama-server (TheRock 7.14 runtime, b10711) and a Vulkan one (b10723).&lt;/li&gt; &lt;/ul&gt; &lt;h2 id=&quot;what-the-model-actually-is&quot;&gt;What the model actually is&lt;/h2&gt; &lt;p&gt;The GGUF metadata explains most of what follows. Qwen3.8-27B has 64 layers with &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;full_attention_interval = 4&lt;/code&gt;: 16 full-attention layers and 48 Gated DeltaNet layers, and the attention layers have 4 KV heads with a 256 head dimension. Only the attention layers keep a KV cache, so a full 131k context costs about 7.2 GiB at f16, roughly 55 KB per token, 3.6 GiB at q8_0 (measured as the f16 minus q8_0 allocation delta), and 1.9 GiB at q4_0. Those are loaded allocations, not estimates. That cheapness is what makes four slots at 131k possible on one card.&lt;/p&gt; &lt;p&gt;The GGUF also carries the model’s own multi-token prediction head, the &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;nextn&lt;/code&gt; layer, which llama.cpp can use as a speculative drafter with &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;--spec-type draft-mtp&lt;/code&gt;. It is the single biggest lever in this note.&lt;/p&gt; &lt;p&gt;I cap the context at 131072 on purpose. llama.cpp issue 27756 reports this hybrid emitting an instant end-of-sequence beyond roughly 130k positions on both CUDA and CPU, so a larger window is a window the model cannot use.&lt;/p&gt; &lt;h2 id=&quot;the-bandwidth-wall&quot;&gt;The bandwidth wall&lt;/h2&gt; &lt;p&gt;Start with &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;llama-bench&lt;/code&gt; on the HIP build, one stream, flash attention on, q8_0 KV, a 2048-token prompt and 512 generated tokens:&lt;/p&gt; &lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;llama-bench &lt;span class=&quot;nt&quot;&gt;-m&lt;/span&gt; /path/to/Qwen3.8-27B-IQ4_NL.gguf &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-ngl&lt;/span&gt; 99 &lt;span class=&quot;nt&quot;&gt;-fa&lt;/span&gt; 1 &lt;span class=&quot;nt&quot;&gt;-ctk&lt;/span&gt; q8_0 &lt;span class=&quot;nt&quot;&gt;-ctv&lt;/span&gt; q8_0 &lt;span class=&quot;nt&quot;&gt;-p&lt;/span&gt; 2048 &lt;span class=&quot;nt&quot;&gt;-n&lt;/span&gt; 512 &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;Prefill is pp2048 and decode is tg512, both in tokens per second. Five Unsloth quantizations of the same model:&lt;/p&gt; &lt;table&gt; &lt;thead&gt; &lt;tr&gt; &lt;th&gt;quant&lt;/th&gt; &lt;th&gt;GiB&lt;/th&gt; &lt;th&gt;prefill t/s&lt;/th&gt; &lt;th&gt;decode t/s&lt;/th&gt; &lt;/tr&gt; &lt;/thead&gt; &lt;tbody&gt; &lt;tr&gt; &lt;td&gt;IQ4_NL&lt;/td&gt; &lt;td&gt;15.21&lt;/td&gt; &lt;td&gt;1287&lt;/td&gt; &lt;td&gt;29.8&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;UD-Q4_K_XL&lt;/td&gt; &lt;td&gt;16.34&lt;/td&gt; &lt;td&gt;1151&lt;/td&gt; &lt;td&gt;26.7&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;UD-Q5_K_M&lt;/td&gt; &lt;td&gt;18.40&lt;/td&gt; &lt;td&gt;946&lt;/td&gt; &lt;td&gt;24.5&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;UD-Q5_K_XL&lt;/td&gt; &lt;td&gt;19.43&lt;/td&gt; &lt;td&gt;904&lt;/td&gt; &lt;td&gt;23.9&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;UD-Q6_K&lt;/td&gt; &lt;td&gt;20.46&lt;/td&gt; &lt;td&gt;821&lt;/td&gt; &lt;td&gt;23.2&lt;/td&gt; &lt;/tr&gt; &lt;/tbody&gt; &lt;/table&gt; &lt;p&gt;Decode scales approximately inversely with file size. IQ4_NL is 15.21 GiB, which is 16.33 GB; multiply by 29.8 tokens per second and you get about 487 GB/s of effective bandwidth, about 76% of the card’s 640 GB/s. Single-stream decode on a dense model reads the whole file once per token, and no kernel setting moves that number.&lt;/p&gt; &lt;p&gt;I measured the settings people recommend against the IQ4_NL baseline and every one landed within about 3%: llama.cpp b10850 instead of b10687 (1275 prefill, 30.5 decode), &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;GGML_CUDA_FORCE_MMQ=1&lt;/code&gt; (1259), &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;GGML_CUDA_FORCE_CUBLAS=1&lt;/code&gt; (1262), &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;ROCBLAS_USE_HIPBLASLT=1&lt;/code&gt; (1246), a micro-batch of 1024 (1291) or 2048 (1270). The hybrid “dead band” reported for this architecture did not show up: tg128 was 29.99 and tg512 was 29.81. At 16k of depth the card still prefills at 755 and decodes at 26.6.&lt;/p&gt; &lt;p&gt;ROCm 10 came out on 2026-08-26, the first major version built on TheRock. I measured it on 2026-09-08 through Lemonade’s prebuilt ROCm 10.1 bundle: 1213 prefill and 30.1 decode, so prefill 6% lower and decode within 1% of ROCm 7.2.4. For llama.cpp on gfx1201, the ROCm version is not a lever.&lt;/p&gt; &lt;h2 id=&quot;pick-the-backend-for-the-workload&quot;&gt;Pick the backend for the workload&lt;/h2&gt; &lt;p&gt;The same IQ4_NL file through the HIP and Vulkan builds:&lt;/p&gt; &lt;table&gt; &lt;thead&gt; &lt;tr&gt; &lt;th&gt;backend&lt;/th&gt; &lt;th&gt;prefill t/s&lt;/th&gt; &lt;th&gt;decode t/s&lt;/th&gt; &lt;/tr&gt; &lt;/thead&gt; &lt;tbody&gt; &lt;tr&gt; &lt;td&gt;HIP (ROCm 7.2.4)&lt;/td&gt; &lt;td&gt;1287&lt;/td&gt; &lt;td&gt;29.8&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;Vulkan (AMDVLK 2025.Q2.1)&lt;/td&gt; &lt;td&gt;239&lt;/td&gt; &lt;td&gt;32.9&lt;/td&gt; &lt;/tr&gt; &lt;/tbody&gt; &lt;/table&gt; &lt;p&gt;In the bench harness Vulkan decodes 10% faster and prefills 5.4 times slower on this dense model. For a coding agent that hands over tens of thousands of tokens of source before asking a question, the prefill gap is the whole latency budget, and HIP wins outright. The obvious reading of the decode column is that Vulkan should still be the chat backend, where prompts are short and decode is the whole experience. I believed that until I measured it through a server.&lt;/p&gt; &lt;p&gt;The chat profile is one slot, a 600-token prompt, 32k context, q8_0 KV, the same IQ4_NL file, through the same standalone &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;llama-server&lt;/code&gt;:&lt;/p&gt; &lt;table&gt; &lt;thead&gt; &lt;tr&gt; &lt;th&gt;backend, drafter&lt;/th&gt; &lt;th&gt;decode t/s&lt;/th&gt; &lt;th&gt;accept&lt;/th&gt; &lt;th&gt;prefill t/s&lt;/th&gt; &lt;/tr&gt; &lt;/thead&gt; &lt;tbody&gt; &lt;tr&gt; &lt;td&gt;Vulkan, none&lt;/td&gt; &lt;td&gt;23.8 / 23.4&lt;/td&gt; &lt;td&gt; &lt;/td&gt; &lt;td&gt;205&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;Vulkan, MTP draft 2&lt;/td&gt; &lt;td&gt;20.9 / 28.0&lt;/td&gt; &lt;td&gt;61%&lt;/td&gt; &lt;td&gt;74&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;HIP, MTP draft 2&lt;/td&gt; &lt;td&gt;54.3 / 47.5&lt;/td&gt; &lt;td&gt;77%&lt;/td&gt; &lt;td&gt;710&lt;/td&gt; &lt;/tr&gt; &lt;/tbody&gt; &lt;/table&gt; &lt;p&gt;The 10% raw-decode edge does not survive a real server. Vulkan without speculation lands at 23.8, below its own bench number, and adding MTP on the AMDVLK path is a net loss: batched draft verification is slow there, so acceptance of 61% buys nothing and prefill falls to 74 tokens per second. HIP with MTP decodes at about twice Vulkan’s best in the same profile. On this dense model, HIP plus MTP wins for chat as well as for agents, by roughly two to one, and the decision is not a workload question at all.&lt;/p&gt; &lt;p&gt;Vulkan stays a mixture-of-experts choice on this card. On Ornith-35B-A3B, AMDVLK decodes at twice HIP’s rate (168 versus 83 tokens per second), which is why my router runs dense models on the HIP worker and MoE models on the Vulkan worker.&lt;/p&gt; &lt;h2 id=&quot;the-drafter&quot;&gt;The drafter&lt;/h2&gt; &lt;p&gt;The bandwidth wall bounds how many tokens come out of one read of the weights. Speculative decoding gets more than one: a cheap drafter proposes a few tokens and the big model verifies them in a single batched pass. Qwen3.8-27B ships its own drafter in the MTP head, so the questions are how deep to draft and whether a separate drafter does better.&lt;/p&gt; &lt;p&gt;These runs go through a real &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;llama-server&lt;/code&gt;: a 131k unified KV pool shared by four slots, a 12k-token prompt, 512 generated tokens, and the thinking-mode sampler at temperature 1.0. Where two decode numbers appear, they are two measurements of the same configuration.&lt;/p&gt; &lt;table&gt; &lt;thead&gt; &lt;tr&gt; &lt;th&gt;config&lt;/th&gt; &lt;th&gt;VRAM MiB&lt;/th&gt; &lt;th&gt;decode t/s&lt;/th&gt; &lt;th&gt;accept&lt;/th&gt; &lt;th&gt;prefill t/s&lt;/th&gt; &lt;/tr&gt; &lt;/thead&gt; &lt;tbody&gt; &lt;tr&gt; &lt;td&gt;no speculation&lt;/td&gt; &lt;td&gt;21896&lt;/td&gt; &lt;td&gt;27.1&lt;/td&gt; &lt;td&gt; &lt;/td&gt; &lt;td&gt;1031&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;MTP draft 2 (chosen)&lt;/td&gt; &lt;td&gt;24054&lt;/td&gt; &lt;td&gt;46.2 / 43.0&lt;/td&gt; &lt;td&gt;66%&lt;/td&gt; &lt;td&gt;979&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;MTP draft 3&lt;/td&gt; &lt;td&gt;24651&lt;/td&gt; &lt;td&gt;41.9 / 46.0&lt;/td&gt; &lt;td&gt;47%&lt;/td&gt; &lt;td&gt;972&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;MTP draft 4 (old preset)&lt;/td&gt; &lt;td&gt;25249&lt;/td&gt; &lt;td&gt;40.0 / 37.6&lt;/td&gt; &lt;td&gt;41%&lt;/td&gt; &lt;td&gt;971&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;DFlash2 Q8_0, n 7&lt;/td&gt; &lt;td&gt;28755&lt;/td&gt; &lt;td&gt;33.4 / 29.2&lt;/td&gt; &lt;td&gt;31%&lt;/td&gt; &lt;td&gt;739&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;DFlash2 Q8_0, n 5&lt;/td&gt; &lt;td&gt;27557&lt;/td&gt; &lt;td&gt;36.8 / 36.3&lt;/td&gt; &lt;td&gt;37%&lt;/td&gt; &lt;td&gt;810&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;DFlash2 Q4_K_M, n 5&lt;/td&gt; &lt;td&gt;26686&lt;/td&gt; &lt;td&gt;43.7 / 39.6&lt;/td&gt; &lt;td&gt;44%&lt;/td&gt; &lt;td&gt;880&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;UD-Q6_K, MTP draft 2&lt;/td&gt; &lt;td&gt;29133&lt;/td&gt; &lt;td&gt;38.2 / 33.0&lt;/td&gt; &lt;td&gt;77%&lt;/td&gt; &lt;td&gt;682&lt;/td&gt; &lt;/tr&gt; &lt;/tbody&gt; &lt;/table&gt; &lt;p&gt;The MTP head is free and it is the biggest win available. It ships inside the GGUF, its draft context costs 2.1 GiB of VRAM, and it takes single-stream decode from 27 to 46 tokens per second.&lt;/p&gt; &lt;p&gt;Tune the draft depth with the sampler you will actually run. Two weeks earlier I had swept this at temperature 0 and settled on draft 4 at 87% acceptance. Under the real thinking-mode sampler, draft 4 accepts 41% and draft 2 wins.&lt;/p&gt; &lt;p&gt;DFlash2 (a block-diffusion drafter from incoai, supported in b10687) is slower than MTP at every setting on this stack, costs 15 to 28% of prefill, and takes 5 to 7 GiB of VRAM out of the KV budget. The published 3.43 times speedup was measured on CUDA with GSM8K prompts. On a long-context summarize-then-code task with sampling, acceptance is 31 to 44%.&lt;/p&gt; &lt;p&gt;Under four concurrent streams, speculation buys almost nothing. “Four-way cached” means the four 12k prompts were prefilled first and then decoded together; the numbers are aggregate throughput:&lt;/p&gt; &lt;table&gt; &lt;thead&gt; &lt;tr&gt; &lt;th&gt;config&lt;/th&gt; &lt;th&gt;4-way cached t/s&lt;/th&gt; &lt;/tr&gt; &lt;/thead&gt; &lt;tbody&gt; &lt;tr&gt; &lt;td&gt;no speculation&lt;/td&gt; &lt;td&gt;47.5&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;MTP draft 2&lt;/td&gt; &lt;td&gt;53.2&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;MTP draft 3&lt;/td&gt; &lt;td&gt;53.6&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;UD-Q6_K, MTP draft 2&lt;/td&gt; &lt;td&gt;39.7&lt;/td&gt; &lt;/tr&gt; &lt;/tbody&gt; &lt;/table&gt; &lt;p&gt;The GPU is already saturated by the batch. Speculation buys latency when the device is idle and costs compute when it is not, so it helps the one worker waiting on a reply and does little for four running at once.&lt;/p&gt; &lt;p&gt;Firing four uncached 12k prompts at once collapses each stream to 5 to 19 tokens per second while prefill chunks interleave with decode. That is continuous batching working as designed, and in an agent loop the prompt cache hides it because most of each turn’s prompt is the previous turn’s.&lt;/p&gt; &lt;h2 id=&quot;kv-cache-type&quot;&gt;KV cache type&lt;/h2&gt; &lt;p&gt;With IQ4_NL, MTP draft 2, 131k of unified context and four slots, the KV cache type changes memory and nothing else:&lt;/p&gt; &lt;table&gt; &lt;thead&gt; &lt;tr&gt; &lt;th&gt;KV type&lt;/th&gt; &lt;th&gt;VRAM MiB&lt;/th&gt; &lt;th&gt;decode t/s&lt;/th&gt; &lt;th&gt;4-way t/s&lt;/th&gt; &lt;th&gt;headroom MiB&lt;/th&gt; &lt;/tr&gt; &lt;/thead&gt; &lt;tbody&gt; &lt;tr&gt; &lt;td&gt;q4_0&lt;/td&gt; &lt;td&gt;22326&lt;/td&gt; &lt;td&gt;43.3 / 45.0&lt;/td&gt; &lt;td&gt;54.1&lt;/td&gt; &lt;td&gt;10107&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;q8_0&lt;/td&gt; &lt;td&gt;24054&lt;/td&gt; &lt;td&gt;46.2 / 43.0&lt;/td&gt; &lt;td&gt;53.2&lt;/td&gt; &lt;td&gt;8377&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;f16&lt;/td&gt; &lt;td&gt;27668&lt;/td&gt; &lt;td&gt;44.9 / 45.1&lt;/td&gt; &lt;td&gt;56.4&lt;/td&gt; &lt;td&gt;4758&lt;/td&gt; &lt;/tr&gt; &lt;/tbody&gt; &lt;/table&gt; &lt;p&gt;All three are within noise on speed. The KV type is purely a trade of memory for quality on this card, so the rule is simple: spend headroom on f16 KV before anything else, and quantize the cache only when the alternative is dropping context or slots. My default preset carries f16 KV on IQ4_NL, which leaves 4758 MiB of headroom, about 4.6 GiB. UD-Q6_K runs q8_0 KV instead: f16 at 131k does load, but with 41 MiB to spare (32543 MiB resident), and even at 96k it leaves only 1993 MiB. Neither is a margin I want under a compute graph, so the Q6_K preset takes the q8_0 cache.&lt;/p&gt; &lt;h2 id=&quot;the-preset-i-run&quot;&gt;The preset I run&lt;/h2&gt; &lt;p&gt;This is the &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;[qwen3.8-27b-dense]&lt;/code&gt; block from the router’s preset file on the machine that serves CLIO Coder:&lt;/p&gt; &lt;div class=&quot;language-ini highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;nn&quot;&gt;[qwen3.8-27b-dense]&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;py&quot;&gt;model&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;.../Qwen3.8-27B-IQ4_NL.gguf&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;mmproj&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;.../mmproj-F16.gguf&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;ctx-size&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;131072&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;kv-unified&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;true&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;parallel&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;4&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;cache-type-k&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;f16&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;cache-type-v&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;f16&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;flash-attn&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;true&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;batch-size&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;2048&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;ubatch-size&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;512&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;spec-type&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;draft-mtp&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;spec-draft-n-max&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;2&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;n-gpu-layers-draft&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;99&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;temperature&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;1.0&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;top-p&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;0.95&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;top-k&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;20&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;min-p&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;presence-penalty&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;0&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;repeat-penalty&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;1.0&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;jinja&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;true&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;reasoning&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;off&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;reasoning-effort&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;high&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;chat-template-kwargs&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;{&quot;enable_thinking&quot;: false, &quot;preserve_thinking&quot;: true}&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;ctx-checkpoints&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;64&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;checkpoint-min-step&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;1024&lt;/span&gt; &lt;span class=&quot;py&quot;&gt;sleep-idle-seconds&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;&lt;span class=&quot;s&quot;&gt;300&lt;/span&gt; &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;Unified KV makes the 131k context one pool shared by the four slots, so any single request may use all of it and slots do not multiply the allocation. The checkpoint pair saves the recurrent state of the DeltaNet layers at intervals, which is what lets prompt-cache reuse work on a hybrid at all. &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;sleep-idle-seconds&lt;/code&gt; unloads the worker after five idle minutes so the card is free for whatever else the router serves.&lt;/p&gt; &lt;p&gt;Through the production router with this preset (f16 KV, thinking off), a cold 12k prompt decodes at 47.5 tokens per second single-stream with 67% draft acceptance and 892 tokens per second of prefill. Four cached streams give 47.7 tokens per second aggregate, 20 to 22.5 per stream. Those two numbers measure different things. Per-stream is the decode rate one stream sees while all four generate. The aggregate divides every generated token by the wall time of the whole four-request batch, including the prompt work the cache did not cover, which is why 47.7 aggregate coexists with 20 to 22.5 per stream. The server sits at 27526 MiB resident with 5098 MiB of headroom, and its log confirms &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;n_slots = 4, n_ctx_slot = 131072, kv_unified = true&lt;/code&gt;. An earlier run of the same recipe with q8_0 KV gave 44.0 single-stream at 61.5% acceptance, 966 prefill, 51.7 aggregate, and 24157 MiB resident, which is the q8_0 row of the KV table seen through the router rather than the bench harness.&lt;/p&gt; &lt;p&gt;One caution about that comparison. The preset keeps the thinking-mode sampler values (temperature 1.0, top-p 0.95, top-k 20) and only turns thinking off by default, so the production runs and the sweeps above share a sampler. What differs is whether the model emits reasoning before its answer, and acceptance moves with the text being drafted, so the two sets of numbers are close but not the same experiment. The model card’s non-thinking sampler is an alternative, not what runs here.&lt;/p&gt; &lt;p&gt;Thinking is off in the preset and turned on per request. I checked: by default &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;reasoning_content&lt;/code&gt; comes back empty, and the same question with &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;chat_template_kwargs&lt;/code&gt; set to &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;{&quot;enable_thinking&quot;: true, &quot;reasoning_effort&quot;: &quot;low&quot;}&lt;/code&gt; returns about 110 to 120 characters of reasoning. When a request enables thinking without naming an effort, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;reasoning-effort = high&lt;/code&gt; in the preset is the server default. In an OpenAI-compatible body, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;chat_template_kwargs&lt;/code&gt; accepts &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;enable_thinking&lt;/code&gt; (true or false), &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;reasoning_effort&lt;/code&gt; (low, medium, high, or xhigh; on this model high is an alias of xhigh), and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;preserve_thinking&lt;/code&gt; (true or false).&lt;/p&gt; &lt;p&gt;Vision goes through the same endpoint: send an &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;image_url&lt;/code&gt; content part and the projector loaded next to the model handles it. I verified it on both presets with a generated 128 px PNG of a red circle. The IQ4_NL model described one round shape; the Q6_K model described one red circle.&lt;/p&gt; &lt;p&gt;The preset file carries a second section, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;[qwen3.8-27b-dense-q6]&lt;/code&gt;, for UD-Q6_K with the same recipe and q8_0 KV. It sits at 29199 MiB resident with 3.4 GiB of headroom and decodes at 33 to 38 tokens per second single-stream, with 77% draft acceptance and 682 tokens per second of prefill. That is the quality option: about 10 tokens per second slower and a 20 to 30% slower prefill for the larger weights.&lt;/p&gt; &lt;h2 id=&quot;fitting-it-on-24-gb&quot;&gt;Fitting it on 24 GB&lt;/h2&gt; &lt;p&gt;If your card has 24 GB rather than 32, the same recipe fits with a few deliberate cuts. The resident components measured on the R9700:&lt;/p&gt; &lt;table&gt; &lt;thead&gt; &lt;tr&gt; &lt;th&gt;component&lt;/th&gt; &lt;th&gt;MiB&lt;/th&gt; &lt;/tr&gt; &lt;/thead&gt; &lt;tbody&gt; &lt;tr&gt; &lt;td&gt;IQ4_NL weights&lt;/td&gt; &lt;td&gt;15575&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;mmproj-F16&lt;/td&gt; &lt;td&gt;885&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;MTP draft context&lt;/td&gt; &lt;td&gt;~2160&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;compute buffers, recurrent state&lt;/td&gt; &lt;td&gt;~1800&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;KV at 131k, f16&lt;/td&gt; &lt;td&gt;~7200&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;KV at 131k, q8_0&lt;/td&gt; &lt;td&gt;~3600&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;KV at 131k, q4_0&lt;/td&gt; &lt;td&gt;~1900&lt;/td&gt; &lt;/tr&gt; &lt;/tbody&gt; &lt;/table&gt; &lt;p&gt;A 24 GB card has 24576 MiB. Leave 1500 for fragmentation and the compute graph and about 23000 is usable. Weights, projector, MTP, and buffers come to 20420, which leaves 2580 for the KV cache: 131k does not fit at f16 or q8_0, and q4_0 at 1900 fits with almost nothing to spare.&lt;/p&gt; &lt;p&gt;The drop order that keeps the agent use case intact:&lt;/p&gt; &lt;ol&gt; &lt;li&gt;KV f16 to q8_0. Saves 3.6 GiB at 131k and costs nothing in speed.&lt;/li&gt; &lt;li&gt;Context 131k to 64k. Halves the KV again.&lt;/li&gt; &lt;li&gt;The projector, if you do not need vision. 0.9 GiB.&lt;/li&gt; &lt;li&gt;MTP last. It is worth 19 tokens per second of decode and nothing else in the list is.&lt;/li&gt; &lt;/ol&gt; &lt;p&gt;The concrete 24 GB recipe for an agent: IQ4_NL, MTP draft 2, q8_0 KV, 64k unified context, 2 slots, projector optional. Expect about 21.5 GiB resident and the same 44 tokens per second single-stream, provided the bandwidth matches. An RX 9070 XT is 640 GB/s like the R9700; a 7900 XTX is 960 GB/s and will decode faster.&lt;/p&gt; &lt;p&gt;For chat only on 24 GB: the HIP build, 1 slot, 32k context, MTP draft 2, q8_0 KV. That is the chat profile measured above, and it decodes at 47.5 to 54.3 tokens per second on the R9700. Do not reach for Vulkan on a dense model here; the chat table shows why.&lt;/p&gt; &lt;p&gt;Do not go below IQ4_NL for speed. UD-IQ4_XS (13.27 GiB) gains about 6% decode for a quality drop that is not worth it.&lt;/p&gt; &lt;h2 id=&quot;strix-halo-laptops&quot;&gt;Strix Halo laptops&lt;/h2&gt; &lt;p&gt;The other machine this model has to run on is a Strix Halo laptop: a Ryzen AI Max+ PRO 395 with the Radeon 8060S (gfx1151) and 128 GB of unified memory, on Windows 11 with Lemonade 11.9.0. Unified memory changes the question. Context is not the constraint there, since 131k of KV is a rounding error in 128 GB. Bandwidth is the constraint, at about 256 GB/s, so the MTP head matters even more than on the desktop card.&lt;/p&gt; &lt;p&gt;The profile: &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;unsloth/Qwen3.8-27B-GGUF&lt;/code&gt; in UD-Q4_K_XL with the projector, 131k context, one slot, 12k-token prompts, thinking off, the same sampler. Lemonade offers two llama.cpp backends, ROCm (TheRock 7.14 runtime, llama-server b10711) and Vulkan (b10723), and I measured both through its server on 2026-09-08.&lt;/p&gt; &lt;table&gt; &lt;thead&gt; &lt;tr&gt; &lt;th&gt;backend, drafter&lt;/th&gt; &lt;th&gt;decode t/s&lt;/th&gt; &lt;th&gt;accept&lt;/th&gt; &lt;th&gt;prefill t/s&lt;/th&gt; &lt;/tr&gt; &lt;/thead&gt; &lt;tbody&gt; &lt;tr&gt; &lt;td&gt;Vulkan, none&lt;/td&gt; &lt;td&gt;10.1&lt;/td&gt; &lt;td&gt; &lt;/td&gt; &lt;td&gt;53&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;Vulkan, MTP draft 2&lt;/td&gt; &lt;td&gt;14 to 18&lt;/td&gt; &lt;td&gt;72%&lt;/td&gt; &lt;td&gt;118&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;ROCm, none&lt;/td&gt; &lt;td&gt;9.9&lt;/td&gt; &lt;td&gt; &lt;/td&gt; &lt;td&gt;169&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;ROCm, MTP draft 2&lt;/td&gt; &lt;td&gt;12 to 14&lt;/td&gt; &lt;td&gt;54 to 78%&lt;/td&gt; &lt;td&gt;205 to 230&lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;ROCm, MTP draft 4&lt;/td&gt; &lt;td&gt;10.2&lt;/td&gt; &lt;td&gt;38%&lt;/td&gt; &lt;td&gt; &lt;/td&gt; &lt;/tr&gt; &lt;tr&gt; &lt;td&gt;ROCm, MTP draft 6&lt;/td&gt; &lt;td&gt;7.7&lt;/td&gt; &lt;td&gt;24%&lt;/td&gt; &lt;td&gt; &lt;/td&gt; &lt;/tr&gt; &lt;/tbody&gt; &lt;/table&gt; &lt;p&gt;The ROCm rows needed a fix before they existed. At the default micro-batch of 512, and again at 384, the ROCm llama-server crashed on every prompt past about 3k tokens. Lemonade’s watchdog reset the process and returned HTTP 500 to the client. Short prompts never triggered it, so the deployed configuration had looked healthy for as long as nobody sent it a real agent turn. With &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;-b 512 -ub 256&lt;/code&gt; the ROCm backend works, and those are the numbers in the table. Two slots made the second MTP stream misbehave, so the laptop runs one.&lt;/p&gt; &lt;p&gt;The conclusion splits by workload in a way the R9700 did not. ROCm wins agent turns because its prefill is about two to three times Vulkan’s: a 12k prompt plus 256 generated tokens takes about 80 seconds on ROCm and about 120 on Vulkan. Vulkan wins short chat on decode alone. Draft depth follows the same rule as the desktop: 2 is the right number under this sampler, and 4 or 6 lose acceptance faster than they gain tokens.&lt;/p&gt; &lt;p&gt;The saved Lemonade options are now ROCm, 131072 context, MTP draft 2, one slot, and the micro-batch fix:&lt;/p&gt; &lt;div class=&quot;language-text highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;lemonade load Qwen3.8-27B-GGUF --llamacpp rocm --no-merge-args \ --ctx-size 131072 \ --llamacpp-args &quot;--spec-type draft-mtp --spec-draft-n-max 2 --parallel 1 -b 512 -ub 256&quot; \ --save-options &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;One more thing worth knowing about a laptop that stays on. An instance that had been resident for days produced a string of slashes at 5 tokens per second once. Unloading and reloading the model cleared it. A health check that only reads status would have reported that server as fine; the check has to generate something and look at it.&lt;/p&gt; &lt;h2 id=&quot;what-was-not-a-lever&quot;&gt;What was not a lever&lt;/h2&gt; &lt;p&gt;To save you the afternoon: the ROCm version (7.2.4 against 10.1), the llama.cpp build (b10687 against b10850), GEMM backend selection and hipBLASLt, and the KV cache type made no difference to speed on this model and this card. Larger micro-batches did nothing for prefill, and an earlier greedy sweep showed they hurt MTP acceptance. Vulkan looked like a lever for chat in the bench harness and was not one in a server. What is left is the size of the model file, the drafter, and the HIP backend under it. Everything else is memory layout.&lt;/p&gt; &lt;h2 id=&quot;limitations&quot;&gt;Limitations&lt;/h2&gt; &lt;ul&gt; &lt;li&gt;One desktop card, one laptop, one model family, one week of data. The bandwidth arithmetic should hold on other RDNA4 parts; the specific numbers will not.&lt;/li&gt; &lt;li&gt;Every number is a snapshot of llama.cpp b10687 on 2026-09-08. Speculative decoding and the hybrid cache code are moving quickly.&lt;/li&gt; &lt;li&gt;AMDVLK 2025.Q2.1 is end of life, so the Vulkan numbers reflect a driver that will not improve. RADV or a newer AMDVLK may close the batched-verification gap; I have not measured them.&lt;/li&gt; &lt;li&gt;The four-way aggregate numbers are for cached prompts. Cold concurrent prefill behaves very differently, as described above.&lt;/li&gt; &lt;li&gt;Throughput is not coding quality. I have not ranked these model or KV quantizations on task quality, and the KV rule above is a configuration preference, not a measured quality score.&lt;/li&gt; &lt;li&gt;The paired decode numbers are two measurements, not confidence intervals. Prompt depth, sampling, cache reuse, and concurrent arrivals all move the result.&lt;/li&gt; &lt;li&gt;The 24 GB recipe is a budget derived from components measured on a 32 GB card, not a run on a 24 GB card.&lt;/li&gt; &lt;li&gt;The laptop numbers are for Lemonade 11.9.0 on Windows 11 with one slot. The ROCm micro-batch crash and its workaround are specific to TheRock 7.14 and llama-server b10711 on gfx1151 and may not survive the next Lemonade release.&lt;/li&gt; &lt;/ul&gt; &lt;h2 id=&quot;where-this-fits&quot;&gt;Where this fits&lt;/h2&gt; &lt;p&gt;A harness is only as good as the runtime under it. &lt;a href=&quot;/work/clio-coder/&quot;&gt;CLIO Coder&lt;/a&gt; is one hands-on component of &lt;a href=&quot;/work/clio/&quot;&gt;CLIO&lt;/a&gt;, the context architecture I lead inside &lt;a href=&quot;/work/iowarp/&quot;&gt;IOWarp&lt;/a&gt;, and the premise of that program is that the model is not the system. The receipts, the bounded context, and the typed authority in the harness only matter if tokens arrive fast enough for an agent to finish a turn. On this card, speculative decoding is the difference between 27 and 46 tokens per second from the same file, and prefill and slots are what decide whether an agent’s turn finishes at all. Those are the numbers a chat guide never shows you, and they are the ones the harness lives on.&lt;/p&gt; </content> </entry> <entry> <title type="html">Run CLIO Coder against a model on your own GPU</title> <link rel="alternate" type="text/html" href="https://akougkas.io/journal/2026/clio-coder-on-your-own-gpu/"/> <id>https://akougkas.io/journal/2026/clio-coder-on-your-own-gpu/</id> <published>2026-08-26T18:35:00-05:00</published> <updated>2026-08-27T00:00:00-05:00</updated> <summary type="html">Install CLIO Coder from npm, point it at a local llama.cpp or LM Studio server, run one headless turn, and read the receipt it seals.</summary> <content type="html" xml:base="https://akougkas.io/journal/2026/clio-coder-on-your-own-gpu/">&lt;p class=&quot;entry-note&quot;&gt;&lt;strong&gt;September 27, 2026:&lt;/strong&gt; This walkthrough retains its original tested CLIO Coder v0.3.7 instructions. Starting with &lt;a href=&quot;https://github.com/iowarp/clio-coder/releases/tag/v0.5.6&quot;&gt;v0.5.6&lt;/a&gt;, the old autonomy names are rejected; &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;default&lt;/code&gt; and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;yolo&lt;/code&gt; are the supported levels, and read-only is a separate dispatch restriction. Use the &lt;a href=&quot;https://coder.iowarp.ai/docs.html&quot;&gt;current product guides&lt;/a&gt; for a new installation. The full workflow below has not been retested against v0.5.7.&lt;/p&gt; &lt;p&gt;Most of the scientists I work with have exactly one GPU and code they cannot send to a cloud API. &lt;a href=&quot;/work/clio-coder/&quot;&gt;CLIO Coder&lt;/a&gt; was built for that room: a coding agent that runs against a model on your own hardware and seals a receipt you can read afterwards.&lt;/p&gt; &lt;p&gt;CLIO Coder treats a model as a named &lt;strong&gt;target&lt;/strong&gt;: a runtime, an endpoint, a model id, and credentials. Everything below is documented in the v0.3.7 README and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;--help&lt;/code&gt;; I have only put it in the order I actually follow on a fresh machine.&lt;/p&gt; &lt;h2 id=&quot;prerequisites&quot;&gt;Prerequisites&lt;/h2&gt; &lt;ul&gt; &lt;li&gt;Node.js 22.19 or newer and npm.&lt;/li&gt; &lt;li&gt;Linux or macOS. Windows is best effort until a stable release.&lt;/li&gt; &lt;li&gt;A local inference server. Any of llama.cpp, Ollama, LM Studio, vLLM, or SGLang works; the runtime ids are &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;llamacpp&lt;/code&gt;, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;ollama-native&lt;/code&gt;, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;lmstudio&lt;/code&gt;, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;vllm&lt;/code&gt;, and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;sglang&lt;/code&gt;.&lt;/li&gt; &lt;li&gt;A model loaded in that server. If you have one GPU with 24 GB or more, the release was hardened against &lt;strong&gt;Qwen3.8-27B&lt;/strong&gt; (the &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;unsloth/Qwen3.8-27B-GGUF&lt;/code&gt; quantizations) on llama.cpp and LM Studio. A 4-bit quantization at 131072 context fits in 24 GB with a q8_0 KV cache.&lt;/li&gt; &lt;/ul&gt; &lt;p&gt;Version as of August 26, 2026:&lt;/p&gt; &lt;div class=&quot;language-text highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;$ clio-coder --version Clio Coder 0.3.7 &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;h2 id=&quot;1-install&quot;&gt;1. Install&lt;/h2&gt; &lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;npm &lt;span class=&quot;nb&quot;&gt;install&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-g&lt;/span&gt; @iowarp/clio-coder clio-coder &lt;span class=&quot;nt&quot;&gt;--version&lt;/span&gt; &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;From source instead, pinned to the release:&lt;/p&gt; &lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;git clone &lt;span class=&quot;nt&quot;&gt;--branch&lt;/span&gt; v0.3.7 https://github.com/iowarp/clio-coder.git &lt;span class=&quot;nb&quot;&gt;cd &lt;/span&gt;clio-coder npm run &lt;span class=&quot;nb&quot;&gt;install&lt;/span&gt;:local &lt;span class=&quot;nb&quot;&gt;export &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;PATH&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$HOME&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/.local/bin:&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$PATH&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;nb&quot;&gt;hash&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-r&lt;/span&gt; &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;npm run install:local&lt;/code&gt; builds the CLI, links it at &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;${CLIO_CODER_BIN_DIR:-$HOME/.local/bin}/clio-coder&lt;/code&gt;, and initializes the home directory. If an older install is on your &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;PATH&lt;/code&gt;, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;command -v clio-coder&lt;/code&gt; shows which binary the bare name reaches.&lt;/p&gt; &lt;h2 id=&quot;2-start-the-model-server&quot;&gt;2. Start the model server&lt;/h2&gt; &lt;p&gt;For llama.cpp, start it with the flags the README asks for so tool calls and reasoning parse cleanly:&lt;/p&gt; &lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;llama-server &lt;span class=&quot;nt&quot;&gt;--jinja&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;--reasoning&lt;/span&gt; on &lt;span class=&quot;nt&quot;&gt;-m&lt;/span&gt; /path/to/Qwen3.8-27B-Q4_K_M.gguf &lt;span class=&quot;nt&quot;&gt;-c&lt;/span&gt; 131072 &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;LM Studio needs nothing beyond loading the model. Note the port your server listens on; 8080 is llama.cpp’s default and 1234 is LM Studio’s.&lt;/p&gt; &lt;h2 id=&quot;3-configure-a-target&quot;&gt;3. Configure a target&lt;/h2&gt; &lt;p&gt;The interactive way:&lt;/p&gt; &lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;clio-coder configure &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;The wizard lists runtimes, asks for the endpoint and model, probes it, and saves it as both the chat and worker target. Bare &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;clio-coder&lt;/code&gt; opens the same wizard when nothing usable is configured.&lt;/p&gt; &lt;p&gt;The scripted way, which does the same thing without prompts:&lt;/p&gt; &lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;clio-coder configure &lt;span class=&quot;nt&quot;&gt;--id&lt;/span&gt; local-lmstudio &lt;span class=&quot;nt&quot;&gt;--runtime&lt;/span&gt; lmstudio &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;--url&lt;/span&gt; http://localhost:1234 &lt;span class=&quot;nt&quot;&gt;--model&lt;/span&gt; your-model-id &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;--set-orchestrator&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;--set-fleet-default&lt;/span&gt; clio-coder targets &lt;span class=&quot;nt&quot;&gt;--probe&lt;/span&gt; &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;clio-coder targets&lt;/code&gt; prints every configured target with its tier, runtime, auth state, URL, model, health, and capability flags. &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;--probe&lt;/code&gt; actually connects. The model id has to match what the server advertises: if LM Studio has a different model resident than the one in your target, the turn fails immediately with a message naming the resident instances instead of silently routing to whatever is loaded. That is deliberate.&lt;/p&gt; &lt;h2 id=&quot;4-run-one-headless-turn&quot;&gt;4. Run one headless turn&lt;/h2&gt; &lt;p&gt;From any repository:&lt;/p&gt; &lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;clio-coder run &lt;span class=&quot;s2&quot;&gt;&quot;Summarize this repository layout and its entry points.&quot;&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;--json&lt;/span&gt; &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;--json&lt;/code&gt; streams JSONL events: a &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;session&lt;/code&gt; record with the session id, working directory, target, model, and Clio version; &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;agent_start&lt;/code&gt;, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;turn_start&lt;/code&gt;, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;message_start&lt;/code&gt;, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;message_end&lt;/code&gt;, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;turn_end&lt;/code&gt;, and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;agent_end&lt;/code&gt;; and tool events in between when the model calls tools. The &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;agent_end&lt;/code&gt; event carries measured usage (input, output, cache reads and writes, reasoning tokens, API calls, and cost in dollars). Add &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;--autonomy read-only&lt;/code&gt; if you want the guarantee that nothing can be mutated or executed on this turn; every mutating call is denied and the denial is recorded.&lt;/p&gt; &lt;p&gt;To hand the same task to a fleet agent instead of the main loop:&lt;/p&gt; &lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;clio-coder run &lt;span class=&quot;s2&quot;&gt;&quot;&amp;lt;task&amp;gt;&quot;&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;--agent&lt;/span&gt; coder &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;That run gets its own receipt.&lt;/p&gt; &lt;h2 id=&quot;5-read-the-receipt&quot;&gt;5. Read the receipt&lt;/h2&gt; &lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;clio-coder evidence list clio-coder evidence inspect &amp;lt;runId&amp;gt; &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;A receipt covers routing intent, the resolved route, worker attestation, priced cost, phase timing, tool activity, safety decisions, and result conformance. Inside an interactive session, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/view verify &amp;lt;runId&amp;gt;&lt;/code&gt; shows the same thing. &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;clio-coder trace&lt;/code&gt; reads the durable dispatch trace.&lt;/p&gt; &lt;h2 id=&quot;6-go-interactive&quot;&gt;6. Go interactive&lt;/h2&gt; &lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;nb&quot;&gt;cd &lt;/span&gt;your-project clio-coder context init &lt;span class=&quot;c&quot;&gt;# drafts CLIO-CODER.md from the source tree&lt;/span&gt; clio-coder &lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt; &lt;p&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;context init&lt;/code&gt; writes the project handbook Clio loads every session and can adopt an existing &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;CLAUDE.md&lt;/code&gt;, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;AGENTS.md&lt;/code&gt;, or &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;GEMINI.md&lt;/code&gt; with provenance. In the session, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/settings&lt;/code&gt; changes the model or autonomy level, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/context&lt;/code&gt; and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/cost&lt;/code&gt; show what is in the window and what it costs, and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/help&lt;/code&gt; lists the rest.&lt;/p&gt; &lt;h2 id=&quot;limitations&quot;&gt;Limitations&lt;/h2&gt; &lt;ul&gt; &lt;li&gt;CLIO Coder is experimental. Interfaces may change between minor versions, and model-specific behavior varies by target. Keep the repository under version control and review what it proposes.&lt;/li&gt; &lt;li&gt;A local model’s usefulness depends on the quantization, the context length you can afford, and whether tool calls parse cleanly. The model catalog in the repository records what has been checked per family; other models may need &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;--jinja&lt;/code&gt; or a different chat template.&lt;/li&gt; &lt;li&gt;The receipt records what the harness observed. It does not make the model’s reasoning correct.&lt;/li&gt; &lt;li&gt;Connecting a Claude or ChatGPT subscription over OAuth uses the same path as the vendors’ own tools; using subscription credentials outside a vendor’s first-party apps may not align with their terms. That is your call, not Clio’s.&lt;/li&gt; &lt;li&gt;Windows is best effort.&lt;/li&gt; &lt;/ul&gt; &lt;p&gt;If something in here does not match what you see, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;clio-coder doctor&lt;/code&gt; plus &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;clio-coder --version&lt;/code&gt; and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;node --version&lt;/code&gt; is what I need in a bug report. Redact secrets and private prompts first.&lt;/p&gt; &lt;h2 id=&quot;where-this-fits&quot;&gt;Where this fits&lt;/h2&gt; &lt;p&gt;The receipt is the point. CLIO Coder is one hands-on surface inside &lt;a href=&quot;/work/clio/&quot;&gt;CLIO&lt;/a&gt;, the context architecture I lead across &lt;a href=&quot;/work/iowarp/&quot;&gt;IOWarp&lt;/a&gt;, and the same ideas about bounded selection, typed authority, and sealed evidence are what I argue for in &lt;a href=&quot;/journal/2026/context-is-part-of-the-system/&quot;&gt;Context is part of the system&lt;/a&gt;. If a local model is useful here, it is because the harness around it does the work a model cannot do for itself.&lt;/p&gt; </content> </entry> </feed>