Paper Instruments Says It Built Fable-5 for $300K

A Startup Says It Turned GLM Into a Fable-5-Class Model

On October 14th, a company called Paper Instruments released Ultramarine 1 Flash. It’s a model tailored for professional work in finance, legal, consulting, and operations teams — producing outputs like due-diligence memos, financial models, and board reports. The base is GLM-5.3 Flash, an open-weight¹ model that China’s Z.ai released under an MIT license, on top of which the company ran large-scale reinforcement learning² for post-training³. The API went live immediately, and the weights are slated for release soon. This is the same company that previously shipped Kos-1, a medical model.

The numbers made the rounds first. CEO Daanish Khazi wrote on X that a team of 5 post-trained the model for about $300,000, all costs included, and the company’s page describes it as “Fable-class, at 60x less cost.” The opening paragraph of the technical report states that at $0.13 per task, it beat both Claude Fable 5 ($8.19) and GPT-5.6 Sol ($2.09).

I want to look at 2 things here. First, how far these numbers actually hold up. Depending on which models were included in the comparison table, under what settings, and who did the grading, the scope of “outperformed” shifts quite a bit. Second, the method itself. In coding, you can reward a model simply for passing tests, but a due-diligence report has to be judged not just on whether the facts are right, but on whether the judgment behind it was sound. How that scoring was built is probably the part of this announcement that will matter longest. Back in issue 208 from August 22nd, writing about Harvey, I argued that the work environment itself is the asset — this time, I’ll also look at the fact that the environment was built by a model, not by people.

It Won 4 of 6, and the Table Also Hides Pricier and Cheaper Rivals

The report uses six benchmarks for knowledge work. Four are external — JobBench, FrontierFinance, AutomationBench, and APEX-Agents 1.1 — plus DiligenceBench, which Paper Instruments built itself, and PI Knowledge Work, an internal evaluation. Ultramarine beat its base model, GLM-5.3 Flash, on all six, by an average of 4.1 points, while using 6.5% fewer tokens. The strongest evidence that post-training actually worked is this “versus base model” line: same model, same harness, with training as the only variable.

Against Fable 5, it led on four of the six: JobBench 62.9 vs. 57.4, PI Knowledge Work 59.3 vs. 57.8, FrontierFinance 53.3 vs. 49.2, and AutomationBench 50.2 vs. 46.2. It lost the other two — DiligenceBench 50.6 vs. 52.9 and APEX-Agents 1.1 57.0 vs. 63.6 — and on both of those, Gemini 3.8 Flash actually topped the leaderboard. Worth noting: one of the two losses is on a benchmark the company built itself. On two coding benchmarks included for reference, Terminal-Bench 2.1 and DeepSWE, it scored 1 to 2 points below the base model. The table itself shows that pushing training toward professional work costs a little coding ability.

The “60x cheaper” claim comes from just one source: PI Knowledge Work. At cost per task, Ultramarine scored 59.3 for $0.13, versus Fable 5’s 57.8 for $8.19 — a 63x cost difference. On the same table, Claude Opus 5.5 scores 61.9 at $6.96 and GPT-6 Sol scores 61.4 at $1.26, beating Ultramarine by 2.6 and 2.0 points respectively. These are the comparisons the report’s opening paragraph chose to highlight.

Ultramarine 1 Flash — Paper InstrumentsUltramarine 1 Flash is an open model for professional work. It makes the decks, models, and memos that people in finance, law, consulting, and operations put in front of clients, at a fraction of thepaperinstruments.com

There’s more in the chart. Claude Fable 5.1 scores 65.5 at $8.64, and GPT-6 Astra scores 64.2 at $5.64 — both roughly 6 points higher. On the other end, GPT-6 Luna scores 58.3 at $0.11, cheaper than Ultramarine and only 1 point lower. The company doesn’t hide this either: it notes that 27 of the 39 comparison runs were both more expensive and lower-scoring. Flip that around, and 12 were either cheaper or higher-scoring. And every model was run at its highest supported reasoning setting. Fable 5’s $8.19 is its maximum-reasoning figure; at a lower setting, it scores 50.8 for $1.82. The “60x” figure compares against Fable 5’s most expensive setting.

The base-model line is worth another look too. GLM-5.3 Flash scores 52.0 at $0.10 on the same evaluation. So post-training bought a 7.3-point gain for an extra 3 cents per task. Since the API token price is the same as the base model’s, that means Ultramarine used slightly more tokens here. The 6.5% savings the report cites is an aggregate across all six benchmarks.

The test conditions matter too. PI Knowledge Work runs on the company’s own harness, Feather, with grading done by GPT-5.6 Terra. Ultramarine is the model trained on that exact harness. The external benchmarks each used their own official harnesses, and the comparison scores are a mix — some run directly by the company, others pulled from public leaderboards. The report’s evaluation notes specify, benchmark by benchmark, which is which. As of the 16th, when I’m writing this, I haven’t found any independent evaluation from an outside organization.

The figures of $300,000 and five people don’t appear in the technical report at all — they come from the CEO’s thread on X, and details like GPU hours or training duration aren’t in the report either. What the report does confirm is that training used LoRA⁴ at rank 64. Like Harvey’s Tenet, this means the base weights were left untouched and only a thin adapter layer was trained — which is part of why the cost is so low.

How to Score Work That Has No Right Answer

Reinforcement learning works by having a model attempt a task, receive a score, and do more of whatever earned a higher score. Coding turned out to be a natural fit for reinforcement learning because scoring it is easy: a test passes, you get 1; it fails, you get 0. The report itself notes that most agentic post-training research to date has clustered around these “verifiable” tasks — coding chief among them.

Professional work doesn’t come with that kind of scorer. You can’t check a due-diligence memo or a litigation timeline against a regex or a unit test; the quality of the answer hinges on judgment and feel. So Paper Instruments built a separate scoring sheet for every single task. In the report’s own language, that’s a rubric⁵.

The part I kept rereading was the explanation of how to write a single scoring item. The report frames each item as drawing a boundary line across the space of possible answers. An item copied straight from one answer key will fail other, equally valid answers; a vague item like “provides appropriate recommendations” will reward plausible-sounding wrong ones. Somewhere in between, an item has to spell out the substantive condition for earning points — along with whatever facts, standards, and context are needed to verify it.

The example task included in the report makes this concrete. In a fictional deal called “Project Aurora,” the task is to revise the forecast and disclosure sections of board pre-read materials using the new management forecast (v7). Scoring item 13 reads: “Does the response distinguish between what the board can approve now (using v7 as the sole basis for forecasts and valuation, and the qualitative positioning for the roadshow) and what still requires legal counsel’s sign-off (external-facing language and the forecast table for the next disclosure cycle)?” Phrase the same idea as “use the language that v7 is the sole forecast basis and present the approval request first,” and two equally valid answers get dropped. Phrase it as “provide the board with a useful forecast update,” and three incomplete answers slip through. The item also carries penalties: approving the use of v7 figures directly in investor materials without legal counsel’s sign-off costs 50 points; presenting multiples based on the old v5 baseline as if they were current costs 25 points. Even what not to do is written into the rubric.

Each item is judged met or unmet, multiplied by a weight, and summed into a reward. The model does the judging. The company says it uses a published rubric library and a scoring prompt calibrated to match human expert judgments. In Issue 166 I wrote about how what you can verify differs depending on whether you check with code or hand the evaluation to a model — this company has scaled up the second path to training scale. Doing that requires reining in the arbitrariness of model-based scoring through item design, and the boundary-line logic above is the answer.

Where do the tasks come from? The report says it considered three sources: real user task logs, expert-written tasks, and synthetic tasks. User logs were discarded because they carry implicit assumptions. A user who asked about “the margins of the top 10 medical device companies” might have meant top 10 by revenue — but build a task from that log, and a correct answer based on market cap or profit gets penalized. Expert-written tasks avoid ambiguity, but maintaining both breadth and scoring quality by hand is expensive, and narrow tasks still exclude valid answers. So the company went entirely synthetic.

The scale is as follows: four domains — finance, law, consulting, operations — spanning 19 functional areas and 88 environment types. Finance alone accounts for 34 types: M&A data rooms, IPO execution rooms, Chapter 11 debtor’s offices, data-center project-finance desks, and the like. Environments pull in real SEC filings (10-Ks, 8-Ks, attachments like credit agreements) and public web data, while contracts and internal emails are synthesized from seed data. Business systems — CRM, ERP, warehouse management, process control — are also mocked up and wired in via MCP. Combine the axes within each functional area — output format, sources of complexity, stakeholders, conflicting data, deal stage — and each area yields somewhere between 10 million and 10 billion possible combinations. Building a single task out of that space takes dozens of agent loops and multiple rounds of review: checking whether it’s grounded, realistic to actual practice, correctly calibrated in difficulty, and whether the instructions and scoring criteria line up. A separate classifier filters out answer leakage and reward hacking. Finally, tasks too easy for the base model’s performance level get filtered out as well.

The deliverables are real Excel, PowerPoint, and Word files. Because models frequently corrupt files silently while editing documents with libraries like python-docx, the company forked that library into a version that refuses unsafe operations and used it for training. At scoring time, those files are parsed into text and handed to the scoring model.

The System Prompt Was Three Sentences

The harness⁶ feeding into training and evaluation is worth a look too. The company surveyed the tool conventions of coding harnesses like Claude Code, Codex, Cursor, Kimi Code, OpenCode, OpenHands, and Qwen Code, then built a new Python harness on top of the common conventions it found. Training across multiple harnesses simultaneously was outside the budget, so the goal was an “average” harness that would transfer well to whichever harness it ended up running on. The toolset is just read, edit, write, grep, glob, bash plus background tasks, and a to-do list.

The striking choice was stripping out instructions. During training, the system prompt was three sentences, about 50 tokens: work within the given tools and workspace in a Linux sandbox, don’t expose secrets, and put result files in the designated path. By comparison, the default system prompts of the real-world harnesses surveyed ran between 1,800 and 4,400 tokens. Tool descriptions, too, stated only “what it does and what it guarantees” rather than “when and how to use it,” and planning and verification procedures were left out. The reason is straightforward: instruction-heavy prompts raise evaluation scores but constrain the model’s exploration, locking in a particular way of working before training even happens. The production-grade prompt was only added in at the very last stage of training.

The reward function also includes a length penalty. It docks points when generated tokens exceed the allowance or turns run long, and docks points for malformed tool calls too. The 6.5% drop in token usage is a direct result of this penalty. Images weren’t part of training at all — if the model tries to read an image, it just gets back text saying “vision isn’t available in this environment.”

What’s Different From the Harvey Case

Compared side by side with Harvey’s thesis from issue 208, which I covered two months ago, the differences become clear. Harvey ran roughly 150 B300s for two months on top of Kimi K3, using 1,750 environments modeled on the actual work lawyers do and 75,000 grading items written by lawyers — lawyers it hired through Mercor and Snorkel. At the time, I wrote that the model is the consumable and the work environment is the capital asset.

Paper Instruments didn’t build that capital asset by hand. Harvey, too, made a point of saying it didn’t use client data, but it hired lawyers to write the tasks and grading items. Paper Instruments handed even that step over to a model. It excluded expert-written tasks on cost grounds and churned out 88 types of environments through a generation pipeline instead. The two companies’ cost claims aren’t directly comparable, because the benchmarks differ. Harvey said its approach costs less than 1/4 of leading frontier models; Paper Instruments only claimed 1/60 in its own internal evaluation.

There are also similarities. Both trained thin LoRA layers on top of Chinese open-weight models. Both rewarded using fewer tokens. Both evaluated their own models on their own benchmarks. And both left the hardest tasks to frontier models regardless. Even in Ultramarine’s own table, Fable 5.1 and GPT-6 Astra sit above the 6-point line.

Oswarld’s Lens

In my own work, this announcement snags on two points.

One is air-gapped networks. When I’m delivering to a company or designing an AI rollout, I plan on the premise that “the model always gets upgraded.” Even when setting up an air-gapped network for a public institution, I structure the project around the rough capability level of the open-weight model expected six to twelve months out, not the one available today. Ultramarine adds one more variable to that premise. Instead of waiting for the next open-weight release, the question becomes how far the cost of tuning an existing open-weight model up to a specific task has fallen. If it really is $300,000 as the company claims, that’s a figure worth weighing within the budget of a single air-gapped project. But that figure is just the training cost — the hardware needed to run the 320-billion-parameter base model inside the network is separate. As we saw in Issue 178, memory requirements track total parameters, not active parameters⁷.

The other is a division of labor I already use. I’ve long since handed overall planning, orchestration, and initial blueprints to Claude or GPT-6, while routing the long, repetitive work to DeepSeek or GLM. Even burning through trillions of tokens on Ollama Cloud is far cheaper than Claude or GPT-6, and I find the results perfectly usable if you set up harness files like Skill.md or soul.md properly. So an announcement about a specialized task model built on top of GLM isn’t an unfamiliar combination to me. Except that Paper Instruments stripped those harness instructions down to 50 tokens during training. The instructions I layer on when I use the model and the instructions they removed during training are the same thing — just at different points in time. Once the weights are released, what I want to know first is how the results differ with my harness files on versus off.

As I always say, not everyone needs superintelligence for every task. There’s a model suited to the task and the environment. In Issue 208, I wrote that models are consumables while the work environment is capital equipment; this announcement pushes one step further, toward the idea that models can now stamp out even that environment. If so, the asset that remains isn’t the environment itself but the taxonomy used to stamp it out, the skill of using grading criteria as boundaries, and the filters that block the gaming tricks models pick up during training. The report said it would release the weights and training code, but it says nothing about releasing the pipeline that builds the 88 environments. The question for Korea shifts along with it. With Harvey, the question was whether anyone had written Korean legal work into 1,750 environments; now it becomes who will seed DART filings — Korea’s equivalent of SEC filings — instead of SEC filings, and write the document practices that Korean law firms and accounting firms actually follow into the grading criteria.

Closing

Ultramarine demonstrated two things. First, that even open-ended document work without a single correct answer can be converted into reinforcement-learning rewards, using grading rubrics as the boundary. Second, that the company claims this was done by five people for $300,000. The report spells out the method behind the first claim. The second claim can only be confirmed once the weights are released and someone else runs the numbers on a different harness. Until then, the right way to read “it beat Fable 5” is this: on the company’s own benchmark, it beat the most expensive configuration of Fable 5 on four out of six tasks.

---

💬 Reader, if a model scored about the same but cost 1/60th as much, would you switch? If not, what would you need to see confirmed first? Let me know in the comments.

📨 If you have a colleague evaluating open-weight models for work, pass this along.

References & Further Reading

Primary sources

• Paper Instruments, “Ultramarine 1 Flash: Scaling Non-Verifiable RL”, 2026.10.14. ··· This is the primary source for this piece. The rubric design, the 88 environments, the Feather harness, the training setup (LoRA rank 64), the benchmark tables with evaluation notes, and the Project Aurora example all come from here.
• Paper Instruments, “Ultramarine 1 Flash” product page. ··· This is the source for the “Fable-class” and “60x cheaper” language, the API pricing (input $0.15, cached input $0.03, output $0.50, per million tokens), and the cost-versus-score chart.
• Z.ai, “GLM-5.3-Flash” model card, Hugging Face. ··· This is the source for the base model specs: 320 billion total parameters, 18 billion active, MIT license, native multimodality.
• Runtime Wire, “Paper Instruments launches Ultramarine 1 Flash for professional work”, 2026.10. ··· An article summarizing CEO Danish Khaji’s X thread. This is the source for the “five people,” “roughly $300,000,” and “weights released two weeks later” claims.

Past issues worth reading alongside this one

• Harvey, the Legal AI OpenAI Backed, Runs on a Chinese Open-Weight Model (Issue 208) ··· The prequel to today’s piece. The story of how Harvey built a model in two months using 75,000 lawyer-written grading items and 1,750 environments.
• Two Different Checks for Verifying AI Output (Issue 166) ··· An issue on what you can confirm when grading with code versus when you hand evaluation to a model.
• Open Weights You Still Can’t Run Yourself (Issue 178) ··· An issue that walks through, in numbers, why an MoE model’s memory requirements track its total parameter count rather than its active one.

The author is Oswarld (Kwangseob Ahn). Current roles: Adjunct Professor at Sejong University, Strategy Consultant at INLEVEL9. Career, research, books, and recent work are kept current on the About page. Latest · July 2026: HEMA-2: A Consolidation-Aware Tri-Memory Architecture with Multi-Channel Scheduling for Lifelong Conversational AI.

📝 Glossary

Footnotes

1. Open weights: A release model in which a model’s trained weight files are made available for anyone to download, run on their own servers, and modify. GLM-5.3 Flash uses the MIT license, which permits commercial modification and redistribution as well. ↩

2. Reinforcement learning: A training method in which a model performs a task, receives a score (reward) for the outcome, and has its weights adjusted to make higher-scoring behaviors more frequent. It first proved itself on tasks that can be graded automatically, like coding. ↩

3. Post-training: The process of further training an already-built model on domain-specific data and rewards to tailor it for a particular use. This is distinct from pre-training, which builds a model from scratch. ↩

4. LoRA (Low-Rank Adaptation): A method that leaves the original model’s weights untouched and instead attaches a thin auxiliary layer alongside them, training only that layer. The “rank” sets the size of this auxiliary layer. It’s cheap to train, but most of the base model’s characteristics remain intact. ↩

5. Rubric: An item-by-item scoring sheet used to evaluate an output. Each item specifies whether a condition is met and how many points it’s worth; in this model, the rubric’s score becomes the reward for reinforcement learning. ↩

6. Harness: The outer framework that puts a model to actual work. It contains the rules for what tools the model can use, how tool-call results are returned to it, and how context gets trimmed during long tasks. Coding agents like Claude Code or Codex are well-known examples of harnesses. ↩

7. Active parameters: The number of parameters a mixture-of-experts (MoE) model actually uses in computation when processing a single token. GLM-5.3 Flash engages only 18 billion of its 320 billion total parameters per token, but since it’s unknown in advance which experts will be called, the entire model still has to be loaded into memory. ↩