Most people know language models from chat, where you type and the model writes back. With a decision model you send some text and a question with a fixed list of allowed answers, such as "is this support ticket urgent?" or "which team should handle it?", and get back a probability for each answer and no text at all.
We have just published smaller versions of five open decision models, plus two of Google's small Gemma 4 chat models, as Artemis Edge on Hugging Face. This post explains what decision models are and why shrinking one needs a different test from shrinking a chat model. Each model has its own article with the story behind it and the full results.
Real runs, recorded live on a desktop CPU (AMD Ryzen 9 9950X): each scene sends the same request to the original and our file. Kev-4B plays a short catch-the-star game with the 2.5 GB file and routes support tickets with our smallest file (2.0 GB), Kev-9B picks an agent's next tool with the 4.8 GB file, and Clef-Flash reviews invoice images with the 4.9 GB file. The game, tickets, traces and invoices are synthetic.
A question with allowed answers
A request to a decision model has two parts. The state is the thing to decide about: a customer message, an invoice, an agent's log, sometimes an image. The questions each have a type and a set of allowed answers. The common types are yes or no, pick one option from a list, and a score on an ordered scale.
The shape of a request and its answers. The probabilities here are illustrative.
The model reads the state once and scores every allowed answer in a single pass. Its only possible outputs are the options in the request, so there is nothing to parse and it cannot answer outside the list. The probability attached to each answer is what makes it useful: a program can act on a confident answer straight away and send an uncertain one to a person.
Where they are useful
They fit any step where software has to choose from a known set of options. A support queue can use one to route tickets and flag the urgent ones, a trust and safety team to triage reports, a finance team to check invoices against rules. Agent builders use them to ask "should the agent call a tool here, and which one?" before spending a full chat-model call on it. Cloudflare's own example is classifying websites by category for its threat intelligence team (Cloudflare blog).
These jobs used to need a classifier trained for each label set. A decision model takes the labels as part of the request, so adding a category means editing the request, with no new data or retraining. Because it scores the options in one pass instead of writing an answer token by token, each decision is cheap to compute.
A short history, mostly from the last month
TypeSafe AI introduced the idea as "System One models" with its hosted model Jev on 15 September 2026, along with the /v1/systemone request format that the open models now share (TypeSafe blog). Open models followed quickly. On 1 October, Cloudflare released Clef and Clef-Flash under Apache 2.0, Perplexity released pplx-decider, and Jared Palmer released Kev 1.0, a family of open decision models anyone can train. A day later llama.cpp added a /v1/systemone endpoint, so these models run locally with the same server people already use for chat models (Hugging Face blog).
curl http://localhost:8080/v1/systemone -H 'Content-Type: application/json' -d '{
"state": "My Stripe integration keeps failing. Please help ASAP.",
"questions": {"urgency": {"type": "noul", "instructions": "Does this message express urgency?"}}
}'
noul is the format's name for a yes or no question.
Why compressing them is a different problem
The originals are large: Clef and Perplexity's Decider are each about 54 GB at full precision. People shrink models like these by storing the weights at lower precision, and a 4-bit file is usually the first thing anyone downloads.
When compression damages a chat model, the damage usually shows in the text and someone notices. A decision model's output is one option and a number. A compressed file that picks "billing" where the original picked "technical" looks just as plausible as one that agrees, and nothing in the answer says it changed.
The confidence matters too. Teams set thresholds on it, for example acting automatically above 90% and asking a person below that. A file that keeps the same top answer but shifts the probabilities changes how many cases reach a person and how many mistakes get through, without any visible sign.
So we test a compressed decision model on how often it gives the same answer as the original across many real decisions, and on whether it ever overturns an answer the original was sure of. Benchmark accuracy alone can hide changes, because a file can get different questions wrong and still score the same.
How we checked every file
Every decision-model file was compared with its full-precision original, and with the most common public files where they exist, at identical llama.cpp settings:
- 2,000 decisions from the test split of the public typed-decisions dataset (400 cases with about five questions each),
- 1,000 further decisions that played no part in making the files (for Kev-9B, the default file was chosen partly on them, so there they are a validation set),
- a 300-item sample of the public Decision Index benchmarks,
- 330 image questions for the models that read images,
- shuffled answer options and repeat runs.
The headline number in every article is agreement: the share of the 2,000 test decisions on which a file gives the same answer as the original. It is not accuracy. We also count how many answers the original was more than 80% sure of that a file changed. The tests ran on an NVIDIA A100, an RTX 5090, a GB10 (DGX Spark), an RTX 3060 and a desktop CPU; each article says which.
The Gemma models are chat models, so they were checked differently: how often each file predicts the same next token as the original, how closely its first 32 generated tokens match, and a 310-item task sample.
The opening support ticket is an illustration and its probabilities are made up. The collection after it shows each model's original size against our default file, with agreement taken from the model cards. Clef-Flash is shown at its A100 figure; on CPU it is 95.0%.
Why we did this
Artemis, our code optimisation platform, runs on LLMs and decision models. Compressing them suits it well, because the work means trying many candidate builds against a measurable objective, with a benchmark the search cannot change. We gave Artemis Discovery the job of finding files that are much smaller and still decide like the original, with agreement measured on decisions instead of generic text. The files on Hugging Face were compressed by TurinTech with techniques discovered by Artemis. The method stays private; the results and the test setup are all on the model cards.
The results are mixed in places, and the cards say so. Against the usual public 4-bit files (Q4_K_M), ours are 16 to 25% smaller and agree with the original 0.2 to 2.3 points less often. At about the same size, ours keep more of the original's decisions: the public IQ3_M files of Clef and Clef-Flash agree 87.6% and 89.5% of the time, against 95.6% and 95.0% for ours. For Perplexity's Decider, ours are the only public files that run on stock llama.cpp (checked on 13 October 2026), and they are smaller and closer to the original than a plain 4-bit file made from it. Each article has the full comparison.
These are independent conversions. They are not affiliated with or endorsed by Cloudflare, Perplexity, Google or the Kev authors, and the credit for the models belongs to them.
The models
Decision models (collection):
| Model | Original | Our default file | Same answer as the original | Article |
|---|---|---|---|---|
| Clef 27B (Cloudflare) | 54.1 GB | 14.8 GB | 95.6% (A100) | Clef 27B |
| Perplexity Decider 27B | 53.8 GB | 15.3 GB | 96.8% (A100) | Perplexity Decider |
| Clef-Flash 9B (Cloudflare) | 18.2 GB | 4.9 GB | 94.1% (A100), 95.0% (CPU) | Clef-Flash 9B |
| Kev-9B | 17.9 GB | 4.8 GB | 92.2% (A100) | Kev-9B and Kev-4B |
| Kev-4B | 8.4 GB | 2.5 GB | 93.0% (RTX 5090) | Kev-9B and Kev-4B |
Agreement is on 2,000 test decisions per model, measured on the hardware in brackets. On an NVIDIA A100 the default Clef and Decider files need 15.0 and 16.6 GiB of GPU memory, against 49.9 and 50.8 GiB for the originals, so about a third.
Compressed LLMs (collection): the first-choice file for Gemma 4 E4B is 4.2 GB, down from 14.9 GB, and for Gemma 4 E2B 2.3 GB, down from 9.3 GB. Each repository also has smaller files, down to 3.6 GB and 2.1 GB. Both models are covered in one article.
Everything runs on stock llama.cpp. The most useful check is still on your own data: run the original and a smaller file on a few hundred of your real decisions and count how often they agree. If you want a version fine-tuned on your own decisions, built for your hardware, or compressed from a private model, book a call.
