telltail · engineering notes

From a paid detector to your CPU.

We got within 95% of a commercial detector with a model we trained ourselves, matched or beat the best cloud LLMs, cut serving cost by more than 100×, and then moved it onto your device — so it’s private by construction.

This is the path we took, what each step cost, and what we’d do again.

Measured, not hyped. Every number below comes from our own eval harnesses; where a number is rough, we say so.

rung 1 · the reference

Start with someone else’s answer key.

Before training anything, we needed ground truth for “reads AI-written” on real browsing prose. A commercial detector gave us that: per-text verdicts we could tune against and, later, a scoreboard to beat. It was also the most expensive way to answer the question — every check a paid API call, every post’s text leaving the machine. A reference, not a product.

Lesson one, which never stopped applying: disputed texts — where judges disagree — are not ground truth either way. We learned to tune only on texts where several independent judges agree, and to treat the rest as informative disagreement.

rung 2 · the generalist

Ask a big model twenty-two questions.

No single prompt separates machine prose from human prose. So we asked a general LLM — TypeSafe’s Jev — a battery of small questions about each text (assistant register, burstiness, first-hand specifics, editorial oddity…) and fitted a scoring rule over the answers. The honest work was all in the questions: the best rule got close to the commercial reference on texts where the judges agreed, and every plateau after that yielded only to new evidence, never to new weights.

The price was throughput and money: roughly 1,500 billed input tokens per check, on the order of $60 per million checks — about $6 a month for one heavy scroller, and hundreds a day at real scale. Plus the thing that mattered most: every check sent the reader’s prose to someone else’s server.

rung 3 · our own model

Distill the judges into one head.

A generalist answering 22 questions per post is theatrical overkill, so we fine-tuned small encoders to answer one baked authorship question directly. The teachers were the same LLM judges from rung 2 — soft labels, averaged — plus certain labels from texts whose authorship we knew for sure (pre-2021 writing is human; assistant replies are machine). The surprise was where the model knew it: mean-pooled middle layers beat the final layer, and the cheapest head — plain logistic regression — beat the MLPs.

Four consumer GPUs served on the order of ten million checks a day for dollars in power. But the prose still travelled to our machines, and four GPUs don’t scale to every reader for free.

rung 4 · your device

A 34 MB student that runs on your CPU.

The same distillation recipe, shrunk once more: a MiniLM-L12 encoder trained against the judges’ soft labels, exported to ONNX, quantized to per-channel int8 — about 34 MB with its vocabulary. It ships inside a browser extension and scores posts with onnxruntime WebAssembly, on the order of ninety milliseconds per check on an ordinary laptop, for $0 per check. The student is graded on the exact rows the big models were measured on, and a release ships only when it holds its false-alarm rate on fresh held-out data. Thresholds are set per site, because a false alarm on LinkedIn and one on X are not the same mistake.

This is the rung where privacy stops being a promise and becomes architecture: there is no server in the loop, so there is nothing to leak, log, or subpoena.

the cost ladder

Each step cut cost and added control.

StageWhat answersOrder of cost
Commercial detectorTheir API, their verdictreference price per call
General LLM, ~22 questions~1,500 billed tokens per check~$60 per million checks
Self-hosted fine-tune (4 GPUs)One baked question, our hardwarea few dollars a day per ~10M checks
On-device student (34 MB int8)Your CPU, ~90 ms per check$0 per check

Token prices and throughput are our measured orders of magnitude, 2026. The point is the shape — four orders down the ladder — not the third decimal. Accuracy is stated relative to the reference (within 95% of Pangram on held-out sets) rather than as one headline number, because the goal was never to label every post: it was to label the ones we can with confidence.

always improving

Signed updates, memory manners.

  1. 01  /  verify

    Every model file is signed

    The extension fetches a version catalog about once a day and checks an Ed25519 signature plus per-file hashes before anything runs — keeping the previous model as fallback. The requests carry no identifiers.

  2. 02  /  contain

    One runtime, then it’s gone

    Inference lives in exactly one worker (Firefox) or offscreen document (Chrome) with one session. After five idle minutes the runtime is terminated — not released, terminated — and rebuilt lazily on the next check.

  3. 03  /  bound

    Backpressure, not backlogs

    Inference is strictly sequential through a capped queue; overflow is marked “not checked” instead of piling up. No check may ever hurt the reader’s machine — that was the requirement, stated first.

telltail

Take it browsing.

Free forever. No account, no key, no signup.