Distillation, quietly
A tailored model does not have to be a large one. Once a private model meets your bar, distillation can pour that behaviour into smaller, faster weights that cost less to run and are easier to own.
Idris Fen, Co-founder & CTO · May 28, 2026 · 5 min read
There is a habit of reaching for the largest model available and stopping there. It is understandable. Larger models are more forgiving. But once a model has been fitted to your data and has cleared your eval, size becomes a cost rather than a comfort, and there is a quieter option.
Teaching a smaller model the same behaviour
Distillation trains a smaller student model to reproduce the behaviour of a larger teacher. When the teacher is already a private model tailored to you and scoring well on your held-out eval, the student inherits that fit. You keep the voice and the standard, and you shed the weight you do not need.
Why it matters for an owned model
- Lower cost to serve. Smaller weights mean cheaper inference, which matters most for the models you run all day.
- Easier to own. A model that fits on your own hardware is a model you can genuinely keep behind your own walls.
- Faster to answer. Latency is part of the voice. A quick, correct reply reads as competence.
We only distil once a model has cleared the bar, and we re-run the held-out eval on the student before anything is deployed. The delta over the base model has to survive the shrink, or the shrink does not ship.
The best owned model is the smallest one that still clears your bar.Idris Fen
Made to measure does not mean oversized. It means fitted. Distillation is how a tailored model stays fitted while getting lighter to wear.
Written by Idris Fen, Co-founder & CTO
← All postsA model made to measure.
Start free and tailor a sample in a minute, or book a fitting and we will build you a private model of your own.