An apprentice watches the expert, then earns the work

The mismatch

There is one job in your stack that runs all day. Reading an invoice into fields. Sorting a complaint. One kind of input, one kind of output, and an answer that is either right or wrong.

Now look at what you are paying for it. In July 2026 OpenAI reported that during a security test, with safety limits lowered, two of its research models found a way out of the test environment and into another company's systems, where the answers to their own exam were kept. That is a lab result, not a normal day. But it is the kind of capability being priced into an API call. Yours fills in a form and writes the same reply, all day.

Nobody chose that. The big model worked first, and there is never an obvious day to switch. But every company knows what to do with work that repeats: you have your most expensive person train someone, watch a while, then step back to the hard cases. Apprentice does that with models.

The shape of it

The part nobody plans for

Watch a company automate one of these jobs. The subject-matter expert knows every odd case in it. Another team knows how to build with models. Neither can finish alone, so the knowledge moves between them in meetings. Someone tunes a prompt until it handles the cases raised in the room, then it fails on the ones nobody mentioned. Weeks go into translation.

Apprentice removes the translation. The expert is shown real answers and says which are right, in a screen built for someone who has never written a prompt. Each fix becomes a new example, so the prompt is tuned against their decisions, not a summary of a meeting. What that set has to contain is its own subject: the golden dataset guide covers which rows count as verified and which only look like it.

Phase 1. A better prompt

Someone wrote your prompt by hand and shipped the version that seemed to work. Nobody measured it, because there was nothing to measure it against. Your examples are that missing thing: Apprentice rewrites the instructions, scores each version, and keeps the best, holding some back so the winner is judged on work it never saw. Hosted optimization starts at 20 verified examples, and returns plain text you paste into your own code, so the first thing we ask of you locks you into nothing. The scoring is described in full in how prompt optimization is scored.

Phase 2. A model of your own

Hosted training starts at 500 expert-checked examples. We train a small open model on this one job until it does it as well as the expensive one did. Only this job: the open-ended work stays where it is, and what moves is the part that repeats, where your volume and your bill sit. The result is not a weaker assistant. It is a specialist.

Then it earns the traffic: a copy of real requests with the answers thrown away, then five percent of live traffic, then more. The big model stays behind it, and turning it off is a setting, not a release.

One thing to state plainly. Promotion is currently recorded rather than routed. When a model passes its eval gate the console records the promotion, and the gateway that moves live traffic to it is being built. Every surface says so until it ships.

What we have measured

Three jobs on public datasets, scored out of 100 on examples held back from training. Every score is a run you can repeat: the data, the commands and the notebooks are at github.com/singhabhishekkk/apprentice-benchmark.

Classifying documents is the row to read twice: our first model scored 80.0, below the optimized prompt, and a second reached 86.7 on the same test. That is what the gate is for. A candidate that does not beat the prompt does not take the work, and one internal control run moved nothing at all: 84.44 before, 84.44 after.

Now the price. GPT-5.4-mini lists at $0.75 and $4.50 per million tokens in and out. The 4B open model serves at $0.03 and $0.15 on DeepInfra. The input ratio is 25x and the output ratio is 30x, so whatever the mix, the saving lands between 96.0 and 96.7 percent: 96% less is a floor, not a best case, for the job that scored higher. List prices, so check the sum.

Every report also says what a fixed answer would have scored, because 70 out of 100 means nothing if always saying "no" scores 70. A request that falls back to the big model is billed at the old price and counted as a swap that did not work.

Trying it

Capture is one line: a callback on the LangChain model you already have, or a wrapper around your OpenAI client. More frameworks are on the way. We observe, we do not choose the model, and if we are slow your calls carry on. Training runs on your own Mac through mlx-lm, so there is no compute bill and no data leaves it, though such a model is self-reported and must pass our evaluation to take traffic. Which surface to open first is its own question: the skill, the SDK, and the console.

Why the loop, and why now

A one-time swap decays: prices drop, a better model arrives, the saving leaks away. The growing set of checked examples is the defence. Each correction improves the next run, which has to pass the same test to keep its traffic.

There is also a date. OpenAI stops accepting new fine-tuning jobs on 6 January 2027. Trained models keep answering until the base retires, so nothing breaks that morning; what ends is retraining. The paths out are laid out in migrating off OpenAI fine-tuning.

Related: what a golden dataset is, why prompts get worse over time, and the receipt benchmark in full. Or see every guide.

Common questions