Same answers, smaller bill, proven by evals.
No company asks a senior manager to classify email all day. They train an apprentice. Apprentice runs that loop: your verified examples in, a fine-tuned small model out. It takes over only when it passes your eval gate.
Free first run · one callback to install · no traffic moves until your evals pass
The 4B you own beat GPT-5.4-mini, even with its prompt optimized.
The manager works.
Your frontier model keeps answering. One callback turns every real call into training data.
The apprentice learns.
Your team verifies rows into a gold set. Generated rows never promote themselves.
The gate decides.
Only a model that beats your held-out bar takes traffic. If it slips, the manager steps back in.
Proven on a public run. Rerun it yourself.
Before: every request at frontier prices. After: a small model you own, scoring higher on your own eval set.
Two public runs, the largest on 200 real scanned receipts with a 60-row held-out slice the optimizer never saw: honest, and yours to rerun. In a pilot we run the same loop on your data.
Receipts: full write-up · rerun every number · the MI300X-trained adapter · runapprentice on PyPI
Cheaper to serve.
One line to connect.
Add one callback to your existing call and Apprentice captures every request, your code unchanged. Eval-gated routing, promotion, and rollback are coming soon, through the same one line.
Install: uv add 'runapprentice[langchain]' · Works with the LangChain chat models you already use; the direct OpenAI SDK is supported too.
OpenAI SDK
LangChainWatch. Learn. Take over.
Capture real calls. Prove quality on gold rows. Traffic shifts only after the gate passes.
Even the frontier labs document this: OpenAI Model Distillation and Google Research show a smaller model can match a frontier model on a task at lower cost.
No dataset? Generate a starter set
A model drafts realistic examples from the rows and instructions you give it. Every generated row lands as raw and earns gold only in review, so the score is never inflated by generated data.
Share the review with your experts
Verifying does not have to be your job. Invite a subject-matter expert by email, choose whether they can only verify or also write corrections, and whether they see your production prompt. Their verdicts build the gold set.
Verify rows into a gold set
One keystroke per verdict. The rows you verify become the examples this task learns from, and the held-out slice it is judged on.
Optimize the prompt, prove it
GEPA rewrites your prompt and scores both versions on held-out gold rows: 72.92 to 79.58 on the receipts benchmark. You see the exact diff that earned it. How the optimization step works.
Train, gate, take over
The fine-tuned 4B beats the optimized teacher, 89.17 vs 79.58 on the same held-out rows. Only a model that passes your gate can take over, and rollback is one click.
Same fine-tune. Three places to run it.
All three speak the OpenAI API: point OPENAI_BASE_URL at the new endpoint and keep your LangChain code unchanged. Rollback is one env var.
Your Mac
One command, apprentice train --local, fine-tunes on device with MLX and scores a held-out slice. Your data never leaves the laptop.
Your cluster
The agent skill writes the vLLM Deployment and Service into your own repo, mirroring your manifests. Traffic never leaves your Kubernetes cluster.
How the cluster path works →Managed coming soon
We run the GPU behind api.runapprentice.com. You point at the endpoint.
Built for the teams that can't afford to be wrong.
Every traffic shift requires your gates to pass. No exceptions. All three guarantees ship with model replacement coming soon
Eval-gated rollout.
Nothing reaches production unless it passes your gold set. You set the thresholds. We fail closed.
Instant rollback.
One click. Traffic flips back to the frontier model in under a second. Every rollback is permanent audit log with a metrics snapshot.
Your data, your cloud.
Fine-tuning runs inside your VPC. Model weights never leave your infrastructure. We see traffic shape, not content.
Before you ask.
The things engineering leaders check before they trust a takeover.
Will a smaller model actually hold quality?
It only goes live if it passes your eval gate. Every replacement is scored against your own gold set, and below the bar your frontier model keeps serving. You see the held-out number before anything ships.
What happens to our data?
By default, captured content is sent to the Apprentice API. Redact it client-side before it leaves your process: the redact hook runs in your code, so sensitive fields never reach us. We never train shared models on your data. With model replacement, coming soon, fine-tuning runs in your own VPC and model weights never leave your infrastructure.
Are we locked in?
No. Capture is one callback you can remove any time, and the optimized prompt is plain text you keep. With model replacement, coming soon, your frontier model stays on hot standby with one-click rollback, and the tuned model is an open-weights Qwen derivative you keep and run anywhere.
How is this different from fine-tuning it ourselves?
You bring the golden set; Apprentice runs the loop. Today that is prompt optimization with an honest held-out score. Coming soon, the same loop trains a small model, gates it on your evals, and routes traffic with instant rollback. The takeover stays measured, not a leap of faith.
Which tasks does this work for?
Repeatable tasks with checkable outputs. JSON extraction and classification are the first classes: the loop needs answers your team can verify as right or wrong. Open-ended chat is a poor fit today.
How fast do we see a result?
Same day. Capture or upload one dataset, verify rows, run optimize, and you see the held-out score next to your baseline. No number is shown that a run did not produce.
What does a pilot involve?
Trying Apprentice is free: upload one dataset and see your score change the same day. The migration itself runs as a design partnership with a small number of teams: one workflow, a written success metric, and a hard end date.
Everything you need to run it yourself.
Works where your agent works
One skill, three agents: a Claude Code plugin, a Codex plugin, and a bare skill folder that GitHub Copilot CLI picks up. It never calls an API and never touches your code.
/plugin marketplace add singhabhishekkk/apprentice-skill/plugin install apprentice@apprenticeStart your first task.
See your score move today.
Teams spending $20k+/month on frontier APIs are exactly who this is for. Trying it is free. The migration runs as a design partnership: one workflow, a written success metric, a hard end date.
Sources: Menlo Ventures, 2025 State of Generative AI in the Enterprise and a16z, 100 Enterprise CIOs 2025
Migrating from OpenAI fine-tuning? Read the migration guide →