The skill, the SDK, and the console

The one-line version

npx skills add singhabhishekkk/apprentice-skill

pip install runapprentice

runapprentice.com

Said another way: the skill supplies judgment, the SDK supplies execution, and the console supplies a second pair of eyes. Teams that get stuck almost always picked a surface that cannot supply the thing they were actually short of.

Three questions, in order

What each one can and cannot do

The gaps are on purpose. Each surface is missing the powers it has no business holding.

One line deserves emphasis. Only the console can turn a row gold, because a gold row means a human confirmed that exact output, and neither a markdown file nor a CLI flag is a human. The one exception is local CLI mode, where a CSV you pass to --data is treated as gold on the grounds that you asserted it.

The skill, in detail

It is one markdown file, MIT licensed, and you can read the whole thing in less time than this section takes. It installs into Claude Code, Codex, and Copilot CLI. What it does is narrow on purpose: it watches for code, not conversation, that sends the same shape of request to an expensive model over and over. A batch job classifying rows, a ticket router, a moderation pass, an endpoint serving one prompt shape to many users. When it sees one it says so once, with benchmark numbers it can source, and stops.

The list of things it will not do is the point of it:

• It never calls the Apprentice API. It holds no key and needs no account.
• It never installs anything and never edits your code.
• It will not flag a one-off prompt, a chat feature, or creative writing, because noise would make you stop trusting it.
• It will not quote a number it cannot source, and it is told to re-check the public benchmark rather than recite figures from memory.

After you opt in it becomes useful in a second way: it knows the two things that bite on a first integration, namely how to wire capture and feedback to a real signal rather than a guessed one, and that the optimized prompt is instruction text rather than a template, so calling .format() on it raises and .replace() silently drops your input.

Why it is a skill and not an MCP server

This question comes up every time, and the honest answer is that the two solve different problems. MCP is the right shape when an agent needs live, authenticated access to a system: it carries credentials, holds state, and hands the agent tools to call. A skill carries procedural knowledge instead, costs no context until something triggers it, and can be read end to end before you install it.

Our case needs judgment, not access. The skill's entire job is to recognize a pattern and say one sentence about it. Wrapping that in a server would add an authenticated process, a running dependency, and a path for an agent to start paid work without you in the loop. We would rather the agent tell you and stop. If you later want programmatic access, that already exists and it is called the SDK, where the calls are in your code and you can read them.

The SDK and CLI, in detail

pip install runapprentice. Two jobs, and they are separable.

Capture. capture() records the input and output of a production call and returns a trace id; feedback() attaches whether it worked. Both fail open, so a problem on our side returns nothing rather than breaking your request path. There is a LangChain callback and support for raw OpenAI clients on both Chat Completions and Responses. Log the trace id next to your own request id: it is the thread back from a row in the console to the request in your system.

The loop. The CLI covers login, status, optimize, train, jobs, job, and cancel. Two of those run entirely on your machine with no account at all:

apprentice optimize extract --local --data golden.csv
apprentice train extract --local --data golden.csv --effort high

The first runs GEPA prompt optimization using your own OpenAI key. The second fine-tunes a small open model on an Apple silicon Mac through mlx-lm, with no cloud compute involved. This is the fastest path to a real number, and it is the one to reach for when somebody wants evidence before they want an account.

What the SDK cannot do is close the loop on data quality. It has no review queue, so nothing it runs turns a row gold, and there is nowhere in it to ask a colleague for a second opinion. That is not an oversight. It is the same reason we did not build the skill as a server.

The console, in detail

Every task in the console has nine pages, and each one exists because something in the loop needs a place where a person can look at it: Overview, Dataset, Review, Runs, Compare, Models, Drift, Connect, and Collaborators.

Three of those you cannot get anywhere else.

• Review is where a row becomes gold. A verdict, a correction, a flag from a fixed taxonomy. This is the only place in the whole product where the tier of a row moves up by human decision.
• Collaborators lets you invite the person who actually knows the task, a support lead or an analyst, to review rows without giving them anything else. On a real team this is usually the constraint, not the compute.
• Drift charts your captured traffic and the feedback your application sends, day by day, so a slide in quality arrives as a falling line instead of an anecdote from support.

The console is also the answer when you have nothing at all. It can generate a starter dataset from your task description and prompt, and those rows land in the raw tier like everything else: generated data never counts until a human confirms it. More on that line in the golden dataset guide.

One thing to state plainly. Promotion is currently recorded rather than routed. When a model passes its eval gate the console records the promotion, and the gateway that moves live traffic to it is being built. Every surface says so until it ships.

How they fit together in practice

The common path runs skill, then CLI, then console, but plenty of teams skip a step and nothing breaks. Once you are signed in they share one backend, so a hosted run you start on the CLI shows up under Runs, and rows a reviewer verifies in the console are the rows your next optimize scores against.

The keyless local paths are the exception, and it is worth being exact about it. --local runs entirely on your machine: nothing is uploaded, and unless an API key is configured the run is never registered, so it will not appear in the console or in apprentice jobs. That is the tradeoff for needing no account.

The row floors differ the same way. Hosted optimization needs at least 20 verified rows and hosted training needs at least 500 human-verified gold rows; those are the gates the API enforces, and the second is high on purpose because it is the point where something else starts doing the work. Local mode is far looser: optimize --local needs only two rows, enough for one train and one held-out score, and train --local just needs enough rows left after the holdout split to fill a batch. Two rows will produce a number, not a trustworthy one, so treat the hosted floors as the honest guidance and the local ones as permission to experiment.

Related: what a golden dataset is, how prompt optimization is scored, and why prompts get worse over time. Or see every guide.

Common questions