Fine-Tuning, Prompting, or RL? Only One Is Usually Your Job

Someone on the team wants to fine-tune a model for the support-ticket classifier before anyone has actually tried a better prompt on it. That instinct is common, and it is usually wrong. Prompting, supervised fine-tuning, and reinforcement learning are 3 different levers for making a model better at your task, and picking the wrong one first wastes weeks and real budget.

What actually changes when you prompt, fine-tune, or train with RL?#

Prompting changes what you send the model, not the model itself: instructions, examples, and retrieved context steered at inference time. Fine-tuning changes the model's weights using labeled input-output pairs so a behavior becomes the default. Reinforcement learning changes the weights too, but by rewarding good outputs instead of copying labeled ones.

That distinction matters more than the vocabulary suggests. A prompt is something you rewrite in a minute and roll back for free. A fine-tuning job takes real setup, a real dataset, and hours to run, and it produces a new artifact you now own and have to version. An RL run needs a reward signal, meaningful compute, and days of iteration before it changes the model's behavior at all, which is why it sits at the far end of the effort scale.

Prompting has also gotten less manual than it used to be. Frameworks like DSPy treat prompt construction as an optimization problem: you define the task and a metric, and the framework searches for the instructions and examples that score best against it, instead of you hand-tuning wording by feel. That is closer to real engineering than trial and error, and it is a big part of why the gap between "just prompt it" and "we need to fine-tune this" narrowed through 2026.

Every number here is read from the vendor's own documentation or the original paper, not a summary of a summary.

When does prompting alone actually win?#

Prompting wins whenever a frontier model already knows how to do the task and just needs the right instructions, examples, or retrieved context to do it reliably. It is the cheapest option, the fastest to iterate, and the easiest to reverse. Its ceiling is fixed: it cannot teach a model a skill the underlying model never learned.

Work through prompting in a fixed order before concluding it has failed. Start with plain instructions and a clear definition of done. Add 2 to 3 worked examples if the output format keeps drifting. Add a schema or structured-output mode if the model keeps missing fields. Add retrieval or a tool call if the model is guessing at facts it was never given. Only after all 4 of those still fail on real inputs, not a handful of demo cases, does the ceiling become a genuine fine-tuning or RL question.

Matching the task to the right frontier model by default is worth doing before any of this, since a stronger default model alone sometimes closes the gap for free. A weak model with a great prompt still loses to a better model with an average one; check that variable before you touch training data at all.

Most tasks that look like a fine-tuning problem resolve somewhere in the first 3 steps.

When is supervised fine-tuning actually worth the setup?#

Fine-tuning is worth it when a task is narrow and repeated, you have real labeled examples, and the failure is about format, tone, or domain vocabulary rather than missing reasoning ability. It teaches a model to default to a pattern it could already produce occasionally, not a capability it never had. Skip it without real examples already on hand.

The self-serve version of this got harder to reach in 2026. OpenAI is winding down its fine-tuning platform on a phased timeline: no organization that had not previously run fine-tuning could start after 7 May 2026, active users needed inference on a fine-tuned model within the past 60 days to keep creating jobs after 2 July 2026, and every customer loses the ability to start a new job at all on 6 January 2027. Source OpenAI has not published a stated reason beyond the notice itself, but the timing follows roughly 2 years of frontier models getting steadily better at following complex instructions and formats without any weight changes.

Google moved earlier and further. The public Gemini API and AI Studio have had no fine-tunable model since Gemini 1.5 Flash-001 was deprecated back in May 2025, though supervised fine-tuning for current Gemini 2.5 models still runs on Vertex AI, Google's enterprise cloud platform. Source Anthropic never opened general fine-tuning on its own public API; the paths that exist are Amazon Bedrock, which supports an older Claude 3 Haiku, or a custom enterprise engagement. Model distillation, covered in depth separately, is the growing exception: training a small model on a larger model's own outputs is a form of supervised fine-tuning every major lab is still actively shipping.

If you still want to fine-tune an open-weight model yourself, that lane stayed open. Providers like Together AI and Fireworks AI sell LoRA fine-tuning by parameter tier and let you keep the resulting weights, which is a real option once a hosted vendor's own platform has narrowed.

Every pro here is a real mechanism, every con is a real cost, and the 2026 wind-downs raise the bar on the con side.

How do the 3 techniques actually compare side by side?#

Prompting costs nothing beyond API calls and reverses in a minute. Fine-tuning needs real labeled data and a training run before it does anything, and reversing it means a new job. Reinforcement learning needs a reward signal, meaningful compute, and days of iteration, and it is the only one of the 3 that can teach a genuinely new skill.

Use this as the quick reference; each row below gets its own reasoning in the sections around it.

What does reinforcement learning actually add, and why is it rarely your job?#

Reinforcement learning updates a model's weights by scoring outputs against a reward signal instead of copying labeled answers, which is how it teaches skills no single example could demonstrate. RLHF rewards human preference, RLAIF swaps in AI feedback, and RLVR rewards a verifiable outcome like a passing test. Frontier labs already ran this on the model you call.

RLHF is the classic version: a reward model trained on human preference comparisons, then the policy optimized against that reward. RLAIF replaces the human comparisons with AI-generated ones; Anthropic's Constitutional AI is the standard example, using a set of written principles instead of a person to judge each output. RLVR skips the learned reward model entirely and checks the answer directly: a unit test passes, a math answer matches, a build compiles. Group Relative Policy Optimization, GRPO, is the algorithm behind most of the reasoning-model gains since 2024. Introduced in DeepSeek's DeepSeekMath paper, it drops the separate value model that older methods like PPO need, instead sampling several answers to the same prompt and rewarding the ones that rank higher within that group. Source That is cheaper than PPO-style training, but cheaper is relative: it still means sampling many completions per prompt, scoring every one, and running that loop for days.

The closest thing to an RL product a small team can actually buy is OpenAI's Reinforcement Fine-Tuning, and even that shows how narrow the lane is. RFT works only on reasoning models, currently limited to o4-mini, and bills at $100 per hour of core training time rather than per token. Source It also needs a reward a program can check automatically: a passing test, a matching number, a compiling build. Most day-to-day tasks, routing a ticket, drafting a reply, extracting a field, have no such checker, which is exactly why RL for those tasks stays inside the labs that train the base model, not on your invoice.

Reward hacking is the other reason RL stays rare outside the labs. A model under RL pressure finds the shortest path to a high reward score, and that path does not always match the behavior you actually wanted. A coding agent rewarded for passing tests can learn to delete the test instead of fixing the bug. Catching that requires a tighter feedback loop and more careful reward design than most small teams have the time or the eval infrastructure to build.

Fine-tuning reshapes what a model already does. Reinforcement learning is the one lever that can teach it something new.

How do you actually decide, in order?#

Work the ladder in a fixed sequence rather than guessing: check whether a genuinely good prompt still fails, check whether the task is narrow with real examples on hand, then check whether success can be verified automatically. Each answer routes you to a different lever, and most tasks resolve in the first 2 questions.

3 checks decide the lever: whether a good prompt still fails, whether real examples exist, and whether success can be verified automatically.

The flowchart above is the whole framework compressed into 3 checks. If a well-built prompt, examples, structured output, and retrieval all still fail, you are past the prompting stage for real. If the task is narrow and you have real labeled examples, fine-tuning is the default next step, not RL. Reinforcement learning only earns consideration when success has a cheap, automatic way to verify it, and even then it is worth pricing against just calling a stronger frontier model first.

That ordering is deliberate. Prompting is reversible in a minute, fine-tuning is reversible with a new training job, and RL is the slowest and priciest to unwind, so the ladder checks the cheap, fast option before it ever pays for the expensive one.

What should you set up this week?#

Pick one task you are already tempted to fine-tune and run it through the ladder this week: rewrite the prompt with examples and structured output, count how many real labeled cases you actually have, and check whether success can be verified automatically. That one exercise decides more than another article about the theory ever will.

Cost is the variable most teams underweight in this decision. Right-sizing the model is lever 1 of 7 for cutting an AI bill, and it usually resolves before fine-tuning or RL would even get funded: a cheaper model with a better prompt often beats a training project that takes a month to ship. Start there, exhaust prompting properly, and only escalate the lever once the cheaper one has genuinely run out of room.