Trade-off questions — RAG vs fine-tuning vs prompting

The decision tree

The right answer is almost always: try in this order — prompt → RAG → fine-tune. Each step is a leap in cost and complexity. Justify the leap before taking it.

Prompting only: the task is general enough that the model already knows how to do it; you just need to specify behavior. Cheapest, fastest, easiest to iterate. Try this first, always.

RAG: the task requires up-to-date or domain-specific knowledge the model doesn't have. Knowledge changes; behavior doesn't. RAG is the right tool when the gap is content, not skill.

Fine-tuning: the task requires specific behavior the model can't be coaxed into via prompting — narrow output style, very specific reasoning pattern, domain language unfamiliar to the base model. Or: latency/cost optimization (a small fine-tuned model serves 80% of traffic at 1/30th the cost).