Fine-Tuning LLMs
When and how to fine-tune vs prompt engineer
Skills taught: llms.fine-tuning
Fine-Tuning vs Prompt Engineering
Fine-tuning trains the model's weights on your data, permanently specialising its behaviour. It is expensive and usually unnecessary — good prompting covers 90% of use cases. But when consistency and style precision are critical at scale, fine-tuning unlocks the remaining 10%.
Fine-Tuning vs Prompt Engineering
- Prompt Engineering: No training — instructions and examples in every request
- Fine-Tuning: New model checkpoint trained on your labeled examples
OpenAI Fine-Tuning Workflow
from openai import OpenAI
import json
client = OpenAI()
# Step 1: Prepare training data (JSONL format)
training_examples = [
{
"messages": [
{"role": "system", "content": "You write product descriptions in our brand voice: confident, friendly, and focused on benefits."},
{"role": "user", "content": "Product: Wireless noise-cancelling headphones with 30h battery"},
{"role": "assistant", "content": "Drown out the world. These headphones wrap you in silence so you can focus, create, or just breathe — for up to 30 hours without reaching for a charger."}
]
},
# ... 100+ more examples
]
# Write to JSONL file
with open("training_data.jsonl", "w") as f:
for ex in training_examples:
f.write(json.dumps(ex) + "\n")
# Step 2: Upload training file
upload = client.files.create(
file=open("training_data.jsonl", "rb"),
purpose="fine-tune"
)
print(f"File ID: {upload.id}")
# Step 3: Create fine-tuning job
job = client.fine_tuning.jobs.create(
training_file=upload.id,
model="gpt-4o-mini", # cheap base model
hyperparameters={"n_epochs": 3}
)
print(f"Job ID: {job.id}")
# Step 4: Check status (poll or use webhooks)
status = client.fine_tuning.jobs.retrieve(job.id)
print(f"Status: {status.status}") # validating_files → running → succeeded
from openai import OpenAI
client = OpenAI()
# Step 5: Use your fine-tuned model (no system prompt needed!)
FINE_TUNED_MODEL = "ft:gpt-4o-mini:your-org:product-copy:AbCd1234"
response = client.chat.completions.create(
model=FINE_TUNED_MODEL,
messages=[{
"role": "user",
"content": "Product: Reusable bamboo coffee cups (4-pack)"
}]
)
print(response.choices[0].message.content)
# Output is now consistently in your brand voice — no system prompt needed
LoRA: Efficient Fine-Tuning of Open Models
LoRA (Low-Rank Adaptation) fine-tunes open-source models (Llama, Mistral) by adding small trainable rank-decomposition matrices to attention layers. It trains 1000x fewer parameters than full fine-tuning — achievable on a single GPU in hours.
# pip install peft transformers datasets
from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments
from peft import LoraConfig, get_peft_model, TaskType
from trl import SFTTrainer
# Load base model (Llama or Mistral)
model_name = "mistralai/Mistral-7B-Instruct-v0.3"
model = AutoModelForCausalLM.from_pretrained(model_name, load_in_4bit=True) # QLoRA
tokenizer = AutoTokenizer.from_pretrained(model_name)
# Configure LoRA — only train rank-decomposition matrices
lora_config = LoraConfig(
task_type=TaskType.CAUSAL_LM,
r=16, # rank of low-rank matrices
lora_alpha=32, # scaling factor
lora_dropout=0.1,
target_modules=["q_proj", "v_proj"] # which attention layers
)
model = get_peft_model(model, lora_config)
model.print_trainable_parameters() # e.g., 0.1% of total parameters!
# Train with SFTTrainer
trainer = SFTTrainer(
model=model,
train_dataset=your_dataset, # Hugging Face dataset
args=TrainingArguments(
output_dir="./lora_output",
per_device_train_batch_size=4,
num_train_epochs=3,
)
)
trainer.train()
Tip: Always try prompt engineering + few-shot examples before fine-tuning. Fine-tuning costs money, time, and locks you into static behavior. If prompting works at 95% quality, it's usually not worth fine-tuning for the remaining 5%.
Warning: Fine-tuning does NOT reliably add new factual knowledge. Use RAG for that. Fine-tuning is for style, format, and task specialisation. A fine-tuned model still hallucinates facts it doesn't know.
Frequently Asked Questions
Can I fine-tune Claude?
Anthropic offers fine-tuning for select enterprise customers through their API. For most developers, prompt engineering with the base model is the recommended path. OpenAI offers fine-tuning on gpt-4o-mini and gpt-3.5-turbo for all API customers.
What is catastrophic forgetting?
When fine-tuning on a narrow dataset, models can 'forget' general capabilities. LoRA mitigates this because the base model weights stay frozen — only the small adapters are trained.
How much does fine-tuning cost?
OpenAI gpt-4o-mini fine-tuning: ~$3/1M training tokens. A typical dataset of 1000 examples × 500 tokens = 500K tokens → ~$1.50 to train. Inference with the fine-tuned model costs ~2x the base model. Hugging Face LoRA on Mistral-7B: free on a rented A100 GPU for a few hours.