Deploying ML Models
Saving models, building APIs, and serving predictions
Skills taught: ml.deployment, ml.model-serving, ml.fastapi, ml.joblib
Saving and Serving Models
Deployment is the step that turns a trained model into business value. The workflow: save the model and preprocessing pipeline → build an API that loads them → containerise with Docker → deploy to a server or cloud service.
Step 1: Save the Full Pipeline
Always save the complete preprocessing pipeline alongside the model. If you trained on scaled features but deploy only the model (without the scaler), predictions will be completely wrong.
import joblib
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import GradientBoostingClassifier
import os
# Build a Pipeline that includes preprocessing
pipe = Pipeline([
('scaler', StandardScaler()),
('model', GradientBoostingClassifier(n_estimators=200))
])
pipe.fit(X_train, y_train)
# Save to a versioned directory
os.makedirs('models/v1', exist_ok=True)
joblib.dump(pipe, 'models/v1/pipeline.joblib')
joblib.dump(X.columns.tolist(), 'models/v1/feature_names.joblib') # save column order
print(f'Model saved. Test score: {pipe.score(X_test, y_test):.3f}')
# Load later
loaded_pipe = joblib.load('models/v1/pipeline.joblib')
feature_names = joblib.load('models/v1/feature_names.joblib')
Step 2: Build a REST API with FastAPI
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, validator
import joblib
import numpy as np
import pandas as pd
from typing import Optional
# Load model on startup (not on every request)
pipe = joblib.load('models/v1/pipeline.joblib')
feature_names = joblib.load('models/v1/feature_names.joblib')
app = FastAPI(title='ML Model API', version='1.0')
class PredictRequest(BaseModel):
age: float
income: float
tenure_months: int
num_products: int
has_credit_card: int
@validator('age')
def age_must_be_positive(cls, v):
if v <= 0:
raise ValueError('age must be positive')
return v
class PredictResponse(BaseModel):
prediction: int
probability: float
model_version: str = 'v1'
@app.post('/v1/predict', response_model=PredictResponse)
def predict(req: PredictRequest):
# Build feature DataFrame in the correct column order
data = pd.DataFrame([req.dict()], columns=feature_names)
try:
pred = int(pipe.predict(data)[0])
prob = float(pipe.predict_proba(data)[0, 1])
return PredictResponse(prediction=pred, probability=prob)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@app.get('/health')
def health():
return {'status': 'ok', 'model': 'v1'}
Step 3: Run and Test Locally
# Start the server:
# uvicorn api:app --reload --port 8000
import requests
payload = {
'age': 42.0,
'income': 75000.0,
'tenure_months': 24,
'num_products': 2,
'has_credit_card': 1
}
response = requests.post('http://localhost:8000/v1/predict', json=payload)
result = response.json()
print(f'Prediction: {result["prediction"]}') # 0 or 1
print(f'Probability: {result["probability"]:.3f}') # P(class=1)
print(f'Status: {response.status_code}') # 200 = OK
Step 4: Containerise with Docker
# Dockerfile — saves as a text file, not .py
# FROM python:3.11-slim
# WORKDIR /app
# COPY requirements.txt .
# RUN pip install --no-cache-dir -r requirements.txt
# COPY . .
# EXPOSE 8000
# CMD ["uvicorn", "api:app", "--host", "0.0.0.0", "--port", "8000"]
# requirements.txt:
# fastapi==0.111.0
# uvicorn==0.30.0
# scikit-learn==1.5.0
# pandas==2.2.0
# joblib==1.4.0
# numpy==1.26.0
# Build and run:
import subprocess
print('docker build -t ml-api .')
print('docker run -p 8000:8000 ml-api')
Tip: Always save the scaler, imputer, and encoder objects alongside the model — not just the model weights. Predictions on unscaled or un-encoded data will be completely wrong.
Warning: Model drift: when real-world data distributions change over time, your model's performance degrades silently. Set up monitoring to compare prediction distributions and actual outcomes. Retrain when performance drops below your threshold.
FastAPI automatically generates interactive API documentation at /docs (Swagger UI). This lets you test endpoints directly in the browser without writing any extra code.
Frequently Asked Questions
What is model drift?
Model performance degrades over time because the real-world data distribution changes. For example, a fraud detection model trained on 2024 patterns may miss new fraud techniques in 2025. Monitor predictions vs actual outcomes and retrain regularly.
Should I use Flask or FastAPI for ML APIs?
FastAPI is the modern choice: it's significantly faster, generates automatic OpenAPI docs at /docs, validates inputs with Pydantic, and supports async out of the box. Flask is simpler but slower and lacks automatic validation.
What is the difference between online and batch serving?
Online serving (REST API) returns predictions in real time for individual or small groups of inputs — used for interactive applications. Batch serving runs predictions over large datasets on a schedule — used for overnight scoring or bulk processing.