Zero-Downtime Deploys
Ship new versions without a maintenance window — blue-green deployments, rolling updates, health checks, and graceful shutdown
Skills taught: deployment.blue-green, deployment.rolling-deploy, deployment.health-checks, deployment.graceful-shutdown, deployment.connection-draining
The problem with naive deploys
A naive deploy: stop the old process, start the new one. During the stop-start gap (even if it's only 2-3 seconds), requests fail with connection refused errors. Users mid-action get errors. For a site with 1,000 concurrent users, a 3-second downtime window affects potentially hundreds of in-flight requests. Zero-downtime deployment means the transition from old to new is invisible to users — no errors, no dropped connections, no failed requests.
Strategy 1: Rolling deploy (PM2)
# Start app in cluster mode (multiple workers = rolling updates possible)
pm2 start ecosystem.config.js
# ecosystem.config.js:
# instances: 'max' (or a number like 4)
# exec_mode: 'cluster'
# Deploy update with zero downtime:
git pull origin main
npm ci --production
npm run build
# 'reload' rotates workers one by one (zero-downtime)
# 'restart' kills all then starts (brief downtime)
pm2 reload myapp
# What happens during pm2 reload:
# PM2 has 4 workers (0, 1, 2, 3) all serving traffic
# → Starts worker 4 (new version), waits for it to be ready
# → Routes traffic to workers 0, 1, 2, 4
# → Gracefully stops worker 3 (finishes in-flight requests)
# → Starts worker 5, routes traffic to 0, 1, 2, 4, 5
# → Gracefully stops worker 2
# ... continues until all workers are new version
# → Removes temporary workers 4, 5
# Total downtime: 0ms
Graceful shutdown — don't drop connections
import express from 'express';
import http from 'http';
const app = express();
const server = http.createServer(app);
// Track active connections for graceful drain
const connections = new Set<any>();
server.on('connection', (conn) => {
connections.add(conn);
conn.on('close', () => connections.delete(conn));
});
server.listen(3000, () => {
console.log('Server running on :3000');
});
// SIGTERM: sent by PM2/Docker/Kubernetes when stopping the process
process.on('SIGTERM', gracefulShutdown);
process.on('SIGINT', gracefulShutdown); // Ctrl+C in development
async function gracefulShutdown() {
console.log('SIGTERM received, starting graceful shutdown');
// 1. Stop accepting new connections
server.close(() => {
console.log('HTTP server closed');
});
// 2. Wait for active requests to finish (max 30 seconds)
const shutdownTimeout = setTimeout(() => {
console.error('Graceful shutdown timeout — forcing exit');
process.exit(1);
}, 30_000);
// 3. Close database connections
await db.end();
console.log('Database connections closed');
// 4. Close Redis connection
await redis.quit();
clearTimeout(shutdownTimeout);
process.exit(0);
}
Strategy 2: Blue-Green deployment
CONCEPT:
Run two identical production environments: Blue and Green
At any time, one is LIVE (serving traffic) and one is IDLE
DEPLOY FLOW:
1. Blue is live (current version v1.0)
2. Deploy v1.1 to Green (idle)
3. Run smoke tests against Green directly
e.g. curl https://green.internal/health
4. Flip the load balancer: route all traffic to Green
5. Blue is now idle (still running v1.0)
6. Monitor Green for 10-30 minutes
7. If problem: flip load balancer back to Blue (instant rollback)
8. If good: decommission/repurpose Blue for next deploy
BENEFITS:
✓ Zero-downtime cutover
✓ Instant rollback (just flip the switch)
✓ Test the new version on real infrastructure before going live
COST:
✗ Need double the servers (or use feature flags/health check routing)
✗ More complex setup
USED BY:
- Large companies with critical uptime requirements
- AWS has built-in blue-green for ECS, CodeDeploy
- Kubernetes: separate deployments behind a Service
Health checks — how load balancers know you're ready
// Every app that wants zero-downtime deploys MUST have a /health endpoint
// The load balancer / deploy system polls it to know when the new version is ready
app.get('/health', async (req, res) => {
// Quick liveness check — just proves the process is running
res.json({ status: 'ok', version: process.env.APP_VERSION });
});
// Better: readiness check (proves app can actually serve traffic)
app.get('/ready', async (req, res) => {
try {
// Check database is reachable
await db.query('SELECT 1');
// Check Redis is reachable
await redis.ping();
res.json({
status: 'ready',
db: 'ok',
redis: 'ok',
uptime: process.uptime(),
version: process.env.APP_VERSION,
});
} catch (error) {
// Return 503 — load balancer will route away from this instance
res.status(503).json({
status: 'not ready',
error: (error as Error).message,
});
}
});
// Docker Compose healthcheck:
// healthcheck:
// test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
// interval: 10s
// timeout: 5s
// retries: 3
// start_period: 30s
Tip: Database migrations are the hardest part of zero-downtime deploys. If your new code expects a new column that doesn't exist yet, deploying new code before running the migration will cause errors. The safe pattern: (1) deploy a migration that only ADDS things (new column with nullable default), (2) deploy new code that works with both old and new schema, (3) optionally backfill data, (4) remove old code that references old schema. Never run a migration that changes or drops something at the same time as deploying code that relies on it.
Frequently Asked Questions
Do I need zero-downtime deploys for my small app?
If you have fewer than 100 users and deploy during off-hours, the 2-3 second downtime of a naive deploy probably doesn't matter. Zero-downtime matters when: you have users in multiple timezones (there's no 'safe' downtime window), you process financial transactions (a dropped request = a lost sale), or you have an SLA. PM2 cluster mode reload is low-effort and worth setting up regardless — it doesn't add complexity once configured.