Redis Streams
XADD, XREAD, consumer groups, and durable messaging
Skills taught: redis.streams
What Makes Streams Different
Redis Streams are an append-only log of messages. Unlike Pub/Sub, messages are persisted. Unlike Lists, streams have unique IDs, support multiple consumers sharing work, and track which messages each consumer has acknowledged.
- Stream Entry
- A message in a Redis Stream, identified by a unique auto-generated ID (milliseconds-sequence, e.g. 1699100000000-0) and containing one or more field-value pairs.
Adding and Reading Messages
# Append an event (* = auto-generate ID)
XADD events * action "click" button "submit" userId "42"
# Returns: "1699100000000-0" (timestamp-sequence ID)
# Append with explicit ID
XADD events 1699100001000-0 action "purchase" amount "99.99"
# Read all messages from the beginning
XRANGE events - +
# Returns: list of [id, [field, value, ...]]
# Read from a specific ID onwards
XRANGE events 1699100000000-0 +
# Read newest messages (blocking — waits up to 5 seconds)
XREAD COUNT 10 BLOCK 5000 STREAMS events $
# $ means "only new messages after now"
# Count entries
XLEN events
# Trim stream to last 1000 entries
XADD events MAXLEN ~ 1000 * action "view"
Consumer Groups
Consumer groups allow multiple workers to share a stream. Each message is delivered to exactly one consumer in the group. If a worker crashes without acknowledging, the message stays in the Pending Entries List (PEL) for recovery.
# Create a consumer group starting from new messages ($)
XGROUP CREATE events workers $ MKSTREAM
# Worker 1 reads up to 5 messages
XREADGROUP GROUP workers consumer1 COUNT 5 STREAMS events >
# > means "give me messages not yet delivered to this group"
# Worker 2 reads next 5 messages
XREADGROUP GROUP workers consumer2 COUNT 5 STREAMS events >
# Acknowledge a processed message
XACK events workers 1699100000000-0
# Check pending (unacknowledged) messages
XPENDING events workers - + 10 # list up to 10 pending
# Claim a stuck message (idle for 60+ seconds)
XCLAIM events workers consumer2 60000 1699100000000-0
# Auto-claim all messages idle for 60+ seconds (Redis 6.2+)
XAUTOCLAIM events workers consumer2 60000 0-0 COUNT 10
Consumer Groups in Node.js
import Redis from 'ioredis';
const redis = new Redis();
const STREAM = 'events';
const GROUP = 'workers';
const CONSUMER = `worker-${process.pid}`;
// Ensure group exists
try {
await redis.xgroup('CREATE', STREAM, GROUP, '