Advanced Retrieval Strategies
Reranking, hybrid search, and contextual retrieval
Skills taught: llms.retrieval
Why Basic Retrieval Falls Short
Pure vector search misses exact keyword matches (product codes, model numbers, names) and retrieves chunks whose meaning is lost without context. Advanced retrieval combines multiple signals to significantly improve answer quality.
- Hybrid search: combines dense vector search with sparse BM25 keyword search
- Reranking: re-scores top-20 retrieved candidates with a cross-encoder for better relevance ordering
- Contextual retrieval: prepends document context to each chunk before embedding so chunks are meaningful in isolation
- Multi-query expansion: generates paraphrased queries to broaden retrieval coverage
Reranking
# pip install cohere
import cohere
co = cohere.Client("YOUR_COHERE_KEY")
# Step 1: Over-retrieve (top 20 from vector search)
candidates = [
"Redis is an in-memory database for caching",
"How to set up Redis Sentinel for high availability",
"Redis cache eviction policies: LRU, LFU, TTL",
"MongoDB document database for storing JSON data",
"Redis Cluster for distributed caching at scale",
# ... 15 more candidates
]
query = "How do I configure Redis cache expiration?"
# Step 2: Cross-encoder rerank — much more accurate than cosine similarity
results = co.rerank(
query=query,
documents=candidates,
top_n=3, # keep only top 3
model="rerank-english-v3.0"
)
print("Top 3 reranked results:")
for result in results.results:
print(f" Score: {result.relevance_score:.3f} | {candidates[result.index][:60]}")
Hybrid Search with BM25 + Vector
# Reciprocal Rank Fusion (RRF) to combine dense + sparse results
from rank_bm25 import BM25Okapi # pip install rank_bm25
import numpy as np
docs = [
"Redis TTL command sets expiry on keys",
"Redis cache eviction with maxmemory-policy",
"MongoDB TTL index for automatic document expiration",
"Python TTL cache decorator for function caching",
]
query = "Redis TTL expiration"
# BM25 keyword ranking
tokenized = [d.lower().split() for d in docs]
bm25 = BM25Okapi(tokenized)
bm25_scores = bm25.get_scores(query.lower().split())
bm25_rank = np.argsort(-bm25_scores) # descending
# (Assume we also have vector search ranks from ChromaDB)
vector_rank = [0, 2, 1, 3] # simulated vector search order
# RRF combination
def rrf_score(ranks: list[list[int]], k: int = 60) -> np.ndarray:
scores = np.zeros(len(docs))
for rank_list in ranks:
for position, doc_idx in enumerate(rank_list):
scores[doc_idx] += 1.0 / (k + position + 1)
return scores
combined = rrf_score([list(bm25_rank), vector_rank])
final_order = np.argsort(-combined)
print("Hybrid search results:")
for idx in final_order:
print(f" RRF score: {combined[idx]:.4f} | {docs[idx]}")
Contextual Retrieval (Anthropic)
Anthropic's contextual retrieval technique prepends a short, LLM-generated context description to each chunk before embedding it. This solves the problem of chunks like 'It increased by 42%' that have no meaning without knowing what 'it' refers to.
import anthropic
client = anthropic.Anthropic()
def add_context_to_chunk(full_document: str, chunk: str) -> str:
"""Prepend document context to a chunk before embedding."""
response = client.messages.create(
model="claude-haiku-5-5", # cheap model for context generation
max_tokens=200,
system="Provide a short (2-3 sentence) context for the following chunk.",
messages=[{"role": "user", "content": f"""
<document>
{full_document[:2000]} <!-- truncate for large docs -->
</document>
<chunk>
{chunk}
</chunk>
Write a brief context paragraph that explains what this chunk is about,
referencing the broader document. Output only the context, no preamble."""
}]
)
context = response.content[0].text
return f"{context}\n\n{chunk}" # prepend context to chunk
# Example
doc = "Q3 2026 Financial Report for Acme Corp. Revenue grew 18% YoY..."
chunk = "The growth rate accelerated to 42% in the enterprise segment."
contextual_chunk = add_context_to_chunk(doc, chunk)
print(contextual_chunk)
# Output: "This chunk is from Acme Corp's Q3 2026 financial report and discusses
# enterprise revenue growth. The 42% refers to YoY growth in enterprise revenue.
#
# The growth rate accelerated to 42% in the enterprise segment."
Tip: Always evaluate retrieval quality separately from generation quality. A bad retriever makes even the best LLM fail. Use RAGAS or a held-out Q&A set to measure retrieval recall before tuning the generation prompt.
Warning: Reranking adds ~200-500ms latency per query and costs additional API calls. Only add it when you have confirmed retrieval quality issues. Over-retrieve (top-20), rerank to top-5 — not the other way around.
Frequently Asked Questions
What is contextual retrieval?
Anthropic's technique of using an LLM to generate a short context description for each chunk and prepending it before embedding. This makes chunks self-contained and dramatically improves retrieval recall, especially for chunks that reference prior content.
What is Reciprocal Rank Fusion (RRF)?
An algorithm to combine ranked lists from multiple retrieval methods (e.g. BM25 + vector). Each document gets a score = Σ 1/(k + rank), where k=60. Documents appearing high in multiple lists rank highest.
Should I always use reranking?
Not necessarily. Measure your baseline retrieval quality first. Reranking adds 200-500ms and API cost. Use it when you confirm that precision at top-5 is low — when you're retrieving the right 20 but selecting the wrong 5.