Vector Databases
Embeddings, similarity search, and vector stores
Skills taught: llms.vector-db
Vector Similarity Search
Vector databases store embedding vectors and perform approximate nearest-neighbour (ANN) search to find semantically similar content in milliseconds. They are the retrieval backbone of every production RAG system.
- Embedding Vector
- A list of floats (e.g., 1536 dimensions for text-embedding-3-small) representing the semantic meaning of text. Similar texts have geometrically close vectors. Distance metrics: cosine similarity, dot product, Euclidean distance.
- Cosine Similarity
- A distance metric: cos(θ) = (A·B)/(|A||B|). Ranges from -1 (opposite) to 1 (identical). Used because it measures angle, not magnitude — so text length doesn't bias similarity scores.
# pip install chromadb
import chromadb
client = chromadb.Client() # In-memory (dev/prototype)
# For persistence: chromadb.PersistentClient(path="./chroma_db")
collection = client.create_collection(
name="docs",
metadata={"hnsw:space": "cosine"} # cosine similarity
)
# Add documents (ChromaDB auto-embeds with default model)
collection.add(
documents=[
"Redis is an in-memory key-value store used for caching and pub/sub",
"MongoDB is a NoSQL document database storing JSON-like BSON documents",
"PostgreSQL is a relational database with ACID compliance and SQL support",
"Pinecone is a managed vector database optimised for semantic search at scale",
],
ids=["doc1", "doc2", "doc3", "doc4"]
)
# Semantic search
results = collection.query(
query_texts=["What database should I use for caching?"],
n_results=2
)
print(results["documents"]) # ['Redis is an in-memory...', ...] — correct!
print(results["distances"]) # Cosine distances
Choosing a Vector Database
ChromaDB (local) vs Pinecone (managed cloud)
- ChromaDB: Open-source, runs locally or self-hosted. Great for prototyping and small-medium scale.
- Pinecone: Fully managed cloud vector database. Scales to billions of vectors.
# pip install pinecone-client openai
from pinecone import Pinecone
from openai import OpenAI
pc = Pinecone(api_key="YOUR_PINECONE_KEY")
oai = OpenAI()
# Create or connect to an index
index = pc.Index("docs-index")
def embed(text: str) -> list[float]:
return oai.embeddings.create(
input=text, model="text-embedding-3-small"
).data[0].embedding
# Upsert vectors with metadata
vectors = [
{"id": "chunk_1", "values": embed("Redis caching guide"), "metadata": {"source": "redis_docs.pdf", "page": 1}},
{"id": "chunk_2", "values": embed("MongoDB aggregation pipeline"), "metadata": {"source": "mongo_docs.pdf", "page": 5}},
]
index.upsert(vectors=vectors)
# Query with metadata filter
results = index.query(
vector=embed("How do I cache API responses?"),
top_k=3,
filter={"source": {"$eq": "redis_docs.pdf"}}, # only search Redis docs
include_metadata=True
)
for match in results.matches:
print(f"Score: {match.score:.3f} | Source: {match.metadata['source']}")
pgvector: Vector Search in PostgreSQL
# pip install psycopg2-binary pgvector
import psycopg2
from pgvector.psycopg2 import register_vector
from openai import OpenAI
oai = OpenAI()
# Connect and enable extension
conn = psycopg2.connect("postgresql://user:pass@localhost/mydb")
register_vector(conn)
cur = conn.cursor()
# Create table with vector column
cur.execute("""
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE IF NOT EXISTS docs (
id SERIAL PRIMARY KEY,
content TEXT,
embedding VECTOR(1536),
source VARCHAR
);
CREATE INDEX IF NOT EXISTS docs_embedding_idx ON docs
USING hnsw (embedding vector_cosine_ops);
""")
conn.commit()
# Insert a document
content = "Redis is used for session storage and caching"
embedding = oai.embeddings.create(input=content, model="text-embedding-3-small").data[0].embedding
cur.execute("INSERT INTO docs (content, embedding, source) VALUES (%s, %s, %s)",
(content, embedding, "redis_guide.pdf"))
conn.commit()
# Cosine similarity search
query_emb = oai.embeddings.create(input="How to cache data?", model="text-embedding-3-small").data[0].embedding
cur.execute("""
SELECT content, 1 - (embedding <=> %s::vector) AS similarity
FROM docs ORDER BY similarity DESC LIMIT 3""", (query_emb,))
for row in cur.fetchall():
print(f"Similarity: {row[1]:.3f} | {row[0][:60]}")
Tip: For production, use managed vector DBs like Pinecone, Weaviate, or pgvector (Postgres extension). ChromaDB is excellent for prototyping but requires self-managed scaling.
Frequently Asked Questions
What embedding model should I use?
For English-only: text-embedding-3-small (OpenAI, 1536 dims, cheap and fast). For multilingual: multilingual-e5-large or Cohere embed-multilingual-v3. Always use the SAME model for indexing and querying.
How many vectors can ChromaDB handle?
ChromaDB works well up to ~1M vectors on a single machine. Beyond that, use a managed solution (Pinecone, Weaviate, Qdrant) or pgvector with dedicated storage.
What is metadata filtering?
Most vector databases allow filtering by document metadata before or alongside vector search. Example: search only documents from 2024 or only from department='engineering'. This prevents irrelevant results from other data segments.