AI/ML Deep Dive: RAG Systems

Think of it as hashing a sentence into a 1,536-dimensional space — except this hash doesn't aim to minimize collisions; it aims to put semantically close sentences at close coordinates. The cosine distance between two vectors is their semantic similarity.

Traditional full-text search (grep, ElasticSearch) only matches the literal characters — search "car" and you miss "automobile"; search "how to save money" and you miss "personal finance tips." Once you encode meaning as a numeric vector, "find documents with similar meaning" becomes "find points with similar coordinates" — a database problem. This is the foundation of every RAG (Retrieval-Augmented Generation) system.

Training uses massive "sentence pair + similar/not-similar label" data to learn a mapping f(text) → vector where "semantically similar → vectors near each other." Common dimensions are 384 / 768 / 1536 / 3072. At runtime, one API call turns any string into a vector.

# Intuition: similarity ≈ cosine of the angle between vectors
"The cat loves fish"          → [0.12, -0.34, 0.88, ...]   ┐
"My cat loves canned tuna"    → [0.15, -0.30, 0.91, ...]   ├ small angle ⇒ similarity ≈ 0.92
"Rate hikes hit the housing"  → [0.71, 0.05, -0.42, ...]   ┘ large angle vs above ⇒ similarity ≈ 0.08

Analogy: the classic Word2Vec example — king - man + woman ≈ queen. Semantic relations get encoded as directions in vector space.

from openai import OpenAI
import numpy as np

client = OpenAI()

def embed(text):
    # One-liner: string → 1536-dim vector
    r = client.embeddings.create(model="text-embedding-3-small", input=text)
    return np.array(r.data[0].embedding)

def cos_sim(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

a = embed("How to limit my child's screen time")
b = embed("Strategies for managing kids' device usage")
c = embed("How Fed rate hikes affect the stock market")

print(cos_sim(a, b))  # ~0.85 high
print(cos_sim(a, c))  # ~0.12 low

• Jay Alammar's Illustrated Word2Vec — the classic visual intro
• OpenAI's official Embeddings docs

It's MySQL where the WHERE clause "= or LIKE" gets replaced by "ORDER BY cosine distance LIMIT 10" — a database whose indexes are optimized for "find the nearest few among millions of vectors."

Once you have embeddings, "find the K most similar documents" becomes "compute cosine similarity against millions of 1,536-dim vectors and take top-K." Brute force is O(N) — seconds at scale. Unacceptable online. Vector databases (Pinecone, Weaviate, Milvus, pgvector) use Approximate Nearest Neighbor (ANN) indexes to push queries down to milliseconds.

The most common index is HNSW (Hierarchical Navigable Small World): organize vectors as a multi-layer graph — sparse top layer with hub nodes, dense bottom layer with all points. Queries hop quickly through the top layer to the rough region, then descend to refine. Like a skip list of pointers, in vector space.

# Traditional SQL                      # Vector DB
SELECT * FROM docs                      collection.query(
WHERE category = 'finance'      ←→        query_vector=embed(q),
ORDER BY title;                           filter={"category":"finance"},
                                          top_k=10)
# Key difference:
# Traditional DB index = B+ Tree, exact-field ordering
# Vector DB index = HNSW/IVF, approximate ordering by high-dimensional distance

Notice "approximate" — for speed, vector DBs do not guarantee 100% recall of the true top-K. 95%+ recall is usually enough for everyday use.

import chromadb
from openai import OpenAI

openai = OpenAI()
client = chromadb.Client()
col = client.create_collection("my_notes")

# 1) Write: text + auto-embedding + metadata
docs = ["Compounding is the eighth wonder of the world", "Screen time affects children's attention", ...]
embeds = [openai.embeddings.create(model="text-embedding-3-small",
                                   input=d).data[0].embedding for d in docs]
col.add(documents=docs, embeddings=embeds,
        metadatas=[{"topic": "finance"}, {"topic": "parenting"}],
        ids=["n1", "n2"])

# 2) Query: natural language → top-K relevant items
q_emb = openai.embeddings.create(model="text-embedding-3-small",
                                 input="how to make money work for me").data[0].embedding
hits = col.query(query_embeddings=[q_emb], n_results=3,
                 where={"topic": "finance"})  # metadata filter
print(hits["documents"])  # → ["Compounding is the eighth wonder of the world", ...]

• Pinecone's HNSW illustrated tutorial — friendly intro to ANN indexes
• Chroma official docs — the easiest local vector DB to start with

Like running EXPLAIN before a SQL query: plain "embedding + top-K" works in toy scenarios, but production mixes keyword search, metadata filters, query rewriting, and chunking strategies — tuning a multi-layer "retrieve → rank → generate" pipeline like you'd tune a multi-level cache hit rate.

The first lesson after shipping a naive RAG: 80% of wrong answers come from "the correct document was never retrieved," not from the LLM being weak. Retrieval strategies tackle exactly this: how to chunk documents, how to rewrite queries, how to mix vector + keyword + metadata (hybrid search), single-shot or multi-hop retrieval — all in service of aligning "what the user asks" with "the relevant slice of the knowledge base."

Production RAG pipeline:

User question q
   ↓ Query rewriting
      LLM turns "where's that note about compounding from last week?" into
      "compounding investing long-term returns" — better for embedding recall
   ↓ Hybrid retrieval
      (1) Vector top 20    ─┐
      (2) BM25 top 20      ─┼→ Union (~30-40 items)
      (3) Metadata filter  ─┘
   ↓ Re-ranking (next section)
      A more precise model trims 30-40 down to top 5
   ↓ Paste into prompt → LLM generates answer

The other critical decision is chunking: split long documents into 200-800 token blocks before embedding. Too large → noisy recall; too small → loss of context. The common approach is "split by semantic boundaries with 50-100 token overlap between adjacent chunks."

from langchain.retrievers import EnsembleRetriever, BM25Retriever
from langchain_community.vectorstores import Chroma
from langchain_openai import OpenAIEmbeddings

# 1) Set up vector and BM25 retrievers
vec_retriever = Chroma(embedding_function=OpenAIEmbeddings()).as_retriever(k=10)
bm25_retriever = BM25Retriever.from_documents(docs, k=10)

# 2) Hybrid: weighted fusion (0.6 vector + 0.4 keyword)
hybrid = EnsembleRetriever(
    retrievers=[vec_retriever, bm25_retriever],
    weights=[0.6, 0.4]
)

# 3) Query rewriting: let the LLM rephrase user input into a more searchable form
def retrieve(user_question):
    rewritten = llm.invoke(f"Rewrite this question as keyword search: {user_question}")
    return hybrid.invoke(rewritten)

• Anthropic's Contextual Retrieval engineering note — practical
• LangChain Retrieval concepts — strategy comparisons

Like a two-stage hiring funnel: HR screens 100 résumés with keywords (coarse), interviewers read 5 résumés closely (fine). The reranker is the "interviewer" in a RAG pipeline — a slower but more accurate model that trims 30-50 retrieved chunks down to the most relevant 5.

Embedding retrieval is fast but "coarse" — it compresses a passage into a single vector, losing detail. Re-rankers reverse the trade: feed pairs of (query, candidate document) into a more precise model (typically a cross-encoder) that outputs a "relevance score." Higher compute cost, but only on the few dozen recalled candidates, so total cost stays sane. Adding a reranker often lifts end-to-end RAG answer correctness by 10-20 points.

Bi-encoder (embedding) vs cross-encoder (re-ranker):

# Bi-encoder (embedding): query and doc encoded independently
emb_q   = encode(query)        ─┐
emb_doc = encode(doc)          ─┼→ cosine_similarity → score
# Pros: docs can be pre-embedded; only query is embedded at runtime; O(1) compare
# Cons: query and doc don't see each other — fine-grained matching is poor

# Cross-encoder (re-ranker): query and doc concatenated and fed to the model
score = model(f"[CLS] {query} [SEP] {doc}")
# Pros: every layer lets query and doc tokens attend to each other — much sharper matching
# Cons: every pair runs the model — N×M complexity, no full-corpus scans

Hence the two-stage approach: bi-encoder narrows millions to dozens; cross-encoder ranks the dozens finely.

from sentence_transformers import CrossEncoder
import cohere  # or use Cohere's hosted reranker API

# Option A: local cross-encoder (open source)
reranker = CrossEncoder("BAAI/bge-reranker-base")
query = "How to help my kid focus on homework"
candidates = [d1, d2, ..., d30]   # 30 candidates from the vector store

scores = reranker.predict([(query, d) for d in candidates])
top5 = sorted(zip(candidates, scores), key=lambda x: -x[1])[:5]

# Option B: hosted API (less work)
co = cohere.Client()
result = co.rerank(query=query, documents=candidates,
                   top_n=5, model="rerank-multilingual-v3.0")
top5 = [r.document["text"] for r in result.results]

# Splice top5 into the prompt and let the LLM produce the final answer

• Pinecone's Rerankers tutorial — concepts to code
• Cohere Rerank docs — the most-used hosted reranker API