In Java you'd reach for a search library and trust its ranking; here you build the ranking function yourself, because "the retriever found the wrong chunk" is the single most common way a RAG pipeline fails, and you can't fix what you can't compute by hand. Vector search ranked chunks by embedding distance; BM25 ranks them by term statistics — no model, no vectors, just word counts and how rare each word is. Most production retrievers run both and fuse the results, which is next.
BM25(corpus, k1=1.5, b=0.75) — corpus is a list of already
tokenised documents (list[list[str]]). k1 controls how fast a
repeated term's contribution saturates; b controls how much a document's
length counts against it.idf(term) — the standard Robertson/Spärck-Jones form,
log((N - df + 0.5) / (df + 0.5)). Note what that means: a term in every
document gets an idf at or below zero — it is common enough to carry no signal, and
the classic "+1 inside the log" variant you'll see in some libraries hides that fact.score(query, doc_idx) — sum over the query's terms of
idf(t) * tf*(k1+1) / (tf + k1*(1 - b + b*len(doc)/avgdl)). b=0
must switch the length term off entirely; k1=0 must make the score stop caring
how many times a term repeats past the first hit.rank(query) -> list[int] — document indices, best match first, sorted
descending by score; ties broken by the earlier document index (stable).recall_at_k(ranked_ids, relevant_set, k),
mrr(ranked_ids, relevant_set), ndcg_at_k(ranked_ids, relevant_set, k)
— the three numbers every retrieval eval report leads with, computed over a ranking you
already have (not tied to BM25 at all — f1-build reuses these on a
real hybrid retriever later in this queue).Check 2 recomputes the exact BM25 formula independently and compares to
your score() to 1e-6 — matching it means your idf, your tf-saturation term and
your length-normalisation term are all individually correct, not just "close on this one
example". Check 3 sets b=0 on two documents that share a term's frequency but
differ only in padding: if your length ratio is right, the padding must not move the score
at all.
df), not term
occurrences. N is the number of documents. Use
math.log((N - df + 0.5) / (df + 0.5)) exactly — no "+1" inside the log, or a
term in every document stops being ≤ 0 and check 1 fails.__init__; don't re-tokenise on every call. avgdl is the mean
document length over the whole corpus, computed once. For a term absent from the document,
skip it — it contributes 0, not a division by zero.b=0, you multiplied instead of adding it into the 1 - b + b*(...)
factor, or you applied length normalisation somewhere else in the formula.k1=0 the whole
k1*(1 - b + b*len/avgdl) term vanishes and tf*(k1+1)/tf collapses
to 1 regardless of tf. If your score still grows with tf at
k1=0, you hardcoded k1+1 as a constant instead of using the
parameter.(-score, doc_index) so Python's sort gives you both
descending order and a deterministic tie-break in one call; don't rely on dict ordering.1/1 = 1.0, not 1/0.1 / log2(rank + 1) using the 1-indexed
rank, i.e. 1 / log2(i + 2) for a 0-indexed loop variable i. A
linear discount (1/(i+1)) happens to still reach 1.0 on a perfect ranking, so
it looks right until you check a mixed one.