Exercise ex-rrf-mmr — fusing rankings without comparable scores, then re-ranking for diversity

In Java, combining two ranked lists feels like a merge: sort by whichever Comparator, or average two numbers that both look like "how good". But a BM25 score and a cosine similarity are not on the same scale — one is unbounded and log-shaped, the other lives in [-1, 1] — averaging them is often meaningless. Reciprocal Rank Fusion (RRF) sidesteps the mismatch entirely: it never touches a raw score, only where each document landed in each list. ex-bm25's rank() output and a vector search's ranking fuse here exactly as they will in f1-build's hybrid retriever. Once you have one fused list, Maximal Marginal Relevance (MMR) re-ranks it again — trading a little relevance for diversity, so the top slots aren't five near-duplicate chunks of the same paragraph.

~75 minruns in the browser 7 checksex-rrf-mmr

What you're building

rrf(rankings: list[list[int]], k=60) -> list[int] — rankings is a list of already-ranked id lists, best id first. Fused score of an id is the sum, over every ranking that contains it, of 1 / (k + rank) where rank is its 1-indexed position in that ranking. An id missing from a ranking contributes nothing from it — not a penalty, not a worst-case rank. Return every id seen anywhere, highest fused score first, ties broken by the smaller id.
mmr(query_vec, doc_vecs, lam, n) -> list[int] — doc_vecs is (m, d), query_vec is (d,). Starting with the single most relevant document (highest cosine similarity to query_vec), repeatedly pick the remaining candidate maximising lam * cos(query_vec, doc) - (1 - lam) * max(cos(doc, already_picked)) until n are picked or the corpus runs out. Never pick the same index twice.

Check 1 is a genuine trap for the plausible-but-wrong "sum the raw ranks" implementation. Rank sums and reciprocal-rank sums both reward "ranked highly everywhere", so on many small examples they agree; they part ways when one document has a very good rank in one list and a very ordinary rank in the other. Here doc 0 lands 1st and 6th while doc 1 lands 3rd and 3rd: a rank sum prefers doc 1 (6 vs 7), reciprocal rank prefers doc 0 (0.643 vs 0.5), because 1/(k+rank) rewards a top rank far more than it punishes a middling one. Check 3 is the same idea from a different angle — push k from 1 to 1000 and watch which document wins flip.

If you get stuck

rrf — accumulate into a dict as you walk each ranking: score[doc_id] += 1 / (k + i + 1) for a 0-indexed loop variable i (so the first entry, i=0, gets rank 1). Sort by (-score[d], d) so Python's sort gives descending score and a deterministic tie-break in one call. Don't special-case a missing id — a dict.get default of 0 for "not yet scored" already does the right thing when you first see it.
the k=1 vs k=1000 check — if both give the same winner, you are not using k at all (the classic sign of a raw-rank-sum implementation hiding behind a k parameter it never reads).
mmr's first pick — pure relevance, no diversity term involved yet, because there is nothing picked to be diverse from.
mmr's later picks — the diversity term is max similarity between the candidate and each already-picked document. Two ways to get this wrong look reasonable: measure the picked documents against the query, and the term stops depending on the candidate — subtracting the same constant from every score never changes which one wins, so lam silently does nothing past the first pick; or measure the candidate against the query, and you have only re-weighted relevance — lam*rel - (1-lam)*rel is still relevance order (or its reverse), never diversity. Check 5 has a fixture for each.
never repeating an id — skip any index already in your picked list before scoring it as a candidate. Skipping it is not automatic: a document's similarity to itself is exactly 1.0, the maximum possible, so a high-relevance document can still win the argmax again if you forget to exclude it — most visibly once lam is large.
n larger than the corpus — clamp with n = min(n, len(doc_vecs)) up front; the loop then naturally exhausts the corpus instead of looping forever or re-picking.

Part of the Illustrated AI/ML catalog · Phase 6 · LLMs, RAG & Agents. Built on ex-bm25 (its rank() output) and vector search; feeds advanced RAG and f1-build's hybrid retriever.