Hybrid search and reranking for better retrieval

Read a sample · Chapter 01 of 06

A longer retrieval pipeline is useful only when its additional work improves the result you need. Name the role of each stage before comparing scores.

Imagine a document collection containing invoice instructions, error-code notes and camera guides. A query may contain an exact identifier, a broad description or both. The first-stage retrievers return ordered document IDs. A fusion stage combines their lists. A later reranking stage changes the order of a bounded candidate set. The final context selector may shorten that list again before an answer is composed.

The named primary references at the end explain reciprocal rank fusion and the retrieve-then-rerank pattern. A bi-encoder can represent queries and documents separately; a cross-encoder can inspect a query and candidate text together. That describes possible model architecture, not the implementation in this workbook. Here we isolate the ranking plumbing using supplied ordered IDs and supplied scores. We do not download a model, build a keyword index or run semantic inference.

The three queries below are invented teaching fixtures. The keyword and dense lists are hand-authored possible outputs, and the scores are a scripted reranker response. They are neither human study data nor measurements from Mickai infrastructure. The relevant set is the chosen answer key for each toy query. It stays separate from the reranker scores so the program cannot use the evaluation answer key to choose an order.

Save the following block as fixtures.py in a new local folder. You will create ranking.py, demo.py and test_ranking.py alongside it. Use Python 3.12 for the tested path. No package install, account, GPU or paid model call is needed. These files contain only synthetic IDs; do not replace them with confidential queries while experimenting with an external service.

"""Invented rankings and scores, not results from a trained model."""
CASES = [
    {
        "query": "read scanned invoice",
        "keyword": ["invoice-layout", "scan-settings", "ocr-check"],
        "dense": ["ocr-check", "invoice-layout", "camera-guide"],
        "relevant": {"ocr-check"},
        "scores": {"ocr-check": .9, "invoice-layout": .4,
                   "scan-settings": .55, "camera-guide": .2},
    },
    {
        "query": "reset error E17",
        "keyword": ["e17-reset", "generic-restart", "e71-reset"],
        "dense": ["generic-restart", "e71-reset", "e17-reset"],
        "relevant": {"e17-reset"},
        "scores": {"generic-restart": .9, "e71-reset": .8,
                   "e17-reset": .3},
    },
    {
        "query": "volcano map",
        "keyword": [], "dense": [], "scores": {},
        "relevant": {"volcano-atlas"},
    },
]

The volcano query is intentionally empty at both retrieval stages even though the answer key names a relevant document. This is a candidate-generation miss. Reordering an empty list cannot fix it. In a real system, distinguish no matching records, an upstream error, an expired permission scope and an intentionally empty collection; those states need different diagnostics even if the user-facing list is empty.

Read a sample · Chapter 02 of 06

Reciprocal rank fusion gives each appearance a contribution based on its position. Keep the rank offset separate from the number of results you retrieve or return.

For a document at one-based rank r, this lab adds 1 / (offset + r). Its fused score is the sum over the lists containing that document. An absent document contributes nothing for that list. With offset 60, the first position contributes 1/61. The offset controls the difference between positions; it is not the nearest-neighbour count, candidate window, evaluation cutoff or final context length.

This approach does not add a keyword engine score directly to a vector similarity. Rank fusion uses order and discards the magnitude of the original score gaps. That is a deliberate trade-off: a large gap and a tiny gap become indistinguishable if they produce the same ordering. Preserve raw scores separately in a diagnostic record if you need to inspect why a retriever produced its list.

In the invoice fixture, invoice-layout appears at keyword rank 1 and dense rank 2. Its score is 1/61 + 1/62, about 0.03252. ocr-check appears at keyword rank 3 and dense rank 1, giving 1/63 + 1/61, about 0.03227. Fusion therefore places invoice-layout first. Agreement between retrievers is a ranking signal here, not proof that the top document answers the question.

Save this block as the first part of ranking.py. The next two chapters append the rest. IDs are short lowercase local keys. Each source ranking may contain an ID once; the same ID in different lists is expected. Duplicate IDs within one list are rejected instead of counting as extra votes. List sizes and offsets are bounded for this teaching tool, not as a production capacity claim.

"""Small local ranking lab. Inputs are ordered IDs, not raw engine scores."""
from collections import defaultdict
from math import isfinite, fsum
import re


def ids(value):
    if type(value) is not list or len(value) > 1000:
        raise ValueError("Expected a list of at most 1000 IDs")
    if any(type(x) is not str or
           re.fullmatch(r"[a-z][a-z0-9-]{0,63}", x) is None
           for x in value):
        raise ValueError("Invalid document ID")
    if len(set(value)) != len(value):
        raise ValueError("Duplicate ID within a ranking")
    return value


def rrf(rankings, *, offset=60, limit=10):
    if type(rankings) is not list or not 1 <= len(rankings) <= 8:
        raise ValueError("Expected 1 to 8 rankings")
    if type(offset) is not int or not 1 <= offset <= 10000:
        raise ValueError("Invalid rank offset")
    if type(limit) is not int or not 1 <= limit <= 1000:
        raise ValueError("Invalid result limit")
    terms = defaultdict(list)
    for ranking in rankings:
        for rank, doc_id in enumerate(ids(ranking), start=1):
            terms[doc_id].append(1 / (offset + rank))
    scored = [(doc_id, fsum(parts))
              for doc_id, parts in terms.items()]
    return sorted(scored, key=lambda row: (-row[1], row[0]))[:limit]

The tie policy is explicit: equal fused scores are ordered by document ID. This makes repeated runs reproducible and independent of which retriever list is supplied first. The function builds its own contribution lists and does not mutate the supplied rankings. It uses fsum for the floating-point sum; printed scores may be rounded for display, but sorting uses the unrounded values.

Eight source lists are permitted by this local contract. They need not be independent. If you submit the same ranking through several duplicated query channels, its candidates receive several contributions and may dominate. Channel selection and weighting are modelling decisions. This implementation deliberately has equal channel weights and rejects malformed data; it does not infer which channels deserve more influence.

Read a sample · Chapter 03 of 06

Reranking revisits candidates already retrieved. Treat its score response as a contract you can validate, and retain a defined tie policy.

Watch the order change

Keyword ranking: invoice-layout → scan-settings → ocr-check. Dense ranking: ocr-check → invoice-layout → camera-guide.

Fuse the two lists with equal weights and one-based ranks: add 1 / (60 + rank) for every list containing the document. Missing documents contribute zero. Keep all four unique candidates, then sort by the invented reranker scores. Those scores are not probabilities. The two score columns use different scales and must not be compared directly.

Invoice query: rows in fused order, scores rounded for display

Document ID	RRF score	Fixture score	Final position
invoice-layout	0.032522	0.40	3
ocr-check	0.032266	0.90	1
scan-settings	0.016129	0.55	2
camera-guide	0.015873	0.20	4

For ocr-check, 1/63 + 1/61 = 0.032266 (rounded). For invoice-layout, 1/61 + 1/62 = 0.032522 (rounded), so it ranks first after fusion. Reranking orders the same candidates as ocr-check → scan-settings → invoice-layout → camera-guide.

All three fictional queries: reciprocal rank at cutoff 3 (RR@3)

Query	Keyword	Fusion	Rerank
read scanned invoice	1/3	1/2	1
reset error E17	1	1/2	1/3
volcano map	0	0	0

The invoice example improves, the error-code example regresses, and the volcano query retrieves no candidates at any stage. Its relevant volcano-atlas item stays outside the candidate set. A reranker cannot recover it. All rankings, scores and relevance judgements here are invented teaching fixtures, not measured search-engine or model performance.

Append the next block to ranking.py. It accepts a list of candidate IDs and exactly one numeric score per candidate. Higher means earlier in this example. Real model score ranges and interpretation vary; a large raw value is not automatically a probability or confidence estimate. This lab permits finite negative scores and does not impose an invented zero-to-one range.

The keys must match the candidate set exactly. A missing score may indicate truncation or a failed request; an extra ID may indicate a stale or mismatched response. Silently filling missing scores with zero would obscure that distinction. Booleans, strings, infinities and NaN are rejected. An empty shortlist with an empty score map remains a valid empty result.

def rerank(candidates, scores, *, limit=10):
    ids(candidates)
    if type(limit) is not int or not 1 <= limit <= 1000:
        raise ValueError("Invalid result limit")
    if type(scores) is not dict or set(scores) != set(candidates):
        raise ValueError("Need exactly one score per candidate")
    for score in scores.values():
        if type(score) not in (int, float):
            raise ValueError("Expected real finite scores")
        try:
            finite = isfinite(score)
        except OverflowError:
            finite = False
        if not finite:
            raise ValueError("Expected real finite scores")
    # Stable sorting preserves candidate order when scores tie.
    return sorted(candidates, key=lambda x: -scores[x])[:limit]

Sorting is stable: tied reranker scores retain the incoming fused order. That differs from fusion, where document ID breaks ties. Both policies are legitimate local choices because they are explicit and tested. The function does not use relevance labels, document titles or the original raw retriever scores. It cannot add a new document, regardless of how plausible another title would look.

The fusion limit determines which candidates reach the reranker. If the invoice shortlist is cut to one item, it contains only invoice-layout. ocr-check is already gone and cannot become the final result. A larger window can admit useful candidates, but requires more text processing or model pairs in a real system. Measure that cost instead of assuming a fixed window works for every collection and query type.

Save demo.py as follows. The score-map comprehension deliberately selects only the IDs in the current candidate window. This keeps a stored fixture reusable across different window sizes without weakening the strict rerank function. In a real adapter, send the selected document texts, keep their IDs with them, and map returned scores back using the documented response contract. Do not zip unrelated result orders by accident.

from fixtures import CASES
from ranking import rrf, rerank, metrics

for case in CASES:
    fused = rrf([case["keyword"], case["dense"]], limit=4)
    candidates = [doc_id for doc_id, score in fused]
    # Subset recorded scores after the candidate window is chosen.
    scores = {doc_id: case["scores"][doc_id] for doc_id in candidates}
    final = rerank(candidates, scores, limit=4)
    print(case["query"])
    stages = [("keyword", case["keyword"]),
              ("fusion", candidates), ("rerank", final)]
    for name, ranking in stages:
        result = metrics(ranking, case["relevant"], cutoff=3)
        print(f"  {name}: {','.join(ranking) or '(empty)'}")
        print(f"    R@3={result['recall']:.3f} "
              f"RR@3={result['rr']:.3f}")

Do not run demo.py until the metrics function from the next chapter has been appended. The query text is present for interpretation, but our replay does not compute a score from it. Replacing the fixtures with a cross-encoder or another learned scorer is a separate integration task. Record the model revision, maximum input length and any document truncation so later quality changes can be investigated.

Read a sample · Chapter 04 of 06

A single relevance number can hide a serious regression. Use an explicit answer key, cutoff and missing-judgement policy, then inspect every teaching case.

Append metrics to ranking.py. Recall at a cutoff is the number of judged relevant IDs found in the returned prefix divided by the size of the relevant set. Reciprocal rank at that cutoff is one divided by the position of the first relevant result, or zero if none appears in the prefix. They answer different questions: coverage of known relevant items versus how soon the first useful result appears.

def metrics(ranking, relevant, *, cutoff=3):
    ids(ranking)
    if type(relevant) is not set or not relevant:
        raise ValueError("Need at least one relevant judgement")
    ids(list(relevant))
    if type(cutoff) is not int or not 1 <= cutoff <= 1000:
        raise ValueError("Invalid metric cutoff")
    head = ranking[:cutoff]
    hits = sum(doc_id in relevant for doc_id in head)
    reciprocal = next((1 / rank for rank, doc_id in
                       enumerate(head, 1) if doc_id in relevant), 0.0)
    return {"recall": hits / len(relevant), "rr": reciprocal}

This function requires at least one relevant judgement. An unjudged query is not assigned a zero-quality score, because that would mix absent evaluation evidence with a measured miss. A deliberately judged no-answer query needs a separate measure, such as whether the system abstains appropriately. For this exercise, volcano-atlas is explicitly relevant and missing, so the volcano query has valid zero recall and zero reciprocal rank.

Run python demo.py from the folder containing all four files. The output below is from the tested program. At cutoff three every invoice and E17 ranking still contains the one relevant item, so recall stays at one. Reciprocal rank exposes the movement: invoice improves from one-third to one-half to one, while E17 drops from one to one-half to one-third.

read scanned invoice
  keyword: invoice-layout,scan-settings,ocr-check
    R@3=1.000 RR@3=0.333
  fusion: invoice-layout,ocr-check,scan-settings,camera-guide
    R@3=1.000 RR@3=0.500
  rerank: ocr-check,scan-settings,invoice-layout,camera-guide
    R@3=1.000 RR@3=1.000
reset error E17
  keyword: e17-reset,generic-restart,e71-reset
    R@3=1.000 RR@3=1.000
  fusion: generic-restart,e17-reset,e71-reset
    R@3=1.000 RR@3=0.500
  rerank: generic-restart,e71-reset,e17-reset
    R@3=1.000 RR@3=0.333
volcano map
  keyword: (empty)
    R@3=0.000 RR@3=0.000
  fusion: (empty)
    R@3=0.000 RR@3=0.000
  rerank: (empty)
    R@3=0.000 RR@3=0.000

These opposing movements are intentional. The E17 query needs an exact identifier; the invented dense list and scorer prefer the generic restart note. The keyword and final-rerank reciprocal-rank averages across the first two queries are both two-thirds, despite substantial per-query changes. Including the missed volcano query lowers both means to four-ninths. Neither equality establishes that the systems are interchangeable for users.

The answer keys are complete only by definition of this small fixture. In a real collection, unjudged results may contain useful evidence that the annotation process missed. Define relevance with examples, record who or what supplied the judgements, and review disagreements. Keep evaluation labels separate from training, query tuning and reranker prompts so a good score does not merely reflect access to the answers.

Read a sample · Chapter 05 of 06

Tests should detect a wrong ranking, not merely confirm that the program returns a list. Small counterexamples make ranking mistakes easier to diagnose.

Save the following as test_ranking.py. Run python -m unittest -v test_ranking. The tested version passes ten test methods. The tests separately check one-based contributions, absent-list behaviour, both tie policies, duplicates, score alignment, invalid numbers, descending reranker order, metric cutoffs, missing judgements and result limits.

import unittest
from ranking import rrf, rerank, metrics


class RankingTests(unittest.TestCase):
    def test_one_based_rank_sum(self):
        result = dict(rrf([["a", "b"], ["b"]], offset=10))
        self.assertAlmostEqual(result["a"], 1/11)
        self.assertAlmostEqual(result["b"], 1/12 + 1/11)

    def test_absence_adds_nothing(self):
        self.assertEqual(rrf([["a"], []]), [("a", 1/61)])
        self.assertEqual(rrf([[], []]), [])

    def test_ties_are_reproducible(self):
        self.assertEqual([x[0] for x in rrf([["b"], ["a"]])],
                         ["a", "b"])
        self.assertEqual(rerank(["b", "a"], {"a": 2, "b": 2}),
                         ["b", "a"])

    def test_duplicates_are_not_extra_votes(self):
        with self.assertRaises(ValueError):
            rrf([["a", "a"]])

    def test_reranker_cannot_invent_or_lose_scores(self):
        for scores in ({}, {"a": 1, "b": 2}):
            with self.assertRaises(ValueError):
                rerank(["a"], scores)

    def test_bad_scores_fail(self):
        for bad in (float("nan"), float("inf"), True, "0.8"):
            with self.assertRaises(ValueError):
                rerank(["a"], {"a": bad})

    def test_highest_score_first(self):
        self.assertEqual(rerank(["a", "b"], {"a": -2, "b": 3}),
                         ["b", "a"])

    def test_metric_cutoff_and_denominator(self):
        self.assertEqual(metrics(["x", "a", "b"], {"a", "b"},
                                 cutoff=2), {"recall": .5, "rr": .5})
        self.assertEqual(metrics([], {"a"}),
                         {"recall": 0.0, "rr": 0.0})

    def test_unjudged_query_is_not_zero_quality(self):
        with self.assertRaises(ValueError):
            metrics(["a"], set())

    def test_window_bounds(self):
        self.assertEqual(len(rrf([["a", "b"]], limit=1)), 1)
        for bad in (0, -1, True, 1.5):
            with self.assertRaises(ValueError):
                rrf([["a"]], limit=bad)


if __name__ == "__main__":
    unittest.main()

A useful additional test uses a two-document universe where you can calculate every expected order by hand. The release verification goes further: it enumerates all partial rankings over four IDs, combines every pair at offsets 1, 10 and 60 and limits 1 and 4, and compares the implementation with an independent rational-arithmetic formula. That produces 25,350 configurations. It also swaps channel order, checks malformed inputs and exercises metric cutoffs.

The 76,817 recorded assertions include ranking orders, numeric comparisons, invariance and boundary checks. They demonstrate the stated behaviour on a small finite domain, not correctness for every possible production input. Long lists are bounded here, but the implementation is an in-memory teaching tool. It does not implement network timeouts, request cancellation, byte limits or an external service boundary.

Three isolated mutations were also rejected by genuine failed expectations. Starting ranks at zero changes the first contribution. Sorting reranker scores ascending prefers worse scores under this contract. Ignoring the metric cutoff counts results that should not be evaluated. A test suite that survives all three changes would not be checking the central promises of this lab.

Read a sample · Chapter 06 of 06

The next step is a measured adapter to a real retrieval system. Preserve the stage boundaries so an upstream improvement or failure can be isolated.

Start with a small, permission-appropriate corpus and a versioned query set. Connect one keyword retriever and one dense retriever, both returning the same stable document or chunk IDs. Preserve the original text and source reference behind each ID. Two chunks from one document are distinct candidates only if your identity and evaluation policies say so; otherwise duplicated passages can occupy the context budget and distort apparent coverage.

Apply the required eligibility restrictions before content is sent to a scorer or answer model. Recheck access at delivery where the application requires it. Ranking is not an authorisation mechanism: a low score does not protect a restricted record. Do not infer permissions from relevance labels, query wording or a generated instruction inside a retrieved passage. This course does not implement identity, permission filtering or prompt-injection protection.

Keep an execution record for each evaluated query. Include the corpus snapshot, retriever configuration, candidate depths, ordered IDs, fusion offset, reranker revision, input truncation, selected context and observed timings. Record hashes or suitable references rather than indiscriminately logging sensitive source text. The list below is an experiment worksheet, not a requirement to send private data to an analytics vendor.

Scroll sideways to see every column.

Compare keyword-only, dense-only, fusion and fusion-plus-reranking on the same held-out queries. Include exact identifiers, paraphrases, ambiguous wording, stale documents and known no-answer cases. Break down results by query type and inspect regressions before choosing a configuration. The supplied toy demo is deliberately too small for an inference about expected quality, statistical significance, cost or user satisfaction.

For latency, measure the real stage boundaries. Two retrievers running concurrently need not cost the sum of their separate durations, yet shared capacity, queueing and network overhead may dominate. Candidate text length and batching affect the reranker. Report observed distributions and the test conditions, not only one warm local request. Do not turn the timing of this rank-only Python script into a claim about an embedding model or GPU.

Specify fallback behaviour before an outage. If one retriever fails, either return a clearly recorded degraded result from the other under an agreed policy, or fail the request; do not quietly report an empty successful list. If the reranker fails, a validated fused shortlist may be an acceptable fallback after the same permission checks. Record the chosen path so offline evaluation can distinguish normal and degraded runs.

Finally, evaluate the composed answer separately from retrieval. A relevant passage can be misquoted or ignored, and a fluent answer can be unsupported. Preserve source references through the final context selection and check whether the answer actually follows from them. Hybrid retrieval and reranking are ways to select material; they do not certify that a generated conclusion is correct.