HBC-RAG · retrieval granularity study · legal contracts · November 2026

Hop-Bounded Agentic RAG for Legal Contract Question Answering

How many passages should an agentic RAG system retrieve per step when the answer is a clause buried in a contract? Measured on 140 questions from LegalBench-RAG over 468 contracts and policies, scored by character span against human-labelled evidence.

0gold snippet hit, coarse (fine 47%)
0character recall, coarse (fine 42%)
0median latency, coarse (fine 8.9s)
0lower cost per run

What improved

Four passages per step instead of one, DeepSeek V4.1 Flash, 140 questions. Intervals are paired bootstrap, 95%.

fine, 1 per hopcoarse, 4 per hop
Gold snippet hit
+0.11
Character recall
+0.12
Character precision
-0.05
+11 pts gold snippet hit (CI +5 to +17) +12 pts character recall (CI +6 to +19) 53% lower median latency39% lower p95 latency 47% fewer hops26% lower cost

What got worse: character precision, 0.120 to 0.065. Coarse retrieval pulls in more text that is not gold, so it suits a pipeline where a reader or reranker trims context.

Method

Results

Retrieval quality by policy and model
Fine vs coarse retrieval, by model

DeepSeek V4.1 Flash, 140 questions

policysnippet hitchar recallchar precisionhopsp50p95cost
fine, 1 passage per hop0.4690.4230.1202.848.93s50.1s$0.0630
coarse, 4 passages per hop0.5750.5470.0651.514.17s30.8s$0.0467

Coarse minus fine: snippet hit +0.106 (95% CI +0.046 to +0.166); character recall +0.124 (95% CI +0.060 to +0.186); character precision -0.055 (95% CI -0.082 to -0.031).

MiMo V2.6 Flash, 100 questions

policysnippet hitchar recallchar precisionhopsp50p95cost
fine, 1 passage per hop0.5090.4410.1292.9015.72s34.2s$0.0608
coarse, 4 passages per hop0.5620.5350.0641.598.58s16.6s$0.0501

Coarse minus fine: snippet hit +0.053 (95% CI -0.015 to +0.123); character recall +0.094 (95% CI +0.023 to +0.166); character precision -0.065 (95% CI -0.100 to -0.034).

Why it works, and what it costs

Latency distribution
Coarse finishes sooner across the whole distribution
Hops distribution
Coarse usually answers in one or two hops
Recall precision trade-off
Recall gained, precision spent
Hit by source
The gain holds on all three sources

By source (DeepSeek)

sourcequestionshit, finehit, coarseprecision, fineprecision, coarse
CUAD contracts600.3990.5230.0870.048
ContractNLI NDAs500.5270.5970.1240.058
PrivacyQA policies300.5140.6450.1770.111

Where coarse wins on one question

Consider the Non-Disclosure Agreement between Stony Hill and Buyer; Does the document specify whether the Receiving Party is required to destroy or return Confidential Information upon the termination of the Agreement?
fine hit 0.00 in 1 hops, coarse hit 1.00 in 2 hop(s)

Consider Seeed's Non-Disclosure Agreement; Does the document state that Confidential Information shall only include technical information?
fine hit 0.00 in 1 hops, coarse hit 0.50 in 1 hop(s)

Consider Kenway's Non-Disclosure Agreement; Does the document indicate that the Agreement does not grant the Receiving Party any rights to the Confidential Information?
fine hit 0.00 in 4 hops, coarse hit 1.00 in 1 hop(s)

What this does and does not show

Related: the same system on HotpotQA and MuSiQue, with the cache study, is in the benchmark report.