SAADI

Understand Once, Retrieve Many Times: Stateful Accumulation of Document Information for Single‑Pass Retrieval of Distributed Evidence in Long Documents

Zeinab Sadat Taghavi^1,2
Lars Klein¹
Mary-Anne Hartley¹
Andreas Marfurt²

¹EPFL · ²HSLU · Switzerland

Paper Code BibTeX

SAADI reads the document once, in order.

SAADI reads each long document once, building entity memory and summaries, so a single-pass retriever can find distributed evidence with no LLM calls at query time.

16 of 20

Agent comparisons in which SAADI is ahead at matched evidence exposure

+12.4 points

Largest HR@10 gain over the base retriever · +0.3 to +12.4 across 20 settings

0 calls

LLM calls at query time · at most 1.25 calls per chunk, once, at index time

Read the abstract

Long-document retrieval is challenging when relevant evidence is spread across distant passages or multiple hops, especially under a limited retrieval budget. Single-pass top-k retrievers have predictable compute and latency at query time, whereas agentic systems pursue higher recall by reading a document over several steps. We move progressive document understanding to index time, where a single-pass top-k retriever can use it without query-time large language model (LLM) calls or adaptive trajectories.

SAADI (Stateful Accumulation of Document Information) builds a query-agnostic document state at index time with one state-update call per chunk. Each call extracts the chunk's entities, conditioned on records recalled from the entity memory and on a hierarchical summary of the prefix. The chunk is linked to the resulting records and to the summary covering it, and the original chunks remain the evidence and retrieval units. At query time, two auxiliary scores are computed from a chunk's linked records and its covering summary and fused with a sparse or dense base retriever's score in a single top-k pass. Index-time LLM calls, including summary folds, are fixed before any query arrives, at most 1.25 per chunk in the default configuration. Each call reads a context that grows only logarithmically with the document, and appending chunks extends the state without rebuilding it.

Across four datasets and five retrievers, SAADI improves HR@10 over its base retriever by 0.3 to 12.4 points (1.1 to 4.0 under BM25). It ranks first or second in 34 of 40 dataset, retriever, and depth cells and, at comparable average evidence exposure, is ahead of agentic retrievers in sixteen of twenty pairs. The gains persist to 256K-token contexts and at every evidence-hop count. Agentic costs recur with every query. After a break-even number of queries per document, which we report in model calls and in tokens, they exceed SAADI's one-time construction cost. Ablations show that the entity-memory and summary signals each improve retrieval, and that fusing both is best in every setting. Thus document-level connections behind distributed and multi-hop evidence can be established once at index time, at a cost known before the first query.

The connection is in the document

But not always in the passage. A question names Alyona Ivanovna; the passage that answers it calls her only “the old woman.”

“…the fourth floor on this staircase would be untenanted except by the old woman.”
An earlier passage

“How much will you give me for the watch, Alyona Ivanovna?”
A later passage

SAADI connects the two while it reads, before any question arrives.

01 · Index time

Understanding grows as SAADI reads

One pass builds an entity memory and a hierarchical history.

Document: Crime and Punishment · DostoevskyC1

Current passage C1

A few words about Dostoevsky himself may help the English reader to understand his work. … and so the fourth floor on this staircase would be untenanted except by the old woman.

Original text remains unchanged

Entity memory Empty

Record / old woman

Tenant of a flat on the fourth floor.

Alias → Alyona Ivanovna

← Recall relevant records

Hierarchical history Recent chunks

The latest chunk enters the history.

Illustrative sequence using selected excerpts; the summary hierarchy follows the folding procedure.

Step 1

Begin with an empty memory. Read the first chunk. Extract names, aliases and descriptions into reusable entity records.

Step 2

Keep more than a name. Dostoevsky and the old woman become records with descriptions. A single chunk can update several entities.

Step 3

Recall. Connect. Update. Recall the old woman while reading the next passage. Add “Alyona Ivanovna” as an alias to the same record.

Step 4

Five chunks. One summary. Fold five consecutive chunks into an L1 summary. Keep reading, recalling memory and extending the history.

Step 5

Five L1 summaries become L2. Fold again at the next level. Each update uses recent chunks together with earlier summaries at different levels.

Step 6

Five L2 summaries become L3. The same rule repeats. Memory keeps the entities; the hierarchy compresses the document’s earlier context.

02 · Query time

The same passage, a better connection

Three signals are fused in one ranking over the original chunks, with no model call.

Question

Who was Alyona Ivanovna?

Original text

“…except by the old woman.”

Linked memory

old woman
↔ Alyona Ivanovna

Covering L1 summary

Raskolnikov visits the old woman, Alyona Ivanovna, to pawn a watch.

Normalize & fuse

Passage rankingText signal

Top k

Original passage · C1

“…untenanted except by the old woman.”

Illustrative query and ranking movement, not measured ranks. The passage is never rewritten.

Step 1

Start with the passage text. The question names Alyona, but the earlier passage refers to her only as “the old woman.”

Step 2

Add the memory connection. The linked record connects “old woman” with “Alyona Ivanovna,” adding a signal to the passage’s score.

Step 3

Add the summary context. The covering L1 summary adds context. Fused signals lift the original passage into the top-k set.

How SAADI works

An index-time pass builds a query-agnostic document state: an active-entity memory and a hierarchical history. At query time, scores derived from that state adjust a single ranking over the original chunks.

SAADI pipeline: index each document once to build entity memory and hierarchical summaries; at query time, fuse text, memory and summary scores to return top-k original passages with zero LLM calls.

Overview of SAADI. (a) The offline pass. Each chunk in turn is interpreted by one model call that reads it together with the records recalled from the active-entity memory and the hierarchical history of the prefix, and every f steps the oldest units fold into one summary at each level. (b) Query time, where the text, memory, and summary scores are fused into one ranking over the original chunks, which are never rewritten and are returned without any model call.

Index time Built once per document, offline

1
Chunk

Split the document at sentence boundaries into fixed-length chunks. Chunks are never rewritten.
2
Recall

Score the chunk against stored names, aliases and descriptions with BM25, and recall a few records.
3
Update

One LLM call reads the chunk, the recalled records and the history, then adds or extends entity records.
4
Fold

Append the chunk to the history. Every f = 5 units at one level fold into one summary above.

Query time One pass, no model calls

1
Score three channels

The same base retriever scores each chunk’s text, its linked records and its covering L1 summary.
2
Normalize and fuse

Each channel is z-scored over the candidates and combined with weights fixed once.
3
Return original chunks

The top-k chunks come back as original text. No query rewriting, no trajectory, no LLM call.

$$S_{\text{SAADI}}(Q, C_i) = \lambda_T\,\hat S_{\text{text}}(Q, C_i) + \lambda_M\,\hat S_{\text{mem}}(Q, C_i) + \lambda_S\,\hat S_{\text{sum}}(Q, C_i)$$

Default weights $(\lambda_T, \lambda_M, \lambda_S) = (1, 0.5, 0.3)$, selected once on 100 queries and held fixed for every dataset and retriever.

Read the full method

A passage may refer to a character only as the old woman while her name appears pages away, or be relevant only because of what the surrounding section is about. That connecting information is in the document but not in the chunk. SAADI establishes it once, while reading the document in order, and makes it available to every query that follows.

Index time · built once per document, offline

Chunk. The document is segmented at sentence boundaries into non-overlapping chunks of a fixed token budget. Chunks are never rewritten, merged, or contextualized.
Recall. Before each chunk is processed, a bounded set of entity records is recalled from the active-entity memory by scoring the chunk against the stored names, aliases, and descriptions with BM25.
Update. One LLM call reads the chunk, the recalled records, and the hierarchical history, and returns the entities the chunk is about. The memory adds new records or enriches existing ones; nothing is removed. Chunks are linked to entity names, so an alias found later reaches every chunk linked to that entity.
Fold. The chunk is appended to the history. Whenever f consecutive units at one level are complete, they are replaced by one summary at the level above, so the context each call reads grows only logarithmically.

Query time · one pass, no model calls

Three channels, one retriever. Each chunk is scored by the same sparse or dense base retriever on its own text, on the entity records linked to it (field by field), and on the level-one summary covering it.
Normalize and fuse. The channels are z-scored over the candidate chunks and combined with weights fixed once for every dataset and retriever, so the text anchors the ranking and the auxiliary channels adjust it.
Return original chunks. The top-k chunks are returned as original text. Records and summaries change the order, not the objects ranked. No query rewriting, no adaptive trajectory, no LLM call.

Results

Four datasets, two of them multi-hop. Hit rate counts a query at depth k only when all of its evidence chunks are in the top k.

LooGLE

Long-document QA

QASPER-64K

Long-context scientific QA

MuSiQue-32K

Multi-hop

NovelHopQA

Multi-hop

5.1 · Retrieval effectiveness

Does SAADI improve the retriever it is built on?

Yes, in every setting: HR@10 rises by 0.3 to 12.4 points across four datasets and five retrievers.

Each cell shows the base retriever alone, then SAADI on the same retriever, with the gain in points. Darker chips mark gains of five points or more.

Retriever	LooGLE			MuSiQue-32K			QASPER-64K			NovelHopQA
	Base	SAADI	Δ	Base	SAADI	Δ	Base	SAADI	Δ	Base	SAADI	Δ
BM25	95.4	96.5	+1.1	74.0	77.7	+3.7	22.1	26.1	+4.0	20.0	22.0	+2.0
Contriever	73.3	85.7	+12.4	76.1	83.7	+7.6	23.3	25.8	+2.5	11.8	18.2	+6.4
BGE-M3	89.8	90.1	+0.3	87.4	88.3	+0.9	29.3	30.9	+1.6	20.7	22.1	+1.4
jina-embeddings-v3	80.1	82.0	+1.9	85.8	88.1	+2.3	24.8	27.2	+2.4	19.1	21.1	+2.0
Qwen3-Embedding-8B	85.9	86.7	+0.8	89.0	90.0	+1.0	32.6	34.1	+1.5	19.8	20.6	+0.8

HR@10 in percent · Table 1. Among fixed-depth methods on the same retriever, SAADI is first or second in 34 of 40 dataset, retriever, and depth cells and never below third. Full tables, including HR@5, are in the paper.

5.2 · Agentic retrieval

How does a single pass compare with agents?

SAADI is ahead in 16 of 20 matched comparisons, without a model call per query.

Agents decide how much to read, so each is compared with SAADI under BM25 at its matched evidence depth. Agents spend 2.0 to 4.2 model calls on every query.

Explore the curves ↓

Mean hit rate (%) across four datasets

AgentSAADI at matched depth

RAG-R1

47.8

51.4

Agentic-R

39.5

46.2

A-RAG

33.8

40.2

GRIP

40.5

45.6

UMA

54.9

57.2

Scale: 0 to 60%. Each pair uses its own matched depth, so the means do not describe a single fixed top-k setting · Table 2.

5.2 · Figures 2 and 3

Where do the gains hold?

At matched evidence exposure, as documents grow to 256K tokens, and at every hop count.

Compare datasets, follow the curves, and select methods to show or hide them.

Hit rate at comparable evidence exposure

Compare SAADI’s retrieval depth with the evidence admitted by agentic methods.

Hit rate (%)

Hover, tap or focus a point for its value. Select a method below to show or hide it.

Source: paper, Figure 2 and Tables 1, 22–23.Download chart data ↓

View plotted values

5.3 · Ablations

What does the document state contribute?

The history improves extraction; linked records and summaries each improve scoring, and fusing both is best.

Rebuilding the index under four conditioning variants, with query-time scoring held fixed, shows that the hierarchical history is what makes conditioned extraction improve the index. At query time the linked records and the covering summaries are both effective scoring signals. The construction model matters less than the retriever.

+3.0 points

Largest gain from conditioning extraction on the history · MuSiQue-32K

4 of 4

Datasets on which fusing records and summaries scores best, with each signal helping on its own

≤ 1.4 points

HR@10 change when the 30B construction model is replaced by a 7B, 24B or 70B model

Table 27 and Appendices C.3 and F.7.

Cost known before the first query

SAADI’s index-time calls are fixed by the chunk count N and the folding factor f. An agent pays again on every query.

SAADI pays once

One state-update call per chunk plus summary folds: fewer than 1.25 calls per chunk at f = 5, and no calls at query time.

Agents pay per query

An agent makes 2.0 to 4.2 model calls on every query. Past the break-even point Q★, it has spent more than SAADI’s one-time construction.

The number of index-time model calls is fixed by the chunk count N and the folding factor f. One state-update call is made per chunk, and a fold at level i once per fⁱ chunks:

$$T(N, f) = N + \sum_{i=1}^{\lfloor \log_f N \rfloor} \left\lfloor \frac{N}{f^{\,i}} \right\rfloor \;<\; \frac{f}{f-1}\,N$$

At f = 5 the hierarchy adds fewer than a quarter of a call per chunk. Appending a chunk costs one update plus at most $\lfloor \log_f (N{+}1) \rfloor$ folds, amortized to 1.25 calls, and the state is extended rather than rebuilt. Each call reads a context bounded by $(f-1)(\lfloor \log_f t \rfloor + 1)$ units.

An agent that averages $\bar c$ calls per query, with no index-time cost, reaches the same total after

$$Q^{\star} = \frac{T(N, f)}{\bar c} \;\approx\; \frac{f}{f-1} \cdot \frac{N}{\bar c}$$

queries per document. The same accounting covers token usage and methods that also pay an index-time cost.

Break-even against agentic retrieval

Queries per document at which an agent’s cumulative pre-answer model calls reach SAADI’s one-time construction cost. Ranges span the five agents; UMA’s own offline memory construction is subtracted first.

Dataset	Agent calls / query	Break-even (queries / doc)
LooGLE	2.0 to 2.7	11 to 17
MuSiQue-32K	2.5 to 4.2	114 to 192
NovelHopQA	2.1 to 2.7	169 to 256
QASPER-64K	2.0 to 3.3	279 to 457

A deployment expecting fewer queries per document than the break-even point is cheaper served by the agent, and one expecting more by SAADI. Measured in generated tokens, the break-even points are about an order of magnitude larger · Table 2.

BibTeX

@article{taghavi2026saadi,
  title   = {Understand Once, Retrieve Many Times: {SAADI} for Single-Pass
             Retrieval of Distributed Evidence in Long Documents},
  author  = {Taghavi, Zeinab Sadat and Klein, Lars and Hartley, Mary-Anne
             and Marfurt, Andreas},
  journal = {Preprint},
  year    = {2026},
  url     = {https://github.com/ZeinabTaghavi/SAADI}
}