RAG raptor¶

RAPTOR (Recursive Abstractive Processing for Tree-Organized Retrieval) builds a tree on top of leaf chunks: leaves are embedded, clustered, and each cluster is summarised by an LLM into a parent node; the process recurses on summaries until a root is reached. Every node at every level is returned and indexed, enabling "collapsed tree" retrieval where a query can match a fine-grained leaf or a high-level summary in the same vector store.

RAPTOR composes existing Helix primitives: an embed.Embedder for clustering and an llm.ChatModel for summarisation. Wrap the model with cache.NewCachingChatModel so repeated cluster summaries reuse prior LLM calls. It plugs between rag-chunk and rag-embed: the level-tagged []*rag.Chunk it emits feeds straight into the rest of the pipeline.

Implementation¶

`raptor` — recursive cluster-summary tree builder¶

rag:
  raptor:
    embedder: ${EMBEDDER_REF}
    model: ${LLM_REF}
    max_levels: 3
    cluster_size: 5
    summary_prompt: "..."   # optional override
    seed: 1

Field	Type	Default	Description
`embedder`	ref	—	Required. `embed.Embedder` used for clustering.
`model`	ref	—	Required. `llm.ChatModel` used for cluster summarisation.
`max_levels`	int	3	Maximum summary levels above the leaves.
`cluster_size`	int	5	Target avg nodes per cluster; `k = ceil(n / cluster_size)`.
`summary_prompt`	string	`DefaultSummaryPrompt`	System prompt for summarisation.
`seed`	int64	1	Clustering RNG seed for determinism.

Constructor: raptor.New(cfg) (*Builder, error). Build the tree with builder.Build(ctx, leaves). Returns all leaves + summary nodes, ordered leaves-first then ascending levels, each tagged with:

Metadata	Value
`chunk_level`	`"0"` for leaves, `"1"`, `"2"`, … for summary levels (matches `HierarchicalChunker`)
`raptor`	`"leaf"` or `"summary"`
`children`	comma-separated child chunk IDs (summary nodes only)

Per level the builder:

Embeds the current level's nodes (Embedder.Embed).
Computes k = ceil(n / ClusterSize) clamped so clustering strictly reduces.
K-means clusters embeddings with 50 iterations and the seed.
For each non-empty cluster: concatenates members and prompts the LLM with summary_prompt for an information-dense summary.

Source: pkg/rag/raptor/raptor.go, pkg/rag/raptor/cluster.go.

When to use RAPTOR¶

RAPTOR's tree shape pays off when:

The corpus is large enough that a flat top-K retrieval misses the big picture (queries like "what does this codebase do?").
Documents are heterogeneous (mixed topics, mixed depth) — summary nodes give the retriever something to match on for high-level queries that no single leaf chunk answers.
You want one index serving both detailed and high-level questions without running separate pipelines.

It does not help when:

The corpus is small (the tree adds latency without much benefit).
All queries are concrete fact lookups (leaves already win).

How to wire it¶

The typical pipeline:

rag-ingest -> rag-chunk -> rag-raptor -> rag-embed -> rag-store

rag-raptor runs after chunking. Its output is the leaves plus all summary nodes — feed the union into the embed stage and the store. At query time, regular simple vector retrieval already returns both levels (because they're all in one store, tagged by chunk_level). For more sophisticated routing, combine with parent or auto_merging retrievers using the chunk_level metadata.

Cost notes¶

A RAPTOR build over n leaves with cluster size c and max levels L:

Embedding cost: n for the leaves, plus ~n/c + n/c^2 + ... (geometric series, ~n/(c-1)) for the summaries.
LLM summarisation cost: same as the summary count above.
Wrap the model in cache.NewCachingChatModel so re-runs reuse prior summaries (this is the canonical use case for the chat-model cache).