RAG store¶

A vector store persists *rag.Embedding vectors (with their original content + metadata) and serves top-K cosine-similarity search at query time. The store sits between the embed stage (writes) and the retrieve stage (reads). All implementations satisfy the store.VectorStore interface:

type VectorStore interface {
    Upsert(ctx, embeddings []*rag.Embedding) error
    Search(ctx, vector []float32, topK int) ([]*rag.SearchResult, error)
    Delete(ctx, ids []string) error
    Collection() string
}

Helix ships thirteen backends: two in-process (InMemoryStore, MemoryStore) and eleven remote (Pinecone, Qdrant, Milvus, Chroma, Elastic, OpenSearch, Weaviate, pgvector, PostGIS, Redis Vector, memx), plus an in-process registry that lets ingest and retrieve pipelines share an unnamed default in-memory store. The HTTP-backed stores hand-roll their REST calls (no per-vendor SDK), so component compile times stay small.

One backend, postgis, adds a geospatial capability on top of vector search (store.SpatialSearcher): it co-locates a PostGIS geometry column with the pgvector embeddings so a single query fuses ANN similarity with a spatial filter. See postgis and the geospatial rag-retrieve filter.

Implementations¶

memory — slice-backed in-memory store¶

rag:
  store:
    type: memory
    collection: kb

InMemoryStore (in store.go) holds a []*rag.Embedding and performs brute-force cosine scan. Process-global via store.GetOrCreateInMemoryStore(name) — rag-store, rag-retrieve, and the memvec:// component all resolve store: memory (and the memvec:// URI) through this single registry, so an ingest pipeline writing to kb and a retrieve pipeline reading from kb share data without an external service. The registry's mechanics are in pkg/rag/store/registry.go. Source: pkg/rag/store/store.go, pkg/rag/store/registry.go, pkg/dsl/reifier.go.

memx-memory — map-backed in-memory store¶

rag:
  store:
    type: memx-memory
    collection: kb

MemoryStore (in memory.go) is a more performant variant using a map[string]*memoryEntry for O(1) ID lookup. Defensive metadata + vector copying prevents aliasing. Same interface, suitable for collections up to ~100K vectors. Source: pkg/rag/store/memory.go.

pinecone — Pinecone serverless¶

rag:
  store:
    type: pinecone
    host: https://my-index-abc123.svc.pinecone.io
    api_key: ${PINECONE_API_KEY}
    namespace: prod

REST-only client (/vectors/upsert, /query, /vectors/delete). The namespace doubles as the Collection() identifier. Use when you want a managed serverless index with per-namespace isolation. Source: pkg/rag/store/pinecone.go.

qdrant — Qdrant REST¶

rag:
  store:
    type: qdrant
    url: http://localhost:6333
    collection: kb
    api_key: ${QDRANT_API_KEY}

Free-form ChunkIDs like paul-graham-essay-chunk-0 are mapped to deterministic UUID-v5 point IDs (Qdrant only accepts uint64 or UUID). The original ChunkID round-trips via the payload's chunk_id field. Use for self-hosted high-performance HNSW with rich filtering. Source: pkg/rag/store/qdrant.go.

milvus — Milvus v2 REST¶

rag:
  store:
    type: milvus
    address: localhost:19530
    collection: kb
    dimension: 1536
    token: ${MILVUS_TOKEN}

Talks to the v2 REST API. The dimension must match the embedder. Auto-creates the collection on first Upsert if missing. Source: pkg/rag/store/milvus.go.

chromadb — ChromaDB REST¶

rag:
  store:
    type: chromadb
    base_url: http://localhost:8000
    collection: kb
    tenant: default_tenant
    database: default_database

Requires Init(ctx) to resolve the collection ID before first use. Source: pkg/rag/store/chromadb.go.

elasticsearch — Elasticsearch with dense_vector¶

rag:
  store:
    type: elasticsearch
    url: http://localhost:9200
    index: rag-kb
    username: elastic
    password: ${ES_PASSWORD}
    # or:
    # api_key: ${ES_API_KEY}

Index requires dense_vector field mapping (set by the caller, not the store). Use when you already run ES for log/document search and want vectors alongside. Source: pkg/rag/store/elasticsearch.go.

opensearch — OpenSearch k-NN¶

rag:
  store:
    type: opensearch
    base_url: http://localhost:9200
    index_name: rag-kb
    dimensions: 1536
    username: admin
    password: ${OS_PASSWORD}

Auto-creates the index with k-NN mappings on Init(ctx). The AWS managed OpenSearch variant of the ES integration. Source: pkg/rag/store/opensearch.go.

pgvector — PostgreSQL with pgvector extension¶

rag:
  store:
    type: pgvector
    conn_string: postgres://user:pass@localhost:5432/db
    table_name: vector_store
    dimensions: 1536

Uses pgx/v5 connection pooling. Init(ctx) creates the table with vector(dimensions) if absent. Best choice when you already operate Postgres and want one less system. Source: pkg/rag/store/pgvector.go.

postgis — PostgreSQL with pgvector + PostGIS (geospatial RAG)¶

rag:
  store:
    type: postgis
    conn_string: postgres://user:pass@localhost:5432/db
    table_name: geo_vector_store
    dimensions: 1536

postgis is pgvector plus a PostGIS geometry(Geometry, 4326) column and a GIST spatial index — one table holding both the embedding and the geometry, so a single query combines cosine similarity with a spatial predicate. Init(ctx) enables the vector and postgis extensions and creates the table + index.

Geometry is taken from each embedding's metadata: geo.geometry (WKT) if present, otherwise a POINT synthesized from geo.lon / geo.lat. Rows with no geo metadata store a NULL geometry and are excluded from spatial searches. The GeoJSON ingest reader (ingest.GeoJSONToDocuments) populates these keys per feature. Source: pkg/rag/store/postgis.go.

It implements the optional store.SpatialSearcher capability. When a rag-retrieve step carries a geospatial filter, retrieval is restricted to that region and fused with the vector query (radius via ST_DWithin, bbox via ST_MakeEnvelope); on a non-spatial store the same filter is ignored and the configured strategy runs unchanged:

# radius mode — within 5 km of (lat 40.7128, lon -74.006):
- rag-embed: { provider: openai, model: text-embedding-3-small }
- rag-retrieve:
    store: postgis
    connection: postgres://user:pass@localhost:5432/db
    collection: places
    strategy: simple
    top_k: 10
    geo_lat: 40.7128
    geo_lon: -74.006
    geo_radius_meters: 5000
# bounding-box mode — replace the three geo_* keys above with:
#   geo_bbox: [-74.1, 40.6, -73.9, 40.8]   # [minLon, minLat, maxLon, maxLat]

Requires a PostgreSQL with both the pgvector and postgis extensions available. Best choice when documents have a location and queries are "semantically similar and near here".

redis — Redis with RediSearch¶

rag:
  store:
    type: redis
    addr: localhost:6379
    password: ${REDIS_PASSWORD}
    db: 0
    index_name: vector_idx
    prefix: doc:
    dimensions: 1536
    algorithm: FLAT

Needs Redis Stack / RediSearch module. Init(ctx) creates the index. Source: pkg/rag/store/redis_vector.go.

weaviate — Weaviate REST¶

rag:
  store:
    type: weaviate
    url: http://localhost:8080
    collection: KB
    api_key: ${WEAVIATE_API_KEY}

Class must follow Weaviate naming (capitalised). Source: pkg/rag/store/weaviate.go.

memx — dual-mode memx (ES API + Postgres wire)¶

rag:
  store:
    type: memx
    conn_string: http://localhost:9000
    database: default
    table_name: memx_documents
    dimensions: 1536
    metric: cosine

ES-API mode is ~38% faster than SQL mode (bypasses parser). Use SQL mode when you need joins/analytics alongside vectors. Source: pkg/rag/store/memx.go.

Choosing between them¶

pgvector vs qdrant — if you already operate Postgres, pgvector is the simplest answer; performance is sufficient up to a few million vectors. Beyond that, or if you need per-collection isolation with high QPS, prefer Qdrant. Qdrant's metadata filtering is also richer.

pgvector vs postgis — use postgis when your documents have a location and you want to filter retrieval by distance or bounding box in the same query as the vector search (store locators, asset inspections, region-scoped knowledge). It needs the PostGIS extension; otherwise it is a drop-in superset of pgvector.

elasticsearch vs opensearch — pick the one that matches your existing search stack. Both are fully supported here; OpenSearch is the AWS-managed fork.

pinecone vs qdrant cloud — Pinecone has the smoother managed experience; Qdrant gives you the option to self-host the same database later.

See also¶

• embed — produces the *rag.Embedding records this stage stores.
• retrieve — queries this stage; some retrievers (e.g. auto_merging, parent) require two stores (child + parent).
• graph — GraphRAG uses a separate graph store, not these.
• RAG guide — end-to-end recipes.