docling://¶

Document parser (PDF, DOCX, PPTX, HTML, image) that delegates to IBM / Red Hat's docling-serve. Producer-only — feed it bytes (or a URL via header), receive parsed markdown / JSON / HTML / doctags on the body.

Built on github.com/iguanesolutions/go-docling@v0.1.2. No hand-rolled multipart, no openapi codegen — the upstream Go client carries the typed surface (functional options, format enums, async task polling). We pin the version; if upstream goes stale we fork (replace github.com/iguanesolutions/go-docling => internal/docling).

URI shape¶

docling://<host>:<port>?<param>=<value>…
docling://?endpoint=http://docling:5001&<param>=<value>…

Either form works. The path-segment form is the canonical Helix style; the ?endpoint=… form is handy when the URL is templated.

When to use this vs. rag-ingest¶

Body shape¶

Request:

• []byte → uploaded as multipart files field, filename from HelixDoclingFilename header (defaults to input.pdf).
• string → treated identically to []byte.
• io.Reader → streamed into the multipart body.
• empty + HelixDoclingSourceURL header → docling-serve fetches the URL itself (avoids streaming large files through Helix).

Response:

• Body: rendered output as a string. The format matches output= (see below). For output=json, the body is a JSON string literal — parse via read(body()) in Starlang.

Parameters¶

Connection¶

Operation¶

Output¶

OCR & parsing¶

VLM pipeline (pipeline=vlm)¶

What it is. Standard PDF parsing assembles text from PDF stream ops + heuristic layout analysis. The VLM pipeline takes a different approach: docling renders each page to an image and feeds it to a multimodal LLM that outputs structured text (markdown / doctags / OTSL tables). This trades latency + cost for accuracy on tricky layouts — multi-column papers, scientific figures, handwritten notes, medical forms.

Three mutually-exclusive selectors. Pick one:

Precedence when multiple are set: local repo > external URL > built-in.

Selector params¶

Shared params (apply to whichever selector you picked)¶

Mode-specific params¶

Picture description (companion to doPictureDescription=true)¶

What it is. pipeline=vlm converts WHOLE PAGES via a VLM. Picture description is narrower — keep pipeline=standard, but ALSO caption each detected figure / chart / diagram inline. Caption text goes under the image in the markdown output, so downstream RAG retrieval can hit figures via their caption.

Why you'd turn it on: medical reports with embedded waveforms or imaging studies, scientific papers with figures whose alt-text matters, internal slide decks where the message lives in the chart.

Two mutually-exclusive captioners. API mode wins if both URL and RepoID are set:

Choosing between OCR, picture description, and full VLM pipeline¶

These three are easy to confuse — they target different problems:

You can combine ocr=true with pipeline=vlm (OCR runs first, VLM sees both the image AND the OCR text) — useful for scanned medical forms where dosage numbers must be preserved exactly.

Headers¶

Examples¶

1. Convert a PDF dropped into a watched folder¶

- id: ingest-pdfs
  from: file:///var/data/in?include=.*\.pdf$&delete=true
  steps:
    - to: docling://docling:5001?ocr=true&tableMode=accurate&output=md
    - to: log://?level=info

2. Use as an agent tool¶

tools:
  - id: parse_doc
    description: "Parse a PDF/DOCX/PPTX into markdown."
    input:
      url: { type: string, description: "Document URL", required: true }
    steps:
      - set-header:
          name: HelixDoclingSourceURL
          starlang: input.url
      - to: docling://docling:5001?ocr=true

3. Async conversion with task polling¶

- id: big-pdf
  from: direct://convert-bigpdf
  steps:
    # 1. Submit asynchronously.
    - to: docling://docling:5001?operation=convert-async&ocr=true
    # 2. HelixDoclingTaskID is now on the headers; persist + return.
    - transform:
        starlang: |
          %sl 2.0
          output application/json
          input headers
          ---
          { "task_id": headers["HelixDoclingTaskID"], "status": "submitted" }

4. Replace a hand-rolled HTTP call¶

Before — 7 lines of http:// + headers + remove-headers:

- remove-headers: { pattern: HelixHttp* }
- set-header: { name: Content-Type, constant: multipart/form-data }
- to: http://docling:5001/v1/convert/file?method=POST&timeout=600s
- transform:
    starlang-file: scripts/extract_markdown.sl

After — one line:

- to: docling://docling:5001?output=md&timeout=600s

5. Selective page extraction¶

- to: docling://docling:5001?pageRange=1-1&output=json

6. Built-in VLM pipeline (no external service)¶

Use docling's bundled granite_vision model to re-parse pages via a vision LLM. Best for complex multi-column layouts where heuristic parsing produces jumbled reading order.

- to: docling://docling:5001?pipeline=vlm&vlmPipelineModel=granite_vision&output=md&timeout=10m

docling-serve must be running with the granite-vision weights loaded (check its startup config / docker-compose env).

7. External VLM — delegate to GPT-4o for layout understanding¶

You don't run a VLM inside docling-serve; instead docling sends each page image to your OpenAI-compatible endpoint and uses the response. Cheaper to operate (no GPU on docling host) but pay-per-token at the VLM provider.

⚠️ Operator prerequisite — --enable-remote-services. docling-serve refuses all outbound VLM / picture-description calls by default as an SSRF safeguard. Helix will pass your URL through correctly, but docling will log Connections to remote services is only allowed when set explicitly and the worker will fail unless docling-serve was started with --enable-remote-services (CLI flag) or DOCLING_SERVE_ENABLE_REMOTE_SERVICES=true (env var). This is a docling-serve config — Helix can't override it client-side.

Same prerequisite applies to all external-API params: vlmPipelineModelURL, pictureDescriptionURL, and any SourceHTTP fetch with auth headers.

- to: |
    docling://docling:5001
    ?pipeline=vlm
    &vlmPipelineModelURL=https://api.openai.com/v1/chat/completions
    &vlmPipelineModelResponseFormat=markdown
    &vlmPipelineModelTimeout=120
    &vlmPipelineModelConcurrency=4
    &vlmPipelineModelScale=2.0
    &output=md
    &timeout=15m

URL-encode &s when inlining the URI. The cleaner form uses params::

- to: docling://docling:5001
  params:
    pipeline: vlm
    vlmPipelineModelURL: https://api.openai.com/v1/chat/completions
    vlmPipelineModelResponseFormat: markdown
    vlmPipelineModelTimeout: 120
    vlmPipelineModelConcurrency: 4
    vlmPipelineModelScale: 2.0
    output: md
    timeout: 15m

Authentication for the VLM API is configured on the docling-serve side (typically env vars like OPENAI_API_KEY mounted into the container) — Helix passes the URL only. Helix's apiKey= is for docling-serve's own auth, not for the downstream VLM.

8. Custom VLM prompt for clinical PDFs¶

Both the whole-page VLM pipeline AND the picture-description captioner take a prompt override. Useful for healthcare to preserve dosages and suppress hallucination. The prompt is shared local / API.

- to: docling://docling:5001
  params:
    pipeline: vlm
    vlmPipelineModel: granite_vision
    vlmPipelineModelResponseFormat: markdown
    vlmPipelineModelPrompt: |
      Convert this medical document page to clean Markdown.
      Rules:
      - Preserve every drug name, dosage, route, and frequency verbatim.
      - Render dosage tables as proper Markdown tables (no flattening).
      - Use Markdown headings for section titles (## for top-level).
      - Do NOT summarise. Do NOT paraphrase.
      - If a section is unreadable, write [UNREADABLE] in its place.
    output: md

9. Picture description (caption figures only, keep standard parsing)¶

Standard pipeline is fast and accurate for text — pay the VLM cost ONLY on detected figures. Captions land inline in the markdown so RAG retrieval can hit "figure showing waveform after treatment" even though the original PDF only had alt-less images.

- to: docling://docling:5001
  params:
    output: md
    doPictureDescription: "true"
    pictureDescriptionURL: https://api.openai.com/v1/chat/completions
    pictureDescriptionPrompt: |
      Describe the clinical content of this image in 1-2 sentences.
      Avoid speculation. Write "unreadable" if blurred.
    pictureDescriptionAreaThreshold: "0.05"   # skip tiny icons / logos
    pictureDescriptionTimeout: "30"
    pictureDescriptionConcurrency: "4"

10. Combine OCR + VLM (scanned forms with critical numbers)¶

For scanned medical forms where the layout is complex AND dosage numbers must be exact, run BOTH: OCR extracts text precisely, VLM reconstructs the page structure with that text as grounding.

- to: docling://docling:5001
  params:
    pipeline: vlm
    vlmPipelineModel: granite_vision
    vlmPipelineModelResponseFormat: markdown
    vlmPipelineModelPrompt: "Preserve all numbers exactly. Use OCR text where present; do not re-OCR."
    ocr: "true"
    forceOCR: "true"
    ocrEngine: tesseract
    ocrLang: "en"
    output: md
    timeout: 20m

Errors¶

The 200-with-failure-status case is treated as an error so route-level onException blocks can catch it; the rendered md_content (often empty in failure cases) is still written to the body for diagnostics, and HelixDoclingErrors carries the per-component messages.

Source¶

• Component: pkg/component/docling/
• Upstream client: github.com/iguanesolutions/go-docling pinned to v0.1.2
• Vendor strategy: pin the version; fork into internal/docling if upstream stalls.

See also¶

• rag-ingest — RAG-pipeline reader (backend: docling)
• http:// — the underlying HTTP transport docling-serve speaks