`csv://`¶

Read and write delimited text (CSV / TSV / …) via Go's standard-library encoding/csv. Helix-native. Its headline feature is a streaming consumer that reads a file row-by-row and dispatches rows in bounded batches, so a multi-hundred-MB / million-row file flows in constant memory and never trips the 30 s single-eval cap that a whole-file read(payload,"text/csv") hits.

For the row-oriented, agent-driven counterpart see the csv.read_rows tool.

URI shape¶

csv:///abs/path/to/file.csv?[options]
csv://relative/path.csv?op=write
csv://?file=/tmp/out.csv&op=read&delimiter=tab

The file path can be in the URI path, the file= parameter, or the HelixCsvFile header (per-message override).

Parameters¶

Param	Type	Default	Description
`file`	string	—	Path to the file. Optional when supplied by the URI path or `HelixCsvFile`.
`op`	enum	`read`	Producer op: `read` / `write` / `append`.
`hasHeader`	boolean	`true`	Treat row 1 as the column-header row on read (keys for the output objects); emit one on write from the first body row's keys.
`delimiter`	string	`,`	Field separator. A single character, or a friendly name: `tab`, `comma`, `semicolon`, `pipe`, `space`.
`maxRows`	integer	`0`	Cap data rows on read (0 = unlimited).
`maxCols`	integer	`0`	Cap columns on read (0 = unlimited).

Consumer parameters (`from: csv://…`)¶

Param	Type	Default	Description
`delay`	duration	`30s`	Interval between polls.
`initialDelay`	duration	`1s`	Wait before the first poll.
`repeatCount`	integer	`0`	Cap on number of polls (0 = infinite).
`fireOn`	enum	`changed`	`changed` = dispatch only when the file's mtime advanced; `always` = every poll.
`splitRows`	boolean	`false`	Dispatch one exchange per row instead of one per file. Per-row body is `map[string]any` (hasHeader=true) or `[]string`.
`stream`	boolean	`false`	Stream the file row-by-row (`encoding/csv`) and dispatch rows in batches of `batch`, instead of materialising the whole file. Enables constant-memory reads of very large (1M+ row) files without tripping the 30 s single-eval cap.
`batch`	integer	`1000`	Rows per dispatched exchange when `stream=true` (0 → 1000). Larger batches mean fewer exchanges but more rows per downstream evaluation. `splitRows=true` forces one row per exchange.

Operations¶

op	Input `Body()`	Output `Body()`	Notes
`read` (default)	`nil`	`[]map[string]any` (hasHeader=true) or `[][]string`	Honors `maxRows`/`maxCols`. For huge files, prefer the streaming consumer.
`write`	`[]map[string]any` / `[][]any` / `[][]string` / `map[string]any` / `[]any`	unchanged (sets `HelixCsvFileWritten`)	Overwrites the file. Header row auto-emitted from the first row's keys when `hasHeader=true`.
`append`	same as `write`	same as `write`	Appends below the existing rows. Creates the file (like `write`) if absent.

Headers¶

Per-message overrides (In): HelixCsvFile, HelixCsvOp, HelixCsvHasHeader.

Out-headers (set on the In message): HelixCsvRowCount (rows read/written/appended, or rows in the current streamed batch), HelixCsvBatchIndex (0-based, streaming only), HelixCsvFileWritten (write/append).

Examples¶

Stream a very large file (1M+ rows, constant memory)¶

Reads row-by-row and hands downstream one batch at a time — each batch is its own evaluation, so it never hits the 30 s cap a whole-file read would:

workflows:
  - id: ingest-orders
    trigger:
      uri: "csv:///data/orders.csv?stream=true&batch=10000&fireOn=always&repeatCount=1"
    steps:
      - log:
          level: info
          showBody: false
          message: "batch ${header.HelixCsvBatchIndex}: ${header.HelixCsvRowCount} rows"
      - target: "sql://orders/insert?driver=postgres&dsn=${env:PG_DSN}&batch=true"

Read a small file mid-route¶

- target: "csv:///data/lookup.csv?op=read&delimiter=semicolon"
# body is now []map[string]any keyed by the header row

Write rows to a TSV¶

- set-body:
    starlang: '[ { id: "1", name: "Ada" }, { id: "2", name: "Alan" } ]'
- target: "csv:///tmp/out.tsv?op=write&delimiter=tab"

Validation¶

helix validate checks every csv:// URI's params against the schema (schemas/components/csv.json): unknown keys warn; an op outside [read, write, append] errors.

csv://¶