`replicate://`¶

Replicate — hosted open-source models behind a prediction-job API rather than a chat-completions API. Helix creates a prediction and polls until the job completes, then returns the joined output. This is structurally different from every other LLM scheme in Helix — Replicate isn't OpenAI-compatible.

URI shape¶

replicate://chat?model=<owner/name>&apiKey=<token>&temperature=<float>&maxTokens=<int>&baseURL=<url>

Parameters¶

Param	Type	Default	Producer/Consumer/Both	Description
`model`	string	`meta/meta-llama-3.1-405b-instruct`	Producer	Replicate model id in `<owner>/<name>` form. See https://replicate.com/explore.
`apiKey`	string	(from constructor)	Producer	Replicate API token. Sent as `Authorization: Bearer <token>`.
`baseURL`	string	`https://api.replicate.com/v1`	Producer	Override for proxies.
`temperature`	float	`0.7`	Producer	Sampling temperature; passed in the model's `input` block.
`maxTokens`	int	`4096`	Producer	Output cap (`max_tokens`).

Operation defaults to chat. Producer-only. Default timeout 300 s because predictions can take a long time.

Headers¶

Reads:

Header	Effect
`systemPrompt`	Concatenated into the prompt with a `<system>...</system>` wrapper, then `User: ...\nAssistant:` template, because Replicate's API expects a single `prompt` string rather than messages.

Sets:

Header	Description
`replicate.model`	Replicate model id used.
`replicate.id`	Prediction id (`r-...`).
`replicate.finishReason`	Final prediction status (`succeeded`, `failed`, `canceled`).
`replicate.usage.inputTokens` / `outputTokens` / `totalTokens`	Token counts from the prediction `metrics`, when the model reports them.

Examples¶

- to: replicate://chat?model=meta/meta-llama-3.1-405b-instruct&apiKey=${REPLICATE_API_TOKEN}

Mistral via Replicate:

- to: replicate://chat?model=mistralai/mistral-7b-instruct-v0.3&apiKey=${REPLICATE_API_TOKEN}&maxTokens=512

Behavior notes¶

Create + poll. The producer first POSTs to {baseURL}/models/{owner}/{name}/predictions with Prefer: wait (which asks Replicate to inline-complete quick predictions). If the response status is starting or processing, it polls prediction.urls.get every 1 s until status reaches succeeded, failed, or canceled.
Prompt template. The chat prompt is User: {body}\nAssistant:, optionally prefixed with <system>{systemPrompt}</system>\n\n. This is the convention most chat-tuned Replicate models expect.
Output normalisation. Replicate models return outputs as either a string or a []string (token streams). Helix joins arrays with no separator and converts other shapes via fmt.Sprintf("%v", ...).
300 s timeout at the HTTP-client level; long predictions still get an opportunity to complete via the poll loop.
No streaming, no embeddings, no native messages array on this scheme.

Source¶

pkg/component/replicate/replicate.go
pkg/component/replicate/replicate_test.go

replicate://¶