`huggingface://`¶

Hugging Face Inference API — hosted inference for any model on the Hub. Uses Hugging Face's OpenAI-compatible chat-completions surface — same wire shape as openai:// except the model name lives in the URL path rather than the request body.

URI shape¶

huggingface://chat?model=<repo>&apiKey=<token>&temperature=<float>&maxTokens=<int>&baseURL=<url>

Parameters¶

Param	Type	Default	Producer/Consumer/Both	Description
`model`	string	`meta-llama/Meta-Llama-3.1-70B-Instruct`	Producer	Hugging Face Hub repo id (e.g. `mistralai/Mistral-7B-Instruct-v0.3`). Must be a chat-enabled model with text-generation-inference support.
`apiKey`	string	(from constructor)	Producer	Hugging Face access token. Sent as `Authorization: Bearer <token>`.
`baseURL`	string	`https://api-inference.huggingface.co`	Producer	Base URL. Override for Inference Endpoints (dedicated deployments).
`temperature`	float	`0.7`	Producer	Sampling temperature.
`maxTokens`	int	`4096`	Producer	Output cap.

Operation defaults to chat; embeddings is not implemented on this scheme. Producer-only. Default timeout 180 s (Inference API can pay a model-warm-up tax on cold calls).

Headers¶

Reads systemPrompt and prepends it as a system message. Sets namespaced response headers: huggingface.model, huggingface.id, huggingface.finishReason, huggingface.usage.promptTokens, huggingface.usage.completionTokens, huggingface.usage.totalTokens.

Examples¶

Llama 3.1 70B via the public Inference API:

- to: huggingface://chat?model=meta-llama/Meta-Llama-3.1-70B-Instruct&apiKey=${HF_API_KEY}

A dedicated Inference Endpoint:

- to: huggingface://chat?model=my-org/my-finetune&baseURL=https://abc123.us-east-1.aws.endpoints.huggingface.cloud&apiKey=${HF_API_KEY}

Behavior notes¶

URL path encodes the model. The endpoint pattern is {baseURL}/models/{model}/v1/chat/completions. The model field also appears in the request body for OpenAI compatibility.
Cold start. First requests to a serverless model can take 20–60 s while HF spins up the worker; the 180 s timeout is sized to cover this. Subsequent calls are fast.
Not every Hub model works — only models with text-generation-inference compatibility expose the chat API.
All other behaviour matches openai://.

Source¶

pkg/component/huggingface/huggingface.go
pkg/component/huggingface/huggingface_test.go

huggingface://¶