Runs a text-generation request against a local or cloud model and returns the completion, the reason generation stopped, and the provider-reported prompt and completion token counts. The request controls the maximum completion tokens, sampling temperature, top-k and top-p cutoffs, an optional deterministic seed, string and token-id stop sequences, and an optional GBNF grammar for constrained generation. Chat models expect their chat template already applied in the prompt. Local models require a build with the local execution feature; cloud providers require the matching provider feature and an API key.

Parameters

Name	Type	Required	Description
model	string	yes	Model spec.
frequency_penalty	number	no	Frequency penalty.
grammar	string	no	GBNF grammar for constrained generation (local).
logit_bias	object	no	Per-token logit bias (token id → bias).
logprobs	boolean	no	Whether to return log-probabilities.
max_tokens	integer	no	Maximum completion tokens.
messages	ChatMessage[]	no	Chat messages (system/user/assistant/tool).
min_p	number	no	Min-p sampling cutoff.
mirostat	Mirostat	no	Mirostat sampling.
model_config	ModelConfig	no	Per-model load/context configuration.
presence_penalty	number	no	Presence penalty.
prompt	string	no	Raw completion prompt (base models / full control).
repeat_last_n	integer	no	Repetition penalty look-back window.
repeat_penalty	number	no	Repetition penalty.
response_format	ResponseFormat	no	Output format constraint.
seed	integer	no	Deterministic sampling seed.
stop	string[]	no	Stop sequences.
stop_token_ids	integer[]	no	Token-id stop sequences (local).
temperature	number	no	Sampling temperature.
tfs_z	number	no	Tail-free sampling z.
tool_choice	ToolChoice	no	How the model should choose among tools.
tools	Tool[]	no	Tools (functions) the model may call.
top_k	integer	no	Top-k sampling cutoff.
top_logprobs	integer	no	Number of top log-probabilities to return per token.
top_p	number	no	Nucleus sampling cutoff.
typical_p	number	no	Typical-p (locally typical) sampling.

Returns

ChatResponse.

Errors

• failed_precondition.engine.runtime_closed
• not_found.engine.branch
• inference.unsupported_operation
• inference.missing_model
• inference.model_load_failed
• inference.local_runtime_failed
• inference.missing_api_key
• inference.provider_auth_failed
• inference.provider_unavailable
• inference.provider_timeout
• inference.provider_rate_limited
• inference.invalid_request
• inference.provider_malformed_response
• inference.unsupported_provider
• inference.unsupported_parameter
• inference.registry_corrupt

Invocation

• CLI: strata inference generate
• Wire type: inference_generate