inference
generate
Generate text with an inference model.
strata inference generatedb.ai.chat(messages: 'Union[str, Iterable[Mapping[str, Any]], None]' = None, *, model: 'str', prompt: 'Optional[str]' = None, response_format: 'Union[str, Mapping[str, Any], None]' = None, json_schema: 'Optional[Mapping[str, Any]]' = None, **knobs: 'Any') -> 'ChatCompletion'wire inference_generate
Runs a text-generation request against a local or cloud model and returns the completion, the reason generation stopped, and the provider-reported prompt and completion token counts. The request controls the maximum completion tokens, sampling temperature, top-k and top-p cutoffs, an optional deterministic seed, string and token-id stop sequences, and an optional GBNF grammar for constrained generation. Chat models expect their chat template already applied in the prompt. Local models require a build with the local execution feature; cloud providers require the matching provider feature and an API key.
To see the complete help for this command, run:
strata inference generate --helphelp(db.ai.chat)
Example
No CLI example in this release.
db.ai.chat("Summarize Strata in one line.",schema = {"type": "object", "properties": {"city": {"type": "string"}}}db.ai.chat("Where is the Eiffel Tower?",tools = [{"type": "function",db.ai.chat("Weather in Paris?",
Parameters
| Name | Type | Required | Description |
|---|---|---|---|
| model | string | yes | Model spec. |
| request | ChatRequest | yes | Generation request. |
Returns
ChatResponse
- dataChatResponse
- typestring
Errors
Recover by code. Retry policy and commit outcome come from the shipped error registry.
| Code | Retry | Commit outcome |
|---|---|---|
| failed_precondition.engine.runtime_closed | never | not_started |
| not_found.engine.branch | never | not_applicable |
| inference.unsupported_operation | never | not_applicable |
| inference.missing_model | after_state_change | not_applicable |
| inference.model_load_failed | after_state_change | not_applicable |
| inference.local_runtime_failed | unknown | not_applicable |
| inference.missing_api_key | after_state_change | not_applicable |
| inference.provider_auth_failed | after_state_change | not_applicable |
| inference.provider_unavailable | same_request | not_applicable |
| inference.provider_timeout | same_request | not_applicable |
| inference.provider_rate_limited | after_state_change | not_applicable |
| inference.invalid_request | never | not_applicable |
| inference.provider_malformed_response | unknown | not_applicable |
| inference.unsupported_provider | never | not_applicable |
| inference.unsupported_parameter | never | not_applicable |
| inference.registry_corrupt | never | not_applicable |