Inference commands

cache_status

Report loaded model cache state.

Reports the runtime model cache as three lists of specs: the generation, embedding, and ranking models currently loaded in memory. Use it to check what is resident before generating, or to confirm that `inference unload` freed the models you expected. The lists reflect only in-memory engines, not models available on disk.

To see the complete help for this command, run:

strata inference cache-status --help

Example

strata inference cache-statusgeneration_models  -embedding_models   -ranking_models     -

db.ai.cache_status()["generation_models"]

capability

Report capabilities for a model spec.

Parses a model spec into a provider and model name and reports what that combination supports without running the model. The result states whether generation, tokenization, embedding, and ranking are available, whether the operation requires network access or an API key, whether this binary was compiled with the provider feature needed to execute, whether the runtime currently permits network calls, and the known embedding dimension. Model specs are catalog names (`tinyllama`), catalog `name:quant` pairs (`tinyllama:q8_0`), local GGUF paths, or provider specs (`anthropic:claude-...`).

To see the complete help for this command, run:

strata inference capability --help

Example

strata inference capability openai:gpt-4o-mini  # Pure metadata - no request is sent to the provider.provider                  openaimodel                     gpt-4o-miniavailability              readycan_generate              truecan_embed                 truecan_rank                  falsecan_tokenize              falseembedding_dim             0requires_api_key          truerequires_network          truenetwork_enabled           trueprovider_feature_enabled  truesupports_tools            truesupports_json_object      truesupports_json_schema      truesupports_logprobs         true

db.ai.capability("openai:gpt-4o-mini")["provider"]

detokenize

Detokenize token ids with a local model.

Decodes an ordered list of token ids back into text using a local model's vocabulary, returning the reconstructed string. Detokenization is a local-only operation: it requires a build with the local execution feature and returns `inference.unsupported_operation` for cloud provider specs.

To see the complete help for this command, run:

strata inference detokenize --help

embed

Embed one or more texts into vectors.

Embeds text with an embedding-capable model and returns one vector per input, in order. The `input` field takes either a single string or an array of strings, so single and batch embedding share one command. The vector dimension is fixed by the model. Local embedding models require a build with the local execution feature; cloud embedding providers (OpenAI, Google) require the matching provider feature and an API key.

To see the complete help for this command, run:

strata inference embed --help

Example

db.ai.embed(["hello", "world"],

generate

Generate text with an inference model.

Runs a text-generation request against a local or cloud model and returns the completion, the reason generation stopped, and the provider-reported prompt and completion token counts. The request controls the maximum completion tokens, sampling temperature, top-k and top-p cutoffs, an optional deterministic seed, string and token-id stop sequences, and an optional GBNF grammar for constrained generation. Chat models expect their chat template already applied in the prompt. Local models require a build with the local execution feature; cloud providers require the matching provider feature and an API key.

To see the complete help for this command, run:

strata inference generate --help

Example

db.ai.chat("Summarize Strata in one line.",schema = {"type": "object", "properties": {"city": {"type": "string"}}}db.ai.chat("Where is the Eiffel Tower?",tools = [{"type": "function",db.ai.chat("Weather in Paris?",

rank

Rank passages against a query.

Scores each candidate passage against a query with a ranking model and returns one outcome per passage. Each item carries the passage's original index and either a relevance score or a per-item error with a stable code and a redacted message, so callers can reorder passages by score while keeping them tied to their inputs. Ranking is a local-only operation: it requires a build with the local execution feature and a ranking-capable model.

To see the complete help for this command, run:

strata inference rank --help

status

Report what this binary can do before anything is attempted.

Reports the inference facts that are knowable up front: whether this build can execute local models and download them, which providers are compiled in, which of those have an API key and where it was found, and how many catalogued models are already on disk.

To see the complete help for this command, run:

strata inference status --help

Example

strata inference status  # reports build, providers, keys, and on-disk models without attempting a callbuild	cloud providers only	local models: this build runs cloud models only, and a bare model name means a local model. Either name a cloud model instead - `openai:<model>`, `google:<model>` or `anthropic:<model>` - or run `strata inference install-local` to add local execution.providers  openai	no key -- `strata config set openai.api_key <key>`, or export OPENAI_API_KEY  anthropic	no key -- `strata config set anthropic.api_key <key>`, or export ANTHROPIC_API_KEY  google	no key -- `strata config set google.api_key <key>`, or export GOOGLE_API_KEY  local	not in this buildmodels	…

{p["provider"] for p in db.ai.status()["providers"]} >= {"openai", "anthropic"}isinstance(db.ai.status()["local_execution"], bool)(db.ai.status()["config_file"] or {"state": "absent"})["state"] in (

tokenize

Tokenize text with a local model.

Encodes text into the token id sequence a local model would see and returns the ids in order. Set `add_special` to include the model's special tokens (such as beginning-of-sequence markers). Tokenization is a local-only operation: it requires a build with the local execution feature and returns `inference.unsupported_operation` for cloud provider specs.

To see the complete help for this command, run:

strata inference tokenize --help

unload

Unload cached inference models.

Removes cached model engines from the runtime to free memory. Pass a model spec to unload one entry, or omit it to unload every cached generation, embedding, and ranking model. The result reports whether any cached entry was actually removed. This affects only the in-memory runtime cache; it never deletes downloaded model files from disk.

To see the complete help for this command, run:

strata inference unload --help

Example

strata inference unloadno cached entry

db.ai.unload()["unloaded"]