Inference commands
11 commands. Each entry gives the invocation and an example, in the interface you pick; open the details for parameters, return shape and error codes.
cache_status
Report loaded model cache state.
strata inference cache-status
db.ai.cache_status() -> 'Any'
Reports the runtime model cache as three lists of specs: the generation, embedding, and ranking models currently loaded in memory. Use it to check what is resident before generating, or to confirm that `inference unload` freed the models you expected. The lists reflect only in-memory engines, not models available on disk.
To see the complete help for this command, run:
strata inference cache-status --helphelp(db.ai.cache_status)
Example
strata inference cache-statusdb.ai.cache_status()["generation_models"]capability
Report capabilities for a model spec.
strata inference capability
db.ai.capability(model: 'str') -> 'Any'
Parses a model spec into a provider and model name and reports what that combination supports without running the model. The result states whether generation, tokenization, embedding, and ranking are available, whether the operation requires network access or an API key, whether this binary was compiled with the provider feature needed to execute, whether the runtime currently permits network calls, and the known embedding dimension. Model specs are catalog names (`tinyllama`), catalog `name:quant` pairs (`tinyllama:q8_0`), local GGUF paths, or provider specs (`anthropic:claude-...`).
To see the complete help for this command, run:
strata inference capability --helphelp(db.ai.capability)
Example
strata inference capability openai:gpt-4o-mini # Pure metadata - no request is sent to the provider.db.ai.capability("openai:gpt-4o-mini")["provider"]detokenize
Detokenize token ids with a local model.
strata inference detokenize
db.ai.detokenize(ids: 'Sequence[int]', *, model: 'str') -> 'Any'
Decodes an ordered list of token ids back into text using a local model's vocabulary, returning the reconstructed string. Detokenization is a local-only operation: it requires a build with the local execution feature and returns `inference.unsupported_operation` for cloud provider specs.
To see the complete help for this command, run:
strata inference detokenize --helphelp(db.ai.detokenize)
embed
Embed one or more texts into vectors.
strata inference embed
db.ai.embed(input: 'Union[str, Sequence[str]]', *, model: 'str', dimensions: 'Optional[int]' = None, normalize: 'Optional[bool]' = None, input_type: 'Optional[str]' = None, instruction: 'Optional[str]' = None) -> 'Embeddings'
Embeds text with an embedding-capable model and returns one vector per input, in order. The `input` field takes either a single string or an array of strings, so single and batch embedding share one command. The vector dimension is fixed by the model. Local embedding models require a build with the local execution feature; cloud embedding providers (OpenAI, Google) require the matching provider feature and an API key.
To see the complete help for this command, run:
strata inference embed --helphelp(db.ai.embed)
Example
No CLI example in this release.
db.ai.embed(["hello", "world"],generate
Generate text with an inference model.
strata inference generate
db.ai.chat(messages: 'Union[str, Iterable[Mapping[str, Any]], None]' = None, *, model: 'str', prompt: 'Optional[str]' = None, response_format: 'Union[str, Mapping[str, Any], None]' = None, json_schema: 'Optional[Mapping[str, Any]]' = None, **knobs: 'Any') -> 'ChatCompletion'
Runs a text-generation request against a local or cloud model and returns the completion, the reason generation stopped, and the provider-reported prompt and completion token counts. The request controls the maximum completion tokens, sampling temperature, top-k and top-p cutoffs, an optional deterministic seed, string and token-id stop sequences, and an optional GBNF grammar for constrained generation. Chat models expect their chat template already applied in the prompt. Local models require a build with the local execution feature; cloud providers require the matching provider feature and an API key.
To see the complete help for this command, run:
strata inference generate --helphelp(db.ai.chat)
Example
No CLI example in this release.
db.ai.chat("Summarize Strata in one line.",schema = {"type": "object", "properties": {"city": {"type": "string"}}}db.ai.chat("Where is the Eiffel Tower?",tools = [{"type": "function",db.ai.chat("Weather in Paris?",
rank
Rank passages against a query.
strata inference rank
db.ai.rank(query: 'str', passages: 'Sequence[str]', *, model: 'str') -> 'Any'
Scores each candidate passage against a query with a ranking model and returns one outcome per passage. Each item carries the passage's original index and either a relevance score or a per-item error with a stable code and a redacted message, so callers can reorder passages by score while keeping them tied to their inputs. Ranking is a local-only operation: it requires a build with the local execution feature and a ranking-capable model.
To see the complete help for this command, run:
strata inference rank --helphelp(db.ai.rank)
tokenize
Tokenize text with a local model.
strata inference tokenize
db.ai.tokenize(text: 'str', *, model: 'str', add_special: 'bool' = False) -> 'Any'
Encodes text into the token id sequence a local model would see and returns the ids in order. Set `add_special` to include the model's special tokens (such as beginning-of-sequence markers). Tokenization is a local-only operation: it requires a build with the local execution feature and returns `inference.unsupported_operation` for cloud provider specs.
To see the complete help for this command, run:
strata inference tokenize --helphelp(db.ai.tokenize)
unload
Unload cached inference models.
strata inference unload
db.ai.unload(model: 'Optional[str]' = None) -> 'Any'
Removes cached model engines from the runtime to free memory. Pass a model spec to unload one entry, or omit it to unload every cached generation, embedding, and ranking model. The result reports whether any cached entry was actually removed. This affects only the in-memory runtime cache; it never deletes downloaded model files from disk.
To see the complete help for this command, run:
strata inference unload --helphelp(db.ai.unload)
Example
strata inference unloaddb.ai.unload()["unloaded"]Models
models list
List catalog inference models.
strata inference models list
db.ai.models.list() -> 'Any'
Lists every model in Strata's built-in catalog as a terminal page. Each entry reports the model's task (embed, generate, or rank), architecture, default quantization, embedding dimension, HuggingFace repository, approximate artifact size, and whether the model artifact is already present in the local model directory. Use `inference models local` to see only the downloaded models, or `inference models pull` to fetch one.
To see the complete help for this command, run:
strata inference models list --helphelp(db.ai.models.list)
Example
strata inference models listsorted(set(m["task"] for m in db.ai.models.list()["items"]))models local
List locally downloaded inference models.
strata inference models local
db.ai.models.local() -> 'Any'
Lists the catalog models that have at least one quantization variant present in the local model directory, as a terminal page. Entries carry the same facts as `inference models list` but are restricted to models that can run without a further download. The local model directory is resolved from `STRATA_MODELS_DIR`, falling back to `~/.strata/models`.
To see the complete help for this command, run:
strata inference models local --helphelp(db.ai.models.local)
models pull
Download an inference model locally.
strata inference models pull
db.ai.models.pull(model: 'str') -> 'Any'
Resolves a catalog name or model spec and downloads the model artifact into the local model directory, returning the resolved local path. Honors `STRATA_MODELS_DIR` for the destination and `STRATA_HF_ENDPOINT` and `STRATA_HF_TOKEN` (or `HF_TOKEN`) for gated HuggingFace repositories. Downloading requires network access and a build with the local execution feature; cloud-only builds return `inference.unsupported_operation`.
To see the complete help for this command, run:
strata inference models pull --helphelp(db.ai.models.pull)