Environment variables
Every process setting (LLMQ_*) that LLMQueue.from_env() reads, with its default and purpose.
Model settings live in the registry document, and timeouts and pool sizes in Tuning (see
Configuration).
Where values come from
llmq = LLMQueue.from_env() # os.environ
llmq = LLMQueue.from_env(secrets=lookup) # lookup(name) first, then os.environ
llmq = LLMQueue.from_env(environ={"LLMQ_STORE_URL": "..."}, # no environment variables at all
secrets=lookup)
environ= replaces os.environ with a mapping you build (from a config file, Secrets
Manager, a test fixture). Nothing is read from the process environment then.
secrets= is a function from a name to a value or None; each LLMQ_* name is looked up
there first, then in environ. LLMQ_STORE_PASSWORD and LLMQ_QUEUE_STORE_PASSWORD are read
through it on every new store connection, so a rotated password needs no restart. Account
secrets (api_key_ref, external_id_ref) are looked up the same way.
Every value is checked at from_env; all problems are raised together as one ConfigError.
LLMQ_ENVIRONMENT: strict or dev
| Value |
Mode |
What it means |
dev, local, test, ci |
dev |
Plain redis:// stores allowed, no id salt needed, any base_url host, LLMQ_DEBUG_CAPTURE_FROM_CONFIG allowed |
anything else (prod, staging, ...) |
strict |
The checks below |
| unset |
strict |
Same as any other name (since 0.1.0b2; before, unset meant dev) |
production is read as prod. Strict mode requires:
- store URLs with
rediss:// (or valkeys://) and the certificate and host name checked; a
localhost URL may stay plain. ssl_cert_reqs=required and ssl_check_hostname=true are
written into the URL so every client built from it verifies explicitly; a repeated query
parameter is refused;
LLMQ_ID_HASH_SALT, at least 16 characters (it keys the hash of ids such as phone numbers;
a short key does not stop brute force);
LLMQ_KEY_PREFIX;
LLMQ_BASE_URL_ALLOWED_HOSTS when the registry has an openai account;
LLMQ_DEBUG_CAPTURE_FROM_CONFIG off (it stores prompt text).
flowchart LR
E{LLMQ_ENVIRONMENT} -- "dev / local / test / ci" --> D[dev: checks relaxed]
E -- "unset or any other name" --> S[strict: TLS, salt,<br/>allowed hosts, no debug capture]
Numbers have a range: whole-number variables refuse 200.0 or 1e3, TTLs stay within
2147483647 seconds (what EXPIRE takes), and every bound is listed below. An empty value
(LLMQ_LEASE_SECS=, as compose's ${VAR:-} writes) counts as unset.
Stores
| Variable |
Default |
Meaning |
LLMQ_STORE_URL |
required |
The state store: redis://, rediss://, valkey:// or valkeys:// |
LLMQ_STORE_PASSWORD |
|
Optional; read through secrets= on every new connection (rotation) |
LLMQ_QUEUE_STORE_URL |
the state store |
Optional separate store for queues and doorbells (same TLS rule) |
LLMQ_QUEUE_STORE_PASSWORD |
|
As LLMQ_STORE_PASSWORD, for the queue store |
LLMQ_STORE_MAX_CONNECTIONS |
200 |
Store connection pool per engine (worker slots + callers + 3), 4 to 100000. Sync calls without a started queue use at most 20 |
LLMQ_QUEUE_FAILOVER_SECS |
5 |
With a separate queue store: send errors this long before calls run direct (0 to 3600) |
LLMQ_QUEUE_WAIT_REPLICAS |
0 (off) |
A send waits for this many replicas to acknowledge it (0 to 16) ... |
LLMQ_QUEUE_WAIT_MS |
100 |
... for at most this long, up to 60000 (must stay below the store read timeout, see Tune) |
Environment, names and process
| Variable |
Default |
Meaning |
LLMQ_ENVIRONMENT |
unset (strict) |
See above |
LLMQ_ID_HASH_SALT |
|
Key of the user / session id hash (HMAC); required when strict, at least 16 characters. Use the same value in every service that shares the store |
LLMQ_KEY_PREFIX |
llmq in dev and test; required in strict mode |
Prefix of this environment's store keys: name it per app and environment (myapp-prod). 1-64 letters, digits, _, . or - (no braces, colons or glob characters: it is part of every key) |
LLMQ_QUEUE_PREFIX |
empty |
Prefix of this environment's queue names; 0-64 of the same characters |
LLMQ_MODE |
both |
both (caller and worker), caller or worker |
LLMQ_METRICS_SINK |
emf |
emf (CloudWatch embedded metrics), prometheus (extra metrics) or none. One sink per process |
Registry
| Variable |
Default |
Meaning |
LLMQ_REGISTRY |
|
The registry document (JSON) |
LLMQ_REGISTRY_FILE |
|
A file holding it (set one of the two, not both). A repeated JSON key is refused |
LLMQ_REGISTRY_LIVE_EDIT |
safe |
safe, all or off, when the document has no live_edit field (see Live edits) |
LLMQ_CFG_RELOAD_SECS |
30 |
How often each process reloads the registry (1 to 3600) |
LLMQ_AUDIT_RETENTION_DAYS |
90 |
How long registry audit entries are kept (at most 3650) |
LLMQ_BASE_URL_ALLOWED_HOSTS |
empty |
Comma-separated bare host names (no scheme, port or path: api.openai.com) an openai account's base_url may name. Empty: any host in dev; required when strict and the registry has an openai account |
LLMQ_DEBUG_CAPTURE_FROM_CONFIG |
false |
Allows a model's debug_capture (prompt text in the store for an hour); refused when strict |
Calls and lifecycle
| Variable |
Default |
Meaning |
LLMQ_LEASE_SECS |
60 |
Lease of a running call's grant; renewed by the heartbeat every third of it (at most 3600) |
LLMQ_CALLER_TTL_SECS |
30 |
How long a caller process counts as alive without a heartbeat. Must be at least LLMQ_LEASE_SECS / 3 plus a margin (max(5, beat / 2)), checked at start |
LLMQ_RESULT_TTL_SECS |
300 |
How long a stored result lives; at least Tuning.engine.result_recheck_secs (60), or start-up fails |
LLMQ_RESULT_DEDUPE_SECS |
0 |
Keep a result with a library-made id this long after it was read (0: delete on read) |
LLMQ_STATUS_TTL_SECS |
1800 |
How long a call's status record lives (at least 60) |
LLMQ_DRAIN_SECS |
25 |
Default drain_secs of stop() (0 to 3600; 0 returns held work at once); fits the Kubernetes default grace period of 30 s (see Operate) |
LLMQ_LOCAL_LIMIT_CAP |
10 |
Calls per process and model run without the store under store_down: static_cap |
Account secrets
| Variable |
Meaning |
LLMQ_ACCOUNT_* |
Secrets named by a registry account's api_key_ref or external_id_ref. Only names with this prefix (upper case, digits, _) are accepted, so a registry can never name the store password or the pod's AWS keys |
Tests only
| Variable |
Meaning |
LLMQ_BEDROCK_ENDPOINT_URL, LLMQ_STS_ENDPOINT_URL |
Point Bedrock / STS at fakes (fakellm, moto); https only in strict mode |