Bedrock APIs and IAM¶
Which Amazon Bedrock API each call uses, how to choose it per model, and the IAM permissions the pods' role needs.
Which API a call uses¶
| Call | Bedrock API | IAM action |
|---|---|---|
converse / converse_sync, model converse_api: "converse" (default) |
Converse |
bedrock:InvokeModel |
converse / converse_sync, model converse_api: "stream" |
ConverseStream, assembled into a Converse reply |
bedrock:InvokeModelWithResponseStream |
converse_stream |
ConverseStream |
bedrock:InvokeModelWithResponseStream |
embed / embed_sync |
InvokeModel |
bedrock:InvokeModel |
flowchart TD
C([chat call]) --> S{Caller streams?}
S -- "yes: converse_stream" --> CS["ConverseStream<br/>InvokeModelWithResponseStream"]
S -- no --> A{model converse_api}
A -- "converse (default)" --> CV["Converse<br/>InvokeModel<br/>attempt timer bounds the whole answer"]
A -- stream --> CS2["ConverseStream underneath<br/>first-response + idle timers"]
Before 0.1.0b2 every chat call used ConverseStream underneath, so the role always needed
bedrock:InvokeModelWithResponseStream, and models without tool use over streaming could not
call tools. Non-stream calls now use plain Converse by default.
Choosing converse_api¶
"models": {
"gpt120b": {"type": "chat", "provider": "bedrock", "model_id": "openai.gpt-oss-120b-1:0",
"converse_api": "stream", "...": "..."}
}
converse(default): one request, one answer. It needs onlybedrock:InvokeModeland works for every model that supports tools in Converse. The attempt timer (attempt_timeout_secs) bounds the whole answer, so a long generation needs a timeout that fits it.stream: the answer streams underneath and is assembled for the caller. The first-response timer (attempt_timeout_secs, at most the time left) and the idle timer between chunks (stream_idle_secs) apply, so a region that accepts the connection and never answers is given up quickly, while a long healthy answer is never cut. Use it for long answers, or when you want hung regions found fast.
converse_stream calls always use ConverseStream, whatever converse_api says. The setting is
set per model or in defaults.
IAM policy¶
The pods' role (or each assumed account role) needs, for the models and regions in the registry:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "LlmQueueInvoke",
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream"
],
"Resource": [
"arn:aws:bedrock:ap-south-1::foundation-model/*",
"arn:aws:bedrock:ap-south-2::foundation-model/*",
"arn:aws:bedrock:*:123456789012:inference-profile/*"
]
}
]
}
- Drop
bedrock:InvokeModelWithResponseStreamonly if no model usesconverse_api: "stream"and no code callsconverse_stream(the LangChain adapters never stream). - Narrow
Resourceto the model ids and regions in your registry. Cross-region inference profiles need both the profile ARN and the foundation models it routes to. - For accounts with
role_arn, the pods' own role needssts:AssumeRoleon that role (with theexternal_id_refvalue as external id when set), and the assumed role needs the policy above. - Large results in S3 (
S3Offload) need S3 and KMS permissions that differ for workers and callers: see Keep large results in S3.
Errors from IAM and credentials¶
| What Bedrock says | What the library does |
|---|---|
AccessDeniedException for one model in one region (not enabled there) |
A target error: that deployment is counted for health, the call moves on |
| Expired token, bad signature, clock skew | A credentials error: the account is skipped in that region with backoff, the client is rebuilt, an alert-worthy log |
| Every account's credentials refused | CallFailed("credentials_unavailable"), retry_after_secs 60 |
See Attempts and retries for every outcome.