Support service_tier selection for Amazon Bedrock (#2098)

* feat: add HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER env var

Adds support for setting Bedrock service tier (flex/priority/reserved)
via environment variable, following the same pattern as the existing
Groq and OpenAI service tier support.

- config.py: env constant, default, dataclass field, os.getenv() load
- llm_wrapper.py: bedrock_service_tier param plumbing
- litellm_llm.py: inject service_tier kwarg for bedrock/ models
- configuration.md: table entry + Bedrock example block
- models.md/mdx: Bedrock tip block update

Closes #2072

* Add validation + tests for HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER

- validate() rejects invalid values (e.g. 'standard') with clear error
- Empty string treated as unset (matching llm_output_language pattern)
- Tests: default, flex, priority, reserved, invalid value, empty string

* fix(api): thread bedrock_service_tier from config into LLM providers

The new HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER flag was plumbed through
LLMProvider/create_llm_provider/LiteLLMLLM but nothing ever constructed
an LLMProvider with the resolved config value, so the env var was inert
(service_tier was never injected into the Bedrock call).

- memory_engine.py: pass bedrock_service_tier=config.llm_bedrock_service_tier
  to all four LLMConfig constructions (default/retain/reflect/consolidation)
- llm_wrapper.py: LLMProvider.from_env() reads the env var too, so ad-hoc
  constructions honor it
- test_bedrock_service_tier.py: plumbing tests asserting the tier reaches
  the LiteLLM call kwargs for bedrock/ models, is omitted otherwise, and
  is guarded off non-Bedrock models

* style(test): ruff format test_config_validation.py (fix verify-generated-files)

---------

Co-authored-by: Hermes Agent (Rob) <hermes-sa@mobilinkd.com>
Co-authored-by: Nicolò Boschi <boschi1997@gmail.com>

hindsight-api-slim/hindsight_api/config.py

@@ -141,6 +141,7 @@ ENV_LLM_TIMEOUT = "HINDSIGHT_API_LLM_TIMEOUT"
 ENV_LLM_REASONING_EFFORT = "HINDSIGHT_API_LLM_REASONING_EFFORT"
 ENV_LLM_GROQ_SERVICE_TIER = "HINDSIGHT_API_LLM_GROQ_SERVICE_TIER"
 ENV_LLM_OPENAI_SERVICE_TIER = "HINDSIGHT_API_LLM_OPENAI_SERVICE_TIER"
+ENV_LLM_BEDROCK_SERVICE_TIER = "HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER"
 ENV_LLM_EXTRA_BODY = "HINDSIGHT_API_LLM_EXTRA_BODY"
 ENV_LLM_DEFAULT_HEADERS = "HINDSIGHT_API_LLM_DEFAULT_HEADERS"
 ENV_LLM_STRICT_SCHEMA = "HINDSIGHT_API_LLM_STRICT_SCHEMA"
@@ -156,6 +157,7 @@ ENV_LLM_LITELLMROUTER_CONFIG = "HINDSIGHT_API_LLM_LITELLMROUTER_CONFIG"
 # Defaults for service tiers
 DEFAULT_LLM_GROQ_SERVICE_TIER = "auto"  # "on_demand", "flex", or "auto"
 DEFAULT_LLM_OPENAI_SERVICE_TIER = None  # None (default) or "flex" (50% cheaper)
+DEFAULT_LLM_BEDROCK_SERVICE_TIER = None  # None (default), "flex", "priority", or "reserved"
 DEFAULT_LLM_EXTRA_BODY = None  # None = no extra body params; JSON dict merged into OpenAI extra_body
 DEFAULT_LLM_DEFAULT_HEADERS = (
     None  # None = no extra headers; JSON dict passed as default_headers to provider SDK clients
@@ -1214,6 +1216,7 @@ class HindsightConfig:
     llm_reasoning_effort: str
     llm_groq_service_tier: str  # Groq: "on_demand", "flex", or "auto"
     llm_openai_service_tier: str | None  # OpenAI: None (default) or "flex" (50% cheaper)
+    llm_bedrock_service_tier: str | None  # Bedrock: None (default), "flex", "priority", or "reserved"
     llm_extra_body: (
         dict | None
     )  # Extra body params merged into OpenAI-compatible API calls (e.g. {"chat_template_kwargs": {"enable_thinking": true}})
@@ -1762,6 +1765,16 @@ class HindsightConfig:
                 f"Invalid semantic_min_similarity: {self.semantic_min_similarity}. Must be between 0.0 and 1.0"
             )
 
+        # Validate bedrock_service_tier
+        valid_bedrock_tiers = (None, "flex", "priority", "reserved")
+        if self.llm_bedrock_service_tier not in valid_bedrock_tiers:
+            raise ValueError(
+                f"Invalid HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER: "
+                f"{self.llm_bedrock_service_tier!r}. Must be one of: "
+                f"{', '.join(t for t in valid_bedrock_tiers if t is not None)}. "
+                f"Note: 'standard' is not a valid Bedrock service tier -- use unset for default tier."
+            )
+
         # When LLM provider is "none", force chunks-only mode and disable LLM-dependent features
         if self.llm_provider == "none":
             self.retain_extraction_mode = "chunks"
@@ -1875,6 +1888,7 @@ class HindsightConfig:
             llm_reasoning_effort=os.getenv(ENV_LLM_REASONING_EFFORT, DEFAULT_LLM_REASONING_EFFORT),
             llm_groq_service_tier=os.getenv(ENV_LLM_GROQ_SERVICE_TIER, DEFAULT_LLM_GROQ_SERVICE_TIER),
             llm_openai_service_tier=os.getenv(ENV_LLM_OPENAI_SERVICE_TIER, DEFAULT_LLM_OPENAI_SERVICE_TIER),
+            llm_bedrock_service_tier=os.getenv(ENV_LLM_BEDROCK_SERVICE_TIER) or None,
             llm_extra_body=json.loads(os.getenv(ENV_LLM_EXTRA_BODY, "null")),
             llm_default_headers=json.loads(os.getenv(ENV_LLM_DEFAULT_HEADERS, "null")),
             llm_strict_schema=os.getenv(ENV_LLM_STRICT_SCHEMA, str(DEFAULT_LLM_STRICT_SCHEMA)).lower() in ("true", "1"),

hindsight-api-slim/hindsight_api/engine/llm_wrapper.py

@@ -249,6 +249,7 @@ def create_llm_provider(
     reasoning_effort: str,
     groq_service_tier: str | None = None,
     openai_service_tier: str | None = None,
+    bedrock_service_tier: str | None = None,
     extra_body: dict[str, Any] | None = None,
     default_headers: dict[str, str] | None = None,
     vertexai_project_id: str | None = None,
@@ -269,6 +270,7 @@ def create_llm_provider(
         reasoning_effort: Reasoning effort level for supported providers.
         groq_service_tier: Groq service tier (for Groq provider) - "on_demand", "flex", or "auto".
         openai_service_tier: OpenAI service tier (for OpenAI provider) - None (default) or "flex" (50% cheaper).
+        bedrock_service_tier: Bedrock service tier (for Bedrock provider) - None (default), "flex", "priority", or "reserved".
         extra_body: Extra request-body params merged into the provider's native
             call. Threaded into OpenAI-compatible, Fireworks, Anthropic, Gemini/
             VertexAI and LiteLLM providers (each merges them in its own parameter
@@ -401,6 +403,7 @@ def create_llm_provider(
             model=bedrock_model,
             reasoning_effort=reasoning_effort,
             extra_body=extra_body,
+            bedrock_service_tier=bedrock_service_tier,
         )
 
     elif provider_lower == "llamacpp":
@@ -478,6 +481,7 @@ class LLMProvider:
         reasoning_effort: str = "low",
         groq_service_tier: str | None = None,
         openai_service_tier: str | None = None,
+        bedrock_service_tier: str | None = None,
         gemini_safety_settings: list | None = None,
         prompt_cache_enabled: bool = False,
         extra_body: dict[str, Any] | None = None,
@@ -495,6 +499,7 @@ class LLMProvider:
             reasoning_effort: Reasoning effort level for supported providers.
             groq_service_tier: Groq service tier ("on_demand", "flex", "auto") - from config.
             openai_service_tier: OpenAI service tier (None or "flex") - from config.
+            bedrock_service_tier: Bedrock service tier (None, "flex", "priority", "reserved") - from config.
             gemini_safety_settings: Safety settings for Gemini/VertexAI providers.
             extra_body: Extra request-body params merged into the provider's native call
                 (OpenAI-compatible, Fireworks, Anthropic, Gemini/VertexAI, LiteLLM).
@@ -517,6 +522,7 @@ class LLMProvider:
         # Service tiers from hierarchical config (not env vars)
         self.groq_service_tier = groq_service_tier
         self.openai_service_tier = openai_service_tier
+        self.bedrock_service_tier = bedrock_service_tier
         # Gemini safety settings (instance default; can be overridden per-request via context var)
         self.gemini_safety_settings = gemini_safety_settings
         # Gemini prompt caching: when True, retain extraction (and any future
@@ -679,6 +685,7 @@ class LLMProvider:
             reasoning_effort=self.reasoning_effort,
             groq_service_tier=self.groq_service_tier,
             openai_service_tier=self.openai_service_tier,
+            bedrock_service_tier=self.bedrock_service_tier,
             extra_body=self.extra_body,
             default_headers=self.default_headers,
             vertexai_project_id=vertexai_project_id,
@@ -1120,6 +1127,7 @@ class LLMProvider:
             DEFAULT_LLM_REASONING_EFFORT,
             ENV_LLM_API_KEY,
             ENV_LLM_BASE_URL,
+            ENV_LLM_BEDROCK_SERVICE_TIER,
             ENV_LLM_DEFAULT_HEADERS,
             ENV_LLM_EXTRA_BODY,
             ENV_LLM_MODEL,
@@ -1151,6 +1159,7 @@ class LLMProvider:
             reasoning_effort=os.getenv(ENV_LLM_REASONING_EFFORT, DEFAULT_LLM_REASONING_EFFORT),
             extra_body=extra_body,
             default_headers=default_headers,
+            bedrock_service_tier=os.getenv(ENV_LLM_BEDROCK_SERVICE_TIER) or None,
         )
 
 

hindsight-api-slim/hindsight_api/engine/memory_engine.py

@@ -821,6 +821,7 @@ class MemoryEngine(MemoryEngineInterface):
             extra_body=config.llm_extra_body,
             default_headers=config.llm_default_headers,
             litellmrouter_config=config.llm_litellmrouter_config,
+            bedrock_service_tier=config.llm_bedrock_service_tier,
         )
 
         # Store client and model for convenience (deprecated: use _llm_config.call() instead)
@@ -853,6 +854,7 @@ class MemoryEngine(MemoryEngineInterface):
             extra_body=config.llm_extra_body,
             default_headers=config.llm_default_headers,
             litellmrouter_config=config.retain_llm_litellmrouter_config or config.llm_litellmrouter_config,
+            bedrock_service_tier=config.llm_bedrock_service_tier,
         )
 
         # Reflect LLM config - for think/observe operations (can use lighter models)
@@ -880,6 +882,7 @@ class MemoryEngine(MemoryEngineInterface):
             extra_body=config.llm_extra_body,
             default_headers=config.llm_default_headers,
             litellmrouter_config=config.reflect_llm_litellmrouter_config or config.llm_litellmrouter_config,
+            bedrock_service_tier=config.llm_bedrock_service_tier,
         )
 
         # Consolidation LLM config - for mental model consolidation (can use efficient models)
@@ -907,6 +910,7 @@ class MemoryEngine(MemoryEngineInterface):
             extra_body=config.llm_extra_body,
             default_headers=config.llm_default_headers,
             litellmrouter_config=config.consolidation_llm_litellmrouter_config or config.llm_litellmrouter_config,
+            bedrock_service_tier=config.llm_bedrock_service_tier,
         )
 
         # Initialize cross-encoder reranker (cached for performance)

hindsight-api-slim/hindsight_api/engine/providers/litellm_llm.py

@@ -49,6 +49,7 @@ class LiteLLMLLM(LLMInterface):
         reasoning_effort: str = "low",
         timeout: float = 300.0,
         extra_body: dict[str, Any] | None = None,
+        bedrock_service_tier: str | None = None,
         **kwargs: Any,
     ):
         super().__init__(provider, api_key, base_url, model, reasoning_effort, **kwargs)
@@ -60,6 +61,7 @@ class LiteLLMLLM(LLMInterface):
         # drops any the target model rejects (litellm.drop_params=True below).
         # Sourced from llm_extra_body (env: HINDSIGHT_API_LLM_EXTRA_BODY).
         self._extra_body: dict[str, Any] = extra_body or {}
+        self.bedrock_service_tier = bedrock_service_tier
 
         try:
             import litellm
@@ -119,6 +121,10 @@ class LiteLLMLLM(LLMInterface):
         for key, value in self._extra_body.items():
             kwargs.setdefault(key, value)
 
+        # Bedrock service tier: flex (50% cheaper), priority, or reserved
+        if self.model.startswith("bedrock/") and self.bedrock_service_tier is not None:
+            kwargs["service_tier"] = self.bedrock_service_tier
+
         return kwargs
 
     # ── per-model output-tokens cap (shared with Router subclass) ────────────

hindsight-api-slim/tests/test_bedrock_service_tier.py

@@ -0,0 +1,61 @@
+"""Plumbing tests for the Bedrock service tier (HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER).
+
+These assert the *wiring* — that a configured tier actually reaches the LiteLLM
+call kwargs — rather than just that the env var parses into config (covered by
+test_config_validation.py). The config value is threaded
+config -> LLMProvider -> create_llm_provider -> LiteLLMLLM, and only LiteLLMLLM
+injects ``service_tier`` for ``bedrock/`` models, so the checks live here.
+"""
+
+from hindsight_api.engine.llm_wrapper import LLMConfig
+from hindsight_api.engine.providers.litellm_llm import LiteLLMLLM
+
+_MESSAGES = [{"role": "user", "content": "hi"}]
+
+
+def _make_litellm(model: str, tier: str | None) -> LiteLLMLLM:
+    return LiteLLMLLM(provider="bedrock", api_key="", base_url="", model=model, bedrock_service_tier=tier)
+
+
+def test_bedrock_model_injects_service_tier():
+    """A configured tier is injected as ``service_tier`` for bedrock/ models."""
+    llm = _make_litellm("bedrock/us.amazon.nova-2-lite-v1:0", "flex")
+    kwargs = llm._build_common_kwargs(messages=_MESSAGES)
+    assert kwargs["service_tier"] == "flex"
+
+
+def test_bedrock_model_without_tier_omits_service_tier():
+    """No tier configured -> no ``service_tier`` key (Bedrock default tier)."""
+    llm = _make_litellm("bedrock/us.amazon.nova-2-lite-v1:0", None)
+    kwargs = llm._build_common_kwargs(messages=_MESSAGES)
+    assert "service_tier" not in kwargs
+
+
+def test_non_bedrock_model_never_gets_service_tier():
+    """The bedrock/ prefix guard keeps the kwarg off non-Bedrock LiteLLM models."""
+    llm = LiteLLMLLM(
+        provider="litellm",
+        api_key="k",
+        base_url="",
+        model="fireworks_ai/accounts/fireworks/models/llama-v3p1-70b-instruct",
+        bedrock_service_tier="flex",
+    )
+    kwargs = llm._build_common_kwargs(messages=_MESSAGES)
+    assert "service_tier" not in kwargs
+
+
+def test_llm_config_threads_tier_to_provider_impl():
+    """End-to-end: LLMConfig -> create_llm_provider -> LiteLLMLLM carries the tier.
+
+    This is the bridge the env var depends on; if MemoryEngine ever stops
+    passing ``bedrock_service_tier`` through, the value silently defaults to
+    None and the flag becomes inert.
+    """
+    llm = LLMConfig(
+        provider="bedrock",
+        api_key="",
+        base_url="",
+        model="us.amazon.nova-2-lite-v1:0",
+        bedrock_service_tier="flex",
+    )
+    assert llm._provider_impl.bedrock_service_tier == "flex"

hindsight-api-slim/tests/test_config_validation.py

@@ -23,6 +23,7 @@ def setup_test_env():
         "HINDSIGHT_API_LLM_PROVIDER",
         "HINDSIGHT_API_LLM_MODEL",
         "HINDSIGHT_API_LLM_REASONING_EFFORT",
+        "HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER",
         "HINDSIGHT_API_SEMANTIC_MIN_SIMILARITY",
         "HINDSIGHT_API_DATABASE_URL",
         "HINDSIGHT_API_MIGRATION_DATABASE_URL",
@@ -465,3 +466,68 @@ def test_recall_max_candidates_per_source_loaded_from_env(monkeypatch):
 
     config = HindsightConfig.from_env()
     assert config.recall_max_candidates_per_source == 150
+
+
+# ---------------------------------------------------------------------------
+# Bedrock service tier (HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER)
+# ---------------------------------------------------------------------------
+
+
+def test_bedrock_service_tier_defaults_to_none(monkeypatch):
+    """Bedrock service tier defaults to None (standard tier) when unset."""
+    from hindsight_api.config import HindsightConfig
+
+    monkeypatch.delenv("HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER", raising=False)
+    monkeypatch.setenv("HINDSIGHT_API_LLM_PROVIDER", "mock")
+
+    config = HindsightConfig.from_env()
+    assert config.llm_bedrock_service_tier is None
+
+
+def test_bedrock_service_tier_flex(monkeypatch):
+    """Flex tier (50% cost savings) is accepted."""
+    from hindsight_api.config import HindsightConfig
+
+    monkeypatch.setenv("HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER", "flex")
+    monkeypatch.setenv("HINDSIGHT_API_LLM_PROVIDER", "mock")
+
+    config = HindsightConfig.from_env()
+    assert config.llm_bedrock_service_tier == "flex"
+
+
+def test_bedrock_service_tier_priority(monkeypatch):
+    """Priority tier (guaranteed throughput) is accepted."""
+    from hindsight_api.config import HindsightConfig
+
+    monkeypatch.setenv("HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER", "priority")
+    monkeypatch.setenv("HINDSIGHT_API_LLM_PROVIDER", "mock")
+
+    config = HindsightConfig.from_env()
+    assert config.llm_bedrock_service_tier == "priority"
+
+
+def test_bedrock_service_tier_reserved(monkeypatch):
+    """Reserved tier (provisioned capacity) is accepted."""
+    from hindsight_api.config import HindsightConfig
+
+    monkeypatch.setenv("HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER", "reserved")
+    monkeypatch.setenv("HINDSIGHT_API_LLM_PROVIDER", "mock")
+
+    config = HindsightConfig.from_env()
+    assert config.llm_bedrock_service_tier == "reserved"
+
+
+def test_bedrock_service_tier_rejects_invalid_value(monkeypatch):
+    """ "standard" is not a valid Bedrock service tier and must be rejected."""
+    from hindsight_api.config import HindsightConfig
+
+    monkeypatch.setenv("HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER", "standard")
+    monkeypatch.setenv("HINDSIGHT_API_LLM_PROVIDER", "mock")
+
+    with pytest.raises(ValueError) as exc_info:
+        HindsightConfig.from_env()
+
+    error_message = str(exc_info.value)
+    assert "HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER" in error_message
+    assert "standard" in error_message
+    assert "'standard' is not a valid Bedrock service tier" in error_message

hindsight-docs/docs/developer/configuration.md

@@ -175,6 +175,7 @@ For non-English banks (especially CJK) and the language/extraction-language trad
 | `HINDSIGHT_API_LLM_REASONING_EFFORT` | Reasoning effort for providers/models that support it (for example `low`, `medium`, `high`, `xhigh`) | `low` |
 | `HINDSIGHT_API_LLM_GROQ_SERVICE_TIER` | Groq service tier: `on_demand`, `flex`, `auto` | `auto` |
 | `HINDSIGHT_API_LLM_OPENAI_SERVICE_TIER` | OpenAI service tier: `flex` for 50% cost savings (OpenAI Flex Processing) | None (default) |
+| `HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER` | Bedrock service tier: `flex` for 50% cost savings (best-effort inference), `priority` (guaranteed throughput), or `reserved` (provisioned capacity) | Unset (default tier) |
 | `HINDSIGHT_API_LLM_EXTRA_BODY` | JSON dict of extra request-body params (e.g. `temperature`, `top_p`, `max_tokens`) merged into every LLM call. Applied across the OpenAI-compatible, Fireworks, Anthropic, Gemini/VertexAI and LiteLLM (incl. Bedrock/Router) providers. Each provider merges them in its own native parameter space, so use that provider's field names (e.g. `max_tokens` for OpenAI/Anthropic vs `max_output_tokens` for Gemini). Also useful for custom model servers (e.g. vLLM `chat_template_kwargs`). | `null` |
 | `HINDSIGHT_API_LLM_DEFAULT_HEADERS` | JSON dict passed as `default_headers` to provider SDK clients. Used by operators routing through proxies / request-tracing middleware (e.g. Cloudflare AI Gateway, Helicone, corporate proxies). Currently wired into the Anthropic provider; other providers can opt in. | `null` |
 | `HINDSIGHT_API_LLM_STRICT_SCHEMA` | Grammar-enforce structured output via `json_schema` `strict: true` instead of the soft "schema-in-prompt + `json_object`" path. Use it with weaker self-hosted models that return prose preambles, markdown ` ```json ` fences, or invalid JSON — which otherwise fail to parse and wedge retain/consolidation. Applies to OpenAI-compatible backends (OpenAI, llama.cpp, vLLM) and LiteLLM; Gemini already enforces its native `response_schema` regardless, and providers without a strict mode ignore it. | `false` |
@@ -294,6 +295,8 @@ export HINDSIGHT_API_LLM_MODEL=us.amazon.nova-2-lite-v1:0
 export AWS_ACCESS_KEY_ID=your-access-key
 export AWS_SECRET_ACCESS_KEY=your-secret-key
 export AWS_REGION_NAME=us-east-1
+# Optional: Use Flex tier for 50% cost savings (with variable latency)
+# export HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER=flex
 
 # LiteLLM (100+ providers via LiteLLM SDK)
 # Azure OpenAI via LiteLLM

hindsight-docs/docs/developer/models.mdx

@@ -33,7 +33,7 @@ See [Configuration](./configuration#llm-provider) for setup examples.
 :::
 
 :::tip AWS Bedrock
-Set `HINDSIGHT_API_LLM_PROVIDER=bedrock` to use AWS Bedrock models directly. Model names use Bedrock model IDs (e.g., `us.amazon.nova-2-lite-v1:0`). No API key is required — authentication uses AWS credentials (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION_NAME`) or IAM roles.
+Set `HINDSIGHT_API_LLM_PROVIDER=bedrock` to use AWS Bedrock models directly. Model names use Bedrock model IDs (e.g., `us.amazon.nova-2-lite-v1:0`). No API key is required — authentication uses AWS credentials (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION_NAME`) or IAM roles. For 50% cost savings on throughput, set `HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER=flex` (see [Configuration](./configuration#llm-provider)).
 
 See [Configuration](./configuration#llm-provider) for setup examples.
 :::

skills/hindsight-docs/references/developer/configuration.md

@@ -175,6 +175,7 @@ For non-English banks (especially CJK) and the language/extraction-language trad
 | `HINDSIGHT_API_LLM_REASONING_EFFORT` | Reasoning effort for providers/models that support it (for example `low`, `medium`, `high`, `xhigh`) | `low` |
 | `HINDSIGHT_API_LLM_GROQ_SERVICE_TIER` | Groq service tier: `on_demand`, `flex`, `auto` | `auto` |
 | `HINDSIGHT_API_LLM_OPENAI_SERVICE_TIER` | OpenAI service tier: `flex` for 50% cost savings (OpenAI Flex Processing) | None (default) |
+| `HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER` | Bedrock service tier: `flex` for 50% cost savings (best-effort inference), `priority` (guaranteed throughput), or `reserved` (provisioned capacity) | Unset (default tier) |
 | `HINDSIGHT_API_LLM_EXTRA_BODY` | JSON dict of extra request-body params (e.g. `temperature`, `top_p`, `max_tokens`) merged into every LLM call. Applied across the OpenAI-compatible, Fireworks, Anthropic, Gemini/VertexAI and LiteLLM (incl. Bedrock/Router) providers. Each provider merges them in its own native parameter space, so use that provider's field names (e.g. `max_tokens` for OpenAI/Anthropic vs `max_output_tokens` for Gemini). Also useful for custom model servers (e.g. vLLM `chat_template_kwargs`). | `null` |
 | `HINDSIGHT_API_LLM_DEFAULT_HEADERS` | JSON dict passed as `default_headers` to provider SDK clients. Used by operators routing through proxies / request-tracing middleware (e.g. Cloudflare AI Gateway, Helicone, corporate proxies). Currently wired into the Anthropic provider; other providers can opt in. | `null` |
 | `HINDSIGHT_API_LLM_STRICT_SCHEMA` | Grammar-enforce structured output via `json_schema` `strict: true` instead of the soft "schema-in-prompt + `json_object`" path. Use it with weaker self-hosted models that return prose preambles, markdown ` ```json ` fences, or invalid JSON — which otherwise fail to parse and wedge retain/consolidation. Applies to OpenAI-compatible backends (OpenAI, llama.cpp, vLLM) and LiteLLM; Gemini already enforces its native `response_schema` regardless, and providers without a strict mode ignore it. | `false` |
@@ -294,6 +295,8 @@ export HINDSIGHT_API_LLM_MODEL=us.amazon.nova-2-lite-v1:0
 export AWS_ACCESS_KEY_ID=your-access-key
 export AWS_SECRET_ACCESS_KEY=your-secret-key
 export AWS_REGION_NAME=us-east-1
+# Optional: Use Flex tier for 50% cost savings (with variable latency)
+# export HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER=flex
 
 # LiteLLM (100+ providers via LiteLLM SDK)
 # Azure OpenAI via LiteLLM

skills/hindsight-docs/references/developer/models.md

@@ -50,7 +50,7 @@ Hindsight works with any provider that exposes an OpenAI-compatible API (e.g., A
 See [Configuration](./configuration#llm-provider) for setup examples.
 > **💡 AWS Bedrock**
 > 
-Set `HINDSIGHT_API_LLM_PROVIDER=bedrock` to use AWS Bedrock models directly. Model names use Bedrock model IDs (e.g., `us.amazon.nova-2-lite-v1:0`). No API key is required — authentication uses AWS credentials (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION_NAME`) or IAM roles.
+Set `HINDSIGHT_API_LLM_PROVIDER=bedrock` to use AWS Bedrock models directly. Model names use Bedrock model IDs (e.g., `us.amazon.nova-2-lite-v1:0`). No API key is required — authentication uses AWS credentials (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION_NAME`) or IAM roles. For 50% cost savings on throughput, set `HINDSIGHT_API_LLM_BEDROCK_SERVICE_TIER=flex` (see [Configuration](./configuration#llm-provider)).
 
 See [Configuration](./configuration#llm-provider) for setup examples.
 > **💡 Built-in llama.cpp (fully local, no API key)**