feat(llm): add GitHub Copilot subscription provider (#3597)

Adds a `github-copilot` LLM provider backed by the official GitHub Copilot SDK, using the signed-in Copilot entitlement with no `HINDSIGHT_API_LLM_API_KEY`.

Verified end-to-end against a live Copilot subscription: retain, consolidation and reflect all run on the provider.

Review fixes on top of the original submission:
- default to `gpt-5.6-terra`; the submitted `gpt-5.6-sol` is not in the model list the runtime serves, so every call failed and the provider was unusable out of the box
- make a rejected request configuration terminal instead of a runtime failure; an unavailable model was invalidating the shared runtime and respawning `copilot --headless` once per attempt (5 spawns at max_retries=3, ~12 at the default of 10)
- let a token-authenticated host run without `~/.copilot`, so the documented COPILOT_GITHUB_TOKEN / GH_TOKEN / GITHUB_TOKEN path works in containers and CI
- regenerate skills/hindsight-docs (the provider list also feeds the generated faq.md)

Co-authored-by: Max Marino <dudujuju828@users.noreply.github.com>
This commit is contained in:
Max Marino
2026-08-19 14:48:00 +01:00
committed by GitHub
parent 08a119eeb6
commit 1d9a6a381c
37 changed files with 1702 additions and 42 deletions
+7 -2
View File
@@ -2,11 +2,11 @@
# Copy this file to .env and fill in your values
# LLM Configuration (Required)
# Supported providers: openai, openai-responses, groq, ollama, gemini, anthropic, lmstudio, vertexai, minimax, deepseek, zai, atlas, volcano
# Supported providers: openai, openai-responses, groq, ollama, gemini, anthropic, lmstudio, vertexai, minimax, deepseek, zai, atlas, volcano, openai-codex, claude-code, github-copilot
HINDSIGHT_API_LLM_PROVIDER=openai
HINDSIGHT_API_LLM_API_KEY=your-api-key-here
HINDSIGHT_API_LLM_MODEL=gpt-4o-mini
HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
# HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
# Reasoning effort for providers/models that support it. Examples: none, low, medium, high, xhigh.
# Set it and the value is sent as given, whatever the model is called — use `none` to stop a
# self-hosted reasoning model (vLLM, Ollama, llama.cpp, TGI) emitting thinking blocks. Unset,
@@ -74,6 +74,11 @@ HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
# HINDSIGHT_API_LLM_API_KEY=your-anthropic-api-key
# HINDSIGHT_API_LLM_MODEL=claude-sonnet-4-20250514
# Example: GitHub Copilot subscription via the official Copilot SDK
# Sign in with Copilot CLI first; no HINDSIGHT_API_LLM_API_KEY is needed.
# HINDSIGHT_API_LLM_PROVIDER=github-copilot
# HINDSIGHT_API_LLM_MODEL=gpt-5.6-terra
# Example: Google Vertex AI configuration
# HINDSIGHT_API_LLM_PROVIDER=vertexai
# HINDSIGHT_API_LLM_MODEL=google/gemini-2.0-flash-001
+1 -1
View File
@@ -89,7 +89,7 @@ cd hindsight-control-plane && npm run dev
### Core Engine (hindsight-api-slim/hindsight_api/engine/)
- `memory_engine.py`: Main orchestrator for retain/recall/reflect operations
- `llm_wrapper.py`: LLM abstraction supporting OpenAI, Anthropic, Gemini, VertexAI, Groq, MiniMax, Ollama, LM Studio, LiteLLM, Claude Code
- `llm_wrapper.py`: LLM abstraction supporting OpenAI, Anthropic, Gemini, VertexAI, Groq, MiniMax, Ollama, LM Studio, LiteLLM, Claude Code, GitHub Copilot
- `embeddings.py`: Embedding generation (local sentence-transformers or TEI)
- `cross_encoder.py`: Reranking (local or TEI)
- `entity_resolver.py`: Entity extraction and normalization
+1 -1
View File
@@ -77,7 +77,7 @@ docker run -it --pull always --name hindsight --restart unless-stopped -p 8888:8
>API: http://localhost:8888
>UI: http://localhost:9999
Hindsight works with **25+ LLM providers** via `HINDSIGHT_API_LLM_PROVIDER` — hosted (`openai`, `anthropic`, `gemini`, `groq`, `bedrock`, `vertexai`, `minimax`, `deepseek`, `atlas`, …), fully local (`ollama`, `lmstudio`, `llamacpp`), any OpenAI-compatible endpoint, and gateways (`litellm`, `litellmrouter`) that reach the rest. Existing subscriptions work too: `openai-codex` (ChatGPT Plus/Pro) and `claude-code` (Claude Pro/Max) need no API key. See [supported models](https://hindsight.vectorize.io/developer/models).
Hindsight works with **25+ LLM providers** via `HINDSIGHT_API_LLM_PROVIDER` — hosted (`openai`, `anthropic`, `gemini`, `groq`, `bedrock`, `vertexai`, `minimax`, `deepseek`, `atlas`, …), fully local (`ollama`, `lmstudio`, `llamacpp`), any OpenAI-compatible endpoint, and gateways (`litellm`, `litellmrouter`) that reach the rest. Existing subscriptions work too: `openai-codex` (ChatGPT Plus/Pro), `claude-code` (Claude Pro/Max) and `github-copilot` (GitHub Copilot) need no API key. See [supported models](https://hindsight.vectorize.io/developer/models).
#### Docker (external PostgreSQL)
+1 -1
View File
@@ -74,7 +74,7 @@ class HindsightEmbedded:
Args:
profile: Profile name for data isolation (default: "default")
llm_provider: LLM provider ("groq", "openai", "ollama", "gemini", "anthropic",
"lmstudio"). Omit to inherit; the server default is "openai".
"lmstudio", "github-copilot"). Omit to inherit; the server default is "openai".
llm_api_key: API key for the LLM provider. Omit to inherit; pass "" to
explicitly run without a key (local services that need no auth).
llm_model: Model name to use. Omit to inherit; the server picks a default
+1 -1
View File
@@ -80,7 +80,7 @@ Configure via environment variables:
| Variable | Description | Default |
|----------|-------------|---------|
| `HINDSIGHT_API_DATABASE_URL` | PostgreSQL connection string | `pg0` (embedded) |
| `HINDSIGHT_API_LLM_PROVIDER` | `openai`, `anthropic`, `gemini`, `groq`, `ollama`, `lmstudio` | `openai` |
| `HINDSIGHT_API_LLM_PROVIDER` | LLM provider, including `openai`, `anthropic`, `gemini`, `groq`, `ollama`, `lmstudio`, and `github-copilot` | `openai` |
| `HINDSIGHT_API_LLM_API_KEY` | API key for LLM provider | - |
| `HINDSIGHT_API_LLM_MODEL` | Model name | `gpt-4o-mini` |
| `HINDSIGHT_API_HOST` | Server bind address | `0.0.0.0` |
@@ -911,6 +911,7 @@ PROVIDER_DEFAULT_MODELS = {
"vertexai": "google/gemini-3.1-flash-lite",
"openai-codex": "gpt-5.4-mini",
"claude-code": "claude-sonnet-4-5-20250929",
"github-copilot": "gpt-5.6-terra",
"mock": "mock-model",
"none": "none",
"litellm": "gpt-4o-mini",
@@ -264,6 +264,7 @@ _PROVIDERS_WITHOUT_API_KEY = frozenset(
"llamacpp",
"openai-codex",
"claude-code",
"github-copilot",
"mock",
"none",
"vertexai",
@@ -374,6 +375,7 @@ def create_llm_provider(
CodexLLM,
FireworksLLM,
GeminiLLM,
GitHubCopilotLLM,
LiteLLMLLM,
LiteLLMRouterLLM,
LlamaCppLLM,
@@ -410,6 +412,16 @@ def create_llm_provider(
reasoning_effort=reasoning_effort,
)
elif provider_lower == "github-copilot":
return GitHubCopilotLLM(
provider=provider,
api_key=api_key,
base_url=base_url,
model=model,
reasoning_effort=reasoning_effort,
timeout=timeout,
)
elif provider_lower == "mock":
return MockLLM(
provider=provider,
@@ -774,6 +786,7 @@ class LLMProvider:
"vertexai",
"openai-codex",
"claude-code",
"github-copilot",
"mock",
"none",
"minimax",
@@ -1476,9 +1489,7 @@ class LLMProvider:
if not api_key and not requires_api_key(provider):
pass # Provider handles its own auth
elif not api_key:
raise ValueError(
f"{ENV_LLM_API_KEY} environment variable is required (unless using openai-codex, claude-code, or litellm)"
)
raise ValueError(f"{ENV_LLM_API_KEY} environment variable is required for provider '{provider}'")
base_url = os.getenv(ENV_LLM_BASE_URL, "")
model = os.getenv(ENV_LLM_MODEL) or _get_default_model_for_provider(provider)
@@ -9,6 +9,7 @@ from .claude_code_llm import ClaudeCodeLLM
from .codex_llm import CodexLLM
from .fireworks_llm import FireworksLLM
from .gemini_llm import GeminiLLM
from .github_copilot_llm import GitHubCopilotLLM
from .litellm_llm import LiteLLMLLM
from .litellm_router_llm import LiteLLMRouterLLM
from .llamacpp_llm import LlamaCppLLM
@@ -23,6 +24,7 @@ __all__ = [
"CodexLLM",
"FireworksLLM",
"GeminiLLM",
"GitHubCopilotLLM",
"LlamaCppLLM",
"LiteLLMLLM",
"LiteLLMRouterLLM",
@@ -0,0 +1,869 @@
"""
GitHub Copilot LLM provider using the official Copilot SDK.
This provider uses the GitHub identity already authenticated by Copilot CLI, or
one of the SDK-supported GitHub token environment variables. It does not use
``HINDSIGHT_API_LLM_API_KEY``.
"""
from __future__ import annotations
import asyncio
import json
import logging
import os
import shutil
import tempfile
import threading
import time
from collections.abc import Awaitable, Mapping
from contextlib import AbstractAsyncContextManager, nullcontext
from dataclasses import dataclass, field
from pathlib import Path
from typing import TYPE_CHECKING, Any, Callable
from pydantic import ValidationError
from hindsight_api.engine.llm_interface import LLM_TOOL_CHOICE_AUTO, LLMInterface, LLMToolChoice, LLMToolChoiceMode
from hindsight_api.engine.llm_trace import LLMResponseUsage, stash_response_usage
from hindsight_api.engine.response_models import LLMToolCall, LLMToolCallResult, TokenUsage
from hindsight_api.metrics import get_metrics_collector
from hindsight_api.worker.stage import set_stage
logger = logging.getLogger(__name__)
if TYPE_CHECKING:
from copilot import CopilotClient
from copilot.session_events import AssistantMessageData, SessionEvent
from copilot.tools import Tool
_STRUCTURED_TOOL_NAME = "structured_response"
_DEFAULT_TIMEOUT_SECONDS = 120.0
_RUNTIME_CLEANUP_TIMEOUT_SECONDS = 5.0
_SESSION_CLEANUP_TIMEOUT_SECONDS = 2.0
_TEMPLATE_OPENAI_BASE_URL = "https://api.openai.com/v1"
_SUPPORTED_REASONING_EFFORTS = frozenset({"low", "medium", "high", "xhigh", "max"})
@dataclass
class _PromptParts:
system_prompt: str
user_prompt: str
@dataclass
class _TurnCapture:
assistant_messages: list[AssistantMessageData] = field(default_factory=list)
input_tokens: int = 0
output_tokens: int = 0
cached_tokens: int = 0
thoughts_tokens: int = 0
finish_reason: str | None = None
def handle_event(self, event: SessionEvent) -> None:
from copilot.session_events import AssistantMessageData, AssistantUsageData
data = event.data
if isinstance(data, AssistantMessageData):
self.assistant_messages.append(data)
elif isinstance(data, AssistantUsageData):
self.input_tokens += data.input_tokens or 0
self.output_tokens += data.output_tokens or 0
self.cached_tokens += data.cache_read_tokens or 0
self.thoughts_tokens += data.reasoning_tokens or 0
self.finish_reason = data.finish_reason or self.finish_reason
def token_usage(self) -> TokenUsage:
return TokenUsage(
input_tokens=self.input_tokens,
output_tokens=self.output_tokens,
total_tokens=self.input_tokens + self.output_tokens,
cached_tokens=self.cached_tokens,
thoughts_tokens=self.thoughts_tokens,
)
@dataclass
class _InvocationResult:
content: str
tool_calls: list[LLMToolCall]
usage: TokenUsage
finish_reason: str | None
_TOKEN_ENV_VARS = ("COPILOT_GITHUB_TOKEN", "GH_TOKEN", "GITHUB_TOKEN")
def _has_token_credentials() -> bool:
"""Whether the SDK can authenticate without a signed-in Copilot CLI account."""
return any(os.environ.get(name) for name in _TOKEN_ENV_VARS)
def _sync_copilot_auth_metadata(target_home: Path, source_home: Path | None = None) -> None:
"""Copy only the account selector into the isolated runtime home.
The normal Copilot home also contains user hooks, plugins, MCP servers,
skills, and session state. Pointing Hindsight's internal runtime there made
every internal LLM request run the Hindsight hooks again, creating an
explosive recursion loop. ``config.json`` carries the signed-in account
selection, so copying only that file preserves the account choice without
importing executable config. The credential itself is never in this file:
the runtime resolves it from the system keychain or from a `gh` CLI login.
"""
target_home.mkdir(parents=True, exist_ok=True)
source = (source_home or Path.home() / ".copilot") / "config.json"
if not source.is_file():
if _has_token_credentials():
# A headless deployment (container, CI) authenticates from the token
# environment and never runs Copilot CLI, so there is no signed-in
# account on disk to carry over. An empty home is correct there.
logger.info(
"No Copilot account metadata at %s; authenticating from the token environment instead",
source,
)
return
raise RuntimeError(
f"GitHub Copilot account metadata was not found at {source}. "
"Start Copilot CLI and sign in, or set COPILOT_GITHUB_TOKEN, GH_TOKEN, or GITHUB_TOKEN."
)
shutil.copy2(source, target_home / "config.json")
class _SharedCopilotRuntime:
"""One Copilot runtime shared by every Hindsight LLM lane in this process."""
def __init__(self, runtime_url: str) -> None:
self.runtime_url = runtime_url
self.ref_count = 0
self._started = False
self._lifecycle_lock = asyncio.Lock()
self._copilot_home = None if runtime_url else Path(tempfile.mkdtemp(prefix="hindsight-github-copilot-"))
self.client = self._new_client()
def _new_client(self) -> CopilotClient:
from copilot import CopilotClient, RuntimeConnection
connection = RuntimeConnection.for_uri(self.runtime_url) if self.runtime_url else None
base_directory = None
if self._copilot_home is not None:
_sync_copilot_auth_metadata(self._copilot_home)
base_directory = str(self._copilot_home)
return CopilotClient(
connection=connection,
working_directory=tempfile.gettempdir(),
base_directory=base_directory,
use_logged_in_user=True,
session_idle_timeout_seconds=300,
mode="copilot-cli",
)
async def ensure_started(self) -> None:
if self._started:
return
async with self._lifecycle_lock:
if self._started:
return
client = self.client
try:
await client.start()
auth = await client.get_auth_status()
if not auth.isAuthenticated:
raise RuntimeError(
"GitHub Copilot is not authenticated. Start Copilot CLI and sign in, "
"or set COPILOT_GITHUB_TOKEN, GH_TOKEN, or GITHUB_TOKEN."
)
self._started = True
except BaseException:
self.client = self._new_client()
self._started = False
await self._force_stop_client(client)
raise
async def get_client(self) -> CopilotClient:
await self.ensure_started()
return self.client
async def invalidate(self, expected_client: CopilotClient, reason: str) -> None:
"""Replace a dead or wedged runtime without disrupting a newer generation."""
async with self._lifecycle_lock:
if self.client is not expected_client:
return
logger.warning("Resetting shared GitHub Copilot runtime: %s", reason)
self.client = self._new_client()
self._started = False
await self._force_stop_client(expected_client)
@staticmethod
async def _force_stop_client(client: CopilotClient) -> None:
try:
await asyncio.wait_for(client.force_stop(), timeout=_RUNTIME_CLEANUP_TIMEOUT_SECONDS)
except Exception:
logger.warning("Failed to force-stop GitHub Copilot runtime", exc_info=True)
async def stop(self) -> None:
async with self._lifecycle_lock:
if self._started:
client = self.client
self._started = False
try:
await asyncio.wait_for(client.stop(), timeout=_RUNTIME_CLEANUP_TIMEOUT_SECONDS)
except Exception:
logger.warning("GitHub Copilot runtime cleanup failed; forcing shutdown", exc_info=True)
await self._force_stop_client(client)
if self._copilot_home is not None:
try:
shutil.rmtree(self._copilot_home)
except OSError:
logger.warning(
"Failed to remove isolated Copilot home %s",
self._copilot_home,
exc_info=True,
)
_runtime_registry_lock = threading.Lock()
_runtime_registry: dict[str, _SharedCopilotRuntime] = {}
def _acquire_runtime(runtime_url: str) -> _SharedCopilotRuntime:
key = runtime_url.strip().rstrip("/")
with _runtime_registry_lock:
runtime = _runtime_registry.get(key)
if runtime is None:
runtime = _SharedCopilotRuntime(key)
_runtime_registry[key] = runtime
runtime.ref_count += 1
return runtime
def _normalize_runtime_url(base_url: str) -> str:
value = base_url.strip().rstrip("/")
# Older generated .env files actively set the OpenAI default URL. Treat
# that one known template value as unset so switching only provider/model
# works; every other non-empty value remains explicit and is validated by
# the Copilot SDK as a headless runtime URI.
if value == _TEMPLATE_OPENAI_BASE_URL.rstrip("/"):
logger.info("Ignoring the OpenAI template base URL for the github-copilot provider")
return ""
return value
async def _release_runtime(runtime: _SharedCopilotRuntime) -> None:
should_stop = False
with _runtime_registry_lock:
if runtime.ref_count > 0:
runtime.ref_count -= 1
if runtime.ref_count == 0 and _runtime_registry.get(runtime.runtime_url) is runtime:
del _runtime_registry[runtime.runtime_url]
should_stop = True
if should_stop:
await runtime.stop()
def _build_prompt(messages: list[dict[str, Any]]) -> _PromptParts:
system_parts: list[str] = []
conversation: list[dict[str, Any]] = []
for message in messages:
role = str(message.get("role", "user"))
if role == "system":
content = message.get("content")
if isinstance(content, str) and content:
system_parts.append(content)
continue
conversation.append(message)
system_prompt = "\n\n".join(system_parts).strip()
if not system_prompt:
system_prompt = "Act as a stateless language-model backend and follow the supplied conversation."
if (
len(conversation) == 1
and conversation[0].get("role") == "user"
and isinstance(conversation[0].get("content"), str)
and set(conversation[0]).issubset({"role", "content"})
):
user_prompt = conversation[0]["content"]
else:
serialized = json.dumps(conversation, ensure_ascii=False, default=str)
user_prompt = (
"Continue the conversation represented by the JSON messages below. "
"Produce only the next assistant turn.\n\n"
f"<conversation_messages>{serialized}</conversation_messages>"
)
return _PromptParts(system_prompt=system_prompt, user_prompt=user_prompt)
def _normalize_tool_arguments(arguments: Any) -> dict[str, Any]:
if arguments is None:
return {}
if isinstance(arguments, Mapping):
return dict(arguments)
if isinstance(arguments, str):
parsed = json.loads(arguments)
if isinstance(parsed, dict):
return parsed
raise ValueError(f"GitHub Copilot returned non-object tool arguments: {type(arguments).__name__}")
def _is_authentication_error(error: Exception) -> bool:
text = str(error).lower()
return any(
marker in text
for marker in (
"not authenticated",
"authentication failed",
"unauthorized",
"sign in",
"login required",
"invalid token",
)
)
def _is_quota_error(error: Exception) -> bool:
text = str(error).lower()
return any(marker in text for marker in ("quota", "usage limit", "weekly limit", "ai credits"))
def _is_configuration_error(error: BaseException) -> bool:
"""Whether the runtime will reject this request identically on every attempt.
These surface as JsonRpcError, which is otherwise indistinguishable from a
transport failure. Treating one as a runtime failure costs a full Copilot
CLI restart per attempt and can never succeed, so they are terminal.
"""
text = str(error).lower()
# Gate on "model" so a transient outage phrased as "service is not
# available" stays retryable; only the model selection is terminal.
if "model" not in text:
return False
return any(
marker in text
for marker in (
"is not available",
"unknown model",
"not found",
"invalid model",
"unsupported model",
)
)
def _is_runtime_failure(error: BaseException) -> bool:
if _is_configuration_error(error):
return False
if isinstance(error, (TimeoutError, ConnectionError, OSError)):
return True
if type(error).__name__ in {"ProcessExitedError", "JsonRpcError"}:
return True
text = str(error).lower()
return any(
marker in text
for marker in (
"client not connected",
"process exited",
"connection closed",
"connection lost",
"broken pipe",
"transport closed",
"unexpected eof",
)
)
class GitHubCopilotLLM(LLMInterface):
"""LLM provider backed by the authenticated GitHub Copilot runtime."""
def __init__(
self,
provider: str,
api_key: str,
base_url: str,
model: str,
reasoning_effort: str | None = None,
timeout: float | None = None,
**kwargs: Any,
) -> None:
super().__init__(provider, api_key, base_url, model, reasoning_effort, **kwargs)
self._released = False
self._timeout = timeout
if self.reasoning_effort == "none":
self.reasoning_effort = None
elif self.reasoning_effort is not None and self.reasoning_effort not in _SUPPORTED_REASONING_EFFORTS:
raise ValueError(
f"Unsupported GitHub Copilot reasoning effort {self.reasoning_effort!r}. "
f"Use one of: {', '.join(sorted(_SUPPORTED_REASONING_EFFORTS))}, or leave it unset."
)
self.base_url = _normalize_runtime_url(base_url)
self._runtime = _acquire_runtime(self.base_url)
async def verify_connection(self) -> None:
try:
await self._runtime.ensure_started()
await self.call(
messages=[{"role": "user", "content": "Reply with exactly: ok"}],
max_completion_tokens=10,
scope="verification",
max_retries=0,
)
logger.info("GitHub Copilot connection verified successfully")
except Exception as error:
raise RuntimeError(f"GitHub Copilot connection verification failed: {error}") from error
async def _invoke(
self,
messages: list[dict[str, Any]],
sdk_tools: list[Any],
available_tools: list[str],
system_suffix: str = "",
timeout_seconds: float = _DEFAULT_TIMEOUT_SECONDS,
) -> _InvocationResult:
prompt = _build_prompt(messages)
system_prompt = prompt.system_prompt
if system_suffix:
system_prompt = f"{system_prompt}\n\n{system_suffix}"
capture = _TurnCapture()
session = None
session_id: str | None = None
client = None
runtime_invalidated = False
loop = asyncio.get_running_loop()
deadline = loop.time() + timeout_seconds
async def wait_with_deadline(factory: Callable[[], Awaitable[Any]]) -> Any:
remaining = deadline - loop.time()
if remaining <= 0:
raise TimeoutError(f"GitHub Copilot attempt exceeded {timeout_seconds:g}s")
return await asyncio.wait_for(factory(), timeout=remaining)
try:
client = await wait_with_deadline(self._runtime.get_client)
session = await wait_with_deadline(
lambda: client.create_session(
model=self.model,
reasoning_effort=self.reasoning_effort,
tools=sdk_tools,
available_tools=available_tools,
system_message={"mode": "replace", "content": system_prompt},
enable_experimental_mode=False,
enable_session_telemetry=False,
enable_file_change_tracking=False,
skip_custom_instructions=True,
custom_agents_local_only=True,
coauthor_enabled=False,
manage_schedule_enabled=False,
streaming=False,
include_sub_agent_streaming_events=False,
mcp_oauth_token_storage="in-memory",
embedding_cache_storage="in-memory",
enable_config_discovery=False,
skip_embedding_retrieval=True,
enable_on_demand_instruction_discovery=False,
enable_file_hooks=False,
enable_host_git_operations=False,
enable_session_store=False,
enable_skills=False,
memory={"enabled": False},
on_event=capture.handle_event,
)
)
session_id = session.session_id
response = await wait_with_deadline(
lambda: session.send_and_wait(
prompt.user_prompt,
timeout=max(deadline - loop.time(), 0.001),
)
)
if response is not None:
from copilot.session_events import AssistantMessageData
if isinstance(response.data, AssistantMessageData) and response.data not in capture.assistant_messages:
capture.assistant_messages.append(response.data)
tool_calls: list[LLMToolCall] = []
seen_call_ids: set[str] = set()
content = ""
for assistant_message in capture.assistant_messages:
if assistant_message.content:
content = assistant_message.content
for request in assistant_message.tool_requests or []:
if request.tool_call_id in seen_call_ids:
continue
seen_call_ids.add(request.tool_call_id)
tool_calls.append(
LLMToolCall(
id=request.tool_call_id,
name=request.name,
arguments=_normalize_tool_arguments(request.arguments),
)
)
return _InvocationResult(
content=content,
tool_calls=tool_calls,
usage=capture.token_usage(),
finish_reason=capture.finish_reason,
)
except asyncio.CancelledError:
if client is not None:
await self._runtime.invalidate(client, "attempt was cancelled")
runtime_invalidated = True
raise
except Exception as error:
if client is not None and _is_runtime_failure(error):
await self._runtime.invalidate(client, str(error))
runtime_invalidated = True
raise
finally:
if session is not None and client is not None and not runtime_invalidated:
cleanup_ok = await self._cleanup_session(
client=client,
session=session,
session_id=session_id,
deadline=deadline,
)
if not cleanup_ok:
await self._runtime.invalidate(client, "transient session cleanup timed out or failed")
def _timeout_seconds(self) -> float:
return self._timeout if self._timeout is not None else _DEFAULT_TIMEOUT_SECONDS
@staticmethod
async def _cleanup_session(
client: CopilotClient,
session: Any,
session_id: str | None,
deadline: float,
) -> bool:
loop = asyncio.get_running_loop()
async def cleanup_step(factory: Callable[[], Awaitable[Any]]) -> None:
remaining = deadline - loop.time()
if remaining <= 0:
raise TimeoutError("GitHub Copilot attempt deadline expired before session cleanup")
await asyncio.wait_for(
factory(),
timeout=min(remaining, _SESSION_CLEANUP_TIMEOUT_SECONDS),
)
try:
await cleanup_step(session.disconnect)
if session_id is not None:
await cleanup_step(lambda: client.delete_session(session_id))
return True
except Exception:
logger.warning("Failed to clean up transient Copilot session %s", session_id, exc_info=True)
return False
@staticmethod
def _terminal_tool(name: str, description: str, parameters: dict[str, Any]) -> Tool:
from copilot.tools import Tool, ToolInvocation, ToolResult
async def capture_tool(_invocation: ToolInvocation) -> ToolResult:
# The assistant.message event carries the full tool request. Marking
# this tool terminal prevents Copilot's own agent loop from feeding
# this placeholder result back to the model; Hindsight executes the
# real tool in its outer reflect loop.
return ToolResult(text_result_for_llm="Tool call captured.", result_type="success")
return Tool(
name=name,
description=description,
parameters=parameters,
handler=capture_tool,
skip_permission=True,
defer="never",
is_terminal=True,
)
async def call(
self,
messages: list[dict[str, str]],
response_format: Any | None = None,
max_completion_tokens: int | None = None,
temperature: float | None = None,
scope: str = "memory",
max_retries: int = 10,
initial_backoff: float = 1.0,
max_backoff: float = 60.0,
skip_validation: bool = False,
strict_schema: bool = False,
return_usage: bool = False,
cached_prefix: str | None = None,
attempt_context: Callable[[], AbstractAsyncContextManager[None]] | None = None,
) -> Any:
start_time = time.time()
for attempt in range(max_retries + 1):
try:
sdk_tools: list[Any] = []
available_tools: list[str] = []
system_suffix = ""
if response_format is not None:
schema = response_format.model_json_schema()
sdk_tools = [
self._terminal_tool(
_STRUCTURED_TOOL_NAME,
"Return the structured response matching the required schema.",
schema,
)
]
available_tools = [f"custom:{_STRUCTURED_TOOL_NAME}"]
system_suffix = (
f"You MUST call the {_STRUCTURED_TOOL_NAME!r} tool exactly once. Do not answer with prose."
)
async with attempt_context() if attempt_context is not None else nullcontext():
set_stage(f"llm.github_copilot.{scope}.attempt={attempt + 1}/{max_retries + 1}")
invocation = await self._invoke(
messages=list(messages),
sdk_tools=sdk_tools,
available_tools=available_tools,
system_suffix=system_suffix,
timeout_seconds=self._timeout_seconds(),
)
stash_response_usage(
LLMResponseUsage(
input_tokens=invocation.usage.input_tokens,
output_tokens=invocation.usage.output_tokens,
cached_tokens=invocation.usage.cached_tokens,
)
)
if response_format is not None:
structured_call = next(
(call for call in invocation.tool_calls if call.name == _STRUCTURED_TOOL_NAME),
None,
)
if structured_call is None:
raise RuntimeError("GitHub Copilot did not return the required structured_response tool call")
result = (
structured_call.arguments
if skip_validation
else response_format.model_validate(structured_call.arguments)
)
else:
if not invocation.content:
raise RuntimeError("GitHub Copilot returned an empty response")
result = invocation.content
duration = time.time() - start_time
self._record_success(
messages=messages,
result=result,
invocation=invocation,
scope=scope,
duration=duration,
)
if return_usage:
return result, invocation.usage
return result
except ValidationError:
raise
except Exception as error:
if _is_configuration_error(error):
raise RuntimeError(f"GitHub Copilot rejected the request configuration: {error}") from error
if _is_authentication_error(error):
raise RuntimeError(
"GitHub Copilot authentication failed. Start Copilot CLI and sign in, "
"or set COPILOT_GITHUB_TOKEN, GH_TOKEN, or GITHUB_TOKEN. "
"HINDSIGHT_API_LLM_API_KEY is not used by this provider."
) from error
if _is_quota_error(error):
raise RuntimeError(f"GitHub Copilot usage limit reached: {error}") from error
if attempt >= max_retries:
raise
backoff = min(initial_backoff * (2**attempt), max_backoff)
logger.warning(
"GitHub Copilot error (attempt %d/%d): %s",
attempt + 1,
max_retries + 1,
error,
)
await asyncio.sleep(backoff)
raise RuntimeError("GitHub Copilot call failed after all retries")
async def call_with_tools(
self,
messages: list[dict[str, Any]],
tools: list[dict[str, Any]],
max_completion_tokens: int | None = None,
temperature: float | None = None,
scope: str = "tools",
max_retries: int = 5,
initial_backoff: float = 1.0,
max_backoff: float = 30.0,
tool_choice: LLMToolChoice = LLM_TOOL_CHOICE_AUTO,
cached_prefix: str | None = None,
cached_prefix_message_count: int = 0,
attempt_context: Callable[[], AbstractAsyncContextManager[None]] | None = None,
) -> LLMToolCallResult:
start_time = time.time()
for attempt in range(max_retries + 1):
try:
selected_tools = tools
system_suffix = ""
if tool_choice.mode is LLMToolChoiceMode.NONE:
selected_tools = []
elif tool_choice.mode is LLMToolChoiceMode.NAMED:
selected_name = tool_choice.selected_function_name
selected_tools = [tool for tool in tools if tool.get("function", {}).get("name") == selected_name]
if not selected_tools:
raise ValueError(f"Requested GitHub Copilot tool {selected_name!r} is not available")
system_suffix = f"You MUST call the {selected_name!r} tool. Do not answer with prose."
elif tool_choice.mode is LLMToolChoiceMode.REQUIRED:
system_suffix = "You MUST call at least one available tool. Do not answer with prose."
sdk_tools = []
available_tools = []
for tool in selected_tools:
function = tool.get("function", {})
name = function.get("name", "")
if not name:
continue
sdk_tools.append(
self._terminal_tool(
name=name,
description=function.get("description", ""),
parameters=function.get("parameters", {}),
)
)
available_tools.append(f"custom:{name}")
async with attempt_context() if attempt_context is not None else nullcontext():
set_stage(f"llm.github_copilot.tools.attempt={attempt + 1}/{max_retries + 1}")
invocation = await self._invoke(
messages=messages,
sdk_tools=sdk_tools,
available_tools=available_tools,
system_suffix=system_suffix,
timeout_seconds=self._timeout_seconds(),
)
stash_response_usage(
LLMResponseUsage(
input_tokens=invocation.usage.input_tokens,
output_tokens=invocation.usage.output_tokens,
cached_tokens=invocation.usage.cached_tokens,
)
)
if tool_choice.mode is LLMToolChoiceMode.NAMED:
selected_name = tool_choice.selected_function_name
if not any(call.name == selected_name for call in invocation.tool_calls):
raise RuntimeError(f"GitHub Copilot did not call the required {selected_name!r} tool")
elif tool_choice.mode is LLMToolChoiceMode.REQUIRED and not invocation.tool_calls:
raise RuntimeError("GitHub Copilot did not call any tool when a tool call was required")
duration = time.time() - start_time
result = LLMToolCallResult(
content=invocation.content or None,
tool_calls=invocation.tool_calls,
finish_reason="tool_calls" if invocation.tool_calls else invocation.finish_reason or "stop",
input_tokens=invocation.usage.input_tokens,
output_tokens=invocation.usage.output_tokens,
cached_tokens=invocation.usage.cached_tokens,
thoughts_tokens=invocation.usage.thoughts_tokens,
)
self._record_success(
messages=messages,
result=result,
invocation=invocation,
scope=scope,
duration=duration,
)
return result
except Exception as error:
if _is_configuration_error(error):
raise RuntimeError(f"GitHub Copilot rejected the request configuration: {error}") from error
if _is_authentication_error(error):
raise RuntimeError(
"GitHub Copilot authentication failed. Start Copilot CLI and sign in, "
"or set COPILOT_GITHUB_TOKEN, GH_TOKEN, or GITHUB_TOKEN."
) from error
if _is_quota_error(error):
raise RuntimeError(f"GitHub Copilot usage limit reached: {error}") from error
if attempt >= max_retries:
raise
backoff = min(initial_backoff * (2**attempt), max_backoff)
logger.warning(
"GitHub Copilot tool-call error (attempt %d/%d): %s",
attempt + 1,
max_retries + 1,
error,
)
await asyncio.sleep(backoff)
raise RuntimeError("GitHub Copilot tool call failed after all retries")
def _record_success(
self,
messages: list[dict[str, Any]],
result: Any,
invocation: _InvocationResult,
scope: str,
duration: float,
) -> None:
usage = invocation.usage
metrics = get_metrics_collector()
metrics.record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
duration=duration,
input_tokens=usage.input_tokens,
output_tokens=usage.output_tokens,
cached_input_tokens=usage.cached_tokens,
thoughts_tokens=usage.thoughts_tokens,
success=True,
)
try:
from hindsight_api.tracing import _serialize_for_span, get_span_recorder
get_span_recorder().record_llm_call(
provider=self.provider,
model=self.model,
scope=scope,
messages=messages,
response_content=_serialize_for_span(result),
input_tokens=usage.input_tokens,
output_tokens=usage.output_tokens,
cached_tokens=usage.cached_tokens,
duration=duration,
finish_reason=invocation.finish_reason,
error=None,
tool_calls=[
{"id": call.id, "name": call.name, "arguments": call.arguments} for call in invocation.tool_calls
]
or None,
)
except Exception:
logger.debug("GitHub Copilot span recording failed", exc_info=True)
if duration > 10.0:
logger.info(
"slow llm call: scope=%s, model=%s/%s, time=%.3fs",
scope,
self.provider,
self.model,
duration,
)
async def cleanup(self) -> None:
if self._released:
return
self._released = True
await _release_runtime(self._runtime)
def supports_attempt_scoped_concurrency(self) -> bool:
return True
@@ -141,6 +141,7 @@ PROVIDER_NAME_MAPPING = {
"lmstudio": "lmstudio",
"openai-codex": "openai",
"claude-code": "anthropic",
"github-copilot": "github",
"mock": "mock",
}
+1
View File
@@ -95,6 +95,7 @@ dependencies = [
"aiohttp>=3.14.3", # GHSA-cq5v-8q36-5273: OOB heap read in the C response parser, plus earlier DoS fixes
"pygments>=2.20.0", # ReDoS via inefficient GUID regex fix
"claude-agent-sdk>=0.2.82",
"github-copilot-sdk>=1.0.11",
"boto3>=1.42.74",
"croniter>=2.0.0", # Cron parsing for scheduled mental model refresh
"json-repair>=0.63.2", # Structural repair of malformed LLM JSON (last-resort parse fallback); >=0.60.1 also fixes the circular-$ref unbounded-CPU DoS
@@ -0,0 +1,601 @@
from __future__ import annotations
import asyncio
import time
from types import SimpleNamespace
from unittest.mock import AsyncMock, MagicMock
import pytest
from pydantic import BaseModel
from hindsight_api.config import PROVIDER_DEFAULT_MODELS
from hindsight_api.engine.llm_interface import LLM_TOOL_CHOICE_REQUIRED, LLMToolChoice
from hindsight_api.engine.llm_wrapper import create_llm_provider, requires_api_key
from hindsight_api.engine.providers import github_copilot_llm as provider_module
from hindsight_api.engine.providers.github_copilot_llm import GitHubCopilotLLM
class _StructuredAnswer(BaseModel):
answer: str
class _FakeSession:
def __init__(self, on_event, events, *, error: Exception | None = None) -> None:
self.session_id = "copilot-session"
self._on_event = on_event
self._events = events
self._error = error
self.disconnected = False
self.aborted = False
async def send_and_wait(self, _prompt: str, *, timeout: float):
if self._error is not None:
raise self._error
for event in self._events:
self._on_event(event)
assistant_events = [event for event in self._events if event.data.__class__.__name__ == "AssistantMessageData"]
return assistant_events[-1] if assistant_events else None
async def disconnect(self) -> None:
self.disconnected = True
async def abort(self) -> None:
self.aborted = True
class _FakeClient:
def __init__(self, events, *, error: Exception | None = None) -> None:
self.events = events
self.error = error
self.create_kwargs = None
self.create_count = 0
self.session = None
self.deleted_sessions: list[str] = []
async def create_session(self, **kwargs):
self.create_count += 1
self.create_kwargs = kwargs
self.session = _FakeSession(kwargs["on_event"], self.events, error=self.error)
return self.session
async def delete_session(self, session_id: str) -> None:
self.deleted_sessions.append(session_id)
class _FakeRuntime:
def __init__(self, client: _FakeClient) -> None:
self.client = client
self.runtime_url = ""
self.ref_count = 1
self.started = False
self.invalidations: list[str] = []
async def ensure_started(self) -> None:
self.started = True
async def get_client(self):
self.started = True
return self.client
async def invalidate(self, _expected_client, reason: str) -> None:
self.invalidations.append(reason)
async def stop(self) -> None:
self.started = False
def _assistant_event(content: str, tool_requests=None):
from copilot.session_events import AssistantMessageData
return SimpleNamespace(
data=AssistantMessageData(
content=content,
message_id="message-1",
tool_requests=tool_requests,
)
)
def _usage_event():
from copilot.session_events import AssistantUsageData
return SimpleNamespace(
data=AssistantUsageData(
model="gpt-5.6-terra",
input_tokens=120,
output_tokens=30,
cache_read_tokens=20,
reasoning_tokens=10,
finish_reason="stop",
)
)
def _provider(monkeypatch, events, *, error: Exception | None = None) -> tuple[GitHubCopilotLLM, _FakeClient]:
client = _FakeClient(events, error=error)
runtime = _FakeRuntime(client)
monkeypatch.setattr(provider_module, "_acquire_runtime", lambda _url: runtime)
provider = GitHubCopilotLLM(
provider="github-copilot",
api_key="",
base_url="",
model="gpt-5.6-terra",
timeout=15,
)
return provider, client
def test_provider_registration_and_default_model(monkeypatch):
assert requires_api_key("github-copilot") is False
assert PROVIDER_DEFAULT_MODELS["github-copilot"] == "gpt-5.6-terra"
runtime = _FakeRuntime(_FakeClient([]))
monkeypatch.setattr(provider_module, "_acquire_runtime", lambda _url: runtime)
provider = create_llm_provider(
provider="github-copilot",
api_key="",
base_url="",
model="gpt-5.6-terra",
reasoning_effort=None,
)
assert isinstance(provider, GitHubCopilotLLM)
def test_openai_template_base_url_is_ignored(monkeypatch):
runtime = _FakeRuntime(_FakeClient([]))
acquired_urls: list[str] = []
def acquire(url: str):
acquired_urls.append(url)
return runtime
monkeypatch.setattr(provider_module, "_acquire_runtime", acquire)
provider = GitHubCopilotLLM(
provider="github-copilot",
api_key="",
base_url="https://api.openai.com/v1",
model="gpt-5.6-terra",
)
assert provider.base_url == ""
assert acquired_urls == [""]
def test_isolated_copilot_home_copies_only_account_metadata(tmp_path):
source_home = tmp_path / "source"
target_home = tmp_path / "target"
source_home.mkdir()
(source_home / "config.json").write_text(
'{"lastLoggedInUser":{"host":"https://github.com","login":"user"}}',
encoding="utf-8",
)
(source_home / "hooks").mkdir()
(source_home / "hooks" / "dangerous.json").write_text("{}", encoding="utf-8")
provider_module._sync_copilot_auth_metadata(target_home, source_home)
assert (target_home / "config.json").read_text(encoding="utf-8") == (source_home / "config.json").read_text(
encoding="utf-8"
)
assert not (target_home / "hooks").exists()
def test_invalid_reasoning_effort_does_not_acquire_runtime(monkeypatch):
acquire = MagicMock()
monkeypatch.setattr(provider_module, "_acquire_runtime", acquire)
with pytest.raises(ValueError, match="Unsupported GitHub Copilot reasoning effort"):
GitHubCopilotLLM(
provider="github-copilot",
api_key="",
base_url="",
model="gpt-5.6-terra",
reasoning_effort="extreme",
)
acquire.assert_not_called()
@pytest.mark.asyncio
async def test_plain_call_isolates_session_and_reports_usage(monkeypatch):
provider, client = _provider(monkeypatch, [_assistant_event("ok"), _usage_event()])
result, usage = await provider.call(
messages=[
{"role": "system", "content": "Reply concisely."},
{"role": "user", "content": "Say ok."},
],
return_usage=True,
max_retries=0,
)
assert result == "ok"
assert usage.input_tokens == 120
assert usage.output_tokens == 30
assert usage.cached_tokens == 20
assert usage.thoughts_tokens == 10
assert client.create_kwargs["model"] == "gpt-5.6-terra"
assert client.create_kwargs["available_tools"] == []
assert client.create_kwargs["system_message"] == {
"mode": "replace",
"content": "Reply concisely.",
}
assert client.create_kwargs["enable_file_hooks"] is False
assert client.create_kwargs["enable_config_discovery"] is False
assert client.create_kwargs["skip_custom_instructions"] is True
assert client.create_kwargs["enable_session_store"] is False
assert client.create_kwargs["enable_skills"] is False
assert client.create_kwargs["memory"] == {"enabled": False}
assert client.session.disconnected is True
assert client.deleted_sessions == ["copilot-session"]
@pytest.mark.asyncio
async def test_structured_call_uses_terminal_schema_tool(monkeypatch):
from copilot.session_events import AssistantMessageToolRequest
request = AssistantMessageToolRequest(
name="structured_response",
tool_call_id="structured-1",
arguments={"answer": "captured"},
)
provider, client = _provider(
monkeypatch,
[_assistant_event("", [request]), _usage_event()],
)
result = await provider.call(
messages=[{"role": "user", "content": "Return an answer."}],
response_format=_StructuredAnswer,
max_retries=0,
)
assert result == _StructuredAnswer(answer="captured")
tool = client.create_kwargs["tools"][0]
assert tool.name == "structured_response"
assert tool.parameters == _StructuredAnswer.model_json_schema()
assert tool.is_terminal is True
assert tool.skip_permission is True
assert client.create_kwargs["available_tools"] == ["custom:structured_response"]
@pytest.mark.asyncio
async def test_named_tool_choice_exposes_only_requested_tool(monkeypatch):
from copilot.session_events import AssistantMessageToolRequest
request = AssistantMessageToolRequest(
name="recall",
tool_call_id="recall-1",
arguments={"query": "project decisions"},
)
provider, client = _provider(
monkeypatch,
[_assistant_event("", [request]), _usage_event()],
)
tools = [
{
"type": "function",
"function": {
"name": "recall",
"description": "Recall memories",
"parameters": {
"type": "object",
"properties": {"query": {"type": "string"}},
"required": ["query"],
},
},
},
{
"type": "function",
"function": {
"name": "done",
"description": "Finish",
"parameters": {"type": "object", "properties": {}},
},
},
]
result = await provider.call_with_tools(
messages=[{"role": "user", "content": "Find project decisions."}],
tools=tools,
tool_choice=LLMToolChoice.named("recall"),
max_retries=0,
)
assert result.finish_reason == "tool_calls"
assert result.tool_calls[0].name == "recall"
assert result.tool_calls[0].arguments == {"query": "project decisions"}
assert [tool.name for tool in client.create_kwargs["tools"]] == ["recall"]
assert client.create_kwargs["available_tools"] == ["custom:recall"]
assert "MUST call the 'recall' tool" in client.create_kwargs["system_message"]["content"]
@pytest.mark.asyncio
async def test_named_tool_choice_retries_then_rejects_prose(monkeypatch):
provider, client = _provider(monkeypatch, [_assistant_event("I will answer directly."), _usage_event()])
tools = [
{
"type": "function",
"function": {
"name": "recall",
"description": "Recall memories",
"parameters": {"type": "object", "properties": {}},
},
}
]
with pytest.raises(RuntimeError, match="did not call the required 'recall' tool"):
await provider.call_with_tools(
messages=[{"role": "user", "content": "Recall first."}],
tools=tools,
tool_choice=LLMToolChoice.named("recall"),
max_retries=1,
initial_backoff=0,
)
assert client.create_count == 2
@pytest.mark.asyncio
async def test_required_tool_choice_rejects_prose(monkeypatch):
provider, _client = _provider(monkeypatch, [_assistant_event("No tool needed."), _usage_event()])
tools = [
{
"type": "function",
"function": {
"name": "recall",
"description": "Recall memories",
"parameters": {"type": "object", "properties": {}},
},
}
]
with pytest.raises(RuntimeError, match="did not call any tool"):
await provider.call_with_tools(
messages=[{"role": "user", "content": "Use a tool."}],
tools=tools,
tool_choice=LLM_TOOL_CHOICE_REQUIRED,
max_retries=0,
)
@pytest.mark.asyncio
async def test_timeout_aborts_and_deletes_session(monkeypatch):
provider, client = _provider(monkeypatch, [], error=TimeoutError("slow"))
with pytest.raises(TimeoutError, match="slow"):
await provider.call(
messages=[{"role": "user", "content": "wait"}],
max_retries=0,
)
assert provider._runtime.invalidations == ["slow"]
assert client.session.disconnected is False
assert client.deleted_sessions == []
@pytest.mark.asyncio
async def test_attempt_timeout_includes_runtime_startup(monkeypatch):
client = _FakeClient([])
class _SlowRuntime(_FakeRuntime):
async def get_client(self):
await asyncio.sleep(1)
return self.client
runtime = _SlowRuntime(client)
monkeypatch.setattr(provider_module, "_acquire_runtime", lambda _url: runtime)
provider = GitHubCopilotLLM(
provider="github-copilot",
api_key="",
base_url="",
model="gpt-5.6-terra",
timeout=0.01,
)
started = time.monotonic()
with pytest.raises(TimeoutError):
await provider.call(
messages=[{"role": "user", "content": "wait"}],
max_retries=0,
)
assert time.monotonic() - started < 0.2
@pytest.mark.asyncio
async def test_runtime_failure_invalidates_shared_client(monkeypatch):
provider, _client = _provider(monkeypatch, [], error=RuntimeError("Client not connected"))
with pytest.raises(RuntimeError, match="Client not connected"):
await provider.call(
messages=[{"role": "user", "content": "hello"}],
max_retries=0,
)
assert provider._runtime.invalidations == ["Client not connected"]
@pytest.mark.asyncio
async def test_session_cleanup_timeout_invalidates_runtime(monkeypatch):
client = _FakeClient([_assistant_event("ok"), _usage_event()])
class _SlowCleanupSession(_FakeSession):
async def disconnect(self) -> None:
await asyncio.sleep(1)
async def create_session(**kwargs):
client.create_count += 1
client.create_kwargs = kwargs
client.session = _SlowCleanupSession(kwargs["on_event"], client.events)
return client.session
client.create_session = create_session
runtime = _FakeRuntime(client)
monkeypatch.setattr(provider_module, "_acquire_runtime", lambda _url: runtime)
provider = GitHubCopilotLLM(
provider="github-copilot",
api_key="",
base_url="",
model="gpt-5.6-terra",
timeout=0.05,
)
started = time.monotonic()
assert (
await provider.call(
messages=[{"role": "user", "content": "say ok"}],
max_retries=0,
)
== "ok"
)
assert time.monotonic() - started < 0.2
assert runtime.invalidations == ["transient session cleanup timed out or failed"]
@pytest.mark.asyncio
async def test_shared_runtime_invalidation_replaces_dead_client(monkeypatch, tmp_path):
old_client = MagicMock()
old_client.force_stop = AsyncMock()
new_client = MagicMock()
clients = iter([old_client, new_client])
monkeypatch.setattr(
provider_module.tempfile,
"mkdtemp",
lambda **_kwargs: str(tmp_path / "runtime-home"),
)
monkeypatch.setattr(
provider_module._SharedCopilotRuntime,
"_new_client",
lambda _self: next(clients),
)
runtime = provider_module._SharedCopilotRuntime("")
runtime._started = True
await runtime.invalidate(old_client, "connection closed")
assert runtime.client is new_client
assert runtime._started is False
old_client.force_stop.assert_awaited_once()
@pytest.mark.asyncio
async def test_shared_runtime_stops_only_after_last_release(monkeypatch, tmp_path):
with provider_module._runtime_registry_lock:
provider_module._runtime_registry.clear()
monkeypatch.setattr(
provider_module.tempfile,
"mkdtemp",
lambda **_kwargs: str(tmp_path / "runtime-home"),
)
monkeypatch.setattr(
provider_module._SharedCopilotRuntime,
"_new_client",
lambda _self: MagicMock(),
)
runtime_a = provider_module._acquire_runtime("")
runtime_b = provider_module._acquire_runtime("")
runtime_a.stop = AsyncMock()
assert runtime_a is runtime_b
assert runtime_a.ref_count == 2
await provider_module._release_runtime(runtime_a)
runtime_a.stop.assert_not_awaited()
await provider_module._release_runtime(runtime_b)
runtime_a.stop.assert_awaited_once()
def test_prompt_serializes_tool_history():
prompt = provider_module._build_prompt(
[
{"role": "system", "content": "Use memory tools."},
{"role": "user", "content": "What changed?"},
{
"role": "assistant",
"tool_calls": [
{
"id": "call-1",
"type": "function",
"function": {"name": "recall", "arguments": '{"query":"changes"}'},
}
],
},
{"role": "tool", "tool_call_id": "call-1", "content": '{"memories":["A"]}'},
]
)
assert prompt.system_prompt == "Use memory tools."
assert '"tool_call_id": "call-1"' in prompt.user_prompt
assert '"name": "recall"' in prompt.user_prompt
class JsonRpcError(RuntimeError):
"""Stand-in for the SDK error class, which is matched by name."""
@pytest.mark.asyncio
async def test_configuration_error_is_terminal_and_spares_the_runtime(monkeypatch):
"""An unavailable model is rejected identically forever.
It arrives as JsonRpcError, which _is_runtime_failure would otherwise treat
as a dead transport restarting the Copilot CLI once per attempt for an
error that can never succeed.
"""
error = JsonRpcError('Request session.create failed with message: Model "gpt-9" is not available.')
provider, client = _provider(monkeypatch, [], error=error)
with pytest.raises(RuntimeError, match="rejected the request configuration"):
await provider.call(messages=[{"role": "user", "content": "hi"}], max_retries=3, initial_backoff=0)
assert client.create_count == 1
assert provider._runtime.invalidations == []
@pytest.mark.asyncio
async def test_configuration_error_is_terminal_for_tool_calls(monkeypatch):
error = JsonRpcError("Model is not available.")
provider, client = _provider(monkeypatch, [], error=error)
with pytest.raises(RuntimeError, match="rejected the request configuration"):
await provider.call_with_tools(
messages=[{"role": "user", "content": "hi"}],
tools=[{"type": "function", "function": {"name": "noop", "parameters": {}}}],
max_retries=3,
initial_backoff=0,
)
assert client.create_count == 1
assert provider._runtime.invalidations == []
def test_missing_account_metadata_falls_back_to_token_credentials(monkeypatch, tmp_path):
"""A container that never ran Copilot CLI authenticates from the token env."""
monkeypatch.delenv("COPILOT_GITHUB_TOKEN", raising=False)
monkeypatch.delenv("GH_TOKEN", raising=False)
monkeypatch.setenv("GITHUB_TOKEN", "ghp_token")
target = tmp_path / "isolated-home"
provider_module._sync_copilot_auth_metadata(target, source_home=tmp_path / "absent")
assert target.is_dir()
assert not (target / "config.json").exists()
def test_missing_account_metadata_without_credentials_raises(monkeypatch, tmp_path):
for name in provider_module._TOKEN_ENV_VARS:
monkeypatch.delenv(name, raising=False)
with pytest.raises(RuntimeError, match="COPILOT_GITHUB_TOKEN"):
provider_module._sync_copilot_auth_metadata(tmp_path / "isolated-home", source_home=tmp_path / "absent")
def test_transient_service_outage_is_not_mistaken_for_a_config_error():
""" "is not available" alone must stay retryable — only the model is terminal."""
assert provider_module._is_configuration_error(JsonRpcError("Copilot service is not available.")) is False
assert provider_module._is_runtime_failure(JsonRpcError("Copilot service is not available.")) is True
assert provider_module._is_configuration_error(JsonRpcError('Model "gpt-9" is not available.')) is True
@@ -0,0 +1,33 @@
from pathlib import Path
from runpy import run_path
def test_all_copied_llm_registries_allow_github_copilot_without_an_api_key(monkeypatch):
root = Path(__file__).resolve().parents[2]
registry_files = sorted((root / "hindsight-integrations").glob("**/llm.py"))
monkeypatch.setenv("HINDSIGHT_API_LLM_PROVIDER", "github-copilot")
monkeypatch.setenv("HINDSIGHT_API_LLM_MODEL", "gpt-5.6-terra")
monkeypatch.delenv("HINDSIGHT_API_LLM_API_KEY", raising=False)
assert registry_files
for path in registry_files:
namespace = run_path(str(path))
no_key_required = namespace.get("NO_KEY_REQUIRED")
assert isinstance(no_key_required, set), f"{path} has no no-key provider registry"
assert "github-copilot" in no_key_required, f"{path} does not register github-copilot"
detected = namespace["detect_llm_config"]({})
assert detected["provider"] == "github-copilot"
assert detected["api_key"] == ""
assert detected["model"] == "gpt-5.6-terra"
def test_openclaw_no_key_registries_include_github_copilot():
root = Path(__file__).resolve().parents[2]
paths = [
root / "hindsight-integrations" / "openclaw" / "src" / "index.ts",
root / "hindsight-integrations" / "openclaw" / "src" / "setup-lib.ts",
]
for path in paths:
assert '"github-copilot"' in path.read_text(encoding="utf-8")
+1
View File
@@ -31,6 +31,7 @@ def test_provider_name_mapping():
assert PROVIDER_NAME_MAPPING["ollama"] == "ollama"
assert PROVIDER_NAME_MAPPING["openai-codex"] == "openai"
assert PROVIDER_NAME_MAPPING["claude-code"] == "anthropic"
assert PROVIDER_NAME_MAPPING["github-copilot"] == "github"
def test_truncate_content_short():
+2 -2
View File
@@ -80,8 +80,8 @@ Configure via environment variables:
| Variable | Description | Default |
|----------|-------------|---------|
| `HINDSIGHT_API_DATABASE_URL` | PostgreSQL connection string | `pg0` (embedded) |
| `HINDSIGHT_API_LLM_PROVIDER` | `openai`, `anthropic`, `gemini`, `groq`, `ollama`, `lmstudio` | `openai` |
| `HINDSIGHT_API_LLM_API_KEY` | API key for LLM provider | - |
| `HINDSIGHT_API_LLM_PROVIDER` | LLM provider, including `openai`, `anthropic`, `gemini`, `groq`, `ollama`, `lmstudio`, and `github-copilot` | `openai` |
| `HINDSIGHT_API_LLM_API_KEY` | API key for providers that require one; unused by `github-copilot` | - |
| `HINDSIGHT_API_LLM_MODEL` | Model name | `gpt-4o-mini` |
| `HINDSIGHT_API_HOST` | Server bind address | `0.0.0.0` |
| `HINDSIGHT_API_PORT` | Server port | `8888` |
@@ -213,8 +213,8 @@ For non-English banks (especially CJK) and the language/extraction-language trad
| Variable | Description | Default |
|----------|-------------|---------|
| `HINDSIGHT_API_LLM_PROVIDER` | Provider: `openai`, `openai-responses`, `openai-codex`, `claude-code`, `anthropic`, `gemini`, `groq`, `minimax`, `deepseek`, `zai`, `opencode-go`, `nous`, `xai-oauth`, `fireworks`, `ollama`, `ollama-cloud`, `lmstudio`, `llamacpp`, `vertexai`, `bedrock`, `litellm`, `litellmrouter`, `volcano`, `openrouter`, `requesty`, `none` | `openai` |
| `HINDSIGHT_API_LLM_API_KEY` | API key for LLM provider | - |
| `HINDSIGHT_API_LLM_PROVIDER` | Provider: `openai`, `openai-responses`, `openai-codex`, `claude-code`, `github-copilot`, `anthropic`, `gemini`, `groq`, `minimax`, `deepseek`, `zai`, `opencode-go`, `nous`, `xai-oauth`, `fireworks`, `ollama`, `ollama-cloud`, `lmstudio`, `llamacpp`, `vertexai`, `bedrock`, `litellm`, `litellmrouter`, `volcano`, `openrouter`, `requesty`, `none` | `openai` |
| `HINDSIGHT_API_LLM_API_KEY` | API key for providers that require one; unused by `github-copilot` | - |
| `HINDSIGHT_API_LLM_MODEL` | Model name | `gpt-5-mini` |
| `HINDSIGHT_API_LLM_BASE_URL` | Custom LLM endpoint | Provider default |
| `HINDSIGHT_API_LLM_MAX_CONCURRENT` | Max concurrent LLM requests | `32` |
+1 -1
View File
@@ -11,7 +11,7 @@ Guide to setting up a local development environment for contributing to Hindsigh
- Python 3.11+
- [uv](https://docs.astral.sh/uv/) - Fast Python package manager
- Docker and Docker Compose
- An LLM API key (OpenAI, Groq, or Ollama)
- An LLM provider credential, or a local/subscription-backed provider such as Ollama or GitHub Copilot
## Local Development Setup
+28
View File
@@ -268,6 +268,34 @@ export HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=your-gcp-project-id
---
### GitHub Copilot Setup
Use a GitHub Copilot subscription for Hindsight's extraction, consolidation, and reflection calls through the official GitHub Copilot SDK.
**Prerequisites:**
- An active GitHub Copilot entitlement
- Copilot CLI signed in under the same operating-system user that runs Hindsight
**Configure Hindsight:**
```bash
export HINDSIGHT_API_LLM_PROVIDER=github-copilot
export HINDSIGHT_API_LLM_MODEL=gpt-5.6-terra
# No HINDSIGHT_API_LLM_API_KEY is needed.
```
The provider starts one shared Copilot runtime for all Hindsight LLM lanes. That runtime uses a hook-free temporary `COPILOT_HOME` containing only the signed-in account selection; the credential itself is resolved by the runtime from the system keychain or an existing `gh` CLI login. Each call then uses an isolated transient session with repository instructions, skills, Copilot memory, built-in tools, and the cross-session store disabled. This prevents the memory integration from recursively retaining its own extraction calls.
For automation, the Copilot SDK also accepts `COPILOT_GITHUB_TOKEN`, `GH_TOKEN`, or `GITHUB_TOKEN`. With one of those set, Copilot CLI never has to have been run on the host, so containers and CI images work with no `~/.copilot` at all. GitHub Actions and server-to-server deployments must have the appropriate Copilot organization policy and token permissions.
**Important notes:**
- Usage counts against the authenticated account or organization's Copilot allowance and AI Credits.
- `HINDSIGHT_API_LLM_BASE_URL` optionally points to an existing headless Copilot runtime, such as `http://127.0.0.1:4321`; it is not an LLM-provider endpoint for this provider.
- GitHub-hosted Copilot sessions do not expose temperature or maximum-output-token controls through the SDK, so those Hindsight settings are not applied.
- Embeddings and reranking continue to use Hindsight's separately configured providers; their local defaults require no API key.
---
### OpenAI Codex Setup (ChatGPT Plus/Pro)
Use your ChatGPT Plus or Pro subscription for Hindsight without separate OpenAI Platform API costs.
@@ -19,6 +19,7 @@
{"id": "requesty", "label": "Requesty", "iconKey": "openai-compatible", "defaultModel": "openai/gpt-4o-mini"},
{"id": "openai-codex", "label": "OpenAI Codex", "iconKey": "openai", "defaultModel": "gpt-5.4-mini"},
{"id": "claude-code", "label": "Claude Code", "iconKey": "anthropic", "defaultModel": "claude-sonnet-4-5-20250929"},
{"id": "github-copilot","label": "GitHub Copilot", "iconKey": "terminal", "defaultModel": "gpt-5.6-terra"},
{"id": "bedrock", "label": "AWS Bedrock", "iconKey": "cloud", "defaultModel": "us.amazon.nova-2-lite-v1:0"},
{"id": "fireworks", "label": "Fireworks AI", "iconKey": "zap", "defaultModel": "accounts/fireworks/models/llama-v3p1-8b-instruct", "batchApi": true},
{"id": "nous", "label": "Nous Portal", "iconKey": "openai-compatible", "defaultModel": "deepseek/deepseek-v4-flash"},
+3 -3
View File
@@ -60,8 +60,8 @@ hindsight-embed configure --profile staging
```
This will:
- Let you choose an LLM provider (OpenAI, Groq, Google, Ollama)
- Configure your API key
- Let you choose an LLM provider (OpenAI, Groq, Google, Ollama, GitHub Copilot)
- Configure credentials when the provider requires them
- Set the model
- Start the daemon with your configuration
@@ -151,7 +151,7 @@ Run `hindsight-embed configure` for a guided setup that saves to `~/.hindsight/e
|----------|-------------|---------|
| `HINDSIGHT_EMBED_PROFILE` | Profile name to use (overrides active profile) | None (uses default profile) |
| `HINDSIGHT_API_LLM_API_KEY` | LLM API key (or use `OPENAI_API_KEY`); required only when the selected provider uses an API key | Provider-dependent |
| `HINDSIGHT_API_LLM_PROVIDER` | LLM provider (`openai`, `groq`, `google`, `ollama`) | `openai` |
| `HINDSIGHT_API_LLM_PROVIDER` | LLM provider (`openai`, `groq`, `gemini`, `ollama`, `github-copilot`) | `openai` |
| `HINDSIGHT_API_LLM_MODEL` | LLM model | `gpt-4o-mini` |
| `HINDSIGHT_EMBED_API_URL` | Use external API server instead of starting local daemon | None (starts local daemon) |
| `HINDSIGHT_EMBED_API_TOKEN` | Authentication token for external API (sent as Bearer token) | None |
+22 -12
View File
@@ -126,9 +126,14 @@ def get_config():
venvs (e.g. `uvx hindsight-embed`) where `hindsight-api` isn't installed.
"""
load_config_file()
provider = os.environ.get("HINDSIGHT_API_LLM_PROVIDER", "openai")
return {
"llm_api_key": os.environ.get("HINDSIGHT_API_LLM_API_KEY") or os.environ.get("OPENAI_API_KEY"),
"llm_provider": os.environ.get("HINDSIGHT_API_LLM_PROVIDER", "openai"),
"llm_api_key": (
None
if provider in NO_API_KEY_PROVIDERS
else os.environ.get("HINDSIGHT_API_LLM_API_KEY") or os.environ.get("OPENAI_API_KEY")
),
"llm_provider": provider,
"llm_model": os.environ.get("HINDSIGHT_API_LLM_MODEL"),
}
@@ -140,7 +145,9 @@ PROVIDER_API_KEYS = {
"gemini": "GEMINI_API_KEY",
"ollama": None,
"vertexai": None,
"github-copilot": None,
}
NO_API_KEY_PROVIDERS = frozenset(provider for provider, key_env in PROVIDER_API_KEYS.items() if key_env is None)
def do_configure(args):
@@ -194,29 +201,31 @@ def _has_non_interactive_env() -> bool:
"""Whether the env vars required by _do_configure_from_env are already set.
Returns True when an API key is present, OR when the provider is one that
doesn't need a key (ollama, vertexai — the latter authenticates via a
service-account file path). Prevents the interactive prompt from kicking
in when the user clearly wants CI/scripted behavior.
doesn't need a key. Prevents the interactive prompt from kicking in when
the user clearly wants CI/scripted behavior.
"""
if os.environ.get("HINDSIGHT_API_LLM_API_KEY") or os.environ.get("OPENAI_API_KEY"):
return True
return os.environ.get("HINDSIGHT_API_LLM_PROVIDER") in ("ollama", "vertexai")
return os.environ.get("HINDSIGHT_API_LLM_PROVIDER") in NO_API_KEY_PROVIDERS
def _do_configure_from_env():
"""Non-interactive configuration from environment variables (for CI)."""
# Check for required environment variables
api_key = os.environ.get("HINDSIGHT_API_LLM_API_KEY") or os.environ.get("OPENAI_API_KEY")
provider = os.environ.get("HINDSIGHT_API_LLM_PROVIDER", "openai")
api_key = (
None
if provider in NO_API_KEY_PROVIDERS
else os.environ.get("HINDSIGHT_API_LLM_API_KEY") or os.environ.get("OPENAI_API_KEY")
)
# Don't gate on PROVIDER_API_KEYS — that's only the interactive-menu set
# (5 entries). hindsight-api's PROVIDER_DEFAULT_MODELS supports ~18
# Don't gate on PROVIDER_API_KEYS — that's only the small interactive-menu
# set. hindsight-api's PROVIDER_DEFAULT_MODELS supports many more
# providers (anthropic, claude-code, bedrock, openrouter, ...). Let the
# daemon validate; rejecting here would block valid configurations.
# Check for API key (required for non-ollama and non-vertexai providers)
# vertexai uses GCP service account credentials instead of an API key
if not api_key and provider not in ("ollama", "vertexai"):
# These providers authenticate locally rather than through an LLM API key.
if not api_key and provider not in NO_API_KEY_PROVIDERS:
print("Error: Cannot run interactive configuration without a terminal.", file=sys.stderr)
print("", file=sys.stderr)
print("For non-interactive (CI) mode, set environment variables:", file=sys.stderr)
@@ -373,6 +382,7 @@ def _do_configure_interactive(profile_name: str | None = None, port: int | None
("Groq (fast & free tier)", "groq"),
("Google Gemini", "gemini"),
("Ollama (local, no API key)", "ollama"),
("GitHub Copilot (signed-in account, no LLM API key)", "github-copilot"),
]
provider = _prompt_choice("Select your LLM provider:", providers, default=1)
+7 -2
View File
@@ -2,11 +2,11 @@
# Copy this file to .env and fill in your values
# LLM Configuration (Required)
# Supported providers: openai, openai-responses, groq, ollama, gemini, anthropic, lmstudio, vertexai, minimax, deepseek, zai, atlas, volcano
# Supported providers: openai, openai-responses, groq, ollama, gemini, anthropic, lmstudio, vertexai, minimax, deepseek, zai, atlas, volcano, openai-codex, claude-code, github-copilot
HINDSIGHT_API_LLM_PROVIDER=openai
HINDSIGHT_API_LLM_API_KEY=your-api-key-here
HINDSIGHT_API_LLM_MODEL=gpt-4o-mini
HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
# HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
# Reasoning effort for providers/models that support it. Examples: none, low, medium, high, xhigh.
# Set it and the value is sent as given, whatever the model is called — use `none` to stop a
# self-hosted reasoning model (vLLM, Ollama, llama.cpp, TGI) emitting thinking blocks. Unset,
@@ -74,6 +74,11 @@ HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
# HINDSIGHT_API_LLM_API_KEY=your-anthropic-api-key
# HINDSIGHT_API_LLM_MODEL=claude-sonnet-4-20250514
# Example: GitHub Copilot subscription via the official Copilot SDK
# Sign in with Copilot CLI first; no HINDSIGHT_API_LLM_API_KEY is needed.
# HINDSIGHT_API_LLM_PROVIDER=github-copilot
# HINDSIGHT_API_LLM_MODEL=gpt-5.6-terra
# Example: Google Vertex AI configuration
# HINDSIGHT_API_LLM_PROVIDER=vertexai
# HINDSIGHT_API_LLM_MODEL=google/gemini-2.0-flash-001
@@ -594,7 +594,7 @@ def test_configure_from_env_accepts_providers_outside_interactive_menu(temp_home
"""Regression test for issue #1360.
`_do_configure_from_env` previously rejected any provider not in the small
interactive-menu set (`PROVIDER_API_KEYS` 5 entries) with "Unknown
interactive-menu set (`PROVIDER_API_KEYS`) with "Unknown
provider". hindsight-api supports ~18 providers (anthropic, claude-code,
bedrock, openrouter, ...), so the gate blocked valid CI configurations.
Validation belongs in the daemon, not in the CLI's UX-only menu list.
@@ -616,6 +616,29 @@ def test_configure_from_env_accepts_providers_outside_interactive_menu(temp_home
assert "HINDSIGHT_API_LLM_PROVIDER=anthropic" in contents
def test_configure_from_env_accepts_github_copilot_without_api_key(temp_home, monkeypatch):
"""GitHub Copilot authenticates through Copilot CLI rather than an LLM API key."""
from hindsight_embed import cli
config_dir = temp_home / ".hindsight"
monkeypatch.setattr(cli, "CONFIG_DIR", config_dir)
monkeypatch.setattr(cli, "CONFIG_FILE", config_dir / "embed")
monkeypatch.setenv("HINDSIGHT_API_LLM_PROVIDER", "github-copilot")
monkeypatch.setenv("HINDSIGHT_API_LLM_MODEL", "gpt-5.6-terra")
monkeypatch.delenv("HINDSIGHT_API_LLM_API_KEY", raising=False)
monkeypatch.setenv("OPENAI_API_KEY", "unrelated-openai-key")
assert cli._has_non_interactive_env() is True
assert cli._do_configure_from_env() == 0
contents = (config_dir / "embed").read_text()
assert "HINDSIGHT_API_LLM_PROVIDER=github-copilot" in contents
assert "HINDSIGHT_API_LLM_MODEL=gpt-5.6-terra" in contents
active_lines = [line for line in contents.splitlines() if line and not line.startswith("#")]
assert not any(line.startswith("HINDSIGHT_API_LLM_API_KEY=") for line in active_lines)
def _windows_scripts_dir(tmp_path: Path, *, with_pythonw: bool) -> Path:
"""Build a fake Windows venv Scripts dir with hindsight-api.exe.
@@ -23,10 +23,11 @@ PROVIDER_DETECTION = [
{"name": "ollama", "key_env": ""},
{"name": "openai-codex", "key_env": ""},
{"name": "claude-code", "key_env": ""},
{"name": "github-copilot", "key_env": ""},
]
# Providers that don't require an API key
NO_KEY_REQUIRED = {"ollama", "openai-codex", "claude-code"}
NO_KEY_REQUIRED = {"ollama", "openai-codex", "claude-code", "github-copilot"}
def _find_provider(name):
@@ -23,10 +23,11 @@ PROVIDER_DETECTION = [
{"name": "ollama", "key_env": ""},
{"name": "openai-codex", "key_env": ""},
{"name": "claude-code", "key_env": ""},
{"name": "github-copilot", "key_env": ""},
]
# Providers that don't require an API key
NO_KEY_REQUIRED = {"ollama", "openai-codex", "claude-code"}
NO_KEY_REQUIRED = {"ollama", "openai-codex", "claude-code", "github-copilot"}
def _find_provider(name):
@@ -23,9 +23,10 @@ PROVIDER_DETECTION = [
{"name": "ollama", "key_env": ""},
{"name": "openai-codex", "key_env": ""},
{"name": "claude-code", "key_env": ""},
{"name": "github-copilot", "key_env": ""},
]
NO_KEY_REQUIRED = {"ollama", "openai-codex", "claude-code"}
NO_KEY_REQUIRED = {"ollama", "openai-codex", "claude-code", "github-copilot"}
def _find_provider(name):
@@ -23,9 +23,10 @@ PROVIDER_DETECTION = [
{"name": "ollama", "key_env": ""},
{"name": "openai-codex", "key_env": ""},
{"name": "claude-code", "key_env": ""},
{"name": "github-copilot", "key_env": ""},
]
NO_KEY_REQUIRED = {"ollama", "openai-codex", "claude-code"}
NO_KEY_REQUIRED = {"ollama", "openai-codex", "claude-code", "github-copilot"}
def _find_provider(name):
@@ -23,10 +23,11 @@ PROVIDER_DETECTION = [
{"name": "ollama", "key_env": ""},
{"name": "openai-codex", "key_env": ""},
{"name": "claude-code", "key_env": ""},
{"name": "github-copilot", "key_env": ""},
]
# Providers that don't require an API key
NO_KEY_REQUIRED = {"ollama", "openai-codex", "claude-code"}
NO_KEY_REQUIRED = {"ollama", "openai-codex", "claude-code", "github-copilot"}
def _find_provider(name):
+6 -1
View File
@@ -1270,7 +1270,12 @@ export function formatMemories(results: MemoryResult[]): string {
}
// Providers that authenticate via OAuth or run locally — no API key needed.
const NO_KEY_REQUIRED_PROVIDERS = new Set(["ollama", "openai-codex", "claude-code"]);
const NO_KEY_REQUIRED_PROVIDERS = new Set([
"ollama",
"openai-codex",
"claude-code",
"github-copilot",
]);
export function detectLLMConfig(pluginConfig?: PluginConfig): {
provider?: string;
@@ -352,6 +352,15 @@ describe("applyEmbeddedMode", () => {
expect(pc.llmApiKey).toBeUndefined();
});
it("omits llmApiKey for github-copilot", () => {
const pc: Record<string, unknown> = {
llmApiKey: { source: "env", provider: "default", id: "STALE" },
};
applyEmbeddedMode(pc, { llmProvider: "github-copilot" });
expect(pc.llmProvider).toBe("github-copilot");
expect(pc.llmApiKey).toBeUndefined();
});
it("throws when a key-requiring provider is given without a key", () => {
const pc: Record<string, unknown> = {};
expect(() => applyEmbeddedMode(pc, { llmProvider: "openai" })).toThrow(
@@ -49,6 +49,7 @@ export type SetupMode = "cloud" | "api" | "embedded";
export const NO_KEY_PROVIDERS: ReadonlySet<string> = new Set([
"claude-code",
"github-copilot",
"openai-codex",
"ollama",
]);
@@ -23,9 +23,10 @@ PROVIDER_DETECTION = [
{"name": "ollama", "key_env": ""},
{"name": "openai-codex", "key_env": ""},
{"name": "claude-code", "key_env": ""},
{"name": "github-copilot", "key_env": ""},
]
NO_KEY_REQUIRED = {"ollama", "openai-codex", "claude-code"}
NO_KEY_REQUIRED = {"ollama", "openai-codex", "claude-code", "github-copilot"}
def _find_provider(name):
@@ -213,8 +213,8 @@ For non-English banks (especially CJK) and the language/extraction-language trad
| Variable | Description | Default |
|----------|-------------|---------|
| `HINDSIGHT_API_LLM_PROVIDER` | Provider: `openai`, `openai-responses`, `openai-codex`, `claude-code`, `anthropic`, `gemini`, `groq`, `minimax`, `deepseek`, `zai`, `opencode-go`, `nous`, `xai-oauth`, `fireworks`, `ollama`, `ollama-cloud`, `lmstudio`, `llamacpp`, `vertexai`, `bedrock`, `litellm`, `litellmrouter`, `volcano`, `openrouter`, `requesty`, `none` | `openai` |
| `HINDSIGHT_API_LLM_API_KEY` | API key for LLM provider | - |
| `HINDSIGHT_API_LLM_PROVIDER` | Provider: `openai`, `openai-responses`, `openai-codex`, `claude-code`, `github-copilot`, `anthropic`, `gemini`, `groq`, `minimax`, `deepseek`, `zai`, `opencode-go`, `nous`, `xai-oauth`, `fireworks`, `ollama`, `ollama-cloud`, `lmstudio`, `llamacpp`, `vertexai`, `bedrock`, `litellm`, `litellmrouter`, `volcano`, `openrouter`, `requesty`, `none` | `openai` |
| `HINDSIGHT_API_LLM_API_KEY` | API key for providers that require one; unused by `github-copilot` | - |
| `HINDSIGHT_API_LLM_MODEL` | Model name | `gpt-5-mini` |
| `HINDSIGHT_API_LLM_BASE_URL` | Custom LLM endpoint | Provider default |
| `HINDSIGHT_API_LLM_MAX_CONCURRENT` | Max concurrent LLM requests | `32` |
@@ -11,7 +11,7 @@ Guide to setting up a local development environment for contributing to Hindsigh
- Python 3.11+
- [uv](https://docs.astral.sh/uv/) - Fast Python package manager
- Docker and Docker Compose
- An LLM API key (OpenAI, Groq, or Ollama)
- An LLM provider credential, or a local/subscription-backed provider such as Ollama or GitHub Copilot
## Local Development Setup
@@ -39,6 +39,7 @@ Used for fact extraction, entity resolution, mental model consolidation, and ans
- Requesty
- OpenAI Codex
- Claude Code
- GitHub Copilot
- AWS Bedrock
- Fireworks AI
- Nous Portal
@@ -121,6 +122,7 @@ Beyond basic generation, some providers support optional features that lower cos
| Requesty (`requesty`) | — | — |
| OpenAI Codex (`openai-codex`) | — | — |
| Claude Code (`claude-code`) | — | — |
| GitHub Copilot (`github-copilot`) | — | — |
| AWS Bedrock (`bedrock`) | — | — |
| Fireworks AI (`fireworks`) | ✅ | — |
| Nous Portal (`nous`) | — | — |
@@ -186,6 +188,7 @@ Each provider has a recommended default model that's used when `HINDSIGHT_API_LL
| `requesty` | `openai/gpt-4o-mini` |
| `openai-codex` | `gpt-5.4-mini` |
| `claude-code` | `claude-sonnet-4-5-20250929` |
| `github-copilot` | `gpt-5.6-terra` |
| `bedrock` | `us.amazon.nova-2-lite-v1:0` |
| `fireworks` | `accounts/fireworks/models/llama-v3p1-8b-instruct` |
| `nous` | `deepseek/deepseek-v4-flash` |
@@ -332,6 +335,34 @@ export HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=your-gcp-project-id
---
### GitHub Copilot Setup
Use a GitHub Copilot subscription for Hindsight's extraction, consolidation, and reflection calls through the official GitHub Copilot SDK.
**Prerequisites:**
- An active GitHub Copilot entitlement
- Copilot CLI signed in under the same operating-system user that runs Hindsight
**Configure Hindsight:**
```bash
export HINDSIGHT_API_LLM_PROVIDER=github-copilot
export HINDSIGHT_API_LLM_MODEL=gpt-5.6-terra
# No HINDSIGHT_API_LLM_API_KEY is needed.
```
The provider starts one shared Copilot runtime for all Hindsight LLM lanes. That runtime uses a hook-free temporary `COPILOT_HOME` containing only the signed-in account selection; the credential itself is resolved by the runtime from the system keychain or an existing `gh` CLI login. Each call then uses an isolated transient session with repository instructions, skills, Copilot memory, built-in tools, and the cross-session store disabled. This prevents the memory integration from recursively retaining its own extraction calls.
For automation, the Copilot SDK also accepts `COPILOT_GITHUB_TOKEN`, `GH_TOKEN`, or `GITHUB_TOKEN`. With one of those set, Copilot CLI never has to have been run on the host, so containers and CI images work with no `~/.copilot` at all. GitHub Actions and server-to-server deployments must have the appropriate Copilot organization policy and token permissions.
**Important notes:**
- Usage counts against the authenticated account or organization's Copilot allowance and AI Credits.
- `HINDSIGHT_API_LLM_BASE_URL` optionally points to an existing headless Copilot runtime, such as `http://127.0.0.1:4321`; it is not an LLM-provider endpoint for this provider.
- GitHub-hosted Copilot sessions do not expose temperature or maximum-output-token controls through the SDK, so those Hindsight settings are not applied.
- Embeddings and reranking continue to use Hindsight's separately configured providers; their local defaults require no API key.
---
### OpenAI Codex Setup (ChatGPT Plus/Pro)
Use your ChatGPT Plus or Pro subscription for Hindsight without separate OpenAI Platform API costs.
+1
View File
@@ -89,6 +89,7 @@ Browse all supported integrations in the Integrations Hub.
- Requesty
- OpenAI Codex
- Claude Code
- GitHub Copilot
- AWS Bedrock
- Fireworks AI
- Nous Portal
Generated
+15
View File
@@ -1448,6 +1448,19 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/a0/61/5c78b91c3143ed5c14207f463aecfc8f9dbb5092fb2869baf37c273b2705/gitdb-4.0.12-py3-none-any.whl", hash = "sha256:67073e15955400952c6565cc3e707c554a4eea2e428946f7a4c162fab9bd9bcf", size = 62794, upload-time = "2025-01-02T07:20:43.624Z" },
]
[[package]]
name = "github-copilot-sdk"
version = "1.0.11"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "httpx" },
{ name = "pydantic" },
{ name = "python-dateutil" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/67/ac/175cbb71fe637d963a885248a421040c9d500ea6390ed3b88bc68ecf51ca/github_copilot_sdk-1.0.11-py3-none-any.whl", hash = "sha256:6f664c7b843c34ab5a7455f7effb4d339eae75969d2b72f43c2e6214c9c637dc", size = 486719, upload-time = "2026-08-14T16:12:20.01Z" },
]
[[package]]
name = "gitpython"
version = "3.1.59"
@@ -1688,6 +1701,7 @@ dependencies = [
{ name = "fastapi", extra = ["standard"] },
{ name = "fastmcp" },
{ name = "filelock" },
{ name = "github-copilot-sdk" },
{ name = "google-auth" },
{ name = "google-genai" },
{ name = "greenlet" },
@@ -1819,6 +1833,7 @@ requires-dist = [
{ name = "filelock", specifier = ">=3.20.1" },
{ name = "filelock", marker = "extra == 'test'", specifier = ">=3.20.1" },
{ name = "flashrank", marker = "extra == 'local-ml'", specifier = ">=0.2.0" },
{ name = "github-copilot-sdk", specifier = ">=1.0.11" },
{ name = "google-auth", specifier = ">=2.0.0" },
{ name = "google-genai", specifier = ">=1.72.0" },
{ name = "greenlet", specifier = ">=3.2.4,<3.4.0" },