5b4d76e8b4
Merge pull request #1173 from dotnet/abhitejjohn-issue-pr-triage-37b
...
Harden DevOps health investigation and reporting
2026-09-16 10:45:24 -07:00
f62c08fd35
Merge remote-tracking branch 'origin/abhitejjohn-issue-pr-triage-37b' into abhitejjohn-issue-pr-triage-37b
...
# Conflicts:
# .github/workflows/devops-health-check.lock.yml
# .github/workflows/devops-health-check.md
# .github/workflows/devops-health-groom.lock.yml
# .github/workflows/devops-health-groom.md
# .github/workflows/devops-health-investigate.lock.yml
# .github/workflows/devops-health-investigate.md
# eng/evaluation/test_token_failover.py
2026-09-16 10:34:52 -07:00
6bef1296b6
Harden durable health outbox lifecycle
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 10:21:44 -07:00
ba609ba4d3
fix: enforce health output invariants
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 19:19:53 +02:00
9324c99248
fix: protect transactional dashboard updates
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 19:05:09 +02:00
9bea2ddfce
fix: bind health reports to canonical metadata
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 18:49:35 +02:00
e8672ab524
fix: preserve unresolved health outbox
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 18:41:06 +02:00
bfae896b4b
Restore validated health publication boundaries
...
Keep typed privileged publishers and three-part correlations, validate the complete investigation template and reference links, and preserve active groom rows with executable regression coverage.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 09:29:49 -07:00
26e9e7a07a
fix: fail closed on malformed health payloads
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 18:28:42 +02:00
6244d86613
fix: validate authoritative health state
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 18:15:09 +02:00
0443f5a6ea
fix: validate health publisher payloads
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 18:06:58 +02:00
4f45bb0406
fix: preserve active health investigation rows
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 17:55:15 +02:00
5b6b1ad29e
fix: guard health workflow privileged writes
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 17:47:09 +02:00
956c21c559
fix: use absolute health dashboard links
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 17:32:38 +02:00
d5118cf124
fix: align health investigation dispatch runtime
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 17:26:44 +02:00
00fce762d0
fix: close health workflow output gaps
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 17:19:00 +02:00
0b2aa5fb82
fix: enforce health investigation provenance
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 17:09:01 +02:00
37f60122f7
Merge remote-tracking branch 'origin/abhitejjohn-issue-pr-triage-37b' into abhitejjohn-issue-pr-triage-37b
...
# Conflicts:
# .github/workflows/devops-health-check.lock.yml
# .github/workflows/devops-health-check.md
# .github/workflows/devops-health-groom.lock.yml
# .github/workflows/devops-health-groom.md
# .github/workflows/devops-health-investigate.lock.yml
# eng/evaluation/test_token_failover.py
2026-09-16 16:58:52 +02:00
f6f984213d
fix: validate persisted health investigation results
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 16:54:59 +02:00
99003944c2
Reject invalid health state dates
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 07:53:24 -07:00
b05fc73ba6
Validate persisted health results
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 07:48:02 -07:00
e9cf0dd388
Merge remote-tracking branch 'origin/abhitejjohn-issue-pr-triage-37b' into abhitejjohn-issue-pr-triage-37b
...
# Conflicts:
# .github/aw/shared/devops-health.lock.md
# .github/workflows/devops-health-check.lock.yml
# .github/workflows/devops-health-check.md
# .github/workflows/devops-health-groom.lock.yml
# .github/workflows/devops-health-groom.md
# .github/workflows/devops-health-investigate.lock.yml
# eng/evaluation/test_token_failover.py
2026-09-16 16:19:40 +02:00
9196fe0324
Merge remote-tracking branch 'origin/abhitejjohn-issue-pr-triage-37b' into abhitejjohn-issue-pr-triage-37b
...
# Conflicts:
# .github/aw/shared/devops-health.lock.md
# .github/workflows/devops-health-check.lock.yml
# .github/workflows/devops-health-check.md
# .github/workflows/devops-health-groom.lock.yml
# .github/workflows/devops-health-groom.md
# .github/workflows/devops-health-investigate.lock.yml
# eng/evaluation/test_token_failover.py
2026-09-16 16:17:22 +02:00
1cbe1538ba
Harden health investigation publishing
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 07:16:27 -07:00
8da2d259da
fix: harden health workflow publication
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 16:13:37 +02:00
1219cb0e2e
Make health publication transactional
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 06:48:32 -07:00
260cdf8e20
Retry deferred health investigations
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 05:09:11 -07:00
8e4d26089a
Fail closed on invalid groomer state
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 04:36:18 -07:00
d77d2479b7
Keep health workflow failures dashboard-bound
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 04:20:09 -07:00
c46731f0b9
Preserve health findings when checks are unavailable
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 03:53:32 -07:00
4512ccf571
Fail closed on invalid health state
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 03:29:39 -07:00
fa260e82f2
Close dashboard state integrity gaps
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-15 18:46:27 -07:00
8b93dfec96
Serialize health dashboard state updates
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-15 18:27:39 -07:00
9f548cc03f
Paginate health investigation comments
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-15 17:47:16 -07:00
29ebd31633
Harden health workflow state handling
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-15 16:52:31 -07:00
769886c466
Close health workflow mutation paths
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-15 15:30:23 -07:00
f4f0ed317a
Address refactoring PR review feedback
...
Correct the judge-comparison label and remove stale dotnet-breaking-changes registry entries left by the rebase.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
Copilot-Session: 8c45529b-2515-483d-9e51-e6c0b7cb6852
2026-09-15 10:31:37 -07:00
1f73f15936
Fix evaluation workflow expression limit
...
Move target discovery into a tested PowerShell script so GitHub Actions can parse and dispatch the workflow.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
Copilot-Session: 8c45529b-2515-483d-9e51-e6c0b7cb6852
2026-09-15 09:30:14 -07:00
72fc029f9f
Update experiment name in evaluation workflow and enhance csharp-refactoring skill documentation and tests
2026-09-15 09:18:35 -07:00
08e6568a8b
Bind health outputs to canonical dashboard
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-15 08:29:41 -07:00
6deb481989
Merge main into infrastructure health fixes
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-15 08:03:19 -07:00
b7f11542fa
Make infra investigation report only
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-15 08:00:47 -07:00
36222bf32d
Add first-class custom-agent evaluation coverage ( #1165 )
...
* feat(evaluation): add custom agent coverage
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): address agent review feedback
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): reject linked fixture sources
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): preserve agent result invariants
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): fail closed on agent errors
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): preserve completion regressions
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): preserve nested command quotes
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): harden native agent evidence
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): honor declared agent layout
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): resolve declared agent sources
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): secure agent path discovery
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): reject linked dependencies
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): centralize path safety checks
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): diagnose ambiguous dependencies
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): reject linked allowed roots
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): preserve skill agent isolation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): normalize dashboard evidence
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): preserve agent gate semantics
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): fail closed on incomplete evidence
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): preserve completion evidence
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): reject overflowing durations
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): stage verified plugin skills
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): block shell network access
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): reject linked MCP config files
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): trust manual dispatch path safety
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): keep agent plugin activation diagnostic
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): count failed tool completions
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix(evaluation): synchronize agent event capture
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
---------
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-15 16:47:35 +02:00
387a0677b0
Require trusted evidence for automatic fixes
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-15 04:43:22 -07:00
6d15f2236e
Enforce health investigation dispatch cap
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-15 04:18:11 -07:00
a6f67ce526
Guard infra remediation from untrusted input
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-15 03:51:34 -07:00
43dec99816
Use MAI Code 1.1 for evaluations ( #1174 )
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-15 11:15:05 +02:00
234c399a95
Close workflow validation gaps
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-15 02:11:01 -07:00
460a01882f
Define and validate native Codex plugin support ( #1164 )
...
* fix: validate native Codex plugin support
Separate Codex-compatible plugin components from GitHub Copilot custom agents, validate Codex manifest fields and MCP shapes, and document the native agent installation boundary.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix: strengthen Codex plugin verification
Validate Codex MCP per-tool settings and add a pinned native smoke lane that installs the marketplace, discovers skills and MCP, and calls binlog_overview.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix: avoid restarting Codex MCP smoke client
Use the real tool call as the app-server initialization so the Linux smoke test does not start and tear down a redundant status client first.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix: harden Codex validation and smoke cleanup
Constrain smoke artifacts to a dedicated child directory, validate Codex manifest field shapes and required skill paths, and document the separate Agents SDK handoff model.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix: align Codex smoke and hooks parsing
Use the clean Codex installation's default provider for the direct MCP call and reject heterogeneous hooks arrays like the runtime parser.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix: verify every Codex plugin skill
Compare native skill discovery with the full repository inventory, match Codex lexical path rejection, and allow null optional MCP tool settings.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
* fix: address Codex compatibility review
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
---------
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-15 10:47:14 +02:00
16f5eaacc1
Keep workflow reviewers read-only
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-14 23:38:46 -07:00
Abhitej John