diff --git a/.gitattributes b/.gitattributes index bf094344..76e96e55 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,2 +1,4 @@ # Normalize all text files to LF line endings * text=auto eol=lf + +.github/workflows/*.lock.yml linguist-generated=true merge=ours \ No newline at end of file diff --git a/.github/aw/actions-lock.json b/.github/aw/actions-lock.json new file mode 100644 index 00000000..a1ea7db8 --- /dev/null +++ b/.github/aw/actions-lock.json @@ -0,0 +1,14 @@ +{ + "entries": { + "actions/github-script@v8": { + "repo": "actions/github-script", + "version": "v8", + "sha": "ed597411d8f924073f98dfc5c65a23a2325f34cd" + }, + "github/gh-aw/actions/setup@v0.45.4": { + "repo": "github/gh-aw/actions/setup", + "version": "v0.45.4", + "sha": "ac090214a48a1938f7abafe132460b66752261af" + } + } +} diff --git a/.github/aw/shared/devops-health.lock.md b/.github/aw/shared/devops-health.lock.md new file mode 100644 index 00000000..8e8d734e --- /dev/null +++ b/.github/aw/shared/devops-health.lock.md @@ -0,0 +1,409 @@ + + + +# DevOps Health Check — Compiled Knowledge + +This document contains the health check catalog, fingerprinting rules, output templates, and operational guidance for the DevOps Daily Health Check agentic workflow. + +--- + +## 1. Fingerprinting Rules + +Every health finding MUST be assigned a deterministic **fingerprint** — a string ID derived from the finding's category and key attributes (but NOT timestamps, run IDs, or other ephemeral data). The same real-world issue MUST produce the same fingerprint on every run. + +### 1.1 Pipeline Fingerprints + +``` +fingerprint = "pipeline:{workflow_name}:{job_name}:{failed_step}:{conclusion}" +``` + +- Normalize `workflow_name` by lowercasing and replacing spaces with hyphens +- Normalize `job_name` and `failed_step` the same way +- Same workflow + job + step + conclusion = same finding (even across different run IDs) +- A workflow that fails in a _different_ step is a _different_ finding +- For timeouts/cancellations: `pipeline:{workflow_name}:{job_name}:timeout` + +**Examples:** +| Finding | Fingerprint | +|---------|-------------| +| Evaluation workflow, evaluate job, "Run skill-validator" step failed | `pipeline:evaluation:evaluate:run-skill-validator:failure` | +| Evaluation workflow, evaluate job, "Build validator" step failed | `pipeline:evaluation:evaluate:build-validator:failure` | +| validate-skills workflow, validate job timed out | `pipeline:validate-skills:validate:timeout` | + +### 1.2 Quality Fingerprints + +``` +fingerprint = "quality:{skill_name}:{scenario_name}:{signal}" + where signal ∈ { "{flag_name}", "regressed", "no-uplift", "high-variance" } +``` + +- Extract `skill_name` and `scenario_name` from bench entry `name` field + - Format: `"{skill}/{scenario} - {metric}"` → parse text before ` - ` and split on `/` +- `{flag_name}` = any non-standard boolean property found on a bench entry (e.g., `notActivated`, `timedOut`, `testOverfitted`, or any future flag added to `generate-benchmark-data.ps1`) +- Anomaly flags are **dynamically discovered**: any property beyond `name`/`unit`/`value` on a bench entry is treated as an anomaly flag + +**Examples:** +| Finding | Fingerprint | +|---------|-------------| +| dump-collect/basic-dump has notActivated flag | `quality:dump-collect:basic-dump:notActivated` | +| csharp-scripts/basic-script quality dropped 2.3 points | `quality:csharp-scripts:basic-script:regressed` | +| dotnet-pinvoke/marshal-array skilled ≤ vanilla | `quality:dotnet-pinvoke:marshal-array:no-uplift` | +| analyzing-dotnet-performance/memory-leak high stddev | `quality:analyzing-dotnet-performance:memory-leak:high-variance` | + +### 1.3 Coverage Fingerprints + +``` +fingerprint = "coverage:{skill_name}:no-tests" +``` + +### 1.4 Benchmark Staleness Fingerprints + +``` +fingerprint = "quality:benchmark-stale:{component_name}" +``` + +### 1.5 PR Fingerprints + +``` +fingerprint = "pr:{pr_number}:{signal}" + where signal ∈ { "stale", "no-review", "failing-checks", "stale-draft" } +``` + +- `pr_number` is the integer PR number (not the node ID) +- A PR that was "stale" last run and is still stale → EXISTING +- A PR that was "stale" but got merged/closed → RESOLVED + +### 1.6 Infrastructure Fingerprints + +``` +fingerprint = "infra:{config_key}" + where config_key ∈ { "no-codeowners", "no-dependabot", "relaxed-skill-validation", + "verdict-warn-only", "pages-deployment-failed", + "unpinned-action:{action_name}" } +``` + +### 1.7 Resource Fingerprints + +``` +fingerprint = "resource:{metric}:{threshold_breach}" +``` + +- `resource:eval-duration:warning` — eval avg > 50 min +- `resource:eval-duration:critical` — eval avg > 55 min +- `resource:cost-increase` — weekly compute hours up >20% + +--- + +## 2. Diff Algorithm + +``` +previous_fps = cache_memory_load("health-check-fingerprints") ?? {} +current_fps = {} + +for each finding in all_collected_findings: + fp = compute_fingerprint(finding) + current_fps[fp] = finding + +new_findings = { fp: f for fp, f in current_fps if fp NOT IN previous_fps } +existing_findings = { fp: f for fp, f in current_fps if fp IN previous_fps } +resolved_findings = { fp: f for fp, f in previous_fps if fp NOT IN current_fps } + +# Update occurrence tracking +for fp in existing_findings: + existing_findings[fp].occurrences = previous_fps[fp].occurrences + 1 + existing_findings[fp].first_seen = previous_fps[fp].first_seen + +for fp in new_findings: + new_findings[fp].occurrences = 1 + new_findings[fp].first_seen = today + +cache_memory_save("health-check-fingerprints", current_fps) +cache_memory_save("health-check-history", append( + load("health-check-history"), + { date: today, new_count, existing_count, resolved_count, by_severity } +)) +``` + +### 2.1 Sorting Within Diff Categories + +Within each category (NEW, EXISTING, RESOLVED): +1. **Primary**: Severity descending — 🔴 Critical → 🟡 Warning → 🔵 Info +2. **Secondary**: Category — pipeline → quality → pr → infra → resource +3. **Tertiary**: Alphabetical by title + +--- + +## 3. Severity Rules Reference + +### Pipeline + +| Check | Condition | Severity | +|-------|-----------|----------| +| P1 | `evaluation` workflow failed | 🔴 Critical | +| P1 | Other workflow failed | 🟡 Warning | +| P1 | Matches `known-noise` pattern | 🔵 Info (demoted) | +| P2 | Any cancelled/timed-out run | 🟡 Warning | +| P3 | Eval avg duration > 55 min | 🔴 Critical | +| P3 | Eval avg duration > 50 min | 🟡 Warning | + +### Quality + +| Check | Condition | Severity | +|-------|-----------|----------| +| Q1 | `notActivated` flag on bench entry | 🔴 Critical | +| Q1 | Any other anomaly flag | 🟡 Warning | +| Q2 | Quality drop > 2.0 points vs 7-day avg | 🔴 Critical | +| Q2 | Quality drop > 1.0 points vs 7-day avg | 🟡 Warning | +| Q3 | Skilled ≤ Vanilla quality | 🟡 Warning | +| Q4 | Quality stddev > 1.5 over 7 days | 🟡 Warning | +| Q5 | Skill has no eval tests | 🟡 Warning | +| Q6 | Benchmark data > 24h old | 🟡 Warning | + +### PR + +| Check | Condition | Severity | +|-------|-----------|----------| +| R1 | Open > 7 days, no review | 🟡 Warning | +| R2 | Open > 14 days (any review) | 🟡 Warning | +| R3 | All checks failing | 🟡 Warning | +| R4 | Draft, no activity > 7 days | 🔵 Info | + +### Infrastructure + +| Check | Condition | Severity | +|-------|-----------|----------| +| I1 | No CODEOWNERS file | 🟡 Warning | +| I2 | No Dependabot config | 🟡 Warning | +| I3 | `fail-on-warning: false` in validate-skills | 🟡 Warning | +| I4 | `--verdict-warn-only` in evaluation | 🔵 Info | +| I5 | Pages deployment failed | 🔴 Critical | +| I6 | Unpinned third-party action | 🔵 Info | + +### Resource + +| Check | Condition | Severity | +|-------|-----------|----------| +| U3 | Weekly compute up >20% | 🟡 Warning | + +--- + +## 4. Benchmark Data Format + +Dashboard data files on `gh-pages` at `data/{component}.json` have this structure: + +```json +{ + "lastUpdate": 1740000000000, + "repoUrl": "", + "entries": { + "Quality": [ + { + "commit": { "id": "abc1234", "message": "...", "timestamp": "..." }, + "date": 1740000000000, + "tool": "customBiggerIsBetter", + "model": "claude-opus-4.6", + "benches": [ + { + "name": "skillName/scenarioName - Skilled Quality", + "unit": "Score (0-10)", + "value": 7.8 + }, + { + "name": "skillName/scenarioName - Vanilla Quality", + "unit": "Score (0-10)", + "value": 5.2 + }, + { + "name": "skillName/scenarioName - Skilled Quality", + "unit": "Score (0-10)", + "value": 0.0, + "notActivated": true + } + ] + } + ], + "Efficiency": [ + { + "commit": { ... }, + "date": 1740000000000, + "tool": "customSmallerIsBetter", + "model": "claude-opus-4.6", + "benches": [ + { + "name": "skillName/scenarioName - Skilled Time", + "unit": "seconds", + "value": 45.2, + "timedOut": true + } + ] + } + ] + } +} +``` + +### Key Points for Parsing: +- `date` is Unix epoch in **milliseconds** +- Bench entry `name` format: `"{skill}/{scenario} - {metric}"` +- Standard fields: `name`, `unit`, `value` — anything else is an anomaly flag +- Both `Quality` and `Efficiency` arrays carry the same anomaly flags +- Quality scores range 0-10 (mapped from 0-5 judge scale) +- The latest entry is the last element in the array (`entries.Quality[-1]`) +- For 7-day rolling averages, filter entries by `date` field (not array index) + +### Parsing Skill/Scenario from Bench Name: + +``` +bench.name = "dump-collect/basic-dump - Skilled Quality" +parts = bench.name.split(" - ") +# parts[0] = "dump-collect/basic-dump" +# parts[1] = "Skilled Quality" + +skill_scenario = parts[0].split("/") +# skill_scenario[0] = "dump-collect" (skill name) +# skill_scenario[1] = "basic-dump" (scenario name) +``` + +### Detecting Anomaly Flags: + +For each bench entry, check all properties. Any property key that is NOT `name`, `unit`, or `value` is an anomaly flag: + +``` +standard_fields = {"name", "unit", "value"} +flags = { key: value for key, value in bench_entry if key not in standard_fields and value == true } +``` + +This approach automatically discovers new flag types added in the future. + +--- + +## 5. Component Discovery + +Components are discovered by scanning the file system: + +```bash +find src/*/plugin.json -maxdepth 2 +``` + +Each `src/{name}/` directory containing a `plugin.json` is a component. The dashboard data file is at `data/{name}.json` on the `gh-pages` branch. + +To fetch benchmark data via the GitHub API (without `curl`): +``` +GET https://raw.githubusercontent.com/{owner}/{repo}/gh-pages/data/{component}.json +``` + +--- + +## 6. Known Noise Patterns + +The `cache-memory` key `known-noise` stores a list of fingerprint prefixes or patterns that should be demoted to 🔵 Info severity. Example patterns: + +- `pipeline:copilot-code-review` — org-level workflow with known chronic failures +- `infra:verdict-warn-only` — intentional configuration, always Info + +When a finding's fingerprint matches any known-noise pattern (prefix match), demote its severity to 🔵 Info. The finding is still reported in the output (in the EXISTING section if recurring) — it is NOT hidden. + +New patterns can be added via the `/health-check suppress ` slash command, which appends to the `known-noise` list in `cache-memory`. + +--- + +## 7. Investigation Dispatch Rules + +Only 🆕 NEW findings that meet these criteria qualify for investigation dispatch: + +| Condition | Action | +|-----------|--------| +| 🆕 + 🔴 Critical | **Always dispatch** | +| 🆕 + 🟡 Warning + `pipeline` or `quality` category | **Dispatch** | +| 🆕 + 🟡 Warning + `pr` or `infra` category | **Skip** | +| 🆕 + 🔵 Info | **Never dispatch** | +| 📌 EXISTING or ✅ RESOLVED | **Never dispatch** | + +**Budget cap:** Maximum 10 dispatches per run. +**Priority order when cap is hit:** +1. 🔴 Critical findings first +2. Pipeline findings before quality +3. Other categories last + +--- + +## 8. Output Templates + +### 8.1 Issue Title + +``` +🏥 Repository Health Dashboard +``` + +### 8.2 Issue Label + +``` +devops-health +``` +- Color: `#0E8A16` +- Description: `Daily automated health check report` + +### 8.3 First Run Notice + +If no previous fingerprints exist in `cache-memory`: + +```markdown +> ⚠️ This is the first health check run. All findings appear as new. +> Starting from the next run, only changes will be highlighted. +``` + +### 8.4 Trends Arrow Legend + +| Condition | Arrow | Meaning | +|-----------|-------|---------| +| Δ positive and good (e.g., success rate up) | ✅ | Improving | +| Δ positive and bad (e.g., compute hours up) | ↗️ | Increasing (watch) | +| Δ negative and good (e.g., open PRs down) | ✅ | Improving | +| Δ negative and bad (e.g., success rate down) | ⚠️ | Degrading | +| Δ ≈ 0 | ➡️ | Stable | + +### 8.5 Investigation Island Template + +```markdown + +⏳ Investigation dispatched — results arriving shortly... + +``` + +--- + +## 9. Operational Guardrails + +### 9.1 API Rate Limits +- Use targeted, date-filtered queries to minimize API calls +- The `github` MCP toolset handles pagination automatically +- Space dispatches 5 seconds apart + +### 9.2 Issue Body Size +- GitHub issues have a ~65,535 character limit +- If body exceeds 60k: truncate EXISTING section (keep top 20 by severity) +- Footer: `> … N additional existing findings omitted` +- The daily comment always includes complete summary counts + +### 9.3 Cache Memory Keys + +| Key | Contents | Updated | +|-----|----------|---------| +| `health-check-fingerprints` | Map of fingerprint → finding (with occurrences, first_seen) | Every run | +| `health-check-history` | Array of daily summaries (date, counts by diff type and severity) | Appended each run | +| `known-noise` | Array of fingerprint patterns to demote to Info | Via `/health-check suppress` | + +### 9.4 Graceful Degradation + +If any data source is unavailable: +- Skip that check category entirely +- Note the skip in the output: `> ⚠️ Skipped {category} checks: {reason}` +- Do NOT fail the entire workflow +- Continue with available data + +### 9.5 Cache Memory Loss + +If `cache-memory` returns no previous state: +- Treat all findings as 🆕 NEW +- Display the first-run notice (§8.3) +- The diff will resume automatically on the next run diff --git a/.github/aw/shared/devops-investigate.lock.md b/.github/aw/shared/devops-investigate.lock.md new file mode 100644 index 00000000..003c777b --- /dev/null +++ b/.github/aw/shared/devops-investigate.lock.md @@ -0,0 +1,309 @@ + + + +# DevOps Investigation — Compiled Knowledge + +This document contains category-specific investigation playbooks, root-cause patterns, and remediation templates for the DevOps Health Investigation worker agent. + +--- + +## 1. Pipeline Investigation Playbook + +When `finding_type == "pipeline"`: + +### Step-by-Step Protocol + +1. **Fetch the failed run** using `resource_url`: + ``` + GET /repos/{owner}/{repo}/actions/runs/{run_id} + ``` + Extract: workflow name, conclusion, created_at, updated_at, triggering_actor, head_sha. + +2. **Identify the failed job and step**: + ``` + GET /repos/{owner}/{repo}/actions/runs/{run_id}/jobs + ``` + Find the job with `conclusion: failure`. Within that job, find the step with `conclusion: failure`. + +3. **Read the failed step's logs** (last 200 lines): + ``` + GET /repos/{owner}/{repo}/actions/jobs/{job_id}/logs + ``` + Extract error messages, exception types, exit codes. Look for patterns: + - `.NET SDK version mismatch` → check `global.json` + - `process exited with code 1` → build/test failure + - `Error: ENOMEM` or `killed` → resource exhaustion + - `rate limit` or `403` → API throttling + - `timeout` → long-running operation exceeded limit + +4. **Fetch the last 5 successful runs** of the same workflow: + ``` + GET /repos/{owner}/{repo}/actions/workflows/{workflow_id}/runs?branch=main&status=success&per_page=5 + ``` + +5. **Compare: what changed between last success and this failure?** + - Get the `head_sha` of the last successful run + - Get the `head_sha` of the failed run + - Compare commits between them: + ``` + GET /repos/{owner}/{repo}/compare/{success_sha}...{failure_sha} + ``` + - Look for changes to: workflow YAML files, build scripts, `global.json`, dependency files, the code being tested. + +6. **Check if the failure is in repo code or a GitHub Action version update**: + - Compare action versions between the failing and last successful workflow YAML + - Check if a new action version was released recently + +7. **Determine root cause** with confidence level: + - **High**: Error message explicitly identifies the cause (e.g., "SDK version 9.0.200 not found") + - **Medium**: Timing strongly correlates with a specific commit + - **Low**: No clear evidence — multiple possibilities + +8. **Generate 1–3 specific remediation steps**: + - Include exact file paths, version numbers, or commands + - Order by recommended priority + +9. **Check for existing tracking**: + ``` + GET /repos/{owner}/{repo}/issues?state=open&labels=bug + ``` + Search for issues mentioning the same workflow or error. + +### Common Pipeline Root Causes + +| Pattern | Typical Cause | Remediation | +|---------|---------------|-------------| +| `SDK version not found` | `global.json` pins a version not installed on the runner | Update `global.json` or add `setup-dotnet` step | +| `process exited with code 1` in test step | Test failure (assertion or runtime error) | Check test output for specific failure | +| `Error: HttpError: rate limit exceeded` | GitHub API rate limiting | Add retry logic or reduce API calls | +| `The operation was canceled` | Timeout (default 360 min for Actions) | Optimize the step or increase timeout | +| `No space left on device` | Runner disk full (14 GB limit) | Add cleanup steps or reduce artifact size | +| Action `X` failed with `Node.js 16 actions are deprecated` | Action needs version update | Update action to latest version | + +--- + +## 2. Quality Investigation Playbook + +When `finding_type == "quality"`: + +### Step-by-Step Protocol + +1. **Fetch benchmark data** for the affected component (last 14+ entries): + ``` + GET https://raw.githubusercontent.com/{owner}/{repo}/gh-pages/data/{component}.json + ``` + +2. **Analyze the trend**: + - Extract the time series for the affected scenario's "Skilled Quality" bench + - Is this a **sudden drop** (step function) or **gradual degradation** (slope)? + - Sudden drops (>2 points between consecutive entries) typically indicate a specific triggering change + - Gradual degradation may indicate model drift or accumulating prompt issues + +3. **For anomaly flags** (`notActivated`, `timedOut`, `testOverfitted`, etc.): + a. Identify which flag(s) are set on the bench entry + b. For `notActivated`: + - The skill failed to activate — the model didn't use any skill-provided context + - Check the skill definition file (`src/{component}/skills/{skill}/SKILL.md`) for syntax or trigger issues + - Check if the skill's `description` field changed recently + - Check if the test prompt still aligns with the skill's trigger keywords + c. For `timedOut`: + - The evaluation exceeded the time limit + - Check recent complexity changes in the skill or test + - Check if the timeout threshold was changed + d. For other/unknown flags: + - Describe the flag name and its value + - Check `eng/dashboard/generate-benchmark-data.ps1` for context on when this flag is set + - Report the flag as-is with whatever context is available + +4. **For regression** (quality score dropped): + - Identify the exact entry where quality dropped (compare consecutive entries) + - Get the commit SHA from the regression entry's `commit` field + - Check what changed in that commit: + ``` + GET /repos/{owner}/{repo}/commits/{sha} + ``` + - Look for changes to: skill definition, test definition, shared knowledge files, prompt templates + +5. **For high variance**: + - Compute the range (max - min) across recent entries + - Check if variance is skill-specific or affects multiple skills (model instability) + - Check if test prompts are ambiguous (allowing valid but different approaches) + +6. **For no-uplift** (skilled ≤ vanilla): + - Compare the skill's prompt additions to the vanilla baseline + - Check if the skill knowledge is relevant to the test scenario + - Check if the skill is triggering correctly (not `notActivated`) + +7. **Check recent skill/test changes**: + ``` + GET /repos/{owner}/{repo}/commits?path=src/{component}/skills/{skill}&per_page=5 + GET /repos/{owner}/{repo}/commits?path=src/{component}/tests/{skill}&per_page=5 + ``` + +8. **Determine root cause** with confidence level. + +9. **Generate remediation steps** specific to skill quality: + - For activation issues: suggest trigger keyword adjustments + - For regression: identify the specific change and suggest reverting or fixing + - For variance: suggest test prompt clarification or eval parameter tuning + +### Common Quality Root Causes + +| Pattern | Typical Cause | Remediation | +|---------|---------------|-------------| +| `notActivated` on all scenarios of a skill | Skill description doesn't match test prompts | Review skill trigger keywords vs test prompts | +| `notActivated` on one scenario only | Test prompt is too different from skill scope | Adjust test prompt or broaden skill description | +| Sudden quality drop after commit X | Skill definition or knowledge was changed | Review diff of commit X; consider partial revert | +| Gradual quality decline over 7+ days | Model behavior drift or prompt degradation | Re-evaluate skill knowledge for staleness | +| Skilled ≤ Vanilla consistently | Skill knowledge may confuse rather than help | Review skill content for misleading information | +| `timedOut` on complex scenarios | Scenario requires too many tool calls or reasoning steps | Simplify scenario or increase timeout | +| High variance (stddev > 2.0) | Ambiguous test prompt allows multiple valid approaches | Tighten test prompt expectations | + +--- + +## 3. PR Investigation Playbook + +When `finding_type == "pr"`: + +### Step-by-Step Protocol + +1. **Fetch PR details**: + ``` + GET /repos/{owner}/{repo}/pulls/{pr_number} + ``` + Extract: title, author, created_at, updated_at, draft status, labels, body summary. + +2. **Fetch PR timeline** (reviews, comments, status changes): + ``` + GET /repos/{owner}/{repo}/pulls/{pr_number}/reviews + GET /repos/{owner}/{repo}/issues/{pr_number}/comments + ``` + +3. **For no-review PRs**: + - Check if CODEOWNERS would auto-assign reviewers + - Check `git blame` or recent contributors to the changed files for potential reviewers + - Look at the PR size (files changed, lines) — large PRs may discourage review + +4. **For stale PRs**: + - Check last activity date (any comment or push) + - Check if the author is still active (recent commits/PRs) + - Check if there are related issues that are still relevant + +5. **For failing checks**: + ``` + GET /repos/{owner}/{repo}/commits/{head_sha}/check-runs + ``` + - Identify which checks fail + - Cross-reference with known pipeline issues from the same health check + - Determine if check failures are PR-specific or repo-wide + +6. **Provide actionable summary**: + - What's blocking the PR + - Who should review + - Whether the PR is still relevant + +--- + +## 4. Infrastructure Investigation Playbook + +When `finding_type == "infra"`: + +### Step-by-Step Protocol + +1. **Audit the configuration**: + - For missing files (CODEOWNERS, dependabot): confirm absence and explain impact + - For relaxed settings: read the config file and explain what the setting does + +2. **Check if intentional**: + - Search for issues or PRs that discuss the configuration choice + - Check commit history of the config file for context + +3. **Compare with best practices**: + - Reference GitHub's recommended security settings + - Note any compliance or security implications + +4. **For Pages deployment failures**: + ``` + GET /repos/{owner}/{repo}/pages/builds + ``` + - Read the latest build log + - Identify the failure cause (build error, quota, DNS, etc.) + +--- + +## 5. Resource Investigation Playbook + +When `finding_type == "resource"`: + +### Step-by-Step Protocol + +1. **Gather usage data**: + ``` + GET /repos/{owner}/{repo}/actions/runs?per_page=100 + ``` + - Compute daily/weekly compute hours by summing run durations + - Break down by workflow + +2. **Identify cost drivers**: + - Which workflows consume the most time? + - Has a new workflow been added recently? + - Did an existing workflow's duration increase? + +3. **For eval duration warnings**: + - Check if the number of skills/scenarios being evaluated increased + - Check if individual scenario duration increased + - Look for parallelism changes in the workflow configuration + +4. **Provide optimization suggestions**: + - Can any workflows be consolidated? + - Are there unnecessary re-runs (e.g., missing `concurrency` groups)? + - Can caching reduce execution time? + +--- + +## 6. Report Format + +All investigation results follow this template: + +```markdown +🔍 **Investigation Complete** — [Worker Run #{run_number}]({run_url}) + +**Root cause:** {Clear, evidence-based description of what went wrong and why. +Include specific error messages, commit SHAs, or file paths as evidence.} + +**Confidence:** {High|Medium|Low} — {One sentence justifying the confidence level} + +**Blast radius:** {What else is affected by this issue. Be specific about which +components, workflows, or metrics are impacted.} + +**Suggested fix:** +1. {Most recommended action — include specific file, line, or command} +2. {Alternative action if applicable} +3. {Additional step if needed} + +**Related:** {List related commits (with SHA + author), PRs (with #number), or +issues (with #number). Say "None found" if nothing is related.} +``` + +### Confidence Level Guidelines + +| Level | Criteria | Example | +|-------|----------|---------| +| **High** | Direct evidence links cause to effect | Error log says "SDK 9.0.200 not found"; commit changed SDK version | +| **Medium** | Strong circumstantial correlation | Quality dropped on the same day a skill file was modified | +| **Low** | Possible but speculative | Multiple recent changes could explain the issue; no clear winner | + +--- + +## 7. Common Cross-Category Patterns + +These patterns span multiple check categories and may help identify systemic issues: + +| Pattern | Indicates | +|---------|-----------| +| Pipeline failure + stale benchmark data | Pipeline is blocking data publication | +| Multiple quality regressions on same date | Common causal commit | +| PR failing checks + same check failing on main | Repo-wide issue, not PR-specific | +| `notActivated` + recent skill definition change | Skill trigger broke | +| Eval duration spike + new skill/scenario added | Expected growth, not a bug | +| Cost increase + new scheduled workflow | Expected growth, not waste | diff --git a/.github/workflows/devops-health-check.lock.yml b/.github/workflows/devops-health-check.lock.yml new file mode 100644 index 00000000..b3bf291e --- /dev/null +++ b/.github/workflows/devops-health-check.lock.yml @@ -0,0 +1,1413 @@ +# +# ___ _ _ +# / _ \ | | (_) +# | |_| | __ _ ___ _ __ | |_ _ ___ +# | _ |/ _` |/ _ \ '_ \| __| |/ __| +# | | | | (_| | __/ | | | |_| | (__ +# \_| |_/\__, |\___|_| |_|\__|_|\___| +# __/ | +# _ _ |___/ +# | | | | / _| | +# | | | | ___ _ __ _ __| |_| | _____ ____ +# | |/\| |/ _ \ '__| |/ /| _| |/ _ \ \ /\ / / ___| +# \ /\ / (_) | | | | ( | | | | (_) \ V V /\__ \ +# \/ \/ \___/|_| |_|\_\|_| |_|\___/ \_/\_/ |___/ +# +# This file was automatically generated by gh-aw (v0.45.4). DO NOT EDIT. +# +# To update this file, edit the corresponding .md file and run: +# gh aw compile +# Not all edits will cause changes to this file. +# +# For more information: https://github.github.com/gh-aw/introduction/overview/ +# +# Orchestrator workflow that collects repo health signals daily (pipelines, skill quality, PRs, infrastructure), computes a fingerprint-based diff against the previous run, updates a pinned health dashboard issue, and dispatches investigation workers for new critical/warning findings. +# +# Resolved workflow manifest: +# Imports: +# - ../aw/shared/devops-health.lock.md +# +# frontmatter-hash: 58af52a6c82092d7371c7d8ccbecb2f0637c094063b34a2e9a3a94c14fddd988 + +name: "DevOps Daily Health Check" +"on": + discussion: + types: + - created + - edited + discussion_comment: + types: + - created + - edited + issue_comment: + types: + - created + - edited + issues: + types: + - opened + - edited + - reopened + pull_request: + types: + - opened + - edited + - reopened + pull_request_review_comment: + types: + - created + - edited + schedule: + - cron: "40 17 * * *" + workflow_dispatch: null + +permissions: {} + +concurrency: + group: "gh-aw-${{ github.workflow }}-${{ github.event.issue.number || github.event.pull_request.number }}" + +run-name: "DevOps Daily Health Check" + +jobs: + activation: + needs: pre_activation + if: > + (needs.pre_activation.outputs.activated == 'true') && (((github.event_name == 'issues' || github.event_name == 'issue_comment' || + github.event_name == 'pull_request' || github.event_name == 'pull_request_review_comment' || github.event_name == 'discussion' || + github.event_name == 'discussion_comment') && ((github.event_name == 'issues') && ((startsWith(github.event.issue.body, '/health-check ')) || + (github.event.issue.body == '/health-check')) || (github.event_name == 'issue_comment') && (((startsWith(github.event.comment.body, '/health-check ')) || + (github.event.comment.body == '/health-check')) && (github.event.issue.pull_request == null)) || (github.event_name == 'issue_comment') && + (((startsWith(github.event.comment.body, '/health-check ')) || (github.event.comment.body == '/health-check')) && + (github.event.issue.pull_request != null)) || (github.event_name == 'pull_request_review_comment') && + ((startsWith(github.event.comment.body, '/health-check ')) || (github.event.comment.body == '/health-check')) || + (github.event_name == 'pull_request') && ((startsWith(github.event.pull_request.body, '/health-check ')) || + (github.event.pull_request.body == '/health-check')) || (github.event_name == 'discussion') && ((startsWith(github.event.discussion.body, '/health-check ')) || + (github.event.discussion.body == '/health-check')) || (github.event_name == 'discussion_comment') && ((startsWith(github.event.comment.body, '/health-check ')) || + (github.event.comment.body == '/health-check')))) || (!(github.event_name == 'issues' || github.event_name == 'issue_comment' || + github.event_name == 'pull_request' || github.event_name == 'pull_request_review_comment' || github.event_name == 'discussion' || + github.event_name == 'discussion_comment'))) + runs-on: ubuntu-slim + permissions: + contents: read + discussions: write + issues: write + pull-requests: write + outputs: + comment_id: "" + comment_repo: "" + slash_command: ${{ needs.pre_activation.outputs.matched_command }} + steps: + - name: Setup Scripts + uses: github/gh-aw/actions/setup@ac090214a48a1938f7abafe132460b66752261af # v0.45.4 + with: + destination: /opt/gh-aw/actions + - name: Checkout .github and .agents folders + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + sparse-checkout: | + .github + .agents + fetch-depth: 1 + persist-credentials: false + - name: Check workflow file timestamps + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_WORKFLOW_FILE: "devops-health-check.lock.yml" + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/check_workflow_timestamp_api.cjs'); + await main(); + - name: Create prompt with built-in context + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }} + GH_AW_GITHUB_ACTOR: ${{ github.actor }} + GH_AW_GITHUB_EVENT_COMMENT_ID: ${{ github.event.comment.id }} + GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: ${{ github.event.discussion.number }} + GH_AW_GITHUB_EVENT_ISSUE_NUMBER: ${{ github.event.issue.number }} + GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER: ${{ github.event.pull_request.number }} + GH_AW_GITHUB_REPOSITORY: ${{ github.repository }} + GH_AW_GITHUB_RUN_ID: ${{ github.run_id }} + GH_AW_GITHUB_WORKSPACE: ${{ github.workspace }} + GH_AW_IS_PR_COMMENT: ${{ github.event.issue.pull_request && 'true' || '' }} + run: | + bash /opt/gh-aw/actions/create_prompt_first.sh + cat << 'GH_AW_PROMPT_EOF' > "$GH_AW_PROMPT" + + GH_AW_PROMPT_EOF + cat "/opt/gh-aw/prompts/xpia.md" >> "$GH_AW_PROMPT" + cat "/opt/gh-aw/prompts/temp_folder_prompt.md" >> "$GH_AW_PROMPT" + cat "/opt/gh-aw/prompts/markdown.md" >> "$GH_AW_PROMPT" + cat "/opt/gh-aw/prompts/cache_memory_prompt.md" >> "$GH_AW_PROMPT" + cat << 'GH_AW_PROMPT_EOF' >> "$GH_AW_PROMPT" + + GitHub API Access Instructions + + The gh CLI is NOT authenticated. Do NOT use gh commands for GitHub operations. + + + To create or modify GitHub resources (issues, discussions, pull requests, etc.), you MUST call the appropriate safe output tool. Simply writing content will NOT work - the workflow requires actual tool calls. + + Temporary IDs: Some safe output tools support a temporary ID field (usually named temporary_id) so you can reference newly-created items elsewhere in the SAME agent output (for example, using #aw_abc1 in a later body). + + **IMPORTANT - temporary_id format rules:** + - If you DON'T need to reference the item later, OMIT the temporary_id field entirely (it will be auto-generated if needed) + - If you DO need cross-references/chaining, you MUST match this EXACT validation regex: /^aw_[A-Za-z0-9]{3,8}$/i + - Format: aw_ prefix followed by 3 to 8 alphanumeric characters (A-Z, a-z, 0-9, case-insensitive) + - Valid alphanumeric characters: ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789 + - INVALID examples: aw_ab (too short), aw_123456789 (too long), aw_test-id (contains hyphen), aw_id_123 (contains underscore) + - VALID examples: aw_abc, aw_abc1, aw_Test123, aw_A1B2C3D4, aw_12345678 + - To generate valid IDs: use 3-8 random alphanumeric characters or omit the field to let the system auto-generate + + Do NOT invent other aw_* formats — downstream steps will reject them with validation errors matching against /^aw_[A-Za-z0-9]{3,8}$/i. + + Discover available tools from the safeoutputs MCP server. + + **Critical**: Tool calls write structured data that downstream jobs process. Without tool calls, follow-up actions will be skipped. + + **Note**: If you made no other safe output tool calls during this workflow execution, call the "noop" tool to provide a status message indicating completion or that no actions were needed. + + + + The following GitHub context information is available for this workflow: + {{#if __GH_AW_GITHUB_ACTOR__ }} + - **actor**: __GH_AW_GITHUB_ACTOR__ + {{/if}} + {{#if __GH_AW_GITHUB_REPOSITORY__ }} + - **repository**: __GH_AW_GITHUB_REPOSITORY__ + {{/if}} + {{#if __GH_AW_GITHUB_WORKSPACE__ }} + - **workspace**: __GH_AW_GITHUB_WORKSPACE__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_ISSUE_NUMBER__ }} + - **issue-number**: #__GH_AW_GITHUB_EVENT_ISSUE_NUMBER__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER__ }} + - **discussion-number**: #__GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER__ }} + - **pull-request-number**: #__GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_COMMENT_ID__ }} + - **comment-id**: __GH_AW_GITHUB_EVENT_COMMENT_ID__ + {{/if}} + {{#if __GH_AW_GITHUB_RUN_ID__ }} + - **workflow-run-id**: __GH_AW_GITHUB_RUN_ID__ + {{/if}} + + + GH_AW_PROMPT_EOF + if [ "$GITHUB_EVENT_NAME" = "issue_comment" ] && [ -n "$GH_AW_IS_PR_COMMENT" ] || [ "$GITHUB_EVENT_NAME" = "pull_request_review_comment" ] || [ "$GITHUB_EVENT_NAME" = "pull_request_review" ]; then + cat "/opt/gh-aw/prompts/pr_context_prompt.md" >> "$GH_AW_PROMPT" + fi + cat << 'GH_AW_PROMPT_EOF' >> "$GH_AW_PROMPT" + + GH_AW_PROMPT_EOF + cat << 'GH_AW_PROMPT_EOF' >> "$GH_AW_PROMPT" + {{#runtime-import ../aw/shared/devops-health.lock.md}} + GH_AW_PROMPT_EOF + cat << 'GH_AW_PROMPT_EOF' >> "$GH_AW_PROMPT" + {{#runtime-import .github/workflows/devops-health-check.md}} + GH_AW_PROMPT_EOF + - name: Interpolate variables and render templates + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/interpolate_prompt.cjs'); + await main(); + - name: Substitute placeholders + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_ALLOWED_EXTENSIONS: '' + GH_AW_CACHE_DESCRIPTION: '' + GH_AW_CACHE_DIR: '/tmp/gh-aw/cache-memory/' + GH_AW_GITHUB_ACTOR: ${{ github.actor }} + GH_AW_GITHUB_EVENT_COMMENT_ID: ${{ github.event.comment.id }} + GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: ${{ github.event.discussion.number }} + GH_AW_GITHUB_EVENT_ISSUE_NUMBER: ${{ github.event.issue.number }} + GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER: ${{ github.event.pull_request.number }} + GH_AW_GITHUB_REPOSITORY: ${{ github.repository }} + GH_AW_GITHUB_RUN_ID: ${{ github.run_id }} + GH_AW_GITHUB_WORKSPACE: ${{ github.workspace }} + GH_AW_IS_PR_COMMENT: ${{ github.event.issue.pull_request && 'true' || '' }} + GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_ACTIVATED: ${{ needs.pre_activation.outputs.activated }} + GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_MATCHED_COMMAND: ${{ needs.pre_activation.outputs.matched_command }} + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + + const substitutePlaceholders = require('/opt/gh-aw/actions/substitute_placeholders.cjs'); + + // Call the substitution function + return await substitutePlaceholders({ + file: process.env.GH_AW_PROMPT, + substitutions: { + GH_AW_ALLOWED_EXTENSIONS: process.env.GH_AW_ALLOWED_EXTENSIONS, + GH_AW_CACHE_DESCRIPTION: process.env.GH_AW_CACHE_DESCRIPTION, + GH_AW_CACHE_DIR: process.env.GH_AW_CACHE_DIR, + GH_AW_GITHUB_ACTOR: process.env.GH_AW_GITHUB_ACTOR, + GH_AW_GITHUB_EVENT_COMMENT_ID: process.env.GH_AW_GITHUB_EVENT_COMMENT_ID, + GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: process.env.GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER, + GH_AW_GITHUB_EVENT_ISSUE_NUMBER: process.env.GH_AW_GITHUB_EVENT_ISSUE_NUMBER, + GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER: process.env.GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER, + GH_AW_GITHUB_REPOSITORY: process.env.GH_AW_GITHUB_REPOSITORY, + GH_AW_GITHUB_RUN_ID: process.env.GH_AW_GITHUB_RUN_ID, + GH_AW_GITHUB_WORKSPACE: process.env.GH_AW_GITHUB_WORKSPACE, + GH_AW_IS_PR_COMMENT: process.env.GH_AW_IS_PR_COMMENT, + GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_ACTIVATED: process.env.GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_ACTIVATED, + GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_MATCHED_COMMAND: process.env.GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_MATCHED_COMMAND + } + }); + - name: Validate prompt placeholders + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + run: bash /opt/gh-aw/actions/validate_prompt_placeholders.sh + - name: Print prompt + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + run: bash /opt/gh-aw/actions/print_prompt_summary.sh + - name: Upload prompt artifact + if: success() + uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6 + with: + name: prompt + path: /tmp/gh-aw/aw-prompts/prompt.txt + retention-days: 1 + + agent: + needs: activation + runs-on: ubuntu-latest + permissions: + actions: read + contents: read + issues: read + pull-requests: read + env: + DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} + GH_AW_ASSETS_ALLOWED_EXTS: "" + GH_AW_ASSETS_BRANCH: "" + GH_AW_ASSETS_MAX_SIZE_KB: 0 + GH_AW_MCP_LOG_DIR: /tmp/gh-aw/mcp-logs/safeoutputs + GH_AW_SAFE_OUTPUTS: /opt/gh-aw/safeoutputs/outputs.jsonl + GH_AW_SAFE_OUTPUTS_CONFIG_PATH: /opt/gh-aw/safeoutputs/config.json + GH_AW_SAFE_OUTPUTS_TOOLS_PATH: /opt/gh-aw/safeoutputs/tools.json + GH_AW_WORKFLOW_ID_SANITIZED: devopshealthcheck + outputs: + checkout_pr_success: ${{ steps.checkout-pr.outputs.checkout_pr_success || 'true' }} + has_patch: ${{ steps.collect_output.outputs.has_patch }} + model: ${{ steps.generate_aw_info.outputs.model }} + output: ${{ steps.collect_output.outputs.output }} + output_types: ${{ steps.collect_output.outputs.output_types }} + secret_verification_result: ${{ steps.validate-secret.outputs.verification_result }} + steps: + - name: Setup Scripts + uses: github/gh-aw/actions/setup@ac090214a48a1938f7abafe132460b66752261af # v0.45.4 + with: + destination: /opt/gh-aw/actions + - name: Checkout repository + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + - name: Create gh-aw temp directory + run: bash /opt/gh-aw/actions/create_gh_aw_tmp_dir.sh + # Cache memory file share configuration from frontmatter processed below + - name: Create cache-memory directory + run: bash /opt/gh-aw/actions/create_cache_memory_dir.sh + - name: Restore cache-memory file share data + uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0 + with: + key: memory-${{ env.GH_AW_WORKFLOW_ID_SANITIZED }}-${{ github.run_id }} + path: /tmp/gh-aw/cache-memory + restore-keys: | + memory-${{ env.GH_AW_WORKFLOW_ID_SANITIZED }}- + - name: Configure Git credentials + env: + REPO_NAME: ${{ github.repository }} + SERVER_URL: ${{ github.server_url }} + run: | + git config --global user.email "github-actions[bot]@users.noreply.github.com" + git config --global user.name "github-actions[bot]" + # Re-authenticate git with GitHub token + SERVER_URL_STRIPPED="${SERVER_URL#https://}" + git remote set-url origin "https://x-access-token:${{ github.token }}@${SERVER_URL_STRIPPED}/${REPO_NAME}.git" + echo "Git configured with standard GitHub Actions identity" + - name: Checkout PR branch + id: checkout-pr + if: | + github.event.pull_request + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + with: + github-token: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/checkout_pr_branch.cjs'); + await main(); + - name: Generate agentic run info + id: generate_aw_info + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const fs = require('fs'); + + const awInfo = { + engine_id: "copilot", + engine_name: "GitHub Copilot CLI", + model: process.env.GH_AW_MODEL_AGENT_COPILOT || "", + version: "", + agent_version: "0.0.410", + cli_version: "v0.45.4", + workflow_name: "DevOps Daily Health Check", + experimental: false, + supports_tools_allowlist: true, + run_id: context.runId, + run_number: context.runNumber, + run_attempt: process.env.GITHUB_RUN_ATTEMPT, + repository: context.repo.owner + '/' + context.repo.repo, + ref: context.ref, + sha: context.sha, + actor: context.actor, + event_name: context.eventName, + staged: false, + allowed_domains: ["defaults"], + firewall_enabled: true, + awf_version: "v0.19.1", + awmg_version: "v0.1.4", + steps: { + firewall: "squid" + }, + created_at: new Date().toISOString() + }; + + // Write to /tmp/gh-aw directory to avoid inclusion in PR + const tmpPath = '/tmp/gh-aw/aw_info.json'; + fs.writeFileSync(tmpPath, JSON.stringify(awInfo, null, 2)); + console.log('Generated aw_info.json at:', tmpPath); + console.log(JSON.stringify(awInfo, null, 2)); + + // Set model as output for reuse in other steps/jobs + core.setOutput('model', awInfo.model); + - name: Validate COPILOT_GITHUB_TOKEN secret + id: validate-secret + run: /opt/gh-aw/actions/validate_multi_secret.sh COPILOT_GITHUB_TOKEN 'GitHub Copilot CLI' https://github.github.com/gh-aw/reference/engines/#github-copilot-default + env: + COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} + - name: Install GitHub Copilot CLI + run: /opt/gh-aw/actions/install_copilot_cli.sh 0.0.410 + - name: Install awf binary + run: bash /opt/gh-aw/actions/install_awf_binary.sh v0.19.1 + - name: Determine automatic lockdown mode for GitHub MCP Server + id: determine-automatic-lockdown + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_GITHUB_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN }} + GH_AW_GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }} + with: + script: | + const determineAutomaticLockdown = require('/opt/gh-aw/actions/determine_automatic_lockdown.cjs'); + await determineAutomaticLockdown(github, context, core); + - name: Download container images + run: bash /opt/gh-aw/actions/download_docker_images.sh ghcr.io/github/gh-aw-firewall/agent:0.19.1 ghcr.io/github/gh-aw-firewall/squid:0.19.1 ghcr.io/github/gh-aw-mcpg:v0.1.4 ghcr.io/github/github-mcp-server:v0.30.3 node:lts-alpine + - name: Write Safe Outputs Config + run: | + mkdir -p /opt/gh-aw/safeoutputs + mkdir -p /tmp/gh-aw/safeoutputs + mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs + cat > /opt/gh-aw/safeoutputs/config.json << 'GH_AW_SAFE_OUTPUTS_CONFIG_EOF' + {"add_comment":{"max":1},"create_issue":{"max":1},"dispatch_workflow":{"max":10,"workflow_files":{"devops-health-investigate":".lock.yml"},"workflows":["devops-health-investigate"]},"missing_data":{},"missing_tool":{},"noop":{"max":1},"update_issue":{"max":1}} + GH_AW_SAFE_OUTPUTS_CONFIG_EOF + cat > /opt/gh-aw/safeoutputs/tools.json << 'GH_AW_SAFE_OUTPUTS_TOOLS_EOF' + [ + { + "description": "Create a new GitHub issue for tracking bugs, feature requests, or tasks. Use this for actionable work items that need assignment, labeling, and status tracking. For reports, announcements, or status updates that don't require task tracking, use create_discussion instead. CONSTRAINTS: Maximum 1 issue(s) can be created.", + "inputSchema": { + "additionalProperties": false, + "properties": { + "body": { + "description": "Detailed issue description in Markdown. Do NOT repeat the title as a heading since it already appears as the issue's h1. Include context, reproduction steps, or acceptance criteria as appropriate.", + "type": "string" + }, + "labels": { + "description": "Labels to categorize the issue (e.g., 'bug', 'enhancement'). Labels must exist in the repository.", + "items": { + "type": "string" + }, + "type": "array" + }, + "parent": { + "description": "Parent issue number for creating sub-issues. This is the numeric ID from the GitHub URL (e.g., 42 in github.com/owner/repo/issues/42). Can also be a temporary_id (e.g., 'aw_abc123', 'aw_Test123') from a previously created issue in the same workflow run.", + "type": [ + "number", + "string" + ] + }, + "temporary_id": { + "description": "Unique temporary identifier for referencing this issue before it's created. Format: 'aw_' followed by 3 to 8 alphanumeric characters (e.g., 'aw_abc1', 'aw_Test123'). Use '#aw_ID' in body text to reference other issues by their temporary_id; these are replaced with actual issue numbers after creation.", + "pattern": "^aw_[A-Za-z0-9]{3,8}$", + "type": "string" + }, + "title": { + "description": "Concise issue title summarizing the bug, feature, or task. The title appears as the main heading, so keep it brief and descriptive.", + "type": "string" + } + }, + "required": [ + "title", + "body" + ], + "type": "object" + }, + "name": "create_issue" + }, + { + "description": "Add a comment to an existing GitHub issue, pull request, or discussion. Use this to provide feedback, answer questions, or add information to an existing conversation. For creating new items, use create_issue, create_discussion, or create_pull_request instead. IMPORTANT: Comments are subject to validation constraints enforced by the MCP server - maximum 65536 characters for the complete comment (including footer which is added automatically), 10 mentions (@username), and 50 links. Exceeding these limits will result in an immediate error with specific guidance. CONSTRAINTS: Maximum 1 comment(s) can be added.", + "inputSchema": { + "additionalProperties": false, + "properties": { + "body": { + "description": "The comment text in Markdown format. This is the 'body' field - do not use 'comment_body' or other variations. Provide helpful, relevant information that adds value to the conversation. CONSTRAINTS: The complete comment (your body text + automatically added footer) must not exceed 65536 characters total. Maximum 10 mentions (@username), maximum 50 links (http/https URLs). A footer (~200-500 characters) is automatically appended with workflow attribution, so leave adequate space. If these limits are exceeded, the tool call will fail with a detailed error message indicating which constraint was violated.", + "type": "string" + }, + "item_number": { + "description": "The issue, pull request, or discussion number to comment on. This is the numeric ID from the GitHub URL (e.g., 123 in github.com/owner/repo/issues/123). If omitted, the tool will attempt to resolve the target from the current workflow context (triggering issue, PR, or discussion).", + "type": "number" + } + }, + "required": [ + "body" + ], + "type": "object" + }, + "name": "add_comment" + }, + { + "description": "Update an existing GitHub issue's title, body, labels, assignees, or milestone WITHOUT closing it. This tool is primarily for editing issue metadata and content. While it supports changing status between 'open' and 'closed', use close_issue instead when you want to close an issue with a closing comment. Body updates support replacing, appending to, prepending content, or updating a per-run \"island\" section. CONSTRAINTS: Maximum 1 issue(s) can be updated.", + "inputSchema": { + "additionalProperties": false, + "properties": { + "assignees": { + "description": "Replace the issue assignees with this list of GitHub usernames (e.g., ['octocat', 'mona']).", + "items": { + "type": "string" + }, + "type": "array" + }, + "body": { + "description": "Issue body content in Markdown. For 'replace', this becomes the entire body. For 'append'/'prepend', this content is added with a separator and an attribution footer. For 'replace-island', only the run-specific section is updated.", + "type": "string" + }, + "issue_number": { + "description": "Issue number to update. This is the numeric ID from the GitHub URL (e.g., 789 in github.com/owner/repo/issues/789). Required when the workflow target is '*' (any issue).", + "type": [ + "number", + "string" + ] + }, + "labels": { + "description": "Replace the issue labels with this list (e.g., ['bug', 'tracking:foo']). Labels must exist in the repository.", + "items": { + "type": "string" + }, + "type": "array" + }, + "milestone": { + "description": "Milestone number to assign (e.g., 1). Use null to clear.", + "type": [ + "number", + "string" + ] + }, + "operation": { + "description": "How to update the issue body: 'append' (default - add to end with separator), 'prepend' (add to start with separator), 'replace' (overwrite entire body), or 'replace-island' (update a run-specific section).", + "enum": [ + "replace", + "append", + "prepend", + "replace-island" + ], + "type": "string" + }, + "status": { + "description": "New issue status: 'open' to reopen a closed issue, 'closed' to close an open issue.", + "enum": [ + "open", + "closed" + ], + "type": "string" + }, + "title": { + "description": "New issue title to replace the existing title.", + "type": "string" + } + }, + "type": "object" + }, + "name": "update_issue" + }, + { + "description": "Report that a tool or capability needed to complete the task is not available, or share any information you deem important about missing functionality or limitations. Use this when you cannot accomplish what was requested because the required functionality is missing or access is restricted.", + "inputSchema": { + "additionalProperties": false, + "properties": { + "alternatives": { + "description": "Any workarounds, manual steps, or alternative approaches the user could take (max 256 characters).", + "type": "string" + }, + "reason": { + "description": "Explanation of why this tool is needed or what information you want to share about the limitation (max 256 characters).", + "type": "string" + }, + "tool": { + "description": "Optional: Name or description of the missing tool or capability (max 128 characters). Be specific about what functionality is needed.", + "type": "string" + } + }, + "required": [ + "reason" + ], + "type": "object" + }, + "name": "missing_tool" + }, + { + "description": "Log a transparency message when no significant actions are needed. Use this to confirm workflow completion and provide visibility when analysis is complete but no changes or outputs are required (e.g., 'No issues found', 'All checks passed'). This ensures the workflow produces human-visible output even when no other actions are taken.", + "inputSchema": { + "additionalProperties": false, + "properties": { + "message": { + "description": "Status or completion message to log. Should explain what was analyzed and the outcome (e.g., 'Code review complete - no issues found', 'Analysis complete - all tests passing').", + "type": "string" + } + }, + "required": [ + "message" + ], + "type": "object" + }, + "name": "noop" + }, + { + "description": "Report that data or information needed to complete the task is not available. Use this when you cannot accomplish what was requested because required data, context, or information is missing.", + "inputSchema": { + "additionalProperties": false, + "properties": { + "alternatives": { + "description": "Any workarounds, manual steps, or alternative approaches the user could take (max 256 characters).", + "type": "string" + }, + "context": { + "description": "Additional context about the missing data or where it should come from (max 256 characters).", + "type": "string" + }, + "data_type": { + "description": "Type or description of the missing data or information (max 128 characters). Be specific about what data is needed.", + "type": "string" + }, + "reason": { + "description": "Explanation of why this data is needed to complete the task (max 256 characters).", + "type": "string" + } + }, + "required": [], + "type": "object" + }, + "name": "missing_data" + }, + { + "_workflow_name": "devops-health-investigate", + "description": "Dispatch the 'devops-health-investigate' workflow with workflow_dispatch trigger. This workflow must support workflow_dispatch and be in .github/workflows/ directory in the same repository.", + "inputSchema": { + "additionalProperties": false, + "properties": { + "correlation_id": { + "description": "Unique ID linking this investigation to the health check run", + "type": "string" + }, + "finding_id": { + "description": "Fingerprint ID of the finding to investigate", + "type": "string" + }, + "finding_severity": { + "description": "Severity: critical | warning | info", + "type": "string" + }, + "finding_title": { + "description": "Human-readable title of the finding", + "type": "string" + }, + "finding_type": { + "description": "Category: pipeline | quality | pr | infra | resource", + "type": "string" + }, + "health_issue_number": { + "description": "Issue number of the pinned health dashboard", + "type": "string" + }, + "resource_url": { + "description": "URL to the primary resource (run, PR, etc.)", + "type": "string" + } + }, + "required": [ + "finding_title", + "finding_type", + "health_issue_number", + "resource_url", + "correlation_id", + "finding_id", + "finding_severity" + ], + "type": "object" + }, + "name": "devops_health_investigate" + } + ] + GH_AW_SAFE_OUTPUTS_TOOLS_EOF + cat > /opt/gh-aw/safeoutputs/validation.json << 'GH_AW_SAFE_OUTPUTS_VALIDATION_EOF' + { + "add_comment": { + "defaultMax": 1, + "fields": { + "body": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "item_number": { + "issueOrPRNumber": true + } + } + }, + "create_issue": { + "defaultMax": 1, + "fields": { + "body": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "labels": { + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "parent": { + "issueOrPRNumber": true + }, + "repo": { + "type": "string", + "maxLength": 256 + }, + "temporary_id": { + "type": "string" + }, + "title": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 128 + } + } + }, + "missing_tool": { + "defaultMax": 20, + "fields": { + "alternatives": { + "type": "string", + "sanitize": true, + "maxLength": 512 + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 256 + }, + "tool": { + "type": "string", + "sanitize": true, + "maxLength": 128 + } + } + }, + "noop": { + "defaultMax": 1, + "fields": { + "message": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + } + } + }, + "update_issue": { + "defaultMax": 1, + "fields": { + "body": { + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "issue_number": { + "issueOrPRNumber": true + }, + "status": { + "type": "string", + "enum": [ + "open", + "closed" + ] + }, + "title": { + "type": "string", + "sanitize": true, + "maxLength": 128 + } + }, + "customValidation": "requiresOneOf:status,title,body" + } + } + GH_AW_SAFE_OUTPUTS_VALIDATION_EOF + - name: Generate Safe Outputs MCP Server Config + id: safe-outputs-config + run: | + # Generate a secure random API key (360 bits of entropy, 40+ chars) + # Mask immediately to prevent timing vulnerabilities + API_KEY=$(openssl rand -base64 45 | tr -d '/+=') + echo "::add-mask::${API_KEY}" + + PORT=3001 + + # Set outputs for next steps + { + echo "safe_outputs_api_key=${API_KEY}" + echo "safe_outputs_port=${PORT}" + } >> "$GITHUB_OUTPUT" + + echo "Safe Outputs MCP server will run on port ${PORT}" + + - name: Start Safe Outputs MCP HTTP Server + id: safe-outputs-start + env: + DEBUG: '*' + GH_AW_SAFE_OUTPUTS_PORT: ${{ steps.safe-outputs-config.outputs.safe_outputs_port }} + GH_AW_SAFE_OUTPUTS_API_KEY: ${{ steps.safe-outputs-config.outputs.safe_outputs_api_key }} + GH_AW_SAFE_OUTPUTS_TOOLS_PATH: /opt/gh-aw/safeoutputs/tools.json + GH_AW_SAFE_OUTPUTS_CONFIG_PATH: /opt/gh-aw/safeoutputs/config.json + GH_AW_MCP_LOG_DIR: /tmp/gh-aw/mcp-logs/safeoutputs + run: | + # Environment variables are set above to prevent template injection + export DEBUG + export GH_AW_SAFE_OUTPUTS_PORT + export GH_AW_SAFE_OUTPUTS_API_KEY + export GH_AW_SAFE_OUTPUTS_TOOLS_PATH + export GH_AW_SAFE_OUTPUTS_CONFIG_PATH + export GH_AW_MCP_LOG_DIR + + bash /opt/gh-aw/actions/start_safe_outputs_server.sh + + - name: Start MCP Gateway + id: start-mcp-gateway + env: + GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }} + GH_AW_SAFE_OUTPUTS_API_KEY: ${{ steps.safe-outputs-start.outputs.api_key }} + GH_AW_SAFE_OUTPUTS_PORT: ${{ steps.safe-outputs-start.outputs.port }} + GITHUB_MCP_LOCKDOWN: ${{ steps.determine-automatic-lockdown.outputs.lockdown == 'true' && '1' || '0' }} + GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + run: | + set -eo pipefail + mkdir -p /tmp/gh-aw/mcp-config + + # Export gateway environment variables for MCP config and gateway script + export MCP_GATEWAY_PORT="80" + export MCP_GATEWAY_DOMAIN="host.docker.internal" + MCP_GATEWAY_API_KEY=$(openssl rand -base64 45 | tr -d '/+=') + echo "::add-mask::${MCP_GATEWAY_API_KEY}" + export MCP_GATEWAY_API_KEY + export MCP_GATEWAY_PAYLOAD_DIR="/tmp/gh-aw/mcp-payloads" + mkdir -p "${MCP_GATEWAY_PAYLOAD_DIR}" + export DEBUG="*" + + export GH_AW_ENGINE="copilot" + export MCP_GATEWAY_DOCKER_COMMAND='docker run -i --rm --network host -v /var/run/docker.sock:/var/run/docker.sock -e MCP_GATEWAY_PORT -e MCP_GATEWAY_DOMAIN -e MCP_GATEWAY_API_KEY -e MCP_GATEWAY_PAYLOAD_DIR -e DEBUG -e MCP_GATEWAY_LOG_DIR -e GH_AW_MCP_LOG_DIR -e GH_AW_SAFE_OUTPUTS -e GH_AW_SAFE_OUTPUTS_CONFIG_PATH -e GH_AW_SAFE_OUTPUTS_TOOLS_PATH -e GH_AW_ASSETS_BRANCH -e GH_AW_ASSETS_MAX_SIZE_KB -e GH_AW_ASSETS_ALLOWED_EXTS -e DEFAULT_BRANCH -e GITHUB_MCP_SERVER_TOKEN -e GITHUB_MCP_LOCKDOWN -e GITHUB_REPOSITORY -e GITHUB_SERVER_URL -e GITHUB_SHA -e GITHUB_WORKSPACE -e GITHUB_TOKEN -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e GITHUB_JOB -e GITHUB_ACTION -e GITHUB_EVENT_NAME -e GITHUB_EVENT_PATH -e GITHUB_ACTOR -e GITHUB_ACTOR_ID -e GITHUB_TRIGGERING_ACTOR -e GITHUB_WORKFLOW -e GITHUB_WORKFLOW_REF -e GITHUB_WORKFLOW_SHA -e GITHUB_REF -e GITHUB_REF_NAME -e GITHUB_REF_TYPE -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GH_AW_SAFE_OUTPUTS_PORT -e GH_AW_SAFE_OUTPUTS_API_KEY -v /tmp/gh-aw/mcp-payloads:/tmp/gh-aw/mcp-payloads:rw -v /opt:/opt:ro -v /tmp:/tmp:rw -v '"${GITHUB_WORKSPACE}"':'"${GITHUB_WORKSPACE}"':rw ghcr.io/github/gh-aw-mcpg:v0.1.4' + + mkdir -p /home/runner/.copilot + cat << GH_AW_MCP_CONFIG_EOF | bash /opt/gh-aw/actions/start_mcp_gateway.sh + { + "mcpServers": { + "github": { + "type": "stdio", + "container": "ghcr.io/github/github-mcp-server:v0.30.3", + "env": { + "GITHUB_LOCKDOWN_MODE": "$GITHUB_MCP_LOCKDOWN", + "GITHUB_PERSONAL_ACCESS_TOKEN": "\${GITHUB_MCP_SERVER_TOKEN}", + "GITHUB_READ_ONLY": "1", + "GITHUB_TOOLSETS": "repos,issues,pull_requests,actions" + } + }, + "safeoutputs": { + "type": "http", + "url": "http://host.docker.internal:$GH_AW_SAFE_OUTPUTS_PORT", + "headers": { + "Authorization": "\${GH_AW_SAFE_OUTPUTS_API_KEY}" + } + } + }, + "gateway": { + "port": $MCP_GATEWAY_PORT, + "domain": "${MCP_GATEWAY_DOMAIN}", + "apiKey": "${MCP_GATEWAY_API_KEY}", + "payloadDir": "${MCP_GATEWAY_PAYLOAD_DIR}" + } + } + GH_AW_MCP_CONFIG_EOF + - name: Generate workflow overview + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { generateWorkflowOverview } = require('/opt/gh-aw/actions/generate_workflow_overview.cjs'); + await generateWorkflowOverview(core); + - name: Download prompt artifact + uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6 + with: + name: prompt + path: /tmp/gh-aw/aw-prompts + - name: Clean git credentials + run: bash /opt/gh-aw/actions/clean_git_credentials.sh + - name: Execute GitHub Copilot CLI + id: agentic_execution + # Copilot CLI tool arguments (sorted): + # --allow-tool github + # --allow-tool safeoutputs + # --allow-tool shell(cat) + # --allow-tool shell(date) + # --allow-tool shell(diff) + # --allow-tool shell(echo) + # --allow-tool shell(find) + # --allow-tool shell(grep) + # --allow-tool shell(head) + # --allow-tool shell(jq) + # --allow-tool shell(ls) + # --allow-tool shell(pwd) + # --allow-tool shell(sort) + # --allow-tool shell(tail) + # --allow-tool shell(uniq) + # --allow-tool shell(wc) + # --allow-tool shell(yq) + # --allow-tool write + timeout-minutes: 20 + run: | + set -o pipefail + sudo -E awf --env-all --container-workdir "${GITHUB_WORKSPACE}" --allow-domains api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --enable-host-access --image-tag 0.19.1 --skip-pull \ + -- /bin/bash -c '/usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --add-dir "${GITHUB_WORKSPACE}" --disable-builtin-mcps --allow-tool github --allow-tool safeoutputs --allow-tool '\''shell(cat)'\'' --allow-tool '\''shell(date)'\'' --allow-tool '\''shell(diff)'\'' --allow-tool '\''shell(echo)'\'' --allow-tool '\''shell(find)'\'' --allow-tool '\''shell(grep)'\'' --allow-tool '\''shell(head)'\'' --allow-tool '\''shell(jq)'\'' --allow-tool '\''shell(ls)'\'' --allow-tool '\''shell(pwd)'\'' --allow-tool '\''shell(sort)'\'' --allow-tool '\''shell(tail)'\'' --allow-tool '\''shell(uniq)'\'' --allow-tool '\''shell(wc)'\'' --allow-tool '\''shell(yq)'\'' --allow-tool write --add-dir /tmp/gh-aw/cache-memory/ --allow-all-paths --share /tmp/gh-aw/sandbox/agent/logs/conversation.md --prompt "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"${GH_AW_MODEL_AGENT_COPILOT:+ --model "$GH_AW_MODEL_AGENT_COPILOT"}' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log + env: + COPILOT_AGENT_RUNNER_TYPE: STANDALONE + COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} + GH_AW_MCP_CONFIG: /home/runner/.copilot/mcp-config.json + GH_AW_MODEL_AGENT_COPILOT: ${{ vars.GH_AW_MODEL_AGENT_COPILOT || '' }} + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }} + GITHUB_HEAD_REF: ${{ github.head_ref }} + GITHUB_REF_NAME: ${{ github.ref_name }} + GITHUB_STEP_SUMMARY: ${{ env.GITHUB_STEP_SUMMARY }} + GITHUB_WORKSPACE: ${{ github.workspace }} + XDG_CONFIG_HOME: /home/runner + - name: Configure Git credentials + env: + REPO_NAME: ${{ github.repository }} + SERVER_URL: ${{ github.server_url }} + run: | + git config --global user.email "github-actions[bot]@users.noreply.github.com" + git config --global user.name "github-actions[bot]" + # Re-authenticate git with GitHub token + SERVER_URL_STRIPPED="${SERVER_URL#https://}" + git remote set-url origin "https://x-access-token:${{ github.token }}@${SERVER_URL_STRIPPED}/${REPO_NAME}.git" + echo "Git configured with standard GitHub Actions identity" + - name: Copy Copilot session state files to logs + if: always() + continue-on-error: true + run: | + # Copy Copilot session state files to logs folder for artifact collection + # This ensures they are in /tmp/gh-aw/ where secret redaction can scan them + SESSION_STATE_DIR="$HOME/.copilot/session-state" + LOGS_DIR="/tmp/gh-aw/sandbox/agent/logs" + + if [ -d "$SESSION_STATE_DIR" ]; then + echo "Copying Copilot session state files from $SESSION_STATE_DIR to $LOGS_DIR" + mkdir -p "$LOGS_DIR" + cp -v "$SESSION_STATE_DIR"/*.jsonl "$LOGS_DIR/" 2>/dev/null || true + echo "Session state files copied successfully" + else + echo "No session-state directory found at $SESSION_STATE_DIR" + fi + - name: Stop MCP Gateway + if: always() + continue-on-error: true + env: + MCP_GATEWAY_PORT: ${{ steps.start-mcp-gateway.outputs.gateway-port }} + MCP_GATEWAY_API_KEY: ${{ steps.start-mcp-gateway.outputs.gateway-api-key }} + GATEWAY_PID: ${{ steps.start-mcp-gateway.outputs.gateway-pid }} + run: | + bash /opt/gh-aw/actions/stop_mcp_gateway.sh "$GATEWAY_PID" + - name: Redact secrets in logs + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/redact_secrets.cjs'); + await main(); + env: + GH_AW_SECRET_NAMES: 'COPILOT_GITHUB_TOKEN,GH_AW_GITHUB_MCP_SERVER_TOKEN,GH_AW_GITHUB_TOKEN,GITHUB_TOKEN' + SECRET_COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} + SECRET_GH_AW_GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }} + SECRET_GH_AW_GITHUB_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN }} + SECRET_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + - name: Upload Safe Outputs + if: always() + uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6 + with: + name: safe-output + path: ${{ env.GH_AW_SAFE_OUTPUTS }} + if-no-files-found: warn + - name: Ingest agent output + id: collect_output + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }} + GH_AW_ALLOWED_DOMAINS: "api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com" + GITHUB_SERVER_URL: ${{ github.server_url }} + GITHUB_API_URL: ${{ github.api_url }} + GH_AW_COMMAND: health-check + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/collect_ndjson_output.cjs'); + await main(); + - name: Upload sanitized agent output + if: always() && env.GH_AW_AGENT_OUTPUT + uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6 + with: + name: agent-output + path: ${{ env.GH_AW_AGENT_OUTPUT }} + if-no-files-found: warn + - name: Upload engine output files + uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6 + with: + name: agent_outputs + path: | + /tmp/gh-aw/sandbox/agent/logs/ + /tmp/gh-aw/redacted-urls.log + if-no-files-found: ignore + - name: Parse agent logs for step summary + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: /tmp/gh-aw/sandbox/agent/logs/ + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/parse_copilot_log.cjs'); + await main(); + - name: Parse MCP Gateway logs for step summary + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/parse_mcp_gateway_log.cjs'); + await main(); + - name: Print firewall logs + if: always() + continue-on-error: true + env: + AWF_LOGS_DIR: /tmp/gh-aw/sandbox/firewall/logs + run: | + # Fix permissions on firewall logs so they can be uploaded as artifacts + # AWF runs with sudo, creating files owned by root + sudo chmod -R a+r /tmp/gh-aw/sandbox/firewall/logs 2>/dev/null || true + # Only run awf logs summary if awf command exists (it may not be installed if workflow failed before install step) + if command -v awf &> /dev/null; then + awf logs summary | tee -a "$GITHUB_STEP_SUMMARY" + else + echo 'AWF binary not installed, skipping firewall log summary' + fi + - name: Upload cache-memory data as artifact + uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6 + if: always() + with: + name: cache-memory + path: /tmp/gh-aw/cache-memory + - name: Upload agent artifacts + if: always() + continue-on-error: true + uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6 + with: + name: agent-artifacts + path: | + /tmp/gh-aw/aw-prompts/prompt.txt + /tmp/gh-aw/aw_info.json + /tmp/gh-aw/mcp-logs/ + /tmp/gh-aw/sandbox/firewall/logs/ + /tmp/gh-aw/agent-stdio.log + /tmp/gh-aw/agent/ + if-no-files-found: ignore + + conclusion: + needs: + - activation + - agent + - detection + - safe_outputs + - update_cache_memory + if: (always()) && (needs.agent.result != 'skipped') + runs-on: ubuntu-slim + permissions: + actions: write + contents: read + discussions: write + issues: write + pull-requests: write + outputs: + noop_message: ${{ steps.noop.outputs.noop_message }} + tools_reported: ${{ steps.missing_tool.outputs.tools_reported }} + total_count: ${{ steps.missing_tool.outputs.total_count }} + steps: + - name: Setup Scripts + uses: github/gh-aw/actions/setup@ac090214a48a1938f7abafe132460b66752261af # v0.45.4 + with: + destination: /opt/gh-aw/actions + - name: Download agent output artifact + continue-on-error: true + uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6 + with: + name: agent-output + path: /tmp/gh-aw/safeoutputs/ + - name: Setup agent output environment variable + run: | + mkdir -p /tmp/gh-aw/safeoutputs/ + find "/tmp/gh-aw/safeoutputs/" -type f -print + echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/safeoutputs/agent_output.json" >> "$GITHUB_ENV" + - name: Process No-Op Messages + id: noop + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_NOOP_MAX: 1 + GH_AW_WORKFLOW_NAME: "DevOps Daily Health Check" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/noop.cjs'); + await main(); + - name: Record Missing Tool + id: missing_tool + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_WORKFLOW_NAME: "DevOps Daily Health Check" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/missing_tool.cjs'); + await main(); + - name: Handle Agent Failure + id: handle_agent_failure + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_WORKFLOW_NAME: "DevOps Daily Health Check" + GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }} + GH_AW_WORKFLOW_ID: "devops-health-check" + GH_AW_SECRET_VERIFICATION_RESULT: ${{ needs.agent.outputs.secret_verification_result }} + GH_AW_CHECKOUT_PR_SUCCESS: ${{ needs.agent.outputs.checkout_pr_success }} + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/handle_agent_failure.cjs'); + await main(); + - name: Handle No-Op Message + id: handle_noop_message + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_WORKFLOW_NAME: "DevOps Daily Health Check" + GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }} + GH_AW_NOOP_MESSAGE: ${{ steps.noop.outputs.noop_message }} + GH_AW_NOOP_REPORT_AS_ISSUE: "true" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/handle_noop_message.cjs'); + await main(); + + detection: + needs: agent + if: needs.agent.outputs.output_types != '' || needs.agent.outputs.has_patch == 'true' + runs-on: ubuntu-latest + permissions: {} + timeout-minutes: 10 + outputs: + success: ${{ steps.parse_results.outputs.success }} + steps: + - name: Setup Scripts + uses: github/gh-aw/actions/setup@ac090214a48a1938f7abafe132460b66752261af # v0.45.4 + with: + destination: /opt/gh-aw/actions + - name: Download agent artifacts + continue-on-error: true + uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6 + with: + name: agent-artifacts + path: /tmp/gh-aw/threat-detection/ + - name: Download agent output artifact + continue-on-error: true + uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6 + with: + name: agent-output + path: /tmp/gh-aw/threat-detection/ + - name: Echo agent output types + env: + AGENT_OUTPUT_TYPES: ${{ needs.agent.outputs.output_types }} + run: | + echo "Agent output-types: $AGENT_OUTPUT_TYPES" + - name: Setup threat detection + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + WORKFLOW_NAME: "DevOps Daily Health Check" + WORKFLOW_DESCRIPTION: "Orchestrator workflow that collects repo health signals daily (pipelines, skill quality, PRs, infrastructure), computes a fingerprint-based diff against the previous run, updates a pinned health dashboard issue, and dispatches investigation workers for new critical/warning findings." + HAS_PATCH: ${{ needs.agent.outputs.has_patch }} + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/setup_threat_detection.cjs'); + await main(); + - name: Ensure threat-detection directory and log + run: | + mkdir -p /tmp/gh-aw/threat-detection + touch /tmp/gh-aw/threat-detection/detection.log + - name: Validate COPILOT_GITHUB_TOKEN secret + id: validate-secret + run: /opt/gh-aw/actions/validate_multi_secret.sh COPILOT_GITHUB_TOKEN 'GitHub Copilot CLI' https://github.github.com/gh-aw/reference/engines/#github-copilot-default + env: + COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} + - name: Install GitHub Copilot CLI + run: /opt/gh-aw/actions/install_copilot_cli.sh 0.0.410 + - name: Execute GitHub Copilot CLI + id: agentic_execution + # Copilot CLI tool arguments (sorted): + # --allow-tool shell(cat) + # --allow-tool shell(grep) + # --allow-tool shell(head) + # --allow-tool shell(jq) + # --allow-tool shell(ls) + # --allow-tool shell(tail) + # --allow-tool shell(wc) + timeout-minutes: 20 + run: | + set -o pipefail + COPILOT_CLI_INSTRUCTION="$(cat /tmp/gh-aw/aw-prompts/prompt.txt)" + mkdir -p /tmp/ + mkdir -p /tmp/gh-aw/ + mkdir -p /tmp/gh-aw/agent/ + mkdir -p /tmp/gh-aw/sandbox/agent/logs/ + copilot --add-dir /tmp/ --add-dir /tmp/gh-aw/ --add-dir /tmp/gh-aw/agent/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --allow-tool 'shell(cat)' --allow-tool 'shell(grep)' --allow-tool 'shell(head)' --allow-tool 'shell(jq)' --allow-tool 'shell(ls)' --allow-tool 'shell(tail)' --allow-tool 'shell(wc)' --share /tmp/gh-aw/sandbox/agent/logs/conversation.md --prompt "$COPILOT_CLI_INSTRUCTION"${GH_AW_MODEL_DETECTION_COPILOT:+ --model "$GH_AW_MODEL_DETECTION_COPILOT"} 2>&1 | tee /tmp/gh-aw/threat-detection/detection.log + env: + COPILOT_AGENT_RUNNER_TYPE: STANDALONE + COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} + GH_AW_MODEL_DETECTION_COPILOT: ${{ vars.GH_AW_MODEL_DETECTION_COPILOT || '' }} + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GITHUB_HEAD_REF: ${{ github.head_ref }} + GITHUB_REF_NAME: ${{ github.ref_name }} + GITHUB_STEP_SUMMARY: ${{ env.GITHUB_STEP_SUMMARY }} + GITHUB_WORKSPACE: ${{ github.workspace }} + XDG_CONFIG_HOME: /home/runner + - name: Parse threat detection results + id: parse_results + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/parse_threat_detection_results.cjs'); + await main(); + - name: Upload threat detection log + if: always() + uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6 + with: + name: threat-detection.log + path: /tmp/gh-aw/threat-detection/detection.log + if-no-files-found: ignore + + pre_activation: + if: > + ((github.event_name == 'issues' || github.event_name == 'issue_comment' || github.event_name == 'pull_request' || + github.event_name == 'pull_request_review_comment' || github.event_name == 'discussion' || github.event_name == 'discussion_comment') && + ((github.event_name == 'issues') && ((startsWith(github.event.issue.body, '/health-check ')) || (github.event.issue.body == '/health-check')) || + (github.event_name == 'issue_comment') && (((startsWith(github.event.comment.body, '/health-check ')) || + (github.event.comment.body == '/health-check')) && (github.event.issue.pull_request == null)) || (github.event_name == 'issue_comment') && + (((startsWith(github.event.comment.body, '/health-check ')) || (github.event.comment.body == '/health-check')) && + (github.event.issue.pull_request != null)) || (github.event_name == 'pull_request_review_comment') && + ((startsWith(github.event.comment.body, '/health-check ')) || (github.event.comment.body == '/health-check')) || + (github.event_name == 'pull_request') && ((startsWith(github.event.pull_request.body, '/health-check ')) || + (github.event.pull_request.body == '/health-check')) || (github.event_name == 'discussion') && ((startsWith(github.event.discussion.body, '/health-check ')) || + (github.event.discussion.body == '/health-check')) || (github.event_name == 'discussion_comment') && ((startsWith(github.event.comment.body, '/health-check ')) || + (github.event.comment.body == '/health-check')))) || (!(github.event_name == 'issues' || github.event_name == 'issue_comment' || + github.event_name == 'pull_request' || github.event_name == 'pull_request_review_comment' || github.event_name == 'discussion' || + github.event_name == 'discussion_comment')) + runs-on: ubuntu-slim + permissions: + discussions: write + issues: write + pull-requests: write + outputs: + activated: ${{ (steps.check_membership.outputs.is_team_member == 'true') && (steps.check_command_position.outputs.command_position_ok == 'true') }} + matched_command: ${{ steps.check_command_position.outputs.matched_command }} + steps: + - name: Setup Scripts + uses: github/gh-aw/actions/setup@ac090214a48a1938f7abafe132460b66752261af # v0.45.4 + with: + destination: /opt/gh-aw/actions + - name: Add eyes reaction for immediate feedback + id: react + if: github.event_name == 'issues' || github.event_name == 'issue_comment' || github.event_name == 'pull_request_review_comment' || github.event_name == 'discussion' || github.event_name == 'discussion_comment' || (github.event_name == 'pull_request') && (github.event.pull_request.head.repo.id == github.repository_id) + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_REACTION: "eyes" + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/add_reaction.cjs'); + await main(); + - name: Check team membership for command workflow + id: check_membership + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_REQUIRED_ROLES: admin,maintainer,write + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/check_membership.cjs'); + await main(); + - name: Check command position + id: check_command_position + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_COMMANDS: "[\"health-check\"]" + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/check_command_position.cjs'); + await main(); + + safe_outputs: + needs: + - agent + - detection + if: ((!cancelled()) && (needs.agent.result != 'skipped')) && (needs.detection.outputs.success == 'true') + runs-on: ubuntu-slim + permissions: + actions: write + contents: read + discussions: write + issues: write + pull-requests: write + timeout-minutes: 15 + env: + GH_AW_ENGINE_ID: "copilot" + GH_AW_WORKFLOW_ID: "devops-health-check" + GH_AW_WORKFLOW_NAME: "DevOps Daily Health Check" + outputs: + create_discussion_error_count: ${{ steps.process_safe_outputs.outputs.create_discussion_error_count }} + create_discussion_errors: ${{ steps.process_safe_outputs.outputs.create_discussion_errors }} + process_safe_outputs_processed_count: ${{ steps.process_safe_outputs.outputs.processed_count }} + process_safe_outputs_temporary_id_map: ${{ steps.process_safe_outputs.outputs.temporary_id_map }} + steps: + - name: Setup Scripts + uses: github/gh-aw/actions/setup@ac090214a48a1938f7abafe132460b66752261af # v0.45.4 + with: + destination: /opt/gh-aw/actions + - name: Download agent output artifact + continue-on-error: true + uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6 + with: + name: agent-output + path: /tmp/gh-aw/safeoutputs/ + - name: Setup agent output environment variable + run: | + mkdir -p /tmp/gh-aw/safeoutputs/ + find "/tmp/gh-aw/safeoutputs/" -type f -print + echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/safeoutputs/agent_output.json" >> "$GITHUB_ENV" + - name: Process Safe Outputs + id: process_safe_outputs + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"max\":1},\"create_issue\":{\"max\":1},\"dispatch_workflow\":{\"max\":10,\"workflow_files\":{\"devops-health-investigate\":\".lock.yml\"},\"workflows\":[\"devops-health-investigate\"]},\"missing_data\":{},\"missing_tool\":{},\"update_issue\":{\"allow_body\":true,\"max\":1}}" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/safe_output_handler_manager.cjs'); + await main(); + + update_cache_memory: + needs: + - agent + - detection + if: always() && needs.detection.outputs.success == 'true' + runs-on: ubuntu-latest + permissions: {} + steps: + - name: Setup Scripts + uses: github/gh-aw/actions/setup@ac090214a48a1938f7abafe132460b66752261af # v0.45.4 + with: + destination: /opt/gh-aw/actions + - name: Download cache-memory artifact (default) + uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6 + continue-on-error: true + with: + name: cache-memory + path: /tmp/gh-aw/cache-memory + - name: Save cache-memory to cache (default) + uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0 + with: + key: memory-${{ env.GH_AW_WORKFLOW_ID_SANITIZED }}-${{ github.run_id }} + path: /tmp/gh-aw/cache-memory + diff --git a/.github/workflows/devops-health-check.md b/.github/workflows/devops-health-check.md new file mode 100644 index 00000000..c7c4e530 --- /dev/null +++ b/.github/workflows/devops-health-check.md @@ -0,0 +1,461 @@ +--- +name: "DevOps Daily Health Check" +description: > + Orchestrator workflow that collects repo health signals daily (pipelines, + skill quality, PRs, infrastructure), computes a fingerprint-based diff + against the previous run, updates a pinned health dashboard issue, and + dispatches investigation workers for new critical/warning findings. + +on: + schedule: daily + slash_command: health-check + workflow_dispatch: + +permissions: + contents: read + actions: read + issues: read + pull-requests: read + +imports: + - ../aw/shared/devops-health.lock.md + +tools: + github: + toolsets: [repos, issues, pull_requests, actions] + cache-memory: + bash: ["cat", "grep", "head", "tail", "find", "ls", "wc", "jq", "date", "sort", "uniq", "diff"] + edit: + +safe-outputs: + create-issue: + max: 1 + update-issue: + max: 1 + add-comment: + max: 1 + dispatch-workflow: + workflows: + - devops-health-investigate + max: 10 + +network: + allowed: + - defaults +--- + +# DevOps Daily Health Check — Orchestrator + +You are a DevOps health monitoring agent. Your job is to collect repo health signals, compute a diff against the previous run, and produce a comprehensive yet actionable health dashboard. + +## High-Level Workflow + +1. **Data Collection** (deterministic — use API calls and bash tools) +2. **Fingerprint & Diff** (compare against previous run via `cache-memory`) +3. **Analysis** (LLM-powered: correlate findings, identify root causes, write summary) +4. **Output** (update pinned issue + post daily comment) +5. **Triage Dispatch** (dispatch investigation workers for new critical/warning findings) + +--- + +## Step 1: Data Collection + +### 1.1 Discover Components + +Scan the repository to find all skill components: + +``` +find src/*/plugin.json -maxdepth 2 +``` + +Each `src/{name}/` directory containing a `plugin.json` is a component. The corresponding dashboard data file is `data/{name}.json` on `gh-pages`. + +### 1.2 Pipeline Health (P1–P4) + +**P1 — Failed workflow runs on `main` in last 24h:** +``` +GET /repos/{owner}/{repo}/actions/runs?branch=main&status=failure&per_page=30 +``` +Filter to runs created within the last 24 hours. For each failed run: +- Extract `workflow_name`, `conclusion`, `job_name`, `failed_step` +- Fingerprint: `pipeline:{workflow_name}:{job_name}:{failed_step}:{conclusion}` +- Severity: 🔴 Critical if `evaluation` workflow fails; 🟡 Warning for others +- **Noise suppression:** Check if the finding matches any pattern in the `known-noise` list from `cache-memory`. If it matches, demote severity to 🔵 Info. + +**P2 — Cancelled/timed-out runs in last 24h:** +``` +GET /repos/{owner}/{repo}/actions/runs?branch=main&status=cancelled&per_page=10 +``` +- Fingerprint: `pipeline:{workflow_name}:{job_name}:timeout` +- Severity: 🟡 Warning + +**P3 — Evaluation duration trend:** +``` +GET /repos/{owner}/{repo}/actions/workflows/evaluation.yml/runs?branch=main&per_page=30 +``` +Compute average run duration over the last 14 days. +- 🟡 Warning if avg > 50 min (83% of 60-min timeout) +- 🔴 Critical if avg > 55 min +- Fingerprint: `resource:eval-duration:{bucket}` (bucket = "warning" or "critical") + +**P4 — Workflow failure rate (7-day rolling):** +``` +GET /repos/{owner}/{repo}/actions/runs?branch=main&per_page=100 +``` +Group by workflow name, compute success/failure ratio over the last 7 days. +- 🔵 Info (metric only — reported in trends table, not fingerprinted) + +### 1.3 Skill Quality (Q1–Q7) + +Fetch benchmark data for each discovered component: +``` +GET https://raw.githubusercontent.com/{owner}/{repo}/gh-pages/data/{component}.json +``` + +**Q1 — Skill inventory overview table:** +Compile a comprehensive table of all skills combining local discovery with benchmark data. For each skill, classify its health status: +- **🟢 OK** — Skill has tests, scenarios pass, skilled > vanilla, no anomaly flags +- **🟡 Warning** — Skill is functional but has issues: timeouts, overfitting, or high variance (stddev > 1.5) +- **🟡 Low Value** — Some scenarios show skilled ≤ vanilla (but others show uplift) +- **🔴 No Value** — All scenarios show skilled ≤ vanilla (skill adds nothing) +- **🔴 Critical** — Skill not activated by the agent (`notActivated` flag) +- **⚪ Untested** — No test directory, no eval.yaml, or eval.yaml has 0 scenarios +- **⚪ No Data** — Skill exists locally but has no benchmark data + +This table is informational (🔵 Info) and not fingerprinted. It is rendered in the issue body as a dedicated "Skill Inventory" section. + +For each skill, compute: +- **Avg Skilled score**: average of all scenario "Skilled Quality" bench values in the latest entry +- **Avg Vanilla score**: average of all scenario "Vanilla Quality" bench values in the latest entry +- **Delta**: Skilled − Vanilla +- **Scenario count**: number of scenarios with benchmark data +- **Issue summary**: comma-separated list of issues (timeout, overfitting, no-uplift, high-variance, etc.) + +**Q2 — Bench entries with anomaly flags:** +Scan the latest entry in **both** `entries.Quality` and `entries.Efficiency` arrays. For each bench entry, check for any property beyond the standard `name`/`unit`/`value` fields. Any extra boolean property is an anomaly flag (e.g., `notActivated`, `timedOut`, `testOverfitted`, or future flags). +- Extract the skill name and scenario from the bench `name` field (format: `"{skill}/{scenario} - {metric}"`) +- 🔴 Critical if `notActivated` (skill broken) +- 🟡 Warning for all other flags +- Fingerprint: `quality:{skill}:{scenario}:{flag-name}` +- **Deduplicate:** If the same skill/scenario/flag appears in both Quality and Efficiency arrays, report it only once. + +**Q3 — Quality regression (>1.0 point drop vs 7-day rolling avg):** +For each scenario's "Skilled Quality" bench, compare the latest value to the rolling average of all entries from the last 7 calendar days (filter by `date` field). +- 🔴 Critical if drop > 2.0 points +- 🟡 Warning if drop > 1.0 points +- Fingerprint: `quality:{skill}:{scenario}:regressed` + +**Q4 — Skilled ≤ Vanilla (skill adds no value):** +For the latest entry, compare `"{skill}/{scenario} - Skilled Quality"` vs `"{skill}/{scenario} - Vanilla Quality"` bench values. +- 🟡 Warning if Skilled ≤ Vanilla +- Fingerprint: `quality:{skill}:{scenario}:no-uplift` + +**Q5 — High variance across runs:** +Compute the standard deviation of `"Skilled Quality"` scores across all entries from the last 7 calendar days. +- 🟡 Warning if stddev > 1.5 +- Fingerprint: `quality:{skill}:{scenario}:high-variance` + +**Q6 — Skills without eval tests:** +``` +find src/*/skills/ -mindepth 1 -maxdepth 1 -type d +``` +For each skill directory, check if a corresponding test directory exists under `src/{component}/tests/{skill-name}/`. +If the test directory exists, verify that `eval.yaml` exists and contains at least one scenario. +- 🟡 Warning if no test directory, no eval.yaml, or eval.yaml has no scenarios +- Fingerprint: `coverage:{skill}:no-tests` + +**Q7 — Benchmark data staleness:** +Check if the latest entry's `date` timestamp is > 24h old (compare to current time). +- 🟡 Warning (pipeline may not be publishing) +- Fingerprint: `quality:benchmark-stale:{component}` + +### 1.4 PR & Review Health (R1–R5) + +``` +GET /repos/{owner}/{repo}/pulls?state=open&sort=created&direction=asc&per_page=50 +``` + +**R1 — PRs open > 7 days without review:** +Filter by `created_at` older than 7 days, then check review count (0 reviews). +- 🟡 Warning +- Fingerprint: `pr:{pr_number}:no-review` + +**R2 — PRs open > 14 days (any state of review):** +- 🟡 Warning (possibly abandoned) +- Fingerprint: `pr:{pr_number}:stale` + +**R3 — PRs with all checks failing:** +For each open PR, check its check runs. If all checks are failing: +- 🟡 Warning +- Fingerprint: `pr:{pr_number}:failing-checks` + +**R4 — Draft PRs with no activity > 7 days:** +Filter for `draft=true` and `updated_at` older than 7 days. +- 🔵 Info +- Fingerprint: `pr:{pr_number}:stale-draft` + +**R5 — PR merge velocity trend:** +``` +GET /repos/{owner}/{repo}/pulls?state=closed&sort=updated&direction=desc&per_page=50 +``` +Count merged PRs per day over the last 7 days. +- 🔵 Info (metric only — reported in trends table, not fingerprinted) + +### 1.5 Infrastructure Checks (I1–I6) + +**I1 — Missing CODEOWNERS:** +``` +GET /repos/{owner}/{repo}/contents/CODEOWNERS +``` +If 404, also check `.github/CODEOWNERS` and `docs/CODEOWNERS`. +- 🟡 Warning if none found +- Fingerprint: `infra:no-codeowners` + +**I2 — Missing Dependabot config:** +``` +GET /repos/{owner}/{repo}/contents/.github/dependabot.yml +``` +- 🟡 Warning if 404 +- Fingerprint: `infra:no-dependabot` + +**I3 — Relaxed skill validation:** +Check if `.github/workflows/validate-skills.yml` contains `fail-on-warning: false`. +- 🟡 Warning +- Fingerprint: `infra:relaxed-skill-validation` + +**I4 — Verdict-warn-only mode:** +Check if `.github/workflows/evaluation.yml` contains `--verdict-warn-only`. +- 🔵 Info +- Fingerprint: `infra:verdict-warn-only` + +**I5 — Dashboard deployment health:** +``` +GET /repos/{owner}/{repo}/pages +``` +Check last deployment status. +- 🔴 Critical if deployment failed +- Fingerprint: `infra:pages-deployment-failed` + +**I6 — Third-party action version drift:** +Scan workflow YAML files for non-`actions/*` references. Flag those pinned to tags instead of SHAs. +- 🔵 Info +- Fingerprint: `infra:unpinned-action:{action_name}` + +### 1.6 Resource Usage (U1–U3) + +**U1 — Daily compute hours:** +Sum all workflow run durations from the last 24h. +- 🔵 Info (metric only — for trends table) + +**U2 — Eval runs count:** +Count `evaluation` workflow runs in last 24h. +- 🔵 Info (metric only) + +**U3 — Cost trending up:** +Use `cache-memory` to compare this week's compute hours to last week. +- 🟡 Warning if >20% increase +- Fingerprint: `resource:cost-increase` + +--- + +## Step 2: Fingerprint & Diff + +After collecting all findings, perform the diff: + +1. **Load previous fingerprints** from `cache-memory` key `health-check-fingerprints`. If not available, treat as empty (first run). + +2. **Compute current fingerprints** for all findings collected in Step 1. + +3. **Classify each finding:** + - **🆕 NEW**: fingerprint is in current set but NOT in previous set + - **📌 EXISTING**: fingerprint is in both current and previous sets + - **✅ RESOLVED**: fingerprint is in previous set but NOT in current set + +4. **Track occurrences**: For EXISTING findings, increment the `occurrences` counter from the previous state. Record `first_seen` date from when the finding first appeared. + +5. **Save state** to `cache-memory`: + - `health-check-fingerprints`: current fingerprint set (with occurrence counts and first_seen dates) + - `health-check-history`: append today's summary `{ date, new_count, existing_count, resolved_count, by_severity: { critical, warning, info } }` + +6. **Sort findings** within each diff category: + - Primary sort: severity (🔴 → 🟡 → 🔵) + - Secondary sort: category (pipeline → quality → pr → infra → resource) + +--- + +## Step 3: Analysis + +Using the classified findings, generate: + +1. **Executive summary**: One sentence describing what changed (e.g., "2 new issues detected, 1 resolved — eval pipeline is now healthy but a skill quality regression appeared") + +2. **Correlation insights**: Identify connections between findings. For example: + - A pipeline failure AND stale benchmark data → pipeline likely blocking data publication + - Multiple quality regressions after the same date → look for a common commit + +3. **Recommendations**: Prioritized list of suggested actions. + +--- + +## Step 4: Output + +### 4.1 Find or Create the Pinned Issue + +Search for open issues with label `devops-health`: +- If exactly one exists → update it +- If none exist → create one with title `🏥 Repository Health Dashboard` and label `devops-health` +- If multiple exist → update the most recently created one, close the others + +Before creating/updating, ensure the `devops-health` label exists. If not, create it with color `#0E8A16` and description `Daily automated health check report`. + +### 4.2 Issue Body Format + +Replace the entire issue body with the following structure: + +```markdown +# 🏥 Daily Health Check — {date} + +**Status:** 🔴 {critical_count} critical · 🟡 {warning_count} warnings · 🔵 {info_count} info +**Since yesterday:** 🆕 {new_count} new · ✅ {resolved_count} resolved · 📌 {existing_count} unchanged + +--- + +## 🧩 Skill Inventory + +> Comprehensive health status of all skills derived from Q1–Q7 checks. + +| Status | Component | Skill | Skilled | Vanilla | Δ | Scenarios | Issues | +|--------|-----------|-------|--------:|--------:|--:|----------:|--------| +{For each skill, sorted by component then skill name:} +| {status_emoji} {status_label} | {component} | {skill_name} | {avg_skilled} | {avg_vanilla} | {delta} | {scenario_count} | {issue_summary} | + +**Legend:** 🟢 OK · 🟡 Warning / Low Value · 🔴 No Value / Critical · ⚪ Untested / No Data + +--- + +## 🆕 New Findings ({new_count}) + +> These appeared since the last health check ({previous_date}). + +{For each new finding, render a full section with title, details, link, and suggested action} +{Include investigation placeholder islands for findings that qualify for dispatch — see Step 5} + +--- + +## ✅ Resolved Since Yesterday ({resolved_count}) + +> These were in yesterday's report but are no longer detected. + +{For each resolved finding, render with strikethrough title and resolution info} + +--- + +## 📌 Existing Findings ({existing_count}) + +> These have been present since before today. Sorted by age. + +{Each existing finding in a collapsed
tag with first_seen and occurrence count} + +--- + +## 📊 Trends (7-day) + +| Metric | Today | 7d Avg | Δ | Trend | +|--------|-------|--------|---|-------| +| Eval duration (min) | {today} | {avg} | {delta} | {arrow} | +| Eval success rate | {today} | {avg} | {delta} | {arrow} | +| PRs merged/day | {today} | {avg} | {delta} | {arrow} | +| Open PRs | {today} | {avg} | {delta} | {arrow} | +| Compute hours/day | {today} | {avg} | {delta} | {arrow} | +| Active skills | {count} | {avg} | {delta} | {arrow} | +| Skills with issues | {count} | {avg} | {delta} | {arrow} | + +--- + +🤖 Generated by DevOps Health Check agentic workflow · [Run #{run_number}](link) · {timestamp} UTC +``` + +**Size guard:** If the issue body exceeds 60k characters: +- Show all 🆕 NEW findings in full (up to 10) +- Show all ✅ RESOLVED in full (up to 5) +- Limit 📌 EXISTING to top 20 by severity in collapsed `
` tags +- Append footer: `> … N additional existing findings omitted — see run artifacts for full report.` + +### 4.3 Daily Comment + +Append a short summary comment for the audit trail: + +```markdown +## 📋 Health Check — {date} + +🆕 {new_count} new · ✅ {resolved_count} resolved · 📌 {existing_count} unchanged + +**New:** +{bullet list of new findings with emojis and links} + +**Resolved:** +{bullet list of resolved findings with strikethrough} + +[Full report →]({issue_url}) +``` + +--- + +## Step 5: Triage Dispatch + +For each 🆕 NEW finding that qualifies for investigation, dispatch a worker: + +### 5.1 Dispatch Rules + +| Condition | Action | +|-----------|--------| +| 🆕 NEW + 🔴 Critical | **Always dispatch** | +| 🆕 NEW + 🟡 Warning + category `pipeline` or `quality` | **Dispatch** | +| 🆕 NEW + 🟡 Warning + category `pr` or `infra` | **Skip** (self-explanatory) | +| 🆕 NEW + 🔵 Info | **Never dispatch** | +| 📌 EXISTING (any) | **Never dispatch** | +| ✅ RESOLVED (any) | **Never dispatch** | + +**Budget:** Maximum 10 dispatches per run. If more than 10 qualify, prioritize by: +1. Severity descending (🔴 first) +2. Pipeline findings first +3. Quality findings second + +### 5.2 For Each Dispatched Finding + +1. **Insert a placeholder island** in the issue body (within the 🆕 section, right after the finding details): + +```markdown + +⏳ Investigation dispatched — results arriving shortly... + +``` + +2. **Dispatch the worker:** + +``` +dispatch-workflow: + workflow: devops-health-investigate + inputs: + finding_id: "{fingerprint}" + finding_type: "{category}" + finding_title: "{title}" + finding_severity: "{severity}" + resource_url: "{link}" + health_issue_number: "{issue_number}" + correlation_id: "hc-{date}-{sequence}" +``` + +3. **Wait 5 seconds** between dispatches (platform rate limit). + +--- + +## Guidelines + +- **Be data-driven**: Include specific numbers, durations, percentages, and links. +- **Be precise with fingerprints**: Use the exact fingerprint formulas from the knowledge file. Consistency is critical — the same finding MUST produce the same fingerprint across runs. +- **First run handling**: If `cache-memory` has no previous state, note: "⚠️ This is the first health check run. All findings appear as new. Diff will resume from next run." +- **Graceful degradation**: If an API call fails, skip that check category and note the skip in the output. Don't fail the entire workflow. +- **Noise awareness**: Demote known-noise findings (matching patterns in `cache-memory` `known-noise` list) to 🔵 Info severity, but still show them in the output for audit. +- **Issue body limit**: Keep under 60k characters. Truncate EXISTING section if needed. +- **Links everywhere**: Every finding should include at least one actionable link (to the run, PR, config file, etc.). diff --git a/.github/workflows/devops-health-investigate.lock.yml b/.github/workflows/devops-health-investigate.lock.yml new file mode 100644 index 00000000..0eccab38 --- /dev/null +++ b/.github/workflows/devops-health-investigate.lock.yml @@ -0,0 +1,1174 @@ +# +# ___ _ _ +# / _ \ | | (_) +# | |_| | __ _ ___ _ __ | |_ _ ___ +# | _ |/ _` |/ _ \ '_ \| __| |/ __| +# | | | | (_| | __/ | | | |_| | (__ +# \_| |_/\__, |\___|_| |_|\__|_|\___| +# __/ | +# _ _ |___/ +# | | | | / _| | +# | | | | ___ _ __ _ __| |_| | _____ ____ +# | |/\| |/ _ \ '__| |/ /| _| |/ _ \ \ /\ / / ___| +# \ /\ / (_) | | | | ( | | | | (_) \ V V /\__ \ +# \/ \/ \___/|_| |_|\_\|_| |_|\___/ \_/\_/ |___/ +# +# This file was automatically generated by gh-aw (v0.45.4). DO NOT EDIT. +# +# To update this file, edit the corresponding .md file and run: +# gh aw compile +# Not all edits will cause changes to this file. +# +# For more information: https://github.github.com/gh-aw/introduction/overview/ +# +# Worker agent that performs deep root-cause analysis on a single health check finding. Dispatched by the health check orchestrator. +# +# Resolved workflow manifest: +# Imports: +# - ../aw/shared/devops-investigate.lock.md +# +# frontmatter-hash: e9dba9ea1b00a1f0efc705cf43053f4f0b395b52bcd0477ee569985f5c486797 + +name: "DevOps Health — Deep Investigation" +"on": + workflow_dispatch: + inputs: + correlation_id: + description: Unique ID linking this investigation to the health check run + required: true + finding_id: + description: Fingerprint ID of the finding to investigate + required: true + finding_severity: + description: "Severity: critical | warning | info" + required: true + finding_title: + description: Human-readable title of the finding + required: true + finding_type: + description: "Category: pipeline | quality | pr | infra | resource" + required: true + health_issue_number: + description: Issue number of the pinned health dashboard + required: true + resource_url: + description: URL to the primary resource (run, PR, etc.) + required: true + +permissions: {} + +concurrency: + group: "gh-aw-${{ github.workflow }}" + +run-name: "DevOps Health — Deep Investigation" + +jobs: + activation: + runs-on: ubuntu-slim + permissions: + contents: read + outputs: + comment_id: "" + comment_repo: "" + steps: + - name: Setup Scripts + uses: github/gh-aw/actions/setup@ac090214a48a1938f7abafe132460b66752261af # v0.45.4 + with: + destination: /opt/gh-aw/actions + - name: Checkout .github and .agents folders + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + sparse-checkout: | + .github + .agents + fetch-depth: 1 + persist-credentials: false + - name: Check workflow file timestamps + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_WORKFLOW_FILE: "devops-health-investigate.lock.yml" + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/check_workflow_timestamp_api.cjs'); + await main(); + - name: Create prompt with built-in context + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }} + GH_AW_GITHUB_ACTOR: ${{ github.actor }} + GH_AW_GITHUB_EVENT_COMMENT_ID: ${{ github.event.comment.id }} + GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: ${{ github.event.discussion.number }} + GH_AW_GITHUB_EVENT_ISSUE_NUMBER: ${{ github.event.issue.number }} + GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER: ${{ github.event.pull_request.number }} + GH_AW_GITHUB_REPOSITORY: ${{ github.repository }} + GH_AW_GITHUB_RUN_ID: ${{ github.run_id }} + GH_AW_GITHUB_WORKSPACE: ${{ github.workspace }} + GH_AW_INPUTS_CORRELATION_ID: ${{ inputs.correlation_id }} + GH_AW_INPUTS_FINDING_ID: ${{ inputs.finding_id }} + GH_AW_INPUTS_FINDING_SEVERITY: ${{ inputs.finding_severity }} + GH_AW_INPUTS_FINDING_TITLE: ${{ inputs.finding_title }} + GH_AW_INPUTS_FINDING_TYPE: ${{ inputs.finding_type }} + GH_AW_INPUTS_HEALTH_ISSUE_NUMBER: ${{ inputs.health_issue_number }} + GH_AW_INPUTS_RESOURCE_URL: ${{ inputs.resource_url }} + run: | + bash /opt/gh-aw/actions/create_prompt_first.sh + cat << 'GH_AW_PROMPT_EOF' > "$GH_AW_PROMPT" + + GH_AW_PROMPT_EOF + cat "/opt/gh-aw/prompts/xpia.md" >> "$GH_AW_PROMPT" + cat "/opt/gh-aw/prompts/temp_folder_prompt.md" >> "$GH_AW_PROMPT" + cat "/opt/gh-aw/prompts/markdown.md" >> "$GH_AW_PROMPT" + cat << 'GH_AW_PROMPT_EOF' >> "$GH_AW_PROMPT" + + GitHub API Access Instructions + + The gh CLI is NOT authenticated. Do NOT use gh commands for GitHub operations. + + + To create or modify GitHub resources (issues, discussions, pull requests, etc.), you MUST call the appropriate safe output tool. Simply writing content will NOT work - the workflow requires actual tool calls. + + Temporary IDs: Some safe output tools support a temporary ID field (usually named temporary_id) so you can reference newly-created items elsewhere in the SAME agent output (for example, using #aw_abc1 in a later body). + + **IMPORTANT - temporary_id format rules:** + - If you DON'T need to reference the item later, OMIT the temporary_id field entirely (it will be auto-generated if needed) + - If you DO need cross-references/chaining, you MUST match this EXACT validation regex: /^aw_[A-Za-z0-9]{3,8}$/i + - Format: aw_ prefix followed by 3 to 8 alphanumeric characters (A-Z, a-z, 0-9, case-insensitive) + - Valid alphanumeric characters: ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789 + - INVALID examples: aw_ab (too short), aw_123456789 (too long), aw_test-id (contains hyphen), aw_id_123 (contains underscore) + - VALID examples: aw_abc, aw_abc1, aw_Test123, aw_A1B2C3D4, aw_12345678 + - To generate valid IDs: use 3-8 random alphanumeric characters or omit the field to let the system auto-generate + + Do NOT invent other aw_* formats — downstream steps will reject them with validation errors matching against /^aw_[A-Za-z0-9]{3,8}$/i. + + Discover available tools from the safeoutputs MCP server. + + **Critical**: Tool calls write structured data that downstream jobs process. Without tool calls, follow-up actions will be skipped. + + **Note**: If you made no other safe output tool calls during this workflow execution, call the "noop" tool to provide a status message indicating completion or that no actions were needed. + + + + The following GitHub context information is available for this workflow: + {{#if __GH_AW_GITHUB_ACTOR__ }} + - **actor**: __GH_AW_GITHUB_ACTOR__ + {{/if}} + {{#if __GH_AW_GITHUB_REPOSITORY__ }} + - **repository**: __GH_AW_GITHUB_REPOSITORY__ + {{/if}} + {{#if __GH_AW_GITHUB_WORKSPACE__ }} + - **workspace**: __GH_AW_GITHUB_WORKSPACE__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_ISSUE_NUMBER__ }} + - **issue-number**: #__GH_AW_GITHUB_EVENT_ISSUE_NUMBER__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER__ }} + - **discussion-number**: #__GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER__ }} + - **pull-request-number**: #__GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER__ + {{/if}} + {{#if __GH_AW_GITHUB_EVENT_COMMENT_ID__ }} + - **comment-id**: __GH_AW_GITHUB_EVENT_COMMENT_ID__ + {{/if}} + {{#if __GH_AW_GITHUB_RUN_ID__ }} + - **workflow-run-id**: __GH_AW_GITHUB_RUN_ID__ + {{/if}} + + + GH_AW_PROMPT_EOF + cat << 'GH_AW_PROMPT_EOF' >> "$GH_AW_PROMPT" + + GH_AW_PROMPT_EOF + cat << 'GH_AW_PROMPT_EOF' >> "$GH_AW_PROMPT" + {{#runtime-import ../aw/shared/devops-investigate.lock.md}} + GH_AW_PROMPT_EOF + cat << 'GH_AW_PROMPT_EOF' >> "$GH_AW_PROMPT" + {{#runtime-import .github/workflows/devops-health-investigate.md}} + GH_AW_PROMPT_EOF + - name: Interpolate variables and render templates + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_INPUTS_CORRELATION_ID: ${{ inputs.correlation_id }} + GH_AW_INPUTS_FINDING_ID: ${{ inputs.finding_id }} + GH_AW_INPUTS_FINDING_SEVERITY: ${{ inputs.finding_severity }} + GH_AW_INPUTS_FINDING_TITLE: ${{ inputs.finding_title }} + GH_AW_INPUTS_FINDING_TYPE: ${{ inputs.finding_type }} + GH_AW_INPUTS_HEALTH_ISSUE_NUMBER: ${{ inputs.health_issue_number }} + GH_AW_INPUTS_RESOURCE_URL: ${{ inputs.resource_url }} + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/interpolate_prompt.cjs'); + await main(); + - name: Substitute placeholders + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_GITHUB_ACTOR: ${{ github.actor }} + GH_AW_GITHUB_EVENT_COMMENT_ID: ${{ github.event.comment.id }} + GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: ${{ github.event.discussion.number }} + GH_AW_GITHUB_EVENT_ISSUE_NUMBER: ${{ github.event.issue.number }} + GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER: ${{ github.event.pull_request.number }} + GH_AW_GITHUB_REPOSITORY: ${{ github.repository }} + GH_AW_GITHUB_RUN_ID: ${{ github.run_id }} + GH_AW_GITHUB_WORKSPACE: ${{ github.workspace }} + GH_AW_INPUTS_CORRELATION_ID: ${{ inputs.correlation_id }} + GH_AW_INPUTS_FINDING_ID: ${{ inputs.finding_id }} + GH_AW_INPUTS_FINDING_SEVERITY: ${{ inputs.finding_severity }} + GH_AW_INPUTS_FINDING_TITLE: ${{ inputs.finding_title }} + GH_AW_INPUTS_FINDING_TYPE: ${{ inputs.finding_type }} + GH_AW_INPUTS_HEALTH_ISSUE_NUMBER: ${{ inputs.health_issue_number }} + GH_AW_INPUTS_RESOURCE_URL: ${{ inputs.resource_url }} + GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_ACTIVATED: ${{ needs.pre_activation.outputs.activated }} + GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_MATCHED_COMMAND: ${{ needs.pre_activation.outputs.matched_command }} + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + + const substitutePlaceholders = require('/opt/gh-aw/actions/substitute_placeholders.cjs'); + + // Call the substitution function + return await substitutePlaceholders({ + file: process.env.GH_AW_PROMPT, + substitutions: { + GH_AW_GITHUB_ACTOR: process.env.GH_AW_GITHUB_ACTOR, + GH_AW_GITHUB_EVENT_COMMENT_ID: process.env.GH_AW_GITHUB_EVENT_COMMENT_ID, + GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER: process.env.GH_AW_GITHUB_EVENT_DISCUSSION_NUMBER, + GH_AW_GITHUB_EVENT_ISSUE_NUMBER: process.env.GH_AW_GITHUB_EVENT_ISSUE_NUMBER, + GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER: process.env.GH_AW_GITHUB_EVENT_PULL_REQUEST_NUMBER, + GH_AW_GITHUB_REPOSITORY: process.env.GH_AW_GITHUB_REPOSITORY, + GH_AW_GITHUB_RUN_ID: process.env.GH_AW_GITHUB_RUN_ID, + GH_AW_GITHUB_WORKSPACE: process.env.GH_AW_GITHUB_WORKSPACE, + GH_AW_INPUTS_CORRELATION_ID: process.env.GH_AW_INPUTS_CORRELATION_ID, + GH_AW_INPUTS_FINDING_ID: process.env.GH_AW_INPUTS_FINDING_ID, + GH_AW_INPUTS_FINDING_SEVERITY: process.env.GH_AW_INPUTS_FINDING_SEVERITY, + GH_AW_INPUTS_FINDING_TITLE: process.env.GH_AW_INPUTS_FINDING_TITLE, + GH_AW_INPUTS_FINDING_TYPE: process.env.GH_AW_INPUTS_FINDING_TYPE, + GH_AW_INPUTS_HEALTH_ISSUE_NUMBER: process.env.GH_AW_INPUTS_HEALTH_ISSUE_NUMBER, + GH_AW_INPUTS_RESOURCE_URL: process.env.GH_AW_INPUTS_RESOURCE_URL, + GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_ACTIVATED: process.env.GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_ACTIVATED, + GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_MATCHED_COMMAND: process.env.GH_AW_NEEDS_PRE_ACTIVATION_OUTPUTS_MATCHED_COMMAND + } + }); + - name: Validate prompt placeholders + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + run: bash /opt/gh-aw/actions/validate_prompt_placeholders.sh + - name: Print prompt + env: + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + run: bash /opt/gh-aw/actions/print_prompt_summary.sh + - name: Upload prompt artifact + if: success() + uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6 + with: + name: prompt + path: /tmp/gh-aw/aw-prompts/prompt.txt + retention-days: 1 + + agent: + needs: activation + runs-on: ubuntu-latest + permissions: + actions: read + contents: read + issues: read + pull-requests: read + concurrency: + group: "gh-aw-copilot-${{ github.workflow }}" + env: + DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} + GH_AW_ASSETS_ALLOWED_EXTS: "" + GH_AW_ASSETS_BRANCH: "" + GH_AW_ASSETS_MAX_SIZE_KB: 0 + GH_AW_MCP_LOG_DIR: /tmp/gh-aw/mcp-logs/safeoutputs + GH_AW_SAFE_OUTPUTS: /opt/gh-aw/safeoutputs/outputs.jsonl + GH_AW_SAFE_OUTPUTS_CONFIG_PATH: /opt/gh-aw/safeoutputs/config.json + GH_AW_SAFE_OUTPUTS_TOOLS_PATH: /opt/gh-aw/safeoutputs/tools.json + GH_AW_WORKFLOW_ID_SANITIZED: devopshealthinvestigate + outputs: + checkout_pr_success: ${{ steps.checkout-pr.outputs.checkout_pr_success || 'true' }} + has_patch: ${{ steps.collect_output.outputs.has_patch }} + model: ${{ steps.generate_aw_info.outputs.model }} + output: ${{ steps.collect_output.outputs.output }} + output_types: ${{ steps.collect_output.outputs.output_types }} + secret_verification_result: ${{ steps.validate-secret.outputs.verification_result }} + steps: + - name: Setup Scripts + uses: github/gh-aw/actions/setup@ac090214a48a1938f7abafe132460b66752261af # v0.45.4 + with: + destination: /opt/gh-aw/actions + - name: Checkout repository + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + - name: Create gh-aw temp directory + run: bash /opt/gh-aw/actions/create_gh_aw_tmp_dir.sh + - name: Configure Git credentials + env: + REPO_NAME: ${{ github.repository }} + SERVER_URL: ${{ github.server_url }} + run: | + git config --global user.email "github-actions[bot]@users.noreply.github.com" + git config --global user.name "github-actions[bot]" + # Re-authenticate git with GitHub token + SERVER_URL_STRIPPED="${SERVER_URL#https://}" + git remote set-url origin "https://x-access-token:${{ github.token }}@${SERVER_URL_STRIPPED}/${REPO_NAME}.git" + echo "Git configured with standard GitHub Actions identity" + - name: Checkout PR branch + id: checkout-pr + if: | + github.event.pull_request + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + with: + github-token: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/checkout_pr_branch.cjs'); + await main(); + - name: Generate agentic run info + id: generate_aw_info + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const fs = require('fs'); + + const awInfo = { + engine_id: "copilot", + engine_name: "GitHub Copilot CLI", + model: process.env.GH_AW_MODEL_AGENT_COPILOT || "", + version: "", + agent_version: "0.0.410", + cli_version: "v0.45.4", + workflow_name: "DevOps Health — Deep Investigation", + experimental: false, + supports_tools_allowlist: true, + run_id: context.runId, + run_number: context.runNumber, + run_attempt: process.env.GITHUB_RUN_ATTEMPT, + repository: context.repo.owner + '/' + context.repo.repo, + ref: context.ref, + sha: context.sha, + actor: context.actor, + event_name: context.eventName, + staged: false, + allowed_domains: ["defaults"], + firewall_enabled: true, + awf_version: "v0.19.1", + awmg_version: "v0.1.4", + steps: { + firewall: "squid" + }, + created_at: new Date().toISOString() + }; + + // Write to /tmp/gh-aw directory to avoid inclusion in PR + const tmpPath = '/tmp/gh-aw/aw_info.json'; + fs.writeFileSync(tmpPath, JSON.stringify(awInfo, null, 2)); + console.log('Generated aw_info.json at:', tmpPath); + console.log(JSON.stringify(awInfo, null, 2)); + + // Set model as output for reuse in other steps/jobs + core.setOutput('model', awInfo.model); + - name: Validate COPILOT_GITHUB_TOKEN secret + id: validate-secret + run: /opt/gh-aw/actions/validate_multi_secret.sh COPILOT_GITHUB_TOKEN 'GitHub Copilot CLI' https://github.github.com/gh-aw/reference/engines/#github-copilot-default + env: + COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} + - name: Install GitHub Copilot CLI + run: /opt/gh-aw/actions/install_copilot_cli.sh 0.0.410 + - name: Install awf binary + run: bash /opt/gh-aw/actions/install_awf_binary.sh v0.19.1 + - name: Determine automatic lockdown mode for GitHub MCP Server + id: determine-automatic-lockdown + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_GITHUB_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN }} + GH_AW_GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }} + with: + script: | + const determineAutomaticLockdown = require('/opt/gh-aw/actions/determine_automatic_lockdown.cjs'); + await determineAutomaticLockdown(github, context, core); + - name: Download container images + run: bash /opt/gh-aw/actions/download_docker_images.sh ghcr.io/github/gh-aw-firewall/agent:0.19.1 ghcr.io/github/gh-aw-firewall/squid:0.19.1 ghcr.io/github/gh-aw-mcpg:v0.1.4 ghcr.io/github/github-mcp-server:v0.30.3 node:lts-alpine + - name: Write Safe Outputs Config + run: | + mkdir -p /opt/gh-aw/safeoutputs + mkdir -p /tmp/gh-aw/safeoutputs + mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs + cat > /opt/gh-aw/safeoutputs/config.json << 'GH_AW_SAFE_OUTPUTS_CONFIG_EOF' + {"add_comment":{"max":1},"missing_data":{},"missing_tool":{},"noop":{"max":1},"update_issue":{"max":1}} + GH_AW_SAFE_OUTPUTS_CONFIG_EOF + cat > /opt/gh-aw/safeoutputs/tools.json << 'GH_AW_SAFE_OUTPUTS_TOOLS_EOF' + [ + { + "description": "Add a comment to an existing GitHub issue, pull request, or discussion. Use this to provide feedback, answer questions, or add information to an existing conversation. For creating new items, use create_issue, create_discussion, or create_pull_request instead. IMPORTANT: Comments are subject to validation constraints enforced by the MCP server - maximum 65536 characters for the complete comment (including footer which is added automatically), 10 mentions (@username), and 50 links. Exceeding these limits will result in an immediate error with specific guidance. CONSTRAINTS: Maximum 1 comment(s) can be added.", + "inputSchema": { + "additionalProperties": false, + "properties": { + "body": { + "description": "The comment text in Markdown format. This is the 'body' field - do not use 'comment_body' or other variations. Provide helpful, relevant information that adds value to the conversation. CONSTRAINTS: The complete comment (your body text + automatically added footer) must not exceed 65536 characters total. Maximum 10 mentions (@username), maximum 50 links (http/https URLs). A footer (~200-500 characters) is automatically appended with workflow attribution, so leave adequate space. If these limits are exceeded, the tool call will fail with a detailed error message indicating which constraint was violated.", + "type": "string" + }, + "item_number": { + "description": "The issue, pull request, or discussion number to comment on. This is the numeric ID from the GitHub URL (e.g., 123 in github.com/owner/repo/issues/123). If omitted, the tool will attempt to resolve the target from the current workflow context (triggering issue, PR, or discussion).", + "type": "number" + } + }, + "required": [ + "body" + ], + "type": "object" + }, + "name": "add_comment" + }, + { + "description": "Update an existing GitHub issue's title, body, labels, assignees, or milestone WITHOUT closing it. This tool is primarily for editing issue metadata and content. While it supports changing status between 'open' and 'closed', use close_issue instead when you want to close an issue with a closing comment. Body updates support replacing, appending to, prepending content, or updating a per-run \"island\" section. CONSTRAINTS: Maximum 1 issue(s) can be updated.", + "inputSchema": { + "additionalProperties": false, + "properties": { + "assignees": { + "description": "Replace the issue assignees with this list of GitHub usernames (e.g., ['octocat', 'mona']).", + "items": { + "type": "string" + }, + "type": "array" + }, + "body": { + "description": "Issue body content in Markdown. For 'replace', this becomes the entire body. For 'append'/'prepend', this content is added with a separator and an attribution footer. For 'replace-island', only the run-specific section is updated.", + "type": "string" + }, + "issue_number": { + "description": "Issue number to update. This is the numeric ID from the GitHub URL (e.g., 789 in github.com/owner/repo/issues/789). Required when the workflow target is '*' (any issue).", + "type": [ + "number", + "string" + ] + }, + "labels": { + "description": "Replace the issue labels with this list (e.g., ['bug', 'tracking:foo']). Labels must exist in the repository.", + "items": { + "type": "string" + }, + "type": "array" + }, + "milestone": { + "description": "Milestone number to assign (e.g., 1). Use null to clear.", + "type": [ + "number", + "string" + ] + }, + "operation": { + "description": "How to update the issue body: 'append' (default - add to end with separator), 'prepend' (add to start with separator), 'replace' (overwrite entire body), or 'replace-island' (update a run-specific section).", + "enum": [ + "replace", + "append", + "prepend", + "replace-island" + ], + "type": "string" + }, + "status": { + "description": "New issue status: 'open' to reopen a closed issue, 'closed' to close an open issue.", + "enum": [ + "open", + "closed" + ], + "type": "string" + }, + "title": { + "description": "New issue title to replace the existing title.", + "type": "string" + } + }, + "type": "object" + }, + "name": "update_issue" + }, + { + "description": "Report that a tool or capability needed to complete the task is not available, or share any information you deem important about missing functionality or limitations. Use this when you cannot accomplish what was requested because the required functionality is missing or access is restricted.", + "inputSchema": { + "additionalProperties": false, + "properties": { + "alternatives": { + "description": "Any workarounds, manual steps, or alternative approaches the user could take (max 256 characters).", + "type": "string" + }, + "reason": { + "description": "Explanation of why this tool is needed or what information you want to share about the limitation (max 256 characters).", + "type": "string" + }, + "tool": { + "description": "Optional: Name or description of the missing tool or capability (max 128 characters). Be specific about what functionality is needed.", + "type": "string" + } + }, + "required": [ + "reason" + ], + "type": "object" + }, + "name": "missing_tool" + }, + { + "description": "Log a transparency message when no significant actions are needed. Use this to confirm workflow completion and provide visibility when analysis is complete but no changes or outputs are required (e.g., 'No issues found', 'All checks passed'). This ensures the workflow produces human-visible output even when no other actions are taken.", + "inputSchema": { + "additionalProperties": false, + "properties": { + "message": { + "description": "Status or completion message to log. Should explain what was analyzed and the outcome (e.g., 'Code review complete - no issues found', 'Analysis complete - all tests passing').", + "type": "string" + } + }, + "required": [ + "message" + ], + "type": "object" + }, + "name": "noop" + }, + { + "description": "Report that data or information needed to complete the task is not available. Use this when you cannot accomplish what was requested because required data, context, or information is missing.", + "inputSchema": { + "additionalProperties": false, + "properties": { + "alternatives": { + "description": "Any workarounds, manual steps, or alternative approaches the user could take (max 256 characters).", + "type": "string" + }, + "context": { + "description": "Additional context about the missing data or where it should come from (max 256 characters).", + "type": "string" + }, + "data_type": { + "description": "Type or description of the missing data or information (max 128 characters). Be specific about what data is needed.", + "type": "string" + }, + "reason": { + "description": "Explanation of why this data is needed to complete the task (max 256 characters).", + "type": "string" + } + }, + "required": [], + "type": "object" + }, + "name": "missing_data" + } + ] + GH_AW_SAFE_OUTPUTS_TOOLS_EOF + cat > /opt/gh-aw/safeoutputs/validation.json << 'GH_AW_SAFE_OUTPUTS_VALIDATION_EOF' + { + "add_comment": { + "defaultMax": 1, + "fields": { + "body": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "item_number": { + "issueOrPRNumber": true + } + } + }, + "missing_tool": { + "defaultMax": 20, + "fields": { + "alternatives": { + "type": "string", + "sanitize": true, + "maxLength": 512 + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 256 + }, + "tool": { + "type": "string", + "sanitize": true, + "maxLength": 128 + } + } + }, + "noop": { + "defaultMax": 1, + "fields": { + "message": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + } + } + }, + "update_issue": { + "defaultMax": 1, + "fields": { + "body": { + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "issue_number": { + "issueOrPRNumber": true + }, + "status": { + "type": "string", + "enum": [ + "open", + "closed" + ] + }, + "title": { + "type": "string", + "sanitize": true, + "maxLength": 128 + } + }, + "customValidation": "requiresOneOf:status,title,body" + } + } + GH_AW_SAFE_OUTPUTS_VALIDATION_EOF + - name: Generate Safe Outputs MCP Server Config + id: safe-outputs-config + run: | + # Generate a secure random API key (360 bits of entropy, 40+ chars) + # Mask immediately to prevent timing vulnerabilities + API_KEY=$(openssl rand -base64 45 | tr -d '/+=') + echo "::add-mask::${API_KEY}" + + PORT=3001 + + # Set outputs for next steps + { + echo "safe_outputs_api_key=${API_KEY}" + echo "safe_outputs_port=${PORT}" + } >> "$GITHUB_OUTPUT" + + echo "Safe Outputs MCP server will run on port ${PORT}" + + - name: Start Safe Outputs MCP HTTP Server + id: safe-outputs-start + env: + DEBUG: '*' + GH_AW_SAFE_OUTPUTS_PORT: ${{ steps.safe-outputs-config.outputs.safe_outputs_port }} + GH_AW_SAFE_OUTPUTS_API_KEY: ${{ steps.safe-outputs-config.outputs.safe_outputs_api_key }} + GH_AW_SAFE_OUTPUTS_TOOLS_PATH: /opt/gh-aw/safeoutputs/tools.json + GH_AW_SAFE_OUTPUTS_CONFIG_PATH: /opt/gh-aw/safeoutputs/config.json + GH_AW_MCP_LOG_DIR: /tmp/gh-aw/mcp-logs/safeoutputs + run: | + # Environment variables are set above to prevent template injection + export DEBUG + export GH_AW_SAFE_OUTPUTS_PORT + export GH_AW_SAFE_OUTPUTS_API_KEY + export GH_AW_SAFE_OUTPUTS_TOOLS_PATH + export GH_AW_SAFE_OUTPUTS_CONFIG_PATH + export GH_AW_MCP_LOG_DIR + + bash /opt/gh-aw/actions/start_safe_outputs_server.sh + + - name: Start MCP Gateway + id: start-mcp-gateway + env: + GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }} + GH_AW_SAFE_OUTPUTS_API_KEY: ${{ steps.safe-outputs-start.outputs.api_key }} + GH_AW_SAFE_OUTPUTS_PORT: ${{ steps.safe-outputs-start.outputs.port }} + GITHUB_MCP_LOCKDOWN: ${{ steps.determine-automatic-lockdown.outputs.lockdown == 'true' && '1' || '0' }} + GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + run: | + set -eo pipefail + mkdir -p /tmp/gh-aw/mcp-config + + # Export gateway environment variables for MCP config and gateway script + export MCP_GATEWAY_PORT="80" + export MCP_GATEWAY_DOMAIN="host.docker.internal" + MCP_GATEWAY_API_KEY=$(openssl rand -base64 45 | tr -d '/+=') + echo "::add-mask::${MCP_GATEWAY_API_KEY}" + export MCP_GATEWAY_API_KEY + export MCP_GATEWAY_PAYLOAD_DIR="/tmp/gh-aw/mcp-payloads" + mkdir -p "${MCP_GATEWAY_PAYLOAD_DIR}" + export DEBUG="*" + + export GH_AW_ENGINE="copilot" + export MCP_GATEWAY_DOCKER_COMMAND='docker run -i --rm --network host -v /var/run/docker.sock:/var/run/docker.sock -e MCP_GATEWAY_PORT -e MCP_GATEWAY_DOMAIN -e MCP_GATEWAY_API_KEY -e MCP_GATEWAY_PAYLOAD_DIR -e DEBUG -e MCP_GATEWAY_LOG_DIR -e GH_AW_MCP_LOG_DIR -e GH_AW_SAFE_OUTPUTS -e GH_AW_SAFE_OUTPUTS_CONFIG_PATH -e GH_AW_SAFE_OUTPUTS_TOOLS_PATH -e GH_AW_ASSETS_BRANCH -e GH_AW_ASSETS_MAX_SIZE_KB -e GH_AW_ASSETS_ALLOWED_EXTS -e DEFAULT_BRANCH -e GITHUB_MCP_SERVER_TOKEN -e GITHUB_MCP_LOCKDOWN -e GITHUB_REPOSITORY -e GITHUB_SERVER_URL -e GITHUB_SHA -e GITHUB_WORKSPACE -e GITHUB_TOKEN -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e GITHUB_JOB -e GITHUB_ACTION -e GITHUB_EVENT_NAME -e GITHUB_EVENT_PATH -e GITHUB_ACTOR -e GITHUB_ACTOR_ID -e GITHUB_TRIGGERING_ACTOR -e GITHUB_WORKFLOW -e GITHUB_WORKFLOW_REF -e GITHUB_WORKFLOW_SHA -e GITHUB_REF -e GITHUB_REF_NAME -e GITHUB_REF_TYPE -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GH_AW_SAFE_OUTPUTS_PORT -e GH_AW_SAFE_OUTPUTS_API_KEY -v /tmp/gh-aw/mcp-payloads:/tmp/gh-aw/mcp-payloads:rw -v /opt:/opt:ro -v /tmp:/tmp:rw -v '"${GITHUB_WORKSPACE}"':'"${GITHUB_WORKSPACE}"':rw ghcr.io/github/gh-aw-mcpg:v0.1.4' + + mkdir -p /home/runner/.copilot + cat << GH_AW_MCP_CONFIG_EOF | bash /opt/gh-aw/actions/start_mcp_gateway.sh + { + "mcpServers": { + "github": { + "type": "stdio", + "container": "ghcr.io/github/github-mcp-server:v0.30.3", + "env": { + "GITHUB_LOCKDOWN_MODE": "$GITHUB_MCP_LOCKDOWN", + "GITHUB_PERSONAL_ACCESS_TOKEN": "\${GITHUB_MCP_SERVER_TOKEN}", + "GITHUB_READ_ONLY": "1", + "GITHUB_TOOLSETS": "repos,issues,pull_requests,actions" + } + }, + "safeoutputs": { + "type": "http", + "url": "http://host.docker.internal:$GH_AW_SAFE_OUTPUTS_PORT", + "headers": { + "Authorization": "\${GH_AW_SAFE_OUTPUTS_API_KEY}" + } + } + }, + "gateway": { + "port": $MCP_GATEWAY_PORT, + "domain": "${MCP_GATEWAY_DOMAIN}", + "apiKey": "${MCP_GATEWAY_API_KEY}", + "payloadDir": "${MCP_GATEWAY_PAYLOAD_DIR}" + } + } + GH_AW_MCP_CONFIG_EOF + - name: Generate workflow overview + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { generateWorkflowOverview } = require('/opt/gh-aw/actions/generate_workflow_overview.cjs'); + await generateWorkflowOverview(core); + - name: Download prompt artifact + uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6 + with: + name: prompt + path: /tmp/gh-aw/aw-prompts + - name: Clean git credentials + run: bash /opt/gh-aw/actions/clean_git_credentials.sh + - name: Execute GitHub Copilot CLI + id: agentic_execution + # Copilot CLI tool arguments (sorted): + # --allow-tool github + # --allow-tool safeoutputs + # --allow-tool shell(cat) + # --allow-tool shell(date) + # --allow-tool shell(diff) + # --allow-tool shell(echo) + # --allow-tool shell(find) + # --allow-tool shell(grep) + # --allow-tool shell(head) + # --allow-tool shell(jq) + # --allow-tool shell(ls) + # --allow-tool shell(pwd) + # --allow-tool shell(sort) + # --allow-tool shell(tail) + # --allow-tool shell(uniq) + # --allow-tool shell(wc) + # --allow-tool shell(yq) + # --allow-tool write + timeout-minutes: 20 + run: | + set -o pipefail + sudo -E awf --env-all --container-workdir "${GITHUB_WORKSPACE}" --allow-domains api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --enable-host-access --image-tag 0.19.1 --skip-pull \ + -- /bin/bash -c '/usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --add-dir "${GITHUB_WORKSPACE}" --disable-builtin-mcps --allow-tool github --allow-tool safeoutputs --allow-tool '\''shell(cat)'\'' --allow-tool '\''shell(date)'\'' --allow-tool '\''shell(diff)'\'' --allow-tool '\''shell(echo)'\'' --allow-tool '\''shell(find)'\'' --allow-tool '\''shell(grep)'\'' --allow-tool '\''shell(head)'\'' --allow-tool '\''shell(jq)'\'' --allow-tool '\''shell(ls)'\'' --allow-tool '\''shell(pwd)'\'' --allow-tool '\''shell(sort)'\'' --allow-tool '\''shell(tail)'\'' --allow-tool '\''shell(uniq)'\'' --allow-tool '\''shell(wc)'\'' --allow-tool '\''shell(yq)'\'' --allow-tool write --allow-all-paths --share /tmp/gh-aw/sandbox/agent/logs/conversation.md --prompt "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"${GH_AW_MODEL_AGENT_COPILOT:+ --model "$GH_AW_MODEL_AGENT_COPILOT"}' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log + env: + COPILOT_AGENT_RUNNER_TYPE: STANDALONE + COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} + GH_AW_MCP_CONFIG: /home/runner/.copilot/mcp-config.json + GH_AW_MODEL_AGENT_COPILOT: ${{ vars.GH_AW_MODEL_AGENT_COPILOT || '' }} + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }} + GITHUB_HEAD_REF: ${{ github.head_ref }} + GITHUB_REF_NAME: ${{ github.ref_name }} + GITHUB_STEP_SUMMARY: ${{ env.GITHUB_STEP_SUMMARY }} + GITHUB_WORKSPACE: ${{ github.workspace }} + XDG_CONFIG_HOME: /home/runner + - name: Configure Git credentials + env: + REPO_NAME: ${{ github.repository }} + SERVER_URL: ${{ github.server_url }} + run: | + git config --global user.email "github-actions[bot]@users.noreply.github.com" + git config --global user.name "github-actions[bot]" + # Re-authenticate git with GitHub token + SERVER_URL_STRIPPED="${SERVER_URL#https://}" + git remote set-url origin "https://x-access-token:${{ github.token }}@${SERVER_URL_STRIPPED}/${REPO_NAME}.git" + echo "Git configured with standard GitHub Actions identity" + - name: Copy Copilot session state files to logs + if: always() + continue-on-error: true + run: | + # Copy Copilot session state files to logs folder for artifact collection + # This ensures they are in /tmp/gh-aw/ where secret redaction can scan them + SESSION_STATE_DIR="$HOME/.copilot/session-state" + LOGS_DIR="/tmp/gh-aw/sandbox/agent/logs" + + if [ -d "$SESSION_STATE_DIR" ]; then + echo "Copying Copilot session state files from $SESSION_STATE_DIR to $LOGS_DIR" + mkdir -p "$LOGS_DIR" + cp -v "$SESSION_STATE_DIR"/*.jsonl "$LOGS_DIR/" 2>/dev/null || true + echo "Session state files copied successfully" + else + echo "No session-state directory found at $SESSION_STATE_DIR" + fi + - name: Stop MCP Gateway + if: always() + continue-on-error: true + env: + MCP_GATEWAY_PORT: ${{ steps.start-mcp-gateway.outputs.gateway-port }} + MCP_GATEWAY_API_KEY: ${{ steps.start-mcp-gateway.outputs.gateway-api-key }} + GATEWAY_PID: ${{ steps.start-mcp-gateway.outputs.gateway-pid }} + run: | + bash /opt/gh-aw/actions/stop_mcp_gateway.sh "$GATEWAY_PID" + - name: Redact secrets in logs + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/redact_secrets.cjs'); + await main(); + env: + GH_AW_SECRET_NAMES: 'COPILOT_GITHUB_TOKEN,GH_AW_GITHUB_MCP_SERVER_TOKEN,GH_AW_GITHUB_TOKEN,GITHUB_TOKEN' + SECRET_COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} + SECRET_GH_AW_GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }} + SECRET_GH_AW_GITHUB_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN }} + SECRET_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + - name: Upload Safe Outputs + if: always() + uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6 + with: + name: safe-output + path: ${{ env.GH_AW_SAFE_OUTPUTS }} + if-no-files-found: warn + - name: Ingest agent output + id: collect_output + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }} + GH_AW_ALLOWED_DOMAINS: "api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com" + GITHUB_SERVER_URL: ${{ github.server_url }} + GITHUB_API_URL: ${{ github.api_url }} + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/collect_ndjson_output.cjs'); + await main(); + - name: Upload sanitized agent output + if: always() && env.GH_AW_AGENT_OUTPUT + uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6 + with: + name: agent-output + path: ${{ env.GH_AW_AGENT_OUTPUT }} + if-no-files-found: warn + - name: Upload engine output files + uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6 + with: + name: agent_outputs + path: | + /tmp/gh-aw/sandbox/agent/logs/ + /tmp/gh-aw/redacted-urls.log + if-no-files-found: ignore + - name: Parse agent logs for step summary + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: /tmp/gh-aw/sandbox/agent/logs/ + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/parse_copilot_log.cjs'); + await main(); + - name: Parse MCP Gateway logs for step summary + if: always() + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/parse_mcp_gateway_log.cjs'); + await main(); + - name: Print firewall logs + if: always() + continue-on-error: true + env: + AWF_LOGS_DIR: /tmp/gh-aw/sandbox/firewall/logs + run: | + # Fix permissions on firewall logs so they can be uploaded as artifacts + # AWF runs with sudo, creating files owned by root + sudo chmod -R a+r /tmp/gh-aw/sandbox/firewall/logs 2>/dev/null || true + # Only run awf logs summary if awf command exists (it may not be installed if workflow failed before install step) + if command -v awf &> /dev/null; then + awf logs summary | tee -a "$GITHUB_STEP_SUMMARY" + else + echo 'AWF binary not installed, skipping firewall log summary' + fi + - name: Upload agent artifacts + if: always() + continue-on-error: true + uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6 + with: + name: agent-artifacts + path: | + /tmp/gh-aw/aw-prompts/prompt.txt + /tmp/gh-aw/aw_info.json + /tmp/gh-aw/mcp-logs/ + /tmp/gh-aw/sandbox/firewall/logs/ + /tmp/gh-aw/agent-stdio.log + /tmp/gh-aw/agent/ + if-no-files-found: ignore + + conclusion: + needs: + - activation + - agent + - detection + - safe_outputs + if: (always()) && (needs.agent.result != 'skipped') + runs-on: ubuntu-slim + permissions: + contents: read + discussions: write + issues: write + pull-requests: write + outputs: + noop_message: ${{ steps.noop.outputs.noop_message }} + tools_reported: ${{ steps.missing_tool.outputs.tools_reported }} + total_count: ${{ steps.missing_tool.outputs.total_count }} + steps: + - name: Setup Scripts + uses: github/gh-aw/actions/setup@ac090214a48a1938f7abafe132460b66752261af # v0.45.4 + with: + destination: /opt/gh-aw/actions + - name: Download agent output artifact + continue-on-error: true + uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6 + with: + name: agent-output + path: /tmp/gh-aw/safeoutputs/ + - name: Setup agent output environment variable + run: | + mkdir -p /tmp/gh-aw/safeoutputs/ + find "/tmp/gh-aw/safeoutputs/" -type f -print + echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/safeoutputs/agent_output.json" >> "$GITHUB_ENV" + - name: Process No-Op Messages + id: noop + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_NOOP_MAX: 1 + GH_AW_WORKFLOW_NAME: "DevOps Health — Deep Investigation" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/noop.cjs'); + await main(); + - name: Record Missing Tool + id: missing_tool + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_WORKFLOW_NAME: "DevOps Health — Deep Investigation" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/missing_tool.cjs'); + await main(); + - name: Handle Agent Failure + id: handle_agent_failure + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_WORKFLOW_NAME: "DevOps Health — Deep Investigation" + GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }} + GH_AW_WORKFLOW_ID: "devops-health-investigate" + GH_AW_SECRET_VERIFICATION_RESULT: ${{ needs.agent.outputs.secret_verification_result }} + GH_AW_CHECKOUT_PR_SUCCESS: ${{ needs.agent.outputs.checkout_pr_success }} + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/handle_agent_failure.cjs'); + await main(); + - name: Handle No-Op Message + id: handle_noop_message + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_WORKFLOW_NAME: "DevOps Health — Deep Investigation" + GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }} + GH_AW_NOOP_MESSAGE: ${{ steps.noop.outputs.noop_message }} + GH_AW_NOOP_REPORT_AS_ISSUE: "true" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/handle_noop_message.cjs'); + await main(); + + detection: + needs: agent + if: needs.agent.outputs.output_types != '' || needs.agent.outputs.has_patch == 'true' + runs-on: ubuntu-latest + permissions: {} + concurrency: + group: "gh-aw-copilot-${{ github.workflow }}" + timeout-minutes: 10 + outputs: + success: ${{ steps.parse_results.outputs.success }} + steps: + - name: Setup Scripts + uses: github/gh-aw/actions/setup@ac090214a48a1938f7abafe132460b66752261af # v0.45.4 + with: + destination: /opt/gh-aw/actions + - name: Download agent artifacts + continue-on-error: true + uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6 + with: + name: agent-artifacts + path: /tmp/gh-aw/threat-detection/ + - name: Download agent output artifact + continue-on-error: true + uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6 + with: + name: agent-output + path: /tmp/gh-aw/threat-detection/ + - name: Echo agent output types + env: + AGENT_OUTPUT_TYPES: ${{ needs.agent.outputs.output_types }} + run: | + echo "Agent output-types: $AGENT_OUTPUT_TYPES" + - name: Setup threat detection + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + WORKFLOW_NAME: "DevOps Health — Deep Investigation" + WORKFLOW_DESCRIPTION: "Worker agent that performs deep root-cause analysis on a single health check finding. Dispatched by the health check orchestrator." + HAS_PATCH: ${{ needs.agent.outputs.has_patch }} + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/setup_threat_detection.cjs'); + await main(); + - name: Ensure threat-detection directory and log + run: | + mkdir -p /tmp/gh-aw/threat-detection + touch /tmp/gh-aw/threat-detection/detection.log + - name: Validate COPILOT_GITHUB_TOKEN secret + id: validate-secret + run: /opt/gh-aw/actions/validate_multi_secret.sh COPILOT_GITHUB_TOKEN 'GitHub Copilot CLI' https://github.github.com/gh-aw/reference/engines/#github-copilot-default + env: + COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} + - name: Install GitHub Copilot CLI + run: /opt/gh-aw/actions/install_copilot_cli.sh 0.0.410 + - name: Execute GitHub Copilot CLI + id: agentic_execution + # Copilot CLI tool arguments (sorted): + # --allow-tool shell(cat) + # --allow-tool shell(grep) + # --allow-tool shell(head) + # --allow-tool shell(jq) + # --allow-tool shell(ls) + # --allow-tool shell(tail) + # --allow-tool shell(wc) + timeout-minutes: 20 + run: | + set -o pipefail + COPILOT_CLI_INSTRUCTION="$(cat /tmp/gh-aw/aw-prompts/prompt.txt)" + mkdir -p /tmp/ + mkdir -p /tmp/gh-aw/ + mkdir -p /tmp/gh-aw/agent/ + mkdir -p /tmp/gh-aw/sandbox/agent/logs/ + copilot --add-dir /tmp/ --add-dir /tmp/gh-aw/ --add-dir /tmp/gh-aw/agent/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --allow-tool 'shell(cat)' --allow-tool 'shell(grep)' --allow-tool 'shell(head)' --allow-tool 'shell(jq)' --allow-tool 'shell(ls)' --allow-tool 'shell(tail)' --allow-tool 'shell(wc)' --share /tmp/gh-aw/sandbox/agent/logs/conversation.md --prompt "$COPILOT_CLI_INSTRUCTION"${GH_AW_MODEL_DETECTION_COPILOT:+ --model "$GH_AW_MODEL_DETECTION_COPILOT"} 2>&1 | tee /tmp/gh-aw/threat-detection/detection.log + env: + COPILOT_AGENT_RUNNER_TYPE: STANDALONE + COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} + GH_AW_MODEL_DETECTION_COPILOT: ${{ vars.GH_AW_MODEL_DETECTION_COPILOT || '' }} + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GITHUB_HEAD_REF: ${{ github.head_ref }} + GITHUB_REF_NAME: ${{ github.ref_name }} + GITHUB_STEP_SUMMARY: ${{ env.GITHUB_STEP_SUMMARY }} + GITHUB_WORKSPACE: ${{ github.workspace }} + XDG_CONFIG_HOME: /home/runner + - name: Parse threat detection results + id: parse_results + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/parse_threat_detection_results.cjs'); + await main(); + - name: Upload threat detection log + if: always() + uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6 + with: + name: threat-detection.log + path: /tmp/gh-aw/threat-detection/detection.log + if-no-files-found: ignore + + safe_outputs: + needs: + - agent + - detection + if: ((!cancelled()) && (needs.agent.result != 'skipped')) && (needs.detection.outputs.success == 'true') + runs-on: ubuntu-slim + permissions: + contents: read + discussions: write + issues: write + pull-requests: write + timeout-minutes: 15 + env: + GH_AW_ENGINE_ID: "copilot" + GH_AW_WORKFLOW_ID: "devops-health-investigate" + GH_AW_WORKFLOW_NAME: "DevOps Health — Deep Investigation" + outputs: + create_discussion_error_count: ${{ steps.process_safe_outputs.outputs.create_discussion_error_count }} + create_discussion_errors: ${{ steps.process_safe_outputs.outputs.create_discussion_errors }} + process_safe_outputs_processed_count: ${{ steps.process_safe_outputs.outputs.processed_count }} + process_safe_outputs_temporary_id_map: ${{ steps.process_safe_outputs.outputs.temporary_id_map }} + steps: + - name: Setup Scripts + uses: github/gh-aw/actions/setup@ac090214a48a1938f7abafe132460b66752261af # v0.45.4 + with: + destination: /opt/gh-aw/actions + - name: Download agent output artifact + continue-on-error: true + uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6 + with: + name: agent-output + path: /tmp/gh-aw/safeoutputs/ + - name: Setup agent output environment variable + run: | + mkdir -p /tmp/gh-aw/safeoutputs/ + find "/tmp/gh-aw/safeoutputs/" -type f -print + echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/safeoutputs/agent_output.json" >> "$GITHUB_ENV" + - name: Process Safe Outputs + id: process_safe_outputs + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + env: + GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"max\":1},\"missing_data\":{},\"missing_tool\":{},\"update_issue\":{\"allow_body\":true,\"max\":1}}" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io); + const { main } = require('/opt/gh-aw/actions/safe_output_handler_manager.cjs'); + await main(); + diff --git a/.github/workflows/devops-health-investigate.md b/.github/workflows/devops-health-investigate.md new file mode 100644 index 00000000..8471cf7c --- /dev/null +++ b/.github/workflows/devops-health-investigate.md @@ -0,0 +1,150 @@ +--- +name: "DevOps Health — Deep Investigation" +description: > + Worker agent that performs deep root-cause analysis on a single + health check finding. Dispatched by the health check orchestrator. + +on: + workflow_dispatch: + inputs: + finding_id: + description: "Fingerprint ID of the finding to investigate" + required: true + finding_type: + description: "Category: pipeline | quality | pr | infra | resource" + required: true + finding_title: + description: "Human-readable title of the finding" + required: true + finding_severity: + description: "Severity: critical | warning | info" + required: true + resource_url: + description: "URL to the primary resource (run, PR, etc.)" + required: true + health_issue_number: + description: "Issue number of the pinned health dashboard" + required: true + correlation_id: + description: "Unique ID linking this investigation to the health check run" + required: true + +permissions: + contents: read + actions: read + issues: read + pull-requests: read + +imports: + - ../aw/shared/devops-investigate.lock.md + +tools: + github: + toolsets: [repos, issues, pull_requests, actions] + bash: ["cat", "grep", "head", "tail", "find", "ls", "wc", "jq", "date", "sort", "diff"] + +safe-outputs: + update-issue: + max: 1 + add-comment: + max: 1 + +network: + allowed: + - defaults +--- + +# DevOps Health — Deep Investigation Worker + +You are a specialized investigation agent. You have been dispatched by the DevOps Health Check orchestrator to perform a deep root-cause analysis on **one specific finding**. + +## Your Mission + +Investigate the finding identified by the inputs provided to this workflow run. Determine the root cause, assess the blast radius, and generate actionable remediation steps. Report your findings back to the pinned health issue. + +## Inputs Available + +- `finding_id`: `${{ inputs.finding_id }}` — The fingerprint ID of the finding +- `finding_type`: `${{ inputs.finding_type }}` — Category (pipeline, quality, pr, infra, resource) +- `finding_title`: `${{ inputs.finding_title }}` — Human-readable title +- `finding_severity`: `${{ inputs.finding_severity }}` — Severity level +- `resource_url`: `${{ inputs.resource_url }}` — URL to the primary resource +- `health_issue_number`: `${{ inputs.health_issue_number }}` — Issue to update +- `correlation_id`: `${{ inputs.correlation_id }}` — Links this investigation to the health check run + +--- + +## Investigation Protocol + +### Step 1: Route to Category-Specific Playbook + +Based on `finding_type`, follow the appropriate investigation playbook from the compiled knowledge file: + +- **pipeline** → Pipeline Investigation Playbook +- **quality** → Quality Investigation Playbook +- **pr** → PR Investigation Playbook +- **infra** → Infrastructure Investigation Playbook +- **resource** → Resource Investigation Playbook + +### Step 2: Gather Evidence + +Follow the playbook steps meticulously. For each piece of evidence: +- Record the **source** (API endpoint, file path, log excerpt) +- Note the **timestamp** of the evidence +- Assess **relevance** to the finding + +### Step 3: Determine Root Cause + +Based on the gathered evidence: +1. Identify the **most likely root cause** +2. Assign a **confidence level**: High / Medium / Low + - **High**: Direct evidence (error message explicitly states the cause, code change directly correlates) + - **Medium**: Strong circumstantial evidence (timing correlates, pattern matches known issues) + - **Low**: Inferential (possible but no direct evidence found) +3. Identify the **blast radius** — what else is affected? +4. Check for **related issues** — is this already tracked? + +### Step 4: Generate Remediation Steps + +Provide 1–3 specific, actionable remediation steps. Each step should: +- Be concrete (include file paths, commands, or config changes) +- Be ordered by recommended priority +- Include any caveats or risks + +### Step 5: Report Back + +Update the pinned health issue by replacing the investigation island: + +``` +update-issue: + issue: {health_issue_number} + operation: replace-island + island: "investigation:{finding_id}" + body: | + 🔍 **Investigation Complete** — [Worker Run #{this_run_number}]({this_run_url}) + + **Root cause:** {one-paragraph description with evidence} + + **Confidence:** {High|Medium|Low} — {justification} + + **Blast radius:** {what else is affected} + + **Suggested fix:** + 1. {step 1} + 2. {step 2} + 3. {step 3} (if applicable) + + **Related:** {commits, PRs, issues, or "None found"} +``` + +--- + +## Guidelines + +- **Be factual**: Every claim must be backed by evidence from API responses, logs, or code. +- **Don't hallucinate**: If you cannot determine the root cause, say so honestly. A "Low confidence" finding with honest uncertainty is better than a fabricated "High confidence" answer. +- **Be concise**: The investigation report appears inline in the health dashboard. Keep it focused — 1-2 paragraphs for root cause, 1 paragraph for blast radius, numbered list for fixes. +- **Include source evidence**: Quote specific error messages, log lines, or commit SHAs. Use code blocks for log excerpts. +- **Check recent commits**: For pipeline and quality findings, always check commits between the last successful state and the current failure. +- **Cross-reference**: Look for related open issues or PRs that might already be tracking this problem. +- **Time-box yourself**: If evidence is insufficient after reasonable investigation, report what you found with appropriate confidence level rather than spiraling. diff --git a/eng/agentic-workflows/README.md b/eng/agentic-workflows/README.md new file mode 100644 index 00000000..a2ea815b --- /dev/null +++ b/eng/agentic-workflows/README.md @@ -0,0 +1,78 @@ +# DevOps Agentic Workflows + +Cross-cutting [GitHub Agentic Workflow](https://github.com/github/gh-aw) workflows for repository-wide DevOps automation. + +The workflow source files live in `.github/workflows/` and are compiled with `gh aw compile` to generate `.lock.yml` files (standard GitHub Actions YAML with security hardening). These workflows monitor the _entire repo_ (all components, all pipelines, all PRs), unlike the component-specific workflows in `src/dotnet-msbuild/agentic-workflows/`. + +## Available Workflows + +| Workflow | Description | Trigger | +|----------|-------------|---------| +| [devops-health-check](../../.github/workflows/devops-health-check.md) | Daily orchestrator that collects repo health signals (pipelines, skill quality, PRs, infrastructure), computes a fingerprint-based diff against the previous run, and updates a pinned health dashboard issue | `schedule: daily` (fuzzy daily), `/health-check` slash command, `workflow_dispatch` | +| [devops-health-investigate](../../.github/workflows/devops-health-investigate.md) | Worker agent dispatched by the health check orchestrator to perform deep root-cause analysis on individual findings | `workflow_dispatch` (dispatched by orchestrator via `dispatch-workflow`) | + +## Architecture + +``` +devops-health-check (Orchestrator) + ├─ Collects health signals from 5 categories: + │ Pipeline · Quality · PRs · Infrastructure · Resources + ├─ Fingerprints each finding for stable diff tracking + ├─ Classifies: 🆕 NEW · 📌 EXISTING · ✅ RESOLVED + ├─ Updates pinned health dashboard issue + └─ Dispatches investigation workers (up to 10) + │ + ▼ +devops-health-investigate (Worker × N) + ├─ Investigates ONE finding with fresh context + ├─ Follows category-specific playbook + ├─ Determines root cause + remediation + └─ Updates health issue via replace-island +``` + +## Setup + +1. Install the `gh aw` CLI extension: `gh extension install github/gh-aw` +2. Compile: `gh aw compile` (from the repo root — this compiles all `.md` files in `.github/workflows/`) +3. Commit both the `.md` and generated `.lock.yml` files +4. The health check runs daily, or on-demand via `/health-check` + +## Local Development + +```powershell +# Compile workflows (generates .lock.yml from .md frontmatter) +gh aw compile + +# Compile with validation +gh aw compile --strict + +# Dry-run (validates without triggering on GitHub Actions) +gh aw run devops-health-check --dry-run + +# Run on GitHub Actions (from a pushed branch) +gh aw run devops-health-check --push --ref +``` + +## File Structure + +``` +.github/ +├── workflows/ +│ ├── devops-health-check.md # Orchestrator workflow +│ ├── devops-health-check.lock.yml # Compiled workflow (generated by gh aw compile) +│ ├── devops-health-investigate.md # Worker workflow +│ └── devops-health-investigate.lock.yml # Compiled workflow (generated by gh aw compile) +└── aw/ + └── shared/ + ├── devops-health.lock.md # Health check catalog & fingerprinting rules + └── devops-investigate.lock.md # Investigation playbooks & remediation templates + +eng/agentic-workflows/ +├── README.md # This file (documentation) +└── pr-141-review-fixes.md # PR review analysis +``` + +## Related + +- [src/dotnet-msbuild/agentic-workflows/](../../src/dotnet-msbuild/agentic-workflows/) — MSBuild-specific agentic workflows +- [eng/dashboard/](../dashboard/) — Benchmark dashboard (data source for quality checks)