Parse '/evaluate <sha>' from issue comments with a bash regex instead of a
sed BRE, avoiding word-boundary escape ambiguity across sed implementations
and matching the validation used elsewhere in the gate. Reject over-length
hex strings to the guidance path (fail-safe).
Clarify the workflow_dispatch docs: the PR head travels in the head_sha
input, while a dispatched run's github.sha is the default-branch tip.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 29ef3316-14f6-4dc1-a838-40f8be09f915
Harden the /evaluate pipeline in evaluation.yml so a run is always tied to
one specific reviewed commit captured when the trigger fires, instead of
resolving the live branch head downstream. Every job (gate, discover,
evaluate, report-status) now consumes the single gate-bound SHA.
- Add a pull_request_review trigger (recommended path): the run binds to
review.commit_id, so no SHA needs to be typed. Files changed -> Review
changes -> Submit review with /evaluate in the body.
- Comment path (/evaluate <sha>) now requires an explicit SHA that must
belong to the PR; a bare /evaluate posts copy-paste-ready guidance and
points to the review flow.
- Label and workflow_dispatch paths bind to the head carried in the event
payload / passed by the triage worker.
- Warn (log + PR comment) when the bound commit is not the current head.
- Only write-access triggers (label, dispatch) may cancel an in-progress
run; comment/review triggers queue so an authorized run finishes.
- Reachability check tolerates PRs >250 commits via a compare-ancestry
fallback, staying fail-closed.
- discover fails closed if the bound commit can't be checked out.
- Pass inputs.plugin and actor logins via env instead of inlining.
- Update docs/design/pr-triage-workflows.md (4 entry points, commit binding).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 29ef3316-14f6-4dc1-a838-40f8be09f915
Replace the agentic close-stale-prs.agent.md with a deterministic
stale-PR sweep hosted in pr-triage-batch.yml. The new stale-sweep job
runs .github/scripts/pr-stale-sweep.sh weekly (cron 17 4 * * 1) and on
manual dispatch (stale_sweep=true), warning about and closing PRs with no
non-bot activity for 30/37 days. Preserves the original policy (no-stale
and maestro exemptions, drafts included, non-bot activity timer) without
model calls or token cost.
Deletes close-stale-prs.agent.md and its compiled lock file, and updates
docs/design/pr-triage-workflows.md.
Fixes#915
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The triage worker added the 'evaluate-now' label via GITHUB_TOKEN, but label events emitted by GITHUB_TOKEN do not start workflows (GitHub's recursion guard), so evaluation.yml's pull_request_target:[labeled] entry point never fired for the bot (repro: PR #745). workflow_dispatch and repository_dispatch are the only token-initiated events exempt from that guard.
The worker now dispatches evaluation.yml directly via 'gh workflow run' with a pr_number input, routed through the existing gate job so the path is identical to /evaluate. A dispatched run's head_sha is the default branch (not the PR head), so idempotency now matches the deterministic run name 'Evaluate PR #<n> @ <sha7>'. The 'evaluate-now' label remains a valid human entry point. Worker granted actions:write for 'gh workflow run'.
* PR triage workflows: orchestrator, worker, and evaluate-now label
Implements docs/design/pr-triage-workflows.md:
- pr-triage-batch.yml: hourly orchestrator that classifies open PRs
- pr-triage.yml + pr-triage-act.sh: per-PR worker (state recompute,
label reconciliation, eval-trigger, ping comments with cool-down)
- evaluation.yml: gate job now also handles pull_request_target [labeled]
with the evaluate-now label as a second entry point alongside /evaluate
* Add temporary push triggers for testing pr-triage workflows
* test: live-run pr-triage worker once
* test: re-run worker for cool-down check
* fix: age gate uses created_at and applies only before first ping
* Remove temporary test triggers and inline test marker
* Add pr-malicious-scan agent workflow; replace design doc with brief overview
- New: .github/workflows/pr-malicious-scan.agent.md + compiled .lock.yml.
Static diff scanner for external (non-trusted) PR contributors. Triggers
on pull_request_target [opened/synchronize/reopened] and workflow_dispatch.
Surfaces findings as code-scanning alerts plus a single maintainer-ping
comment per head SHA when high-severity / workflow-tamper / supply-chain
hits. Never executes PR head code.
- docs/design/pr-triage-workflows.md replaced with a brief overview + diagram.
The full implementation plan is kept locally as
docs/design/pr-triage-workflows-plan.md (gitignored).
- pr-triage-batch.yml's existing dispatch-scanner branch now resolves to
the new scanner; orchestrator unchanged.
* Fix markdownlint MD038 (pipe inside code span) in malicious-scan agent
* Remove agentic workflows
The agentic workflows were useful when CCA didn't support loading skills. That's now solved and supported. The agentic workflows come with a couple of issues (limited chars) which make it hard to use. Remove them in favor of the plugin system.
* Remove Agentic Workflows section from README
Removed section about Agentic Workflows from README.
* Remove stale /agentic-workflows references from CONTRIBUTING.md, CODEOWNERS, and docs
Agent-Logs-Url: https://github.com/dotnet/skills/sessions/8f129e2f-156e-4ee9-9897-896cd1981513
Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
* Add exp-mock-usage-analysis skill and evaluation
New experimental skill that detects unused, redundant, or unnecessary
mocks in .NET test suites. Covers Moq, NSubstitute, FakeItEasy, and
manual test doubles.
- SKILL.md with 6-step workflow: gather code, classify mocks, detect
anti-patterns, incorporate runtime data, generate recommendations,
report findings
- eval.yaml with 10 scenarios: trivial type mocking, unused setups,
redundant configs, stable utility mocks, well-placed mocks (no false
positives), NSubstitute support, FakeItEasy support, setup sprawl,
and 2 non-activation boundary tests
- Fixture files for each scenario with production + test code
- CODEOWNERS entry for @dotnet/dotnet-testing
* Remove no-headroom scenarios, increase NSubstitute timeout
Remove 4 scenarios where baseline already scores 5.0/5 (model handles
these well without skill guidance):
- Detect mocking of DTOs, records, and enums
- Recognize well-placed mocks without inventing false positives
- Decline when asked about non-mock test anti-patterns
- Decline mock framework migration request
Increase NSubstitute scenario timeout from 120s to 180s to avoid
baseline timeouts impacting scoring.
Remaining 6 scenarios all show positive skill impact (3→5, 3→4, etc).
* Document areas the model handles well without the skill
Add a subsection under 'When Not to Use' listing mock-related tasks
where the base model already scores 5.0/5 without skill guidance:
- Mocking trivial types (DTOs, records, enums)
- Recognizing well-placed mocks (no false positives)
* Fix md
* Improve
New experimental skill that assesses test maintainability: duplication,
test size, data-driven patterns, display names, builder/helper extraction,
and shared setup.
Amaury Levé