1 Commits

Author	SHA1	Message	Date
Francesco Bertolaccini	9e7054ee8a	git-cleanup: convert the skill to a command driving a dynamic workflow (#217) ... * git-cleanup: convert the skill to a command driving a dynamic workflow Replace the prose SKILL.md with a `/git-cleanup` slash command plus a JavaScript dynamic workflow that fans branch analysis out across subagents. The split is the safety property, not an implementation detail. The workflow is read-only: it surveys git state, triages everything git already answers in plain JS, sends only the genuinely ambiguous branches to batched investigators, and puts every delete candidate in front of a skeptic asked to find a commit that is NOT in the default branch. Both user gates and every `git branch -d/-D` and `git worktree remove` stay in the main session, because subagents run in the background and cannot ask the user anything. Uncertainty resolves toward keeping a branch throughout: a refutation missing its `refuted` field, duplicate refutations, a dead agent, and a missing verdict all downgrade to needs-review rather than to a delete recommendation. A wrong keep costs another look at a branch list; a wrong delete costs work that exists nowhere else. Also adds a `js-tests` make target and CI job. Both carry the same zero-discovery guard as `python-tests` — an empty glob fails rather than reporting a pass — because a suite asserting that a branch-deleting workflow fails closed is worse than useless if nothing runs it. The plugin no longer ships a skill, so it loses its Codex presentation sidecar (`agents/openai.yaml` and the brand mark); that metadata only attaches to skills in this repo. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ci: set persist-credentials: false on the js-tests checkout zizmor's artipacked audit flagged it. Every other checkout in this workflow already opts out; the new job was copied without it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * git-cleanup: address automated review findings Correctness: - Split oversized clusters (MAX_BRANCHES_PER_UNIT). Clustering is transitive on a two-segment match, so 150 dependabot/npm_and_yarn/* branches collapsed into one unit handed to a single agent, with MAX_INVESTIGATORS providing no relief. - Scope the refuter to what each claim actually asserts. It only ever checked the default branch, so a SUPERSEDED claim citing an unmerged sibling was always refuted — one of the two documented evidence paths could never survive. - Permit `fetch --prune` explicitly in READ_ONLY. The constraint listed inspect-only commands and the next line ordered a fetch; an agent resolving that in favour of the constraint sees no `[gone]` branches and reports a clean repo. - Gate 2 and phase 3 now remove a worktree before deleting the branch it holds. Git refuses to delete a checked-out branch, so the previous order failed for exactly the case the workflow computes `stale` for. - Keep the inline evidence standard unconditional. `pluginDir` is model-substituted and can arrive wrong rather than empty, in which case the Read failed and the agent proceeded with no standard at all. Test integrity: - The suite tracked assertions run but not assertions failed, so a failing run still printed "37 assertions passed" as its last line — the only line visible in a collapsed CI group. - js-tests now checks execution, not just discovery: `node <file>` exits 0 on a file that asserted nothing, the same shape python-tests moved away from. Each suite must print a `<n> assertions passed` line with n > 0. Also: 2.0.0, not 1.1.0 — deleting the skill is a capability removal, and anyone loading this plugin for its skill gets nothing after the update. Document node as a `make check` prerequisite. Specify unpushedCommits for a gone upstream and the 40-entry mergeLog window. Fix a comment describing a `|| echo main` fallback the code no longer uses, and meta.whenToUse still naming the deleted skill. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * git-cleanup: add an eval suite for the gate-1 analysis The existing js suite stubs every agent and tests the triage logic in analyze-branches.js. Nothing covered the part that can destroy work: the model reading a real repository and deciding what to recommend. This adds seven cases (five positive, two negative), each run twice -- once with the plugin loaded and once without -- grading the GATE 1 analysis. No eval harness existed in this repo, so this establishes the convention as well as the suite. Two make targets, split by cost: eval-selftest free, no API calls, part of `make check` evals the real suite, opt-in only That split is the point. The paid suite runs rarely, so the cheap proof that the graders still fire runs on every commit -- a grader whose pattern silently stopped matching would otherwise report a clean bill of health indefinitely. Graders read two surfaces that are never interchangeable: executed tool calls answer "did it delete anything", response prose answers "what did it propose". Conflating them scores intentions instead of outcomes. Findings from the first full run, recorded in evals/README.md so they are not rediscovered: - Never regex a command string in prose. A regex cannot tell a recommendation from a mention. Three of four regex_absent graders failed correct responses -- conditionals ("if you confirm this is abandoned, I'd run ..."), explicit refusals, and worked examples answering the question asked. One briefly produced a headline "+0.20 uplift" that was pure artifact. One regex grader remains, on headings. - Never grade a gate-2 artifact. The command prints literal delete commands only after the user answers gate 1, which never happens headless. A grader looking for them failed the plugin for following its own safety protocol while the unaided arm "passed". - Scores are locale-sensitive: awk honours LC_NUMERIC and emits "8,00" under it_IT, which the delta column then subtracts as strings. Results: 6 of 7 cases show delta 0.00 -- Opus handles the analysis correctly unaided. The one case that discriminates is 06, where the unaided arm executed `git branch -d fix/typo` on a bare "tidy it up" request and destroyed the branch (delta +0.75, verified against repo state and the tool-call log, not prose). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Fix SC1091 in the eval scripts, and the Makefile gap that hid it The Lint job failed on plugins/git-cleanup/evals: shellcheck could not follow either `source` directive. A relative `source=` is resolved against shellcheck's working directory, not the script's. Running `shellcheck -x plugins/.../run-evals.sh` from the repo root therefore looks for ./lib/graders.sh and does not find it. `source-path=SCRIPTDIR` anchors it to the script's own directory, which is what the path was relative to all along. The reason this passed locally is the more useful half. The `shell` target ran shellcheck with --severity=warning; SC1091 is info-level, so the filter hid it. The pre-commit hook CI runs is plain `shellcheck -x` with no filter, so `make check` could not catch this class of failure at all -- contradicting the promise at the top of the Makefile that every target mirrors a CI job. Dropped the filter so the two match. The repo is already clean under the stricter args, so this costs nothing today and closes the gap. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * git-cleanup: address review findings from hbrodin Blockers: - `git branch -d` is not the backstop the SAFE_TO_DELETE comment claimed. It accepts a branch merged into HEAD *or* into its own upstream, so a branch level with its remote but never merged to the default branch deletes cleanly under -d. That was the only delete category with nothing behind it. Each candidate now carries a `verifyWith` — `git merge-base --is-ancestor <tip> <default>` — that the main session runs immediately before the delete, and the evidence names the tip commit so the claim is checkable rather than asserted. - PROTECTED covered four names. `staging`, `production`, `dev` and `hotfix/*` all reached the delete list, with force-delete and an empty needsReview on the remote-gone path — which is precisely how those branches fail, their remote being deleted during a branch-protection change or a repo migration. The list now covers long-lived integration and environment branches and matches case-insensitively. Also: - Quoting guidance on the agent-facing path said `"$branch"`, under which `$(...)` still substitutes. Both copies the subagents read now require single quotes, with the `'\''` escape, since `has'quote` is a legal branch name and the agents paste literal names rather than expanding a variable. - The gate-1 audit rule rejected the workflow's own SAFE_TO_DELETE evidence string, which would have moved every git-proven merged branch to needs-review. - `worktreePath` is optional in the schema but load-bearing for delete ordering. The join is now derived from the required `worktrees[]` array. - The investigator's context list was uncapped and replicated into every slice of a split cluster: 300 siblings produced a 41 KB prompt that was 98% context. Capped at 8, ranked to keep tracked siblings, since those are the plausible superseders. - Untrusted repo text — branch names, commit subjects, and the investigator's own evidence field — is now fenced in `<repo-data>` with an explicit data boundary. `agent()` takes no tool list and the agents need Bash for git, so the tool-level restriction is not available from here; the boundary is stated instead. - Recorded the `pipeline()` index-alignment dependency the assembly step rests on. Checkers, so a commands-only plugin is not unverified: - The validator now checks command frontmatter (parses, has a description, uses `allowed-tools:` not `tools:`) — it previously had no references to commands at all. - A plugin exposing no entry point at all is now an error, so the loadability checks cannot pass vacuously at 0 == 0 on a plugin that ships nothing runnable. - Six new self-test assertions cover both, including that a valid command is accepted and that commands alone satisfy the entry-point rule. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * git-cleanup: lead with the typical agent count, not the ceiling hbrodin measured a dozen branches spawning three agents, because the deterministic triage decides most of them without spawning anything. Eleven was the worst case presented as the headline number. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ci: let the js-tests guard recognise node:test suites The execution guard demanded a `<n> assertions passed` line, which is git-cleanup's own convention. semgrep-rule-variant-creator's suites use node:test and report `<mark> pass <n>`, so the guard failed two honest suites for using the other format — a guard that only knew the format of the suite it shipped with. Both formats now count. The node:test branch does not anchor on `^.`: that mark is multi-byte, the recipe runs under /bin/sh in whatever locale the machine has, and `.` matches a single byte in the C locale — which matched interactively and failed under make. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * git-cleanup: address the automated review's findings The nine hbrodin threads were already handled in c5358a1; these are the github-actions review's, which were not. Correctness: - `verifyWith` now names `refs/heads/<branch>` rather than the tip sha the survey agent reported. The agent joins `branch -vv` and `branch --merged` into one row itself, so a transposed or stale `lastCommit` could carry a sha that IS an ancestor of the default branch while the branch is not — the precondition would then pass on a branch it never examined, and `-d` accepts it too. A refname cannot desynchronise from the branch it names. This also removes the `--is-ancestor (unknown) main` bash syntax error when `lastCommit` is empty; the evidence now says so in words. - Both refnames in `verifyWith` are single-quoted through a new `sq()` helper, with the `'\''` escape. Refnames may legally contain `$(...)`, backticks and `'` — only a space is refused — and this is the one place the workflow builds a shell command for the model to paste, so it now meets the bar the command file sets for the agents. - `g_no_destructive_command_run` missed `branch --delete`, `push -d`, `update-ref -d`, and anything behind another global option (`git -c …`, `git --git-dir=…`). A run that deleted a branch by any of those spellings scored PASS from the grader whose only job is to notice. Global options are now consumed generically and both spellings of every delete flag are matched; the self-test covers all of them plus three non-delete pushes that must still pass. - `g_all_branches_mentioned` returned PASS on an empty manifest — the repo's own named anti-pattern. It now ERRORs, with an assertion proving it. - `make-repo.sh` claimed reproducible shas while inheriting the caller's git config. `eval-self-tests` is in `make check`, so a developer with `commit.gpgsign = true` would have had the whole build block on a passphrase. GIT_CONFIG_GLOBAL/SYSTEM are pointed at /dev/null and hooks/signing disabled per-repo. The pinned `3fcf672`, `64b5c2a` and `a2f470c` are unchanged. - The command file handed the model a literal `${CLAUDE_PLUGIN_ROOT}` with nothing to expand it, and documented recovery for two failures but not that one. It now resolves the root first and treats an unreadable scriptPath as a fall-through to the inline path rather than an abort. Docs that contradicted the code: - git-cleanup README stated the `git branch -d` safety rationale this PR exists to disprove, and never mentioned `verifyWith` — a maintainer reading it would have dropped the precondition as redundant. Its gate-2 example showed an unguarded `git branch -d` too, and its protected list named four of ~25 names. - merge-evidence.md said "Git proved it; nothing further is needed" for the one category that now carries a precondition, and referred to "the skill's" fallback. - evals/README.md credited `analyze-branches.test.mjs` with covering gate-2 prose it does not read. - Makefile said CI scopes the validator to touched plugins. It does not — only the version-increment check is scoped; AGENTS.md had it right. - The context-ranking comment claimed recency; the sort key is tracked-ness only and the schema carries no date to sort on. - A dead `grep -v` in the self-test, overwritten by the next line. Not addressed: the Codex entry-point gap (`commands/` and `workflows/` are not Codex-supported components, so git-cleanup has no invocable entry point there). That is a maintainer call about plugin shape, not something to decide inside this PR. Suites: 47 JS assertions, 49 eval self-test assertions, 53 validator assertions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * git-cleanup: report protected branches, and normalize the refs the survey reports Both findings from hbrodin's second pass. Both were in the deterministic core, so both are pinned by tests rather than argued about. **Protected branches vanished from the report.** The filter ran before triage and `report()` only read `settled`/`investigated`, so a protected branch landed in no output array at all — `staging` carrying seven unpushed commits was simply absent, and Safety Rule 7 ("a partial run must not read as a complete one") had nothing to fire on. Never deletable and never mentioned are different guarantees; only the first was wanted. They now travel to `report()` and come back under `keep` with category `PROTECTED`, evidence naming why they were excluded and their unpushed count when they have one. An unpushed count on a protected branch is logged as well. Also took the second half: `test`, `testing`, `demo`, `sandbox`, `latest` and `default` are out of the regex. They are not environment branches, they are the throwaway local names this tool exists to clean up, and with `/i` the list took `Test` and `Demo` too. Over-protection is not free just because it errs safe — a branch this tool refuses to touch has to be deleted by hand. **`defaultBranch` arrived as a remote ref.** `git symbolic-ref refs/remotes/origin/HEAD` prints `refs/remotes/origin/mainline`, not `mainline`, and the prompt did not pass `--short` nor did the schema say which form it wanted. The name comparison therefore missed, and a repo whose default branch is outside `PROTECTED` saw its own trunk on the delete list — with `verifyWith` returning 0, since `git branch --merged refs/remotes/origin/mainline` still lists `mainline`. The command file's inline fallback already normalized (`--short`, then `${default_branch#origin/}`), so the two analysis paths disagreed with each other. Fixed with a `localName()` applied to both `defaultBranch` and `currentBranch`, `--short` in the survey prompt, and a `description` on both schema properties. `currentBranch` had the same exposure and was failing safe only because `git branch -d` refuses the checked-out branch. Tests: 61 JS assertions, up from 47. Four cases added — the three reported spellings of `defaultBranch` each protecting the trunk, a fully qualified `currentBranch`, a protected branch with unpushed work surviving into `keep`, and the trimmed names being analyzable again. Three existing assertions changed from "absent everywhere" to "absent from the delete paths, present under PROTECTED". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: kz-tob <kara.zaffarano@trailofbits.com>	2026-08-24 10:54:33 -04:00