Files
conorbronsdon__avoid-ai-wri…/scripts/promo-drift-report.py
T
Conor Bronsdon cc0c2f43ab Catch promo-surface drift from the release that causes it (#80)
* Catch promo-surface drift from the release that causes it

The pattern count and word-table count are already policed inside this repo:
check-pattern-count.sh derives both from SKILL.md and fails CI if the README
disagrees. Nothing policed the second link — four surfaces in other repos
quote those numbers, and a release moves the number here while they sit still.

That drift cannot be caught from the other side. Their files do not change on
release day, so no pre-commit hook over there ever has a commit to fire on.
It went unnoticed for three weeks: three of the four surfaces were still on
53 pattern categories and 109 table entries.

So the check runs here, on release, with a weekly backstop:

- .ssot.yaml declares the README bullets as canonical and the four surfaces
  as copies (dogfoods conorbronsdon/ssot-check)
- promo-drift.yml clones the surfaces as siblings and runs the check
- promo-drift-report.py turns the JSON into a readable issue body

A surface that cannot be read is reported as NOT CHECKED and does not fail
the job — conorbronsdon-site is private, and failing red every week over a
missing token trains people to ignore the check. It is always printed. Set
SITE_REPO_TOKEN to bring it into coverage; no code change needed.

Verified against real clones of all four surfaces: in-sync exits 0, a stale
number exits 1 with file:line, an unreadable surface exits 0 and says so, and
a reworded canonical bullet is a hard failure rather than a silent pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YA7143ow5qvWZghbeQKiUd

* Drop the site from the manifest; the check now needs no credential

conorbronsdon.com/builds floors both counts instead of quoting them, so there
is nothing left to keep in sync there. That removes the only private surface,
which removes the only reason this workflow wanted a PAT.

Every tracked surface is now a public repo and the job runs on its own
GITHUB_TOKEN. The not-checked path stays — a surface repo can still be
renamed or fail to clone, and that must read as a coverage gap, not a pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YA7143ow5qvWZghbeQKiUd

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 10:58:45 -07:00

99 lines
3.5 KiB
Python
Executable File

#!/usr/bin/env python3
"""Turn `ssot_check.py check --json` output into a readable drift report.
Reads the JSON on stdin, writes markdown to stdout. Exit code decides whether
the workflow opens an issue:
0 every surface that could be read is in sync
1 at least one surface carries a stale number
A surface that could not be read at all (repo renamed, moved, or a clone that
failed) is reported as NOT CHECKED and does *not* fail the job. That case is a
coverage gap, not drift, and conflating the two makes a red check meaningless.
It is always printed, never silently dropped.
"""
import json
import sys
# Statuses ssot-check emits for a copy it managed to read and compare.
REAL_DRIFT = {"drifted", "stale_entry"}
UNREADABLE = {"unverified"}
def main():
try:
result = json.load(sys.stdin)
except json.JSONDecodeError as exc:
print(f"could not parse ssot-check output: {exc}", file=sys.stderr)
return 2
drifted, unreadable, checked = [], [], 0
for fact in result.get("facts", []):
name = fact.get("name", "?")
canon = fact.get("canonical", {})
cval = canon.get("value")
# A broken canonical means the README bullet moved or was reworded.
# That is this repo's own problem and always a hard failure.
if fact.get("status") == "canonical_moved":
drifted.append(
f"- **{name}** — canonical unreadable in "
f"`{canon.get('file')}`: {canon.get('error')}. "
f"The pattern in `.ssot.yaml` no longer matches the README."
)
continue
for copy in fact.get("copies", []):
status = copy.get("status")
if status in REAL_DRIFT:
drifted.append(
f"- **{name}** — `{copy.get('file')}`"
f"{':' + str(copy['line']) if copy.get('line') else ''} "
f"says **{copy.get('value', '?')}**, canonical is "
f"**{cval}**"
+ (f" ({copy['note']})" if copy.get("note") else "")
)
elif status in UNREADABLE:
# ssot-check reports the *source* it tried, so a missing file
# comes back as the bare word "local". Say what that means.
note = copy.get("note") or ""
if note in ("", "local"):
note = ("file not present on the runner — the surface repo "
"was renamed, moved, or failed to clone")
unreadable.append(f"- **{name}** — `{copy.get('file')}` ({note})")
else:
checked += 1
lines = []
if drifted:
lines.append(
"The number moved here and these surfaces still carry the old one.\n"
)
lines.append("### Drifted\n")
lines.extend(drifted)
lines.append("")
if unreadable:
lines.append("### Not checked\n")
lines.append(
"These surfaces could not be read on the runner, so they are "
"unverified either way — not a pass.\n"
)
lines.extend(unreadable)
lines.append("")
lines.append(
f"_{checked} copies verified in sync · "
f"{len(drifted)} drifted · {len(unreadable)} not checked · "
f"ssot-check {result.get('version', '?')} on "
f"{result.get('generated', '?')}_"
)
print("\n".join(lines))
return 1 if drifted else 0
if __name__ == "__main__":
sys.exit(main())