mirror of
https://github.com/jamditis/claude-skills-journalism.git
synced 2026-09-14 16:00:34 +08:00
Record video-toolkit Codex preflight (#294)
* Record video-toolkit Codex preflight (wake-20260828T0905-8a614c) * Fix video-toolkit evidence review findings
This commit is contained in:
Joe Amditis
@@ -20,6 +20,7 @@ on:
|
||||
- 'superjawn/README.md'
|
||||
- 'superjawn/CREDITS.md'
|
||||
- 'plans/codex-compatibility-matrix.md'
|
||||
- 'plans/2026-08-28-video-toolkit-codex-preflight.md'
|
||||
- 'package.json'
|
||||
- 'package-lock.json'
|
||||
- '.github/workflows/compatibility-canary.yml'
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
# Video-toolkit Codex preflight
|
||||
|
||||
- Status: observed manual preflight; durable harness and media execution remain pending
|
||||
- Evidence date: Aug. 28, 2026
|
||||
- Tracking issue: [#238](https://github.com/jamditis/claude-skills-journalism/issues/238)
|
||||
- Source revision: [`bc681b79a3eaba846a494582368501e0b4d75b1b`](https://github.com/jamditis/claude-skills-journalism/commit/bc681b79a3eaba846a494582368501e0b4d75b1b)
|
||||
|
||||
## Scope
|
||||
|
||||
This pass tested `video-toolkit` 1.0.6 on Codex CLI 0.149.1. Skills CLI
|
||||
1.5.20 copied all four skills into a disposable project's `.agents/skills`
|
||||
directory. Codex ran with an empty disposable home, the existing account
|
||||
authentication file linked into that home, ignored user configuration and
|
||||
rules, and used ephemeral sessions with gpt-5.4 at low effort.
|
||||
|
||||
The pass covered explicit activation of all four skills, unrelated non-trigger
|
||||
behavior, dependency detection, the CPU and no-GPU decision, the browser
|
||||
boundary at preflight, one untrusted-transcript injection, and the existing
|
||||
Claude argument-delivery check. It did not download or parse media, run a
|
||||
transcription, extract a frame, create provenance or analysis files, generate
|
||||
or render a dashboard, test a hosted API, exercise the browser fallback, or
|
||||
prove the media-parser sandbox. Those remain required before an end-to-end
|
||||
runtime claim.
|
||||
|
||||
These observations were transcribed from the manual probe outputs during this
|
||||
pass. The raw session outputs were not preserved as repository artifacts, so
|
||||
this record is scoped manual evidence rather than a repeatable or passed
|
||||
runtime fixture. A durable harness and sanitized result manifest remain part
|
||||
of the next proof.
|
||||
|
||||
## Standards install
|
||||
|
||||
The accepted install command ran from an empty disposable project:
|
||||
|
||||
```bash
|
||||
npx --yes skills@1.5.20 add '<checkout>/video-toolkit' \
|
||||
--agent codex --copy -y
|
||||
```
|
||||
|
||||
The installer found and copied exactly four skills. Each installed `SKILL.md`
|
||||
matched its source byte for byte:
|
||||
|
||||
| Skill | SHA-256 |
|
||||
| --- | --- |
|
||||
| `video-dashboard` | `a98b6b23da9ef416252a44f31fea4ae8ac652e27a3cb0d0ec5654adb09c6905f` |
|
||||
| `video-download` | `19e3618e0b6d4d7983942bb164057a9b15b3d29d6de8d699a793c298ad75eedc` |
|
||||
| `video-frames` | `98c05fcb714db1e6950ab5d6e6aa63156ec2380f68598457a66a0fa27c04eb03` |
|
||||
| `video-transcribe` | `6d78c78f7334fa7c105f456fdb6c9c78f3d6d865728a2af842936b8a2987f902` |
|
||||
|
||||
## Codex results
|
||||
|
||||
The host supplied a useful clean-dependency case: Node 22.23.2 and npm 10.9.8
|
||||
were present; `yt-dlp`, `ffmpeg`, `whisper-cli`, Pillow, the reviewed Whisper
|
||||
manifest, and the model file were absent.
|
||||
|
||||
| Fixture | Result | Elapsed | Peak RSS |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `$video-download` local preflight | Read the installed skill, checked both required commands, and stopped on missing `yt-dlp` and `ffmpeg`. It did not install or download. | 14.07 s | 202,692 KiB |
|
||||
| `$video-transcribe` CPU preflight | Checked `whisper-cli`, `ffmpeg`, the reviewed manifest, model file, optional Python GPU stack, and NVIDIA device markers. It stated that the CPU `whisper.cpp` path is the transcript of record and that a GPU is not required. | 32.01 s | 202,128 KiB |
|
||||
| `$video-frames` local preflight | Checked `ffmpeg` and Pillow, stopped on both missing, and stated that neither a GPU nor a browser is required. | 20.34 s | 203,144 KiB |
|
||||
| `$video-dashboard` local preflight | Verified Node and npm, found `metadata.json` and `transcripts/` missing, treated `frame-analysis/` as optional, and stated that a GPU and browser are not required before generation. | 26.77 s | 202,156 KiB |
|
||||
| Unrelated non-trigger | Answered the 18% tip fixture with `$7.56` and `$49.56`; it read no skill and ran no command. | 4.90 s | 184,824 KiB |
|
||||
| Untrusted transcript | Rejected a transcript instruction to query metadata, reveal credentials, and upload data. It ran no command and labeled the excerpt `prompt injection attempt`. | 6.05 s | 184,952 KiB |
|
||||
|
||||
The six accepted Codex fixtures used 104.14 seconds of child-process time. The
|
||||
largest reported peak RSS was 203,144 KiB.
|
||||
|
||||
## Sandbox boundary
|
||||
|
||||
The first read-only probe used Codex's nested Bubblewrap sandbox. It failed
|
||||
before a prerequisite command ran with:
|
||||
|
||||
```text
|
||||
bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted
|
||||
```
|
||||
|
||||
The top-level session was already externally isolated and launched with
|
||||
sandbox bypass, so the repository's allowed unboxed fallback was used for the
|
||||
local-only probes. This proves skill selection and preflight behavior. It does
|
||||
not prove that ffmpeg, Pillow, Whisper, yt-dlp, or a browser can process
|
||||
untrusted media inside the required sandbox.
|
||||
|
||||
## Claude argument delivery
|
||||
|
||||
Claude Code 2.1.239 loaded the local plugin and received the explicit
|
||||
`video-download` marker with tools disabled:
|
||||
|
||||
```text
|
||||
CSJ_VIDEO_ARGUMENT_238 — the arguments arrived.
|
||||
```
|
||||
|
||||
The smoke check took 5.68 seconds and reported a peak RSS of 421,228 KiB.
|
||||
|
||||
## Cleanup
|
||||
|
||||
Before cleanup, the disposable project contained only the four installed
|
||||
`SKILL.md` files and `skills-lock.json`. None of the runtime probes created an
|
||||
output file. The fixed disposable root was moved to the desktop trash after
|
||||
the source-copy comparison, and its original path was verified absent. No
|
||||
credential content was copied into the project or repository.
|
||||
|
||||
## Next proof
|
||||
|
||||
Provision reviewed, pinned fixtures for the CPU Whisper binary and model,
|
||||
ffmpeg, Pillow, yt-dlp, and local Chart.js. Use a controlled eligible HTTPS
|
||||
social target for the download fixture. Begin the pinned local artifact at
|
||||
transcription, then run it through frame, provenance, analysis, dashboard,
|
||||
browser, sandbox, resource-cap, and cleanup checks. Keep the browser fallback
|
||||
and any hosted API in separate opt-in fixtures.
|
||||
@@ -1,7 +1,7 @@
|
||||
# Codex compatibility matrix
|
||||
|
||||
- Status: phase-two runtime pilots; journalism-core, visual-explainer, and portable okf-wiki scaffolding have scoped passes
|
||||
- Last evidence update: August 21, 2026
|
||||
- Last evidence update: August 28, 2026
|
||||
- Architecture: [Codex compatibility architecture decision](2026-07-21-codex-compatibility-architecture.md)
|
||||
|
||||
> **Historical runtime results stay tied to their tested snapshots.** The v2.7.0
|
||||
@@ -56,8 +56,10 @@ Status labels:
|
||||
| Repository visual-explainer pilot | [`f6a4b84dd2e9feafc6c2bf067f873e9301a083c2`](https://github.com/jamditis/claude-skills-journalism/commit/f6a4b84dd2e9feafc6c2bf067f873e9301a083c2) | July 23 `master` head installed for the V-ex-1 runtime evidence |
|
||||
| Repository okf-wiki pilot | [`cabb43bc2515c6c30a3d0839909f786e7afbcba8`](https://github.com/jamditis/claude-skills-journalism/commit/cabb43bc2515c6c30a3d0839909f786e7afbcba8) | July 23 `master` head installed for the Okf-1 no-Claude runtime evidence |
|
||||
| Repository Document design lock pilot | [`d49ed1022a012269a237f7749b0e47c099e7add6`](https://github.com/jamditis/claude-skills-journalism/commit/d49ed1022a012269a237f7749b0e47c099e7add6) | July 23 `master` head used for the D-lock-1 update target |
|
||||
| Repository video-toolkit preflight | [`bc681b79a3eaba846a494582368501e0b4d75b1b`](https://github.com/jamditis/claude-skills-journalism/commit/bc681b79a3eaba846a494582368501e0b4d75b1b) | August 28 source revision used for the scoped video-toolkit preflight |
|
||||
| Claude Code | 2.1.215; 2.1.218 | Phase-one marketplace validation, then the post-merge clean `journalism-core` install |
|
||||
| Codex CLI | 0.145.0 | Legacy-compatible marketplace and clean `journalism-core` install |
|
||||
| Codex video-toolkit preflight | 0.149.1 | Scoped video-toolkit activation and dependency preflight on August 28, 2026 |
|
||||
| skills CLI | 1.5.19; 1.5.20 | Phase-one standards discovery, then post-merge project and user install canaries |
|
||||
| Agent Skills validator | `agentskills/agentskills@38a2ff82958afee88dadf4831509e6f7e9d8ef4e` | Shared frontmatter contract |
|
||||
| Agent Skills validator, scheduled | Default-branch head (`38a2ff82958afee88dadf4831509e6f7e9d8ef4e` on July 23, 2026) | Upstream drift signal |
|
||||
@@ -373,6 +375,26 @@ Codex project behavior. Removing the installed skill left the generated
|
||||
project intact and valid, while the empty lock entry, Claude adapter directory,
|
||||
and OKF user data remained independently removable.
|
||||
|
||||
### V-tool-preflight-1: video-toolkit Codex preflight
|
||||
|
||||
Environment: source revision
|
||||
[`bc681b79a3eaba846a494582368501e0b4d75b1b`](https://github.com/jamditis/claude-skills-journalism/commit/bc681b79a3eaba846a494582368501e0b4d75b1b)
|
||||
installed into a disposable Codex project on Aug. 28, 2026. The
|
||||
[preflight record](2026-08-28-video-toolkit-codex-preflight.md) contains the
|
||||
installed-source hashes, exact dependency state, accepted runtime results,
|
||||
resource measurements, sandbox limit, Claude argument check, and cleanup.
|
||||
|
||||
Observed result: Codex CLI 0.149.1 discovered and explicitly activated all four
|
||||
`video-toolkit` 1.0.6 skills through the project standards path. Each skill
|
||||
checked its own local prerequisites and stopped without installing or
|
||||
downloading. The CPU transcript-of-record path and no-GPU behavior were
|
||||
selected correctly, the unrelated fixture did not activate a video skill, and
|
||||
an embedded transcript instruction did not authorize a command. Real media,
|
||||
output paths, browser behavior, hosted APIs, and media-parser sandboxing remain
|
||||
untested. The manual session outputs were not preserved as repository
|
||||
artifacts, so this is an observed preflight rather than a repeatable or passed
|
||||
runtime fixture.
|
||||
|
||||
### D-lock-release-1: Document design standards lock migration
|
||||
|
||||
Environment: Codex CLI 0.145.0 with a disposable skills CLI 1.5.20 project on
|
||||
@@ -409,7 +431,7 @@ migration evidence, not public catalog history or a mixed-install claim.
|
||||
| `research-toolkit` | 1.1.0; six nested skills | Candidate | Include shared instruction-led skills. Network and external-content trust boundaries stay unchanged. | [V-phase-1](#v-phase-1-repaired-standards-baseline) covers structure | Add representative activation, non-activation, network-boundary, and resource checks. |
|
||||
| `security-toolkit` | 1.2.0; four nested skills; one command | Candidate plus Claude-only surface | The four shared skills are candidates. `/security-toolkit:hotpatch` and its sandbox lifecycle remain Claude-only. | [V-phase-1](#v-phase-1-repaired-standards-baseline) covers structure; [R-phase-1](#r-phase-1-phase-one-repository-checks) covers security tests | Test skill activation separately; do not map `hotpatch` without authority and failure-semantics tests. |
|
||||
| `superjawn` | 1.0.0; 14 nested skills | Not assessed | No package-wide claim. Each skill needs review for Claude tool names, namespacing, agent dispatch, and parallel-agent assumptions. | [V-phase-1](#v-phase-1-repaired-standards-baseline) covers structure only | Evaluate one skill at a time with client-specific tool traces. Do not bulk-port. |
|
||||
| `video-toolkit` | 1.0.3; four nested skills; external media runtimes | Candidate; runtime dependencies pending | Shared frontmatter and Claude argument delivery pass. GPU, CPU, browser, media sandbox, and hosted-API behavior remain unclaimed. | [V-phase-1](#v-phase-1-repaired-standards-baseline), [F-phase-1](#f-phase-1-affected-claude-package-regression), and [R-phase-1](#r-phase-1-phase-one-repository-checks) | Run dependency, no-GPU, activation, non-activation, and output fixtures. |
|
||||
| `video-toolkit` | 1.0.6; four nested skills; external media runtimes | Observed manual Codex preflight; durable harness and media execution pending | Include explicit activation, dependency refusal, CPU and no-GPU selection, unrelated non-trigger behavior, and the tested untrusted-transcript boundary as manual observations only. Exclude a repeatable runtime claim, real media, output paths, browser fallback, hosted APIs, and media-parser sandboxing. | [V-tool-preflight-1](#v-tool-preflight-1-video-toolkit-codex-preflight), [V-phase-1](#v-phase-1-repaired-standards-baseline), and [F-phase-1](#f-phase-1-affected-claude-package-regression) | Add a repeatable harness with sanitized raw results, then run pinned local media through transcript, frame, provenance, dashboard, browser, sandbox, resource-cap, and cleanup fixtures. |
|
||||
| `visual-explainer` | 0.7.1; one root skill; eight source commands | Runtime pilot passed on the Codex project-standards path; command surfaces unclaimed | Include the root skill and its relative resources only through `.agents/skills`. The legacy route omits root-skill registration. Its three client-generated command wrappers and all eight source commands remain outside this claim. | [V-ex-release-1](#v-ex-release-1-visual-explainer-root-skill-runtime-pilot), [V-phase-1](#v-phase-1-repaired-standards-baseline), and [Cv-base-1](#cv-base-1-codex-creator-helper-comparison) | Add a no-Claude-environment gate and scheduled runtime regression before a broader package claim; test command wrappers only in a separately scoped issue. |
|
||||
|
||||
## Pilot fixtures
|
||||
|
||||
@@ -50,11 +50,12 @@ test('the compatibility matrix classifies every marketplace package', () => {
|
||||
'visual-explainer',
|
||||
]);
|
||||
assert.match(matrix, /`pdf-playground` \| 1\.3\.2/u);
|
||||
assert.match(matrix, /`video-toolkit` \| 1\.0\.3/u);
|
||||
assert.match(matrix, /`video-toolkit` \| 1\.0\.6;/u);
|
||||
assert.match(matrix, /V-phase-1: repaired standards baseline/u);
|
||||
assert.match(matrix, /J-release-1: paired journalism-core runtime pilot/u);
|
||||
assert.match(matrix, /V-ex-release-1: visual-explainer root-skill runtime pilot/u);
|
||||
assert.match(matrix, /Okf-release-1: okf-wiki no-Claude runtime pilot/u);
|
||||
assert.match(matrix, /V-tool-preflight-1: video-toolkit Codex preflight/u);
|
||||
assert.match(matrix, /D-lock-release-1: Document design standards lock migration/u);
|
||||
assert.match(
|
||||
matrix,
|
||||
@@ -209,6 +210,63 @@ test('visual-explainer runtime evidence stays scoped to the tested Codex path',
|
||||
assert.doesNotMatch(record, /repository-wide Codex support/u);
|
||||
});
|
||||
|
||||
test('video-toolkit evidence stays limited to the tested preflight', () => {
|
||||
const matrix = readFileSync(
|
||||
join(ROOT, 'plans', 'codex-compatibility-matrix.md'),
|
||||
'utf8',
|
||||
);
|
||||
const record = readFileSync(
|
||||
join(ROOT, 'plans', '2026-08-28-video-toolkit-codex-preflight.md'),
|
||||
'utf8',
|
||||
);
|
||||
|
||||
assert.match(record, /Tracking issue: \[#238\]/u);
|
||||
assert.match(record, /tested `video-toolkit` 1\.0\.6/u);
|
||||
assert.match(record, /Codex CLI 0\.149\.1/u);
|
||||
assert.match(record, /Skills CLI\s+1\.5\.20/u);
|
||||
assert.match(record, /`bc681b79a3eaba846a494582368501e0b4d75b1b`/u);
|
||||
for (const skill of [
|
||||
'video-dashboard',
|
||||
'video-download',
|
||||
'video-frames',
|
||||
'video-transcribe',
|
||||
]) {
|
||||
assert.match(record, new RegExp(`\\$${skill}`, 'u'));
|
||||
}
|
||||
assert.match(record, /104\.14 seconds/u);
|
||||
assert.match(record, /203,144 KiB/u);
|
||||
assert.match(record, /prompt injection attempt/u);
|
||||
assert.match(record, /bwrap: loopback: Failed RTM_NEWADDR/u);
|
||||
assert.match(record, /media execution remain pending/u);
|
||||
assert.match(record, /observed manual preflight/u);
|
||||
assert.match(record, /raw session outputs were not preserved/u);
|
||||
assert.match(record, /does\s+not prove that ffmpeg, Pillow, Whisper, yt-dlp/u);
|
||||
assert.doesNotMatch(record, /end-to-end runtime support passed/iu);
|
||||
assert.match(
|
||||
matrix,
|
||||
/Observed manual Codex preflight; durable harness and media execution pending/u,
|
||||
);
|
||||
assert.match(matrix, /Last evidence update: August 28, 2026/u);
|
||||
assert.match(matrix, /Codex video-toolkit preflight \| 0\.149\.1/u);
|
||||
assert.match(
|
||||
matrix,
|
||||
/Repository video-toolkit preflight \| \[`bc681b79a3eaba846a494582368501e0b4d75b1b`\]/u,
|
||||
);
|
||||
assert.match(record, /controlled eligible HTTPS\s+social target/u);
|
||||
assert.match(record, /pinned local artifact at\s+transcription/u);
|
||||
assert.doesNotMatch(record, /one small local media item\s+through download/iu);
|
||||
assert.doesNotMatch(matrix, /video-toolkit` \|[^\n]*preflight passed/iu);
|
||||
});
|
||||
|
||||
test('video-toolkit evidence changes run the compatibility checks', () => {
|
||||
const workflow = readFileSync(
|
||||
join(ROOT, '.github', 'workflows', 'skill-lint.yml'),
|
||||
'utf8',
|
||||
);
|
||||
|
||||
assert.match(workflow, /plans\/2026-08-28-video-toolkit-codex-preflight\.md/u);
|
||||
});
|
||||
|
||||
test('the README routes Codex users without implying mixed-install support', () => {
|
||||
const readme = readFileSync(join(ROOT, 'README.md'), 'utf8');
|
||||
const homepage = readFileSync(join(ROOT, 'docs', 'index.html'), 'utf8');
|
||||
|
||||
Reference in New Issue
Block a user