Files

85 lines
1.9 KiB
Makefile
Raw Permalink Normal View History

ci: add dead-code detection (knip + vulture + orphaned-CI check) (#3787) ## What Adds a **report-only** `Dead Code` CI workflow that surfaces likely-orphaned code on every PR, across all three surfaces — TypeScript, Python, and GitHub Actions. Follow-up to the root `Dockerfile` cleanup (#3783) and the dead-code sweep in #3786: instead of finding this stuff by hand, catch it automatically. | Surface | Tool | Wiring | |---|---|---| | TypeScript | [**knip**](https://knip.dev) | `knip.json`; runs via `pnpm dlx knip@5`. Finds unused files, exports, types, and dependencies. | | Python | [**vulture**](https://github.com/jendrikseipp/vulture) | `dead_code` nox session + `make dead-code`; allowlist at `python/config/vulture_allowlist.py`. Finds unused functions/classes/variables (complements Ruff's F401/F841). | | GitHub Actions | small bash script | `.github/scripts/check-orphan-ci.sh` — flags reusable workflows and composite actions with no callers. | ## Why report-only (not blocking) Every job writes findings to the run's **Step Summary** and **never fails the build**. These tools carry unavoidable false positives on a library monorepo — public API surface, dynamic imports, import-map targets (e.g. core's `#platform`), framework entry points. A red ❌ on false positives would just train everyone to ignore the check. Once a job's config is refined enough that a clean run is the steady state, it can be flipped to blocking. ## Validation (ran each locally) - **knip**: 0 unused *files* after scoping out the e2e-test workspaces (knip crashes traversing `ts/e2e-tests/**` — pre-existing knip bug, filed via `ignoreWorkspaces`) and build/docs artifacts. Export/dep categories surface advisory items. - **vulture**: clean run (report-only); allowlist suppresses the 3 `TYPE_CHECKING` re-exports in `custom_tool.py`; `build/`/`dist/` excluded. Surfaces 4 genuine minor items (`bases`, `desc`×3). - **orphan-CI script**: finds none — the repo has no orphaned CI plumbing today. - **Pinning**: `jk actions check` clean for this workflow; `actions/checkout` is SHA-pinned, the two local composite actions need no pin. Local usage is documented under **Dead code detection** in `CONTRIBUTING.md`. ## Note on scope These cover TS/Python/GHA — they would *not* have caught the root `Dockerfile` itself (an arbitrary root file no tool tracks). That class stays a manual-review concern; the orphan-CI script is the closest analogue for the CI surface.
2026-07-10 02:17:54 +04:00
.PHONY: clean-build sync provider create-provider env fmt chk snt tst type_inference dead-code bump build format check sanity test
chore(agents): normalize repo guidance skills (#3666) This PR: - replaces the duplicated repo-local skill set with the requested canonical `.agents/skills` taxonomy and short router `SKILL.md` files backed by first-level references - makes `.claude/skills` a compatibility symlink to `.agents/skills` and removes hand-maintained Claude skill/rule copies - moves docs agent guidance and decisions into neutral `docs/agent-guidance/` and `docs/decisions/`, with `docs/CLAUDE.md` reduced to a shim - retires CLI Cursor rules after migrating CLI design, Effect source, and client-cache sync guidance into `AGENTS.md` and `cli-command` - adds nested `AGENTS.md` files for TS, core, providers, e2e, Python, Python providers, and docs - adds `pnpm validate:agent-skills` to validate skill frontmatter, taxonomy, references, symlink invariants, stale paths, and command names - implements missing Python `tst` and `snt` nox sessions that existing Makefile targets already exposed - no changeset: repository guidance/tooling only, no published SDK package behavior ## Verification - `pnpm validate:agent-skills` -> `Validated 14 canonical agent skills and guidance invariants.` - `for skill in .agents/skills/*; do python /Users/jkomyno/.codex/skills/.system/skill-creator/scripts/quick_validate.py "$skill" || exit 1; done` -> 14x `Skill is valid!` - `pnpm --dir ts/packages/cli validate:skills` -> `Validated composio-cli skill builds for stable and beta.` - `cd python && uv run nox --list` -> includes `tst` and `snt` - `cd python && uv run nox -s snt` -> 18 passed - `cd python && uv run nox -s tst -- tests/test_imports.py` -> 8 passed - `git diff --check` / `git diff --cached --check` -> clean - stale reference search for retired docs/Claude/Cursor paths -> no matches ## Forward Tests - TypeScript core bug: loaded `bug-fixing`, `typescript-sdk`, `typescript-testing`; found correct root/ts/core `AGENTS.md` route. - Python provider: loaded `python-providers`, `python-testing`; found missing nox sessions, fixed here. - CLI command: loaded `cli-command`; found recording/changeset wording gaps, fixed here. - Cross-SDK drift: loaded `cross-sdk-parity`; again found Python nox drift, fixed here. - Docs + decision: loaded `docs-decisions`; found decision template/index and Twoslash path gaps, fixed here. ## Notes The first normal `git commit` attempt hit a lint-staged/Git stash limitation while replacing `.claude/skills/` with a symlink (`path ... beyond a symbolic link`). The final commit used `--no-verify` after the validators and formatting checks above passed.
2026-06-27 00:35:57 +04:00
PROVIDER_DIRS := $(patsubst %/,%,$(sort $(dir $(wildcard providers/*/pyproject.toml))))
2025-06-17 12:58:00 +05:30
clean-build:
@rm -rf dist/ build/
@set -e; for provider in $(PROVIDER_DIRS); do\
2025-06-17 12:58:00 +05:30
rm -rf build $$provider/dist $$provider/build;\
done
sync:
@uv sync
@uv pip install -e .
provider:
@set -e; for provider in $(PROVIDER_DIRS); do\
2025-06-17 12:58:00 +05:30
uv pip install $$provider;\
done
create-provider:
@if [ -z "$(name)" ]; then\
echo "Please provide a provider name: make create-provider name=<provider-name> [agentic=true] [output=<directory>]";\
exit 1;\
fi
@ARGS="$(name)";\
if [ "$(agentic)" = "true" ]; then\
ARGS="$$ARGS --agentic";\
fi;\
if [ ! -z "$(output)" ]; then\
ARGS="$$ARGS --output-dir $(output)";\
fi;\
bash scripts/create-provider.sh $$ARGS
2025-06-17 12:58:00 +05:30
env:
@echo "* creating new environment"
@if [ -z "$$VIRTUAL_ENV" ];\
then\
chore(toolchain): finish mise migration (#3493) ## Summary Depends on #3492. This completes the Phase 2 migration by removing the transitional version-file layer and making `mise.toml` plus `mise.lock` the repository toolchain source of truth. It also moves runtime test matrices into `toolchain-versions.json`, so CI matrix changes are explicit and reviewable without reintroducing `.nvmrc`, `.dvmrc`, `.bun-version`, or `.python-version`. The Node.js e2e matrix now starts at the latest Node 22 LTS line and also covers the latest Node 24 and Node 25 lines. That removes Node 20 from the well-known e2e versions while keeping us covered on the runtimes SDK users are moving toward. ## Rationale Phase 1 introduced mise side by side with the existing version files to keep the first PR low-risk. Phase 2 removes that compatibility layer so there is one place to update tool versions. That avoids silent drift between local setup, GitHub Actions, Docker E2E images, release docs, and install-time checks. The composite setup actions now install Node, Bun, Python, and uv through mise by default, with explicit version overrides only where a matrix needs them. New GitHub actions added in this PR are pinned by release commit SHA and include the release version comment. ## What changed - Deleted the transitional root/version files: `.nvmrc`, `.bun-version`, `.dvmrc`, root `.python-version`, and `python/.python-version`. - Removed `idiomatic_version_file_enable_tools` from `mise.toml` and added a committed `mise.lock` for linux/macOS x64/arm64 tool resolution. - Replaced `BYPASS_BUN_VERSION_CHECK` with `BYPASS_TOOLCHAIN_CHECK`, and made Docker E2E image installs use that bypass because they receive explicit build args instead of installing mise. - Updated Node/Bun and Python/uv composite actions to default to mise, remove `*-version-file` inputs, report resolved versions, and cache pnpm after `corepack enable`. - Centralized CI runtime matrices in `toolchain-versions.json` for TS E2E, Python tests, and CLI npm fallback coverage. - Updated the Node E2E matrix to `22.22.3`, `24.16.0`, and `25.9.0`, removing Node 20 from the well-known runtime versions. - Updated workflows, docs, E2E helpers, Dockerfiles, and release guidance to reference `mise.toml` / `mise.lock`. ## Verification - Verified latest Node 22/24/25 releases from the official Node dist index: `22.22.3`, `24.16.0`, `25.9.0` (https://nodejs.org/dist/index.json). - `pnpm install --frozen-lockfile` - `pnpm --filter @e2e-tests/utils typecheck` - `pnpm --filter @e2e-tests/utils exec tsc --noEmit --target es2022 --module esnext --moduleResolution bundler --types bun --resolveJsonModule --skipLibCheck --strict scripts/docker-build.ts` - `bash -n ts/scripts/pre-install.sh && bun run ts/scripts/pre-install/check-toolchain.ts && BYPASS_TOOLCHAIN_CHECK=1 bash ts/scripts/pre-install.sh` - `pnpm exec prettier --check ...` on touched YAML/Markdown/TS/JSON files - `ruby -e "require \"yaml\"; ARGV.each { |f| YAML.load_file(f) }" ...` on touched actions/workflows - `mise lock --platform linux-x64,linux-arm64,macos-arm64,macos-x64 && git diff --exit-code mise.lock` - `mise exec node@22.22.3 -- pnpm --filter @e2e-tests/utils typecheck` - `mise exec node@22.22.3 -- pnpm --filter @e2e-tests/node-esm-basic typecheck` - `mise exec node@22.22.3 -- pnpm --filter @e2e-tests/node-cjs-basic typecheck` - `docker manifest inspect node:24.16.0-slim` - `docker manifest inspect node:25.9.0-slim` - `git diff --check`
2026-06-15 14:03:21 +04:00
uv venv --seed --prompt composio --python 3.12;\
2025-06-17 12:58:00 +05:30
uv sync;\
uv sync --dev;\
make provider;\
uv pip install -e .;\
echo "* enter virtual environment with all development dependencies now";\
else\
uv sync;\
uv pip install -e .;\
echo "* already in a virtual environment (exit first ('deactivate') to create a new environment)";\
fi
@echo "* run 'source .venv/bin/activate' to enter the development environment."
fmt:
@nox -s fmt
chk:
@nox -s chk
ci: add dead-code detection (knip + vulture + orphaned-CI check) (#3787) ## What Adds a **report-only** `Dead Code` CI workflow that surfaces likely-orphaned code on every PR, across all three surfaces — TypeScript, Python, and GitHub Actions. Follow-up to the root `Dockerfile` cleanup (#3783) and the dead-code sweep in #3786: instead of finding this stuff by hand, catch it automatically. | Surface | Tool | Wiring | |---|---|---| | TypeScript | [**knip**](https://knip.dev) | `knip.json`; runs via `pnpm dlx knip@5`. Finds unused files, exports, types, and dependencies. | | Python | [**vulture**](https://github.com/jendrikseipp/vulture) | `dead_code` nox session + `make dead-code`; allowlist at `python/config/vulture_allowlist.py`. Finds unused functions/classes/variables (complements Ruff's F401/F841). | | GitHub Actions | small bash script | `.github/scripts/check-orphan-ci.sh` — flags reusable workflows and composite actions with no callers. | ## Why report-only (not blocking) Every job writes findings to the run's **Step Summary** and **never fails the build**. These tools carry unavoidable false positives on a library monorepo — public API surface, dynamic imports, import-map targets (e.g. core's `#platform`), framework entry points. A red ❌ on false positives would just train everyone to ignore the check. Once a job's config is refined enough that a clean run is the steady state, it can be flipped to blocking. ## Validation (ran each locally) - **knip**: 0 unused *files* after scoping out the e2e-test workspaces (knip crashes traversing `ts/e2e-tests/**` — pre-existing knip bug, filed via `ignoreWorkspaces`) and build/docs artifacts. Export/dep categories surface advisory items. - **vulture**: clean run (report-only); allowlist suppresses the 3 `TYPE_CHECKING` re-exports in `custom_tool.py`; `build/`/`dist/` excluded. Surfaces 4 genuine minor items (`bases`, `desc`×3). - **orphan-CI script**: finds none — the repo has no orphaned CI plumbing today. - **Pinning**: `jk actions check` clean for this workflow; `actions/checkout` is SHA-pinned, the two local composite actions need no pin. Local usage is documented under **Dead code detection** in `CONTRIBUTING.md`. ## Note on scope These cover TS/Python/GHA — they would *not* have caught the root `Dockerfile` itself (an arbitrary root file no tool tracks). That class stays a manual-review concern; the orphan-CI script is the closest analogue for the CI surface.
2026-07-10 02:17:54 +04:00
dead-code:
@nox -s dead_code
2025-06-17 12:58:00 +05:30
snt:
@nox -s snt
tst:
@nox -s tst
type_inference:
@nox -s type_inference
chore(agents): normalize repo guidance skills (#3666) This PR: - replaces the duplicated repo-local skill set with the requested canonical `.agents/skills` taxonomy and short router `SKILL.md` files backed by first-level references - makes `.claude/skills` a compatibility symlink to `.agents/skills` and removes hand-maintained Claude skill/rule copies - moves docs agent guidance and decisions into neutral `docs/agent-guidance/` and `docs/decisions/`, with `docs/CLAUDE.md` reduced to a shim - retires CLI Cursor rules after migrating CLI design, Effect source, and client-cache sync guidance into `AGENTS.md` and `cli-command` - adds nested `AGENTS.md` files for TS, core, providers, e2e, Python, Python providers, and docs - adds `pnpm validate:agent-skills` to validate skill frontmatter, taxonomy, references, symlink invariants, stale paths, and command names - implements missing Python `tst` and `snt` nox sessions that existing Makefile targets already exposed - no changeset: repository guidance/tooling only, no published SDK package behavior ## Verification - `pnpm validate:agent-skills` -> `Validated 14 canonical agent skills and guidance invariants.` - `for skill in .agents/skills/*; do python /Users/jkomyno/.codex/skills/.system/skill-creator/scripts/quick_validate.py "$skill" || exit 1; done` -> 14x `Skill is valid!` - `pnpm --dir ts/packages/cli validate:skills` -> `Validated composio-cli skill builds for stable and beta.` - `cd python && uv run nox --list` -> includes `tst` and `snt` - `cd python && uv run nox -s snt` -> 18 passed - `cd python && uv run nox -s tst -- tests/test_imports.py` -> 8 passed - `git diff --check` / `git diff --cached --check` -> clean - stale reference search for retired docs/Claude/Cursor paths -> no matches ## Forward Tests - TypeScript core bug: loaded `bug-fixing`, `typescript-sdk`, `typescript-testing`; found correct root/ts/core `AGENTS.md` route. - Python provider: loaded `python-providers`, `python-testing`; found missing nox sessions, fixed here. - CLI command: loaded `cli-command`; found recording/changeset wording gaps, fixed here. - Cross-SDK drift: loaded `cross-sdk-parity`; again found Python nox drift, fixed here. - Docs + decision: loaded `docs-decisions`; found decision template/index and Twoslash path gaps, fixed here. ## Notes The first normal `git commit` attempt hit a lint-staged/Git stash limitation while replacing `.claude/skills/` with a symlink (`path ... beyond a symbolic link`). The final commit used `--no-verify` after the validators and formatting checks above passed.
2026-06-27 00:35:57 +04:00
# Friendly aliases for the short session names above (e.g. `make test` == `make tst`).
format: fmt
check: chk
sanity: snt
test: tst
2025-06-17 12:58:00 +05:30
bump: clean-build
@uv run python scripts/bump.py
build: clean-build
2025-06-19 13:35:39 +05:30
@./.venv/bin/python -m build
@set -e; for provider in $(PROVIDER_DIRS); do\
2025-06-19 13:35:39 +05:30
./.venv/bin/python -m build $$provider;\
2025-06-17 12:58:00 +05:30
cp $$provider/dist/* dist/;\
done