decision(perf-testing): take the fastest of five runs rather than a single
sample — noise can only slow a run down, so the minimum is the stable estimator
and removes the CI flake risk a one-shot measurement carried
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
intent(security): the third review round showed the previous commit traded one
defect for three — length caps created false negatives, and both the trailing
punctuation restore and its regex were themselves broken
decision(userinfo-match): express the userinfo as a repetition of `<chunk>@`
instead of a greedy `[^/\s]*@`. Both find the same last '@', but the repetition
is unambiguous — one '@' per iteration — so it is linear without needing a length
cap, and a 1000-character JWT credential is redacted again
rejected(length-caps): capping the userinfo at 256 characters silently returned
long credentials verbatim; JWT-style tokens routinely exceed that
rejected(punctuation-restore): re-appending trailing punctuation after the
placeholder leaked the tail of values like `token=secret:...`, restored a
punctuation-only credential in full, and was itself quadratic. The closing quote
of a log line is now absorbed instead — losing a character of the message is the
correct side of that trade
constraint(host-lookahead): the scp-style lookahead stays bounded at 256, which
is above the 253-character hostname limit in RFC 1035, because it re-runs on
every backtrack and is the one remaining quadratic risk
learned(perf-testing): a single absolute timing threshold does not distinguish
linear from quadratic; the test now compares two input sizes and asserts the
growth ratio
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
intent(security): a second review round found that the previous commit's fix for
over-redaction had introduced a worse problem, plus a denial-of-service vector
decision(query-values): revert to consuming the whole parameter value and restore
any trailing punctuation afterwards, instead of treating it as a terminator —
',', '(' and quotes are legal unencoded in a query value, so stopping at them
left most of a credential in the clear to protect a cosmetic detail
constraint(regex-bounds): the userinfo patterns backtrack across every '@' in a
token, so unbounded quantifiers plus the host lookahead are quadratic; a 64KB
input took ~1s and MCP's `remote` argument is untrusted and unbounded, giving an
event-loop stall. Bounded to 256 chars, which is far above any real userinfo and
brings the same input to ~0.2ms
learned(tradeoffs): the earlier choice traded a security property for diagnostic
readability, which is the wrong direction — the readability fix now happens after
redaction rather than by narrowing what gets redacted
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
intent(security): a Codex review of the first commit surfaced seven more paths
where a credentialed URL survived, three of them outside the logging sites the
advisory named
decision(skill-generation): redact skillSourceUrl at its source in remoteAction —
it is only rendered as a link in the generated SKILL.md and never used to reach
the network, so leaving it raw persisted the credential into a file the user is
likely to commit, which outlives any log
constraint(mcp-prompts): packRemoteRepositoryPrompts echoes the repository into a
literal `repository: "..."` tool-call instruction, so redacting there would break
the call it is telling the model to make; only the tool *result* is redacted
decision(scheme-detection): left isExplicitRemoteUrl case-sensitive and redacted
the "Target path does not exist" message instead — making URL detection
case-insensitive is a behavior change that belongs in its own PR
learned(userinfo-regex): terminating the userinfo search at '?' and '#' let a
password containing either survive untouched; only '/' and whitespace may end it,
and the resulting over-redaction of a path-less URL with '@' in its query is the
safer failure
learned(scp-regex): stopping at the first '@' left the tail of a password that
itself contains '@'; matching to the last '@' plus a `host:path` lookahead fixes
that and also stops `failed at 12:30@example.com` from being mangled
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
intent(security): GHSA-w8cw-mgw9-74h7 — remote URLs carrying a user:password or
token reached console output, trace logs, and error messages verbatim (CWE-532),
persisting the credential in terminal scrollback and CI build logs
decision(url-redaction): centralize redaction in shared/urlRedact.ts and apply it
at every logger and error-formatting site, instead of extending the previous
inline regex — that one lived in validateGitUrl's catch block, so it only ran for
malformed URLs and never for the valid credentialed URLs that actually matter
decision(ssh-remotes): leave `git@host:owner/repo` untouched; the username there
is a fixed literal rather than a secret, and redacting it would make ordinary SSH
remotes unreadable in logs for no security gain
learned(git-errors): Node's execFile puts the full command line into
`error.message`, so a failed clone or ls-remote leaks the credential through the
error path as well — not just through the deliberate log statements
learned(cli-trace): `--verbose` dumps the whole options object and the positional
arguments, which leaks `remote` no matter how carefully the individual call sites
redact, so those dumps are redacted too
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Declining the trust prompt raises OperationCancelledError, which handleError
rendered like any other failure: a "✖ …" banner, the --verbose hint, and a
"file an issue / join our Discord" footer. The prompt has already said what was
cancelled, so all of that framed the user's own answer as a bug in Repomix.
Return early for cancellations instead. The exit code stays 1 because the
requested pack did not happen and a wrapper script must not read that as
success. The same path covers the existing skill-location cancel, which had
the identical problem; this PR just puts it in front of many more people.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
intent(tmp-dir): a single grep should find every consumer of the `$TMPDIR/repomix/` umbrella; previously the path was rebuilt independently in mcpToolRuntime and tokenCountCache, with no shared anchor preventing future drift
decision(api-shape): expose only `getRepomixTmpDir()` + the `REPOMIX_TMP_DIR_NAME` constant — callers append their own subdirectory (`mcp-outputs`, `cache`) and own their mkdir/mkdtemp/permissions, since each consumer's lifecycle and creation pattern differs
rejected(scope): folding `--remote`'s `mkdtemp(repomix-)` into the umbrella was considered but skipped — it uses a different prefix-based scheme at the $TMPDIR root, not the shared umbrella, and changing it is a user-visible path change worth handling separately
- validateFileSafety: pin the negative path of `if (config.security.enableSecurityCheck)`
— every other test enabled the check, so a regression that always runs
the security check would have passed silently.
- unifiedWorker:
- Add a positive workerData=securityCheck + ambiguous-task case so the
pair (override + this) distinguishes "inference always wins" from
"inference wins only when it yields a value".
- Stop pretending the handler-cache test verifies caching. Both branches
of `if (cached) return cached;` end with the same Map.set, and Node's
own module cache makes the dynamic import effectively free, so the
cache is unobservable from outside without exposing internals.
Renamed to "repeated calls" with a comment explaining the limitation.
- fileSystemReadDirectoryTool: translate the pre-existing Japanese comment
to English per CLAUDE.md.
- TokenCounter: extract `LoadEncodingFn` type alias instead of the
unusual `typeof loadEncoding`, so a signature drift between the local
function and the deps field would surface at the type level.
- shared/errorHandle: recognize duck-typed OperationCancelledError from
worker boundaries in isRepomixError (it extends RepomixError but the
name was missing from the structured-clone fallback comparison).
Add a regression test for the worker-boundary case.
Test improvements per coderabbit / claude review:
- cliReport: assert skill-directory + relative path on the same log line.
- processConcurrency: restore process.versions.bun by removing the property
when it didn't originally exist, instead of leaving it defined-as-undefined.
- logger: drop the no-op `process.env.REPOMIX_LOG_LEVEL = undefined` (it
coerces to the string "undefined" and is overwritten by the next delete).
- unifiedWorker: replace the tautological cache test with one that proves
cache uniqueness via onWorkerTermination cleanup count; add a test for
task-based inference overriding workerData (bundled-env reuse).
- calculateMetricsWorker: new direct test for the default export's items
vs. single-mode dispatch — unifiedWorker mocks this module so the branch
was otherwise untested.
- packRemoteRepositoryTool: hard-code the expected output path instead of
expect.any(String) to catch arg-swap regressions.
- memoryUtils: tighten getMemoryStats assertions with sanity bounds
(heapUsed <= heapTotal, rss > 0, heapUsagePercent <= 100) so a
unit-conversion regression (bytes vs MB) would fail the test.
intent(asyncMap): tighten doc and test based on coderabbitai review feedback
decision(asyncMap-doc): explicitly note that workers are not cooperatively cancelled after a rejection — sibling workers keep claiming indices
decision(asyncMap-test): replace timing-sensitive `peakActive > 1` with exact `=== 4` — workers spawn synchronously via Promise.all so the cap is hit deterministically
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
intent(empty-dir-check): protect very large repos from FD exhaustion that unbounded Promise.all could trigger
rejected(p-limit): user wants to keep dependencies minimal — built a small in-tree helper instead
decision(asyncMap): single mapWithConcurrency helper rather than a p-limit-style limiter object — only call site is array map
decision(concurrency-limit): 20 in flight — well above libuv default thread pool (4) while still bounded for users who tuned UV_THREADPOOL_SIZE
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
intent(readability): an issue with no `path` (e.g., root-level schema mismatch) previously rendered as `[] message`; emit just `message` when segments are empty. Small quality-of-life for error output
intent(limitation-pin): add an integration test that documents the known ESM unwrap ambiguity — a CJS module shaped like `{ default: {...}, otherKey: ... }` has `otherKey` silently dropped by our heuristic. Non-issue for RepomixConfig today, but worth freezing so the behavior can't drift without someone noticing
intent(interop-consolidation): drop `interopDefault: true` from the jiti setup in configLoad — the explicit ESM namespace unwrap at the call site already handles every module-format case we test (.ts / .mts / .js / .mjs / .cjs). Having both the jiti flag and the manual unwrap was redundant and made the intent fuzzier
intent(error-path-cleanliness): filter out empty path segments before joining in rethrowValidationErrorIfSchemaError — a malformed ValiError item (object without `key`) would otherwise produce `[output..style]`; dropping the empty entry keeps the path readable. Added a dedicated test covering the filter
intent(error-handle): drop the instanceof Error guard in rethrowValidationErrorIfSchemaError so ValiError / ZodError round-tripped through a worker (plain { name, message, issues }) is still recognized — aligns with isError / isRepomixError elsewhere in the file
intent(schema-parity): restore splitOutput's upper bound (Number.MAX_SAFE_INTEGER) in the generated JSON schema so editor hints match the previous zod output; also strip the empty required:[] arrays that @valibot/to-json-schema emits on every object node
intent(esm-unwrap): only unwrap jiti's .default when it's an object, preserving a CJS config that legitimately exports { default: 'plain', ...rest }; plain Symbol.toStringTag === 'Module' was too narrow — jiti returns non-Module namespace wrappers for .ts / .mts files
intent(test-coverage): add tests/shared/errorHandle.test.ts covering the Zod + Valibot + worker-serialized paths through rethrowValidationErrorIfSchemaError; tighten the default-schema assertion in configSchema.test.ts from /expected|invalid/i to toThrow(v.ValiError) + targeted message pattern
decision(path-segment-fallback): return '' for unknown object-shaped path items instead of falling into String(segment) → "[object Object]"; defensive, non-breaking
Add maxWorkerThreads option to WorkerOptions for explicit thread count
capping, then use it to reduce CPU contention when metrics and security
worker pools run concurrently during the pipeline overlap phase.
- Metrics pool: capped at (processConcurrency - 1)
- Security pool: capped at floor(processConcurrency / 2)
On a 4-core machine this reduces concurrent threads from 8 (4+4) to 5
(3+2), avoiding context-switching overhead during gpt-tokenizer warmup.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
File collection was replaced with a promise pool approach in 96ff05dc,
but the worker-related code remained. This removes the now-unused
fileCollectWorker and all references to it from the worker system.
Vitest v4 changed how vi.fn() and vi.mock() work with class constructors.
Arrow functions in mockImplementation no longer work as constructors
when called with 'new' keyword.
Changes:
- Use regular function syntax instead of arrow functions for constructor mocks
- Use vi.hoisted() to define class mocks that can be used in vi.mock() factories
- Replace vi.fn().mockReturnValue() with vi.fn().mockImplementation() for class mocks
- Update mock instance retrieval to use vi.mocked().mock.results[0].value
- Add comprehensive tests for unifiedWorker.ts covering task inference
and worker termination cleanup
- Unify onWorkerTermination to async signature across all worker files
for consistency (fileCollect, securityCheck, calculateMetrics)
Remove code that was added for debugging during development:
- Remove unused isTinypoolWorker function from unifiedWorker.ts
- Remove REPOMIX_DEBUG_WORKER logging from unifiedWorker.ts
- Remove debug logging from defaultActionWorker.ts
- Remove unused getUnifiedWorkerPath export
- Update tests to use workerType instead of workerPath
Add a unified worker entry point that enables full bundling support by
allowing bundled files to spawn workers using themselves. This is a
prerequisite for bundling the website server to improve Cloud Run cold
start times.
Changes:
- Add src/shared/unifiedWorker.ts as single entry point for all workers
- Support both worker_threads and child_process runtimes
- Add REPOMIX_WORKER_TYPE env var for child_process worker type detection
- Add REPOMIX_WORKER_PATH env var for bundled environment worker path
- Add REPOMIX_WASM_DIR env var for WASM file location override
- Update processConcurrency.ts to use unified worker path
- Add debug logging (REPOMIX_DEBUG_WORKER=1) for worker troubleshooting
- Export unified worker handler from main index.ts
Note: This is work in progress. There's a known issue with child_process
runtime where nested worker pools (created inside a worker) may receive
incorrect REPOMIX_WORKER_TYPE environment variable, causing task routing
issues. Investigation ongoing.
- Add test for sizeParse overflow case
- Use RepomixProgressCallback type in outputSplit.ts for consistency
- Improve configuration.md description for splitOutput option
Adds a size-based output splitter via --split-output (kb/mb) and writes numbered parts without splitting within a top-level folder.
Also updates metrics aggregation for multi-part output and adds unit tests.
- Use class names for RepomixError type checking instead of hardcoded strings
- Remove unused RepomixError import from fileProcess.ts
- Simplify comments in errorHandle.ts and fileProcess.ts
- Clean up constructor-based error checking logic
Adjust worker runtime configuration to use child_process for all potentially risky operations, prioritizing stability and isolation over performance.
- Change token-related workers to child_process for better memory isolation:
- calculateGitDiffMetrics: child_process (was worker_threads)
- calculateGitLogMetrics: child_process (was worker_threads)
- calculateOutputMetrics: child_process (was worker_threads)
- calculateSelectiveFileMetrics: child_process (was worker_threads)
- Keep file collection and globby operations as worker_threads (lower risk)
- Remove redundant memory leak risk comments for cleaner code
- Fix test cases to include required runtime parameter and teardown property
- Reorder imports in languageParser.ts for consistency
This conservative approach ensures maximum stability by isolating all token counting operations in separate processes, preventing potential memory leaks from affecting the main process.
- Add WorkerOptions interface to combine numOfTasks, workerPath, and optional runtime
- Update createWorkerPool and initTaskRunner functions to accept WorkerOptions object
- Refactor all usage sites across file processing, metrics, and security modules
- Update corresponding test cases to use new interface
This improves type safety and makes the API more maintainable by avoiding parameter order mistakes.
Add WorkerRuntime type and configurable runtime parameter to createWorkerPool and initTaskRunner functions. This allows choosing between 'worker_threads' and 'child_process' runtimes based on performance requirements.
- Add WorkerRuntime type definition for type safety
- Add optional runtime parameter to createWorkerPool with child_process default
- Add optional runtime parameter to initTaskRunner with child_process default
- Configure fileCollectWorker to use worker_threads for better performance
- Update all test files to use WorkerRuntime type
- Add comprehensive tests for runtime parameter functionality
- Maintain backward compatibility with existing code
The fileCollectWorker now benefits from worker_threads faster startup and shared memory, while other workers continue using child_process for stability.
- Extract git diff token calculation into separate worker and dedicated module
- Parallelize git diff metrics calculation with other metrics computations using Promise.all
- Isolate TokenCounter usage for git diffs within child process worker to prevent memory leaks
- Add comprehensive worker cleanup with exit handler for proper resource management
- Update tests to reflect new worker-based architecture and remove direct TokenCounter mocking
Memory improvements:
- Git diff token calculation now runs in isolated child process
- Enables parallel execution of all three metrics calculations (files, output, git diff)
- Further reduces main process memory footprint by isolating heavy TokenCounter operations
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add generic initTaskRunner function to processConcurrency.ts to eliminate
duplicate initialization logic across multiple modules. This reduces code
duplication and provides consistent worker pool management with proper
type safety through generic parameters.
- Add TaskRunner<T, R> interface and initTaskRunner function
- Remove duplicate createTaskRunner wrappers from 5 modules
- Update all deps parameters to use shared initTaskRunner directly
- Maintain type safety with explicit generic type parameters
- Update corresponding test mocks to match new signature
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Replace the environment variable approach for passing log levels to workers with Tinypool's workerData mechanism, which is more idiomatic for worker thread configuration.
Changes:
- Add setLogLevelByWorkerData() method to handle workerData-based log level setting
- Update Tinypool configuration to use workerData instead of env variables
- Update all 5 worker files to use setLogLevelByWorkerData()
- Remove unused setLogLevelByEnv function and related test mocks
- Update tests to reflect new workerData configuration
This provides better isolation and follows Node.js worker thread best practices.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Set TASKS_PER_THREAD to 100 for better balance between performance and resource usage
- Add comment explaining that worker initialization is expensive
- Update tests to match new thread allocation logic
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Updated all references throughout the codebase:
- Import statements in 5 core modules
- Function calls in file processing, metrics, and security modules
- Test mocks and descriptions
- Maintained backward compatibility and functionality
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Replace Piscina with Tinypool to significantly reduce bundle size (800KB → 38KB) while maintaining full API compatibility and performance. This migration affects all worker thread pools used in file processing, security checks, and metrics calculations.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Kazuki Yamada