mirror of
https://github.com/civitai/civitai.git
synced 2026-09-20 22:08:18 +08:00
fix(bot-detection): stop asset-staging's top rung outscoring its strongest rung (#4954)
* fix(bot-detection): stop asset-staging's top rung outscoring its strongest rung
The `asset-staging` volume ramp ran (zeroAt 1, oneAt 3), scoring a staged
count of 1 at 0, 2 at 0.5 and 3-or-more at 1.0. That ordering is backwards
against what the heuristic is for.
A rising ramp puts its top rung on the largest counts, and on this predicate
the largest counts are the wrong population: a coordinated upload clusters at
exactly TWO assets, an avatar and a header, which is what a profile needs and
no more, while a legitimate profile setup -- a business, or a creator bringing
in a kit -- runs to THREE OR MORE. So the rung the heuristic weighted highest
was the rung carrying disproportionately many legitimate accounts, while the
shape it exists to find sat at half of what it selected.
Move STAGED_ONE_AT from 3 to 2 so the ramp saturates at the firing point:
count 1 scores 0, count 2 and above score 1. A plateau, not a suppression --
the 3+ arm still scores its maximum and still reaches a moderator on its own.
It is the worse of the two arms, not an empty one, so the requirement is that
it stop OUTSCORING a pair, never that it stop scoring. rampScore is monotone
non-decreasing by construction and throws on oneAt <= zeroAt, so a declining
shape is not expressible in the shared helper without a second ramp term; of
the shapes that are expressible, the plateau is one constant.
Costs, recorded in the code rather than left to be discovered:
- The volume half now has no gradient -- it is a step -- so the sub-score
cannot express "more staged than that". The count itself is still disclosed
verbatim to a moderator by explain().
- The two rungs are no longer distinguishable in any counter a run emits, so
a future re-shape has to be graded against moderation outcomes rather than
read off the shadow-phase counters.
- Two series move on deploy with no account behaving differently: a lone
asset-staging blend goes 0.125 -> 0.25 (one confidence bucket for a
lone-signal account), and asset-staging's sole_signal inflates because the
dominance test's runner-up tolerance scales with the leader's score.
The reported population is unchanged at the shipped cut.
Tests: adds "THE ORDERING", which pins score(2) >= score(3) as a comparison
rather than as two literals, and also pins that the 3+ arm keeps scoring and
stays independently reportable -- so the ordering cannot be satisfied by
suppressing it. Watched red against the unmodified ramp
("expected 0.5 to be greater than or equal to 1") and green after.
Existing expectations that moved are the arithmetic consequence of the new
boundary. One case, "scores 0 for ONE staged upload and fires from two", is
deleted: its assertions had become a subset of the saturation case's, so no
mutant could separate them.
Comment changes in ramp.ts, run.ts, scoring.ts and the test files correct
statements this change falsified -- chiefly two that said the volume and burst
halves share boundaries, and one giving a degenerate step as the reason for
rampScore's throw, which the new adjacent-integer pair makes false.
MIN_REPORTED_CONFIDENCE, LONE_SIGNAL_CUT, the heuristic registry and the burst
boundaries are untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(tests): qualify two false absolutes about which boundary pairs rampScore clamps
Review round 8 found one finding, in prose only; no assertion, fixture or
expected value changes, and the file's code is byte-identical (verified by
stripping comments with the TypeScript parser).
The asset-staging loop-vacuity note said `burst <= 1 === volume` holds for
"every boundary mutation, since rampScore clamps". It does not. rampScore
guards `!(oneAt > zeroAt)` and THROWS on a degenerate pair, so such a pair
produces no output in [0, 1] at all and the loop goes red rather than holding
-- measured, `BURST_ONE_AT -> 1` with its pin updated turns 83 of 398 cases
red on that error. The hypothesis the sentence rests on ("output stays in
[0, 1]") and both of its named counter-examples were already correct; only
the appositive was too wide.
That matters more than a nit because the same file retracts this exact class
by name two hundred lines earlier, where the same edit is described as
throwing and taking ~80 cases with it. The two paragraphs disagreed, and the
newer one was the wider.
Second instance of the same shape, same file: the ORDERING case said its
comparisons hold for "EVERY boundary pair with oneAt <= 2" -- a pair with
zeroAt >= oneAt satisfies that quantifier and throws.
Both errors ran in the safe direction: they overstated how vacuous the
assertions are, i.e. understated coverage, so neither gave false confidence
in a guard.
Also corrects a positional reference that the round-7 reorder invalidated --
`spread.burst` is no longer "at the foot" of its case, it is the penultimate
block, since the count-1 control was deliberately moved below it.
Suite 398/398, ESLint and Prettier clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Zachary Lowden
@@ -225,9 +225,15 @@ describe('stagedImageSampleArgs', () => {
|
||||
// 🔴 THE PER-MEMBER CAP IS ALSO THE LARGEST COUNT THE HEURISTIC CAN EVER SEE, which the
|
||||
// filename cap is not — a filename sample folds into a set, while this one IS the measurement.
|
||||
// That is harmless only while the scoring ramp saturates well below it, so the relationship is
|
||||
// asserted rather than left to a reader to notice. The margin WIDENED when the volume boundary
|
||||
// was re-derived downwards, so the assertion is written against the constant rather than
|
||||
// restating its value — a literal here goes stale exactly when the coupling it guards moves.
|
||||
// asserted rather than left to a reader to notice, and it is written against the constant rather
|
||||
// than restating its value — a literal here goes stale exactly when the coupling it guards moves.
|
||||
//
|
||||
// 🔴 READ THE LAST LINE AS 4×, NOT AS THE MARGIN. The actual margin is 25× (50 against a volume
|
||||
// boundary of 2) and prose elsewhere calls it "an order of magnitude"; this assertion only
|
||||
// requires 4×, so it is much weaker than the sentences that cite it. It got WEAKER, not
|
||||
// stronger, when the volume boundary moved down — the bar it enforces fell from 12 to 8 against
|
||||
// an unchanged 50. It is kept at 4× deliberately: the point is to catch a per-member cap
|
||||
// dropped near the ramp, not to re-pin the margin, which the two literals above already do.
|
||||
expect(MAX_STAGED_IMAGE_SAMPLES).toBe(20_000);
|
||||
expect(MAX_STAGED_IMAGES_PER_MEMBER).toBe(50);
|
||||
expect(MAX_STAGED_IMAGES_PER_MEMBER).toBeGreaterThan(STAGED_ONE_AT * 4);
|
||||
|
||||
@@ -665,32 +665,113 @@ describe('asset-staging', () => {
|
||||
// boundary case written as `{ count: STAGED_ZERO_AT }` is vacuous about the constant under
|
||||
// test — measured on this module: a mutant moving `CLUSTER_ZERO_AT` survived exactly that.
|
||||
expect(STAGED_ZERO_AT).toBe(1);
|
||||
expect(STAGED_ONE_AT).toBe(3);
|
||||
expect(STAGED_ONE_AT).toBe(2);
|
||||
expect(BURST_ZERO_AT).toBe(1);
|
||||
expect(BURST_ONE_AT).toBe(3);
|
||||
// 🔴 THE TWO PAIRS ARE NOW EQUAL, AND THAT IS A BLIND SPOT THIS CASE CANNOT COVER: a mutant
|
||||
// that swaps the volume boundaries for the burst ones changes nothing observable. What covers
|
||||
// the burst arm instead is the subset pin below — it is the case that goes red if the burst
|
||||
// pair ever moves BELOW the volume pair, which is the only direction in which this arm can
|
||||
// start affecting the score again.
|
||||
expect([BURST_ZERO_AT, BURST_ONE_AT]).toEqual([STAGED_ZERO_AT, STAGED_ONE_AT]);
|
||||
});
|
||||
|
||||
it('scores 0 for ONE staged upload and fires from two', () => {
|
||||
// One unattached, metadata-free upload is the commonest shape on the site that matches this
|
||||
// predicate at all — somebody started a post and did not finish. Scoring it would fire on a
|
||||
// large share of every day's genuine signups.
|
||||
it('🔴 the volume boundaries stay no wider than the burst ones, so the burst arm cannot lead', () => {
|
||||
// 🔴 ITS OWN CASE, NOT A TAIL ON THE ONE ABOVE, AND THAT IS THE WHOLE REASON IT IS HERE. These
|
||||
// two lines were first written at the end of the CONSTANTS case, directly after four literal
|
||||
// `toBe` pins on the same four constants. Vitest aborts an `it` at the first failed `expect`, so
|
||||
// behind those literals they were UNREACHABLE: every mutant they claimed to catch died two lines
|
||||
// earlier, and a relational assertion that can never be the failing line is documentation
|
||||
// wearing a guard's clothes. Split out, they can fail on their own and the claim below is true.
|
||||
//
|
||||
// LITERAL counts and a LITERAL expected value. 2 staged is (2-1)/(3-1) = 0.5. That it IS a
|
||||
// round half now is a consequence of where the derived boundary landed, not a convenience, so
|
||||
// the mutants it separates are named rather than assumed: dropping the `- zeroAt` gives
|
||||
// 2/(3-1) = 1; dividing by `oneAt` gives 1/3; `zeroAt` 1→0 gives 2/3; `oneAt` 3→4 gives 1/3;
|
||||
// `oneAt` 3→2 saturates to 1. All five differ from 0.5, so none of them survives this case.
|
||||
// WHAT THEY GUARD, stated no wider than it is: the volume ramp must be at or above the burst
|
||||
// ramp at every input — same or lower `zeroAt`, same or lower `oneAt`. Why that relationship is
|
||||
// what makes `max(volume, burst)` identically `volume`, and which half of it is actually load-
|
||||
// bearing at the shipped constants, is on `BURST_ONE_AT`. The literal pins above catch a BLIND
|
||||
// mutant of either constant.
|
||||
//
|
||||
// 🔴 THIS CASE IS NEVER THE SOLE FAILURE, AND CLAIMING IT CATCHES A RE-TUNE "THE LITERALS
|
||||
// CANNOT" WOULD OVERSELL IT. Measured: a dominance-breaking re-tune with its literal pin updated
|
||||
// turns ~10 other cases red too, because every such edit also moves a ramp and the behavioural
|
||||
// cases see it. Its unique value is one step further out — the author who re-tunes, updates the
|
||||
// literal pin AND updates the behavioural expectations, where this relation is the last thing
|
||||
// standing. That is the edit this PR itself performed on the volume side, which is why it is
|
||||
// worth four lines.
|
||||
//
|
||||
// 🔴 THE TWO LINES GUARD OPPOSITE SIDES, AND SAYING "BOTH CATCH A BURST RE-TUNE" WOULD BE ONE
|
||||
// BOUNDARY TOO WIDE. Measured: the `zeroAt` line fires on a BURST re-tune — `BURST_ZERO_AT` to 0
|
||||
// with its pin updated gives `expected 1 to be less than or equal to 0` here. The `oneAt` line
|
||||
// is what catches a VOLUME widening — `STAGED_ONE_AT` to 4 with its pin updated gives
|
||||
// `expected 4 to be less than or equal to 3`.
|
||||
//
|
||||
// A burst re-tune CAN reach the `oneAt` line — `BURST_ONE_AT` to 1 with its pin updated fires it
|
||||
// with `expected 2 to be less than or equal to 1`, because this case reads constants and never
|
||||
// calls `rampScore`. It is simply not a viable edit: that pair throws inside `rampScore` and
|
||||
// takes ~80 other cases with it. An earlier wording here said the line was "not reachable by any
|
||||
// integer burst re-tune", which was a false absolute of exactly the kind this file has spent
|
||||
// several rounds removing — unreachable and unviable are different claims.
|
||||
expect(STAGED_ZERO_AT).toBeLessThanOrEqual(BURST_ZERO_AT);
|
||||
expect(STAGED_ONE_AT).toBeLessThanOrEqual(BURST_ONE_AT);
|
||||
});
|
||||
|
||||
it('🔴 THE ORDERING: three or more staged uploads never OUTSCORES exactly two', () => {
|
||||
// 🔴 THE PROPERTY THE RAMP GOT BACKWARDS, PINNED AS A COMPARISON RATHER THAN AS TWO VALUES.
|
||||
// WHY the old ordering was backwards — which population clusters where, and what the old
|
||||
// boundaries therefore weighted highest — is on `STAGED_ONE_AT` and is deliberately not
|
||||
// reproduced here. It was, for two rounds, and the two copies had already drifted apart on the
|
||||
// detail before anyone noticed; one argument, one place.
|
||||
//
|
||||
// 🔴 ASSERTED AS `>=`, DELIBERATELY, SO IT PINS THE ORDERING AND NOT ONE PARTICULAR CURE. A
|
||||
// plateau (3+ scoring the same as a pair) and a decline (3+ scoring less) both satisfy it, and
|
||||
// both are defensible shapes; what is NOT defensible is the direction, which is the thing that
|
||||
// was measured. A case asserting literal values here would go red on a later re-shaping that
|
||||
// kept the ordering intact, and would then be pinning an implementation rather than a finding.
|
||||
//
|
||||
// 🔴 NOT FULLY SHAPE-NEUTRAL, THOUGH, AND SAYING "IT IS" WOULD BE FALSE. The non-suppression
|
||||
// floor further down requires `at3 >= LONE_SIGNAL_CUT`, which rules out a decline to, say, 0.3 —
|
||||
// a shape that satisfies the ordering and keeps the arm scoring. That is deliberate rather than
|
||||
// an oversight: a 3+ arm scoring below the lone-signal cut is no longer independently reportable,
|
||||
// so an account carried by that arm alone leaves the board entirely. For an arm that still
|
||||
// carries genuine catches — it graded worse than the pair rung, not empty — that is suppression
|
||||
// by another route, and
|
||||
// the floor is where this case says so. A re-shape that genuinely intends it must move the floor
|
||||
// deliberately, which is the point of making it an assertion rather than an assumption.
|
||||
//
|
||||
// 🔴 AND READ THE `>=` COMPARISONS AS REGRESSION COVERAGE, NOT AS A STANDING INVARIANT. `at2` is
|
||||
// the ramp's ceiling at today's constants and `rampScore` clamps at 1, so `at2 >= at3` and the
|
||||
// loop below hold for every NON-DEGENERATE boundary pair with `oneAt <= 2` — a pair with
|
||||
// `zeroAt >= oneAt` satisfies that quantifier and throws instead, which is why the qualifier is
|
||||
// there. They were red at the pre-change
|
||||
// constants — which is what they exist to pin — but going forward the teeth are in the three
|
||||
// assertions after the loop, not in the comparisons. Do not "simplify" this case by deleting
|
||||
// them.
|
||||
const at2 = score(member(), stagedSignals(42, { count: 2, largestSameSecondBurst: 1 }));
|
||||
const at3 = score(member(), stagedSignals(42, { count: 3, largestSameSecondBurst: 1 }));
|
||||
expect(at2).toBeGreaterThanOrEqual(at3);
|
||||
|
||||
// The claim is about the whole arm, not about its first member: `3+` is one population and a
|
||||
// ramp that merely delayed its rise by a step would satisfy the line above while still putting
|
||||
// its top rung on the wrong accounts. Counts pairwise distinct, and distinct from every
|
||||
// constant this case names.
|
||||
//
|
||||
// 🔴 FOUR SAMPLES RATHER THAN ONE, AND UNDER MONOTONICITY THAT WOULD BE THREE TOO MANY — the
|
||||
// largest count would subsume the rest. They earn their place because this case deliberately
|
||||
// admits a NON-MONOTONE re-shape (that is what the `>=` above is for), and under a decline the
|
||||
// binding count need not be the largest.
|
||||
for (const count of [4, 7, 11, 40]) {
|
||||
expect(
|
||||
score(member(), stagedSignals(42, { count, largestSameSecondBurst: 1 }))
|
||||
).toBeLessThanOrEqual(at2);
|
||||
}
|
||||
|
||||
// 🔴 AND NOT SATISFIABLE BY SUPPRESSION, WHICH IS THE CHEAP WAY TO PASS THE LINES ABOVE AND IS
|
||||
// THE WRONG FIX. The 3+ arm is worse than the pair arm; it is not empty, and zeroing it would
|
||||
// discard the genuine catches it still carries. So the arm must keep scoring, and keep scoring
|
||||
// enough to reach a moderator ON ITS OWN — the same `s >= LONE_SIGNAL_CUT` test the firing
|
||||
// point below is derived from, asserted here on the arm that must not be thrown away.
|
||||
expect(at3).toBeGreaterThan(0);
|
||||
// ⚠️ This line cannot fail for any plausible value of `LONE_SIGNAL_CUT` — `at3` is 1 today, so
|
||||
// any cut at or below 1 satisfies it. It is not a guard on the cut; it fires only on a re-shape
|
||||
// that declines the 3+ arm below it, which is the thing it is here for.
|
||||
expect(at3).toBeGreaterThanOrEqual(LONE_SIGNAL_CUT);
|
||||
|
||||
// Nor by flattening the ramp into a constant: ONE staged upload is the commonest shape on the
|
||||
// site that matches this predicate at all, and it must still be worth nothing.
|
||||
expect(score(member(), stagedSignals(42, { count: 1, largestSameSecondBurst: 1 }))).toBe(0);
|
||||
expect(score(member(), stagedSignals(42, { count: 2, largestSameSecondBurst: 1 }))).toBeCloseTo(
|
||||
0.5,
|
||||
12
|
||||
);
|
||||
});
|
||||
|
||||
it('🔴 THE FIRING POINT: a LONE asset-staging signal is REPORTED at two staged uploads, not one', () => {
|
||||
@@ -702,15 +783,23 @@ describe('asset-staging', () => {
|
||||
// one of them moving silently breaks it. So this case runs the REAL registry through the REAL
|
||||
// blend and the REAL partition, and asserts the reported/suppressed verdict itself.
|
||||
//
|
||||
// The derivation it pins: a lone sub-score `s` blends to `s / n` and is compared against
|
||||
// `LONE_SIGNAL_CUT / n`, so the `n`s cancel and the account is reported exactly when
|
||||
// `s >= LONE_SIGNAL_CUT`. With `zeroAt = 1`, `1 / (oneAt - 1) >= 0.45` forces `oneAt <= 3.22…`,
|
||||
// i.e. 3. A pair scores 0.5 and clears; a single upload scores 0 and does not.
|
||||
// The property it pins: a lone signal is reported exactly when `s >= LONE_SIGNAL_CUT`. The
|
||||
// algebra behind that, and the bound it puts on `oneAt`, are on `STAGED_ONE_AT` — this segment
|
||||
// had to edit BOTH copies of that derivation to make one change, which is the argument for
|
||||
// keeping it in one place. Here: a pair saturates at 1 and clears the cut, a single upload
|
||||
// scores 0 and does not.
|
||||
//
|
||||
// The member is built so every OTHER heuristic scores 0 — a common mail provider, no shared
|
||||
// address, no templated text, 7 images over 11 hours (0.64/hour, far under the velocity floor).
|
||||
// The fixture numbers are pairwise distinct and distinct from every constant named below: 7
|
||||
// images, 1 and 2 staged, against 0.5, 0.45 and 0.1125.
|
||||
// 🔴 ONE FIXTURE VALUE DOES COLLIDE WITH AN EXPECTED ONE — the same-second burst is 1, which is
|
||||
// also what the pair's sub-score now saturates to, and a mutant returning the raw burst tally
|
||||
// would satisfy that line. It is separated by the OTHER verdict rather than by the fixture: this
|
||||
// fixture also pins the burst tally at 1 for a count of ONE, where the expected sub-score is 0,
|
||||
// so that mutant fails on `verdict(1)`. (A tally of 0 beside a count of 1 IS buildable — a row
|
||||
// whose timestamp will not parse increments the count and is skipped by the burst fold — it is
|
||||
// simply not what this fixture supplies.) The staged COUNT of 1 also equals the expected
|
||||
// sub-score of 1 in `verdict(2)`; same separation, same reason. The remaining numbers are
|
||||
// pairwise distinct: 7 images against 0.25, 0.45 and 0.1125.
|
||||
const loner = () =>
|
||||
member({
|
||||
all: { images: 7 },
|
||||
@@ -744,36 +833,112 @@ describe('asset-staging', () => {
|
||||
const pair = verdict(2);
|
||||
expect(pair.reported).toBe(1);
|
||||
expect(pair.others).toEqual([0, 0, 0]);
|
||||
expect(pair.staged).toBeCloseTo(0.5, 12);
|
||||
expect(pair.staged).toBe(1);
|
||||
expect(pair.staged as number).toBeGreaterThanOrEqual(LONE_SIGNAL_CUT);
|
||||
expect(pair.confidence).toBeCloseTo(0.125, 12);
|
||||
expect(pair.confidence).toBeCloseTo(0.25, 12);
|
||||
expect(pair.confidence).toBeGreaterThanOrEqual(MIN_REPORTED_CONFIDENCE);
|
||||
// 🔴 TWO OF THESE ASSERTIONS GOT SLACKER IN THE CHANGE THAT MOVED THE SUB-SCORE TO 1, AND THE
|
||||
// CASE'S OWN RATIONALE ABOVE ("any one of the four moving silently breaks it") IS WHY THAT IS
|
||||
// WORTH WRITING DOWN. The confidence is now 0.25, so the window of `MIN_REPORTED_CONFIDENCE`
|
||||
// values this case tolerates doubled from `(0, 0.125]` to `(0, 0.25]` — a mutant raising it to
|
||||
// 0.2 used to flip `reported` to 0 and now does not. And `pair.staged >= LONE_SIGNAL_CUT` is
|
||||
// satisfied by any cut at or below 1, so it can no longer fail for a plausible value. Both
|
||||
// constants are still pinned literally in `scoring.test.ts`, so neither mutant escapes the
|
||||
// suite; what changed is that THIS case stopped being the place they die.
|
||||
});
|
||||
|
||||
it('saturates at the volume boundary and stays there', () => {
|
||||
// Lands exactly ON the boundary, then overshoots it — 3 and 11 against a boundary of 3.
|
||||
it('starts at 0 below the volume boundary, saturates ON it, and stays there', () => {
|
||||
// Below the boundary, exactly ON it, one past it, then far past it — 1, 2, 3 and 11 against a
|
||||
// boundary of 2. The plateau is the shape the ORDERING case requires and this is its other half:
|
||||
// every count above the boundary scores the SAME as the boundary, rather than tailing off.
|
||||
//
|
||||
// 🔴 THE `count: 1` LINE IS WHAT MAKES THE TITLE TRUE, AND WITHOUT IT THE CASE IS BLIND. The
|
||||
// other three counts are all at or above `oneAt`, so all three of those assertions are the upper
|
||||
// clamp: a `score` that returned 1 for EVERY input would satisfy them, and the case could not
|
||||
// distinguish "saturates at 2" from "always 1". One count below the boundary separates them.
|
||||
//
|
||||
// LITERAL counts and LITERAL expected values. The mutants this separates, named rather than
|
||||
// assumed: `zeroAt` 1→0 puts a single upload at (1-0)/(2-0) = 0.5 instead of 0; `oneAt` 2→3 puts
|
||||
// a pair at 0.5 instead of 1, which is the inversion the ORDERING case above exists for;
|
||||
// `oneAt` 2→1 throws at `rampScore`'s own guard.
|
||||
//
|
||||
// (This case absorbed a separate `scores 0 for ONE staged upload and fires from two`, whose two
|
||||
// assertions had become a subset of the first two here — so no mutant could separate them, and
|
||||
// counting it as a second guard was counting the same coverage twice. Its rationale is not
|
||||
// reproduced here: the "commonest shape on the site" argument lives on `STAGED_ZERO_AT` and in
|
||||
// the ORDERING case, and the boundary derivation is on `STAGED_ONE_AT`. A below-boundary control
|
||||
// still appears in several other
|
||||
// cases in this file and in `run.test.ts`, each as that case's own negative control — moving
|
||||
// `STAGED_ZERO_AT` turns all of them red, deliberately. No count is given here: a ledger of them
|
||||
// went stale inside the change that wrote it.)
|
||||
//
|
||||
// 🔴 WHAT THE VOLUME HALF CANNOT SEE, SAID PLAINLY RATHER THAN LEFT TO BE ASSUMED FROM THE
|
||||
// GREEN: with `zeroAt = 1` and `oneAt = 2` there is NO INTEGER strictly between the two
|
||||
// boundaries, so no count reaches `rampScore`'s interpolation line THROUGH THIS CALL SITE —
|
||||
// both ends are clamps. A mutant of that arithmetic (dropping the `- zeroAt`, dividing by
|
||||
// `oneAt` rather than by the span) is therefore invisible through `score`. It is NOT invisible
|
||||
// through this heuristic: the BURST call site runs at (1, 3), a tally of 2 lands strictly
|
||||
// between, and the two assertions that read `assetStagingHalfScores(...).burst` at 0.5 go red on
|
||||
// exactly those mutants. The other three heuristics cover it as well.
|
||||
expect(score(member(), stagedSignals(42, { count: 1, largestSameSecondBurst: 1 }))).toBe(0);
|
||||
expect(score(member(), stagedSignals(42, { count: 2, largestSameSecondBurst: 1 }))).toBe(1);
|
||||
expect(score(member(), stagedSignals(42, { count: 3, largestSameSecondBurst: 1 }))).toBe(1);
|
||||
expect(score(member(), stagedSignals(42, { count: 11, largestSameSecondBurst: 1 }))).toBe(1);
|
||||
});
|
||||
|
||||
it('🔴 the burst half NEVER exceeds the volume half at today’s boundaries', () => {
|
||||
// 🔴 THE HONEST REPLACEMENT FOR A CASE THAT USED TO ASSERT THE OPPOSITE. Until the firing point
|
||||
// moved to two, the burst boundaries sat tighter than the volume ones (4 against 8) and a
|
||||
// same-second pair genuinely scored HIGHER than the same count spread out; a case here asserted
|
||||
// exactly that. Both pairs are now (1, 3), and a same-second group is a SUBSET of the staged
|
||||
// rows, so `largestSameSecondBurst <= count` always and a monotonic ramp over identical
|
||||
// boundaries cannot turn the smaller input into the larger score. `max(volume, burst)` is
|
||||
it('🔴 the SCORE is identically the volume half — the burst arm cannot raise it', () => {
|
||||
// 🔴 THE HONEST REPLACEMENT FOR A CASE THAT USED TO ASSERT THE OPPOSITE. Long ago the burst
|
||||
// boundaries sat tighter than the volume ones (4 against 8) and a same-second pair genuinely
|
||||
// scored HIGHER than the same count spread out; a case here asserted exactly that. A
|
||||
// same-second group is a SUBSET of the staged rows, so `largestSameSecondBurst <= count`
|
||||
// always, and the volume ramp (1, 2) is at or above the burst ramp (1, 3) at every input, so a
|
||||
// monotonic ramp cannot turn the smaller input into the larger score. `max(volume, burst)` is
|
||||
// therefore identically `volume`: the burst arm changes neither whether this heuristic fires
|
||||
// nor how high it scores. Asserting a comparison it can no longer satisfy would be a guard
|
||||
// describing behaviour the code does not have.
|
||||
//
|
||||
// This IS regression coverage for the boundary change and not only a forward-looking guard:
|
||||
// measured red against the pre-change constants with `expected 0.3333… to be less than or
|
||||
// equal to 0.1428…`, i.e. the (2, 2) row, where the old tighter burst pair genuinely produced
|
||||
// the larger score. It doubles as the future guard — it goes red the moment `BURST_ONE_AT`
|
||||
// drops below `STAGED_ONE_AT` again, which is the only edit that can revive this arm, and it is
|
||||
// what keeps the `max` in `staging.ts` from being deleted as dead code without anyone noticing
|
||||
// the arm went with it.
|
||||
// 🔴 THE LOOP BELOW IS VACUOUS ABOUT THE BURST RAMP — NOT MERELY ABOUT ITS BOUNDARIES — AND THE
|
||||
// TITLE WAS RENAMED BECAUSE OF IT. Every row of the table has `count >= 2`, so `volume` is 1 for
|
||||
// all five, and `rampScore` clamps at 1: `burst <= 1 === volume` therefore holds for any burst
|
||||
// implementation whose output STAYS IN [0, 1] — which is every boundary mutation `rampScore`
|
||||
// ACCEPTS, since it clamps those — including one that returned a constant 1. (Not for literally
|
||||
// any implementation: a burst half returning 2, or returning the raw same-second tally unramped,
|
||||
// does fail that line. The clamp is the hypothesis.) 🔴 "ACCEPTS" IS LOAD-BEARING AND AN EARLIER
|
||||
// WORDING OMITTED IT, SAYING "every boundary mutation": a DEGENERATE pair does not produce an
|
||||
// output in [0, 1] at all, it THROWS (`rampScore` guards `!(oneAt > zeroAt)`), so the loop goes
|
||||
// red rather than holding. That is the same false-absolute class the case two hundred lines up
|
||||
// retracts by name, and this file would have held two paragraphs disagreeing about it. The error
|
||||
// ran in the safe direction — it overstated the loop's vacuity, i.e. understated coverage — but
|
||||
// it was still wrong. This case used to double
|
||||
// as the guard that went red if the burst pair were tightened below the volume pair; it cannot
|
||||
// any more, because the volume half is now a STEP at two and is already saturated wherever the
|
||||
// burst half is non-zero.
|
||||
//
|
||||
// WHAT ACTUALLY GUARDS THE BURST PAIR, so nobody deletes it believing this loop has them
|
||||
// covered: the four literal pins in the CONSTANTS case (which kill a blind mutant of either
|
||||
// constant), the separate dominance case beside them (which catches a deliberate re-tune that
|
||||
// updates those literals), and `spread.burst` in the penultimate block of THIS case — 2 in one second on an
|
||||
// account with 5 staged is `rampScore(2, 1, 3) = 0.5`, and both `BURST_ZERO_AT -> 0` (0.666…)
|
||||
// and `BURST_ONE_AT -> 2` (1) fail it. That last one is the only burst-pair guard inside this
|
||||
// case, and it is the reason the case is not merely the `x <= x` its loop has become.
|
||||
//
|
||||
// 🔴 READ THAT LIST AS THE DESIGNATED GUARDS, NOT AS THE COMPLETE SET — it understates, which is
|
||||
// the safe direction, but a reader deciding what is safe to delete needs to know. Dropping
|
||||
// `BURST_ZERO_AT` below `STAGED_ZERO_AT` turns far more cases red than the three named, most of
|
||||
// them ordinary behavioural ones; the measured count is recorded once, on the `max` in
|
||||
// `staging.ts`, rather than copied here where it would go stale separately.
|
||||
//
|
||||
// What the loop pins is that `score` is not `burst` (a burst-half mutant fails it) and not a
|
||||
// sum (1.5 against 1). It does NOT by itself pin "not a constant": every row has `count >= 2`,
|
||||
// so `halves.volume` is 1 throughout and `score(...) === halves.volume` reduces to `1 === 1` —
|
||||
// `score: () => 1` walks the loop untouched. The below-boundary line AFTER it is what closes
|
||||
// that, the same control the two neighbouring cases carry.
|
||||
//
|
||||
// 🔴 AND NONE OF IT KEEPS THE `max` FROM BEING REWRITTEN TO A BARE `volume`, WHICH THIS COMMENT
|
||||
// USED TO CLAIM: that rewrite satisfies `score === volume` BY CONSTRUCTION, and it is recorded
|
||||
// on the `max` in `staging.ts` as a measured survivor of the whole suite. Nothing in this file
|
||||
// catches it, and saying otherwise here would be a coverage claim about the one mutation this
|
||||
// module is measured to miss.
|
||||
//
|
||||
// Every pair respects `burst <= count`, because a pair that does not is a state the evidence
|
||||
// layer cannot build and proves nothing about the shipped code.
|
||||
@@ -790,14 +955,25 @@ describe('asset-staging', () => {
|
||||
expect(halves.burst).toBeLessThanOrEqual(halves.volume);
|
||||
expect(score(member(), s)).toBe(halves.volume);
|
||||
}
|
||||
// And the two values are genuinely different somewhere in that table, so the loop is not
|
||||
// asserting `x <= x` five times over.
|
||||
// 🔴 THE TWO VALUES ARE GENUINELY DIFFERENT HERE, AND THIS IS THE CASE'S BURST-PAIR ASSERTION.
|
||||
// The loop's FIRST line is `x <= x` five times over (see the note at the top); its second line
|
||||
// is a real assertion, and `spread.burst` is where a burst BOUNDARY is observable here.
|
||||
const spread = assetStagingHalfScores(
|
||||
42,
|
||||
stagedSignals(42, { count: 5, largestSameSecondBurst: 2 })
|
||||
);
|
||||
expect(spread.volume).toBe(1);
|
||||
expect(spread.burst).toBeCloseTo(0.5, 12);
|
||||
|
||||
// 🔴 LAST, AND THE ORDER IS LOAD-BEARING. This is the negative control the loop cannot supply —
|
||||
// every row it iterates is at or above the boundary, so `score: () => 1` walks all of them; one
|
||||
// count below the boundary is what closes that. It sits AFTER the spread block deliberately:
|
||||
// placed before it, this line is the first to fail under `BURST_ZERO_AT -> 0` (a count of 1
|
||||
// scores `rampScore(1, 0, 3)` = 0.333 on the burst half), vitest aborts the case there, and
|
||||
// `spread.burst` — the assertion the comment above credits with catching that mutant — never
|
||||
// executes. That is the same unreachability the dominance case was split out to remove, and it
|
||||
// was introduced here by an earlier round's fix before being measured.
|
||||
expect(score(member(), stagedSignals(42, { count: 1, largestSameSecondBurst: 1 }))).toBe(0);
|
||||
});
|
||||
|
||||
it('scores a burst of ONE as nothing — every upload shares its own second', () => {
|
||||
@@ -807,16 +983,21 @@ describe('asset-staging', () => {
|
||||
// stop distinguishing anything — which matters for the `fired_burst` counter and the moderator
|
||||
// clause even now that the half cannot move the score.
|
||||
//
|
||||
// 🔴 ASSERTED ON THE HALF, NOT ON THE BLEND, BECAUSE THE BLEND CANNOT SEE THIS MUTANT — and
|
||||
// that is now true by construction rather than by luck. `max` is identically `volume`, so NO
|
||||
// mutation of a burst constant is visible through `score` at all. Reading the half directly is
|
||||
// the only reachable assertion this arm has. (Measured before the boundaries met: moving
|
||||
// 🔴 ASSERTED ON THE HALF, NOT ON THE BLEND, BECAUSE THE BLEND CANNOT SEE THIS MUTANT AT THE
|
||||
// SHIPPED CONSTANTS. `max` is identically `volume` while the burst pair stays no steeper than
|
||||
// the volume pair, so a mutation of `BURST_ONE_AT` is invisible through `score`. It is NOT true
|
||||
// that no burst mutation is visible at all — dropping `BURST_ZERO_AT` below `STAGED_ZERO_AT`
|
||||
// breaks the dominance and does move the score — but that is the one BOUNDARY direction. Other
|
||||
// mutations of this half are visible through `score` too (replacing it with a constant 1 moves
|
||||
// 16 cases); the narrow claim is about its two boundaries, not about the half. Reading the half
|
||||
// directly is what makes this arm assertable without depending on any of that. (Measured before
|
||||
// two pairs met: moving
|
||||
// `BURST_ZERO_AT` to 0 made a burst of one score 0.25 while the volume half at a count of 3 was
|
||||
// already 0.2857, so the blended expectation passed at BOTH values of the constant.)
|
||||
const facts = stagedSignals(42, { count: 2, largestSameSecondBurst: 1 });
|
||||
expect(assetStagingHalfScores(42, facts).burst).toBe(0);
|
||||
// And the blend is then the volume half's (2-1)/(3-1) and nothing else.
|
||||
expect(score(member(), facts)).toBeCloseTo(0.5, 12);
|
||||
// And the blend is then the volume half saturating at its boundary of two, and nothing else.
|
||||
expect(score(member(), facts)).toBe(1);
|
||||
});
|
||||
|
||||
it('saturates the burst half at its own boundary', () => {
|
||||
@@ -827,18 +1008,28 @@ describe('asset-staging', () => {
|
||||
).toBe(1);
|
||||
});
|
||||
|
||||
it('🔴 combines the two halves with max, NOT a sum', () => {
|
||||
it('🔴 does NOT sum the two halves', () => {
|
||||
// A sum would double-count the same uploads — every burst member is also a count member — and
|
||||
// would make the sub-score stop meaning "how far past ordinary these uploads are". Still worth
|
||||
// pinning although `max` currently resolves to `volume`: the combination is what becomes wrong
|
||||
// first if the boundaries ever diverge again.
|
||||
// first if the volume half ever stops being a step.
|
||||
//
|
||||
// 2 staged in one second is 0.5 on BOTH halves, so max is 0.5 and a sum is 1.0 — a different
|
||||
// number from the operands and from the max, and below the clamp, so the sum mutant is visible
|
||||
// in this function's own return value rather than being flattened to 1 by `scoreAccount`.
|
||||
// 2 staged in one second saturates the volume half at 1 and puts the burst half at 0.5, so max
|
||||
// is 1 and a sum is 1.5 — above the clamp, so the sum mutant is visible in this function's own
|
||||
// return value before `scoreAccount` flattens it. A mutant returning the BURST half gives 0.5
|
||||
// and fails too.
|
||||
//
|
||||
// 🔴 THE TITLE SAYS "DOES NOT SUM" RATHER THAN "COMBINES WITH MAX" BECAUSE THE COMBINATION IS NO
|
||||
// LONGER OBSERVABLE HERE. At the previous boundaries both halves scored 0.5 on this fixture and
|
||||
// the expected 0.5 was a value neither `1` nor `0` could impersonate; now the expected value IS
|
||||
// the ceiling, so `score: () => 1` and `Math.max(volume, burst, 1)` both survive the FIRST line
|
||||
// below. The second — the `count: 1` control — is what kills them, measured: `return 1` fails it
|
||||
// with `expected 1 to be +0`. The mutant this case genuinely cannot see either way is one
|
||||
// returning the VOLUME half: semantically equivalent at these constants, and named as a known
|
||||
// survivor on the `max` in `staging.ts`.
|
||||
const both = stagedSignals(42, { count: 2, largestSameSecondBurst: 2 });
|
||||
expect(score(member(), both)).toBeCloseTo(0.5, 12);
|
||||
expect(score(member(), both)).not.toBeCloseTo(1.0, 6);
|
||||
expect(score(member(), both)).toBe(1);
|
||||
expect(score(member(), stagedSignals(42, { count: 1, largestSameSecondBurst: 1 }))).toBe(0);
|
||||
});
|
||||
|
||||
it('scores an account with nothing staged 0, without throwing', () => {
|
||||
@@ -869,13 +1060,14 @@ describe('asset-staging', () => {
|
||||
// 🔴 THE ASYMMETRY IS THE HONEST PART. "Volume without burst" is a real and common state. Its
|
||||
// mirror — a burst half firing on an account whose volume half did not — was asserted here
|
||||
// until the boundaries met, and it is now UNREACHABLE for any index the evidence layer can
|
||||
// build: `burst <= count` and the two ramps are identical, so `burst > 0` implies
|
||||
// `volume >= burst > 0`. Asserting the old direction would have required a fixture with more
|
||||
// same-second rows than staged rows, which is not a state that exists. So the reachable claim
|
||||
// is stated instead: the two halves are separately readable, one can be zero while the other is
|
||||
// not, and the impossible direction is named rather than faked with an invalid fixture.
|
||||
// build: `burst <= count` and the volume ramp is at or above the burst ramp everywhere, so
|
||||
// `burst > 0` implies `volume >= burst > 0`. Asserting the old direction would have required a
|
||||
// fixture with more same-second rows than staged rows, which is not a state that exists. So the
|
||||
// reachable claim is stated instead: the two halves are separately readable, one can be zero
|
||||
// while the other is not, and the impossible direction is named rather than faked with an
|
||||
// invalid fixture.
|
||||
const volumeOnly = signalsWith({ staged: { 42: { count: 2, largestSameSecondBurst: 1 } } });
|
||||
expect(assetStagingHalfScores(42, volumeOnly).volume).toBeCloseTo(0.5, 12);
|
||||
expect(assetStagingHalfScores(42, volumeOnly).volume).toBe(1);
|
||||
expect(assetStagingHalfScores(42, volumeOnly).burst).toBe(0);
|
||||
|
||||
// Both halves non-zero, read independently and NOT as two names for the max: 9 staged saturates
|
||||
@@ -884,6 +1076,14 @@ describe('asset-staging', () => {
|
||||
const both = signalsWith({ staged: { 42: { count: 9, largestSameSecondBurst: 2 } } });
|
||||
expect(assetStagingHalfScores(42, both).volume).toBe(1);
|
||||
expect(assetStagingHalfScores(42, both).burst).toBeCloseTo(0.5, 12);
|
||||
|
||||
// 🔴 AND A CONTROL ON `volume` ITSELF, WHICH THIS CASE LOST WHEN THE RAMP BECAME A STEP. Both
|
||||
// fixtures above now score 1 on the volume half — at the previous boundaries they were 0.5 and
|
||||
// 1, two distinct values, so a mutant returning a constant for `volume` died inside this case.
|
||||
// It no longer would. One fixture below the boundary restores the separation.
|
||||
const belowBoundary = signalsWith({ staged: { 42: { count: 1, largestSameSecondBurst: 1 } } });
|
||||
expect(assetStagingHalfScores(42, belowBoundary).volume).toBe(0);
|
||||
expect(assetStagingHalfScores(42, belowBoundary).burst).toBe(0);
|
||||
});
|
||||
|
||||
it('🔴 explains itself with the numbers it used, and states the account’s image total', () => {
|
||||
|
||||
@@ -575,32 +575,61 @@ describe('🔴 the seam between the evidence and the scoring', () => {
|
||||
]);
|
||||
|
||||
// 🔴 THE HEURISTIC THAT GOES INERT UNDER THE SEAM MUTANT, AT ITS FIRING POINT. TWO staged
|
||||
// uploads a minute apart is (2-1)/(3-1) = 0.5 on the volume half and nothing on the burst half.
|
||||
// The count is deliberately the SMALLEST that scores anything: a fixture further up the ramp
|
||||
// saturates to 1, and a saturated expectation cannot separate "the evidence arrived" from "the
|
||||
// heuristic returns its ceiling", which is the seam mutant this case exists for.
|
||||
// Read off the reason's `id=0.00` clause, so the expectation is the RENDERED two-decimal form.
|
||||
// Exact rather than approximate: the rendering is part of what a moderator sees, and a value
|
||||
// assertion with slack would pass on a heuristic scoring 0.5049.
|
||||
expect(sub['asset-staging']).toBe(0.5);
|
||||
// uploads a minute apart saturates the volume half at 1 and scores nothing on the burst half.
|
||||
// Read off the reason's `id=0.00` clause, so the expectation is the RENDERED two-decimal form —
|
||||
// which is what a moderator sees. ⚠️ Do not read the exact matcher as buying precision: the
|
||||
// sub-score is rendered with `toFixed(2)` before `subScoresOf` parses it back, so the renderer
|
||||
// sets the resolution and a heuristic scoring 0.999 renders `1.00` and passes this line. What
|
||||
// discriminates a near-miss is the confidence assertion below, on its own — measured, a
|
||||
// `score * 0.999` mutant reaches it and fails with `expected 0.24975 to be close to 0.25`. The
|
||||
// `count: 1` control at the foot of this case catches a constant CEILING, which is a different
|
||||
// mutant; it does not see a near-miss, and an earlier wording here credited it with both.
|
||||
//
|
||||
// 🔴 THIS EXPECTATION IS NOW THE HEURISTIC'S CEILING, AND THAT USED TO BE DELIBERATELY AVOIDED
|
||||
// HERE. While the volume ramp rose to a boundary of 3, two uploads landed mid-ramp at 0.5, and
|
||||
// a mid-ramp value separates "the evidence arrived" from "the heuristic returns its ceiling".
|
||||
// The ramp now saturates AT the firing point (see `STAGED_ONE_AT`), so no count this fixture
|
||||
// can carry is mid-ramp and that separation is no longer available from one run. It is restored
|
||||
// below by a second run at a count the heuristic must score ZERO on: a mutant returning a
|
||||
// constant ceiling fails there, which is the property this case would otherwise have lost.
|
||||
expect(sub['asset-staging']).toBe(1);
|
||||
// 🔴 AND THE THREE RING HEURISTICS SCORE NOTHING, which is the point of the case: this account
|
||||
// is on the board because of its OWN uploads, with no other account involved anywhere in the
|
||||
// run. No previous heuristic could have produced this finding.
|
||||
expect(sub['posting-velocity']).toBe(0);
|
||||
expect(sub['registration-cluster']).toBe(0);
|
||||
expect(sub['content-templating']).toBe(0);
|
||||
// 🔴 AND THE FIRING POINT ITSELF, THROUGH THE WHOLE RUN. One of four heuristics at 0.5 blends
|
||||
// to 0.125, against a shipped cut of 0.1125 — so two staged uploads is a row a moderator
|
||||
// actually receives. `heuristics.test.ts` pins the same property against the registry and the
|
||||
// partition directly; this asserts it survives the run's real reader, evidence layer, scorer
|
||||
// and report rendering, which is the composition no unit case builds.
|
||||
expect(finding?.confidence).toBeCloseTo(0.125, 12);
|
||||
// 🔴 AND THE FIRING POINT ITSELF, THROUGH THE WHOLE RUN. One of four heuristics at 1 blends to
|
||||
// 0.25, against a shipped cut of 0.1125 — so two staged uploads is a row a moderator actually
|
||||
// receives. `heuristics.test.ts` pins the same property against the registry and the partition
|
||||
// directly; this asserts it survives the run's real reader, evidence layer, scorer and report
|
||||
// rendering, which is the composition no unit case builds.
|
||||
expect(finding?.confidence).toBeCloseTo(0.25, 12);
|
||||
expect(finding?.confidence as number).toBeGreaterThanOrEqual(MIN_REPORTED_CONFIDENCE);
|
||||
// ⚠️ The `findingsReported` line is a cohort-size check, NOT part of the firing-point claim —
|
||||
// this scenario runs with `minConfidence: 0`, so every scored member is emitted whatever it
|
||||
// scored and this count cannot distinguish a reported account from a suppressed one. What
|
||||
// carries "a row a moderator actually receives" is the `>= MIN_REPORTED_CONFIDENCE` line above.
|
||||
expect((await scenario.result).findingsReported).toBe(3);
|
||||
|
||||
// The reason names WHAT was seen, not merely that something was.
|
||||
expect(finding?.reason).toContain('no generation metadata');
|
||||
expect(finding?.reason).toContain('attached to no post');
|
||||
|
||||
// 🔴 THE NEGATIVE CONTROL THE SATURATED EXPECTATION ABOVE COSTS THIS CASE OTHERWISE. Same run,
|
||||
// same wiring, ONE staged upload per member instead of two — below the firing point, so the
|
||||
// heuristic must score nothing. A mutant that returns the heuristic's ceiling without reading
|
||||
// the evidence passes every assertion above and fails here.
|
||||
//
|
||||
// Asserted on the SUB-SCORE and the confidence rather than on `findingsReported`: this scenario
|
||||
// runs with `minConfidence: 0`, so every scored member is emitted whatever it scored, and a
|
||||
// finding count would be measuring the option rather than the heuristic.
|
||||
const below = stagingRun(stagedEvidence(1, false));
|
||||
await below.result;
|
||||
const belowFinding = below.reports[0].findings.find((f) => f.userId === 1);
|
||||
expect(belowFinding).toBeDefined();
|
||||
expect(subScoresOf(belowFinding as { reason: string })['asset-staging']).toBe(0);
|
||||
expect(belowFinding?.confidence).toBe(0);
|
||||
});
|
||||
|
||||
it('🔴 the SAME-SECOND half moves the COUNTERS and NOT the score — the arm is inert, not absent', async () => {
|
||||
@@ -608,15 +637,18 @@ describe('🔴 the seam between the evidence and the scoring', () => {
|
||||
//
|
||||
// 🔴 THIS CASE USED TO ASSERT THE OPPOSITE, AND THE CHANGE IS THE POINT. While the burst
|
||||
// boundaries sat tighter than the volume ones (4 against 8) the same-second run scored HIGHER
|
||||
// on the board, and this case pinned that. Both pairs are now (1, 3) — derived from the same
|
||||
// reporting cut — and a same-second group is a SUBSET of the staged rows, so the burst half can
|
||||
// never exceed the volume half and `max` resolves to `volume` for every account. The honest
|
||||
// statement is therefore an EQUALITY on the score and a DIFFERENCE on the counters, and writing
|
||||
// it the old way would be a guard asserting behaviour the shipped code does not have.
|
||||
// on the board, and this case pinned that. A same-second group is a SUBSET of the staged rows
|
||||
// and the volume ramp is at or above the burst ramp at every input, so the burst half can never
|
||||
// exceed the volume half and `max` resolves to `volume` for every account. The honest statement
|
||||
// is therefore an EQUALITY on the score and a DIFFERENCE on the counters, and writing it the
|
||||
// old way would be a guard asserting behaviour the shipped code does not have.
|
||||
//
|
||||
// TWO uploads per member, not three: two is the firing point, so the shared score is the ramp's
|
||||
// midpoint 0.5 rather than its ceiling. An equality asserted at saturation would hold for the
|
||||
// uninteresting reason that both halves had run out of range.
|
||||
// 🔴 THE EQUALITY IS AT SATURATION NOW, AND THAT WEAKENS IT — SAID HERE RATHER THAN LEFT IN THE
|
||||
// GREEN. The volume ramp saturates at the firing point (see `STAGED_ONE_AT`), so there is no
|
||||
// count at which these two runs could be compared mid-ramp, and a mutant that saturated a half
|
||||
// would land on the same 1. The DISCRIMINATION MOVED TO THE COUNTERS, which is also where the
|
||||
// arm's only remaining product is: the spread run must report `fired_burst` 0 against
|
||||
// `fired_volume` 3, which a burst half that saturated or ignored its boundary cannot do.
|
||||
const spread = stagingRun(stagedEvidence(2, false));
|
||||
const spreadResult = await spread.result;
|
||||
const burst = stagingRun(stagedEvidence(2, true));
|
||||
@@ -626,16 +658,27 @@ describe('🔴 the seam between the evidence and the scoring', () => {
|
||||
subScoresOf(s.reports[0].findings.find((f) => f.userId === 1) as { reason: string })[
|
||||
'asset-staging'
|
||||
];
|
||||
// Rendered to two decimals in the reason clause: (2-1)/(3-1) = 0.5 on the volume half in BOTH
|
||||
// runs, and the burst half contributes nothing visible even when it is fully engaged. Neither 0
|
||||
// nor 1, so a mutant that saturates or zeroes a half still cannot land on it.
|
||||
expect(scoreOf(spread)).toBe(0.5);
|
||||
expect(scoreOf(burst)).toBe(0.5);
|
||||
// Rendered to two decimals in the reason clause: the volume half saturates at its boundary of
|
||||
// two in BOTH runs, and the burst half contributes nothing visible even when it is fully
|
||||
// engaged. A mutant that zeroed the VOLUME half fails here — `scoreOf` reads 0 out of the
|
||||
// rendered reason clause against an expected 1. (NOT because the finding would disappear: this
|
||||
// scenario runs with `minConfidence: 0`, so a member scoring 0 is still emitted, which the
|
||||
// negative control in the sibling case relies on.) 🔴 A mutant that zeroed the BURST half does
|
||||
// NOT fail here — `max` is identically `volume`, so `scoreOf` still reads 1 and both lines pass;
|
||||
// its first failure is the `fired_burst` counter below. Nor does one that saturates either half.
|
||||
// That is precisely why the counters carry this case rather than these two lines.
|
||||
expect(scoreOf(spread)).toBe(1);
|
||||
expect(scoreOf(burst)).toBe(1);
|
||||
|
||||
// 🔴 THE COUNTERS ARE WHERE THE BURST HALF STILL EXISTS, AND THEY ARE NOW ITS ONLY PRODUCT
|
||||
// BESIDES THE MODERATOR CLAUSE. This is what the shadow phase reads to answer whether
|
||||
// same-second concentration separates at all — and therefore whether the arm should be
|
||||
// re-tightened below the volume boundary or deleted. Without the decomposition that question
|
||||
// re-shaped or deleted — NOT "re-tightened below the volume boundary", which this sentence said
|
||||
// until the volume ramp became a step at two. A tighter `BURST_ONE_AT` no longer moves any
|
||||
// score. (It is NOT impossible, which an earlier wording of this correction claimed: a burst
|
||||
// pair of (0, 1) is a legal pair strictly below the volume boundary and does revive the arm —
|
||||
// by dropping `zeroAt`, which is what breaks the dominance, not by tightening `oneAt`. See
|
||||
// `BURST_ONE_AT`.) Without the decomposition that question
|
||||
// has no number behind it, which is the failure that kept a zero-firing comment source alive
|
||||
// for five runs one heuristic over.
|
||||
expect(burstResult.counters['heuristic:asset-staging:fired_burst']).toBe(3);
|
||||
|
||||
@@ -284,7 +284,8 @@ describe('the reporting threshold', () => {
|
||||
// against a silently changed lone-signal bar and this case would stay green. With it, changing
|
||||
// the bar requires editing a test that says out loud that the value is inherited — which is the
|
||||
// point at which someone has to supply evidence for a new one. `asset-staging`'s boundaries are
|
||||
// derived against this number, so it is load-bearing for a firing point even while provisional.
|
||||
// CHECKED against this number — they were derived from it until the ordering evidence set that
|
||||
// heuristic's volume boundary instead — so it still bounds a firing point even while provisional.
|
||||
expect(LONE_SIGNAL_CUT).toBe(0.45);
|
||||
// A worked instance, with literals rather than expressions over the constants — the same
|
||||
// reasoning the boundary cases in `heuristics.test.ts` are written with. A registry of four
|
||||
|
||||
@@ -1,12 +1,18 @@
|
||||
/**
|
||||
* The one shape every heuristic in this directory turns a measurement into a score with.
|
||||
*
|
||||
* 🔴 ONE RAMP, NOT THREE. Each heuristic measures something with a different unit — items per hour,
|
||||
* accounts per IP, accounts per fingerprint — but all three answer the same question: "how far past
|
||||
* boring is this". Writing that arithmetic once means the three heuristics differ ONLY in what they
|
||||
* measure and where their two boundaries sit, which is what makes the sub-scores comparable enough
|
||||
* to sit beside each other in one reason string. It is also one place for the off-by-one to live
|
||||
* rather than three, and the boundary is the part every mutation check aims at.
|
||||
* 🔴 ONE RAMP, NOT FOUR. Each heuristic measures something with a different unit — items per hour,
|
||||
* accounts per IP, accounts per email domain, accounts per fingerprint, staged uploads, staged
|
||||
* uploads inside one second — but all of them answer the same question: "how far past boring is
|
||||
* this". Writing that arithmetic once means the heuristics differ ONLY in what they measure and
|
||||
* where their two boundaries sit, which is what makes the sub-scores comparable enough to sit beside
|
||||
* each other in one reason string. It is also one place for the off-by-one to live rather than
|
||||
* seven, and the boundary is the part every mutation check aims at.
|
||||
*
|
||||
* (Four heuristics, seven call sites: `velocity.ts`, `clustering.ts` twice, `similarity.ts` twice,
|
||||
* `staging.ts` twice. The count is stated because the paragraphs below reason about the callers as
|
||||
* a set — one of them specifically about `asset-staging`, which an earlier "NOT THREE" wording had
|
||||
* left out of the list entirely.)
|
||||
*
|
||||
* The two boundaries are named for what they DO, not for what they bound:
|
||||
* - `zeroAt` — the largest value that is still worth nothing. A value equal to it scores exactly 0.
|
||||
@@ -29,9 +35,34 @@
|
||||
*
|
||||
* `oneAt <= zeroAt` throws rather than returning something. It is not a value a caller could have
|
||||
* meant: the two boundaries would be inverted or coincident, every input would land on a degenerate
|
||||
* step, and the resulting heuristic would look calibrated while scoring 0 or 1 and nothing between.
|
||||
* A constant this wrong is a bug at module load, and it is better found there than averaged into a
|
||||
* moderator's queue.
|
||||
* step, and the resulting heuristic would look calibrated while never scoring anything between its
|
||||
* two ends. A constant this wrong is a bug at module load, and it is better found there than
|
||||
* averaged into a moderator's queue.
|
||||
*
|
||||
* 🔴 WHAT THE THROW DOES NOT GUARD, BECAUSE THE OBVIOUS READING IS WRONG AND LEADS SOMEONE TO
|
||||
* DELETE IT: for an ORDERED-BUT-DEGENERATE pair it is not protecting the division. With the throw
|
||||
* removed, `oneAt <= zeroAt` never reaches the interpolation at all — the two clamps below cover the
|
||||
* entire real line (coincident, every finite value is at or outside one end; inverted, the ranges
|
||||
* overlap and `<= zeroAt` is tested first), so there is no division by zero and no `NaN` to catch.
|
||||
* The failure it prevents there is silent, not loud: a ramp that has quietly become a step at
|
||||
* `zeroAt` while its constants still read like a calibrated pair.
|
||||
*
|
||||
* The guard is written `!(oneAt > zeroAt)` rather than `oneAt <= zeroAt`, which is WIDER — it also
|
||||
* fires when a boundary is `NaN`, and that case genuinely does reach the division and return `NaN`.
|
||||
* No call site can produce it today (all seven pass integer module constants), so that is precision
|
||||
* about the guard rather than a hazard; do not narrow the comparison on the strength of the
|
||||
* paragraph above.
|
||||
*
|
||||
* 🔴 AND A STEP IS NOT ITSELF A BUG — ONE CALL SITE SHIPS ONE DELIBERATELY, WITH A VALID PAIR. The
|
||||
* general rule, which is all this file needs to state: a call site whose two boundaries are ADJACENT
|
||||
* INTEGERS, fed integer inputs, never reaches the interpolation below — the ramp is a step there,
|
||||
* deliberately or not — so a mutation of that line is invisible through that call site and must be
|
||||
* killed through one whose boundaries are further apart. Read it as the CALL SITE and not the
|
||||
* heuristic: the two are not the same, and a heuristic with a second, wider call site does kill such
|
||||
* a mutant. One caller ships exactly that shape on purpose; its constants, its reasoning and the
|
||||
* worked coverage consequence are on `STAGED_ONE_AT` and in the case named "starts at 0 below the
|
||||
* volume boundary, saturates ON it, and stays there", not restated here — a caller's constants
|
||||
* copied into the shared helper go stale silently the next time that caller is tuned.
|
||||
*/
|
||||
export function rampScore(value: number, zeroAt: number, oneAt: number): number {
|
||||
if (!(oneAt > zeroAt))
|
||||
|
||||
@@ -38,12 +38,14 @@ import { rampScore } from './ramp';
|
||||
*
|
||||
* 🔴 AND THAT CLASS GOT BIGGER IN THE CHANGE THAT SET THESE BOUNDARIES, WHICH IS THE COST OF THEM.
|
||||
* Firing from TWO staged uploads rather than from eight means a two-file abandoned drag now scores
|
||||
* enough to be reported on its own, where before it scored a fraction of the cut. That is the
|
||||
* deliberate trade — the old boundary was calibrated against a predicate this heuristic does not
|
||||
* ship (see `STAGED_ONE_AT`) and selected almost nobody — but the consequence is that this signal's
|
||||
* precision now rests entirely on the shadow phase measuring it, not on the boundary being cautious.
|
||||
* Anyone reading `sole_signal` for this id is reading the number that decides whether that trade
|
||||
* was right.
|
||||
* enough to be reported on its own, where before it scored a fraction of the cut — and since the
|
||||
* volume ramp was flattened to a step at two (see `STAGED_ONE_AT`) that drag no longer scores a
|
||||
* half, it scores the maximum. That is the deliberate trade — the old boundary was calibrated
|
||||
* against a predicate this heuristic does not ship and selected almost nobody, and the rung above
|
||||
* the pair graded WORSE than the pair rather than better — but the consequence is that this
|
||||
* signal's precision now rests entirely on the shadow phase measuring it, not on the boundary being
|
||||
* cautious. Anyone reading `sole_signal` for this id is reading the number that decides whether
|
||||
* that trade was right.
|
||||
*
|
||||
* 🔴 THE KNOWN FALSE NEGATIVE: A STAGE THAT WAS LATER PUBLISHED. An account that stages forty images
|
||||
* and then attaches them to a post scores 0 here from the moment it does, because `postId` stops
|
||||
@@ -55,10 +57,13 @@ import { rampScore } from './ramp';
|
||||
* 🔴 IT SEES A SAMPLE, NOT A CENSUS, AND HERE THAT BOUNDS THE SCORE RATHER THAN ONLY THE READ. The
|
||||
* read is budgeted (`MAX_STAGED_IMAGE_SAMPLES`) and per-member capped
|
||||
* (`MAX_STAGED_IMAGES_PER_MEMBER`), so `count` is the number SAMPLED and never more than that cap.
|
||||
* The ramp saturates far below it — more than an order of magnitude, and the margin WIDENED when
|
||||
* the boundary moved down to 3 — so no real account is mis-scored by the cap today; that is a
|
||||
* property of where the two numbers sit and not a guarantee, and moving either is the moment to
|
||||
* re-check it. The relationship is asserted in `evidence.test.ts` rather than left to a reader.
|
||||
* The ramp saturates far below it — 50 against a boundary of 2, and the margin WIDENED again when
|
||||
* that boundary moved down — so no real account is mis-scored by the cap today; that is a property
|
||||
* of where the two numbers sit and not a guarantee, and moving either is the moment to re-check it.
|
||||
* ⚠️ `evidence.test.ts` asserts a RELATED BUT WEAKER thing — that the cap exceeds four times the
|
||||
* boundary, i.e. 8, not the 25× above — so read that guard as a floor under a per-member cap dropped
|
||||
* near the ramp, not as a check on this sentence. The margin itself is pinned only by the two
|
||||
* literals beside it.
|
||||
*
|
||||
* 🔴 A ZERO FROM A DEAD SOURCE IS THE DANGEROUS ZERO FOR THIS HEURISTIC SPECIFICALLY. Its index is
|
||||
* empty both when a member staged nothing and when the read never ran, and unlike the ring
|
||||
@@ -81,65 +86,147 @@ export const ASSET_STAGING_ID = 'asset-staging';
|
||||
* and did not finish — and scoring it would fire on a large share of every day's genuine signups,
|
||||
* which is the "fires on 90% of accounts" uselessness the scoring seam exists to make visible.
|
||||
*
|
||||
* 🔴 `oneAt: 3` IS DERIVED FROM THE REPORTING CUT, NOT PICKED — AND THE DERIVATION IS THE REASON IT
|
||||
* IS 3 AND NOT SOMETHING ROUNDER. The requirement is that TWO staged uploads is enough for this
|
||||
* heuristic to put an account on the board ON ITS OWN. `scoreAccount` divides by the WHOLE
|
||||
* 🔴 `oneAt: 2` MAKES THE RAMP SATURATE AT THE FIRING POINT, AND THAT IS THE WHOLE OF THE FIX IT
|
||||
* CARRIES. A rising ramp puts its TOP rung on the largest counts, and on this predicate the largest
|
||||
* counts are the wrong population. A coordinated upload clusters at exactly TWO assets — an avatar
|
||||
* and a header, which is what a profile needs and no more — while a legitimate profile setup, a
|
||||
* business or a creator bringing in a kit, runs to THREE OR MORE. With the previous `oneAt = 3` a
|
||||
* pair scored 0.5 and everything from three up scored 1.0, so the rung this heuristic weighted
|
||||
* HIGHEST was the rung carrying disproportionately many legitimate accounts, and the shape it
|
||||
* exists to find sat at half of what it selected. Saturating at 2 removes the inversion in the one
|
||||
* way the shared ramp can express: `score(2) === score(3+)`.
|
||||
*
|
||||
* 🔴 A PLATEAU, NOT A SUPPRESSION, AND THE DIFFERENCE IS DELIBERATE. `count >= 3` still scores 1.0.
|
||||
* It is the WORSE of the two arms, not an empty one — it still carries genuine catches — so the
|
||||
* requirement this constant satisfies is that it must stop OUTSCORING a pair, never that it stop
|
||||
* scoring. `rampScore` is monotonically non-decreasing by construction and throws on
|
||||
* `oneAt <= zeroAt`, so a DECLINING shape is not expressible in it at all without a second ramp
|
||||
* term; between the two shapes that are expressible, the plateau is one constant and adds no
|
||||
* machinery. Both the ordering and the non-suppression are pinned by the case named "THE ORDERING:
|
||||
* three or more staged uploads never OUTSCORES exactly two" in `__tests__/heuristics.test.ts`,
|
||||
* which asserts `>=` rather than two literals precisely so that a later re-shaping backed by its
|
||||
* own measurement does not have to fight a test pinning this implementation.
|
||||
*
|
||||
* 🔴 IT STILL SATISFIES THE REPORTING-CUT DERIVATION THAT SET THE PREVIOUS VALUE — CHECKED, NOT
|
||||
* ASSUMED, BECAUSE THAT IS THE PROPERTY A READER CARES ABOUT. The requirement is that TWO staged
|
||||
* uploads is enough to put an account on the board ON ITS OWN. `scoreAccount` divides by the WHOLE
|
||||
* registry's weight, so a lone sub-score `s` blends to `s / n` and is compared against
|
||||
* `MIN_REPORTED_CONFIDENCE`, which is `LONE_SIGNAL_CUT / n` — the `n`s cancel, and a lone signal is
|
||||
* reported exactly when `s >= LONE_SIGNAL_CUT`. With `zeroAt = 1` the ramp puts a count of two at
|
||||
* `(2 - 1) / (oneAt - 1)`, so:
|
||||
* `1 / (oneAt - 1)`, so the cut requires `oneAt <= 1 + 1/0.45 = 3.22…`. The previous value took the
|
||||
* LARGEST integer satisfying that; 2 is simply a smaller one, and a pair now scores 1.0 — clear of
|
||||
* the 0.45 cut by the widest margin available rather than by a rounding step. Note what changed
|
||||
* about the derivation's STATUS: `oneAt` is no longer DERIVED from `LONE_SIGNAL_CUT`, it is set by
|
||||
* the ordering above and CHECKED against the cut. The end-to-end property is pinned by the case
|
||||
* named "THE FIRING POINT: a LONE asset-staging signal is REPORTED at two staged uploads, not one".
|
||||
*
|
||||
* 1 / (oneAt - 1) >= LONE_SIGNAL_CUT (0.45) ⟺ oneAt <= 1 + 1/0.45 = 3.22…
|
||||
* 🔴 THE COST, NAMED RATHER THAN TUNED AWAY: THIS HALF NOW HAS NO GRADIENT AT ALL. It is a step —
|
||||
* 0 below two, 1 at two and above — so the sub-score can no longer express "more staged than that"
|
||||
* and a moderator ordering a queue by confidence gets no separation between a pair and forty. That
|
||||
* is the honest consequence of a monotone helper meeting a non-monotone finding, and it is not
|
||||
* hidden: the COUNT itself is still disclosed verbatim to the moderator by `explain`. If the shadow
|
||||
* phase shows the two arms want separating rather than levelling, the edit that earns it is a real
|
||||
* re-shape with its own measurement — not widening this constant back and reinstating the inversion.
|
||||
*
|
||||
* The largest integer satisfying it is 3. A pair then scores 0.5 — clear of the cut by more than a
|
||||
* rounding step, which is deliberate so a mutant nudging either constant cannot land between them —
|
||||
* and three saturates. `oneAt = 4` would put a pair at 0.333, below the cut, which is the behaviour
|
||||
* this change exists to remove. The property is pinned end-to-end, through the real blend and the
|
||||
* real partition, by the case named "THE FIRING POINT: a LONE asset-staging signal is REPORTED at
|
||||
* two staged uploads, not one" in `__tests__/heuristics.test.ts`.
|
||||
* 🔴 AND THE RUN'S OWN COUNTERS CAN NO LONGER PRODUCE THAT MEASUREMENT, WHICH IS THE SHARPEST COST
|
||||
* AND THE EASIEST ONE TO MISS. The two rungs this change was made on are now indistinguishable in
|
||||
* every number a run emits: `fired` is 1 for both, `fired_volume` is 1 for both, `fired_burst`
|
||||
* measures concentration rather than the rung, and `confidence_bucket_*` — which DID separate them,
|
||||
* a lone pair landing in the 10–20 bucket against 20–30 for three or more — now puts both in the
|
||||
* same bucket. The count survives only as free text inside each finding's reason. So a future
|
||||
* re-shape has to be graded the way this one was, against moderation outcomes joined outside this
|
||||
* repository; it cannot be read off the shadow-phase counters. No counter was added here because
|
||||
* one would be a new metric key with no consumer, but nobody should discover this by looking for a
|
||||
* number that is not there.
|
||||
*
|
||||
* 🔴 WHERE THIS CAME FROM, AND WHY THE PREVIOUS VALUE (8) WAS NOT EVIDENCE. The old boundary was
|
||||
* calibrated against a predicate this heuristic does not ship: a RATIO test — "all of the account's
|
||||
* images are staged" — rather than the unratioed COUNT above. Re-measured against the SHIPPED
|
||||
* predicate on a matured cohort (accounts old enough for a moderation outcome to exist, graded
|
||||
* against that cohort's own base actioned rate), the old volume boundary selected a population far
|
||||
* too small to carry any signal, while a firing point of two separated strongly and on a population
|
||||
* large enough to mean something. That measurement — its cohort definition, denominators, per-cell
|
||||
* rates and lift — lives in the private infra repo and is deliberately not restated here, because
|
||||
* this repository is public.
|
||||
* 🔴 TWO SERIES MOVE ON THE DAY THIS SHIPS, WITH NO ACCOUNT BEHAVING DIFFERENTLY. Read them as the
|
||||
* instrument moving, not the population — the same artefact `soleSignalCounters` documents for a
|
||||
* registry-size change, in the opposite direction. (a) Every `count == 2` account's confidence rises
|
||||
* by exactly +0.125, which for one carrying NO other signal is 0.125 → 0.25, i.e. one
|
||||
* `confidence_bucket_*` bucket up; an account that also scores elsewhere can cross TWO bucket edges
|
||||
* on the same +0.125, so do not read the shift as uniformly one bucket. (b)
|
||||
* `heuristic:asset-staging:sole_signal` INFLATES: the dominance test credits a leader at
|
||||
* `leader >= 4 × runnerUp`, and this heuristic's leader score on that population doubled, so the
|
||||
* runner-up tolerance doubles with it and accounts that previously counted for nobody now count
|
||||
* here. Neither series is comparable across this deploy.
|
||||
*
|
||||
* The REPORTED POPULATION does not move AT THE SHIPPED CUT — 0.125 already cleared
|
||||
* `MIN_REPORTED_CONFIDENCE`, and the production caller passes no override — so for the deployed
|
||||
* configuration this is an instrumentation discontinuity and not a detection change. ⚠️ `run.ts`
|
||||
* takes `minConfidence` as an option, and for any run configured in the band (0.125, 0.25] it IS a
|
||||
* detection change: the whole lone-signal pair population moves from suppressed to reported. A
|
||||
* deliberate high-precision grading pass is exactly the run that would sit in that band.
|
||||
*
|
||||
* 🔴 WHERE THIS CAME FROM, AND WHY NEITHER PREVIOUS VALUE (8, THEN 3) WAS EVIDENCE FOR THE TOP
|
||||
* RUNG. The original boundary was calibrated against a predicate this heuristic does not ship: a
|
||||
* RATIO test — "all of the account's images are staged" — rather than the unratioed COUNT above.
|
||||
* Re-measured against the SHIPPED predicate on a matured cohort (accounts old enough for a
|
||||
* moderation outcome to exist, graded against that cohort's own base actioned rate), it moved to 3,
|
||||
* which fixed the FIRING POINT and left the ORDERING untouched — nobody had asked which of the two
|
||||
* scoring rungs graded better, only whether a pair reached the board at all. Grading the rungs
|
||||
* separately is what produced this change: the pair rung graded strongly, and the 3+ rung graded
|
||||
* materially worse than it. Those measurements — cohort definition, denominators, per-cell rates
|
||||
* and lift — live in the private infra repo and are deliberately not restated here, because this
|
||||
* repository is public.
|
||||
*/
|
||||
export const STAGED_ZERO_AT = 1;
|
||||
export const STAGED_ONE_AT = 3;
|
||||
export const STAGED_ONE_AT = 2;
|
||||
|
||||
/**
|
||||
* The same two boundaries for the SAME-SECOND half — and they are now the SAME NUMBERS as the
|
||||
* volume half's, which is a statement about this arm and not a coincidence.
|
||||
* The same two boundaries for the SAME-SECOND half — left where they were when the volume half's
|
||||
* top boundary moved down, which is a statement about this arm and not an oversight.
|
||||
*
|
||||
* `zeroAt: 1` because every staged upload shares its own second with itself, so a burst of one is
|
||||
* what a member with any staged image at all has and it must be worth nothing. Two inside one second
|
||||
* is the smallest that scores, which is what "created together" means at this resolution.
|
||||
*
|
||||
* `oneAt: 3` is the identical cut arithmetic `STAGED_ONE_AT` is derived with: a same-second PAIR
|
||||
* must clear `LONE_SIGNAL_CUT` on its own, and 3 is the largest integer boundary that does.
|
||||
* `oneAt: 3` is the cut arithmetic `STAGED_ONE_AT` used to be derived with: a same-second PAIR must
|
||||
* clear `LONE_SIGNAL_CUT` on its own, and 3 is the largest integer boundary that does. It was NOT
|
||||
* moved alongside the volume boundary, because the finding that moved that one is a statement about
|
||||
* staged COUNT rungs and says nothing about same-second concentration. Changing a constant the
|
||||
* measurement did not cover, merely to keep two numbers looking alike, would be inventing evidence.
|
||||
*
|
||||
* 🔴 THE BURST ARM THEREFORE CHANGES NOTHING ABOUT THE SCORE — NOT WHETHER THIS HEURISTIC FIRES,
|
||||
* NOT HOW HIGH IT SCORES — AND SAYING SO PLAINLY IS THE POINT. A same-second group is a SUBSET of
|
||||
* the staged rows, so `largestSameSecondBurst <= count` ALWAYS: `evidence.ts` builds both from one
|
||||
* walk over the same rows, and a row whose timestamp will not parse increments `count` while being
|
||||
* left out of the burst tally, which can only widen the gap. `rampScore` is monotonic, so feeding
|
||||
* the smaller of two values through the SAME pair of boundaries cannot produce the larger score.
|
||||
* Hence `max(volume, burst) === volume` identically at these constants. Anything implying the burst
|
||||
* half independently widens the net would be false.
|
||||
* 🔴 THE BURST ARM CHANGES NOTHING ABOUT THE SCORE — NOT WHETHER THIS HEURISTIC FIRES, NOT HOW HIGH
|
||||
* IT SCORES — AND SAYING SO PLAINLY IS THE POINT. A same-second group is a SUBSET of the staged
|
||||
* rows, so `largestSameSecondBurst <= count` ALWAYS: `evidence.ts` builds both from one walk over
|
||||
* the same rows, and a row whose timestamp will not parse increments `count` while being left out of
|
||||
* the burst tally, which can only widen the gap. The volume ramp now also DOMINATES this one
|
||||
* pointwise — same `zeroAt`, smaller `oneAt`, so it is at or above the burst ramp at every input —
|
||||
* and `rampScore` is monotonic. Chaining the two: `burst = ramp_b(b) <= ramp_b(count) <=
|
||||
* ramp_v(count) = volume`. Hence `max(volume, burst) === volume` identically. Anything implying the
|
||||
* burst half independently widens the net would be false.
|
||||
*
|
||||
* 🔴 THE IDENTITY HOLDS FOR A RANGE OF BURST PAIRS RATHER THAN ONLY FOR THIS ONE — BUT IT IS NOT
|
||||
* UNCONDITIONAL, AND THE CONDITION IS THE WHOLE POINT. The volume half is a STEP: 0 at a count of
|
||||
* one or less, 1 at two or more. While `BURST_ZERO_AT >= STAGED_ZERO_AT`, a burst needs a count of
|
||||
* at least two to score anything, so the volume half is already saturated wherever the burst half is
|
||||
* non-zero and `burst > volume` is unreachable. **Drop `BURST_ZERO_AT` to 0 and that fails**: an
|
||||
* account with one staged upload has a burst tally of 1, scoring `rampScore(1, 0, 3)` = 0.333 on a
|
||||
* burst half against 0 on the volume half, and the arm moves the score again. That is measured
|
||||
* rather than reasoned; the figures are on the `max` below, stated once. So the honest statement is
|
||||
* CONDITIONAL, and the condition is IMPLIED BY the pointwise-dominance relationship the test named
|
||||
* "the volume boundaries stay no wider than the burst ones" pins — not identical to it. That test
|
||||
* pins both boundaries; at the shipped step shape only the `zeroAt` half is needed, so it guards
|
||||
* something strictly stronger than this paragraph requires.
|
||||
*
|
||||
* What this DOES cost is the LOOP in the case pinning `score === volume`, which is now `x <= x`
|
||||
* over its table and cannot go red for a burst-constant change the way it once did. The guards that
|
||||
* replace it are enumerated at the head of that case in `__tests__/heuristics.test.ts` rather than
|
||||
* restated here — a coverage ledger kept in two places is the one that goes stale.
|
||||
*
|
||||
* 🔴 SO IT HAS NO SCORING RATIONALE TODAY, AND NONE IS INVENTED HERE. It previously had one: a
|
||||
* tighter boundary pair (4 against 8) expressing "concentration is the stronger of the two claims",
|
||||
* which was a real difference in behaviour. Moving the firing point to two consumed that rationale
|
||||
* outright, because the volume half now fires everywhere the burst half possibly could. The
|
||||
* re-measurement does show same-second concentration separating harder than raw volume one count
|
||||
* further up, which is the only thing that would justify tightening this boundary below the volume
|
||||
* one again — it rests on too few members to author a constant from, so it has not been used, and
|
||||
* outright, because the volume half now fires everywhere the burst half possibly could, and
|
||||
* saturating the volume half at two has since put the identity out of reach of a tighter boundary
|
||||
* (the condition and its one exception are two paragraphs up; they are not restated here).
|
||||
* The re-measurement does show same-second concentration separating harder than raw volume one count
|
||||
* further up — it rests on too few members to author a constant from, so it has not been used, and
|
||||
* inventing a gradient the measurement does not support is exactly what this comment exists to
|
||||
* prevent.
|
||||
* prevent. Reviving this arm needs the combination in `score` to stop being a `max` over a DOMINATED
|
||||
* operand — which a tighter `BURST_ONE_AT` cannot do, though dropping `BURST_ZERO_AT` below
|
||||
* `STAGED_ZERO_AT` can (the paragraph above). "More than a boundary edit" would be too strong: the
|
||||
* dominated-ness is what matters, not which kind of edit undoes it.
|
||||
*
|
||||
* What the arm still produces is real and is NOT the score: `assetStagingHalfScores` reports the
|
||||
* halves separately (`heuristic:asset-staging:fired_burst` in `run.ts`), which is how the shadow
|
||||
@@ -171,9 +258,11 @@ export function stagedImageFacts(
|
||||
* same-second concentration doing work the volume half was not already doing" — used to be partly
|
||||
* answerable from the SCORE, because the burst boundaries were tighter. They are not any more, so
|
||||
* `max` is identically `volume` (see `BURST_ONE_AT`) and the score carries no information about
|
||||
* this half whatsoever. The counters `run.ts` builds from these two values are therefore the whole
|
||||
* of the evidence that will decide whether the arm gets re-tightened or deleted. Read carefully:
|
||||
* `burst > 0` here does NOT mean the burst half contributed anything to the account's sub-score.
|
||||
* this half whatsoever — and no TIGHTENING of `BURST_ONE_AT` can restore it, for the reasons and
|
||||
* under the one condition set out on `BURST_ONE_AT`. The counters `run.ts` builds from these two
|
||||
* values are therefore the whole of the evidence that will decide whether the arm gets re-shaped or
|
||||
* deleted. Read carefully: `burst > 0` here does NOT mean the burst half contributed anything to
|
||||
* the account's sub-score.
|
||||
*
|
||||
* They may BOTH be non-zero on one account, deliberately: they are two questions about the same
|
||||
* uploads, not a partition of them. The reverse — `burst > 0` while `volume` is 0 — cannot occur
|
||||
@@ -204,19 +293,44 @@ export const assetStagingHeuristic: BotAccountHeuristic = {
|
||||
// of the count, so summing would also double-count the same rows.
|
||||
//
|
||||
// 🔴 AT TODAY'S CONSTANTS THIS `max` IS IDENTICALLY `volume` — the proof is on `BURST_ONE_AT`,
|
||||
// and the identity is pinned by the case named "the burst half NEVER exceeds the volume half at
|
||||
// today's boundaries". It is kept rather than collapsed to a bare `volume` because the identity is a
|
||||
// property of the two boundary PAIRS being equal, not of the model: the moment either pair moves,
|
||||
// `max` is the correct combination and a hardcoded `volume` would silently drop the burst arm
|
||||
// with nothing failing. Read it as "the arm is inert, not absent" — the pin is what makes a
|
||||
// boundary change that revives it visible instead of assumed.
|
||||
// and the identity is pinned by the case named "the SCORE is identically the volume half — the
|
||||
// burst arm cannot raise it". It is kept rather than collapsed to a bare `volume` because the
|
||||
// identity is a property of the two boundary pairs and of the subset relation, not of the model:
|
||||
// the moment the DOMINANCE breaks — the burst ramp rising somewhere the volume ramp has not —
|
||||
// `max` is the correct combination again and a hardcoded `volume` would silently drop the burst
|
||||
// arm. Read it as "the arm is inert, not absent".
|
||||
//
|
||||
// 🔴 MEASURED, SO THE CLAIM IS NOT LEFT AS REASONING: replacing this line with `return volume`
|
||||
// leaves the whole suite GREEN (418/418). That mutant SURVIVES, and it survives because it is
|
||||
// semantically equivalent at these constants, not because the tests are thin — which is the
|
||||
// difference this comment exists to record. Under `BURST_ONE_AT = 2` the equivalence breaks and
|
||||
// the subset pin goes red, which is the control proving the guard is reachable rather than
|
||||
// decorative.
|
||||
// 🔴 "THE MOMENT THE VOLUME HALF STOPS BEING A STEP" IS THE WRONG TRIGGER, AND THIS COMMENT SAID
|
||||
// IT — MEASURED FALSE. At `STAGED_ONE_AT = 3` with its pin updated, i.e. the volume half exactly
|
||||
// as un-stepped as it was before this change, `max` and `return volume` fail the SAME 8 cases:
|
||||
// zero difference. Un-stepping the volume half does not make the hardcode wrong. Breaking the
|
||||
// dominance does (11 against 6; see below). An earlier wording, "the moment either pair moves",
|
||||
// was wrong the same way — `BURST_ONE_AT -> 4` moves a pair and leaves `max` and the hardcode
|
||||
// still indistinguishable. ⚠️ Read that as a claim about THIS discrimination only: such an edit
|
||||
// is not invisible to the suite, it reddens the two assertions that read the burst half at 0.5.
|
||||
//
|
||||
// 🔴 MEASURED AT TODAY'S CONSTANTS, SO THE CLAIM IS NOT LEFT AS REASONING OR CARRIED OVER FROM AN
|
||||
// OLDER ONE: replacing this line with `return volume` leaves this module's suite fully GREEN —
|
||||
// 398/398, `vitest run --project 'unit*' src/server/services/bot-account-detection`. Note the
|
||||
// denominator is that selection, not the whole app suite. That mutant SURVIVES, and it survives
|
||||
// because it is semantically equivalent at these constants, not because the tests are thin —
|
||||
// which is the difference this comment exists to record.
|
||||
//
|
||||
// 🔴 IT IS STILL KILLABLE BY A BURST BOUNDARY, AND AN EARLIER VERSION OF THIS COMMENT SAID
|
||||
// OTHERWISE. Measured at `BURST_ZERO_AT = 0` WITH ITS LITERAL PIN UPDATED IN THE SAME EDIT — the
|
||||
// condition is load-bearing, because a bare constant edit also fails the pin and gives 12/7
|
||||
// instead — this module's suite fails 11 cases against this `max` and 6 against the mutant. Two
|
||||
// different numbers either way, so the control that proves this guard reachable still exists.
|
||||
// What DID narrow is which boundary works: while the burst pair stays no steeper than the volume
|
||||
// pair (see `BURST_ONE_AT`), the two are semantically identical and the mutant survives, so a
|
||||
// tighter `BURST_ONE_AT` alone no longer separates them. Nothing here should be read as claiming
|
||||
// the burst arm is covered by the score at the shipped constants.
|
||||
//
|
||||
// ⚠️ ALL FOUR NUMBERS ABOVE ARE SUITE-SIZE-DEPENDENT AND HAVE GONE STALE ONCE ALREADY, inside the
|
||||
// change that wrote them: deleting one redundant case moved the `max` column by one and left the
|
||||
// mutant column alone, because that case failed under `max` and passed under the mutant. Re-run
|
||||
// the four arms rather than adjusting them by hand — the claim that matters is that the two
|
||||
// columns DIFFER, not the literals.
|
||||
score: ({ member, signals }) => {
|
||||
const { volume, burst } = assetStagingHalfScores(member.userId, signals);
|
||||
return Math.max(volume, burst);
|
||||
|
||||
@@ -330,21 +330,31 @@ export async function runBotAccountDetection(
|
||||
// 🔴 THIS COMMENT USED TO NAME A QUESTION THESE COUNTERS CAN NO LONGER ANSWER, AND THE CORRECTION
|
||||
// MATTERS BECAUSE THE OLD SENTENCE READ AS COVERAGE. It said they were here to settle "whether the
|
||||
// same-second half ever fires on an account the volume half did not already carry". Since the
|
||||
// firing point moved to two that has a known answer — NEVER — and it is known by arithmetic
|
||||
// rather than by measurement: a same-second group is a subset of the staged rows, so a burst of
|
||||
// two implies a count of at least two, and both halves now share boundaries (see `BURST_ONE_AT`).
|
||||
// `fired_burst > 0 && fired_volume == 0` is unreachable, so a run reporting it is a defect in the
|
||||
// evidence fold, not a finding about accounts. Leaving the old sentence would have had someone
|
||||
// watch a counter for a signal that cannot arrive and read its silence as an answer.
|
||||
// firing point moved to two that has a known answer — NEVER. `fired_burst > 0 && fired_volume == 0`
|
||||
// is unreachable, so a run reporting it is a defect in the evidence fold, not a finding about
|
||||
// accounts. Leaving the old sentence would have had someone watch a counter for a signal that
|
||||
// cannot arrive and read its silence as an answer.
|
||||
//
|
||||
// 🔴 WHY IT IS UNREACHABLE, AND UNDER WHAT CONDITION, IS ON `BURST_ONE_AT` — DO NOT RE-DERIVE IT
|
||||
// FROM THE CONSTANTS HERE. This comment used to, and the derivation it carried ("the two halves
|
||||
// share boundaries") went stale the moment the volume boundary moved. So did the copy on the
|
||||
// constants — both said it, both were wrong, both had to be rewritten, which is the argument for
|
||||
// one derivation in one place rather than a claim that the other copy fared better.
|
||||
//
|
||||
// WHAT THEY CAN STILL SETTLE, which is why they are kept: `fired_burst` is the population of
|
||||
// accounts whose staged uploads arrived in one batch, and the question is whether THAT population
|
||||
// is actioned at a different rate than the accounts carried by volume alone. That is a grading
|
||||
// question over outcomes, answered by joining these counters to moderation results — not by
|
||||
// either counter on its own. If the answer is "no different", the burst arm has no reason to
|
||||
// exist and the honest edit is to delete it; if it separates, that is the evidence for giving it
|
||||
// a tighter boundary than the volume half again, which is the only thing that would make it
|
||||
// affect a score.
|
||||
// exist and the honest edit is to delete it.
|
||||
//
|
||||
// 🔴 IF IT SEPARATES, TIGHTENING `BURST_ONE_AT` IS NOT THE FIX — AND THIS PARAGRAPH SAID IT WAS
|
||||
// UNTIL THE VOLUME RAMP BECAME A STEP. Such an edit changes no SCORE. It is not silent, though,
|
||||
// and saying "it ships green" would send someone to read a guard firing as designed as an
|
||||
// unrelated break: the two assertions that read the burst half AT 0.5 go red, which is exactly
|
||||
// what they are for. (Other assertions read that half at 0 or 1 and are unmoved — the qualifier
|
||||
// is what makes the count right.) What reviving the arm takes instead is on `BURST_ONE_AT` and on
|
||||
// the `max` in `assetStagingHeuristic`.
|
||||
//
|
||||
// Counted over EVERY scored member, matching `fired`'s own population. They may sum to MORE than
|
||||
// `fired` — an account can be both, and attributing it to whichever won a `>` comparison would
|
||||
|
||||
@@ -424,9 +424,10 @@ export const MIN_REPORTED_CONFIDENCE = 0.1125;
|
||||
*
|
||||
* So: PROVISIONAL. It is carried forward unchanged because changing it would move the reported
|
||||
* population of every heuristic at once, which is a decision that wants its own evidence and its
|
||||
* own change — not a side effect of adding a signal. `asset-staging`'s boundaries are derived
|
||||
* AGAINST this number (see `STAGED_ONE_AT`), so it is load-bearing for that heuristic's firing
|
||||
* point; that makes it more important to be honest about its provenance, not less. The
|
||||
* own change — not a side effect of adding a signal. `asset-staging`'s boundaries are CHECKED
|
||||
* against this number (see `STAGED_ONE_AT` — they were derived from it until the ordering evidence
|
||||
* set the volume boundary instead), so it still bounds that heuristic's firing point; that makes it
|
||||
* more important to be honest about its provenance, not less. The
|
||||
* `confidence_bucket_*` counters are what can eventually replace it with a measured value.
|
||||
*/
|
||||
export const LONE_SIGNAL_CUT = 0.45;
|
||||
|
||||
Reference in New Issue
Block a user