fix(merge_meta): Surface ALL competing proposals against existing entities

Two bugs in cmd_prepare_merge's grouped_new_entities loop where src is
already a glossary surface (last unfixed multi-variant collapse path):

1. owner_role == 'source': code only inspected proposals[0]'s
   target_proposal against canonical. If the first proposal happened to
   match canonical and a later proposal differed, no decision was emitted
   but every chunk was hashed as consumed → competing translations lost
   forever. Category differences were never considered at all (the branch
   compared targets only).

   Fix: build distinct (target, category) variants from all proposals,
   filter out ones matching canonical, emit one new `existing_entity_conflict`
   decision listing every differing variant. Choices: keep_current /
   use_variant_N (overwrites BOTH target and category, stamps prior values
   into notes) / record_in_notes (canonical unchanged; all variants
   logged). If every proposal matched canonical, silent no-op as before.

2. owner_role == 'alias': code took proposals[0]'s target/category as the
   sole promotion option. Multiple competing promotion targets (Apple → 苹果
   fruit vs Apple → 苹果公司 company) collapsed to the first; once the
   orchestrator picked promote_to_separate_entity the alternates were
   permanently consumed.

   Fix: new_entity_existing_alias now carries `promoted_variants` (parallel
   to alias_or_new_entity.standalone_variants). Choices: use_variant_N
   (one per distinct promotion variant) / keep_as_alias / skip. Old
   `promote_to_separate_entity` choice removed in favor of `use_variant_0`
   for the single-variant case.

apply-merge dispatches both new shapes; pre-validation accepts the dynamic
choice sets. _is_creator updated for the new use_variant_N branch in
new_entity_existing_alias.

8 new tests across both fixes (matches-canonical surfaced, category-only
diff surfaced, all-match silent no-op, multi-variant promotion exposed,
correct variant selected, etc.). Three existing tests updated for the new
choice/field names. SKILL.md updated. Suite 168 green.
This commit is contained in:
deusyu
2026-04-22 15:55:16 +08:00
parent 2c34bbefc5
commit 70c7e06a3d
3 changed files with 323 additions and 53 deletions
+2 -1
View File
@@ -218,7 +218,8 @@ Each sub-agent emitted an `output_chunk<NNNN>.meta.json` alongside its translate
- `decisions_needed` — items requiring main-agent judgment. Each has `id`, `kind`, an `options` array, and the data needed to pick. Kinds:
- `alias` — `{variant, candidate_source, evidence}`. Choices: `yes_alias` / `no_separate_entity` / `skip`.
- `conflict` — `{entity_source, field, current, proposed, evidence}`. Choices: `keep_current` / `accept_proposed` / `record_in_notes`.
- `new_entity_existing_alias` — `{proposed_source, currently_alias_of, proposed_target, proposed_category, evidence}`. Choices: `promote_to_separate_entity` / `keep_as_alias` / `skip`.
- `new_entity_existing_alias` — sub-agents propose `proposed_source` as a new entity, but it's already someone's alias. `{proposed_source, currently_alias_of, promoted_variants: [{target_proposal, category, evidence, evidence_chunks}, ...]}`. Choices: one `use_variant_N` per distinct (target, category) promotion variant (promote `proposed_source` to standalone with that target+category, removing it from the host's aliases) / `keep_as_alias` / `skip`.
- `existing_entity_conflict` — sub-agents proposed a (target, category) for `entity_source` that differs from the canonical. Multiple distinct differing proposals all get exposed. `{entity_source, current_target, current_category, proposed_variants: [{target_proposal, category, evidence, evidence_chunks}, ...]}`. Choices: `keep_current` / one `use_variant_N` per competing proposal (overwrites both target AND category, stamps the prior values into notes) / `record_in_notes` (canonical unchanged; every proposed variant gets logged to notes).
- `alias_or_new_entity` — `variant` has multiple competing options that can't all coexist under v2's surface-form uniqueness rule. Triggered when (a) `variant` was proposed both as a new standalone entity AND as an alias of one or more candidates, OR (b) `variant` was proposed as an alias of two or more different candidates with no standalone competitor. `{variant, alias_candidates: [{candidate_source, evidence, evidence_chunks}, ...], standalone_variants: [{target_proposal, category, evidence, evidence_chunks}, ...]}`. Choices: one `use_alias_N` per candidate (attach as alias of that candidate), one `use_standalone_N` per competing standalone proposal (add as standalone with that target+category), or `skip`.
- `conflicting_new_entity_proposals` — `{source, variants: [{target_proposal, category, evidence, evidence_chunks}, ...]}`. Choices: `use_variant_0`, `use_variant_1`, ..., `skip`.
- `consumed_chunk_ids` — every meta file scanned this round (regardless of whether it produced a finding). These hashes get recorded in `applied_meta_hashes` on apply.
+125 -48
View File
@@ -44,12 +44,10 @@ EVIDENCE_REFS_CAP = 5
VALID_CHOICES_BY_KIND = {
'alias': frozenset({'yes_alias', 'no_separate_entity', 'skip'}),
'conflict': frozenset({'keep_current', 'accept_proposed', 'record_in_notes'}),
'new_entity_existing_alias': frozenset(
{'promote_to_separate_entity', 'keep_as_alias', 'skip'}
),
# alias_or_new_entity and conflicting_new_entity_proposals have dynamic
# use_standalone_N / use_variant_N choices generated from variant counts;
# validated at apply time against the decision item's options array.
# alias_or_new_entity, conflicting_new_entity_proposals,
# new_entity_existing_alias, and existing_entity_conflict have dynamic
# use_*_N choices generated from variant counts; validated at apply time
# against the decision item's options array.
}
VALID_GENDER_VALUES = ('male', 'female', 'nonbinary', 'unknown')
@@ -302,33 +300,51 @@ def cmd_prepare_merge(temp_dir):
owner_id, owner_role = surface_idx[src]
owner_term = _find_term_by_id(glossary, owner_id)
if owner_role == 'source':
# Source already exists as someone's source → flag as conflict.
# Take the first proposal (sub-agent voted on a translation that
# disagrees with what's already canonical).
p = proposals[0]
if p['target_proposal'] != owner_term['target']:
# src is already canonical. For each distinct (target, category)
# proposal across the batch that DIFFERS from the canonical
# (target, category), expose it as a competing variant. If
# every proposal matches canonical, silent no-op.
canonical_pair = (
owner_term['target'],
owner_term.get('category', ''),
)
proposed_variants = _standalone_variants_for(proposals)
differing = [
v for v in proposed_variants
if (v['target_proposal'], v['category']) != canonical_pair
]
if differing:
options = (
['keep_current']
+ [f'use_variant_{i}' for i in range(len(differing))]
+ ['record_in_notes']
)
decisions_needed.append({
'id': _new_decision_id(),
'kind': 'conflict',
'kind': 'existing_entity_conflict',
'entity_source': src,
'field': 'target',
'current': owner_term['target'],
'proposed': p['target_proposal'],
'evidence': p['evidence'],
'options': ['keep_current', 'accept_proposed', 'record_in_notes'],
'current_target': owner_term['target'],
'current_category': owner_term.get('category', ''),
'proposed_variants': differing,
'options': options,
})
# else: identical target — nothing to do.
# else: every proposal matched canonical — no decision needed.
else: # role == 'alias'
p = proposals[0]
# src is currently another term's alias; sub-agents propose to
# promote it. Expose every distinct (target, category) variant
# so the orchestrator can pick the right promoted form.
promoted_variants = _standalone_variants_for(proposals)
options = (
[f'use_variant_{i}' for i in range(len(promoted_variants))]
+ ['keep_as_alias', 'skip']
)
decisions_needed.append({
'id': _new_decision_id(),
'kind': 'new_entity_existing_alias',
'proposed_source': src,
'currently_alias_of': owner_id,
'proposed_target': p['target_proposal'],
'proposed_category': p['category'],
'evidence': p['evidence'],
'options': ['promote_to_separate_entity', 'keep_as_alias', 'skip'],
'promoted_variants': promoted_variants,
'options': options,
})
elif len(target_cat_pairs) == 1:
# All proposals agree → auto_apply with combined evidence.
@@ -556,6 +572,30 @@ def cmd_apply_merge(temp_dir):
f"must be one of {sorted(valid)}"
)
continue
if kind == 'new_entity_existing_alias':
promoted_variants = d.get('promoted_variants') or []
valid = (
{'keep_as_alias', 'skip'}
| {f'use_variant_{i}' for i in range(len(promoted_variants))}
)
if choice not in valid:
pre_errors.append(
f"decision {d_id!r} (kind={kind}): invalid choice {choice!r}, "
f"must be one of {sorted(valid)}"
)
continue
if kind == 'existing_entity_conflict':
proposed_variants = d.get('proposed_variants') or []
valid = (
{'keep_current', 'record_in_notes'}
| {f'use_variant_{i}' for i in range(len(proposed_variants))}
)
if choice not in valid:
pre_errors.append(
f"decision {d_id!r} (kind={kind}): invalid choice {choice!r}, "
f"must be one of {sorted(valid)}"
)
continue
valid = VALID_CHOICES_BY_KIND.get(kind)
if valid is None:
pre_errors.append(f"decision {d_id!r}: unknown kind {kind!r}")
@@ -664,7 +704,7 @@ def cmd_apply_merge(temp_dir):
kind = d['kind']
choice = d['choice']
if kind == 'new_entity_existing_alias':
return choice == 'promote_to_separate_entity'
return choice.startswith('use_variant_')
if kind == 'alias_or_new_entity':
return choice.startswith('use_standalone_')
if kind == 'conflicting_new_entity_proposals':
@@ -731,31 +771,68 @@ def cmd_apply_merge(temp_dir):
return True, None
if kind == 'new_entity_existing_alias':
if choice == 'promote_to_separate_entity':
proposed_source = d.get('proposed_source')
host_id = d.get('currently_alias_of')
proposed_target = d.get('proposed_target')
proposed_category = d.get('proposed_category', '')
evidence = d.get('evidence', '')
host = _find_term_by_id(glossary, host_id)
if host is None:
return False, (
f"decision {d_id!r}: host term id={host_id!r} not in glossary"
promoted_variants = d.get('promoted_variants') or []
if choice in ('keep_as_alias', 'skip'):
return True, None
m = re.match(r'^use_variant_(\d+)$', choice)
idx = int(m.group(1))
chosen = promoted_variants[idx]
proposed_source = d.get('proposed_source')
host_id = d.get('currently_alias_of')
host = _find_term_by_id(glossary, host_id)
if host is None:
return False, (
f"decision {d_id!r}: host term id={host_id!r} not in glossary"
)
if proposed_source in host.get('aliases', []):
host['aliases'] = [a for a in host['aliases'] if a != proposed_source]
combined_chunks = sorted({
cid for v in promoted_variants for cid in v.get('evidence_chunks', [])
})[:EVIDENCE_REFS_CAP]
glossary['terms'].append({
'id': proposed_source,
'source': proposed_source,
'target': chosen['target_proposal'],
'category': chosen.get('category', ''),
'aliases': [],
'gender': 'unknown',
'confidence': _confidence_for_evidence_count(len(combined_chunks)),
'frequency': 0,
'evidence_refs': combined_chunks,
'notes': f'promoted from alias of {host_id!r}',
})
return True, None
if kind == 'existing_entity_conflict':
entity_source = d.get('entity_source')
term = _find_term_by_surface(glossary, entity_source)
if term is None:
return False, (
f"decision {d_id!r}: entity_source {entity_source!r} not in glossary"
)
if choice == 'keep_current':
return True, None
proposed_variants = d.get('proposed_variants') or []
if choice == 'record_in_notes':
for v in proposed_variants:
_append_note(
term,
f"[conflict] target={v['target_proposal']!r} "
f"category={v['category']!r} evidence={v['evidence']!r}",
)
if proposed_source in host.get('aliases', []):
host['aliases'] = [a for a in host['aliases'] if a != proposed_source]
glossary['terms'].append({
'id': proposed_source,
'source': proposed_source,
'target': proposed_target,
'category': proposed_category,
'aliases': [],
'gender': 'unknown',
'confidence': 'low',
'frequency': 0,
'evidence_refs': [],
'notes': f'promoted from alias of {host_id!r}; evidence={evidence!r}',
})
return True, None
m = re.match(r'^use_variant_(\d+)$', choice)
idx = int(m.group(1))
chosen = proposed_variants[idx]
old_target = term.get('target')
old_category = term.get('category', '')
term['target'] = chosen['target_proposal']
term['category'] = chosen.get('category', '')
_append_note(
term,
f"[updated] target {old_target!r}{chosen['target_proposal']!r}, "
f"category {old_category!r}{chosen.get('category', '')!r}",
)
return True, None
if kind == 'alias_or_new_entity':
+196 -4
View File
@@ -193,8 +193,10 @@ class PrepareMergeBasicTests(unittest.TestCase):
self.assertEqual(d['kind'], 'new_entity_existing_alias')
self.assertEqual(d['proposed_source'], 'Apple')
self.assertEqual(d['currently_alias_of'], 'Banana')
self.assertEqual(len(d['promoted_variants']), 1)
self.assertEqual(d['promoted_variants'][0]['target_proposal'], '苹果')
self.assertEqual(set(d['options']),
{'promote_to_separate_entity', 'keep_as_alias', 'skip'})
{'use_variant_0', 'keep_as_alias', 'skip'})
def test_flags_alias_hypothesis_when_candidate_exists(self):
existing = make_term('Tai', '太一', 'person')
@@ -566,7 +568,7 @@ class ApplyMergeDecisionTests(unittest.TestCase):
d = out['decisions_needed'][0]
run_apply_merge(tmp, {
'auto_apply': [],
'decisions': [{**d, 'choice': 'promote_to_separate_entity'}],
'decisions': [{**d, 'choice': 'use_variant_0'}],
'consumed_chunk_ids': out['consumed_chunk_ids'],
})
g = glossary_mod.load_glossary(os.path.join(tmp, 'glossary.json'))
@@ -1249,7 +1251,7 @@ class CandidateNotCanonicalizedTests(unittest.TestCase):
'auto_apply': [],
'decisions': [
{**alias_d, 'choice': 'yes_alias'},
{**promote_d, 'choice': 'promote_to_separate_entity'},
{**promote_d, 'choice': 'use_variant_0'},
],
'consumed_chunk_ids': out['consumed_chunk_ids'],
})
@@ -1321,7 +1323,7 @@ class DispatchOrderIndependenceTests(unittest.TestCase):
'auto_apply': [],
'decisions': [
{**alias_d, 'choice': 'yes_alias'},
{**promote_d, 'choice': 'promote_to_separate_entity'},
{**promote_d, 'choice': 'use_variant_0'},
],
'consumed_chunk_ids': out['consumed_chunk_ids'],
})
@@ -1520,6 +1522,196 @@ class MultiAliasCandidateTests(unittest.TestCase):
self.assertEqual(out2['decisions_needed'], [])
class ExistingEntityConflictTests(unittest.TestCase):
"""Bug fix: when src is already a glossary source, multi-chunk new_entity
proposals must surface every distinct (target, category) variant that
differs from canonical — not collapse to proposals[0]."""
def test_first_proposal_matches_canonical_other_differs_still_surfaces(self):
# chunk0001 repeats canonical; chunk0002 proposes a different target.
# Old code looked at proposals[0] (matches), did silent no-op, but
# consumed both chunks → chunk0002's signal lost.
existing = make_term('Tai', '太一', 'person')
m1 = empty_meta(new_entities=[{
'source': 'Tai', 'target_proposal': '太一', 'category': 'person',
'evidence': 'matches.',
}])
m2 = empty_meta(new_entities=[{
'source': 'Tai', 'target_proposal': '', 'category': 'person',
'evidence': 'differs.',
}])
with temp_workspace(glossary=make_glossary(existing),
metas={'chunk0001': m1, 'chunk0002': m2}) as tmp:
out, _ = run_prepare_merge(tmp)
self.assertEqual(len(out['decisions_needed']), 1)
d = out['decisions_needed'][0]
self.assertEqual(d['kind'], 'existing_entity_conflict')
self.assertEqual(d['entity_source'], 'Tai')
# Only the differing variant is in proposed_variants.
self.assertEqual(len(d['proposed_variants']), 1)
self.assertEqual(d['proposed_variants'][0]['target_proposal'], '')
def test_category_only_difference_surfaces(self):
# Same target, different category — must surface.
existing = make_term('Apple', '苹果', 'fruit')
m = empty_meta(new_entities=[{
'source': 'Apple', 'target_proposal': '苹果', 'category': 'company',
'evidence': 'Apple Inc.',
}])
with temp_workspace(glossary=make_glossary(existing),
metas={'chunk0001': m}) as tmp:
out, _ = run_prepare_merge(tmp)
self.assertEqual(len(out['decisions_needed']), 1)
d = out['decisions_needed'][0]
self.assertEqual(d['kind'], 'existing_entity_conflict')
self.assertEqual(d['proposed_variants'][0]['category'], 'company')
def test_all_proposals_match_canonical_silent_noop(self):
# If every proposal matches canonical, no decision but chunks consumed.
existing = make_term('Tai', '太一', 'person')
m1 = empty_meta(new_entities=[{
'source': 'Tai', 'target_proposal': '太一', 'category': 'person',
'evidence': 'a.',
}])
m2 = empty_meta(new_entities=[{
'source': 'Tai', 'target_proposal': '太一', 'category': 'person',
'evidence': 'b.',
}])
with temp_workspace(glossary=make_glossary(existing),
metas={'chunk0001': m1, 'chunk0002': m2}) as tmp:
out, _ = run_prepare_merge(tmp)
self.assertEqual(out['decisions_needed'], [])
self.assertEqual(set(out['consumed_chunk_ids']), {'chunk0001', 'chunk0002'})
def test_multi_distinct_variants_all_surfaced(self):
existing = make_term('Tai', '太一', 'person')
m1 = empty_meta(new_entities=[{
'source': 'Tai', 'target_proposal': '泰一', 'category': 'person',
'evidence': 'a.',
}])
m2 = empty_meta(new_entities=[{
'source': 'Tai', 'target_proposal': '', 'category': 'place',
'evidence': 'b.',
}])
with temp_workspace(glossary=make_glossary(existing),
metas={'chunk0001': m1, 'chunk0002': m2}) as tmp:
out, _ = run_prepare_merge(tmp)
d = out['decisions_needed'][0]
self.assertEqual(d['kind'], 'existing_entity_conflict')
self.assertEqual(len(d['proposed_variants']), 2)
self.assertEqual(set(d['options']),
{'keep_current', 'use_variant_0', 'use_variant_1', 'record_in_notes'})
def test_apply_use_variant_updates_target_and_category(self):
existing = make_term('Tai', '太一', 'person')
m = empty_meta(new_entities=[{
'source': 'Tai', 'target_proposal': '', 'category': 'place',
'evidence': '...',
}])
with temp_workspace(glossary=make_glossary(existing),
metas={'chunk0001': m}) as tmp:
out, _ = run_prepare_merge(tmp)
d = out['decisions_needed'][0]
run_apply_merge(tmp, {
'auto_apply': [],
'decisions': [{**d, 'choice': 'use_variant_0'}],
'consumed_chunk_ids': out['consumed_chunk_ids'],
})
g = glossary_mod.load_glossary(os.path.join(tmp, 'glossary.json'))
tai = next(t for t in g['terms'] if t['source'] == 'Tai')
self.assertEqual(tai['target'], '')
self.assertEqual(tai['category'], 'place')
self.assertIn('updated', tai['notes'])
def test_apply_record_in_notes_preserves_canonical(self):
existing = make_term('Tai', '太一', 'person')
m1 = empty_meta(new_entities=[{
'source': 'Tai', 'target_proposal': '泰一', 'category': 'person', 'evidence': 'a.',
}])
m2 = empty_meta(new_entities=[{
'source': 'Tai', 'target_proposal': '泰二', 'category': 'person', 'evidence': 'b.',
}])
with temp_workspace(glossary=make_glossary(existing),
metas={'chunk0001': m1, 'chunk0002': m2}) as tmp:
out, _ = run_prepare_merge(tmp)
d = out['decisions_needed'][0]
run_apply_merge(tmp, {
'auto_apply': [],
'decisions': [{**d, 'choice': 'record_in_notes'}],
'consumed_chunk_ids': out['consumed_chunk_ids'],
})
g = glossary_mod.load_glossary(os.path.join(tmp, 'glossary.json'))
tai = next(t for t in g['terms'] if t['source'] == 'Tai')
self.assertEqual(tai['target'], '太一') # canonical preserved
# Both observations recorded in notes.
self.assertIn('泰一', tai['notes'])
self.assertIn('泰二', tai['notes'])
class NewEntityExistingAliasMultiVariantTests(unittest.TestCase):
"""Bug fix: when src is an existing alias of host, multi-chunk promotion
proposals must surface every distinct (target, category) variant — not
just proposals[0]."""
def test_multi_variant_promotion_surfaces_all_proposals(self):
# Banana has alias Apple. chunk0001 says Apple → 苹果 (fruit);
# chunk0002 says Apple → 苹果公司 (company). Both must be exposed.
existing = make_term('Banana', '香蕉', aliases=['Apple'])
m1 = empty_meta(new_entities=[{
'source': 'Apple', 'target_proposal': '苹果', 'category': 'fruit',
'evidence': 'Apple is red.',
}])
m2 = empty_meta(new_entities=[{
'source': 'Apple', 'target_proposal': '苹果公司', 'category': 'company',
'evidence': 'Apple Inc.',
}])
with temp_workspace(glossary=make_glossary(existing),
metas={'chunk0001': m1, 'chunk0002': m2}) as tmp:
out, _ = run_prepare_merge(tmp)
self.assertEqual(len(out['decisions_needed']), 1)
d = out['decisions_needed'][0]
self.assertEqual(d['kind'], 'new_entity_existing_alias')
self.assertEqual(d['proposed_source'], 'Apple')
self.assertEqual(len(d['promoted_variants']), 2)
targets = {v['target_proposal'] for v in d['promoted_variants']}
self.assertEqual(targets, {'苹果', '苹果公司'})
self.assertEqual(set(d['options']),
{'use_variant_0', 'use_variant_1', 'keep_as_alias', 'skip'})
self.assertEqual(set(out['consumed_chunk_ids']),
{'chunk0001', 'chunk0002'})
def test_multi_variant_apply_picks_correct_promoted_form(self):
existing = make_term('Banana', '香蕉', aliases=['Apple'])
m1 = empty_meta(new_entities=[{
'source': 'Apple', 'target_proposal': '苹果', 'category': 'fruit',
'evidence': 'a.',
}])
m2 = empty_meta(new_entities=[{
'source': 'Apple', 'target_proposal': '苹果公司', 'category': 'company',
'evidence': 'b.',
}])
with temp_workspace(glossary=make_glossary(existing),
metas={'chunk0001': m1, 'chunk0002': m2}) as tmp:
out, _ = run_prepare_merge(tmp)
d = out['decisions_needed'][0]
company_idx = next(
i for i, v in enumerate(d['promoted_variants'])
if v['category'] == 'company'
)
run_apply_merge(tmp, {
'auto_apply': [],
'decisions': [{**d, 'choice': f'use_variant_{company_idx}'}],
'consumed_chunk_ids': out['consumed_chunk_ids'],
})
g = glossary_mod.load_glossary(os.path.join(tmp, 'glossary.json'))
# Banana lost the alias; new Apple created with company target/category.
banana = next(t for t in g['terms'] if t['source'] == 'Banana')
self.assertEqual(banana['aliases'], [])
apple = next(t for t in g['terms'] if t['source'] == 'Apple')
self.assertEqual(apple['target'], '苹果公司')
self.assertEqual(apple['category'], 'company')
class AliasChainTests(unittest.TestCase):
"""Bug fix: alias hypotheses can chain through other pending alias decisions.
chunk0001: Taig → Tai (Tai in glossary). chunk0002: Taighi → Taig (Taig