New-library scorer status

Date: 2026-10-02 (iteration-4 update; scorer sections updated 2026-10-07 when all four scorers were run on the new libraries). Every number below comes from the shipped frontend/public/data/library/<slug>/stats.json, the files under data/domains/<slug>/reasoning/ (including coverage.json and the hand-audit file), or data/spend_ledger.json.

The reasoning engine has five scorers. "Skipped" is not the same as "ran and produced zero".

Summary

"Pairs" are shortlisted claim pairs. Diet and ML-fairness: cosine similarity ≥ 0.80, at most 2 candidates per claim (their documented settings). LLM-cal used its own Phase-3 settings (≥ 0.78, up to 10 per claim; docs/relationship-layer.md), so the counts are not comparable across libraries.

What iteration 4 changed

• Diet: the 58 shortlisted pairs left unclassified after iteration 3 were classified (58 calls, ₹16.36 from the ledger). 3 new flags, all one paper pair on triglycerides and coronary heart disease; hand audit set all three aside (1 artifact, 2 duplicates). Headline unchanged at 5.
• ML-fairness: the last 27 papers were extracted through the batch API (28 batch results including one retry, ₹72.75), then the 29 new shortlisted pairs were classified (₹7.77). 0 flags.
• Both libraries' disagreement checks are now complete for their full extraction sets.

Persistent limitations

Since 2026-10-07 this is the full Phase-4 scorer, the same code and thresholds as LLM calibration (at least 3 independent papers report the same own-work limitation category). Each result is published as a gap card.

Orphaned future work

Ran on diet-and-mortality on 2026-10-07 (two-stage matcher: cosine shortlist, then a language-model check of each candidate pair). Skipped for ml-fairness: its projected cost, doubled as the safety gate requires, did not fit the remaining budget. See multi-domain.md §8.

Structural holes

Ran on both new libraries on 2026-10-07 with topic groups scaled to library size (k = 10) and the leads checked by a language model (at most 15 per library). Counts are in the table above; why they are low is in multi-domain.md §8.

ML-fairness impossibility-result papers (full coverage)

Pairs touching these papers at the documented settings, from the saved verdict files:

Friedler et al.'s paper was never compared with anything: none of its claims reached the similarity threshold against another paper's claims. Why the classifier flags no contradictions in this library is not established. That fairness disagreements are definitional rather than empirical is a hypothesis, not a finding.