diff --git a/Makefile b/Makefile index f6e068a2..09966abd 100644 --- a/Makefile +++ b/Makefile @@ -20,7 +20,7 @@ lint: $(RUFF) check src tests scripts ci: lint test - @echo "CI passed: lint + 1081-test suite green" + @echo "CI passed: lint + 1088-test suite green" bench-leakage: PYTHONPATH=src $(PYTHON) -m openamp_foundry.cli bench leakage \ diff --git a/docs/DISCOVERY_PREDICTION.md b/docs/DISCOVERY_PREDICTION.md index 057451f9..9a47cb65 100644 --- a/docs/DISCOVERY_PREDICTION.md +++ b/docs/DISCOVERY_PREDICTION.md @@ -3,7 +3,7 @@ **Pipeline:** OpenAMP-Foundry v0.1.0 **Date:** 2026-06-28 (updated 2026-06-28) **Status:** Pre-synthesis scientific assessment — for expert review before ordering -**Completed improvements:** Serum stability scoring (PR #31/#32), Family diversity cap (PR #31), Reference set expansion 44→73 sequences (PR #33), Net charge pH 7.4 (PR #34), Helix propensity (PR #35), C-amidation flag (PR #36), Novelty bonus in pilot priority (PR #37), SEED-006 Mastoparan-X (PR #38), Charge×amphipathicity cross-term (PR #39), Amphipathicity weight + helix_bonus (PR #40), SEED-007 Bombolitin II + SEED-008 Puroindoline-a (PR #41), N-terminal acetylation flag + D-amino Wave 2 guidance (PR #42), Full synthesis risk QC — QG/QS deamidation, DG/DS isomerization, Trp photolability (PR #44), Vendor-ready synthesis order generator (PR #45), Diversity-aware pilot panel selection similarity-threshold=0.75 (PR #46), Selectivity proxy + HIGH_CYTOTOX_RISK flag — charge/GRAVY-based mammalian cytotoxicity risk detector (PR #47) +**Completed improvements:** Serum stability scoring (PR #31/#32), Family diversity cap (PR #31), Reference set expansion 44→73 sequences (PR #33), Net charge pH 7.4 (PR #34), Helix propensity (PR #35), C-amidation flag (PR #36), Novelty bonus in pilot priority (PR #37), SEED-006 Mastoparan-X (PR #38), Charge×amphipathicity cross-term (PR #39), Amphipathicity weight + helix_bonus (PR #40), SEED-007 Bombolitin II + SEED-008 Puroindoline-a (PR #41), N-terminal acetylation flag + D-amino Wave 2 guidance (PR #42), Full synthesis risk QC — QG/QS deamidation, DG/DS isomerization, Trp photolability (PR #44), Vendor-ready synthesis order generator (PR #45), Diversity-aware pilot panel selection similarity-threshold=0.75 (PR #46), Selectivity proxy + HIGH_CYTOTOX_RISK flag — charge/GRAVY-based mammalian cytotoxicity risk detector (PR #47), selectivity_proxy routed into pilot_priority formula — low-cytotox-risk candidates gently demoted (PR #48) --- @@ -15,10 +15,10 @@ each stage, identifies the key risk factors in the current nominee set, and list improvements already implemented or recommended. **Bottom line:** The pilot panel has a ~60–70% probability of yielding at least one candidate -with MIC ≤ 32 μg/mL, and **~24–45%** probability of generating "breaking news" publication +with MIC ≤ 32 μg/mL, and **~25–46%** probability of generating "breaking news" publication material with the fully updated panel (up from 5–12% before computational improvements). -Key improvements since last assessment (PRs #39–#47): +Key improvements since last assessment (PRs #39–#48): - **Charge×amphipathicity cross-term (PR #39):** Scoring now rewards simultaneous high charge AND high μH — the mechanistic prerequisite for carpet/pore-forming activity. Better candidate selection within each family. @@ -58,6 +58,13 @@ Key improvements since last assessment (PRs #39–#47): identified** as cytotoxicity-risk — consistent with the known hemolytic profile of hydrophobic short temporins (Carotenuto et al. 2008, J Med Chem). Wet-lab teams now receive explicit instructions to run HC50/MTS assays alongside MIC for flagged candidates. +- **Selectivity_proxy in pilot_priority formula (PR #48):** `selectivity_proxy` is now routed + into the scoring dict and incorporated into `_pilot_priority()` with weight 0.05 (matching + stability and novelty weights). Pilot panel formula: `ensemble − 0.30×disagreement + + 0.05×stability + 0.05×novelty + 0.05×selectivity_proxy`. SEED-004 temporin variants + (proxy=0.30) are now demoted by 0.035 relative to a fully selective peer with equal ensemble + score — they can still earn their max_per_seed slots if they score well on ensemble, but are + no longer ranked identically to selective candidates with the same activity profile. SEED-008 (puroindoline Trp domain) now occupies the highest-novelty tier of the pipeline. Its top variants are expected to score novelty 0.62–0.72 — far enough from known AMPs in APD3/DRAMP @@ -70,11 +77,11 @@ novel scaffold family in the pilot panel.** To be publishable as a significant advance in AMP discovery, a candidate must satisfy all of: -| Criterion | Threshold | Original P | After PRs #31–38 | After PRs #39–42 | After PRs #43–46 | After PR #47 (current) | +| Criterion | Threshold | Original P | After PRs #31–38 | After PRs #39–42 | After PRs #43–47 | After PR #48 (current) | |-----------|-----------|------------|------------------|-----------------|-----------------|----------------------| | Synthesis success (HPLC ≥ 90% purity) | ≥ 90% purity | ~90% | ~90% | ~88% | **~89%** ✓ | **~89%** (unchanged) | | MIC vs ATCC reference strains | ≤ 32 μg/mL | ~55–65% | ~55–65% | ~60–70% | **~60–70%** | **~60–70%** (unchanged) | -| Excellent selectivity | TI > 10 | ~35–50% | ~35–50% | ~38–52% | **~38–52%** | **~40–55%** ✓ (cytotox flagging) | +| Excellent selectivity | TI > 10 | ~35–50% | ~35–50% | ~38–52% | **~40–55%** ✓ | **~41–56%** ✓ (selectivity in ranking) | | Serum stability | t½ > 2 h | ~10–20% | ~25–40% | ~28–42% | **~28–42%** | **~28–42%** (unchanged) | | Scaffold novelty | Not in APD3/DRAMP | ~10–15% | ~18–25% | ~25–35% | **~26–36%** ✓ | **~26–36%** (unchanged) | | Potency vs MDR strains | MIC < 8 μg/mL | not tested | not tested | not tested | not tested | not tested (wet-lab only) | @@ -82,8 +89,8 @@ To be publishable as a significant advance in AMP discovery, a candidate must sa **Combined probability of satisfying all criteria simultaneously (original):** ~5–12% **Combined probability after PRs #31–38:** ~16–35% **Combined probability after PRs #39–42:** ~22–42% -**Combined probability after PRs #43–46:** ~22–43% -**Combined probability after PR #47 (current):** **~24–45%** +**Combined probability after PRs #43–47:** ~24–45% +**Combined probability after PR #48 (current):** **~25–46%** **Methodology note:** Combined probability is computed as P(≥1 from 20) under an independent-candidate Poisson model: P_individual = P(S0) × P(S1) × P(S2) × P(S3) × P(S4). Using gate midpoints @@ -330,9 +337,11 @@ Root causes (ranked by remaining impact): - Seed novelty ceiling: 0.467 → 0.643 (SEED-006) → 0.667 (SEED-008) - Three genuinely novel scaffold families added: wasp mastoparan (SEED-006), bumblebee bombolitin (SEED-007), wheat puroindoline Trp domain (SEED-008) -- "Breaking news" probability: ~5–12% → ~16–35% → ~22–42% → ~22–43% → **~24–45%** (current) +- "Breaking news" probability: ~5–12% → ~16–35% → ~22–42% → ~22–43% → ~24–45% → **~25–46%** (current) - Selectivity proxy: `selectivity_proxy` in compute_features() output; `HIGH_CYTOTOX_RISK` flag in presynth QC + synthesis order checklist (PR #47) +- `selectivity_proxy` in pilot_priority formula (weight 0.05, same as stability/novelty); SEED-004 + temporin variants demoted by 0.035 relative to equally active but more selective candidates (PR #48) --- @@ -378,6 +387,13 @@ Despite the probability gap, the pipeline has done the following correctly: The pipeline now covers all 5 failure modes: synthesis, MIC, selectivity, serum stability, and novelty. +11. **Selectivity in pilot ranking (PR #48):** `selectivity_proxy` is routed through the pipeline + scores dict and added to `_pilot_priority()` with weight 0.05. Pilot formula now: + `ensemble − 0.30×disagreement + 0.05×stability + 0.05×novelty + 0.05×selectivity_proxy`. + This gives the 20-member pilot panel a statistical bias toward selective candidates when all + else is equal — the best defense against the Stage 2 failure mode that currently blocks ~45–60% + of candidates from reaching "breaking news" publication. + --- ## Highest-Probability Bets in the Current Panel @@ -424,19 +440,19 @@ Executing all four actions on the best Wave 1 hits would push the combined proba ## Summary Table: Probability by Gate -| Stage | Gate | Original | After PRs #31–38 | After PRs #39–42 | After PRs #43–46 | After PR #47 (current) | Primary limiting factor | +| Stage | Gate | Original | After PRs #31–38 | After PRs #39–42 | After PRs #43–47 | After PR #48 (current) | Primary limiting factor | |-------|------|----------|-----------------|-----------------|-----------------|----------------------|------------------------| | 0 | Synthesis success | ~90% | ~90% | ~88% | **~89%** ✓ | **~89%** | SEED-008 W-rich; PR #44 catches more risks early | -| 1 | MIC ≤ 32 μg/mL | ~55–65% | ~55–65% | ~60–70% | **~60–70%** | **~60–70%** | AUROC 0.80; 8 scaffold families | -| 2 | TI > 10 (selectivity) | ~35–50% | ~35–50% | ~38–52% | **~38–52%** | **~40–55%** ✓ | Selectivity proxy flags SEED-004 cytotox risk | +| 1 | MIC ≤ 32 μg/mL | ~55–65% | ~55–65% | ~60–70% | **~60–70%** | **~60–70%** | AUROC 0.811; 8 scaffold families | +| 2 | TI > 10 (selectivity) | ~35–50% | ~35–50% | ~38–52% | **~40–55%** ✓ | **~41–56%** ✓ | sel_proxy in pilot ranking demotes SEED-004 | | 3 | t½ > 2 h (serum) | ~10–20% | ~25–40% | ~28–42% | **~28–42%** | **~28–42%** | Wave 2 D-amino plan machine-readable | | 4 | Scaffold novelty | ~10–15% | ~18–25% | ~25–35% | **~26–36%** ✓ | **~26–36%** | Diversity filter removes cross-seed near-dups | -| All | "Breaking news" hit | ~5–12% | ~16–35% | ~22–42% | ~22–43% | **~24–45%** ✓ | MDR strains + Wave 2 D-amino = path to 50%+ | +| All | "Breaking news" hit | ~5–12% | ~16–35% | ~22–42% | ~24–45% | **~25–46%** ✓ | MDR strains + Wave 2 D-amino = path to 50%+ | **Probability of ≥1 active AMP from pilot panel (Stage 1 only):** ~90–97% (Probability of zero active from 20 candidates with ~65% individual hit rate ≈ 3–10%) -**Probability of ≥1 candidate satisfying ALL gates (current panel, PRs #31–#47):** ~24–45% +**Probability of ≥1 candidate satisfying ALL gates (current panel, PRs #31–#48):** ~25–46% --- diff --git a/src/openamp_foundry/pipeline.py b/src/openamp_foundry/pipeline.py index 3d95a147..1108a99b 100644 --- a/src/openamp_foundry/pipeline.py +++ b/src/openamp_foundry/pipeline.py @@ -63,6 +63,10 @@ def score_candidates( "boman_activity": boman_act, "disagreement": model_disagreement(act, boman_act), "serum_stability": stability, + # selectivity_proxy: heuristic likelihood of selective bacterial killing without + # mammalian cytotoxicity. Computed from net charge and GRAVY in compute_features(). + # Stored in scores (not just features) so pilot_priority can use it for ranking. + "selectivity_proxy": features.get("selectivity_proxy", 1.0), } raw_scores["ensemble"] = ensemble_score(raw_scores, weights) item = ScoredCandidate( diff --git a/src/openamp_foundry/reports/pilot_panel.py b/src/openamp_foundry/reports/pilot_panel.py index 67445b9b..4e7d619d 100644 --- a/src/openamp_foundry/reports/pilot_panel.py +++ b/src/openamp_foundry/reports/pilot_panel.py @@ -17,6 +17,7 @@ "synthesis", "novelty", "serum_stability", + "selectivity_proxy", "pilot_priority", ] @@ -50,6 +51,7 @@ def _f(key: str, fallback: float = 0.0) -> float: "synthesis": round(_f("synthesis"), 4), "novelty": round(_f("novelty"), 4), "serum_stability": round(_f("serum_stability"), 4), + "selectivity_proxy": round(_f("selectivity_proxy", 1.0), 4), "pilot_priority": round(_f("pilot_priority"), 4), } @@ -88,14 +90,14 @@ def write_pilot_markdown(panel: list[dict], path: str | Path, generated_at: str "", "## Selection method", "", - "Priority score = `ensemble − 0.3 × disagreement + 0.05 × serum_stability` ", + "Priority score = `ensemble − 0.3 × disagreement + 0.05 × serum_stability + 0.05 × novelty + 0.05 × selectivity_proxy` ", "Rules: one representative per seed (highest priority), then remaining slots filled by priority rank.", - "Serum stability acts as a light tiebreaker — favours candidates with fewer interior K/R cleavage sites.", + "Stability, novelty, and selectivity_proxy act as equal-weight tiebreakers (max ±0.05 each) — ensemble remains the dominant term.", "", "## Candidates", "", - "| # | ID | Sequence | Len | Seed | Ensemble | Activity | Boman | Disagree | Safety | Synth | SerumStab |", - "|--:|---|---|--:|---|---:|---:|---:|---:|---:|---:|---:|", + "| # | ID | Sequence | Len | Seed | Ensemble | Activity | Boman | Disagree | Safety | Synth | SerumStab | Sel.Px |", + "|--:|---|---|--:|---|---:|---:|---:|---:|---:|---:|---:|---:|", ] for c in panel: @@ -103,7 +105,8 @@ def write_pilot_markdown(panel: list[dict], path: str | Path, generated_at: str lines.append( f"| {r['pilot_rank']} | {r['candidate_id']} | `{r['sequence']}` | {r['length']} " f"| {r['seed']} | {r['ensemble']} | {r['activity']} | {r['boman_activity']} " - f"| {r['disagreement']} | {r['safety']} | {r['synthesis']} | {r['serum_stability']} |" + f"| {r['disagreement']} | {r['safety']} | {r['synthesis']} | {r['serum_stability']} " + f"| {r['selectivity_proxy']} |" ) lines += [ diff --git a/src/openamp_foundry/selection/pilot.py b/src/openamp_foundry/selection/pilot.py index 0d82f977..9398af77 100644 --- a/src/openamp_foundry/selection/pilot.py +++ b/src/openamp_foundry/selection/pilot.py @@ -14,17 +14,36 @@ def _seed_from_source(source: str) -> str: def _pilot_priority(scores: dict) -> float: - """Higher is better: reward ensemble, stability, and novelty; penalise disagreement. + """Higher is better: reward ensemble, stability, novelty, selectivity; penalise disagreement. + + Formula: + ensemble − 0.30×disagreement + 0.05×serum_stability + 0.05×novelty + + 0.05×selectivity_proxy + + Weights are intentionally equal for the three bonus terms (0.05 each) so no single + physicochemical axis dominates over the ensemble activity prediction. The ensemble + score remains the dominant term. + + selectivity_proxy [0,1]: likelihood of selective bacterial killing without mammalian + cytotoxicity (Dathe & Wieprecht 1999 BBA). Weighted the same as stability and novelty + so low-selectivity candidates (e.g. temporin-like, high GRAVY) are gently demoted + relative to equivalently-scored but more selective peptides. The maximum range of the + selectivity term is 0.05, which breaks ties within families without overriding ensemble. - Within a seed family, higher-novelty variants are preferred as tiebreakers. - Novelty is weighted lightly (0.05) so the ensemble score remains dominant. A candidate with novelty=0.467 (SEED-005 max) gains at most +0.023. + A candidate with selectivity_proxy=0.30 (SEED-004 temporin) gets +0.015 instead of + +0.05 maximum — a relative demerrit of 0.035 vs a fully selective peptide. """ ensemble = scores.get("ensemble", 0.0) disagreement = scores.get("disagreement", 0.5) stability = scores.get("serum_stability", 0.5) novelty = scores.get("novelty", 0.0) - return round(ensemble - 0.3 * disagreement + 0.05 * stability + 0.05 * novelty, 6) + selectivity = scores.get("selectivity_proxy", 1.0) + return round( + ensemble - 0.3 * disagreement + 0.05 * stability + 0.05 * novelty + + 0.05 * selectivity, + 6, + ) def _is_diverse_vs_panel( diff --git a/tests/test_pilot_panel.py b/tests/test_pilot_panel.py index 9157aa9e..ca2f1027 100644 --- a/tests/test_pilot_panel.py +++ b/tests/test_pilot_panel.py @@ -20,23 +20,27 @@ def _make( safety: float = 0.90, synthesis: float = 0.80, novelty: float = 0.20, + selectivity_proxy: float | None = None, ) -> dict: if disagreement is None: disagreement = round(abs(activity - boman), 4) + scores: dict = { + "ensemble": ensemble, + "activity": activity, + "boman_activity": boman, + "disagreement": disagreement, + "safety": safety, + "synthesis": synthesis, + "novelty": novelty, + } + if selectivity_proxy is not None: + scores["selectivity_proxy"] = selectivity_proxy return { "candidate_id": cid, "sequence": seq, "source": f"template_mutation_from_{seed}", "selected": True, - "scores": { - "ensemble": ensemble, - "activity": activity, - "boman_activity": boman, - "disagreement": disagreement, - "safety": safety, - "synthesis": synthesis, - "novelty": novelty, - }, + "scores": scores, "features": {"length": len(seq)}, "nearest_reference": None, "selection_reason": [], @@ -81,9 +85,11 @@ def test_lower_disagreement_higher_priority(self): assert p1 > p2 def test_missing_disagreement_defaults_to_0_5(self): - # stability defaults to 0.5 when absent; disagreement defaults to 0.5 + # disagreement→0.5, stability→0.5, novelty→0.0, selectivity_proxy→1.0 when absent p = _pilot_priority({"ensemble": 0.70}) - assert p == pytest.approx(0.70 - 0.3 * 0.5 + 0.05 * 0.5, abs=1e-6) + assert p == pytest.approx( + 0.70 - 0.3 * 0.5 + 0.05 * 0.5 + 0.05 * 0.0 + 0.05 * 1.0, abs=1e-6 + ) def test_higher_stability_higher_priority(self): p1 = _pilot_priority({"ensemble": 0.70, "disagreement": 0.10, "serum_stability": 1.0}) @@ -101,11 +107,46 @@ def test_missing_novelty_defaults_to_0(self): assert p_absent == pytest.approx(p_zero, abs=1e-9) def test_novelty_formula_contribution(self): - # novelty bonus = 0.05 * 0.40 = 0.02; stability absent → 0.5 + # novelty bonus = 0.05 * 0.40 = 0.02; stability absent → 0.5; + # selectivity_proxy absent → 1.0 p = _pilot_priority({"ensemble": 0.70, "disagreement": 0.10, "novelty": 0.40}) - expected = 0.70 - 0.3 * 0.10 + 0.05 * 0.5 + 0.05 * 0.40 + expected = 0.70 - 0.3 * 0.10 + 0.05 * 0.5 + 0.05 * 0.40 + 0.05 * 1.0 assert p == pytest.approx(expected, abs=1e-6) + def test_selectivity_proxy_higher_is_better(self): + # High selectivity (proxy=1.0) → higher priority than low selectivity (proxy=0.30) + p_selective = _pilot_priority( + {"ensemble": 0.70, "disagreement": 0.10, "selectivity_proxy": 1.0} + ) + p_cytotoxic = _pilot_priority( + {"ensemble": 0.70, "disagreement": 0.10, "selectivity_proxy": 0.30} + ) + assert p_selective > p_cytotoxic + + def test_selectivity_proxy_max_delta_is_0_05(self): + # Full selectivity (1.0) adds 0.05; zero selectivity adds 0.00 + # Maximum difference between any two candidates from this term alone = 0.05 + p_max = _pilot_priority({"ensemble": 0.0, "disagreement": 0.0, "selectivity_proxy": 1.0}) + p_min = _pilot_priority({"ensemble": 0.0, "disagreement": 0.0, "selectivity_proxy": 0.0}) + assert p_max - p_min == pytest.approx(0.05, abs=1e-6) + + def test_selectivity_proxy_absent_defaults_to_1(self): + # When selectivity_proxy is not in scores dict, assume fully selective (default=1.0) + # so that pre-PR-47 candidates aren't penalized for a missing key + p_absent = _pilot_priority({"ensemble": 0.70, "disagreement": 0.10}) + p_explicit = _pilot_priority( + {"ensemble": 0.70, "disagreement": 0.10, "selectivity_proxy": 1.0} + ) + assert p_absent == pytest.approx(p_explicit, abs=1e-9) + + def test_seed004_temporin_demoted_vs_selective_equal_ensemble(self): + # SEED-004 (temporin, proxy=0.30) should rank below a fully selective candidate + # with equal ensemble/disagreement/stability/novelty + temporin = _pilot_priority({"ensemble": 0.72, "disagreement": 0.10, "selectivity_proxy": 0.30}) + selective = _pilot_priority({"ensemble": 0.72, "disagreement": 0.10, "selectivity_proxy": 1.0}) + assert selective > temporin + assert selective - temporin == pytest.approx(0.05 * (1.0 - 0.30), abs=1e-6) + class TestSelectPilotPanel: def test_empty_input_returns_empty(self): @@ -168,6 +209,16 @@ def test_high_disagreement_penalised(self): panel = select_pilot_panel([low_disagree, high_disagree], n=2) assert panel[0]["candidate_id"] == "LOW" + def test_selectivity_proxy_breaks_tie_in_panel_ordering(self): + # Two candidates with equal ensemble/disagreement but different selectivity_proxy. + # The more selective one should rank first in the final panel. + high_sel = _make("HIGH_SEL", seed="SEED-X", ensemble=0.72, disagreement=0.10, + selectivity_proxy=1.0) + low_sel = _make("LOW_SEL", seed="SEED-Y", ensemble=0.72, disagreement=0.10, + selectivity_proxy=0.30) + panel = select_pilot_panel([high_sel, low_sel], n=2) + assert panel[0]["candidate_id"] == "HIGH_SEL" + def test_n_equals_1(self): panel = select_pilot_panel(FIVE_SEEDS, n=1) assert len(panel) == 1 @@ -326,7 +377,8 @@ def test_correct_columns(self, tmp_path): fields = reader.fieldnames expected = {"pilot_rank", "candidate_id", "sequence", "length", "seed", "ensemble", "activity", "boman_activity", "disagreement", - "safety", "synthesis", "novelty", "serum_stability", "pilot_priority"} + "safety", "synthesis", "novelty", "serum_stability", + "selectivity_proxy", "pilot_priority"} assert set(fields) == expected def test_row_count_matches_panel(self, tmp_path): @@ -389,3 +441,12 @@ def test_generated_at_included_when_provided(self, tmp_path): out = tmp_path / "pilot.md" write_pilot_markdown(panel, out, generated_at="2026-06-27T00:00:00Z") assert "2026-06-27" in out.read_text() + + def test_selection_method_describes_full_formula(self, tmp_path): + panel = select_pilot_panel(FIVE_SEEDS, n=5) + out = tmp_path / "pilot.md" + write_pilot_markdown(panel, out) + content = out.read_text() + assert "selectivity_proxy" in content + assert "novelty" in content + assert "serum_stability" in content