Repository navigation
feat: diversity-aware pilot panel selection (similarity-threshold=0.75) - #46
Merged
Merged
Conversation
…pdate Adds similarity_threshold parameter to select_pilot_panel() and --similarity-threshold CLI arg to pilot-panel. When set (default 0.75 in make pilot), candidates that are too similar to an already- selected member (across any seed family) are filtered and replaced with more structurally distinct nominees. Implements _is_diverse_vs_panel helper using Levenshtein normalized similarity. 8 new tests in TestSelectPilotPanelDiversity covering: near-duplicate exclusion at 0.75 threshold, strict vs permissive thresholds, higher- priority retention, fully diverse panels unaffected, None backward compatibility, rank assignment, and max_per_seed interaction. DISCOVERY_PREDICTION.md updated: PRs #43-#46 listed, Stage 4 probability updated to 26-36% (diversity filter benefit), combined "breaking news" estimate updated to 22-43%. 1044 tests passing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--similarity-thresholdparameter added topilot-panelCLI andselect_pilot_panel()make pilotto eliminate cross-seed near-duplicatesDISCOVERY_PREDICTION.mdupdated to PRs docs: update discovery probability to ~22-42% — SEED-007/008, Wave 2 D-amino plan, cross-term scoring (#43) #43–46 stateWhat changed
src/openamp_foundry/selection/pilot.py_is_diverse_vs_panel(candidate, panel, threshold)helper usingnormalized_similarityselect_pilot_panel()gainssimilarity_threshold: float | None = Noneparameterremaindern, a log warning is emitted (pool diversity limit)similarity_threshold=None(default) is identical to the prior behaviorsrc/openamp_foundry/cli.pypilot-panelsubparser gains--similarity-thresholdfloat argument (default None)_run_pilot_panel()passessimilarity_thresholdtoselect_pilot_panel()Makefilemake pilotnow passes--similarity-threshold 0.75docs/DISCOVERY_PREDICTION.md~26–36%(diversity filter removes cross-seed near-dups)~22–43%Why this improves discovery probability
With 8 diverse seed families and conservative substitution generators, cross-seed near-duplicates are rare but possible when different families converge through parallel substitution paths. Without the filter, such pairs occupy two panel slots but provide the same structural information as one. With
threshold=0.75, the lower-priority near-duplicate is replaced by a more structurally distinct candidate, maximizing structural coverage per synthesis dollar.Expected improvement: +1–2 pp in Stage 4 (scaffold novelty gate) as the panel now guarantees structural distinctness between all 20 members.
Test plan
TestSelectPilotPanelDiversity— 8 new tests covering near-dup exclusion, threshold boundaries, priority retention, diverse panels unaffected, None backward-compat, rank assignment, max_per_seed interactionsimilarity_threshold=Nonegives identical result to no-arg call🤖 Generated with Claude Code