Skip to content

perf(fts): add WAND exactness certificate - #8703

Open
BubbleCal wants to merge 1 commit into
yang/fts-top-level-cross-column-multimatchfrom
yang/oss-2078-wand-exactness-certificate
Open

perf(fts): add WAND exactness certificate#8703
BubbleCal wants to merge 1 commit into
yang/fts-top-level-cross-column-multimatchfrom
yang/oss-2078-wand-exactness-certificate

Conversation

@BubbleCal

@BubbleCal BubbleCal commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Performance issue

The exact field-local top-level MultiMatch path introduced by #8702 uses CompoundQueryExec for every field. This preserves final (score DESC, row_id ASC) ties, but always pays inclusive score-floor and row-ID resolution costs even when the existing Match WAND result already has a strict kth score boundary.

Linear: https://linear.app/lancedb/issue/OSS-2078

Stack parent: #8702

How this PR improves performance

For eligible root Match children, this PR runs the existing global Match WAND chain with k + 1 results and classifies the final boosted scores:

  • Fewer than k + 1 results proves the field result is exhaustive.
  • A strict score[k - 1] > score[k] gap proves that the first k results cannot depend on a discarded row-ID tie.
  • An equal or unsupported boundary is ambiguous and falls back to the existing exact compound scorer.

Each MultiMatch field certifies or falls back independently. The ambiguous path reuses the same opened indices, ready prefilter, and corpus-wide BM25 scorer. The implementation does not change WAND pruning algorithms, external query APIs, or index formats.

The certificate is conservatively disabled for zero/non-finite boosts, wand_factor != 1, legacy indices, physical posting schemas without impacts, injected base scorers, zero/non-finite term weights, unbounded queries, and unsupported planner shapes. No-impact postings fall back to the existing exact compound scorer because partition-local candidate pruning cannot certify a corpus-wide score boundary.

Benchmark results (measured before review fixes)

Environment and methodology:

  • 10M-row MMLB dataset
  • Two independently indexed text columns
  • GCP c4-highmem-16
  • 8 workers
  • 64 GiB index cache
  • 250 frozen queries, k=10/100
  • 5 repetitions
  • Two isolated A1-B1-B2-A2 ABBA blocks
  • Measured parent c581b764b versus this PR 4eab6d16a
  • Current heads are 3d27cf91f / 563efebe7. They add per-field mixed fallback and reject no-impact certificate probes. The measured MMLB corpus uses impact-backed postings, so its fast path is unchanged, but these heads have not been re-benchmarked.
  • Ordered row IDs and score signatures verified before timing

Higher QPS is better; lower latency is better.

Scenario / metric Parent exact This PR Benefit
k=10 QPS 1,615 QPS 1,761 QPS 1.09x
k=10 p95 latency 11.12 ms 6.84 ms 1.63x
k=100 QPS 766 QPS 1,297 QPS 1.69x
k=100 p50 latency 9.97 ms 5.58 ms 1.79x
k=100 p95 latency 19.69 ms 10.84 ms 1.82x
Fresh-cold k=10 Q000001 679.9 ms 566.4 ms 1.20x
Fresh-cold k=100 Q000001 729.7 ms 567.9 ms 1.28x

Correctness and determinism:

  • Parent and this PR: 500 / 500 oracle cases exact.
  • Cross-build row/result differences: 0.
  • Timed signature instability: 0.

Certificate census over 250 queries:

k Field attempts Strict Exhaustive Fallbacks Queries with fallback
10 466 229 2 235 232 / 250
100 466 449 2 15 15 / 250

Metric invariant violations were zero: attempts == strict + exhaustive + fallbacks for every query.

Known tradeoff: the rare k=100 one-field-fallback cohort had about 15% worse warm p95 because it runs WAND and then exact scoring. Overall k=100 p95 improved 1.82x. Reusing first-pass WAND state for ambiguous fallback is a possible follow-up.

RSS stayed within roughly +0.5% to +3% in the fresh-cold runs.

Metrics

  • wand_exactness_certificate_attempts
  • wand_exactness_certificate_strict
  • wand_exactness_certificate_exhaustive
  • wand_exactness_certificate_fallbacks
  • wand_exactness_certificate_candidates

Tests and checks

  • Added strict, exhaustive, boundary-tie fallback, mixed-field, zero-boost, reversed-segment, and partial-coverage coverage through the real field-local MultiMatch path.
  • Added the modern V1 no-impact false-certificate corpus from review, plus mixed-impact rejection and all-impact acceptance controls.
  • cargo fmt --all -- --check
  • git diff --check
  • Pre-commit fmt and typos checks

Per the requested workflow, I did not run local cargo test or cargo clippy; CI will run them.

lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added the K-changes Latest Gatekeeper recommendation requests changes. label Aug 21, 2026
@BubbleCal
BubbleCal force-pushed the yang/oss-2078-wand-exactness-certificate branch from 4eab6d1 to a132925 Compare August 21, 2026 17:02
@lance-gatekeeper lance-gatekeeper Bot removed the K-changes Latest Gatekeeper recommendation requests changes. label Aug 21, 2026
lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added the K-changes Latest Gatekeeper recommendation requests changes. label Aug 21, 2026
@BubbleCal
BubbleCal force-pushed the yang/oss-2078-wand-exactness-certificate branch from a132925 to 563efeb Compare August 21, 2026 17:30
@github-actions github-actions Bot added the A-index Vector index, linalg, tokenizer label Aug 21, 2026
@lance-gatekeeper lance-gatekeeper Bot removed the K-changes Latest Gatekeeper recommendation requests changes. label Aug 21, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate recommendation: approve.

The previous no-impact blocker is fixed: certificate eligibility now requires every opened segment and physical partition to be impact-backed, while unsupported or mixed postings fall back to exact compound scoring. The strict/exhaustive certificate and ambiguous-boundary fallback preserve exact score and row-ID ordering across field-local, mixed-field, and partial-coverage paths.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-index Vector index, linalg, tokenizer K-approved Latest Gatekeeper recommendation permits acceptance. performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant