feat(fts): add cross-column scorer composition

[BubbleCal]

Feature

Each FTS index source scores in its own partition-local DocId domain. Cross-column compound queries need to compose those scorers in one globally ordered row-address domain without assuming independently built indices share local document layouts.

This PR adds the cross-column scorer core:

• validates and caches strictly ordered row-address projections
• maps local scorer iteration, shallow bounds, and advances into row-address order
• materializes an exact fallback for reordered projections
• rejects duplicate row-address mappings that could combine content from different local documents
• lazily merges multiple physical sources for one semantic leaf
• preserves conservative bounds, competitive floors, and equal-score row-address ties
• analyzes compound plans for positive generator coverage, cost, feasibility, and conservative score bounds
• supports staged positive generation followed by candidate-scoped required, optional, and prohibited probing
• performs one exact global top-k collection after composing Match, Phrase, Boolean, MultiMatch, and Boost scorers
• uses candidate-scoped row-address and scoring-length reads when staging is selective

The dataset Scanner does not call this core yet; that integration remains in the final consumer PR.

API and compatibility

There is no file-format change.

This PR exports the low-level async Rust entry point used by the later lance planner integration:

pub async fn cross_column_compound_search(
    columns: &[(String, Vec<Arc<InvertedIndex>>)],
    query: &FtsQuery,
    params: &FtsSearchParams,
    prefilter: Arc<dyn PreFilter>,
    metrics: Arc<dyn MetricsCollector>,
) -> Result<(Vec<u64>, Vec<f32>)>

The API requires a bounded limit, preserves exact (score DESC, row_address ASC) ordering, and returns an error for unsupported or internally inconsistent scorer state.

Existing PartitionDocuments::resolve_addresses and estimated_address_read_bytes behavior remains unchanged from main, so current production cache and I/O behavior is not altered before Scanner integration.

CI failure addressed

The previous head split scorer foundations from their production consumer, leaving 43 groups of private items unused under -D dead-code. This revision folds the cross-column core into the same PR so those components have real production call paths.

Unused leaf-role metadata was deleted. No allow(dead_code), test-only gating, or visibility workaround was added.

Scope boundary

This is PR 3 of the OSS-1603 stack and builds on #8666 and #8667, both merged.

It does not include the previously deferred same-column delayed MUST_NOT probing work from OSS-1705. Cross-column prohibited clauses are supported as ordinary query semantics, but this PR does not change the same-column BooleanScorer or its probing strategy.

Validation

• cargo fmt --all -- --check
• git diff --check origin/main...HEAD
• deterministic projection tests for ordered, deleted, remapped, duplicate, and out-of-order layouts
• materialized fallback and collision regressions
• merge-scorer ordering, lazy initialization, advance, shallow-bound, floor, tie, and duplicate tests
• seeded randomized exhaustive oracle coverage for SUM, MAX, MUST, required-optional, signed Boost, and nested prohibited shapes
• staged generator, candidate resolution, phrase, visibility, and quantized-length tests
• static reachability audit covering every item reported by the failed CI jobs

Per project workflow, GitHub CI is the authoritative Cargo test and Clippy run for this revised head.

Performance validation

No end-to-end performance benefit is claimed yet because the dataset Scanner does not call this entry point in this PR. A main-vs-this-PR dataset benchmark would execute the existing fallback path and measure noise.

The final consumer PR will enable this core and report independent warm/cold ABBA results with latency, throughput, CPU, bytes, requests, cache metrics, and exact result digests.

Stack

1. fix(fts): preserve exact wand score bounds #8666 — WAND scoring and bound exactness (merged)
2. refactor(fts): add exact posting load policies #8667 — posting loading and cache policy (merged)
3. This PR: row-address and cross-column compound scorer core
4. Dataset planner / execution integration and end-to-end benchmark

Part of OSS-1603.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[lance-gatekeeper]

⚠️ Gate recommendation: approve with a non-blocking risk.

The document-granularity mismatch is fixed at the row-only boundary, with Match, Phrase, and MultiMatch validation coverage. The remaining risk is regression coverage: supported cross-column scoring has strong component/property evidence and a successful index-backed end-to-end probe, but the repository still lacks a committed test through the full orchestration path. A staged-versus-eager differential test in the consumer integration would close that gap.