fix(fts): chunk posting reads during segment merge

[BubbleCal]

What is the bug?

FTS segment merge materializes every posting row in an invert.lance partition as one Arrow RecordBatch. Large partitions can exceed the i32 child-offset capacity of the posting, impact, or position List columns while Arrow concatenates pages, causing an offset overflow panic after indexing has otherwise completed.

What issues or incorrect behavior does the bug cause?

Large segmented FTS rebuilds can finish tokenization but fail while finalizing intermediate segments. Because the final index segment is never committed, callers cannot complete the rebuild and automated maintenance can encounter the same deterministic failure again.

How does this PR fix the problem?

• replace the partition-wide posting read in InvertedPartition::into_builder with sequential token-range chunks
• reuse the existing posting chunk decoder so each batch is released before the next range is read
• let merge slices share the current chunk Arrow buffers while prewarm retains compact independently-owned buffers
• bound current shared-position chunks by token count, an approximately 128 MiB on-disk target, and the exact widest List<i32> child count derived from posting metadata
• read compressed V1 per-document positions one token at a time because their nested position-block child count is not stored in metadata
• return a descriptive error if one metadata-bounded token alone cannot fit within the configured child-offset limit

The pre-compression row-based legacy layout keeps its existing row-count fallback and is outside this incident scope.

Token ordering, posting contents, position data, impact data, public APIs, and the on-disk format remain unchanged.

Tests

• cargo test -p lance-index test_into_builder_chunks_postings_by_list_children -- --nocapture
• cargo test -p lance-index test_chunk_posting_mode_controls_buffer_sharing -- --nocapture
• cargo test -p lance-index test_prewarm_streams_in_chunks -- --nocapture
• cargo test -p lance-index test_merge_segments_ -- --nocapture
• cargo check -p lance-index --tests
• cargo fmt --all -- --check
• git diff --check main...HEAD
• cargo clippy -p lance-index --tests -- -D warnings -A clippy::single-range-in-vec-init -A clippy::implicit-clone

Strict local Clippy on Rust 1.97 currently reports pre-existing single-range-in-vec-init and implicit-clone findings in untouched files; allowing only those two lints makes the changed package pass cleanly.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[lance-gatekeeper]

✅ Gate recommendation: approve.

The current-format planner retains exact metadata-based batching and buffer sharing, while compressed V1 LegacyPerDoc reads now use the conservative one-token boundary required by their nested-list layout. The focused coverage verifies both paths.