Skip to content

feat: resolve stable file format to 2.2 - #8657

Open
Xuanwo wants to merge 4 commits into
mainfrom
xuanwo/lance-2-2-stable
Open

feat: resolve stable file format to 2.2#8657
Xuanwo wants to merge 4 commits into
mainfrom
xuanwo/lance-2-2-stable

Conversation

@Xuanwo

@Xuanwo Xuanwo commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Lance 2.2 is the current stable file format, but the centralized release policy and enum default still resolve new datasets to 2.1. As a result, the stable selector and default writes lag behind the intended stable format.

Resolve both paths to 2.2 and align the Java integration expectations. Lance 2.3 remains the next unstable format.

@github-actions github-actions Bot added A-java Java bindings + JNI A-encoding Encoding, IO, file reader/writer bug Something isn't working labels Aug 20, 2026
@Xuanwo Xuanwo changed the title fix: resolve stable file format to 2.2 feat: resolve stable file format to 2.2 Aug 20, 2026
@Xuanwo
Xuanwo marked this pull request as ready for review August 20, 2026 09:26
@github-actions github-actions Bot added the enhancement New feature or request label Aug 20, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate recommendation: request changes.

The 2.2 stable/default promotion is valid, but all Java paths that pair writer output with manually supplied format metadata must advance together. Align the operation helper with the new default so merge and data-replacement transactions keep a single exact file version.

/// Exact v2.2.
/// Exact v2.2 and the current default.
#[default]
V2_2,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing the omitted-version writer default here leaves OperationTestBase.writeLanceDataFile inconsistent: it calls LanceFileWriter.open(..., null), which now writes 2.2, but still returns DataFile metadata with TEST_FILE_FORMAT_MINOR_VERSION = 1. The resulting fragment mixes 2.2 and 2.1 metadata and is rejected. Update the helper so the writer selection and DataFile version come from the same 2.2 source (the previous 2.0→2.1 promotion updated this same constant).

Reproducer

./mvnw -Djava.io.tmpdir=/home/agent/tmp/lance-java.nBZtTr '-Dtest=MergeTest#testMergeNewColumn' test

Observed: IllegalArgumentException: All data files must have the same version. Detected both 2.2 and 2.1.

@lance-gatekeeper lance-gatekeeper Bot added the K-changes Latest Gatekeeper recommendation requests changes. label Aug 20, 2026
@github-actions github-actions Bot added the A-python Python bindings label Aug 20, 2026
@lance-gatekeeper lance-gatekeeper Bot removed the K-changes Latest Gatekeeper recommendation requests changes. label Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-encoding Encoding, IO, file reader/writer A-java Java bindings + JNI A-python Python bindings bug Something isn't working enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant