feat: resolve stable file format to 2.2 - #8657
Conversation
There was a problem hiding this comment.
❌ Gate recommendation: request changes.
The 2.2 stable/default promotion is valid, but all Java paths that pair writer output with manually supplied format metadata must advance together. Align the operation helper with the new default so merge and data-replacement transactions keep a single exact file version.
| /// Exact v2.2. | ||
| /// Exact v2.2 and the current default. | ||
| #[default] | ||
| V2_2, |
There was a problem hiding this comment.
Changing the omitted-version writer default here leaves OperationTestBase.writeLanceDataFile inconsistent: it calls LanceFileWriter.open(..., null), which now writes 2.2, but still returns DataFile metadata with TEST_FILE_FORMAT_MINOR_VERSION = 1. The resulting fragment mixes 2.2 and 2.1 metadata and is rejected. Update the helper so the writer selection and DataFile version come from the same 2.2 source (the previous 2.0→2.1 promotion updated this same constant).
Reproducer
./mvnw -Djava.io.tmpdir=/home/agent/tmp/lance-java.nBZtTr '-Dtest=MergeTest#testMergeNewColumn' test
Observed: IllegalArgumentException: All data files must have the same version. Detected both 2.2 and 2.1.
Lance 2.2 is the current stable file format, but the centralized release policy and enum default still resolve new datasets to 2.1. As a result, the
stableselector and default writes lag behind the intended stable format.Resolve both paths to 2.2 and align the Java integration expectations. Lance 2.3 remains the
nextunstable format.