Use cursor copies for PiecewiseSequence slices#8832
Conversation
Merging this PR will regress 2 benchmarks
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
9adf56d to
c4a0358
Compare
f9a3472 to
afcc04a
Compare
c4a0358 to
c76531d
Compare
80f7b89 to
30870aa
Compare
30870aa to
e777ab4
Compare
afcc04a to
1a17030
Compare
5924fe3 to
79d9e10
Compare
1a17030 to
534b70f
Compare
79d9e10 to
39ee98e
Compare
3be0c74 to
e60ec21
Compare
83c466d to
df091fb
Compare
423876b to
7034fd3
Compare
e31db3e to
9f02349
Compare
7034fd3 to
bdfe8f7
Compare
9f02349 to
43ecd40
Compare
bdfe8f7 to
9c21b5e
Compare
43ecd40 to
97a2e5a
Compare
9c21b5e to
b2069a5
Compare
97a2e5a to
4626cfc
Compare
497ff39 to
5ca0608
Compare
4626cfc to
9eba049
Compare
5ca0608 to
bfd4825
Compare
9eba049 to
3c5e519
Compare
bfd4825 to
1af09ed
Compare
| /// This is useful when the caller knows the final output length up front and wants to avoid | ||
| /// repeatedly growing the initialized length while copying contiguous source slices. | ||
| pub struct SpareBufferWriter<'a, T> { | ||
| buffer: &'a mut BufferMut<T>, |
There was a problem hiding this comment.
I think you just want here &'a mut [T] unless I am missing something
There was a problem hiding this comment.
ah no you need to setlen
| /// | ||
| /// This is useful when the caller knows the final output length up front and wants to avoid | ||
| /// repeatedly growing the initialized length while copying contiguous source slices. | ||
| pub struct SpareBufferWriter<'a, T> { |
There was a problem hiding this comment.
I think before we put this in vortex-buffer we would need to benchmark this against other approaches. We can keep it internal to the take impl and figure out a right abstraction from there. My suggestions were not fully tested
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
3c5e519 to
5d31540
Compare
9183265 to
182567a
Compare
Replace the raw-pointer cursor with a written-count cursor over spare_capacity_mut, and remove copy_slice_unchecked along with the unsafe blocks at the four PiecewiseSequence call sites. The bounds check is one well-predicted branch per slice: benchmarking the gather loop over 16M u64 elements shows checked copies within noise of unchecked ones at every run length (212ms vs 200ms even at the pathological run length of 1, where both beat extend_from_slice's 299ms), so the unchecked variant's extra unsafe surface isn't paying for anything. The writer is now #[must_use], and the remaining unsafety is contained to the copy into the bounds-checked spare-capacity range (the not-yet-stable <[MaybeUninit<T>]>::write_copy_of_slice) and the single set_len in finish, justified by the writer's contiguous-prefix invariant. Signed-off-by: "Robert" <robert@spiraldb.com> Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GUSY34eRKq9du5zQK1FuZq
The writer only exists to serve the PiecewiseSequence gather loops, so keep it pub(crate) in arrays::piecewise_sequence as its own submodule rather than growing vortex-buffer's public API. It only uses BufferMut's public surface, so nothing in vortex-buffer changes behavior. Signed-off-by: "Robert" <robert@spiraldb.com> Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GUSY34eRKq9du5zQK1FuZq
Summary
extend_from_slicein the primitive, decimal, VarBinView, and VarBin PiecewiseSequence consumersset_lenValidation
cargo +nightly fmt --allcargo clippy -p vortex-array --all-targets --all-features -- -D warningscargo bench -p vortex-array --bench listview_rebuildcargo bench -p vortex-array --bench cast_decimalcargo test -p vortex-array piecewise_sequence