fix(datagen): align merge with live schema, surface close errors, tolerate drained generations

[beinan]

Three fixes the rollout store received but DatagenStore never did. Each is latent today and becomes reachable as soon as datagen sees concurrent or long-lived use.

1. Merge used the compile-time schema, not the live one

merge_own_shard built its append batch from datagen_log_schema(). A base table written by an older binary can lack columns the current schema has, and merging a batch built from the compile-time schema into it fails outright. Now aligned via align_batch_to_schema — the rollout store hit exactly this and was fixed in #175.

2. Drop swallowed the writer's close error

let _ = writer.close().await;   // before

A failed close leaks the writer's background tasks and, if the memtable was still buffered, strands rows that are durable in the WAL but never sealed — with no signal anywhere. Now logged, and the no-runtime path says so too. (Rollout equivalent: #190.)

3. A concurrently-drained generation failed the whole lookup

get_blob propagated not-found when a merge drained and deleted a generation between snapshot and open. Those rows are already in the base table, so it now skips and falls through — as the rollout store does.

Sharing

align_batch_to_schema and is_not_found_error become pub(crate) so both stores share one implementation instead of drifting again. This drift is the actual pattern here: all three bugs are cases where rollout was fixed and datagen was not.

Scope note — worth reading

I expected datagen to have the write-lock stall fixed in #199. It does not, and I verified rather than assumed:

• DatagenStore::append is already &mut self (its writer is a bare Option<ShardWriter>, not behind a mutex), so appends need the exclusive lock regardless of merge — the read/write lock split from fix(rollout): stop WAL merge from blocking concurrent appends #199 would buy nothing.
• write_with_resident_writer already retries on fence, so claim_epoch fencing its own writer is handled. Restoring claim_epoch leaves the new test green, confirming this empirically.
• Datagen is not yet wired into any server route or sweeper.

So the epoch change from #199 is deliberately not ported. These three divergences are the real defects.

Testing

New test asserts a merge does not fence the store's own resident writer, so an append immediately after a merge still succeeds and both rows stay readable, and a second merge still converges.

9 datagen tests pass; clippy --all-targets -D warnings and cargo fmt clean.

🤖 Generated with Claude Code