[Bug] dirty-closed rescan misses freshly-marked episodes for a full cursor lap; interrupted drains lose phase-2 accounting #2459

Environment

• memos-local-plugin 2.0.20 (shared-bridge daemon bridge.cjs --daemon)
• DB: ~3.4k closed episodes, ~35k traces; single Hermes agent instance
• Context: recovering from a 10-day scoring outage; ~300 episodes closed with heuristic-zero scores were marked rewardDirty for retro-rescore. The dirty-rescore path itself works (single episodes rescore end-to-end: rTask + trace value write-back + downstream L2/policies/skills), which is what surfaced the two reliability gaps below.

Gap 1 — cursor-lap starvation: no newest-first pass

The scan (collectDirtyClosedEpisodes + autoRescoreDirtyClosedEpisodes; init pass + 10-minute periodic) walks closed episodes newest→oldest, one 500-row page per run, persisting a cursor in kv (pipeline.dirty_closed_scan_cursor.v1). An episode that becomes dirty (explicit marker, close-time reward failure, rTask-null orphan, traceCount mismatch) after the cursor has already paged past its position is not revisited until the walk reaches the end of the closed set and the cursor resets — a full lap of ceil(closed_episodes / 500) × 10 min. In this deployment that is ~70 min; for the 30k+-episode DBs referenced in earlier reports (e.g. #1787) it approaches ~10 h. The periodic scan was added for "episodes that miss the startup scan", but because the walk only moves away from the newest rows, newly dirtied episodes are the least likely to be seen promptly.

Today's operational symptom: after marking a single known-dirty episode, neither the init scan nor periodic ticks collected it for ~15 minutes. Only after DELETE FROM kv WHERE key='pipeline.dirty_closed_scan_cursor.v1' did the next scan collect it (and then a 304-episode batch in one page). Without manual cursor deletion, that batch would have waited up to a full lap.

Gap 2 — interrupted drain loses the accounting pass for the whole batch

recoverDirtyClosedEpisodes(episodes) emits all episode.finalized events, await handle.flush(), and only then runs phase-2: clear rewardDirty on success / bump failedAttempts + lastFailureAt for backoff. If the process shuts down mid-drain (we hit startupRecoveryCancelled via a daemon restart), phase-2 is skipped for every queued episode:

• episodes that already rescored successfully keep their rewardDirty markers → they get re-collected and re-run later (duplicate LLM spend);
• failures don't get their failedAttempts retry counters.

With a 300+ episode batch the drain stays open for 1 h+, so the exposure window is large.

Secondary observation (mechanism unconfirmed): on 2 of 4 daemon restarts performed with a pre-deleted scan cursor and pending dirty episodes, the startup classification produced no collection at all — no kv cursor write, no queue; a later 10-minute periodic collected normally. Possible race with heavy sqlite activity at boot (unproven). Compounding factor: the classification's failure path (init.orphan_scan.failed) logs at debug, so a skipped classification is invisible at default log levels (we only found it by diffing kv cursor writes).

Suggested directions (not prescriptive)

• Add a bounded newest-first pass per tick (first page from before=null, collect dirty, then continue the cursor walk), or reset the cursor when new closed rows appeared since the stored position.
• Make phase-2 incremental (settle each episode as its chain finishes), or best-effort on shutdown before cancelling.
• Raise the classification failure log from debug to warn.

Related: #1808 (origin of the retry/backoff scan), #1966 (ghost trace-count rows), #2355 (restart-orphaned rTask-null episodes), #2370 (rewardDirty lifecycle on reopen), #2382 (recovery replay).