perf(code-index): delta-seal reuse via generation-independent identities (#800)

[ScriptedAlchemy]

Summary

• Completes perf(code-index): make incremental capture and seal delta-proportional #800's remaining seal residue: store generation-independent symbol_identities on each partitioned file-segment descriptor and rebind occurrences at restore, so one-file seal reuse no longer SHA-256-rebounds every unchanged file's symbols.
• Bumps SEALED_GENERATION_FORMAT_REVISION_V1 10 → 11. Revision 10 abstains as SupersededSealedGenerationRevision (rebuild); never silent migrate.
• Builds on the existing chore(campaign): wip snapshot 20260915T204220Z on agent/800-delta-seal (merged onto current base tip).

Motivation

Issue #800: one-file sync still did repository-sized work in the seal reuse gate via rematerialized_symbol_occurrence_id (316k digests at 4,160 files in prior Hotpath). Capture/segment publish were already delta-proportional; this is the remaining encode-side design slice.

Changes

• PartitionedFileSegmentDescriptorV1: persist symbol_identities; symbol_occurrences is restore-only (#[serde(skip)]).
• Reuse gate: capture-row equality + exact sorted unique identity key-list equality (no per-symbol rematerialization digest).
• SEALED_GENERATION_FORMAT_REVISION_V1 = 11 with typed refusal coverage for revision 10.
• Fixture digest update for the format bump; reused-descriptor stability assertion on the one-edit encode path.

Hotpath evidence (small-scale, this host)

Lane	Unchanged files	Published file segments	domain.canonical.sha256 calls
baseline 46698151a	10	1	50
baseline 46698151a	40	1	200
this PR	10	1	1
this PR	40	1	1

Treatment is constant across 4× repo growth. Logs under agent store internal/long-tail/800/.

Test plan

• production_orchestration::partitioned_encode_publishes_only_the_edited_file_segment
• production_orchestration::partitioned_* including stable bytes / rev-10 refusal
• Full production_orchestration:: (49) + sealed_generation_restore:: (5)
• format_gate_accepts_only_the_monolithic_and_partitioned_revisions
• Hotpath one-edit sha256 count (10/40 unchanged)
• Independent change-risk review: ship (no defects)
• Exact-head CI green
• Codex review threads drained to 0 (currently empty)
• Merge into codex/tracedecay-total-redesign-plan-reopened (not master); close perf(code-index): make incremental capture and seal delta-proportional #800 with evidence

Notes

• Hauler is not installed in this VM; cargo ran directly with isolated CARGO_TARGET_DIR / TRACEDECAY_DATA_DIR (never ~/.tracedecay).
• 1k/10k scorecard still needs the large host; this PR lands the fail-closed format slice + focused Hotpath proof.
• Do not touch chore(release): release 0.1.0 #745 / master.

  

[changeset-bot]

⚠️ No Changeset found

Latest commit: 46d4f05

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[ScriptedAlchemy]

Drain: HOLD pending focused delta-seal / partitioned_codec verify on exact head c2a12faaf0 after tip moved. Non-trivial codec change — not merging blind.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-16T00:21:33.532876Z	46d4f05	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[github-actions]

Performance Comparison codex/tracedecay-total-redesign-plan-reopened → agent/800-delta-seal

Total Elapsed Time: 7.75s → 7.76s (+0.1%)
CPU Baseline: 89.90µs → 85.83µs (-4.5%)
Benchmark ID: index-bench-timing

timing - Execution duration of functions.

+------------------------------------------+----------------------------+--------------------------------+--------------------------------+--------------------------------+------------------------------+
| Function                                 | Calls                      | Avg                            | P95                            | Total                          | % Total                      |
+------------------------------------------+----------------------------+--------------------------------+--------------------------------+--------------------------------+------------------------------+
| tracedecay-index-bench                   | 1 → 1 (+0.0%)              | 7.75s → 7.76s (+0.1%)          | 7.76s → 7.76s (+0.0%)          | 7.75s → 7.76s (+0.1%)          | 100.00% → 100.00% (+0.0%)    |
+------------------------------------------+----------------------------+--------------------------------+--------------------------------+--------------------------------+------------------------------+
| domain.canonical.sha256                  | 85787 → 85787 (+0.0%)      | 34.09µs → 34.90µs (+2.4%)      | 106.17µs → 111.10µs (+4.6%)    | 2.92s → 2.99s (+2.4%)          | 37.72% → 38.58% (+2.3%)      |
+------------------------------------------+----------------------------+--------------------------------+--------------------------------+--------------------------------+------------------------------+
| code_index.workers.install               | 81 → 81 (+0.0%)            | 31.64ms → 32.16ms (+1.6%)      | 161.48ms → 161.22ms (-0.2%)    | 2.56s → 2.61s (+2.0%)          | 33.06% → 33.57% (+1.5%)      |
+------------------------------------------+----------------------------+--------------------------------+--------------------------------+--------------------------------+------------------------------+
| code_index.build.and_publish             | 2 → 2 (+0.0%)              | 723.74ms → 735.15ms (+1.6%)    | 1.00s → 1.01s (+1.0%)          | 1.45s → 1.47s (+1.4%)          | 18.67% → 18.95% (+1.5%)      |
+------------------------------------------+----------------------------+--------------------------------+--------------------------------+--------------------------------+------------------------------+
| query.artifact.finalization.advance_wake | 28 → 28 (+0.0%)            | 52.93ms → 51.24ms (-3.2%)      | 398.72ms → 399.77ms (+0.3%)    | 1.48s → 1.43s (-3.4%)          | 19.12% → 18.49% (-3.3%)      |
+------------------------------------------+----------------------------+--------------------------------+--------------------------------+--------------------------------+------------------------------+
| query.artifact.batch.sqlite              | 5 → 5 (+0.0%)              | 277.92ms → 278.47ms (+0.2%)    | 319.03ms → 319.29ms (+0.1%)    | 1.39s → 1.39s (+0.0%)          | 17.92% → 17.94% (+0.1%)      |
+------------------------------------------+----------------------------+--------------------------------+--------------------------------+--------------------------------+------------------------------+
| code_index.extract.parser_artifact       | 276 → 276 (+0.0%)          | 4.06ms → 4.10ms (+1.0%)        | 6.57ms → 6.63ms (+0.9%)        | 1.12s → 1.13s (+0.9%)          | 14.45% → 14.57% (+0.8%)      |
+------------------------------------------+----------------------------+--------------------------------+--------------------------------+--------------------------------+------------------------------+
| query.artifact.batch.postings            | 5 → 5 (+0.0%)              | 218.55ms → 219.36ms (+0.4%)    | 249.17ms → 249.95ms (+0.3%)    | 1.09s → 1.10s (+0.9%)          | 14.09% → 14.13% (+0.3%)      |
+------------------------------------------+----------------------------+--------------------------------+--------------------------------+--------------------------------+------------------------------+
| query.artifact.batch.parallel_prepare    | 5 → 5 (+0.0%)              | 185.60ms → 184.78ms (-0.4%)    | 219.28ms → 216.27ms (-1.4%)    | 928.02ms → 923.88ms (-0.4%)    | 11.97% → 11.90% (-0.6%)      |
+------------------------------------------+----------------------------+--------------------------------+--------------------------------+--------------------------------+------------------------------+
| domain.canonical.json_bytes              | 206681 → 206681 (+0.0%)    | 4.17µs → 4.25µs (+1.9%)        | 4.42µs → 4.50µs (+1.8%)        | 862.78ms → 879.22ms (+1.9%)    | 11.13% → 11.33% (+1.8%)      |
+------------------------------------------+----------------------------+--------------------------------+--------------------------------+--------------------------------+------------------------------+
| code_index.restore.file_admit            | 260 → 260 (+0.0%)          | 3.04ms → 3.11ms (+2.3%)        | 4.77ms → 4.85ms (+1.7%)        | 791.02ms → 807.70ms (+2.1%)    | 10.20% → 10.41% (+2.1%)      |
+------------------------------------------+----------------------------+--------------------------------+--------------------------------+--------------------------------+------------------------------+
| code_index.build.materialize_full        | 1 → 1 (+0.0%)              | 724.46ms → 732.92ms (+1.2%)    | 724.57ms → 732.95ms (+1.2%)    | 724.46ms → 732.92ms (+1.2%)    | 9.34% → 9.44% (+1.1%)        |
+------------------------------------------+----------------------------+--------------------------------+--------------------------------+--------------------------------+------------------------------+
| code_index.collect.materialize_full      | 1 → 1 (+0.0%)              | 710.17ms → 719.09ms (+1.3%)    | 710.41ms → 719.32ms (+1.3%)    | 710.17ms → 719.09ms (+1.3%)    | 9.16% → 9.27% (+1.2%)        |
+------------------------------------------+----------------------------+--------------------------------+--------------------------------+--------------------------------+------------------------------+
| query.artifact.batch.postings.ngram_rows | 5 → 5 (+0.0%)              | 126.01ms → 124.61ms (-1.1%)    | 142.74ms → 142.61ms (-0.1%)    | 630.06ms → 623.05ms (-1.1%)    | 8.13% → 8.03% (-1.2%)        |
+------------------------------------------+----------------------------+--------------------------------+--------------------------------+--------------------------------+------------------------------+
| code_index.build.assemble                | 2 → 2 (+0.0%)              | 281.17ms → 285.97ms (+1.7%)    | 287.31ms → 293.34ms (+2.1%)    | 562.33ms → 571.94ms (+1.7%)    | 7.25% → 7.37% (+1.7%)        |
+------------------------------------------+----------------------------+--------------------------------+--------------------------------+--------------------------------+------------------------------+

---

Generated with hotpath-rs