Skip to content

P1: FastEmbed production journeys red on the #707 tip (artifact_digest refusal, linked-worktree publish timeout) #1097

Description

@ScriptedAlchemy

Three tracedecay --lib --features test-helpers production journeys fail deterministically on the #707 tip (5c1623f) whenever TRACEDECAY_DISTRIBUTION_FASTEMBED_FIXTURE points at the byte-pinned JinaEmbeddingsV2BaseCode package. They skip (vacuously pass) without it, which is why CI and most lanes did not see them. They also fail identically on f52868364 and on every lane branch based there (ci-red-suite, freshness-1071, daemon-1040, compat-1072 — hauler cc-9918..cc-9921, 0 passed / 4 failed each), so the regression predates today's merges. The same journeys passed with the same fixture at 18203ff30 (#908 evidence).

Failing tests

  1. daemon::production_harness::semantic_activation_journey_test::public_semantic_activation_rollback_and_exact_retry_preserve_graph_authority

  2. daemon::production_harness::semantic_restart_journey_test::strict_semantic_answers_again_after_daemon_restart_without_rebuild

    Both: semantic activation input was rejected: semantic profile qualification failed: semantic snapshot preflight failed: semantic runtime rejected the verified evaluation target authority: semantic runtime rejected the transition at inspect_generation.artifact_digestProductionSemanticRuntimeV1::inspect_generation (crates/tracedecay-usecases/src/semantic_runtime/production.rs ~L2760) compares the installed lifecycle state's artifact_digest() with required.artifact_manifest_digest; they no longer agree. The fixture itself installs and verifies fine (model.verify_and_install 38 s in the stage ledger), so the disagreement is between how the evaluation target's pins are derived and how the lifecycle reports the installed artifact.

  3. daemon::production_harness::generation_retention_test::linked_worktree_scope_retention_crash_replay_and_pure_inventory_journey

    mounted linked-worktree production composition: Config { message: "production-composition code index did not publish for '<iso>/linked-b' after 20001 ms; composition gate capacity=2, admitted=1, waiting=0; scheduler_state=None; …" } — the linked worktree's code index never publishes while the primary does. May share a root cause with the stale-pins problem (the linked scope's semantic owner refusing and holding the seat) or be an independent linked-route defect; determine which.

A fourth test (semantic_availability_journey_test::retrieval_answers_before_activation_and_is_unchanged_by_live_semantic_activation) failed only because its helper read one page of a truncated tracedecay_search answer; fixed in 5c1623f.

Leads (git log 18203ff30..f52868364 --no-merges on the semantic runtime): 40847ffe6 refactor(configuration): share one runtime pin between root and usecases, 8e965d384 fix(semantic): restore from sealed source metadata, 76586f52a fix(semantic): pass the acquisition target as one value, 930b252a7 fix(semantic): retire projection cache ownership safely, and the 2f17bc83b reconcile.

Repro

TRACEDECAY_DISTRIBUTION_FASTEMBED_FIXTURE=/fast/tmp/td-753-fastembed-fixture \
cargo test -p tracedecay --lib --features test-helpers -- --test-threads=2 \
  public_semantic_activation_rollback_and_exact_retry_preserve_graph_authority \
  strict_semantic_answers_again_after_daemon_restart_without_rebuild \
  linked_worktree_scope_retention_crash_replay_and_pure_inventory_journey

Fix the production defect at its owning boundary (do not re-pin the fixture or relax the digest check unless the pins are genuinely derived from the wrong authority, and say which). Do not raise the 20 s publish deadline.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions