Nested git repos never reindex on branch switch: watcher only tracks session_project, index_status reports false freshness

[xnng]

Summary

In a multi-repo monorepo workspace (parent git root + many nested independent git checkouts), background auto-watch / reindex only tracks the MCP session project (cwd root). Branch switches and HEAD moves in already-indexed nested repositories are ignored.

Worse: index_status / list_projects surface live branch / head_sha from disk git, while the knowledge graph remains on the previously indexed tree. Agents therefore treat a stale graph as fresh and answer structural questions incorrectly with no error signal.

This is a high-severity reliability issue for agent workflows: silent wrong answers, not a hard failure.

Environment

Item	Value
Version	codebase-memory-mcp 0.9.0 (darwin-arm64 release binary)
OS	macOS (arm64)
Config	auto_index=true, auto_watch=true, auto_index_limit=50000
Client	Agent MCP session with cwd = monorepo parent
Cache	default ~/.cache/codebase-memory-mcp

Setup shape (common agent monorepo)

/path/to/workspace/                 ← git root; MCP session_root
  group/nested-service/             ← nested independent git repo (already indexed)
  other-nested-repos/               ← also indexed as separate projects

• Session project name: derived from parent path (e.g. Users-...-workspace)
• Nested project name: e.g. nested-service (indexed earlier via index_repository)

The nested repo has mutually exclusive branch trees (different modules/files per branch). Example: branch A has module service-variant-a/, branch B has service-variant-b/ instead.

Reproduction

1. Start MCP from the parent monorepo root. Confirm autoindex/watcher only registers the session project:

level=info msg=autoindex.start project=<session-project> path=/path/to/workspace
level=info msg=watcher.watch project=<session-project> path=/path/to/workspace
level=info msg=watcher.baseline project=<session-project> strategy=git files=yes

No watcher.watch line appears for the nested project.

2. Ensure nested repo is indexed and on branch A:

codebase-memory-mcp cli index_status --project nested-service
# nodes ~N, graph contains symbols under service-variant-a/

3. Clean switch nested repo to a very different branch B:

cd group/nested-service
git status --porcelain   # empty
git switch branch-b
# service-variant-a gone from disk; service-variant-b present

4. Wait well beyond poll interval (minutes). Re-check:

codebase-memory-mcp cli index_status --project nested-service

Actual

Surface	After branch switch
Live files on disk	Branch B tree
index_status.git.branch / head_sha	Branch B (live)
Graph nodes / edges	Unchanged from branch A
search_graph for branch-A-only symbols	Still hits (e.g. paths under service-variant-a/)
search_graph for branch-B paths	Missing / empty
search_code (disk grep)	Correctly reflects branch B
stderr watcher.reindex for nested project	Never

Expected

Either:

1. Nested indexed projects under (or beside) the session are watched and reindexed when their HEAD moves; or
2. Query surfaces refuse / flag staleness when graph snapshot ≠ current HEAD; and
3. index_status exposes an indexed HEAD (or stale: true), not only live git metadata that always looks current.

Control: manual reindex works

codebase-memory-mcp cli index_repository \
  --repo-path /path/to/workspace/group/nested-service \
  --name nested-service \
  --mode fast

• Branch B → fewer/different nodes; service-variant-b/* present, variant A gone from file_hashes
• Switch back + reindex → graph matches branch A again

So the pipeline can rebuild correctly; automatic path never targets the nested project.

Note: omitting --name derived a second project name from absolute path, leaving the original project stale. Separate naming footgun, but not the core bug.

Root cause (source read on main / v0.9.0-era tree)

1. Watcher registration is session-scoped only

register_watcher_if_enabled only watches srv->session_project / srv->session_root:

/* Register the session project with the background watcher for ongoing
 * change detection — unless auto_watch is disabled. */
static void register_watcher_if_enabled(cbm_mcp_server_t *srv) {
    if (!srv->watcher || srv->session_project[0] == '\0' || srv->session_root[0] == '\0') {
        return;
    }
    ...
    cbm_watcher_watch(srv->watcher, srv->session_project, srv->session_root);
}

Already-indexed nested projects are never registered unless the MCP session cwd is that nested root.

2. Watcher can detect checkout when registered

watcher.c tracks git rev-parse HEAD and treats HEAD movement (commit / checkout / pull) as a change. Nested repos never reach this path.

3. Parent git HEAD does not move when a nested repo switches branches

Even continuous reindex of the session/parent project does not rebuild the nested project's DB. Nested .git trees are separate indexes.

4. False freshness compounds the failure

index_status / project list git fields read live git state (also discussed in #1213). After a nested branch switch, agents see matching branch/sha and assume the graph is current — while search_graph still returns symbols deleted on disk.

Related open issues (distinct but compounding):

• Watcher re-index leaves MCP server serving a stale cached store (POSIX, silent) #1191 / MCP session serves a stale graph indefinitely after the background watcher re-indexes #1296 — watcher reindex may leave MCP serving a stale cached store (POSIX)
• Branch.head_sha stale after re-indexing an existing project (freezes at creation-time HEAD) #1213 — Branch / head_sha freshness signals unreliable
• Feature request: worktree overlay indexing for branch-aware agent workflows #573 — branch/worktree overlay (longer-term product direction)

This report is specifically: unwatched nested projects + live git metadata on stale graphs.

Impact

• Agent monorepos with many nested git checkouts are a common CBM install shape.
• Silent structural wrong answers on branch-divergent code (different modules / layouts per branch).
• Worse than “index missing”: tools return confident, high-quality looking wrong results.
• No reliable client-side freshness check today without external bookkeeping.

Severity: high for agent reliability (silent incorrect graph).

Suggested fixes (shortest path first)

1. Watch all indexed projects (or all projects whose root_path is under session root / configured roots), not only session_project.
2. Persist indexed_head_sha (and branch) at successful index time; expose in index_status with stale = (indexed_head_sha != live_head_sha).
3. On stale, either auto-queue reindex or return an explicit staleness flag on graph tools so agents do not trust results blindly.
4. Keep Watcher re-index leaves MCP server serving a stale cached store (POSIX, silent) #1191/MCP session serves a stale graph indefinitely after the background watcher re-indexes #1296 store invalidation so successful reindex is visible to long-lived MCP sessions.
5. Docs: state clearly that auto_watch only covers the session project today, and nested multi-repo setups need per-repo reindex or separate sessions until (1) lands.

Workaround (current)

After every git switch / checkout in a nested indexed repo:

codebase-memory-mcp cli index_repository \
  --repo-path <nested-repo> \
  --name <existing-project-name> \
  --mode fast

Or keep one clone per long-lived branch as separate project roots (avoids in-place branch switches).

Offer

Happy to refine the repro into a small fixture (parent repo + nested repo with two branches of divergent file trees) if useful for a regression test.

[bosschen0429]

I reproduced the same false-freshness outcome on Windows with
codebase-memory-mcp 0.10.8, but without requiring a nested repository.

Observed sequence:

1. The project graph was indexed while the working tree contained symbols from
   a later feature branch.
2. The watcher/session was unregistered or restarted.
3. The repository was cleanly switched to another branch/HEAD.
4. On re-registration, the watcher logged a new Git baseline but did not
   reindex.
5. index_status.git.head_sha reflected the live checkout, while
   search_graph continued returning symbols absent from that checkout.

The project database last-write timestamp also predated the current checkout
state, so this was not only a stale in-process reader: the persisted graph
itself had not been rebuilt.

Source inspection appears to explain the lifecycle gap:

• index_status resolves Git context live from root_path.
• watcher last_head is process-local.
• init_baseline() adopts the current HEAD without comparing it against a
  persisted indexed HEAD.
• therefore a clean checkout change that happens while unwatched cannot be
  distinguished from a fresh graph after re-registration.

This means the graph can remain stale indefinitely after the new HEAD is
adopted as the baseline; waiting for another poll is insufficient.

Persisting indexed_head_sha at successful publication, comparing it during
watcher initialization, and exposing indexed-vs-live Git state (or stale)
would cover both this lifecycle case and the false-freshness surface described
here.

[zhangbiyao1129]

Reproducing from macOS (Apple Silicon) on 0.11.0 with two additions that may help
shape the fix: a per-file freshness oracle already exists and works, and an
mtime-only change latches metadata_changed irrecoverably.

1. False freshness reproduces without a branch switch or a nested repo

A project simply left un-reindexed while commits accumulate behaves identically.

Users-kanyun-it-services-its-ms-attendance (~/it-services/its-ms-attendance):

Signal	Value
indexed_at	2026-08-06T08:01:34Z
commits since index	20
last commit	2026-09-18 17:13:35 +0800
index_status.git.head_sha	matches current HEAD exactly
status	ready
parse_partial / skipped	empty

Both surfaced fields look healthy; the graph is six weeks behind. Verified by content,
not inference — of 12 functions added in that window, all 12 are absent, and the
file carrying most of them has zero nodes:

$ for f in DefaultDirectoryMappingConfig LoadDirectoryMappingConfig validPositionGroup \
           mergeDirectoryUnmatchedPool positionGroupForRegion directoryPoolID ...; do
    sqlite3 "$DB" "SELECT COUNT(*) FROM nodes WHERE name='$f';"; done
# 12 checked -> 12 missing
$ sqlite3 "$DB" "SELECT COUNT(*) FROM nodes WHERE file_path LIKE '%directory_unmatched%';"
0

No snapshot exists to compare against: projects holds only name / indexed_at /
root_path, and store_meta is empty. This is consistent with @bosschen0429's
lifecycle analysis — indexed_head_sha is genuinely absent from the store.

2. check_index_coverage already carries the per-file staleness signal

This is the stale signal point 3 asks for, per path, and it works. Clean A/B on a
current-format project, nothing touched but mtime:

# before
lib/model-scope.ts  no_recorded_issue  metadata_match    use_graph_with_best_effort_caveat
# after `touch lib/model-scope.ts` (content byte-identical)
lib/model-scope.ts  no_recorded_issue  metadata_changed  read_source_and_reindex

So the oracle exists and is reachable today. Surfacing it at project level would
reuse machinery that is already there rather than adding a second notion of staleness.

3. An mtime-only change latches metadata_changed permanently

metadata_changed cannot be cleared by re-indexing when the content is unchanged —
neither the default mode nor --mode full refreshes the stored mtime_ns:

# baseline
freshness=metadata_match    rec=1789788292866885000  disk=1789788292866885000
# touch lib/paths.ts  (content byte-identical)
freshness=metadata_changed  rec=1789788292866885000  disk=1789788490508198000
# index_repository (default)
freshness=metadata_changed  rec=1789788292866885000  disk=1789788490508198000
# index_repository --mode full
freshness=metadata_changed  rec=1789788292866885000  disk=1789788490508198000

A content change does clear it (rec and disk converge, metadata_match
returns), because the incremental path compares hashes, correctly concludes the file
changed, and then writes fresh metadata. An mtime-only change is judged "unchanged"
and skipped entirely — including the metadata write.

Consequence: any tool that rewrites mtimes without changing content pins the project
in metadata_changed indefinitely. A git checkout in CI, a git stash/restore, an
editor that re-saves identical bytes, or a tar/rsync restore all do this. I confirmed
recovery requires a real content edit; re-indexing alone never does it.

4. get_code_snippet serves shifted source under this state (observed on 0.11.0)

I could not test PR #1788's source_drift path directly — strings finds no
source_drift symbol in the 0.11.0 binary, so that fix is not in this release. On
0.11.0 the behaviour is the pre-#1788 one, and it is worth restating because it is
silent:

Inserting two lines above a function (indexed start_line: 46; the function is now at
line 48):

$ codebase-memory-mcp cli get_code_snippet --project ... --qualified-name ...toNativePath
start_line: 46
end_line: 49
source: |
   * Only pass paths — a branch name like `feature/x` would become `feature\x`.
   */
  export function toNativePath(p: string): string {
    if (!p || process.platform !== "win32") return p;

The reported range is unchanged, the slice is shifted, and the first line is the tail
of the preceding JSDoc block — presented as the function source. No error, no flag.

Suggestions

Points 1–4 in the issue are the right shape. Three additions:

• Reuse the existing oracle at project level. Report stale when any recorded
  file's metadata no longer matches, using the same reader check_index_coverage
  already uses, instead of a new signal that could disagree with it.
• Refresh mtime_ns on the incremental path even when the content hash matches.
  This removes the one-way door in §3 with a one-line metadata write.
• Scope the fix beyond branch switches. §1 needed no nested repo and no checkout;
  §3 is independent of git entirely. A fix framed only as "watcher should track
  nested HEADs" would leave both in place.

Happy to retest against a candidate build.

[DeusData]

Thank you for the v0.11.0 evidence. The watcher still gates work on project registration, so the earlier dirty-to-clean fix does not settle this nested-project report. The stale generation, persistent metadata mismatch and shifted snippet need separate checks. This remains open; the earlier watcher fix is not being treated as proof that these cases are resolved.

[DeusData]

Thank you, @xnng, @bosschen0429 and @zhangbiyao1129! Between you, you separated the lifecycle cause from the reporting surface. There are now two linked tracks:

• feat(mcp): report truthful index freshness from checkout evidence #1561 persists the checkout a graph was built from and reports a freshness verdict with reasons. That addresses the false freshness.
• fix(watcher): show watch state in index_status, correct auto-sync docs (#2167) #2348 shows in index_status whether a project is actually being watched.

Neither of them makes nested repositories watched: the watcher is still registered only for the session project. That part remains open here.

[njuptlzf]

Confirming this on 0.11.0 / Windows, in a host that serves many session workspaces (the MCP client's workspace here is a non-git parent directory containing several independent git checkouts), and adding the mechanism plus a concrete ask.

Mechanism (measured from ~/.cache/codebase-memory-mcp/logs/cbm-daemon.log): the watch is registered by the client that creates the session-managed daemon, keyed to that client's cwd; clients that merely join an already-active daemon register nothing.

• daemon.start 580 / daemon.lifetime_end 579 (reason=runtime_exited) — the daemon, and its watch, live only as long as the creating client.
• Every daemon.start is followed by exactly one watcher.watch naming the creator's cwd: …-work (the session workspace) 598 times, …-dsh-codebase-memory 118 times, and the nested repo I was actually editing (…-ruankao-ai) 0 times — despite many cli index_repository --repo-path …/ruankao-ai runs (a transient CLI is not the creator, so it acquires no watch).
• Two probes where I spawned my own long-lived MCP client with cwd = <nested repo> while a daemon was already held by the host's client: watcher.watch / baseline / changed / unwatch = 0 / 0 / 0 / 0 for both. Joining does not add a watch, even for a fully interactive client.
• For the non-git parent, watcher.baseline … strategy=none and changed=0 throughout — no polling at all, matching the source comment.

Quantifying the silent-freshness half (same repo): after the index had drifted, I sampled 59 symbols and 20 (34%) returned a line range that does not contain the named symbol — e.g. adminGate claimed 27-34 while the file has it at 47; registerRoutes claimed 36-382 vs actual 56. index_status reports ready throughout, so an agent reads a neighbour function under the correct name, exactly as you describe.

Ask: an explicit registration surface — e.g. daemon watch <path>, an MCP watch_project tool, or a config list of watch roots — so a host serving many workspaces can register the projects it cares about. Worth noting the per-project-MCP-server workaround does not help either: with one shared daemon, the extra clients join it and register nothing.

Cross-referencing #1660 (session-scoped watch); the data above refines its wording: it is not "an interactive client connects", but specifically the daemon-creating client that registers.