feat: coordinate process-local invalidation

[lan17]

Summary

• add opt-in Redis Pub/Sub coherence for trackForInvalidation process-local entries while keeping invalidateRemote(keyType, id, futureBufferMs) as the only application invalidation call
• synchronously evict/fence the origin, atomically advance the Redis watermark and publish a versioned event, and fan the event out to every attached DialCache instance in healthy peer processes
• guard every tracked local publication immediately before the LRU write so in-flight Redis hits, fallbacks, serializers, and health transitions cannot repopulate invalidated local state during the future window
• preserve request-local snapshots, process-flight result sharing, the stored value frame, legacy clients/scripts, and exact remote-only behavior when coordination is omitted
• provide first-party node-redis publication/subscription support, GLIDE publication support, bounded local fence state, lifecycle/health handling, operational documentation, and a repeatable invalidation benchmark

Closes #106.

Why

Redis watermarks already protect tracked Redis values, but a process-local hit stops before Redis and cannot observe that watermark. Mutation-sensitive use cases therefore had to choose between process-local performance and accepting the full local TTL as a staleness window.

This adds a best-effort notification path without moving correctness authority away from Redis and without adding a generation lookup to every local hit.

Architecture

flowchart LR
  M[Source mutation commits] --> I[invalidateRemote]
  I --> P[Provisional synchronous local fence + LRU scan]
  I --> L[Single Redis Lua command]
  L --> W[Advance durable watermark]
  L --> U[Publish v1 namespace event]
  L --> R[Return effective watermark + Redis time]
  R --> O[Origin coordinator fan-out]
  U --> S[Dedicated subscribers]
  S --> X[Peer coordinator fan-out]
  P --> F[Monotonic identity fence]
  O --> F
  X --> F
  F --> E[Evict exact tracked namespace + keyType + id]
  F --> G[Final synchronous publication guard]

Loading

There is no generation in DialCacheKey.urn, no change to the local entry shape, and no Redis or fence-map work on an ordinary local hit.

Redis watermark and event protocol

The coordinated path uses a new parallel INVALIDATE_AND_PUBLISH_CACHE_SCRIPT; the legacy invalidation script and exact integer 1 reply remain unchanged.

The new one-key Lua command:

1. validates the bounded future buffer;
2. reads Redis TIME;
3. preserves the furthest existing watermark and derived TTL;
4. constructs and validates a v1 event before mutation;
5. stores the watermark;
6. publishes on one strictly encoded namespace channel; and
7. returns the same event to the origin.

Timing values remain canonical decimal strings so JavaScript precision is validated before conversion. Receivers use effectiveWatermarkMs - redisNowMs only as a duration and start that duration on performance.now(); pod wall clocks are never compared with Redis epoch time.

The full UTF-8 JSON event is capped at MAX_REDIS_INVALIDATION_EVENT_BYTES (16 KiB). Oversized events reject before SET/PUBLISH. Channel components use strict RFC 3986 encoding so Redis ACL glob metacharacters such as * cannot broaden an exact-channel grant.

Local eviction and publication safety

Each participating DialCache maintains bounded per-identity monotonic deadlines. Every valid signal installs or extends the fence before scanning the LRU for the exact closed-brace tracked prefix, covering all use cases and argument variants while leaving adjacent ids, other key types/namespaces, and untracked entries untouched.

A tracked miss captures the coordinator health epoch. Immediately before the final synchronous LRU set, publication requires:

• coordinator state is ready;
• the captured health epoch is still current; and
• no matching identity or conservative global overflow fence is active.

Fence cardinality is bounded by localMaxSize. If all slots contain active identities, state collapses into one global deadline through the furthest active fence instead of dropping safety state. There is no timer per identity.

Existing process-flight response semantics are unchanged: a caller arriving after invalidation can still join an older exact-key leader and receive its result. The fence controls cache publication, not application-result ordering. Request-local memoization remains the existing outer-request snapshot.

Subscriber health and lifecycle

The backend-neutral coordinator starts unavailable and delivers state/event callbacks synchronously while isolating listener failures. Detected errors, reconnects, connection end, wrong channels, malformed payloads, or invalid custom signals clear tracked local state and suppress tracked publication. A newly acknowledged ready epoch clears once more before allowing new publication.

Protocol-failure epochs prevent a malformed message arriving beside an initial subscribe or resubscribe acknowledgement from being erased by a pending ready callback. Disposal is terminal, reentrantly idempotent, stops stale fan-out, detaches health listeners synchronously, and performs listener-specific unsubscribe without closing the caller-owned client.

The node-redis helper requires a connected, dedicated standalone subscriber and owns only its subscription/listeners. The GLIDE adapter supports the combined publisher script but intentionally does not advertise a first-party subscriber because its available lifecycle surface cannot establish the same health contract.

Compatibility and public surface

• RedisConfig remains an interface so existing extends/implements consumers compile unchanged.
• CoordinatedRedisConfig / DialCacheRedisConfig type the opt-in branch and require DialCacheCoordinatedRedisClient.invalidateAndPublish.
• The backend-neutral event/coordinator types are root-exported.
• The versioned event codec, channel helper, size/version constants, and new Lua source are exported from dialcache/redis-protocol.
• dialcache/node-redis exports the coordinated script registration and subscriber coordinator factory.
• DialCache.dispose() detaches only that instance; it never drains or closes Redis resources.
• Omitting redis.coordinator keeps the v0.11.0 command path, local-hit path, local entry shape, and remote-only invalidation behavior exact.

Delivery and failure semantics

Redis Pub/Sub remains at-most-once: there is no replay, peer acknowledgement, sequence check, or all-pod completion barrier. Detected gaps fail closed for tracked process-local storage, but an undetected missed event can leave a stale peer value until TTL. Strict cross-pod read-after-invalidation still requires disabling the local layer or a durable validation/barrier design.

The combined Lua command stores before PUBLISH. If publication is ACL-denied or the client loses the response after dispatch, invalidateRemote() rejects while the watermark may already have advanced. The origin deliberately keeps its provisional local eviction/fence; callers must treat the outcome as ambiguous and retry according to their mutation protocol.

Redis Cluster and operations

The Lua command has one watermark key and preserves the existing hash-tag routing. It uses ordinary Redis 6.2-compatible Pub/Sub, which propagates cluster-wide; sharded Pub/Sub remains out of scope. Plan one dedicated subscriber connection per process and namespace and include cluster-wide Pub/Sub fan-out plus LRU scans in capacity planning.

Safe rollout keeps tracked process-local ramp at zero until every writer uses the combined publisher and every cache-serving process has an acknowledged subscriber. Validate identity sizes against the 16 KiB limit, then ramp local caching gradually. Rollback disables tracked local caching before removing publishers/subscribers.

Performance

The maintained benchmark alternates seven rounds of identical unconfigured/coordinated healthy local hits, then measures single peer scans at 1k/10k/50k occupancy and the origin's normal three-signal path (provisional + returned event + Pub/Sub echo) at 10k.

Final Node 22.22.0 run:

• unconfigured local hits: 452,621 ops/s
• coordinated healthy local hits: 452,544 ops/s (+0.02% elapsed; noise)
• peer scan: 0.173 ms at 1k, 2.079 ms at 10k, 10.685 ms at 50k
• origin three-signal path at 10k: 6.223 ms per logical invalidation / 2.074 ms per scan

Every equal-watermark signal still scans because equal timing can represent a separate source mutation. Local invalidation metrics therefore count authoritative event applications, not logical mutations: a healthy origin commonly records the returned event and echo, while a peer commonly records one event.

Validation

Full validation ran on Node.js 22.22.0 at cbb2378. The history-only rebase produced ff1b70c with the identical source tree; corepack pnpm check and git diff --check were rerun successfully at the new head:

• corepack pnpm check
  • typecheck passed
  • 21 unit files / 371 tests passed
  • coverage: 97.21% statements, 94.54% branches, 98.36% functions, 97.21% lines
  • build, declarations, and packed ESM/CommonJS consumers passed
• corepack pnpm test:integration
  • 2 integration files / 90 tests passed across Redis 6.2, Valkey 8, and Redis Cluster
  • includes cross-client peer delivery, Cluster cross-node Pub/Sub, SCRIPT FLUSH, no-subscriber publication, pre-mutation event rejection, and ACL-denied PUBLISH after committed SET
• corepack pnpm benchmark:invalidation passed with semantic assertions and the results above
• corepack pnpm audit --prod: no known vulnerabilities
• corepack pnpm audit: no known vulnerabilities
• git diff --check
• two fresh final review passes (correctness/concurrency/API and tests/operations/performance) returned clean after their findings were fixed

Base and concurrent work

#96 landed on main as 500d5e7. This branch was rebased directly onto that commit at ff1b70c; the source tree is byte-for-byte identical to the previously validated head, and the PR now targets main with one feature commit.

#105 is independent but touches overlapping core/docs files, so whichever feature lands second will need a normal rebase/conflict-resolution pass.