feat(ace): ACE Playbook Hooks — Curator Pattern (Spec 18)

codev/plans/18-ace-reference.md

@@ -0,0 +1,81 @@
+# Implementation Plan: ACE Playbook Hooks (Curator Pattern)
+
+## Metadata
+- **Spec**: 18-ace-reference
+- **Epic**: 0007a-context-engineering
+- **Consultation**: GPT 5.3 Codex (FOR 8/10), Gemini 3.1 Pro (AGAINST 9/10)
+- **Protocol**: ASPIR
+- **Branch**: feature-18-ace-reference
+- **FAVA Trails**: `01KKAA7ACN1SRR56YHWYBG33FV` in `mwai/eng/fava-trails/codev-assets/plans/18-ace-reference`
+
+## Overview
+
+Implement the ACE Curator pattern as a FAVA Trails protocol hook module. Six lifecycle hooks provide playbook-driven recall reranking, save-time quality checks, and observer telemetry. Pure Python, zero external dependencies.
+
+## Consultation Changes (vs Spec 18 v4)
+- **Added `after_propose` hook** (unanimous) — rules enter preferences/ via propose_truth
+- **Score clamping** [0.5, 2.0] — prevents runaway ranking
+- Multi-process cache concern deferred (single-process stdio for v1)
+
+## Phases
+
+```json
+{
+  "phases": [
+    {"id": "phase_1", "title": "PlaybookRule dataclass and parser (rules.py)"},
+    {"id": "phase_2", "title": "Hook module (__init__.py) with all 6 lifecycle hooks"},
+    {"id": "phase_3", "title": "Tests (test_ace.py)"},
+    {"id": "phase_4", "title": "README and example rules"}
+  ]
+}
+```
+
+### Phase 1: PlaybookRule + Parser
+- **Objective**: Implement rule engine with ACE-style feedback counters
+- **Files**: `src/fava_trails/protocols/ace/rules.py`
+- **Dependencies**: None
+- **Success Criteria**: PlaybookRule matches/evaluates correctly, _parse_rules handles malformed entries gracefully
+- **Tests**: Unit tests for matches(), evaluate(), _parse_rules()
+
+Details:
+- `PlaybookRule` dataclass: name, rule_type, match, action, weight, helpful_count, harmful_count, section, description, source_thought_id
+- `matches(thought)` — AND logic: source_type, confidence_lt, tags_include, tags_exclude, age_lt_days
+- `evaluate(thought)` — Multiplicative factor with Laplace-smoothed ratio, clamped to [0.5, 2.0]
+- `_parse_rules(raw_thoughts)` — Reads metadata.extra, skips malformed with warning
+
+### Phase 2: Hook Module with 6 Lifecycle Hooks
+- **Objective**: Implement all hooks following SECOM patterns
+- **Files**: `src/fava_trails/protocols/ace/__init__.py`
+- **Dependencies**: Phase 1
+- **Success Criteria**: All hooks use correct Event-Action types, cache works with TTL
+- **Tests**: Hook function tests
+
+Hooks:
+1. `on_startup` — Return StartupOk, lazy warmup
+2. `on_recall` — Lazy-load rules via TrailContext.recall("ace-playbook"), cache 5-min TTL, multiplicative scoring, RecallSelect + Annotate
+3. `before_save` — Anti-pattern Warn + brevity bias Advise (decision < 80 chars)
+4. `after_save` — Cache invalidation on ace-playbook tag + telemetry
+5. `after_propose` — Cache invalidation on ace-playbook tag
+6. `after_supersede` — Correction telemetry + cache invalidation
+
+Config: `playbook_namespace` (str, default "preferences")
+
+### Phase 3: Tests
+- **Objective**: Comprehensive test coverage following test_secom.py patterns
+- **Files**: `tests/test_ace.py`
+- **Dependencies**: Phases 1 and 2
+- **Success Criteria**: All tests pass
+- **Tests**: TestConfigure, TestPlaybookRule, TestParseRules, TestOnStartup, TestOnRecall, TestBeforeSave, TestAfterSave, TestAfterPropose, TestAfterSupersede, TestPipelineIntegration
+
+### Phase 4: README + Example Rules
+- **Objective**: User-facing documentation and sample rules
+- **Files**: `src/fava_trails/protocols/ace/README.md`, `src/fava_trails/protocols/ace/example_rules/*.md`
+- **Dependencies**: Phases 1 and 2
+- **Success Criteria**: README follows SECOM pattern, examples demonstrate all rule types
+- **Tests**: Manual verification
+
+## Risk Assessment
+- **age_lt_days timezone**: Defensive try/except, default True on failure
+- **Cache race**: GIL-protected dict assignment; first writer wins
+- **Missed invalidation**: TTL backstop recovers within 5 minutes
+- **Score explosion**: Clamped to [0.5, 2.0] per evaluation

codev/projects/18-ACE Playbook Hooks/status.yaml

@@ -0,0 +1,22 @@
+id: '18'
+title: ACE Playbook Hooks
+protocol: aspir
+phase: implement
+plan_phases: []
+current_plan_phase: null
+gates:
+  pr:
+    status: pending
+  phase-review-specify-main-iter1:
+    status: approved
+    requested_at: '2026-03-09T22:01:52.018Z'
+    approved_at: '2026-03-09T22:03:28.574Z'
+  phase-review-plan-main-iter1:
+    status: approved
+    requested_at: '2026-03-09T22:10:24.640Z'
+    approved_at: '2026-03-09T22:10:30.692Z'
+iteration: 1
+build_complete: false
+history: []
+started_at: '2026-03-09T22:01:30.232Z'
+updated_at: '2026-03-09T22:10:44.204Z'

src/fava_trails/protocols/ace/README.md

@@ -0,0 +1,193 @@
+# ACE Playbook Hooks (Curator Pattern)
+
+FAVA Trails' reference implementation of the [ACE (Agentic Context Engine)](https://arxiv.org/abs/2510.04618) Curator pattern (Stanford/SambaNova, ICLR 2026). ACE treats agent context as an evolving **playbook** of structured rules that grow and refine through feedback loops.
+
+## Architecture: FAVA Trails as the Curator
+
+FAVA Trails maps to the **Curator role** — the infrastructure that makes the Generator→Reflector→Curator loop safe and observable:
+
+| ACE Role | FAVA Trails Equivalent |
+|----------|------------------------|
+| Generator | External agent using `recall` MCP tool |
+| Reflector | External agent calling `save_thought` + `propose_truth` |
+| Curator | These hooks (governance, scoring, telemetry) |
+| Playbook bullets | Thoughts in `preferences/` with `ace-playbook` tag |
+
+The external Reflector reads `_SAVE_TELEMETRY` and `_SUPERSEDE_STATS` (or queries via MCP `recall`), analyzes patterns, and proposes new rules via `save_thought` into `drafts/`. Rules pass through the Trust Gate via `propose_truth` before becoming active.
+
+## Quick Start
+
+### Option A: Module Reference (zero-friction, no copying)
+
+Add to the `hooks:` section of your `config.yaml`:
+
+```yaml
+# config.yaml (at data repo root)
+hooks:
+  - module: fava_trails.protocols.ace
+    points: [on_startup, on_recall, before_save, after_save, after_propose, after_supersede]
+    order: 10
+    fail_mode: open
+    config:
+      playbook_namespace: preferences
+```
+
+### Option B: Local Copy (for customization)
+
+```bash
+cp -r "$(python -c 'import fava_trails.protocols.ace as p; import os; print(os.path.dirname(p.__file__))')" ./my-hooks/ace/
+```
+
+Then in `config.yaml`:
+```yaml
+hooks:
+  - path: ./my-hooks/ace/
+    points: [on_startup, on_recall, before_save, after_save, after_propose, after_supersede]
+    order: 10
+    fail_mode: open
+    config:
+      playbook_namespace: preferences
+```
+
+### Optional Dependencies
+
+None. ACE hooks are pure Python. The external Reflector is a separate agent or process.
+
+## Config Contract
+
+The protocol receives its config as a plain `dict` via `configure()`. It never reads files directly.
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| `playbook_namespace` | `str` | `"preferences"` | Namespace to load playbook rules from |
+
+## Lifecycle Hooks (6 points)
+
+### `on_startup` — Lazy Cache Warmup
+
+Returns `StartupOk`. The playbook cache warms lazily on the first `on_recall` call.
+
+### `on_recall` — Playbook-Driven Reranking
+
+Applies playbook rules to score and reorder recall results. Rules are lazy-loaded from `preferences/` (or configured namespace) with a 5-minute TTL as a stale-cache backstop.
+
+**Returns:** `RecallSelect` (reorder only — provenance-safe, cannot inject) + `Annotate`.
+
+**Scoring:** ACE-style multiplicative — each matching rule multiplies the base confidence score by its Laplace-smoothed factor. Score is clamped to `[0.5, 2.0]` to prevent runaway ranking.
+
+### `before_save` — Anti-Pattern Guardian + Quality Advisor
+
+Checks cached playbook anti-pattern rules. Adds a brevity-bias advisory for terse decisions (< 80 chars) per ACE research findings.
+
+**Returns:** `Warn` for anti-pattern matches, `Advise` for brevity bias.
+
+### `after_save` — Cache Invalidation + Reflector Telemetry
+
+- Invalidates playbook cache when a thought with `ace-playbook` tag is saved.
+- Accumulates save events in `_SAVE_TELEMETRY` for the external Reflector.
+
+> **after_* hooks are at-most-once, best-effort, non-blocking. Correctness must derive from persisted thoughts queried via MCP, not from hook state.**
+
+### `after_propose` — Promotion Cache Invalidation
+
+Invalidates the playbook cache when a rule enters `preferences/` via `propose_truth`. Ensures newly promoted rules are picked up on the next recall without waiting for the TTL.
+
+### `after_supersede` — Correction Telemetry
+
+Records each supersession as structured telemetry for the external Reflector. Invalidates the playbook cache if a playbook rule was superseded.
+
+## Playbook Rule Authoring
+
+Rules live in `preferences/` namespace with the `ace-playbook` tag. Use standard FAVA Trails MCP tools to create and manage them.
+
+### Rule Format
+
+Rules are stored as FAVA Trails thoughts. Fields are read from `metadata.extra`:
+
+| Field | Type | Default | Description |
+|-------|------|---------|-------------|
+| `rule_type` | `str` | `retrieval_priority` | One of: `retrieval_priority`, `confidence_floor`, `staleness`, `anti_pattern` |
+| `match` | `dict` | `{}` | Match criteria (AND logic — all must pass) |
+| `action` | `dict` | `{}` | Score adjustment: `{"boost": N}` or `{"deprioritize": N}` |
+| `weight` | `int` | `0` | Conflict-resolution priority (higher wins) |
+| `helpful_count` | `int` | `0` | ACE-style feedback counter (increases boost effect) |
+| `harmful_count` | `int` | `0` | ACE-style feedback counter (reduces boost effect) |
+| `section` | `str` | `""` | Playbook section label |
+| `description` | `str` | `""` | Human-readable description |
+
+### Match Criteria
+
+All criteria use AND logic — all specified conditions must pass:
+
+| Key | Type | Description |
+|-----|------|-------------|
+| `source_type` | `str` | Must equal `decision`, `observation`, `inference`, `user_input`, or `tool_output` |
+| `confidence_lt` | `float` | Thought confidence must be strictly less than this value |
+| `tags_include` | `list[str]` | Thought must have ALL listed tags |
+| `tags_exclude` | `list[str]` | Thought must have NONE of the listed tags |
+| `age_lt_days` | `float` | Thought must have been created within N days |
+
+### Scoring Formula
+
+```
+score_multiplier = clamp(action_base * laplace_ratio, 0.5, 2.0)
+
+where:
+  laplace_ratio = (helpful_count + 1) / (helpful_count + harmful_count + 2)
+  action_base   = action["boost"] or action["deprioritize"] or 1.0
+```
+
+`anti_pattern` rules always return `1.0` (neutral) — they signal in `before_save`, not during scoring.
+
+### Creating a Rule via MCP
+
+```python
+# Via fava-trails save_thought MCP tool
+save_thought(
+    trail_name="my-trail",
+    content="# Boost Recent Decisions\n\nBoosts decisions made in the last 7 days.",
+    source_type="user_input",
+    metadata={
+        "tags": ["ace-playbook", "retrieval-priority"],
+        "extra": {
+            "rule_type": "retrieval_priority",
+            "match": {"source_type": "decision", "age_lt_days": 7},
+            "action": {"boost": 1.5},
+            "weight": 10,
+            "helpful_count": 0,
+            "harmful_count": 0,
+            "section": "task_guidance",
+            "description": "Recent decisions rank higher than older ones",
+        }
+    }
+)
+
+# Then promote through Trust Gate
+propose_truth(trail_name="my-trail", thought_id="<ULID>")
+```
+
+## What ACE Adds vs. Flat Storage
+
+| Capability | ACE (flat list) | FAVA Trails ACE Hooks |
+|------------|----------------|----------------------|
+| Delta updates | ✓ | ✓ (save_thought/supersede) |
+| Feedback counters | ✓ (helpful/harmful) | ✓ (in rule metadata.extra) |
+| Correction lineage | ✗ | ✓ (supersession chain with reasons) |
+| Quality gate on rules | ✗ | ✓ (Trust Gate on promotion) |
+| Staging area | ✗ | ✓ (drafts/ → preferences/) |
+| Atomic multi-rule updates | ✗ | ✓ (single JJ commit) |
+| Rule rollback | ✗ | ✓ (JJ op restore) |
+| Human-editable rules | Partial | ✓ (markdown + standard MCP tools) |
+
+## Performance
+
+- `on_recall`: < 50ms for 100 thoughts with 10 rules (in-memory cache hit)
+- `before_save`: < 20ms for anti-pattern check (cached playbook)
+- Playbook cache TTL: 5 minutes (configurable via `_CACHE_TTL_SECONDS`)
+- Cache invalidation: immediate on `after_save`, `after_propose`, `after_supersede` for `ace-playbook` tagged thoughts
+
+## Literature
+
+- Stanford/SambaNova [arXiv:2510.04618](https://arxiv.org/abs/2510.04618) (ICLR 2026)
+- ACL 2025 Reflective Memory Management
+- Reference implementations: [ace-agent/ace](https://github.com/ace-agent/ace) (Apache-2.0), [kayba-ai/agentic-context-engine](https://github.com/kayba-ai/agentic-context-engine) (MIT)