Skip to content

feat(memory): enforce subject-bound atomic extraction - #2078

Open
beatenevo wants to merge 12 commits into
oceanbase:masterfrom
beatenevo:feat/enforce-memory
Open

beatenevo wants to merge 12 commits into
oceanbase:masterfrom
beatenevo:feat/enforce-memory

Conversation

@beatenevo

@beatenevo beatenevo commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Which issue or RFC does this PR close?

Closes #2066.

Rationale for this change

Atomic Memory extraction could preserve imperative or second-person wording from historical evidence, causing durable preferences and constraints to be represented as instructions for a future agent.

This change strengthens the extraction contract so retained memories are subject-bound declarative propositions, while task-specific procedures remain outside Atomic Memory.

What changes are included in this PR?

  • Version the coding and conversation Atomic Memory extraction prompts to .v2.
  • Require preferences, decisions, and constraints to identify their subject explicitly.
  • Prevent instruction-like evidence from becoming commands or second-person memory.
  • Exclude task-specific procedures and troubleshooting workflows unless they independently establish a durable fact, decision, or constraint.
  • Preserve literal commands, code, paths, configuration keys, and quoted factual values.
  • Keep the new rules in invariant instructions so custom prompts cannot remove them.
  • Add a versioned bilingual extraction evaluation with deterministic retention and instruction-leakage scoring.
  • Add safe live-run, replay, reporting, and pinning commands.
  • Include a pinned content-free live-run report and run manifest.
  • Add evaluation tests to the default pytest collection.

The pinned live-run report used:
evaluation/memory/atomic_extraction/reports/atomic-extraction-20261019-03/report.md

  • Provider/model: deepseek / deepseek:deepseek-flash
  • Temperature: 0
  • Max tokens: 1024
  • Timeout: 120s
  • Max requests: 2
  • Overall: 9/9 passed
  • Factual retention: 6/6
  • False rejections: 0
  • Instruction leakage: 0

Are there any user-facing changes?

Atomic Memory extraction now produces subject-bound declarative statements instead of instruction-shaped memories.

There are no public API or schema changes. Existing Atomic Memory records are not rewritten or migrated, and search, ranking, reconciliation, lifecycle, and context preparation behavior remain unchanged.

How was this change tested?

  • make check
  • uv run pytest evaluation/memory/atomic_extraction/tests
    • 24 passed
  • uv run pytest tests/builtin/artifacts/atomic_memory tests/builtin/artifacts/prompt tests/builtin/artifacts/memory
    • 208 passed, 3 skipped
  • Replayed the pinned live run successfully with status=passed.
  • Verified the pinned dataset, prompt identities, and report remain consistent with the current contract.

AI usage statement

OpenAI Codex (GPT-5) was used to assist with implementation, tests, validation, and review fixes. DeepSeek deepseek:deepseek-flash was used only for the documented opt-in live generation evaluation.

Comment thread evaluation/memory/atomic_extraction/manifest.py
Comment thread evaluation/memory/atomic_extraction/scoring.py Outdated
Comment thread evaluation/memory/atomic_extraction/runner.py Outdated
Comment thread evaluation/memory/atomic_extraction/scoring.py Outdated
@AlexStocks AlexStocks added the 1.4.1 v1.4.1 release tracking label Oct 20, 2026
Comment thread evaluation/memory/atomic_extraction/scoring.py
Comment thread evaluation/memory/atomic_extraction/scoring.py Outdated
Comment thread evaluation/memory/atomic_extraction/scoring.py Outdated

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1.4.1 v1.4.1 release tracking

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(memory): enforce declarative Atomic Memory extraction and family routing

4 participants