ci(docs): enforce governed knowledge records - #369
Conversation
tt-a1i
left a comment
There was a problem hiding this comment.
方向对,覆盖面不够
docs/README.md 治理的是 research、design、architecture、Decision、benchmarks。检查器只扫了 docs/research 和 docs/benchmarks。#386/#382 这种 validated 的 architecture/design 记录现在不会被拦。
请扩到 design / architecture / Decision,或明确写「其余类别暂不强制」并改 README。#309 是 Decision-only 且 Windows 路径 fail-open,请吸收进来,不要并排合。
frontmatter 选择加入、「结构 ≠ 验证」是对的。
tt-a1i
left a comment
There was a problem hiding this comment.
Exact-head review: 81320b2
[P1] The new CI contract is opt-out, so it does not enforce the forward-only policy for new records. scripts/check-knowledge-contract.mjs:136-142 classifies every frontmatter-less research/benchmark Markdown file as legacy and skips it. A contributor can therefore add a brand-new record without frontmatter and bypass required metadata, index reachability, links, frozen identities, accounting, failures, limitations, and evidence checks while bun run check remains green.
Decision 0001 says documents from that Decision forward are governed; legacy is a fixed historical set, not a property a new file may claim by omitting metadata. Please encode an immutable legacy allowlist or baseline and fail closed for every new unlisted record. Add a regression proving that a newly added frontmatter-less record fails, while the enumerated pre-Decision records remain accepted.
Verification boundary: exact-head checker/policy review. Existing tests intentionally assert the bypass as legacy exclusion, so their green result confirms rather than mitigates the issue.
tt-a1i
left a comment
There was a problem hiding this comment.
Follow-up review of 3dd872b: the previously reported frontmatter opt-out P1 is addressed by the fixed LEGACY_RECORDS allowlist and the new rejection regression. I reran node --test --experimental-strip-types tests/scripts/knowledge-contract.test.ts in an isolated exact-head worktree: 6 passed, 0 failed (exit 0). This does not constitute full approval or evidence validation.
Please synchronize the remaining opt-in wording in the PR description and Research index with the implementation. The checker currently covers research/benchmarks only; retain an explicit statement of that enforcement scope. The required Windows CI is still red: job 101235332304 reports Timed out waiting for local git info in tests/extensions/git-info/index.test.ts. I have not established that failure as a product regression or a flake; refresh against main and obtain a green required matrix before merge.
3dd872b to
1061c2b
Compare
|
@tt-a1i 已修复 P1:legacy 仅来自不可变显式 allowlist,新建无 frontmatter 记录 fail closed,并补回归与 CI。 |
|
@tt-a1i 已同步 PR 说明与 Research index:仅显式列出的历史记录可免 frontmatter;新记录 fail closed,checker scope 仍为 research/benchmarks。 |
# Conflicts: # docs/research/README.md
|
已推送 |
tt-a1i
left a comment
There was a problem hiding this comment.
Reviewed exact head 166645d.
This is a high-value, low-runtime-risk CI guard, but its central fail-closed evidence promise is still bypassable:
- [P1] evidence-reference and rerun-entry-point are checked only for non-empty strings. Markdown body links are resolved separately, but those metadata values are never resolved. A validated Benchmark with evidence-reference: missing-evidence.json, rerun-entry-point: missing-command, and no body link passes with zero problems. Require a resolvable repository target or valid external/archive identity and add a missing-evidence regression.
- [P1] README.md and TEMPLATE.md are exempted by basename at every depth. A new docs/research/topic/README.md without frontmatter is silently omitted. Restrict exemptions to the explicit category-root files or a fixed path allowlist.
- [P2] Index reachability is a raw substring check, so a target appearing only inside an HTML comment is accepted as indexed. Parse actual Markdown links outside comments/code.
- [P2] Migrated values such as related-issues: #428 parse as comments/null under standard YAML even though the bespoke parser accepts them. Quote them or explicitly define a non-YAML format.
Focused tests, bun run check, hosted CI, and the current-main merge-tree are green, but adversarial fixtures reproduce the P1 bypasses. After these narrow fixes, this PR is worth merging.
Problem
Issue #198 established the forward-only knowledge and evidence contract, but new Research and Benchmark records still have no repository check for required metadata, category-index reachability, or local link integrity. A complete-looking Benchmark can therefore omit frozen identities, accounting, failure classification, limitations, or a retrievable evidence reference without failing CI.
Value
Approach
check-knowledge-contract.mjsfor governed records underdocs/researchanddocs/benchmarks.bun run check, and append the Decision-0001-backed promise to the discipline ledger.Validation
bun run check— passed.bun run test— passed: 1246 Node tests passed, 1 platform test skipped; 30/30 Vitest tests passed.git diff --check— passed.Impact
Refs #198.