spec-plan: per-task Files: has no Delete verb and no completeness rule, so plan-instructed edits fall outside the changes-review scope

[buddhamangler]

Summary

spec-plan's per-task Files: block is the scope that changes-review later reviews against ("extract the list of files each task creates/modifies — the plan files"; "scope to only plan files"). But the plan template gives the author no way to state a deletion, and no rule says that every path a task's own body names — in its Objective, Key Decisions, or DoD — must also appear in Files:. Plans therefore routinely arrive with a Files: list written from what the task edits and missing what it creates, deletes, or touches in passing (a doc comment on an interface, a CLI help string, an option description, a queue/report file the task removes).

Nothing catches it at plan time: pilot spec validate checks the rendering contract, not Files: completeness, and a human reviewer skims past a missing path far more easily than a missing task. It surfaces at implementation time as a changes-review scope complaint against an implementer that did exactly what its plan said — or, worse, the out-of-list edit is scoped out of the review entirely and lands unreviewed.

Version: Pilot Shell v11.0.2, Claude Code on Linux (WSL2).

Where it lives

~/.claude/skills/spec-plan/SKILL.md, the task template (around the **Files:** label):

**Files:**

- Create: `exact/path/to/file.py`
- Modify: `exact/path/to/existing.py`
- Test: `tests/exact/path/to/test.py`

Three verbs, no Delete, and the only rule attached to the block is that it "must list reviewable implementation artifacts". The consumer side, ~/.claude/agents/changes-review.md, reads Files: as the review scope.

Reproduction

1. Run /spec on a task whose natural description mentions a file without "editing" it as the headline, e.g.:

   Add a --force flag to the wipe command. Update the doc comment on IStore to say the flag bypasses the guard. Remove the now-obsolete docs/todo/wipe-guard.md.

2. Read the generated plan's task. In our runs the task body named all three files, but Files: listed only the command file and its test:

   **Files:**
   - Modify: `src/cli/WipeCommand.cs`
   - Test: `tests/cli/WipeCommandTests.cs`
   

   IStore.cs (doc-comment edit) and docs/todo/wipe-guard.md (deletion) are absent — the second because the template has no verb for it.

3. pilot spec validate <plan> → exit 0.

4. Implement. changes-review scopes to plan files, so the IStore.cs hunk is either flagged as out-of-scope or never reviewed, and the deletion is invisible to the review.

Across one five-task plan we counted five omissions of this shape in three tasks (a doc-comment target, a CLI-help/option-description file whose text a DoD bullet asserted on, the test class a DoD's own verify command ran, plus one create and one delete), found only by a manual sweep after approval.

Suggested fix

Two small changes to spec-plan, no CLI change:

1. Add Delete: (and optionally Rename:) to the Files: template so a removal has somewhere to go.
2. Add a rule to the task-writing step: every repository path named anywhere in the task body (Objective, Key Decisions / Notes, Definition of Done — including the file a verify command runs) must appear in Files:. Derive the list from the task body rather than from the code the author was looking at; a path that only appears as "update the doc comment on X" or "the --help text in Y" is still a modification.

Optionally, pilot spec validate could enforce (2) mechanically — it already parses the task blocks, and a backtick-quoted path in the body that is missing from Files: is a cheap, deterministic check.

[maxritter]

Fixed in b05f19c — thank you, this is an unusually well-traced report. The framing that Files: is the review scope rather than a summary is exactly right, and it's the reason both of your failure modes exist: the out-of-list edit is either argued about as drift or scoped out of the review entirely.

Both suggestions are in, and the optional third one too:

1. Delete: (and Rename:) added to the template, so a removal finally has somewhere to go.
2. A completeness rule in spec-plan Step 7: every repository path named anywhere in the task body — Objective, Key Decisions / Notes, Definition of Done, including the file a Verify: command runs — must appear in Files: under the verb that says what the task does to it. Derived from the task body, not from whatever the author had open.
3. pilot spec validate now enforces the mechanical part as a files-incomplete warning, and Step 10.0 runs the validator as a gate before reviewers or approval.

One honest note on scope, since it's narrower than your rule as written. The check reads a task's Objective, DoD and User Action — not Key Decisions / Notes. Step 7 asks that field for pattern references (follow the pattern at src/foo.ts:42), so a bare path there is usually somewhere to look, and flagging it would train authors to ignore the check. A file:line ref is treated as a read-only pointer everywhere and is exempt, which gives authors a way to cite a file they only read. Scope is also the union of every task's block rather than each task's own, because reviewers receive the plan's blocks together — only a path that no task lists can actually land unreviewed.

That tuning was measured, not guessed: run against the six real plans in this repo, the naive version produced 7 findings that were mostly pattern references; the shipped version produces 1, and it's a genuine omission (a test file a DoD's own verify command runs that no block lists — the exact shape you described). Your IStore.cs / wipe-guard.md repro fires on both paths.

Two consumers had to follow, one of which would have bitten immediately: spec-verify treated every Files: entry as "must exist in the diff", so a Delete: line would have reported the task that did exactly what it was told as unimplemented. That check is now inverted for deletions and split for renames. Both changes-review variants also now treat a changed file the task body names but the block omits as in scope — reviewed, with the gap reported as a plan defect rather than left as unreviewable drift.

Counting the five omissions across your five-task plan is what made the case concrete. Thanks for doing that legwork.