Repository navigation
Run Copilot Review Once per Change at the Promotion - #3403
Conversation
A promotion re-reviews content its feature pull requests already carried, so the hosted review runs once per change, at the develop -> main promotion, and a feature pull request carries the attested local pass and CI. Measured 2026-10-16 to 2026-10-23, the four most active fleet repositories ran 418 Copilot reviews and about 2,200 workflow runs for 382 merged pull requests. Drop the copilot_code_review rule from the develop ruleset, keep it in main. pr_review.py no longer needs a first Copilot round before an attested local pass covers a head on a pull request into a non-default branch, so wait requests nothing there and status reads coverage=local on an attested head. A pull request into the default branch is unchanged. drive-pr step 3 says so, and the script README, the reviewer reference, and the Copilot instructions drop the old first-round rule. Decision (ptr727, 2026-10-23) on #3391, item 4. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
State drive-pr's exceptions where wait still requests a round and what clears exit 49, keep the holds and local_cover docstrings distinct, drop the operational bullet's now-shared Copilot rule and the stale "both ruleset payloads" in TODO.md, and split the sentences this change rewrote. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
A recorded local strict-review pass covers head |
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configuration
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved review findings remain in the review guidance and feature-head coverage messaging.
3 open issues
What changed in this PR
This pull request changes fleet review policy so feature pull requests rely on an attested local review and CI, while Copilot review runs at the develop → main promotion.
Changes:
- Removes the Copilot review rule from the
developruleset while retaining it formain. - Updates
pr_review.pyand its tests for local coverage on non-default branches and hosted review at promotion. - Synchronizes documentation, review guidance, and generated skill distributions.
| File | Description |
|---|---|
TODO.md |
Updates the settled ruleset note. |
tests/test_pr_review.py |
Covers feature and promotion review behavior. |
scripts/README.md |
Documents the revised Copilot request flow. |
scripts/pr_review.py |
Implements local coverage for eligible feature heads. |
repo-config/README.md |
Documents the ruleset split and promotion review model. |
repo-config/develop.json |
Removes Copilot review from develop. |
docs/pr-reviewer-reference.md |
Updates operational reviewer guidance. |
.github/skills/drive-pr/SKILL.md |
Updates the generated drive-pr instructions. |
.github/copilot-instructions.md |
Updates the Copilot review runbook. |
.claude-plugin/fleet-skills/skills/drive-pr/SKILL.md |
Regenerates the Claude skill mirror. |
.claude-plugin/fleet-skills/.source-digests/drive-pr |
Updates the generated skill digest. |
.agents/skills/drive-pr/SKILL.md |
Updates the canonical drive-pr workflow instructions. |
Review effort: Lite
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #3403 +/- ##
==========================================
Coverage ? 81.60%
==========================================
Files ? 19
Lines ? 10231
Branches ? 0
==========================================
Hits ? 8349
Misses ? 1882
Partials ? 0
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…ested Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
A recorded local strict-review pass covers head |
…view-at-promotion-only # Conflicts: # .claude-plugin/fleet-skills/.source-digests/drive-pr
|
A recorded local strict-review pass covers head |
## Summary Brings the review rule text in line with two changes already on `develop`: Copilot's hosted review runs once per change, at the promotion, and a `pre-existing` finding is listed in the pull request body rather than filed unless it is a correctness defect with a stated failure. Prose only, no code. Decision (ptr727, 2026-10-23) on #3391, items 2 and 4. ### Copilot reviews once per change, at the promotion Applies the replacement text from #3403's "Follow-up: pr-review-conduct Merge Gate text", in place: - `.agents/skills/pr-review-conduct/SKILL.md` "Merge Gate" item 2: every head on a pull request into a branch other than the default can carry an attested local pass (`coverage=local`), and a pull request into the default branch always carries a Copilot round. - Same item 2, promotion evidence path: a constituent feature pull request's full coverage comes from its attested local pass or a Copilot round, and a Copilot table row on the promotion itself counts as per-file evidence. The "lacked that coverage" example reads "no attested local pass and no review round". - "Expected review loop" step 3: one defined moment, the head of a pull request into the default branch. Elsewhere `wait` reads the attestation as `coverage=local` where Merge Gate item 2 allows that reading, and requests a round only where a partial round or a truncated review history is on record (the review found the dictated sentence dropped the exception `pr_review.py` keeps). - "Copilot's absence blocks, and is answered elsewhere": "on a fix push into" becomes "on a pull request into". - `GOVERNANCE.md` "PR Review Etiquette": the two first-round sentences, a sentence naming the review wait's partial-history request, a pointer to the Merge Gate for when the published pass counts as coverage, and the following sentence's "for such a fix push", which referred to the retired case and is recast around "on such a pull request". ### Class-sweep sibling takes the pre-existing outcome - `GOVERNANCE.md` "Verification Discipline": a pre-existing sibling in an untouched file stays out of the diff, and such a sibling takes the outcome a `pre-existing` finding takes under `pr-review-conduct` outcome 4, by pointer. - `pr-review-conduct` outcome 1: "filing the rest" becomes "recording the rest as outcome 4 records a `pre-existing` finding". - `pr-review-conduct` outcome 4: the sentence saying outcome 1 "keeps its own filing rule" for a sweep sibling says instead that the sibling takes those bullets as a `pre-existing` finding does. `GOVERNANCE.md` "Design Before Code" is deliberately unchanged. Its "Sites in files a change does not touch move in a change of their own, filed as one issue naming each" is a consolidation rule rather than the class-sweep sibling rule. Its bold lead says every site moves onto the primitive, and `pr-review-conduct` "Triaging findings" files the other sites as one issue to match. Sending those sites to outcome 4 would drop a consolidation finding and leave the lead false, so that change is left to the maintainer. `agent-conduct` and `drive-pr` change only inside generated include regions, regenerated with `python3 scripts/build_dist.py`, and the `.github/skills/` and `.claude-plugin/` mirrors are regenerated. ## Verification - `python3 scripts/build_dist.py --check`: `Generated skill distributions are current.` (exit 0) - `python3 scripts/prose_lint.py . --diff origin/develop`: exit 0, `scope: 13 of 380 file(s) read, 95 changed line(s)` at 466adc1 ## Local Review Four recorded `local-strict-review` passes at the `frontier` tier, the last over the pushed head with no findings. The two `introduced` findings the third pass left open, the one-moment rule against `wait`'s partial-history request and the implicit antecedent of the Merge Gate pointer, were raised again by Copilot and are fixed in 466adc1. Declined: `sentence-length` reports on edited sentences. The check is opt-in, and most of those sentences are either the dictated replacement text or pre-existing sentences with one clause changed. ## Pre-existing findings, not filed - `pre-existing`, `scripts/pr_review.py` module docstring (`attest`) and `scripts/README.md` `pr_review.py` section: "A promotion reads the comment only in its Merge Gate's per-file fallback" is stale since feature pull requests merge on `coverage=local`, so the promotion case's first bullet also reads the attestation. - `pre-existing`, `.agents/skills/pr-review-conduct/SKILL.md` "Which Reviewers a Repository Actually Has": the bold lead "Copilot's absence blocks, and is answered elsewhere" overstates the case for a pull request into a non-default branch, where an attested head merges with Copilot absent. The dictated replacement left the rest of that bullet as it is. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary Promotes these pull requests from develop. Each carries a maintainer decision recorded on #3391. - [#3399](#3399): states the write guard's threat model and documents two shapes it keeps allowed, a quoted run-time `-C` value and an `xargs` replace string run as a shell script (item 1). - [#3401](#3401): lists a pre-existing review finding in the pull request body instead of filing an issue for it (item 2). - [#3402](#3402): keeps one standing handoff issue per track, edited in place each round, with a `close` subcommand that ends a track (item 5). - [#3400](#3400): adds a weekly backlog and review-cost report script. - [#3403](#3403): runs the Copilot review once per change, at the promotion, with the attested local pass covering a pull request into develop (item 4). - [#3404](#3404): aligns the remaining review rule text with the promotion-only Copilot review (items 2 and 4). - [#3406](#3406): fixes the findings this promotion's first Copilot round raised on the content above, in the `wait` note for a head no local pass covers, the pr-review-conduct heading, and the backlog report's label line and cost scoping. Closes #3382 Closes #3385 ## Pre-existing findings, not filed - The [Copilot round on 4a77a1d](#3405 (review)) headline says documentation does not consistently describe current-head Copilot refusal handling. `local_cover` in `scripts/pr_review.py` refuses an attested head whose current Copilot round is a refusal, and `wait` exits `41` there. The `wait` entry of the `pr_review.py` docstring ("An attested head is covered at once.") and the `scripts/README.md` paragraph on the defined review moments ("Where it is attested, `wait` polls the head's checks instead.") leave that exception out. `pr-review-conduct` Merge Gate item 2 states it ("A refusal is not that coverage"). Both omissions are on `main` already, so this promotion did not introduce them. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Summary
Copilot's hosted review runs once per change, at the
develop -> mainpromotion pull request, and not on feature pull requests intodevelop.Why: measured on 2026-10-16 to 2026-10-23, the four most active fleet repositories ran 418 Copilot reviews and about 2,200 workflow runs for 382 merged pull requests, and a promotion re-reviews content its feature pull requests already carried. So the hosted review runs once per change, at the promotion, and the feature pull request carries the local pass and CI. Decision (ptr727, 2026-10-23) on #3391, item 4.
repo-config/develop.json: thecopilot_code_reviewrule is removed.repo-config/main.jsonis unchanged and keeps it.scripts/pr_review.py:round_before_headis removed fromholdsandlocal_cover, and the function itself is deleted since nothing else reads it. On a pull request whose base is not the repository's default branch,waitrequests no Copilot review, and an attested local pass alone satisfies coverage on the head (review_on_head=local coverage=local), with the digest saying so. Exceptions kept as they were: a partial Copilot round on record, a review history past the 100-review window, or a request already pending. A pull request into the default branch is unchanged. Every exit code keeps its meaning:49is still "no round covers this head, none was requested, and no attestation vouches for it", now reachable on the first head too. The docstring, the--requesthelp, the held note, the exit-49 message, and theCOVERAGE IS READ FROM THE LOCAL PASSdigest line are edited in place.tests/test_pr_review.py: constructed fixtures for both base-branch cases. A feature pull request with no Copilot round is never requested (unattested exits49, attested exits0withcoverage=local),statusreadsreview_on_head=localthere, a promotion with no round readsreview_on_head=NOand no local cover, and a promotion is still requested with or without an attestation. The tests that asserted the old first-round rule now assert the new one, andholds/local_coverare tested directly on both bases. No test calls the live API..agents/skills/drive-pr/SKILL.mdstep 3, plus its regenerated mirrors only: request no Copilot round on the feature PR, read the attestation ascoverage=local, and what clears exit49. The promotion steps are unchanged.repo-config/README.md"Rulesets",scripts/README.md(thepr_review.pyreview-moment paragraph),docs/pr-reviewer-reference.md,.github/copilot-instructions.md, andTODO.md.Applying the Ruleset Change
configure.sh applyupdates an existing ruleset with aPUTthat replaces the whole document, so removing the rule fromdevelop.jsonremoves it from the livedevelopruleset when apply runs, rather than only no longer asserting it.configure.sh checkcompares the rule-type set in both directions, so until apply runs it reports drift on every release-model repository, and the livedevelopruleset keeps requesting a Copilot review when a feature pull request opens. Nothing here ranconfigure.shagainst any repository. Perrepo-config/README.md, run it from a hub checkout atmainonce this change is promoted. For one repository:Then
repo-config/configure.sh check ptr727/ProjectTemplate releaseconfirms it. Each other release-model repository takes the same command with its own name.Follow-up: pr-review-conduct Merge Gate text
.agents/skills/pr-review-conduct/SKILL.mdis held by another lane, so this pull request leaves it alone. Until the follow-up lands, it andGOVERNANCE.md"PR Review Etiquette" still state the first-round rule this change retires.Merge Gate item 2. Replace the sentence that opens "On a pull request into a branch other than the default, a head after Copilot's first round can carry an attested local pass instead" with:
The same follow-up also owes these sentences, which state the same retired rule:
statusandwaitnever read", replaced by: "Request a Copilot round at one defined moment only: the head of a pull request into the default branch, a promotion among them. Auto-trigger is unreliable, so request it explicitly, which step 4'swaitis what does. On a pull request into any other branch,waitrequests nothing and reads step 1's attestation ascoverage=local. Passwait --requestwhere a change deserves a Copilot round anyway." The sentences from "waitalso skips the request" onward stay as they are.GOVERNANCE.md"PR Review Etiquette", a verbatim-carried section: "Request a review at defined moments rather than on every push: when the pull request opens, and on the head of a pull request into the default branch, a promotion among them." becomes "Request a Copilot review at one defined moment rather than on every push: the head of a pull request into the default branch, a promotion among them." And "On a pull request into any other branch, a fix push after the first round is covered instead by the recorded local pass the push already owes, published to the pull request so the review status can read it." becomes "On a pull request into any other branch, every head is covered instead by the recorded local pass the push already owes, published to the pull request so the review status can read it."Verification
Run from the worktree with
COVERAGE_FILE,MYPY_CACHE_DIR,UV_CACHE_DIR, andRUFF_CACHE_DIRunder a scratch cache directory, perOPERATIONS.md. Results, the full suite rerun at the final head as the second test row shows:uvx ruff@latest format --check .257 files already formatted, exit 0uvx ruff@latest check .All checks passed!, exit 0uvx mypy@latest(the[tool.mypy]CI checker inpyproject.toml)Success: no issues found in 63 source files, exit 0python3 -m unittest tests.test_pr_reviewRan 958 tests in 23.203s/OKuvx coverage@latest run --source=scripts,spec,host-setup -m unittest discover -s testsRan 3059 tests in 206.598s/OK (skipped=7), exit 0python3 -m unittest discover -s tests, rerun at the final head5b5ae9adRan 3059 tests in 214.314s/OK (skipped=7)spec/audit.py --selftestunder coverageSELFTEST PASS, exit 0spec/workflow_reuse.py --selftestunder coverageworkflow_reuse selftest OK, exit 0gh-write-guard.py --selftestunder coverageSELFTEST PASS, exit 0python3 scripts/build_dist.py --checkGenerated skill distributions are current., exit 0python3 scripts/canonical_review.py statuspython3 scripts/repo_gate.py[ok ], exit 0python3 scripts/prose_lint.py . --check charset --check semicolon --check dash --check dupword --check spelling --check comment-wrap --check comment-case --check home-path --check dead-path --check issue-refscope: 378 file(s) read, whole tree, exit 0python3 scripts/prose_lint.py --diff origin/developscope: 12 of 378 file(s) read, 130 changed line(s), exit 0 (same result at the final head)jq emptyoverregistry/*.json spec/*.json repo-config/*.jsonpython3 spec/validate.pySpec validation OK: 22 cataloged repos classify cleanly., exit 0Not run:
scripts/docker_lint.py, which needs Docker and repository-exposure approval this seat does not hold, the live-repositoryspec/audit.py, andrepo-config/configure.sh check, which this change was briefed not to run against any repository.OPERATIONS.mdnames the last two for a change torepo-config/, so they are the maintainer's after the apply step above. The opt-in--check sentence-lengthrun is listed under the local review below.Local Review
Five recorded
local-strict-reviewpasses, at thefrontiertier, againstdevelop: 10 findings, then 4, then 2 before the first push, then 3 after Copilot's round, then 0, then 0 again after mergingdevelopto clear a conflict in the generateddrive-prdigest. Every introduced finding is fixed, so none remains open.One first-round finding is answered here rather than in code: a release-model repository keeps its live Copilot rule on
developuntilconfigure.sh applyruns there, and the spec'sbranch.ruleset.developcheck reports drift until then. "Applying the Ruleset Change" above names the command.Copilot's round raised three findings. One, on
docs/pr-reviewer-reference.md, is fixed. Two, on the drive-pr step and the Copilot instructions, are the known disagreement withpr-review-conductandGOVERNANCE.mdthat the follow-up above carries, and are declined in their threads on that boundary.Pre-existing findings, not filed
Per Decision (ptr727, 2026-10-23) on #3391, item 2, these are listed rather than filed. Each is a sentence over the opt-in 25-word
sentence-lengthcap that this branch did not write and only trimmed, rewrapped, or left beside an edit:.github/copilot-instructions.md:31: six sentences in the "GitHub Copilot Review Runbook" paragraph.docs/pr-reviewer-reference.md:27: the 55-word and 28-word sentences after the edited ones.repo-config/README.md:15: the 43-word opener of theoperationalbullet.repo-config/README.md:17: the 38-word opener of themainparagraph.scripts/README.md:181: five sentences in the review-moment paragraph.TODO.md:140: the 42-word Settled bullet.scripts/pr_review.py: the sentences at lines 100, 306 (the 58-word bot-id sentence), 310, and 6067.🤖 Generated with Claude Code