fix(mcp): reword explore guidance as advisory, not a quota - #1570
Open
rongbc wants to merge 1 commit into
Open
Conversation
The trailing explore note said 'Explore budget: N calls ... spend your remaining calls ... Synthesize once you've used N' — agents read it as a hard quota, stopped exploring early, and fell back to grep+Read (more tokens, less complete). Reword it as explicit advisory guidance: the server never rejects or rate-limits extra explore calls, and exploring further is encouraged when the response doesn't cover the flow.
The tool schema description carried the same quota language ('Budget: make at most N calls for this project') — the higher-salience text the agent reads before ever calling. Reword it identically: advisory only, NOT a quota.
Also sync the eval probe string (probe-suite-envelope.mjs) and pin the new wording in tests with anti-regression assertions (medium tier must contain 'NOT a quota' and never 'remaining calls'/'Synthesize once'/'make at most').
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Description — Reword the
codegraph_explorebudget note so agents read it as advisory, not a quotaSummary
This PR fixes a wording problem in the
codegraph_exploreresponse footer. The server appends a note likeExplore budget: 2 calls for this project (3,497 files indexed) … Synthesize once you've used 2to everyresponse on mid-sized projects. My AI coding assistant read that as a hard quota, stopped calling
codegraph_exploreafter two invocations, and fell back to grep + Read — which cost more tokens and produced worse context than a
third explore would have. The note was meant as soft guidance; nothing in it said so. This PR rewords it to be
explicitly advisory ("NOT a quota", "extra calls are never rejected"), and updates the eval probe and tests that
pin the old wording.
Background
codegraph_explorereturns a bounded slice of source per call, tuned by project size(
getExploreOutputBudget/getExploreBudgetinsrc/mcp/tools.ts, introduced in #185). On small-to-mediumrepositories the intent is to keep a single response tight — under the host's inline tool-result ceiling so the
agent doesn't have to Read an externalized file back — and to steer the agent toward a few focused calls instead
of one sprawling query.
The steering takes the form of a footer note appended to every response when the project sits in the
500–4,999-file tier (3,497 files indexed for my project, so this tier is where I hit it). The original text:
Why it scared my AI assistant
The note is generated by the server at
src/mcp/tools.ts:5696and is purely informational — the server neverenforces or rate-limits explore calls, and a third call always succeeds. But the wording reads like an enforced
budget, and that is exactly how my assistant treated it:
spend your remaining calls, the calls sound like a depleting allowance.Synthesize once you've used 2is a direct order to wrap up after two calls.LLM agents tend to comply with imperative text in tool output as if it were a runtime constraint.
work". An agent cannot tell soft guidance from a hard cap unless the text says so explicitly.
2next to3,497 files indexedreinforces the intuition that explore calls areexpensive and rationed.
with
codegraph_exploreand use grep only as a supplement. When the budget note (stop after 2) and therule (keep exploring) pointed in opposite directions, the assistant obeyed the one that looked like a system
constraint — the tool's own quota-flavored text.
What actually happened (2026-08-19, reproducible): my assistant used two explores on my project, saw
Explore budget: 2 calls, assumed the allowance was spent, and switched the follow-up query to grep + Read —reading whole files (
agent_api.c,wifi_service.c, …) that one more explore would have returned more cheaplyand more completely. A third explore still succeeded, proving there was never any enforcement. The only reason
the assistant behaved that way is that the note did not say "this is a suggestion".
Fix overview
src/mcp/tools.ts:5696):Exploration guidance — advisory only, NOT a quota;explicitly state "extra calls are never rejected or rate-limited"; replace the imperative
Synthesize once you've used Nwith a conditional "only stop exploring when the response actually coversthe flow you asked about".
exactly when more calls are genuinely useful), and consider dropping the call count entirely.
scripts/agent-eval/probe-suite-envelope.mjs:88and__tests__/explore-output-budget.test.ts(including a new regression assertion that the tier-2 notecontains "NOT a quota" and never reintroduces "remaining calls" / "Synthesize once").
dist/mcp/tools.jsvianpm run build.Tests
npm run build && npm test(vitest:explore-output-budget.test.ts,explore-allocation-e2e.test.ts).codegraph.js serve --mcp --path <project>on a 3,497-file project, confirm the footercarries the new advisory wording.