fastmcp: Let a project name its own safety tiers - #77
Open
tony wants to merge 22 commits into
Open
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #77 +/- ##
==========================================
+ Coverage 92.94% 93.00% +0.06%
==========================================
Files 276 276
Lines 22753 22966 +213
==========================================
+ Hits 21148 21360 +212
- Misses 1605 1606 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
why: The tier vocabulary was three literals, and an unrecognized tag fell through to `readonly`. A project renaming its tags — libtmux-mcp is about to — would have every tool badged read-only, including the ones that run commands. A badge that names a tier nobody assigned is worse than no badge. what: - Add `fastmcp_safety_tiers`: an ordered vocabulary, highest precedence first, each entry a tag name or a mapping with a tooltip and icon - Default to `destructive` / `mutating` / `readonly`, so a docs build that declares nothing renders exactly as before - Resolve an unmatched tool to no tier, and omit its safety badge rather than reporting the lowest one - Install the vocabulary at `builder-inited`; badges are built from call sites with no `app` in scope, so threading it through each one would reach further than the change needs to
why: The new config value needs a worked example, and the rule that a tool outside the vocabulary loses its badge needs saying where a consumer will look for it. what: - Add a how-to section with a four-tier example, the precedence rule, and the CSS class a new tier name produces
why: The docs job runs only on `main`, so nothing on this branch reaches the live site. what: - Add `scope-overhaul` to the docs workflow's push branches Remove before merging.
why: "Safety" was the misrepresentation, not just the three tag names.
A tier reads as a permission level; these are groups of tools named for
what they do, and this extension has no standing to call one safer than
another in a project whose tags it does not choose.
The rename also surfaced a fourth hardcoded site the first pass missed:
`{fastmcp-summary}` grouped its tables under `readonly` / `mutating` /
`destructive` with fixed headings, so a project with other tags got
empty tables.
what:
- Rename the concept throughout: `fastmcp_toolsets`, `Toolset`,
`build_toolset_badge`, `ToolInfo.toolset`, and the rendered class
`gp-sphinx-fastmcp__toolset-<tag>`
- Drive the summary directive from the declared vocabulary, titling
each section from the tag and describing it from the tooltip
- Ship no default vocabulary: a default badges a project's tools with
words it never used, and an undeclared setting should be visible
rather than silently plausible
- Retag this project's own demo tools, which were still teaching the
tiers on the examples page
why: The stylesheet keyed every colour rule on `-readonly`, `-mutating` and `-destructive`. Renaming the concept moved the class prefix and left the tag half, so `__toolset-teardown` matched no rule and every badge rendered transparent with inherited link-blue — confirmed in the browser, `background-color: rgba(0, 0, 0, 0)`. A rule per tag was always going to break: this extension cannot know what a project calls its toolsets, so it can only ever style the names it guessed. what: - Ship tones — `green`, `blue`, `amber`, `red`, `slate` — and let a project map its tags onto them with `tone` - Default to `slate`: visible, and claiming nothing about a toolset this extension did not name - Drop the emoji `::before` fallbacks keyed on guessed tag names; the icon comes from the declaration - Declare tones for this project's own demo tools
resolve_tool_refs() guarded only on the tool being known, so a tool whose tags match none of the declared fastmcp_toolsets rendered a badge with an empty label, a bare "Toolset: " tooltip and the class gp-sphinx-fastmcp__toolset- , which matches no rule. The tone class did match, so the result was a blank grey pill. build_tool_badge_group() already guards on the toolset being non-empty. Both call sites here now do the same, so an unmatched tool renders as plain text.
The summary directive seeded its groups dict with readonly, mutating and destructive, left from when those three were the whole vocabulary. The render loop keys off the declared toolsets and setdefault() creates any key on demand, so the seeding did nothing but suggest those three tags are still special.
fastmcp-tool-summary groups tools by declared toolset, so a tool carrying none of the declared tags has no section to render in and disappeared from the page with no diagnostic. Since no vocabulary is assumed by default, a project that never sets fastmcp_toolsets lost the whole summary this way. It now warns and names the tools it dropped, matching the warning the directive already emits when no tools are found at all.
Section ids came from the rendered heading, so they tracked display text rather than the tag: two declared tags that title-case to the same words collided, and a tag renamed for presentation moved its anchor. Keying on the tag gives each declared toolset one stable id, and make_id() keeps it a valid HTML id for tags carrying spaces or capitals. This changes existing anchors: a readonly section is now #fastmcp-toolset-readonly rather than #readonly. Links into the old form need updating.
coerce_toolsets() indexed entry["tag"] directly, so a mapping that misspelled the key aborted the build with a bare KeyError naming neither the entry nor conf.py. It also passed any tone straight through, and a tone the stylesheet has no rule for renders a badge with no background, border or text colour and nothing to point at the typo. Both now warn and carry on, as fastmcp_collector_mode already does: a tagless entry is skipped, an unknown tone falls back to slate. TONES names the set the stylesheet actually defines.
TOOLSET_INSPECT, TOOLSET_MANAGE and TOOLSET_TEARDOWN had no callers and hardcoded the readonly / mutating / destructive class names that toolset_class() replaced. Their names had stopped matching their values, so TOOLSET_INSPECT read as the inspect tag while holding gp-sphinx-fastmcp__toolset-readonly.
fastmcp_toolsets told readers that leaving it empty keeps destructive, mutating and readonly. No vocabulary is assumed, so the option's own help contradicted the sentence after it and steered readers away from setting the one option that makes badges render. It also predates tone, which it never mentioned. fastmcp_section_badge_map still offered the same three tags as its example values; it now points at whatever the project declared.
The rename ran as a substitution without a re-read, so it produced "a entry vocabulary", "Build entry sections" and "labeling the entry as an MCP tool" while leaving the tiers and _tier identifiers behind. It also swept a comment describing another file: the stylesheet header claimed sphinx-gp-theme's custom.css matches [aria-label^="Safety entry:"], a selector in neither file, since custom.css still carries the old "Safety tier:" block. Docstrings now say toolset, identifiers follow (tiers to toolsets, _tier to _declared), and the header says what is true: green, amber and red reuse the theme's legacy hex values so a migrating project keeps its colours, while blue and slate are new here. Two docstrings also stated the vocabulary this branch removed. build_toolset_badge documented its argument as one of readonly, mutating or destructive, and Toolset.tooltip promised a "Safety: <tag>" fallback the code never emitted. tutorial.md is a user-facing page the rename never reached.
The active vocabulary is process-global, installed from builder-inited because badges are built at call sites with no app in scope. That makes "when is it cleared?" an easy question to answer wrongly: builder-inited fires once per Sphinx app but build-finished fires after every build(), so pairing them drops the vocabulary on the first build and renders every later one with no tooltip, icon or tone. Building one app twice and asserting the badge still carries its declared tooltip and tone catches that pairing. The badge assertions run under try/finally so the test restores the default instead of leaking its vocabulary into whatever runs next.
The examples page declared teardown, execute and inspect without a tone, so all three rendered slate and the flagship page demonstrated the vocabulary without demonstrating the colour that comes with it. Tones follow libtmux-mcp's mapping, the same page rendered by a real consumer: teardown red, execute amber, inspect green.
Each tone had its own rule repeating the same four declarations with literal hexes, so the stylesheet could only colour the tones it shipped. A project wanting another had nowhere to put it. Tones are now three layers. ``:root`` names the palette, a ``--tone-<name>`` class maps one palette entry onto the badge slot, and a single rule consumes the slot. Restyling a shipped tone means redefining three ``:root`` variables; adding one the extension never heard of means writing one class. Also adds an outline variant, since a term can now ask for one.
One vocabulary was one badge. ``ToolInfo.toolset`` held a single string, so a tool whose tags carried two ideas — a read-only lifecycle tool, say — had to be reported as its risk or as its topic, never both. No ``conf.py`` value could fix that; the model had to change. ``fastmcp_axes`` replaces ``fastmcp_toolsets``. An axis is one independent way of classifying a tool: a tool takes at most one term per axis, and renders one badge per axis in declaration order. Each axis names where its term comes from. ``tags`` matches declared terms against the tool's tags, as before. ``annotations`` reads MCP's own ``ToolAnnotations`` — which the collector already gathered and nothing ever rendered — following the spec, so ``destructiveHint`` describes a tool only once ``readOnlyHint`` is false and an unset hint says nothing rather than defaulting. ``meta:<key>`` reads the mapping MCP passes through to clients untouched. A term carries its own label, tooltip, icon, tone, style, fill and extra classes, so presentation is per term rather than per extension. Badges gain ``__axis-<axis>`` and ``__<axis>-<term>`` classes, so a project can style one axis or one term without going through tones. ``fastmcp-tool-summary`` takes an optional axis name, groups by it, and anchors sections on ``#fastmcp-<axis>-<term>``. ``fastmcp_section_badge_map`` values accept ``term`` or ``axis:term``. ``ToolInfo.toolset`` becomes ``ToolInfo.axes``, and ``ToolInfo.meta`` carries the tool's ``meta`` mapping. Existing tests move to the new API; the snapshot test now pins the axes it renders under, since they are process-global and a Sphinx build earlier in the session would otherwise choose its badge text.
The headline capability had no end-to-end coverage: that one tool is
classified on several axes at once, and that each source reads what it
claims to.
Builds a tool tagged ``mutating``/``lifecycle``, annotated
``readOnlyHint=False`` with ``destructiveHint=False``, and carrying
``meta={"since": "1.2"}``, against three axes reading annotations, tags
and meta. Asserts all three badges render, including that the hint pair
resolves to ``mutating`` per the MCP spec rather than to
``destructive``.
Fails against a renderer that emits one badge per tool.
The how-to described one vocabulary and a fixed tone list, neither of which the extension has any more. Rewritten around what a project actually does: one axis from tags, two axes when the tags carry two ideas, an axis from MCP's annotations or from ``meta``, and a table of what each source reads. Adds the two CSS entry points — redefining a shipped tone's palette variables, and defining a class for a tone the extension does not ship — plus the axis argument to ``fastmcp-tool-summary``.
The demo site's three toolsets become one ``capability`` axis, keeping the tones, tooltips and icons the previous commit gave them. Names the axis rather than leaving it implicit, so the rendered classes on the examples page read ``__axis-capability`` and ``__capability-inspect`` and show what a project's own axis produces.
The S3 sync replaces the whole site with --delete, but the invalidation named only /index.html, /objects.inv and /searchindex.js. Every other page kept serving CloudFront's cached copy until its TTL expired, so a deploy landed in the bucket and stayed invisible for the better part of an hour. A wildcard covers the pages the sync actually replaced, and bills as one path rather than one per page.
Covers the branch's net change in one entry rather than the five successive ones the branch accumulated while the API was still moving: the move from a single vocabulary to named axes, what breaks with it, the annotations and meta sources, per-term presentation, extensible tones, and the badge and summary fixes found along the way.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
sphinx-autodoc-fastmcpbadged tools from three fixed tags —destructive,mutating,readonly— resolved in_collector.pyand ending inelse: safety = "readonly".A project that renames its tags gets every tool badged read-only, including the ones that run commands. libtmux-mcp is about to do exactly that: its capability model replaces the tier ladder with four unordered toolsets (
inspect,manage,execute,teardown), so this has to ship first.What changes
fastmcp_safety_tiersdeclares the vocabulary, in precedence order. Each entry is a tag name or a mapping with atooltipand anicon. The default is the existing three tiers, so a docs build that declares nothing renders unchanged.A tool carrying none of the tags resolves to no tier and renders without a safety badge. Falling back to the lowest tier badges a tool with a name nobody gave it, and read-only is the worst available guess.
The vocabulary is installed at
builder-inited. Badges are built from eight call sites across three modules, one of them a role handler with noappin scope, so threading a parameter through each would reach further than the change needs to.Each tier still gets the CSS class
gp-sphinx-fastmcp__safety-<tag>. The shipped stylesheet colours the three defaults; a project introducing new names styles them in its own CSS. That is documented in the how-to rather than guessed at.Verification
uv run ruff format . --check,uv run ruff check .,uv run mypy .(302 source files),uv run pytest(2275 passed, 161 skipped, 14 snapshots), andjust build-docsare all clean.Every new test was shown failing first. The fallback gate was checked by reinstating
return tiers[-1].tagand confirming two tests go red, then restoring it.Note for review
The final commit adds
scope-overhaulto the docs workflow's push branches so the change could be seen rendered. It is labelled[DO NOT MERGE]and comes off before merge.