Fix #165: make the httpdomain renderer's whitespace delimited options usable

[dstrodtman]

Fixes #165.

Five options in HttpdomainRenderer.option_spec map to None:

"response-examples-for": None,
"request-parameters-order": None,
"example-preference": None,
"request-example-preference": None,
"response-example-preference": None,

docutils reads None in an option_spec as this option is explicitly disabled — assemble_option_dict raises KeyError for it exactly as it does for a name that is not in the spec at all, and the directive machinery turns that into unknown option: "response-examples-for". So none of the five could be used, and the renderer always fell back to its defaults.

The fix is the one the reporter arrived at: give them a converter. All five consumers already expect a sequence of tokens — response-examples-for is membership-tested against status codes, request-parameters-order is lowercased and iterated, and the three preference options are passed to _iterinorder as order_by — so the whitespace splitting that http-methods-order already used is what they want:

.. openapi:httpdomain:: openapi.yml
   :response-examples-for: 200 201 202 2XX 400 404

Rather than repeat the lambda six times I pulled it out as _split_option and reused it for http-methods-order too. It differs from the old lambda in one way: an empty value is rejected with ValueError, which docutils reports as a proper directive error. Two reasons:

• None.split() raised AttributeError on an option written with no value, and since docutils only catches ValueError/TypeError around a converter, that escaped as a traceback rather than a directive error. That was a latent bug in http-methods-order too.
• Parsing an empty value to [] would be quietly destructive for exactly one option. self._response_examples_for = options.get("response-examples-for", self._response_examples_for) treats [] as a present value, so :response-examples-for: with nothing after it would override the default and disable every response example with no error at all. The other four are harmless when empty, but one uniform converter that rejects it seemed better than one that is safe five times out of six.

A limitation worth naming

Whitespace splitting cannot express a media type that carries a parameter, so :response-example-preference: application/json; charset=utf-8 parses as ['application/json;', 'charset=utf-8'] and neither token ever matches. That affects only the three preference options, since the other three take HTTP methods, parameter locations and status codes. Writing the parameter without a space works today, and specs overwhelmingly key content on a bare media type, so I kept whitespace splitting for consistency with http-methods-order rather than special-casing three of the six. If you would rather those three split on commas, say so and I will change them — it is a small diff, and better decided by you than guessed at by me.

Measured on a real spec

Built a generated-looking spec against 0.9.0 installed from PyPI and against this branch, from a neutral working directory so the checkout couldn't shadow the released package:

sphinx-build -W with :response-examples-for: 200 default
  0.9.0        exit 1   ERROR: Error in "openapi" directive: unknown option: "response-examples-for"
  this branch  exit 0   no diagnostics

Worth noting for anyone who tests this: without -W the 0.9.0 build exits 0 while still printing that error and silently ignoring the option, which is how it can go unnoticed.

Tests

tests/renderers/httpdomain/test_options.py is new. Since the whole defect was that these options looked accepted while being inert, the tests that matter are the ones proving each option changes rendered output, so there is one per option: method order, parameter order, 404 examples appearing only once response-examples-for is widened, and the preference options selecting text/plain over the application/json that wins by declaration order. One more asserts a request specific preference beats the shared one. Alongside those, each option is checked to parse to a literal expected list and to reject an empty value. 25 of the 28 cases fail before the change.

The end-to-end half needed two small, additive things from the run_sphinx fixture, with existing callers untouched:

• A return value carrying the captured warning stream, which is what the new tests assert on. The stream is teed to stderr rather than replacing it, so every other caller of the fixture still surfaces build warnings through pytest's capture on failure. unknown option does fail sphinx-build -W with exit 1, but it does not raise out of app.build() on Sphinx 8 or later — the warning is counted and the nonzero exit is applied by the sphinx.cmd.build wrapper, so an API caller like this fixture sees a clean return. On Sphinx 7.4 it does raise SphinxWarning, and the message lands in the exception rather than in the stream. Asserting on the captured warnings detects the regression on both.
• A renderer parameter that writes openapi_default_renderer into the generated conf.py, since these options only exist on the httpdomain renderer while the fixture builds .. openapi::.

On the second point: I first had the fixture emit .. openapi:httpdomain:: instead, and that turned out to warn unknown directive name: openapi:httpdomain on Sphinx 9.0.4. Sphinx reads the colon as a domain prefix, fails to find an openapi domain, warns, and then falls back to the registered name and renders correctly. Harmless for these tests, since the assertion is specific, but it puts a false warning in the stream the tests read, so going through the config value seemed better.

Flagging it in case it is news, though it is unrelated to this fix. Sphinx reserves : in a directive name for a domain prefix, so openapi:httpdomain and openapi:httpdomain:old both take that path, warn, and then fall through to the registered name. I tried Sphinx 7.4.7, 8.1.3, 8.2.3 and 9.0.4 — all four warn and all four render correctly, so this is long-standing rather than a recent change. Neither the colon names nor openapi_default_renderer appear in the README or docs/, so I would guess few users reach it. Happy to file it if it is news to you, but it seemed too cosmetic to open an issue for unprompted.

Full suite passes (521), and pytest --regenerate-rendered-specs tests/ produces no fixture diff, as expected since nothing about rendering changed.

Happy to also document these five options in docs/index.rst if you would like — I left them out because that page's option list is introduced as the options of the openapi directive, and these are specific to the httpdomain renderer, so I did not want to guess at how you would rather structure it. http-methods-order is documented there today with no such distinction.

Context

Found while moving the Ray docs' Jobs API reference onto this extension. With the options unusable, :response-examples-for: cannot be widened past the successful status codes; combined with #166 that reference currently ships with no example payloads at all.

[dstrodtman]

Since this PR changes behavior for anyone already passing these options, I went looking for who that is. Short answer: as far as I can tell, nobody.

I checked the twelve public projects I could find that depend on sphinxcontrib-openapi:

ceph/ceph, cilium/cilium, PowerDNS/pdns, microsoft/CCF, scionproto/scion, getpatchwork/patchwork, performancecopilot/pcp, jupyter-server/enterprise_gateway, stan-dev/httpstan, microsoft/farmvibes-ai, vulnerability-lookup/vulnerability-lookup, openreferral/specification.

None of them passes any of the six options this PR touches. None of them sets openapi_default_renderer either, so they all render with the default old renderer, which these options don't reach. Nobody in that set gets different output after this merges.

Method was per-repo code search for each option name. I ran a positive control first, searching each repo for sphinxcontrib-openapi itself, so a zero result means the option is absent rather than the search being broken.

Worth knowing which of them would pick up a release without doing anything: ceph/ceph, getpatchwork/patchwork, performancecopilot/pcp, and jupyter-server/enterprise_gateway all depend on the package unpinned. The rest pin an exact version, mostly ==0.9.0 or ==0.8.4, so they only see a new release when they bump. The four unpinned ones are all in the group that passes none of these options.

Two limits on this. It only covers projects discoverable through GitHub code search, so private repositories and anything hosted elsewhere aren't in it. And it says nothing about people who tried one of these options, found it did nothing, and removed it again, which is presumably how the original report came about.

Happy to add a changelog note about the behavior change if you want one.