Support help for plugins in sasview

[pkienzle]

Modify docstring conventions for models so that they can be rendered in the SasView help system.

The current docstring markup is done with reStructuredText which only supports full document build.

See #765

[pkienzle]

Consider moving to markdown for model docstrings since that can be rendered directly in the browser.

Use the myst-parser extension to sphinx to support mixed .rst and .md documents.

Summary of the AI conversation for full conversion (periodictable, bumps, sasmodels, sasdata, sasview) follows.

---

Proposal: Transitioning Documentation from Sphinx/reST to MkDocs/Markdown with Runtime Help Integration

🎯 Objective

To modernize the project's documentation workflow by moving from reStructuredText (reST) to Markdown, while implementing a lightweight system to render Python docstrings as HTML within the application's "Help" UI without requiring a full static site generator at runtime.

🛠 Proposed Technical Stack

• Static Site Generator: MkDocs
• API Documentation: mkdocstrings (for automatic Python API generation)
• Theming: Material for MkDocs
• Markdown Extensions: pymdown-extensions (for admonitions and attribute lists)
• Runtime Rendering: Python markdown library $\rightarrow$ Qt HTML widgets

---

📋 Key Technical Decisions

1. Docstring Format & API Documentation

We will transition from .rst to .md. To maintain the ability to automatically generate API documentation from code:

• Use mkdocstrings to extract Python docstrings.
• Use Cross-References in the form of [ClassName][mypackage.module.ClassName] to link between API elements.

2. Runtime "Help" Button Implementation

To avoid running the entire MkDocs engine inside the application, the following pipeline will be used to render help content on demand:

1. Figure generation: Run the models to generate 1D and 2D plots for the default parameter set
2. Extraction: Use importlib to pull the __doc__ string from the target plugin/module.
3. Preprocessing: Apply a Regex pass to translate Sphinx-style links (e.g., :class:) or internal shorthand (e.g., [guide:orientation][]) into full URLs.
4. Conversion: Use the Python markdown library (with extra extensions) to convert the docstring to an HTML string.
5. Display: Inject the resulting HTML into a Qt-supported text widget (e.g., QLabel or QTextEdit).### 3. Handling Cross-References & Indexing
   Since Markdown lacks a global object graph, we will implement a Reference Manifest:

• Create a references.json or a Python dictionary mapping logical IDs (e.g., "orientation_conventions") to their final URLs in the MkDocs site.
• The runtime renderer will replace these IDs with actual hyperlinks before displaying the HTML.

4. Math Rendering in UI

Equations will be handled as follows:

• Simple Math: Use Unicode symbols in docs so no rendering is needed.
• Complex Math: Use KaTeX for complex math, but use unicode symbols when possible so that the raw text is more readable.

---

⚖️ Trade-off Analysis: Sphinx vs. MkDocs

Feature	Sphinx (Current)	MkDocs (Proposed)	Impact
Syntax	reStructuredText	Markdown	$\uparrow$ Writing speed, lower barrier to entry.
Referencing	Global :ref:	Path-based [text](url)	$\downarrow$ Some manual link maintenance.
Visuals	Academic/Basic	Modern/Responsive	$\uparrow$ Better user experience.
PDF Output	Native LaTeX $\rightarrow$ PDF	HTML $\rightarrow$ PDF	$\downarrow$ Loss of professional typesetting.
Build Speed	Slow	Very Fast	$\uparrow$ Faster CI/CD pipelines.

✅ Action Items

• Install mkdocs, mkdocstrings, and pymdown-extensions.
• Convert existing .rst files to .md.
• Implement the markdown conversion pipeline in the application's Help system.
• Create the references.json manifest for global guide links.
• Verify links and formula rendering in the Qt UI.

[llimeht]

Being able render just the documentation for a single model, particularly for marketplace/user plugin models would be nice.

I'm not sure from the above what the vision is for the SasView documentation, where holistic documentation of all models is still desirable - browsable ToC, crosslinks between models, published on the web; that doesn't seem to be captured in the above plan? Is that still rendered via sphinx with the mixed rst/md file pool mentioned - the AI is talking about migrating everything, but your preamble implies not. That's probably just a mismatch in the scoping for the AI, but it's good to have agreement on that scope.

The AI output raised some other queries from me that might help with the next steps

• various parts of the mkdocs / mkdocstrings family are quite young and we tend to see on-going requirements from projects to have particular versions of these tools used for their documentation; we've found this hard downstream. This seems to be slowly stabilising but a risk assessment on the fragility of the system is needed.
• I've only used mkdocstrings in a couple of places and have little first hand experience; I still consistently see people recommending sphinx-apidoc rather than mkdocstrings for complex projects - but a single model might not be a complex enough project to worry.
• a caution about speed: from what I've seen, much of the speed of mkdocs comes from partial rebuilding (using something like mkdocs serve); that means that the first build cost is still there. For a single plugin model, that's OK (and the continuous server model would be unneeded) but for the whole project, the speed up won't be as huge as hoped.
• I'm assuming we wouldn't be proposing to ship SasView with something like mkdocs serve running - open ports, even on localhost, aren't good for a user-facing app.
• the nominated QLabel widget won't work for this - it would be QTextBrowser if the HTML output is simple enough, but that precludes the use of the normal js usage of katex; we'd need unicode maths or SVGs for the maths - I didn't think katex could be pressed into rendering to svg (mathjax can with some effort).
• To actually do katex in Qt would require QtWebEngine but there is on-going work to remove that dependency (and therefore remove PySide6-Addons, so that only PySide6-Essentials is needed). QtWebEngine is a highly problematic dependency that makes SasView harder to port to new architectures, has a massive cost in times for CI, installer, signing, download, installation etc, and means that sasview distributables include code with many known vulnerabilities that we are (informally) triaging to low risk on the basis of rendering only local files.

For a single plugin, is this the sort of thing that can be done with python-markdown on its own?

• there's a well-defined API so introspection of functions and variables isn't that complex
• a single page rendering doesn't need a doc management solution
• I often use the Debian archive as a barometer of 'taste', to look for existing solutions and pause to rethink solutions that are radically different to established norms. Runtime deps on mkdocs are just as rare as those on sphinx, with it being basically only developer tools and plugins for those ecosystems that express such dependencies.
• I'd never normally suggest an NIH solution when stress-tested tools already exist, but looking at doc/genmodel.py makes me wonder if this is already most of the way with a bit more introspection to extract function signatures and a simple HTML renderer?

(Our of curiosity and at the risk of disappearing up one's own fundamental, after I had written the above, I also asked an LLM (GPT-5.6 Sol-High) to critique the AI output. It flagged many of the same issues and also had quite a lot of other things to say about it.)

[pkienzle]

My chat with the AI was regarding the feasibility of replacing reST with markdown for documentation markup. This is why it has a "whole project" feel. From your comments it sounds like the infrastructure isn't solid enough yet.

For the preamble, I asked whether we could support some docs in markdown and some in reST during transition.

I want to support math markup and links to the docs from the model help, so I asked specifically about those issues. KaTex looks feasible now. Linking to the generated docs will require hacks to build a {label: url} mapping and substitute the references during rendering. This will be a maintenance pain point.

The QtWebEngine is a non-issue; we just need to generate html that we can send to the browser. [When I tried this years ago some browsers refused to render javascript for file:/// urls, instead requiring a local server. I don't know if that is still the case.]

markdown to pdf is not well supported.

[pkienzle]

#765 renders the rst with docutils instead of sphinx. It doesn't have the table of contents and the cross-linking that a normal sphinx rendering would produce, but it renders well enough to be usable from a help button.

The sphinx build is leaving an "inventory file" in doc/_build/html/objects.inv. Use that to resolve links within the docs. It can also be used to generate the header/footer lines for linking back to the manual.

The discussion re: markdown can be tabled until the markdown infrastructure is more stable.