Skip to content

Commit 4118ef6

Browse files
Save tutorial projects consistently and verify their fit results (#191)
* Standardize tutorial project save paths * Seed and standardize save paths for Bayesian tutorials * Regenerate tutorial notebooks * Add tutorial-output regression test against fit baseline * Apply pixi run fix auto-fixes * Assert persisted result_kind in tutorial-output test * Run tutorial-output checks in CI * Clean saved tutorial projects before tutorial tests * Skip numeric baseline for platform-sensitive tutorials * Set non-zero defaults for TOF profiles * Trim platform-sensitive skip message * Update tutorial Si structure and doc URLs * Add pattern-display-unification implementation plan * Add top margin above structure scene rectangle * Match single-panel pattern height and x-range to composite * Always render available pattern content; drop include * Amend display-ux ADR for always-on pattern view * Apply formatter to pattern docstring and docs * Drop include= from tutorials for unified pattern view * Reach Phase 1 review gate * Add Raises section to pattern docstring * Set single-panel height via layout construction * Update unit tests for unified pattern view * Remove py3Dmol, relax pillow constraint * Update integration plotting tests for dropped include * Mark Phase 2 verification complete * Record pattern unification as a dedicated ADR * Show pretty units in parameter repr * Omit space_group_Wyckoff loop from IUCr reports * Hide space_group_Wyckoff via _skip_cif_serialization hook * Make Wyckoff skip-serialization hook a staticmethod * Update Wyckoff plan for serialization-hook refactor * Apply latest templates * Assert crysfml engine is loaded in switch-calculator test * Reformat builtins as multiline list * Increase page max-width to 118em * Point report API page at easydiffraction.report package * Track host theme in 3D-view loading box, relabel to plot * Size fit-series scatter plot to the pattern top panel * Increase page and content max-widths * Hide space_group_Wyckoff from parameter tables * Render integer descriptors in parameter table * Pre-bin posterior distribution histogram, not raw samples * Move notebook action buttons to top left above title * Drop duplicate pair scatter hover layer, halve sample cap * Apply ruff format to distribution histogram test * Include integer descriptors in remaining parameter tables * Add regression tests for parameter table rendering * Name tutorial projects to match their save directories * Render pandas tables as inline-styled HTML * Remove dead table theme-sync leftovers * Neutralize MkDocs Material table CSS rules * Rename tutorial projects to match save directories * Temporarily mark all structures as nuclear-only * Match project outputs to save directories * Validate refined tutorial parameters from model and experiment CIFs * Update tutorial projects and remove structure display
1 parent 10a167c commit 4118ef6

101 files changed

Lines changed: 2661 additions & 1167 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.copier-answers.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# WARNING: Do not edit this file manually.
22
# Any changes will be overwritten by Copier.
3-
_commit: v0.11.4
3+
_commit: v0.12.0
44
_src_path: gh:easyscience/templates
55
app_docs_url: https://easyscience.github.io/diffraction-app
66
app_doi: 10.5281/zenodo.18163581

‎.github/workflows/coverage.yml‎

Lines changed: 1 addition & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -63,32 +63,8 @@ jobs:
6363
files: ./coverage-unit.xml
6464
token: ${{ secrets.CODECOV_TOKEN }}
6565

66-
# Job 2: Run integration tests with coverage and upload to Codecov
67-
integration-tests-coverage:
68-
runs-on: ubuntu-latest
69-
70-
steps:
71-
- name: Check-out repository
72-
uses: actions/checkout@v6
73-
74-
- name: Set up pixi
75-
uses: ./.github/actions/setup-pixi
76-
77-
- name: Run integration tests with coverage
78-
run:
79-
pixi run integration-tests-coverage --cov-report=xml:coverage-integration.xml
80-
81-
- name: Upload integration tests coverage to Codecov
82-
if: ${{ !cancelled() }}
83-
uses: ./.github/actions/upload-codecov
84-
with:
85-
name: integration-tests-job
86-
flags: integration
87-
files: ./coverage-integration.xml
88-
token: ${{ secrets.CODECOV_TOKEN }}
89-
9066
# Job 4: Build and publish dashboard (reusable workflow)
9167
run-reusable-workflows:
92-
needs: [docstring-coverage, unit-tests-coverage, integration-tests-coverage] # depend on the previous jobs
68+
needs: [docstring-coverage, unit-tests-coverage] # depend on the previous jobs
9369
uses: ./.github/workflows/dashboard.yml
9470
secrets: inherit

‎.github/workflows/tutorial-tests.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,15 +47,15 @@ jobs:
4747

4848
- name: Test tutorials as python scripts
4949
shell: bash
50-
run: pixi run script-tests
50+
run: pixi run script-tests-checked
5151

5252
- name: Prepare notebooks
5353
shell: bash
5454
run: pixi run notebook-prepare
5555

5656
- name: Test tutorials as notebooks
5757
shell: bash
58-
run: pixi run notebook-tests
58+
run: pixi run notebook-tests-checked
5959

6060
# Job 2: Build and publish dashboard (reusable workflow)
6161
run-reusable-workflows:

‎.prettierignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,9 @@ src/easydiffraction/report/templates/html/vendor/
3131
src/easydiffraction/report/templates/tex/styles/
3232
src/easydiffraction/utils/_vendored/jupyter_dark_detect/
3333

34+
# Tox
35+
.tox
36+
3437
# Misc
3538
.benchmarks
3639
.cache

‎docs/dev/adrs/accepted/crysview-structure-visualization.md‎

Lines changed: 10 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -158,8 +158,7 @@ representation); like the HTML report it can also write a standalone
158158
HTML file to a path. The exact return and save signature is left to the
159159
implementation plan.
160160

161-
Content selection mirrors `pattern(include=...)` rather than inventing a
162-
new vocabulary:
161+
Content selection uses an `include=` argument:
163162

164163
```python
165164
project.display.structure(struct_name='lbco')
@@ -181,13 +180,11 @@ toggles the same features after the initial view is drawn, so `include`
181180
sets the starting state and the modebar refines it.
182181

183182
A companion `project.display.show_structure_options(struct_name=...)`
184-
mirrors the existing `show_pattern_options(expt_name=...)`: it lists
185-
each `include=` option with whether the active engine and the current
186-
structure state support it, and the reason when they do not — for
187-
example `moments` is unavailable until the structure model carries
183+
lists each `include=` option with whether the active engine and the
184+
current structure state support it, and the reason when they do not —
185+
for example `moments` is unavailable until the structure model carries
188186
moment fields, and the `ascii` engine reports the features only the 3D
189-
engines draw. This gives the structure view the same per-option
190-
discoverability the pattern view already offers.
187+
engines draw. This gives the structure view per-option discoverability.
191188

192189
The view also has a spatial extent: which symmetry-equivalent atoms the
193190
scene contains. The scene builder takes the unique (asymmetric-unit)
@@ -613,11 +610,11 @@ a per-call request behave predictably:
613610
## Consequences
614611

615612
- `project.display` gains a spatial view (`structure()`) that
616-
complements the 1D `pattern()` view and reuses the `include=`
617-
vocabulary.
618-
- `project.display` also gains `show_structure_options()`, parallel to
619-
`show_pattern_options()`, so the supported content for a given
620-
structure and engine is discoverable with reasons.
613+
complements the 1D `pattern()` view with an `include=` feature
614+
selector.
615+
- `project.display` also gains `show_structure_options()`, so the
616+
supported content for a given structure and engine is discoverable
617+
with reasons.
621618
- Keeping crystallography in the scene builder and out of renderers lets
622619
several front-ends (Three.js now, Qt Quick 3D later) share one model.
623620
- A switchable `rendering_structure` category

‎docs/dev/adrs/accepted/display-ux.md‎

Lines changed: 11 additions & 75 deletions
Original file line numberDiff line numberDiff line change
@@ -82,8 +82,6 @@ project.display.fit.series(param, versus='diffrn.ambient_temperature')
8282
project.display.posterior.pairs()
8383
project.display.posterior.distribution(param)
8484
project.display.posterior.predictive(expt_name='hrpt')
85-
86-
project.display.show_pattern_options(expt_name='hrpt')
8785
```
8886

8987
`project.analysis.display` is removed from the primary public API. Its
@@ -116,77 +114,14 @@ project.display.pattern(expt_name='hrpt')
116114
project.display.pattern(expt_name='hrpt', x_min=40, x_max=55)
117115
```
118116

119-
By default, `pattern()` uses `include='auto'` and displays as much
120-
useful information as the project state supports:
121-
122-
- measured data if present
123-
- calculated data if linked structure state and calculated intensities
124-
are available
125-
- background if powder Bragg measured and calculated data plus defined
126-
background points are available
127-
- Bragg ticks if powder Bragg measured and calculated data plus
128-
reflection rows are available
129-
- residual if both measured and calculated data are available and the
130-
experiment type supports a residual panel
131-
- excluded regions if available on the experiment
132-
- uncertainty bands where posterior predictive data exists and the chart
133-
engine supports them
134-
135-
Specific subsets are selected with `include`:
136-
137-
```python
138-
project.display.pattern(expt_name='hrpt', include='auto')
139-
project.display.pattern(expt_name='hrpt', include='measured')
140-
project.display.pattern(expt_name='hrpt', include='calculated')
141-
project.display.pattern(
142-
expt_name='hrpt',
143-
include=('measured', 'calculated', 'background', 'residual', 'bragg'),
144-
)
145-
```
146-
147-
`include` was chosen over alternatives:
148-
149-
| Name | Reason not selected |
150-
| ------------- | ----------------------------------------------- |
151-
| `layers` | Sounds graphical rather than user intent. |
152-
| `components` | Precise, but longer. |
153-
| `content` | Too broad. |
154-
| `view` | Better for presets than arbitrary combinations. |
155-
| `series` | Does not fit residual rows or Bragg ticks well. |
156-
| boolean flags | Explicit, but scales poorly. |
157-
158-
Add discovery for supported pattern content:
159-
160-
```python
161-
project.display.show_pattern_options(expt_name='hrpt')
162-
```
163-
164-
The table shows option name, description, availability for the
165-
experiment, whether `include='auto'` includes it, and the reason an
166-
option is unavailable.
167-
168-
Pattern option names:
169-
170-
- `auto`
171-
- `measured`
172-
- `calculated`
173-
- `background`
174-
- `residual`
175-
- `bragg`
176-
- `excluded`
177-
- `uncertainty`
178-
179-
`uncertainty` is available where posterior predictive data exists for a
180-
supported experiment and the active chart engine can render bands. It is
181-
unavailable, with a clear reason, when no posterior predictive data is
182-
present.
183-
184-
Explicit combinations are validated against the same project state used
185-
by `include='auto'`. `background`, `bragg`, and `residual` require both
186-
measured and calculated data in the same view. `excluded` requires
187-
measured, calculated, or uncertainty content in the same view, and
188-
excluded-region overlays currently require the experiment's default
189-
x-axis.
117+
`pattern()` renders every kind of data the project state supports —
118+
measured, calculated, residual, Bragg ticks, background, excluded
119+
regions, and posterior predictive uncertainty, each shown when
120+
available. It takes no view-selection argument. The content rules, the
121+
removed `include` / `show_pattern_options` design, and the shared
122+
single- and multi-panel figure sizing are recorded in the
123+
[Unified Pattern View](pattern-display-unification.md) ADR, which
124+
supersedes the `include`-based pattern design once described here.
190125

191126
## Deterministic And Bayesian Consistency
192127

@@ -230,8 +165,9 @@ users should not need to decide the output type before asking for
230165
information. Some outputs may render as a chart or a table depending on
231166
backend and state.
232167

233-
Separate `measured()` and `calculated()` methods were rejected because
234-
they duplicate `pattern(..., include=...)`.
168+
Separate `measured()` and `calculated()` methods are unnecessary:
169+
`pattern()` shows every available kind of data directly, so there is no
170+
subset for them to select.
235171

236172
## Consequences
237173

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
# ADR: Unified Pattern View
2+
3+
## Status
4+
5+
Accepted and implemented.
6+
7+
## Date
8+
9+
2026-06-04
10+
11+
## Context
12+
13+
`project.display.pattern()` was originally specified by the
14+
[Display UX Facade](display-ux.md) ADR with an `include=` argument that
15+
assembled a view from named layers (`measured`, `calculated`,
16+
`background`, `residual`, `bragg`, `excluded`, `uncertainty`), plus a
17+
`show_pattern_options()` discovery table. `include='auto'` already chose
18+
the most informative combination from project state.
19+
20+
In practice this surfaced several problems:
21+
22+
- The single-panel path (`plot_meas`/`plot_calc` → `plot_powder`) and
23+
the composite path (`build_powder_meas_vs_calc_figure`) computed
24+
figure height and x-range independently, so they drifted. A
25+
measured-only view rendered at the full three-panel height in the lazy
26+
docs runtime — the
27+
[Plotting & Docs Performance](plotting-docs-performance.md) skeleton
28+
fell back to `DEFAULT_HEIGHT * PLOTLY_HEIGHT_PER_UNIT` — and gained
29+
stray left/right autoscale padding the composite did not have.
30+
- `'excluded'` was an opt-in overlay, but excluded regions are a
31+
property of the experiment, not a viewing choice; `include='measured'`
32+
and `include=('measured', 'excluded')` produced different plots of the
33+
same data, which read as redundant.
34+
- For a scientist audience, choosing layers is friction. The project
35+
state already determines what is meaningful to show, which is exactly
36+
what `include='auto'` computed.
37+
38+
## Decision
39+
40+
`pattern(expt_name, x_min=None, x_max=None, *, x=None)` always renders
41+
every kind of data the project state supports — the former
42+
`include='auto'` behaviour is now the only behaviour. The `include`
43+
parameter, the `show_pattern_options()` method, and the option-status
44+
discovery table are removed. Strict-subset views (for example
45+
measured-only once a calculation exists) are intentionally no longer
46+
offered; zooming (`x_min`/`x_max`) and the x-axis variable (`x`) remain.
47+
48+
Excluded regions are always shaded when defined on the experiment,
49+
skipped only when a custom `x` axis variable is selected, because the
50+
overlay cannot be mapped onto an arbitrary axis.
51+
52+
Single-panel and composite charts share one figure-sizing and x-range
53+
core. `plot_powder` builds its layout with the same
54+
`_single_main_panel_height_pixels(...)` height and tight
55+
`_composite_x_range(...)` as the composite main row, and `_get_layout`
56+
already applies the composite margins. A one-row chart is therefore the
57+
top row of the multi-row chart pixel for pixel, by construction, so the
58+
two paths cannot diverge again.
59+
60+
This supersedes the `include`-based pattern design in the
61+
[Display UX Facade](display-ux.md) ADR. The remainder of that ADR — the
62+
facade grouping, renderer categories, and naming rules — still stands.
63+
64+
## Consequences
65+
66+
- `pattern(expt_name=...)` is the whole pattern API surface; tutorials,
67+
docs, and tests no longer pass `include=`.
68+
- The project is in beta, so this replaces the previous API with no
69+
compatibility shim; tutorials and tests are updated to the current
70+
API.
71+
- Sizing and range differences between one- and three-panel views are
72+
prevented structurally, not patched per call.
73+
- `structure(include=...)` and `show_structure_options()` are
74+
unaffected: choosing which 3D features to draw remains a genuine
75+
viewing choice (see
76+
[Crystal Structure 3D Visualization](crysview-structure-visualization.md)).
77+
78+
## Alternatives Considered
79+
80+
Keeping `include` as the view-selection vocabulary (the original
81+
design). `include` had been chosen over `layers`, `components`,
82+
`content`, `view`, `series`, and boolean flags because it read as user
83+
intent and fit residual rows and Bragg ticks. It is removed now because
84+
the only combination users reached for in practice was the automatic
85+
"show everything available" view; the subset combinations added API
86+
surface and a discovery table without a matching workflow, and the
87+
parallel single-panel rendering path was the source of the sizing and
88+
range divergence.
89+
90+
Keeping `'excluded'` as an opt-in overlay, or as a redundant no-op
91+
token, was rejected: excluded regions belong to the experiment, so
92+
shading them is automatic whenever they are present.

‎docs/dev/adrs/accepted/wyckoff-letter-detection.md‎

Lines changed: 21 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,14 @@
22

33
**Status:** Accepted **Date:** 2026-06-01
44

5+
> **Amendment (2026-06-04):** the derived `space_group_Wyckoff` loop is
6+
> now **code-only** — it is excluded from IUCr/report output as well as
7+
> from project CIF. Wording below that calls the loop "report-facing" or
8+
> says the report writer "emits" it (the §"`space_group_Wyckoff` loop"
9+
> decision and the matching Consequences bullet) is superseded: the
10+
> table is reachable in code via `structure.space_group_wyckoff` but is
11+
> never serialized.
12+
513
## Group
614

715
Structure model.
@@ -447,13 +455,17 @@ which fall out naturally below.
447455
already emits the resolved `Wyckoff_symbol` for every atom and now
448456
also emits `_atom_site.site_symmetry_multiplicity`.
449457
- **`space_group_Wyckoff` loop.** The derived category is model-owned
450-
and report-facing, but it is not persisted in project CIF. `Structure`
451-
explicitly excludes it from project-save serialization via
452-
`_serializable_categories()`, overriding `CategoryOwner`'s default of
453-
serializing all owned categories. The IUCr/report writer emits the
454-
`_space_group_Wyckoff.*` loop from the derived category because that
455-
loop is useful report output even though it is redundant persisted
456-
state.
458+
and code-facing only; it is not serialized. The collection suppresses
459+
its own output through `_skip_cif_serialization()` returning `True`,
460+
the same collection-owned hook `atom_site_aniso` uses (conditionally);
461+
every serialization path that consults it honours the suppression —
462+
`category_collection_to_cif` (project CIF) and the report data context
463+
— so the `_space_group_Wyckoff.*` loop never appears in project CIF or
464+
HTML/TeX reports, and the hand-rolled IUCr writer does not emit it
465+
either. The decision lives on the category, so `Structure` needs no
466+
`_serializable_categories()` override. The table is redundant derived
467+
state, reachable in code via `structure.space_group_wyckoff` (amended
468+
2026-06-04).
457469
- **Derived values on read.** Multiplicity is recomputed from the
458470
letter, so any incoming `_atom_site.site_symmetry_multiplicity` is
459471
ignored rather than trusted; the `space_group_Wyckoff` category is
@@ -677,8 +689,8 @@ may miss:
677689
structure's space group (each entry's letter / multiplicity /
678690
site_symmetry / coords match `SPACE_GROUPS`), rebuilds when the space
679691
group changes, refuses all public mutation paths, is empty for an
680-
absent group, is omitted from project CIF, and is emitted in
681-
IUCr/report output;
692+
absent group, and is omitted from both project CIF and IUCr/report
693+
output (code-only, reachable via `structure.space_group_wyckoff`);
682694
- CIF round-trip stability — a written letter reloads verbatim, an
683695
omitted one re-derives to the same value, and an unsupported-group row
684696
keeps `None` multiplicity whether its letter is empty, explicitly

‎docs/dev/adrs/index.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,4 +56,5 @@ folders.
5656
| User-facing API | Accepted | String Paths and Live Descriptors | Separates persisted field selectors from references to live model parameters. | [`string-paths-and-live-descriptors.md`](accepted/string-paths-and-live-descriptors.md) |
5757
| User-facing API | Accepted | Switchable Category API | Places multi-type category selectors on the owner and omits public selectors for fixed or single-type categories. | [`switchable-category-api.md`](accepted/switchable-category-api.md) |
5858
| User-facing API | Accepted | Switchable Category Owned Selectors | Moves the writable `type` selector and `show_supported()` onto the category itself; collapses the CIF duplication. | [`switchable-category-owned-selectors.md`](accepted/switchable-category-owned-selectors.md) |
59+
| User-facing API | Accepted | Unified Pattern View | `pattern()` always renders available data, drops `include`, and unifies single- and three-panel figure sizing. | [`pattern-display-unification.md`](accepted/pattern-display-unification.md) |
5960
| User-facing API | Accepted | Value-Selector Discovery | Gives enumerated value fields a per-descriptor `show_supported()`, beside the three category-level selector families. | [`value-selector-discovery.md`](accepted/value-selector-discovery.md) |

0 commit comments

Comments
 (0)