From 863c681dba6ef25ea406d91bb375bf2494a2ca34 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Fri, 16 Oct 2026 18:41:07 +0200 Subject: [PATCH 01/68] =?UTF-8?q?docs:=20server=20push=20=E2=80=94=20actio?= =?UTF-8?q?ns=20declare=20what=20they=20change,=20the=20server=20tells=20t?= =?UTF-8?q?he=20GUI=20(spec=208)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 --- .../specs/2026-10-16-server-push-design.md | 324 ++++++++++++++++++ 1 file changed, 324 insertions(+) create mode 100644 docs/superpowers/specs/2026-10-16-server-push-design.md diff --git a/docs/superpowers/specs/2026-10-16-server-push-design.md b/docs/superpowers/specs/2026-10-16-server-push-design.md new file mode 100644 index 000000000..501e14588 --- /dev/null +++ b/docs/superpowers/specs/2026-10-16-server-push-design.md @@ -0,0 +1,324 @@ +# Server push: actions declare what they change, the server tells the GUI — design (spec 8) + +The server tells every client that shows a piece of data when that data changes, so the GUI updates without +being asked. What changes is declared **once, on the action**: an action says which record types it +updates or changes, and the framework works out which cached queries, in which clients, show them. A UI +document then declares only what a screen **shows** (its queries) and what an edit **sends** (its +mutations). It no longer declares what to refresh after an action. + +The same mechanism updates the GUI however the change was made: by this screen, by another screen of the +same client, by another user, or by an agent acting through the server. + +## Contents + +[1 Intent](#1-intent) · [2 What exists today](#2-what-exists-today) · [3 Concepts](#3-concepts) · +[4 Declaring records and effects](#4-declaring-records-and-effects) · [5 The catalog](#5-the-catalog) · +[6 The server-to-client channel](#6-the-server-to-client-channel) · [7 Watching](#7-watching) · +[8 Fan-out on the server](#8-fan-out-on-the-server) · [9 The client cache](#9-the-client-cache) · +[10 Documents](#10-documents) · [11 Agents through the server](#11-agents-through-the-server) · +[12 Security](#12-security) · [13 Tests](#13-tests) · [14 Delivery](#14-delivery) · +[15 Out of scope](#15-out-of-scope) + +## 1. Intent + +**Today**, a LIMS capture dialog declares what to refresh: + +```json +"capture": { "model": "sample", "action": "Lims_CaptureConcentration", "body": { ... }, + "invalidates": ["results"] }, +"results": { "model": "sample", "action": "Lims_ListResults", "body": { ... }, + "refreshOn": ["Lims_CaptureConcentration"], "refreshEvery": 5000 } +``` + +Every document that calls `Lims_CaptureConcentration`, and every document that lists results, has to know +about the other. A change made by another client is seen only on the next poll. + +**With this design**, the action says what it changes, once, where it is registered: + +```cpp +BRIDGE_REGISTER_ACTION(SampleModel, CaptureConcentration, "Lims_CaptureConcentration", + morph::effects::updates) +``` + +The documents drop `invalidates`, `refreshOn` and the polling `refreshEvery`. When a capture succeeds: + +- the calling client patches the new `ResultRow` into every cached query that holds it; +- the server tells every other client watching `ResultRow`, and their mounted queries showing that row refetch. + +**Success criteria.** + +1. A document that only shows data and sends edits updates correctly with no refresh declarations. +2. A change made by client A appears in client B's mounted screen within one push latency, with no polling. +3. An agent calling an action through the server updates every open GUI the same way. +4. The examples lose their refresh declarations, about 40 today, and their tests stay green. + +## 2. What exists today + +| Fact | Where | +|---|---| +| The query cache is per client session, keyed by `(model type, instance, action, canonical body)`. `invalidates`, `refreshOn` and `{"refetch"}` mark entries stale. | spec 5 §5 "The query cache" | +| `refreshOn` matches by **model type, not instance**, because two private instances of a database-backed model read the same rows. | spec 5 §5 "Queries" | +| `patch` merges a reply into a cached row found by its `id` field. `optimistic` applies the same shape before the reply. | spec 5 §5 "Mutations" | +| A change by another client is not pushed. Spec 5 §15 names a negotiated `ui-event` kind as the way to add it. | spec 5 §5, §15 | +| A bridge subscription reaches only handlers in the same process, never across connections. | `docs/spec/core/shared_instances.md` | +| `RemoteServer::handle(msg, reply, cid)` answers each request through its own `reply`. The server holds no way to send a connection a message it did not ask for. | `include/morph/core/remote.hpp` | +| Both transports track a `ConnectionId` per connection and call `closeConnection` on disconnect. | `include/morph/net/socket_server.hpp`, `src/qt/qt_websocket_server.cpp` | +| The client matches replies by `callId`. A message that answers no pending call is dropped. | `include/morph/core/wire.hpp` | +| `ActionInfo` carries `modelType`, `calculatedFields`, `secretFields`, `credential` and `listFields`. Nothing says whether an action reads or writes. | `include/morph/ui/document.hpp` | +| `BRIDGE_REGISTER_ACTION` already carries one per-action flag (`loggable`). A view's row identity is a structurally detected `rowKey`, defaulting to `"id"`. | `docs/spec/core/registry.md`, `docs/spec/forms/views.md` | + +## 3. Concepts + +- **Record type.** A reflected aggregate that action results carry and screens show, with a registered name + and a key member. Examples: `ResultRow` keyed by `id`, `AccountInfo` keyed by `id`. The record type, not the + model type, is the unit of change. One record type can be shown through several models, as when + `CustomerModel::ListAccounts` and `AccountModel::GetAccount` both return `AccountInfo`. +- **Effect.** What a successful action did to record types. One of: + - `updates`: the result carries the new state of the `R` records it changed, identified by their keys. + - `changes`: records of `R` were created, removed or changed in a way the result does not carry. + - nothing: the action reads. +- **Change.** What the framework derives from an effect after a successful execute: `{record, keys?}`. Keys + are present for `updates`, taken from the result. They are absent for `changes`, which means "any record of + this type". +- **Watch.** A client's statement of which record types its cached queries hold. It is maintained by the + client and authorised by the server. +- **Event.** A server-to-client message carrying changes, sent to the connections watching their record types. + +The unit is the record type, not the model instance, for the reason `refreshOn` already matches by model type: +two private instances of one database-backed model read the same rows. + +## 4. Declaring records and effects + +### Records + +A record type is declared where its codec already is, in the same structural way as a view's `rowKey`: + +```cpp +// Detected structurally: a member `id`, or an explicit trait. +template <> struct morph::model::RecordTraits { + static constexpr std::string_view name = "Bank_Account"; + static constexpr auto key = &bank::dto::AccountInfo::id; +}; +// or the macro: BRIDGE_REGISTER_RECORD(bank::dto::AccountInfo, "Bank_Account") // key defaults to `id` +``` + +- The key is an integral or `std::string` member, the types `ModelKey` already admits. A record type with no + key can only be named by `changes`. +- The record registry is process-wide and filled during the registration phase, like the action registry, and + it is checked by the same latch. + +### Effects + +Effects are an optional trailing argument of the registration macro. The macro is already where the framework +meets the application's types. A DTO stays free of morph types, which keeps invariant 1: the model does not +know the framework exists. + +```cpp +BRIDGE_REGISTER_ACTION(AccountModel, Deposit, "Deposit", morph::effects::updates) +BRIDGE_REGISTER_ACTION(AccountModel, Transfer, "Transfer", morph::effects::updates) +BRIDGE_REGISTER_ACTION(PayeeModel, AddPayee, "AddPayee", morph::effects::changes) +BRIDGE_REGISTER_ACTION(PayeeModel, RemovePayee, "RemovePayee", morph::effects::changes) +BRIDGE_REGISTER_ACTION(PayeeModel, ListPayees, "ListPayees") // reads: no effects +``` + +- **`updates` finds the records in the result.** The result may be an `R`, a `std::vector`, or an + aggregate whose members, found by reflection, hold an `R` or a `std::vector`. `Transfer` returns both + accounts this way. A result in which none is found is a registration error, reported when the program starts, + not when the action first runs. +- **`changes` needs nothing from the result.** It covers creation and removal, whose effect on a list the + framework cannot compute: whether a new payee belongs in a filtered list is the server's query to answer. +- **Several effects compose:** `morph::effects::updates, morph::effects::changes`. +- **No declaration means the action reads, and nothing is pushed.** This default under-notifies rather than + over-notifies. An action wrongly treated as a write would make every fetch invalidate the queries that + showed it, and they would refetch in a loop. An action that writes and declares nothing behaves as it does + today. §10's load-time check finds the documents that still rely on refresh declarations for it. +- **Only success has effects.** A failed action, or one that was cancelled, changes nothing. + +`ActionTraits` gains `effects()`, returning the list as data (`{record name, kind}`). The framework reads +effects only through that function. + +## 5. The catalog + +`ActionInfo` gains two facts, filled from the registry in local mode and served by `ui-catalog` remotely: + +| Field | Meaning | +|---|---| +| `effects` | The action's effects: `[{record, kind: "updates" \| "changes"}]`. | +| `records` | The record types the action's **result** carries, found by the same reflection as `updates`. A cached query holds exactly these. | + +`records` lets a client decide, without the server, whether a cache entry holds record `R`: the entry's action's +`records` contains `R`. That is the whole link between "R changed" and "this query must update". + +## 6. The server-to-client channel + +The wire gains its first server-initiated message. It is a core facility, not a UI one: the core `hello` +negotiates it, and the UI layer is its first user. + +- **Negotiation.** The core `hello` gains a capability list, and the client offers `events`. A server sends + events only to connections that offered it. A client that never offers it sees today's protocol unchanged. +- **The envelope.** `kind: "event"`, `callId: 0`, `body: {topic, payload}`. The topic is a string, `changes` + for this design. A client routes it by kind before reply matching, so an event is never mistaken for an + orphaned reply. +- **The server side.** `RemoteServer` gains `openConnection(cid, send)`, with `closeConnection(cid)` as its pair. + A transport registers the connection's outbound writer when the connection opens. + `RemoteServer::post(cid, envelope)` writes through it on the transport's thread, under the same size cap + (`kMaxEnvelopeBytes`) and the same write-failure handling as replies. +- **The client side.** `IBackend` gains `setEventHandler(fn)`, delivered on the bridge's executor. The remote + backend calls it for each `event`. The local backend calls it in-process, with no wire: local mode gets the + same behaviour from the same code above the backend. +- **Order.** Events on one connection arrive in the order the server posted them. No order is promised between + an event and an unrelated reply. + +## 7. Watching + +A client watches the record types its cache holds, so the server sends each connection only what it can show. + +- **Lifetime.** A record type is watched while at least one cache entry holds it, mounted or kept (spec 5 §5: + for 5 minutes). The client counts references per record type. It sends `watch` when a count leaves zero and + `unwatch` when it returns to zero. Both are core kinds with `{records: [name, ...]}` and an `ok` reply. +- **Batching.** Watch changes made in one reactive batch go out as one message, so mounting a screen with ten + queries sends one `watch`. +- **Reconnect.** After a reconnect or a backend switch, the client sends its whole watch set again and marks + every entry stale, as spec 6 §9 already does for a backend switch. Events missed while disconnected are not + replayed: the stale marks cover them. +- **Granularity.** Watching is by record type, not by key. Per-key watching would need the server to hold + every key every client shows, and keys change as lists scroll. The event carries keys, and the client filters + them (§9). + +## 8. Fan-out on the server + +After an `execute` succeeds, and before its reply is written: + +1. The server reads the action's effects and derives the changes: record name and, for `updates`, the keys found + in the result. +2. It finds the connections watching those record types, **excluding the connection that made the call**, whose + client applies the effects from its own reply (§9). +3. It filters each connection's view by authorisation (§12) and posts one `changes` event per connection. + +- **Coalescing.** Changes for one connection are merged for up to 50 ms, or until 256 keys are queued, then sent + as one event. Keys are merged per record type; a `changes` for a type absorbs its `updates` keys. One hundred + writes in a burst are then one refetch per showing query, not one hundred. +- **Where the watch table lives.** On the server strand, next to the connection scopes `closeConnection` already + reclaims. A closed connection drops its watches. +- **Every path that executes reaches fan-out:** the UI service, the plain `execute` kind, an offline replay, and + an agent (§11), because fan-out sits on the dispatch path they share. + +## 9. The client cache + +The interpreter's query cache applies a change the same way whether it came from its own reply or from an event. + +| Change | Own reply (the caller) | Event (another connection) | +|---|---|---| +| `updates R` with keys | Patch: each entry holding `R` merges the new records into the rows with those keys, using the existing `patch` merge and the catalog's `listFields` for a list inside a record. No refetch. | Each entry holding `R` whose value contains one of the keys is marked stale, and refetched if mounted. | +| `changes R` | Each entry holding `R` is marked stale, and refetched if mounted. | The same. | + +- **Why other clients refetch rather than receive values.** The new values were computed for the caller. Another + user may see different fields (`secretFields`) or may not be allowed the record at all. A refetch goes through + the receiver's own authorised read. Pushing values to other receivers is §15's first deferred item. +- **Batching.** One event, or one reply's effects, is applied in one reactive batch, so a screen updates in one + frame. +- **`optimistic` is unchanged.** It still shows a guess before the reply. The reply's patch then confirms or + corrects it, exactly as a declared `patch` does today. +- **Entries no query mounts** are only marked stale. They refetch if a screen mounts on them. + +## 10. Documents + +With effects declared, a document needs no refresh declarations. The vocabulary changes in two steps. + +**Step 1, additive within `ui/1`.** `invalidates`, `patch` and `refreshOn` stay valid and keep working, and +effects apply as well. The loader, which already has the catalog, reports: + +- **redundant:** an `invalidates` or `refreshOn` that the mutation's action's effects already cover. This is a + warning, so a document can migrate one screen at a time. +- **uncovered:** a mutation whose action declares no effects, with no `invalidates` or `patch` of its own and + no query naming it in `refreshOn`. After it succeeds, nothing on any screen updates. This is a warning, + because the action may genuinely change nothing that is shown. It points at the actions still to annotate. + +**Step 2, removal.** Once the examples have migrated, `invalidates`, `patch` and `refreshOn` are refused at load. +A document that needs to refetch for a reason no action declares uses the existing `{"refetch": "q"}` command. +`refreshEvery` stays, for data that changes with time rather than through actions, such as a clock or an +external feed. Whether step 2 waits for a vocabulary version (`ui/2`) or happens inside `ui/1` is decided when +the examples have migrated (§14, part 8). + +## 11. Agents through the server + +An agent acts on the application through the same dispatch as everything else, so this design is what makes its +changes visible. + +- **The MCP adapter** is a server-side component. It lists the actions a principal may run (the catalog's + `permitted`, with their JSON shapes from the registry) as MCP tools. It runs each tool call through + `ActionDispatcher` as that principal, with the same `authorize` and `authorizeInstance` checks, the same journal + entry and the same `ui-evaluate` dry run as a GUI client. The agent may commit directly. +- **Fan-out treats the adapter as one more connection**, so every open GUI showing the changed records updates + through §8 and §9. +- **Model access and keys are not part of this design.** The adapter does not call a model provider. An external + agent (Claude Desktop, Claude Code) brings its own. Where keys would live for an assistant built into the client + is still undecided, and it does not affect anything above. + +## 12. Security + +- **A watch reveals nothing by itself.** The server accepts a watch for record type `R` only when the principal + may run at least one action whose `records` contain `R`. Otherwise it refuses with `err`, and the client logs it. +- **An event reveals that records of `R` changed, and their keys.** Keys are ids the receiver could already see + through some action returning `R`, but not necessarily the same records. Where that matters, because a model + filters rows per principal, a record type registered with `keys: private` is pushed without keys to principals + other than the caller's. Every query holding `R` then refetches, which costs more and reveals nothing. +- **Values never travel to other connections** (§9), so `secretFields` and row-level filtering stay on the read + path, where they are enforced today. +- **Rate.** Coalescing (§8) bounds events per connection to 20 a second. A connection that cannot drain its + outbound queue past the transport's existing limit is closed, as for a slow reader of replies. + +## 13. Tests + +Every test below must fail if the feature does nothing, so each names the mutation that should break it. + +- **Records and effects (core).** Registration extracts keys from an `R`, a `vector` and a nested member. A + result with no `R` under `updates` fails registration. *Mutation: drop the reflection walk; key extraction + tests fail.* +- **Channel (core, net, qt).** The same conformance case is run on both transports and on the local backend: a + posted event reaches the client's handler. An event to a client that did not offer `events` is never sent. + An event is not taken for a reply. *Mutation: route events through reply matching; the routing case fails.* +- **Fan-out (core).** Two connections, both watching `R`. An `updates` execute from A posts to B and not to A. + An unwatched type posts nothing, and a burst of 100 coalesces to one event. *Mutation: remove the origin + exclusion; A receives its own event.* +- **Cache (ui).** A recording-backend screen with a list query and no refresh declarations. An own reply patches + the row without a fetch, and the fetch counter stays flat. An event marks it stale and refetches it. + *Mutation: skip the cache step; the shown row keeps its old value.* +- **End to end (examples).** One rung runs two clients against one server. Client A's mutation updates client B's + mounted screen with no `refreshOn` and no `refreshEvery` in either document. *Control: unregister the action's + effect; B does not update.* + +## 14. Delivery + +Each part lands as its own pull request to `master`. Parts with no dependency between them can run side by side. + +| # | Part | Layer | Needs | +|---|---|---|---| +| 1 | Records and effects: `RecordTraits`, `BRIDGE_REGISTER_RECORD`, `morph::effects`, `ActionTraits::effects()`, change derivation from a result | core | none | +| 2 | Server-to-client channel: core `hello` capabilities, `event` kind, `openConnection` and `post`, both transports, `setEventHandler`, in-process delivery in the local backend | core, net, qt | none | +| 3 | Catalog: `ActionInfo::effects` and `records`, local lookup and `ui-catalog` | ui | 1 | +| 4 | Watching and fan-out: `watch` and `unwatch` kinds, the watch table, origin exclusion, coalescing, authorisation (§12) | core | 1, 2 | +| 5 | Client cache: apply effects from own replies and from events; watch reference counting; reconnect | ui | 2, 3 (4 for its remote tests) | +| 6 | Document checks: redundant and uncovered warnings; spec 5 §5 updated | ui | 3, 5 | +| 7 | Examples: declare records and effects in the seven rungs that use refresh declarations, remove those declarations, and add the two-client end-to-end test | examples | 5, 6 | +| 8 | Removal: refuse `invalidates`, `patch` and `refreshOn` at load, in `ui/1` or as `ui/2` (decided then) | ui | 7 | +| 9 | MCP adapter: catalog actions as tools, run as a principal, through fan-out | net, ui | 4 | + +**Ready now:** parts 1 and 2. They are independent, and both are core. **Natural pairs:** 1 ∥ 2, then 3 ∥ 4. +Part 9 can run beside anything after part 4. Part 5 runs beside part 4: the local backend delivers events +in-process, so its local tests need no server fan-out, and only its remote tests wait for part 4. + +## 15. Out of scope + +- **Pushing values to other connections.** Receivers refetch through their own authorised read. Sending the new + record, filtered per receiver, would save that round trip. It needs per-receiver field filtering on the server, + and it is the first follow-up once refetch traffic is measured. +- **Per-key and per-field watching.** Watching is by record type, and the event carries keys. Field-level effects + (`updates` naming fields) would let a client skip refetches for fields a screen does not show. That is + deferred until a measurement shows refetches cost something. +- **Effects computed at run time** beyond the records found in the result, such as an explicit change list a model + returns. `updates` over a reflected result covers the known cases, `Transfer` included. +- **An assistant built into the client, and where its model keys live.** §11 covers agents acting through the + server only. +- **Server push for anything other than changes:** progress of a long action, notifications. The channel (§6) has + topics so they can be added without another negotiation, but none is defined here. From 62763fe0bcf5499909ec0b1cb6eea20ec001d1f5 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Fri, 16 Oct 2026 19:00:25 +0200 Subject: [PATCH 02/68] =?UTF-8?q?docs:=20spec=208=20=E2=80=94=20address=20?= =?UTF-8?q?the=20first=20review=20round?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 --- .../specs/2026-10-16-server-push-design.md | 511 +++++++++++------- 1 file changed, 311 insertions(+), 200 deletions(-) diff --git a/docs/superpowers/specs/2026-10-16-server-push-design.md b/docs/superpowers/specs/2026-10-16-server-push-design.md index 501e14588..eea547f8b 100644 --- a/docs/superpowers/specs/2026-10-16-server-push-design.md +++ b/docs/superpowers/specs/2026-10-16-server-push-design.md @@ -1,13 +1,13 @@ # Server push: actions declare what they change, the server tells the GUI — design (spec 8) The server tells every client that shows a piece of data when that data changes, so the GUI updates without -being asked. What changes is declared **once, on the action**: an action says which record types it -updates or changes, and the framework works out which cached queries, in which clients, show them. A UI -document then declares only what a screen **shows** (its queries) and what an edit **sends** (its -mutations). It no longer declares what to refresh after an action. +being asked. What changes is declared **once, next to the action's registration**: an action says which +record types it updates or changes, or that it changed its own instance's state. The framework then works out +which cached queries, in which clients, show that data. A UI document declares only what a screen **shows** +(its queries) and what an edit **sends** (its mutations). It no longer declares what to refresh after an action. The same mechanism updates the GUI however the change was made: by this screen, by another screen of the -same client, by another user, or by an agent acting through the server. +same client, by another user, by an offline replay, or by an agent acting through the server. ## Contents @@ -21,235 +21,316 @@ same client, by another user, or by an agent acting through the server. ## 1. Intent -**Today**, a LIMS capture dialog declares what to refresh: +**Today**, LIMS's sample-entry screen declares, query by query, which actions refresh it, and each mutation +declares which queries it invalidates (`examples/lims/screens/lims_screens.cpp`): -```json -"capture": { "model": "sample", "action": "Lims_CaptureConcentration", "body": { ... }, - "invalidates": ["results"] }, -"results": { "model": "sample", "action": "Lims_ListResults", "body": { ... }, - "refreshOn": ["Lims_CaptureConcentration"], "refreshEvery": 5000 } +```cpp +auto const header = screen.query("header", sample, {}, + {.when = ready, .refreshOn = sample.actions()}); +auto const load = screen.mutation("load", sample, + {.body = {{&LoadSample::id, sampleId}}, .invalidates = {header, series, measure}, ...}); ``` -Every document that calls `Lims_CaptureConcentration`, and every document that lists results, has to know -about the other. A change made by another client is seen only on the next poll. +Every document that calls an action and every document that shows what it changes has to know about the +other. A change made by another client is seen only on that client's next poll. -**With this design**, the action says what it changes, once, where it is registered: +**With this design**, each action's effects are declared once, beside its registration: ```cpp -BRIDGE_REGISTER_ACTION(SampleModel, CaptureConcentration, "Lims_CaptureConcentration", - morph::effects::updates) +BRIDGE_REGISTER_EFFECTS(CaptureConcentration, morph::effects::changes) +BRIDGE_REGISTER_EFFECTS(LoadSample, morph::effects::changesInstance) ``` -The documents drop `invalidates`, `refreshOn` and the polling `refreshEvery`. When a capture succeeds: +The documents drop `refreshOn`, `invalidates` and the polling `refreshEvery`. When a capture succeeds: + +- the calling client marks every cached query holding `ResultView` stale, and the mounted ones refetch; +- the server tells every other client watching `ResultView`, and theirs do the same. -- the calling client patches the new `ResultRow` into every cached query that holds it; -- the server tells every other client watching `ResultRow`, and their mounted queries showing that row refetch. +When `LoadSample` succeeds, only the caller's queries through that same private instance refetch, because no +other client can see that working copy. **Success criteria.** 1. A document that only shows data and sends edits updates correctly with no refresh declarations. 2. A change made by client A appears in client B's mounted screen within one push latency, with no polling. 3. An agent calling an action through the server updates every open GUI the same way. -4. The examples lose their refresh declarations, about 40 today, and their tests stay green. +4. The examples lose their refresh declarations, and their unit and GUI tests stay green. ## 2. What exists today | Fact | Where | |---|---| -| The query cache is per client session, keyed by `(model type, instance, action, canonical body)`. `invalidates`, `refreshOn` and `{"refetch"}` mark entries stale. | spec 5 §5 "The query cache" | +| The query cache is per client session, keyed by `(model type, instance, action, canonical body)`. `invalidates`, `refreshOn` and `{"refetch"}` mark entries stale. | spec 5 §5 "The query cache"; `include/morph/ui/interpreter.hpp` (the cache key) | | `refreshOn` matches by **model type, not instance**, because two private instances of a database-backed model read the same rows. | spec 5 §5 "Queries" | -| `patch` merges a reply into a cached row found by its `id` field. `optimistic` applies the same shape before the reply. | spec 5 §5 "Mutations" | -| A change by another client is not pushed. Spec 5 §15 names a negotiated `ui-event` kind as the way to add it. | spec 5 §5, §15 | +| `patch` merges a reply into a cached row found by its `id` field. `optimistic` applies the same shape before the reply. `latest` applies only the newest reply. | spec 5 §5 "Mutations" | +| Every success the client sees, from a mutation or from a form, collection or wizard submit, is reported through `session.actionSucceeded(typeId, action)`. That call drives `refreshOn`. | `include/morph/ui/interpreter.hpp`, `ScreenCore::aliasSubmits` | +| A change made by another client is not pushed. Spec 5 §15 names a `ui-event` kind negotiated by `ui-hello` as the way to add it. | spec 5 §5, §15 | | A bridge subscription reaches only handlers in the same process, never across connections. | `docs/spec/core/shared_instances.md` | -| `RemoteServer::handle(msg, reply, cid)` answers each request through its own `reply`. The server holds no way to send a connection a message it did not ask for. | `include/morph/core/remote.hpp` | -| Both transports track a `ConnectionId` per connection and call `closeConnection` on disconnect. | `include/morph/net/socket_server.hpp`, `src/qt/qt_websocket_server.cpp` | +| `RemoteServer::handle(msg, reply, cid)` answers each request through its own `reply`. `openConnection()` returns a scope id. The server holds no way to send a connection a message it did not ask for. | `include/morph/core/remote.hpp` | +| `execute` replies are written from the model's strand, and a timed-out `execute` is answered `err "timeout"` while the model may still finish. | `include/morph/core/remote.hpp` | +| The server stamps the verified principal onto **each envelope**. A connection has no principal of its own. | `include/morph/core/remote.hpp`, `docs/spec/core/backend.md` | +| Both transports track a `ConnectionId` per connection and call `closeConnection` on disconnect. Neither caps its outbound queue. | `include/morph/net/socket_server.hpp`, `include/morph/net/detail/ws_connection.hpp`, `src/qt/qt_websocket_server.cpp`, `docs/spec/security.md` | | The client matches replies by `callId`. A message that answers no pending call is dropped. | `include/morph/core/wire.hpp` | +| The core `hello` request carries only `protocolVersion`, and the generic UI client opens with `ui-hello`, not `hello`. | `include/morph/core/wire.hpp`, `include/morph/ui/service_client.hpp` | | `ActionInfo` carries `modelType`, `calculatedFields`, `secretFields`, `credential` and `listFields`. Nothing says whether an action reads or writes. | `include/morph/ui/document.hpp` | -| `BRIDGE_REGISTER_ACTION` already carries one per-action flag (`loggable`). A view's row identity is a structurally detected `rowKey`, defaulting to `"id"`. | `docs/spec/core/registry.md`, `docs/spec/forms/views.md` | +| `BRIDGE_REGISTER_ACTION` picks its form by argument count, and its fourth argument is `Loggable`. | `include/morph/core/registry.hpp` | +| A key may be raw (integral or `std::string`) or a strong id wrapping one (`WrappedModelKey`). | `include/morph/core/model_key.hpp` | +| Offline replay runs as reconnect → activate → bind → replay. | `docs/spec/offline/offline.md` | +| Within a vocabulary version, changes are additive. | spec 5 §12 "Versions" | +| The examples hold 21 `invalidates` and 13 `refreshOn` occurrences outside their tests. | `grep` over `examples/`, at `a55374f5` | ## 3. Concepts -- **Record type.** A reflected aggregate that action results carry and screens show, with a registered name - and a key member. Examples: `ResultRow` keyed by `id`, `AccountInfo` keyed by `id`. The record type, not the - model type, is the unit of change. One record type can be shown through several models, as when - `CustomerModel::ListAccounts` and `AccountModel::GetAccount` both return `AccountInfo`. -- **Effect.** What a successful action did to record types. One of: +- **Record type.** A reflected aggregate that action results carry and screens show, with a registered name and + optionally a key member: `ResultView` keyed by its `ResultId`, `AccountInfo` keyed by `id`. A record type shown + through several models is one record type: `CustomerModel::ListAccounts` and `AccountModel::GetAccount` both + return `AccountInfo`. +- **Derived record type.** A record computed from others: ledger's `GetBudgetReportResult` from transactions. It + is keyless, and it declares what it is derived from (§4), so a change to its sources reaches it without every + writer naming it. +- **Effect.** What a successful action did. One of: - `updates`: the result carries the new state of the `R` records it changed, identified by their keys. - `changes`: records of `R` were created, removed or changed in a way the result does not carry. + - `changesInstance`: the action changed the state of the model instance it ran on, such as a working copy that + only that instance's queries read. - nothing: the action reads. -- **Change.** What the framework derives from an effect after a successful execute: `{record, keys?}`. Keys - are present for `updates`, taken from the result. They are absent for `changes`, which means "any record of - this type". -- **Watch.** A client's statement of which record types its cached queries hold. It is maintained by the - client and authorised by the server. -- **Event.** A server-to-client message carrying changes, sent to the connections watching their record types. +- **Change.** What the framework derives from an effect after a successful execute: a record topic + `{record, keys?}`, or an instance topic `{model type, instance}`. +- **Watch.** A client's statement of which topics its cached queries hold, authorised by the server. +- **Event.** A server-to-client message carrying changes, sent to the connections watching their topics. -The unit is the record type, not the model instance, for the reason `refreshOn` already matches by model type: -two private instances of one database-backed model read the same rows. +The record type is the unit for shared data. It cannot be the model instance, for the reason `refreshOn` already +matches by model type: two private instances of one database-backed model read the same rows. The instance is the +unit only for state that lives in the instance, which is what `changesInstance` names. ## 4. Declaring records and effects ### Records -A record type is declared where its codec already is, in the same structural way as a view's `rowKey`: - ```cpp -// Detected structurally: a member `id`, or an explicit trait. -template <> struct morph::model::RecordTraits { - static constexpr std::string_view name = "Bank_Account"; - static constexpr auto key = &bank::dto::AccountInfo::id; -}; -// or the macro: BRIDGE_REGISTER_RECORD(bank::dto::AccountInfo, "Bank_Account") // key defaults to `id` +BRIDGE_REGISTER_RECORD(bank::dto::AccountInfo, "Bank_Account") // key: member `id` +BRIDGE_REGISTER_RECORD(lims::ResultView, "Lims_Result") // key: `id`, a ResultId +BRIDGE_REGISTER_RECORD(ledger::dto::GetBudgetReportResult, "Ledger_BudgetReport", + morph::record::keyless, morph::record::derivedFrom) ``` -- The key is an integral or `std::string` member, the types `ModelKey` already admits. A record type with no - key can only be named by `changes`. -- The record registry is process-wide and filled during the registration phase, like the action registry, and - it is checked by the same latch. +- **The key** defaults to a member named `id`, as a view's `rowKey` defaults to `"id"`. It may be any `ModelKey`, + raw or wrapped. On the wire it is the raw scalar, as `ModelKey` already encodes it. `morph::record::key<&T::m>` + names another member, and `morph::record::keyless` declares none. +- **`derivedFrom`** says that a change to any `S` is a change to this record type. Fan-out expands it once, + when the change is derived (§8). A derived type is keyless, because a key-level change to a source does not say + which derived rows it moved. +- The record registry is process-wide and filled during the registration phase, like the action registry, and the + same latch checks it. `derivedFrom` cycles are refused at registration. ### Effects -Effects are an optional trailing argument of the registration macro. The macro is already where the framework -meets the application's types. A DTO stays free of morph types, which keeps invariant 1: the model does not -know the framework exists. +Effects are declared by their own macro, beside the action's registration. This leaves `BRIDGE_REGISTER_ACTION`'s +argument-count forms untouched, including `Loggable` as the fourth argument and the `_FOR_CLIENT` forms. It also +keeps every DTO free of morph types, which preserves the rule that a model does not know the framework exists. ```cpp -BRIDGE_REGISTER_ACTION(AccountModel, Deposit, "Deposit", morph::effects::updates) -BRIDGE_REGISTER_ACTION(AccountModel, Transfer, "Transfer", morph::effects::updates) -BRIDGE_REGISTER_ACTION(PayeeModel, AddPayee, "AddPayee", morph::effects::changes) -BRIDGE_REGISTER_ACTION(PayeeModel, RemovePayee, "RemovePayee", morph::effects::changes) -BRIDGE_REGISTER_ACTION(PayeeModel, ListPayees, "ListPayees") // reads: no effects +BRIDGE_REGISTER_ACTION(TransactionModel, Deposit, "Deposit") +BRIDGE_REGISTER_EFFECTS(Deposit, morph::effects::changes, morph::effects::changes) +BRIDGE_REGISTER_EFFECTS(Transfer, morph::effects::changes, morph::effects::changes) +BRIDGE_REGISTER_EFFECTS(AddPayee, morph::effects::changes) +BRIDGE_REGISTER_EFFECTS(RemovePayee, morph::effects::changes) +// An action that edits one row and returns it, such as a hypothetical EditPayee returning PayeeInfo: +// BRIDGE_REGISTER_EFFECTS(EditPayee, morph::effects::updates) +BRIDGE_REGISTER_EFFECTS(LoadSample, morph::effects::changesInstance) +// ListPayees, GetAccount: no effects, they read. ``` -- **`updates` finds the records in the result.** The result may be an `R`, a `std::vector`, or an - aggregate whose members, found by reflection, hold an `R` or a `std::vector`. `Transfer` returns both - accounts this way. A result in which none is found is a registration error, reported when the program starts, - not when the action first runs. -- **`changes` needs nothing from the result.** It covers creation and removal, whose effect on a list the - framework cannot compute: whether a new payee belongs in a filtered list is the server's query to answer. -- **Several effects compose:** `morph::effects::updates, morph::effects::changes`. -- **No declaration means the action reads, and nothing is pushed.** This default under-notifies rather than - over-notifies. An action wrongly treated as a write would make every fetch invalidate the queries that - showed it, and they would refetch in a loop. An action that writes and declares nothing behaves as it does - today. §10's load-time check finds the documents that still rely on refresh declarations for it. -- **Only success has effects.** A failed action, or one that was cancelled, changes nothing. - -`ActionTraits` gains `effects()`, returning the list as data (`{record name, kind}`). The framework reads -effects only through that function. +- **`BRIDGE_REGISTER_EFFECTS(A, effects...)`** specialises `morph::model::ActionEffects`. `ActionTraits` + gains `effects()`, which returns the list as data (`{kind, record?}`), empty when `A` has no specialisation. + The framework reads effects only through `effects()`. +- **`updates` finds the records in the result.** The result may be an `R`, a `std::vector`, or an aggregate + whose members, found by reflection, hold an `R` or a `std::vector`. If none is found, registration fails + when the program starts, not when the action first runs. `R` must have a key. Bank's `Transfer` returns two + balances (`TransferResult{fromBalanceMinor, toBalanceMinor}`), not two `AccountInfo`s, so it declares + `changes`. +- **`updates` only changes rows that exist.** A key that no cached entry holds is not added to any list, because + whether it belongs there is the server's query to answer. An action that may create a record declares + `changes`. +- **`changes` needs nothing from the result.** It covers creation, removal, and changes the result does not + carry. +- **`changesInstance`** means the instance's own state changed. It reaches the caller's entries through the same + instance (§9), and, for a shared instance, other clients attached to that instance (§7). +- **Several effects compose,** as above. +- **No declaration means the action reads, and nothing is pushed.** A read is safe by construction, because a + query may only run an action with no effects (§10). An action that writes and declares nothing behaves as it + does today, and §10's checks point at it. +- **Only success has effects.** A failed action or a cancelled one changes nothing. + +### What each action's result holds + +Registration also records, per action, the record types its **result** holds, found by the same reflection as +`updates`: `records(A)`. A cached query holds exactly the records of its action. `ListResults` returns +`ListResultsResult{results: vector}`, so its entries hold `Lims_Result`. ## 5. The catalog -`ActionInfo` gains two facts, filled from the registry in local mode and served by `ui-catalog` remotely: +`ActionInfo` gains two facts. Local mode fills them from the registry; remotely, `ui-catalog` serves them, like the +existing ones. | Field | Meaning | |---|---| -| `effects` | The action's effects: `[{record, kind: "updates" \| "changes"}]`. | -| `records` | The record types the action's **result** carries, found by the same reflection as `updates`. A cached query holds exactly these. | +| `effects` | `[{kind: "updates" \| "changes" \| "changesInstance", record?}]`. | +| `records` | The record types the action's result holds (§4), each with whether it is keyed. | -`records` lets a client decide, without the server, whether a cache entry holds record `R`: the entry's action's -`records` contains `R`. That is the whole link between "R changed" and "this query must update". +`records` lets the client decide, without asking the server, whether a cache entry holds record `R`: the entry's +action's `records` contain `R`. `derivedFrom` is never sent. The server expands it before a change leaves (§8), so a +client only ever matches the record types its entries hold. + +An `evaluate` cache entry (`ui-evaluate`, spec 5 §7) holds no records and is never watched: an evaluation depends on +its body, not on stored data. ## 6. The server-to-client channel -The wire gains its first server-initiated message. It is a core facility, not a UI one: the core `hello` -negotiates it, and the UI layer is its first user. - -- **Negotiation.** The core `hello` gains a capability list, and the client offers `events`. A server sends - events only to connections that offered it. A client that never offers it sees today's protocol unchanged. -- **The envelope.** `kind: "event"`, `callId: 0`, `body: {topic, payload}`. The topic is a string, `changes` - for this design. A client routes it by kind before reply matching, so an event is never mistaken for an - orphaned reply. -- **The server side.** `RemoteServer` gains `openConnection(cid, send)`, with `closeConnection(cid)` as its pair. - A transport registers the connection's outbound writer when the connection opens. - `RemoteServer::post(cid, envelope)` writes through it on the transport's thread, under the same size cap - (`kMaxEnvelopeBytes`) and the same write-failure handling as replies. -- **The client side.** `IBackend` gains `setEventHandler(fn)`, delivered on the bridge's executor. The remote - backend calls it for each `event`. The local backend calls it in-process, with no wire: local mode gets the - same behaviour from the same code above the backend. -- **Order.** Events on one connection arrive in the order the server posted them. No order is promised between - an event and an unrelated reply. +The wire gains its first server-initiated message. + +- **Negotiation in `ui-hello`.** `HelloRequest` gains `events: true` and `HelloReply` gains `events: bool`. Both are + additive optional keys. The generic client already opens with `ui-hello`, on the same connection as its executes, + and spec 5 §15 names it as the place to negotiate this. A server sends events only on a connection whose + `ui-hello` offered them and that it accepted. A client that never offers sees today's protocol unchanged. + Negotiation is per connection, so a reconnect negotiates again. +- **The envelope.** `kind: "event"`, `callId: 0`, `body: {topic, payload}`. For this design the topic is + `changes`. A client routes it by kind before reply matching, so an event is never taken for an orphaned reply. +- **Writing to a connection.** `RemoteServer` gains `setConnectionWriter(cid, write)`, the transport's writer for a + connection already opened with `openConnection()`, and `post(cid, envelope)`. `post` is callable from any thread. + It hands the encoded envelope to the writer, which queues it on the transport's own thread, as replies are + queued today. The size cap (`kMaxEnvelopeBytes`) applies. A write failure drops the connection's watches, + as a disconnect does. +- **The outbound cap.** Events are queued per connection, up to 256 pending events or 1 MiB. On overflow the queue + is replaced by one `resync` event, meaning "everything you watch may have changed", and the client marks every + watched entry stale. The connection is not closed, and replies are not affected. This is the only outbound limit + the transports gain. +- **The client side.** `IBackend` gains `setEventHandler(fn)`, delivered on the bridge's executor: + - The remote backends call it for each `event`: the Qt WebSocket backend, the `net` socket backend, and + `SimulatedRemoteBackend`, which the core transport tests use. + - The local backend calls it in-process, with no wire, so local mode gets the same behaviour from the same code + above the backend. +- **Order.** Events on one connection arrive in the order the server posted them. No order is promised between an + event and any reply. ## 7. Watching -A client watches the record types its cache holds, so the server sends each connection only what it can show. - -- **Lifetime.** A record type is watched while at least one cache entry holds it, mounted or kept (spec 5 §5: - for 5 minutes). The client counts references per record type. It sends `watch` when a count leaves zero and - `unwatch` when it returns to zero. Both are core kinds with `{records: [name, ...]}` and an `ok` reply. -- **Batching.** Watch changes made in one reactive batch go out as one message, so mounting a screen with ten - queries sends one `watch`. -- **Reconnect.** After a reconnect or a backend switch, the client sends its whole watch set again and marks - every entry stale, as spec 6 §9 already does for a backend switch. Events missed while disconnected are not - replayed: the stale marks cover them. -- **Granularity.** Watching is by record type, not by key. Per-key watching would need the server to hold - every key every client shows, and keys change as lists scroll. The event carries keys, and the client filters - them (§9). +A client watches the topics its cache holds, so the server sends each connection only what it can show. + +- **Topics.** + - A record type, for every record its entries hold. + - `(model type, instance key)` for every entry read through a **shared** instance whose action holds no + records, which is the case for working-copy reads. + - A private instance is never a topic, because no other connection can reach it. +- **Lifetime.** A topic is watched while at least one cache entry holds it, mounted or kept (spec 5 §5: for + 5 minutes). The client counts references per topic. It sends `watch` when a count leaves zero and `unwatch` when it + returns to zero. Both are UI service kinds (`ui-watch`, `ui-unwatch`) with `{records: [...], instances: [...]}`, + and an `ok` reply that lists any topic refused (§12). +- **Watch before fetch.** Watch changes made in one reactive batch are sent as one message, **before** the + fetches of that batch, on the same connection. The server handles them in arrival order on its strand. A write + that commits after the fetch has read therefore finds the watch in place, and one that commits before it is in + the fetched value. Neither is missed. +- **Reconnect.** After a reconnect or a backend switch, the client negotiates again and sends its whole watch set + before anything else, then marks every entry stale, as spec 6 §9 already does for a backend switch. Events missed + while disconnected are not replayed: the stale marks cover them. Offline replay comes after the watch set, so its + changes arrive as events too (§9). +- **Granularity.** Record topics are per type, not per key. Per-key watching would make the server track every key + every client shows, and those keys change as lists scroll. The event carries keys, and the client filters on them + (§9). ## 8. Fan-out on the server -After an `execute` succeeds, and before its reply is written: - -1. The server reads the action's effects and derives the changes: record name and, for `updates`, the keys found - in the result. -2. It finds the connections watching those record types, **excluding the connection that made the call**, whose - client applies the effects from its own reply (§9). -3. It filters each connection's view by authorisation (§12) and posts one `changes` event per connection. - -- **Coalescing.** Changes for one connection are merged for up to 50 ms, or until 256 keys are queued, then sent - as one event. Keys are merged per record type; a `changes` for a type absorbs its `updates` keys. One hundred - writes in a burst are then one refetch per showing query, not one hundred. -- **Where the watch table lives.** On the server strand, next to the connection scopes `closeConnection` already - reclaims. A closed connection drops its watches. -- **Every path that executes reaches fan-out:** the UI service, the plain `execute` kind, an offline replay, and - an agent (§11), because fan-out sits on the dispatch path they share. +Fan-out sits in `RemoteServer`'s execute path (`dispatchExecute`), which every executing message shares: the +`execute` kind, offline replays (which arrive as executes), and the agent adapter (§11). The UI service has no +execute path of its own (`ui-evaluate` never executes), so it needs nothing. + +After an `execute` succeeds on the model's strand: + +1. The model strand derives the changes: the effects' record topics, with keys read from the result for `updates`, + expanded through `derivedFrom`; and, for `changesInstance` on a shared instance, its instance topic. It posts the + changes to the server strand. The reply is written as it is today, unaffected. +2. On the server strand, the changes are matched against the watch table. **The calling connection is included.** + It also applies the effects from its reply (§9), and a second refetch request for the same entry is deduplicated + by the cache. Including it covers every case in which the caller never sees its own reply: an execute that timed + out on the client but succeeded on the server, a reply that `latest` discarded, and a replay whose reply went to + the offline worker. +3. Each matching watch is re-authorised with the principal stored on it (§12), and the connection's coalescer + queues the change. + +- **Coalescing.** Changes for one connection are merged for up to 50 ms, or until 256 keys are queued, then sent as + one event. Keys are merged per record type, a `changes` absorbs that type's `updates` keys, and instance topics are + deduplicated. A burst of one hundred writes is then one refetch per showing query, not one hundred. +- **The watch table** lives on the server strand, next to the connection scopes `closeConnection` already reclaims. + A closed connection drops its watches. ## 9. The client cache -The interpreter's query cache applies a change the same way whether it came from its own reply or from an event. +The interpreter's query cache applies a change the same way whether it came from a reply or from an event. -| Change | Own reply (the caller) | Event (another connection) | +| Change | From a reply (the caller's own) | From an event | |---|---|---| | `updates R` with keys | Patch: each entry holding `R` merges the new records into the rows with those keys, using the existing `patch` merge and the catalog's `listFields` for a list inside a record. No refetch. | Each entry holding `R` whose value contains one of the keys is marked stale, and refetched if mounted. | | `changes R` | Each entry holding `R` is marked stale, and refetched if mounted. | The same. | - +| `changesInstance` | Each entry read through the same instance is marked stale, and refetched if mounted. | The same, for the shared instance named. | +| `resync` | none | Every watched entry is marked stale, and refetched if mounted. | + +- **Every reply applies its effects.** The single hook is `session.actionSucceeded`, which every success already + goes through: mutations, and form, collection and wizard submits (`aliasSubmits`). It gains the reply's JSON, so + `updates` can patch. A `latest` run whose reply is discarded still applies its effects; only its value is + dropped. An offline replay's reply reaches the offline worker, not a mutation. Its effects reach the cache as an + event, because the caller is not excluded (§8). - **Why other clients refetch rather than receive values.** The new values were computed for the caller. Another - user may see different fields (`secretFields`) or may not be allowed the record at all. A refetch goes through - the receiver's own authorised read. Pushing values to other receivers is §15's first deferred item. + user may see different fields (`secretFields`) or may not be allowed the record at all. A refetch goes through the + receiver's own authorised read. Pushing values to other receivers is the first item in §15. +- **Duplicates are harmless.** The caller sees each change twice, once from its reply and once as an event. Marking + an entry that is already stale, or already refetching, is a no-op, so a duplicate costs nothing. - **Batching.** One event, or one reply's effects, is applied in one reactive batch, so a screen updates in one frame. -- **`optimistic` is unchanged.** It still shows a guess before the reply. The reply's patch then confirms or - corrects it, exactly as a declared `patch` does today. +- **`optimistic` is unchanged.** It still shows a guess before the reply, and the reply's patch confirms or corrects + it, exactly as a declared `patch` does today. - **Entries no query mounts** are only marked stale. They refetch if a screen mounts on them. ## 10. Documents -With effects declared, a document needs no refresh declarations. The vocabulary changes in two steps. +With effects declared, a document needs no refresh declarations. + +**At load, in `ui/1`, additively.** The loader already has the catalog, and it reports: -**Step 1, additive within `ui/1`.** `invalidates`, `patch` and `refreshOn` stay valid and keep working, and -effects apply as well. The loader, which already has the catalog, reports: +- **refused: a query whose action declares effects.** A query must read. This is what makes the default safe: two + clients cannot refetch each other in a ring, because a fetch never produces a change. It is new in what it + refuses, so it applies only to actions that declare effects, which no existing action does. +- **warning, redundant:** an `invalidates`, `patch` or `refreshOn` that the mutation's action's effects already + cover, so a document can migrate one screen at a time. +- **warning, uncovered:** a mutation whose action declares no effects, with no `invalidates` or `patch` of its own + and no query naming it in `refreshOn`. After it succeeds, nothing on any screen updates. It may genuinely change + nothing that is shown, so this is only a warning, but it points at actions still to annotate. -- **redundant:** an `invalidates` or `refreshOn` that the mutation's action's effects already cover. This is a - warning, so a document can migrate one screen at a time. -- **uncovered:** a mutation whose action declares no effects, with no `invalidates` or `patch` of its own and - no query naming it in `refreshOn`. After it succeeds, nothing on any screen updates. This is a warning, - because the action may genuinely change nothing that is shown. It points at the actions still to annotate. +**In `ui/2`, the removal.** `invalidates`, `patch` and `refreshOn` are removed from mutations and queries. Removing +keys is not additive, so it is a new vocabulary version under spec 5 §12. A server that keeps `ui/1` documents for +older clients keeps them working. -**Step 2, removal.** Once the examples have migrated, `invalidates`, `patch` and `refreshOn` are refused at load. -A document that needs to refetch for a reason no action declares uses the existing `{"refetch": "q"}` command. -`refreshEvery` stays, for data that changes with time rather than through actions, such as a clock or an -external feed. Whether step 2 waits for a vocabulary version (`ui/2`) or happens inside `ui/1` is decided when -the examples have migrated (§14, part 8). +- `optimistic` stays, and keeps the `{query, key, with}` shape it has today, now as its own definition rather than + `patch`'s. +- A document that must refetch for a reason no action declares uses the existing `{"refetch": "q"}` command. +- `refreshEvery` stays, for data that changes with time rather than through actions, such as a clock or an external + feed. + +**The C++ authoring layer** (`include/morph/ui/author.hpp`) gains nothing for effects, because they are declared +beside the action, not in a document. It drops `.invalidates`, `.patch` and `.refreshOn` when it emits `ui/2`, and +its golden files (`tests/ui/golden/`) change with it. ## 11. Agents through the server An agent acts on the application through the same dispatch as everything else, so this design is what makes its changes visible. -- **The MCP adapter** is a server-side component. It lists the actions a principal may run (the catalog's - `permitted`, with their JSON shapes from the registry) as MCP tools. It runs each tool call through - `ActionDispatcher` as that principal, with the same `authorize` and `authorizeInstance` checks, the same journal - entry and the same `ui-evaluate` dry run as a GUI client. The agent may commit directly. -- **Fan-out treats the adapter as one more connection**, so every open GUI showing the changed records updates +- **The MCP adapter** is a server-side component. It opens a connection scope with `openConnection()` for each + agent session, and sends each tool call through `RemoteServer::handle(msg, reply, cid)` as an ordinary `execute`. + That keeps every check the GUI gets: authentication and the principal, `authorize` and `authorizeInstance`, + limits, the journal, cancellation and the in-flight count. It lists the actions the principal may run (the + catalog's `permitted`, with their JSON shapes) as MCP tools, and it offers `ui-evaluate` as a dry run. The agent + may commit directly. +- **Fan-out treats the agent's connection like any other,** so every open GUI showing the changed records updates through §8 and §9. - **Model access and keys are not part of this design.** The adapter does not call a model provider. An external agent (Claude Desktop, Claude Code) brings its own. Where keys would live for an assistant built into the client @@ -257,68 +338,98 @@ changes visible. ## 12. Security -- **A watch reveals nothing by itself.** The server accepts a watch for record type `R` only when the principal - may run at least one action whose `records` contain `R`. Otherwise it refuses with `err`, and the client logs it. -- **An event reveals that records of `R` changed, and their keys.** Keys are ids the receiver could already see - through some action returning `R`, but not necessarily the same records. Where that matters, because a model - filters rows per principal, a record type registered with `keys: private` is pushed without keys to principals - other than the caller's. Every query holding `R` then refetches, which costs more and reveals nothing. -- **Values never travel to other connections** (§9), so `secretFields` and row-level filtering stay on the read - path, where they are enforced today. -- **Rate.** Coalescing (§8) bounds events per connection to 20 a second. A connection that cannot drain its - outbound queue past the transport's existing limit is closed, as for a slow reader of replies. +- **Each watch carries a principal.** A connection has no principal of its own: the server verifies one on every + envelope. The watch table therefore stores, with each watch, the verified principal of the `ui-watch` envelope + that created it. + - **A later envelope on the connection with a different verified principal drops all of the connection's + watches.** That covers signing in as someone else, and a client that keeps old watches across a sign-out. The + client's cache is already cleared on a principal change (spec 5 §5), and it watches again. + - Fan-out re-authorises each watch at send time, with its stored principal and that token's expiry. An expired + token drops the watch. The client watches again with its next envelope, which carries a fresh token. +- **What a watch may name.** + - A record type: only when the principal may run at least one action whose `records` contain it. Otherwise the + `ok` reply lists it as refused, and the client logs it. + - An instance topic: only when `authorizeInstance` admits the principal to that instance. +- **An event reveals that records of `R` changed, and their keys.** Keys are ids of records the receiver could + list through some action returning `R`, but not necessarily the same ones. For a model that filters rows per + principal, the record type is registered with `morph::record::privateKeys`. Its changes then reach principals other + than the caller's without keys, and every entry holding `R` refetches. That costs more and reveals nothing. +- **Values never travel to other connections** (§9), so `secretFields` and row-level filtering stay on the read path, + where they are enforced today. +- **Volume.** Coalescing bounds events to 20 a second per connection, and the outbound cap (§6) bounds what a slow + reader can queue. The query-must-read rule (§10) removes the only way the design could feed itself. ## 13. Tests -Every test below must fail if the feature does nothing, so each names the mutation that should break it. - -- **Records and effects (core).** Registration extracts keys from an `R`, a `vector` and a nested member. A - result with no `R` under `updates` fails registration. *Mutation: drop the reflection walk; key extraction - tests fail.* -- **Channel (core, net, qt).** The same conformance case is run on both transports and on the local backend: a - posted event reaches the client's handler. An event to a client that did not offer `events` is never sent. - An event is not taken for a reply. *Mutation: route events through reply matching; the routing case fails.* -- **Fan-out (core).** Two connections, both watching `R`. An `updates` execute from A posts to B and not to A. - An unwatched type posts nothing, and a burst of 100 coalesces to one event. *Mutation: remove the origin - exclusion; A receives its own event.* -- **Cache (ui).** A recording-backend screen with a list query and no refresh declarations. An own reply patches - the row without a fetch, and the fetch counter stays flat. An event marks it stale and refetches it. - *Mutation: skip the cache step; the shown row keeps its old value.* +Every test below must fail if the feature does nothing, so each names the change that should break it. + +- **Records and effects (core).** + - Registration extracts raw and wrapped keys from an `R`, a `vector` and a nested member. + - `updates` over a result with no `R`, and a `derivedFrom` cycle, both fail registration. + - `records(A)` lists the result's record types. + - *Mutation: drop the reflection walk; key extraction fails.* +- **Channel (core, net, qt).** + - One conformance case runs on both transports, on `SimulatedRemoteBackend` and on the local backend: a posted + event reaches the handler. + - A connection that did not offer `events` receives none. + - An event is not taken for a reply. + - Overflow produces exactly one `resync`. + - *Mutation: route events through reply matching; the routing case fails.* +- **Watching and fan-out (core).** + - Two connections watch `R`, and an execute from A posts to both. + - An unwatched type posts nothing, and a burst of 100 is one event. + - A watch made under principal P is dropped by an envelope under Q. + - A watch on a type the principal cannot read is refused. + - An execute that times out at the client but succeeds still reaches the caller as an event. + - *Mutation: skip re-authorisation; the principal case fails.* +- **Client cache (ui).** A recording-backend screen with a list query and no refresh declarations: + - An own `updates` reply patches the row without a fetch, and the fetch counter stays flat. + - A `changes` event refetches it. + - A submit (not a mutation) applies its effects. + - A discarded `latest` reply still applies its effects. + - `changesInstance` refetches only entries of that instance. + - *Mutation: skip the cache step in `actionSucceeded`; the shown row keeps its old value.* +- **Documents (ui).** A query whose action declares effects is refused at load. A redundant `invalidates` warns. An + uncovered mutation warns. - **End to end (examples).** One rung runs two clients against one server. Client A's mutation updates client B's - mounted screen with no `refreshOn` and no `refreshEvery` in either document. *Control: unregister the action's - effect; B does not update.* + mounted screen, with no `refreshOn` and no `refreshEvery` in either document. *Control: remove the action's + `BRIDGE_REGISTER_EFFECTS`; B does not update.* ## 14. Delivery Each part lands as its own pull request to `master`. Parts with no dependency between them can run side by side. - -| # | Part | Layer | Needs | -|---|---|---|---| -| 1 | Records and effects: `RecordTraits`, `BRIDGE_REGISTER_RECORD`, `morph::effects`, `ActionTraits::effects()`, change derivation from a result | core | none | -| 2 | Server-to-client channel: core `hello` capabilities, `event` kind, `openConnection` and `post`, both transports, `setEventHandler`, in-process delivery in the local backend | core, net, qt | none | -| 3 | Catalog: `ActionInfo::effects` and `records`, local lookup and `ui-catalog` | ui | 1 | -| 4 | Watching and fan-out: `watch` and `unwatch` kinds, the watch table, origin exclusion, coalescing, authorisation (§12) | core | 1, 2 | -| 5 | Client cache: apply effects from own replies and from events; watch reference counting; reconnect | ui | 2, 3 (4 for its remote tests) | -| 6 | Document checks: redundant and uncovered warnings; spec 5 §5 updated | ui | 3, 5 | -| 7 | Examples: declare records and effects in the seven rungs that use refresh declarations, remove those declarations, and add the two-client end-to-end test | examples | 5, 6 | -| 8 | Removal: refuse `invalidates`, `patch` and `refreshOn` at load, in `ui/1` or as `ui/2` (decided then) | ui | 7 | -| 9 | MCP adapter: catalog actions as tools, run as a principal, through fan-out | net, ui | 4 | - -**Ready now:** parts 1 and 2. They are independent, and both are core. **Natural pairs:** 1 ∥ 2, then 3 ∥ 4. -Part 9 can run beside anything after part 4. Part 5 runs beside part 4: the local backend delivers events -in-process, so its local tests need no server fan-out, and only its remote tests wait for part 4. +Each part updates the `docs/spec/` pages named in its row, and keeps every new public symbol's Doxygen complete. + +| # | Part | Layer | Needs | Spec pages | +|---|---|---|---|---| +| 1 | Records and effects: `BRIDGE_REGISTER_RECORD`, keys raw and wrapped, `keyless`, `derivedFrom`, `privateKeys`; `BRIDGE_REGISTER_EFFECTS`, `ActionEffects`, `ActionTraits::effects()`; `records(A)`; deriving changes from a result | core | none | `core/registry.md` | +| 2 | Server-to-client channel: the `event` kind, `setConnectionWriter` and `post`, the outbound cap and `resync`, both transports, `SimulatedRemoteBackend`, `setEventHandler`, in-process delivery in the local backend | core, net, qt | none | `core/wire.md`, `core/backend.md`, `security.md` | +| 3 | Catalog and negotiation: `ActionInfo::effects` and `records`, local lookup and `ui-catalog`; `events` in `ui-hello` | ui | 1 | `ui/service.md`, `ui/document.md` | +| 4 | Watching and fan-out: `ui-watch` and `ui-unwatch`, the watch table with principals, fan-out in `dispatchExecute`, `derivedFrom` expansion, coalescing, re-authorisation, `privateKeys` | core, ui | 1, 2, 3 | `core/backend.md`, `ui/service.md`, `security.md` | +| 5 | Client cache: `actionSucceeded` carries the reply and applies effects; events and `resync`; topic reference counting; watch before fetch; reconnect order | ui | 2, 3 (4 for its remote tests) | `ui/interpreter.md`, spec 5 §5 | +| 6 | Document checks: query-must-read refused; redundant and uncovered warnings | ui | 3, 5 | `ui/document.md` | +| 7 | Examples: records and effects in the seven rungs that use refresh declarations, their declarations removed in `ui/1` documents where the warnings say so, and the two-client end-to-end test | examples | 5, 6 | the rungs' READMEs | +| 8 | `ui/2`: remove `invalidates`, `patch` and `refreshOn`; `optimistic` keeps its own shape; the authoring layer and golden files emit `ui/2`; the server keeps serving `ui/1` | ui | 7 | `ui/document.md`, `ui/author.md`, spec 5 §5 and §12 | +| 9 | MCP adapter: a connection scope per agent session, tools from `permitted`, calls through `RemoteServer::handle` | net, ui | 4 | a new `ui/agent.md` | + +**Ready now:** parts 1 and 2. They are independent: part 1 is `registry.hpp` and a new record header, part 2 is +`wire.hpp`, `remote.hpp`, `backend.hpp` and the transports. **Then:** part 3, then parts 4 and 5 side by side. Part 5 +uses the local backend's in-process delivery for its tests until part 4 lands. Part 9 can run beside anything after +part 4. ## 15. Out of scope - **Pushing values to other connections.** Receivers refetch through their own authorised read. Sending the new record, filtered per receiver, would save that round trip. It needs per-receiver field filtering on the server, - and it is the first follow-up once refetch traffic is measured. + and it is the first follow-up once refetch traffic has been measured. - **Per-key and per-field watching.** Watching is by record type, and the event carries keys. Field-level effects - (`updates` naming fields) would let a client skip refetches for fields a screen does not show. That is - deferred until a measurement shows refetches cost something. -- **Effects computed at run time** beyond the records found in the result, such as an explicit change list a model - returns. `updates` over a reflected result covers the known cases, `Transfer` included. -- **An assistant built into the client, and where its model keys live.** §11 covers agents acting through the - server only. -- **Server push for anything other than changes:** progress of a long action, notifications. The channel (§6) has - topics so they can be added without another negotiation, but none is defined here. + would let a client skip refetches for fields a screen does not show. That is deferred until a measurement shows + refetches cost something. +- **Effects computed at run time,** such as a change list a model returns. Declared effects with `derivedFrom` cover + the examples' cases. A run-time list would have to keep the rule that a model does not know the framework exists. +- **Push for non-UI clients.** Negotiation is in `ui-hello`. A plain `Client` that never sends it receives no events. + Offering events in the core `hello` would need that request to carry capabilities, which it does not today. +- **An assistant built into the client, and where its model keys live.** §11 covers agents acting through the server + only. +- **Push for anything other than changes:** progress of a long action, notifications. The `event` envelope has a + topic, so they can be added without another negotiation, but none is defined here. From 6ec60b00a558496a0c8ee60fdbd3bebe56bfee9c Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Fri, 16 Oct 2026 19:23:45 +0200 Subject: [PATCH 03/68] =?UTF-8?q?docs:=20spec=208=20=E2=80=94=20address=20?= =?UTF-8?q?the=20second=20review=20round?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 --- .../specs/2026-10-16-server-push-design.md | 358 +++++++++++------- 1 file changed, 220 insertions(+), 138 deletions(-) diff --git a/docs/superpowers/specs/2026-10-16-server-push-design.md b/docs/superpowers/specs/2026-10-16-server-push-design.md index eea547f8b..88e412178 100644 --- a/docs/superpowers/specs/2026-10-16-server-push-design.md +++ b/docs/superpowers/specs/2026-10-16-server-push-design.md @@ -61,23 +61,27 @@ other client can see that working copy. | Fact | Where | |---|---| | The query cache is per client session, keyed by `(model type, instance, action, canonical body)`. `invalidates`, `refreshOn` and `{"refetch"}` mark entries stale. | spec 5 §5 "The query cache"; `include/morph/ui/interpreter.hpp` (the cache key) | +| A successful fetch clears an entry's `stale` mark unconditionally, and a refetch is skipped while a fetch is pending. | `include/morph/ui/interpreter.hpp` (`settleSuccess`, `refetchCurrent`) | | `refreshOn` matches by **model type, not instance**, because two private instances of a database-backed model read the same rows. | spec 5 §5 "Queries" | -| `patch` merges a reply into a cached row found by its `id` field. `optimistic` applies the same shape before the reply. `latest` applies only the newest reply. | spec 5 §5 "Mutations" | -| Every success the client sees, from a mutation or from a form, collection or wizard submit, is reported through `session.actionSucceeded(typeId, action)`. That call drives `refreshOn`. | `include/morph/ui/interpreter.hpp`, `ScreenCore::aliasSubmits` | +| An alias is `"private"`, keyless `"shared"` (a stateless model), or keyed by an instance expression. | spec 5 §5 "Models" | +| `patch` merges a reply into a cached row found by its `id` field (`rowId`). `optimistic` applies the same shape before the reply. `latest` applies only the newest reply. | spec 5 §5 "Mutations"; `include/morph/ui/interpreter.hpp` | +| Every success the client sees, from a mutation or from a form, collection or wizard submit, is reported through `session.actionSucceeded(typeId, action)`, and that call drives `refreshOn`. | `include/morph/ui/interpreter.hpp`, `ScreenCore::aliasSubmits` | +| A binding holds the server's id for the instance it is bound to (`currentId`). | `docs/spec/core/bridge.md` | | A change made by another client is not pushed. Spec 5 §15 names a `ui-event` kind negotiated by `ui-hello` as the way to add it. | spec 5 §5, §15 | | A bridge subscription reaches only handlers in the same process, never across connections. | `docs/spec/core/shared_instances.md` | -| `RemoteServer::handle(msg, reply, cid)` answers each request through its own `reply`. `openConnection()` returns a scope id. The server holds no way to send a connection a message it did not ask for. | `include/morph/core/remote.hpp` | +| `RemoteServer::handle(msg, reply, cid)` answers each request through its own `reply`, and `openConnection()` returns a scope id that owns the connection's registered models. The server holds no way to send a connection a message it did not ask for. | `include/morph/core/remote.hpp` | +| An `IEnvelopeService` (the UI service) handles the kinds core does not know, through `handle(request, authorizer)`, which receives no connection id. | `include/morph/core/remote.hpp` | | `execute` replies are written from the model's strand, and a timed-out `execute` is answered `err "timeout"` while the model may still finish. | `include/morph/core/remote.hpp` | -| The server stamps the verified principal onto **each envelope**. A connection has no principal of its own. | `include/morph/core/remote.hpp`, `docs/spec/core/backend.md` | -| Both transports track a `ConnectionId` per connection and call `closeConnection` on disconnect. Neither caps its outbound queue. | `include/morph/net/socket_server.hpp`, `include/morph/net/detail/ws_connection.hpp`, `src/qt/qt_websocket_server.cpp`, `docs/spec/security.md` | +| The server stamps the verified principal onto **each envelope**. A connection has no principal of its own, and `IAuthorizer::authenticate` returns only the principal, with no expiry. | `include/morph/core/remote.hpp`, `include/morph/session/session.hpp` | +| Both transports track a `ConnectionId` per connection and call `closeConnection` on disconnect. Neither caps its outbound queue, and the `net` writer closes a peer that stalls past `sendTimeout`. | `include/morph/net/socket_server.hpp`, `include/morph/net/detail/ws_connection.hpp`, `src/qt/qt_websocket_server.cpp`, `docs/spec/security.md` | | The client matches replies by `callId`. A message that answers no pending call is dropped. | `include/morph/core/wire.hpp` | -| The core `hello` request carries only `protocolVersion`, and the generic UI client opens with `ui-hello`, not `hello`. | `include/morph/core/wire.hpp`, `include/morph/ui/service_client.hpp` | +| The core `hello` request carries only `protocolVersion`. Its reply carries capabilities. The generic UI client opens with `ui-hello`, not `hello`. | `include/morph/core/wire.hpp`, `include/morph/core/remote.hpp`, `include/morph/ui/service_client.hpp` | | `ActionInfo` carries `modelType`, `calculatedFields`, `secretFields`, `credential` and `listFields`. Nothing says whether an action reads or writes. | `include/morph/ui/document.hpp` | -| `BRIDGE_REGISTER_ACTION` picks its form by argument count, and its fourth argument is `Loggable`. | `include/morph/core/registry.hpp` | +| `BRIDGE_REGISTER_ACTION` is a full specialisation of `ActionTraits`, chosen by argument count, with `Loggable` as its fourth argument. Per-action facts added later use their own trait or registry: `BRIDGE_REGISTER_VALIDATOR` specialises `ActionValidator`. | `include/morph/core/registry.hpp` | | A key may be raw (integral or `std::string`) or a strong id wrapping one (`WrappedModelKey`). | `include/morph/core/model_key.hpp` | -| Offline replay runs as reconnect → activate → bind → replay. | `docs/spec/offline/offline.md` | +| Offline replay runs as reconnect → activate → `bindContext` → `replay`. | `docs/spec/offline/offline.md`, `include/morph/offline/reconnect_coordinator.hpp` | | Within a vocabulary version, changes are additive. | spec 5 §12 "Versions" | -| The examples hold 21 `invalidates` and 13 `refreshOn` occurrences outside their tests. | `grep` over `examples/`, at `a55374f5` | +| The examples hold 21 `invalidates` and 13 `refreshOn` occurrences outside their tests, in seven rungs. | `grep` over `examples/`, at `a55374f5` | ## 3. Concepts @@ -93,11 +97,13 @@ other client can see that working copy. - `changes`: records of `R` were created, removed or changed in a way the result does not carry. - `changesInstance`: the action changed the state of the model instance it ran on, such as a working copy that only that instance's queries read. - - nothing: the action reads. + - `none`: the action changes nothing that is shown. This is stated explicitly, for an action that attaches or + opens and would otherwise draw a warning (§10). + - nothing declared: the action reads. - **Change.** What the framework derives from an effect after a successful execute: a record topic - `{record, keys?}`, or an instance topic `{model type, instance}`. + `{record, keys?}`, or an instance topic `{model type, instance id}`. - **Watch.** A client's statement of which topics its cached queries hold, authorised by the server. -- **Event.** A server-to-client message carrying changes, sent to the connections watching their topics. +- **Event.** A server-to-client message carrying changes, sent to the connections whose topics they match. The record type is the unit for shared data. It cannot be the model instance, for the reason `refreshOn` already matches by model type: two private instances of one database-backed model read the same rows. The instance is the @@ -105,6 +111,17 @@ unit only for state that lives in the instance, which is what `changesInstance` ## 4. Declaring records and effects +Both are runtime registries, filled by static registrars during the registration phase and checked by its latch. +Neither specialises a trait that `BRIDGE_REGISTER_ACTION` has already used. +- The action's `ActionTraits` is unchanged. Its inline members are already defined when the action is + registered, so a later specialisation that they read would be ill-formed. +- In a header split between `BRIDGE_DECLARE_ACTION` and `BRIDGE_REGISTER_ACTION_SOURCE`, one translation unit + would see a different trait than another. + +Records and effects are matched by **runtime type identity** (`std::type_index`) when the registration phase +closes. The order of `BRIDGE_REGISTER_RECORD`, `BRIDGE_REGISTER_ACTION` and `BRIDGE_REGISTER_EFFECTS` across +translation units therefore does not matter. + ### Records ```cpp @@ -114,21 +131,18 @@ BRIDGE_REGISTER_RECORD(ledger::dto::GetBudgetReportResult, "Ledger_BudgetReport" morph::record::keyless, morph::record::derivedFrom) ``` -- **The key** defaults to a member named `id`, as a view's `rowKey` defaults to `"id"`. It may be any `ModelKey`, - raw or wrapped. On the wire it is the raw scalar, as `ModelKey` already encodes it. `morph::record::key<&T::m>` - names another member, and `morph::record::keyless` declares none. -- **`derivedFrom`** says that a change to any `S` is a change to this record type. Fan-out expands it once, - when the change is derived (§8). A derived type is keyless, because a key-level change to a source does not say - which derived rows it moved. -- The record registry is process-wide and filled during the registration phase, like the action registry, and the - same latch checks it. `derivedFrom` cycles are refused at registration. +- **The key** defaults to a member named `id`, the field the interpreter's `patch` already identifies rows by. It + may be any `ModelKey`, raw or wrapped, and on the wire it is the raw scalar, as `ModelKey` already encodes it. + `morph::record::key<&T::m>` names another member, and `morph::record::keyless` declares none. +- **`derivedFrom`** says that a change to any `S` is a change to this record type. Fan-out expands it when it + derives a change (§8). A derived type is keyless, because a key-level change to a source does not say which + derived rows moved. +- **`morph::record::privateKeys`** marks a type whose keys must not reach other principals (§12). +- `derivedFrom` cycles, and two registrations of one type or one name, are refused when the registration phase + closes. ### Effects -Effects are declared by their own macro, beside the action's registration. This leaves `BRIDGE_REGISTER_ACTION`'s -argument-count forms untouched, including `Loggable` as the fourth argument and the `_FOR_CLIENT` forms. It also -keeps every DTO free of morph types, which preserves the rule that a model does not know the framework exists. - ```cpp BRIDGE_REGISTER_ACTION(TransactionModel, Deposit, "Deposit") BRIDGE_REGISTER_EFFECTS(Deposit, morph::effects::changes, morph::effects::changes) @@ -138,35 +152,43 @@ BRIDGE_REGISTER_EFFECTS(RemovePayee, morph::effects::changes) // An action that edits one row and returns it, such as a hypothetical EditPayee returning PayeeInfo: // BRIDGE_REGISTER_EFFECTS(EditPayee, morph::effects::updates) BRIDGE_REGISTER_EFFECTS(LoadSample, morph::effects::changesInstance) -// ListPayees, GetAccount: no effects, they read. +BRIDGE_REGISTER_EFFECTS(OpenSample, morph::effects::none) +// ListPayees, GetAccount: nothing declared, they read. ``` -- **`BRIDGE_REGISTER_EFFECTS(A, effects...)`** specialises `morph::model::ActionEffects`. `ActionTraits` - gains `effects()`, which returns the list as data (`{kind, record?}`), empty when `A` has no specialisation. - The framework reads effects only through `effects()`. +- **`BRIDGE_REGISTER_EFFECTS(A, effects...)`** expands to a static registrar that records, under + `ActionTraits::typeId()`, the effects as data (`{kind, record type?}`). It must appear where `ActionTraits` + is visible: after `BRIDGE_REGISTER_ACTION`, or after `BRIDGE_DECLARE_ACTION` in the same header. It defines no + specialisation, so a translation unit that does not see it is not inconsistent with one that does. +- **Client-only builds** (`MORPH_CLIENT_ONLY`) compile the registrars too; they hold no model code. A pure remote + client does not consult them: it takes effects and records from `ui-catalog` (§5), as it takes every other + action fact. - **`updates` finds the records in the result.** The result may be an `R`, a `std::vector`, or an aggregate - whose members, found by reflection, hold an `R` or a `std::vector`. If none is found, registration fails - when the program starts, not when the action first runs. `R` must have a key. Bank's `Transfer` returns two - balances (`TransferResult{fromBalanceMinor, toBalanceMinor}`), not two `AccountInfo`s, so it declares - `changes`. + whose members, found by reflection, hold an `R` or a `std::vector`. `R` must have a key. If no `R` is found, + or `R` has no key, startup fails when the registration phase closes, not when the action first runs. Bank's + `Transfer` returns two balances (`TransferResult{fromBalanceMinor, toBalanceMinor}`), not two `AccountInfo`s, so + it declares `changes`. - **`updates` only changes rows that exist.** A key that no cached entry holds is not added to any list, because whether it belongs there is the server's query to answer. An action that may create a record declares `changes`. - **`changes` needs nothing from the result.** It covers creation, removal, and changes the result does not carry. -- **`changesInstance`** means the instance's own state changed. It reaches the caller's entries through the same - instance (§9), and, for a shared instance, other clients attached to that instance (§7). +- **`changesInstance`** means the state of the instance the action ran on changed: + - the caller's entries through that instance refetch (§9); + - for a private instance, the event also goes to the connection that owns it (§8); + - for a keyed shared instance, it also goes to the other clients watching that instance (§7). - **Several effects compose,** as above. -- **No declaration means the action reads, and nothing is pushed.** A read is safe by construction, because a - query may only run an action with no effects (§10). An action that writes and declares nothing behaves as it - does today, and §10's checks point at it. +- **Nothing declared means the action reads, and nothing is pushed.** A query may only run such an action (§10). An + action that writes and declares nothing behaves as it does today, and §10's warnings point at it. - **Only success has effects.** A failed action or a cancelled one changes nothing. ### What each action's result holds -Registration also records, per action, the record types its **result** holds, found by the same reflection as -`updates`: `records(A)`. A cached query holds exactly the records of its action. `ListResults` returns -`ListResultsResult{results: vector}`, so its entries hold `Lims_Result`. +Each action's existing registration also records a reflection walker over its `Result` type, which needs no record +to be registered yet. When the registration phase closes, the walker's type identities are matched against the +record registry, giving `records(A)`. For each record, it gives the **paths** at which the result holds it and +whether it is keyed. A cached query holds exactly the records of its action. `ListResults` returns +`ListResultsResult{results: vector}`, so its entries hold `Lims_Result` at path `results`. ## 5. The catalog @@ -175,12 +197,13 @@ existing ones. | Field | Meaning | |---|---| -| `effects` | `[{kind: "updates" \| "changes" \| "changesInstance", record?}]`. | -| `records` | The record types the action's result holds (§4), each with whether it is keyed. | +| `effects` | `[{kind: "updates" \| "changes" \| "changesInstance" \| "none", record?}]`. | +| `records` | `[{record, keyField?, paths: [...]}]`: the record types the action's result holds, the key member's wire name when keyed, and the JSON paths at which the result holds them (`""` for the result itself, `"results"` for a member list). | -`records` lets the client decide, without asking the server, whether a cache entry holds record `R`: the entry's -action's `records` contain `R`. `derivedFrom` is never sent. The server expands it before a change leaves (§8), so a -client only ever matches the record types its entries hold. +From `records` the client can decide, without asking the server, whether a cache entry holds record `R`, where its +rows are, and which field identifies them. The patch therefore matches rows by the record's key field, not only by +`id`. `derivedFrom` is never sent. The server expands it before a change leaves (§8), so a client only ever matches +the record types its entries hold. An `evaluate` cache entry (`ui-evaluate`, spec 5 §7) holds no records and is never watched: an evaluation depends on its body, not on stored data. @@ -197,19 +220,23 @@ The wire gains its first server-initiated message. - **The envelope.** `kind: "event"`, `callId: 0`, `body: {topic, payload}`. For this design the topic is `changes`. A client routes it by kind before reply matching, so an event is never taken for an orphaned reply. - **Writing to a connection.** `RemoteServer` gains `setConnectionWriter(cid, write)`, the transport's writer for a - connection already opened with `openConnection()`, and `post(cid, envelope)`. `post` is callable from any thread. - It hands the encoded envelope to the writer, which queues it on the transport's own thread, as replies are - queued today. The size cap (`kMaxEnvelopeBytes`) applies. A write failure drops the connection's watches, - as a disconnect does. -- **The outbound cap.** Events are queued per connection, up to 256 pending events or 1 MiB. On overflow the queue - is replaced by one `resync` event, meaning "everything you watch may have changed", and the client marks every - watched entry stale. The connection is not closed, and replies are not affected. This is the only outbound limit - the transports gain. + connection already opened with `openConnection()`, and `post(cid, envelope)`. + - `post` is callable from any thread. It hands the encoded envelope to the writer, which queues it on the + transport's own thread, as replies are queued today. + - The size cap (`kMaxEnvelopeBytes`) applies. + - A write failure drops the connection's watches, as a disconnect does. +- **The outbound cap.** Events are queued per connection, up to 256 pending events or 1 MiB. + - On overflow, the queue is replaced by one `resync` event, meaning "everything you watch may have changed", and + the client marks every watched entry stale. + - Overflow itself closes nothing. A transport's own rules still apply: the `net` writer closes a peer that + stalls past `sendTimeout`, as it does for replies. + - This is the only outbound limit the transports gain. - **The client side.** `IBackend` gains `setEventHandler(fn)`, delivered on the bridge's executor: - - The remote backends call it for each `event`: the Qt WebSocket backend, the `net` socket backend, and - `SimulatedRemoteBackend`, which the core transport tests use. - - The local backend calls it in-process, with no wire, so local mode gets the same behaviour from the same code - above the backend. + - the Qt WebSocket backend, the `net` socket backend and `SimulatedRemoteBackend`, which the core transport tests + use, call it for each `event`; + - the local backend has no server and produces no events in production. Local mode is covered by the effects + every reply applies (§9). The local backend exposes `deliverEvent(envelope)` as a test seam, so the client + cache's event path is tested without a server. - **Order.** Events on one connection arrive in the order the server posted them. No order is promised between an event and any reply. @@ -219,21 +246,30 @@ A client watches the topics its cache holds, so the server sends each connection - **Topics.** - A record type, for every record its entries hold. - - `(model type, instance key)` for every entry read through a **shared** instance whose action holds no - records, which is the case for working-copy reads. - - A private instance is never a topic, because no other connection can reach it. + - `(model type, instance id)` for every entry read through a **keyed shared** instance whose action holds no + records, which is the case for working-copy reads. The instance id is the binding's `currentId`. + - A private instance needs no watch: its owner is known to the server (§8). A keyless `"shared"` alias names a + stateless model, which has no instance state to change. +- **The kinds.** `watch` and `unwatch` are **core** kinds, handled by `RemoteServer` on its strand with the + connection's `cid`, because the watch table belongs to the connection and fan-out reads it in the execute path. + - Core can validate them: after part 1, records are a core registry, and instance topics are checked with + `authorizeInstance`. + - The body is `{records: [...], instances: [{modelType, instanceId}]}`. The `ok` reply lists any topic refused + (§12). + - The UI layer sends them; it does not handle them. - **Lifetime.** A topic is watched while at least one cache entry holds it, mounted or kept (spec 5 §5: for 5 minutes). The client counts references per topic. It sends `watch` when a count leaves zero and `unwatch` when it - returns to zero. Both are UI service kinds (`ui-watch`, `ui-unwatch`) with `{records: [...], instances: [...]}`, - and an `ok` reply that lists any topic refused (§12). -- **Watch before fetch.** Watch changes made in one reactive batch are sent as one message, **before** the - fetches of that batch, on the same connection. The server handles them in arrival order on its strand. A write - that commits after the fetch has read therefore finds the watch in place, and one that commits before it is in - the fetched value. Neither is missed. -- **Reconnect.** After a reconnect or a backend switch, the client negotiates again and sends its whole watch set - before anything else, then marks every entry stale, as spec 6 §9 already does for a backend switch. Events missed - while disconnected are not replayed: the stale marks cover them. Offline replay comes after the watch set, so its - changes arrive as events too (§9). + returns to zero. +- **Watch before fetch.** Watch changes made in one reactive batch are sent as one message, **before** the fetches + of that batch, on the same connection. The server handles `watch` on its strand when the message arrives, and an + `execute` is admitted on that same strand before it is posted to the model strand. The watch is therefore in the + table before the fetch reads anything: + - a write that commits before the fetch reads is in the fetched value; + - one that commits after produces an event, which §9 keeps even if the fetch is still in flight. +- **Reconnect.** The watch set is resent in the `bindContext` step of the offline reconnect sequence (`activate` → + `bindContext` → `replay`), after `ui-hello` has negotiated again and before the offline queue replays. The client + then marks every entry stale, as spec 6 §9 already does for a backend switch. Events missed while disconnected are + not replayed; the stale marks cover them. The replay's own changes arrive as events (§8). - **Granularity.** Record topics are per type, not per key. Per-key watching would make the server track every key every client shows, and those keys change as lists scroll. The event carries keys, and the client filters on them (§9). @@ -246,20 +282,27 @@ execute path of its own (`ui-evaluate` never executes), so it needs nothing. After an `execute` succeeds on the model's strand: -1. The model strand derives the changes: the effects' record topics, with keys read from the result for `updates`, - expanded through `derivedFrom`; and, for `changesInstance` on a shared instance, its instance topic. It posts the - changes to the server strand. The reply is written as it is today, unaffected. -2. On the server strand, the changes are matched against the watch table. **The calling connection is included.** - It also applies the effects from its reply (§9), and a second refetch request for the same entry is deduplicated - by the cache. Including it covers every case in which the caller never sees its own reply: an execute that timed - out on the client but succeeded on the server, a reply that `latest` discarded, and a replay whose reply went to - the offline worker. -3. Each matching watch is re-authorised with the principal stored on it (§12), and the connection's coalescer - queues the change. +1. **The model strand derives the changes.** + - Record topics come from the effects, with keys read from the result for `updates`, and are expanded through + `derivedFrom`. + - For `changesInstance`, the topic is the instance the action ran on. + + It posts the changes to the server strand. The reply is written as it is today, unaffected. +2. **The server strand finds the recipients.** + - Record and keyed-shared-instance topics go to every watching connection, **the calling connection included**. + - A private instance's topic goes only to the connection whose scope owns it. + + Including the caller covers the cases in which it never sees its own reply: an execute that timed out on the + client but succeeded on the server, a reply that `latest` discarded, and a replay whose reply went to the offline + worker. +3. **Each recipient watch is re-authorised** with the session context stored on it (§12), and the connection's + coalescer queues the change. - **Coalescing.** Changes for one connection are merged for up to 50 ms, or until 256 keys are queued, then sent as - one event. Keys are merged per record type, a `changes` absorbs that type's `updates` keys, and instance topics are - deduplicated. A burst of one hundred writes is then one refetch per showing query, not one hundred. + one event. + - Keys are merged per record type, and a `changes` absorbs that type's `updates` keys. + - Instance topics are deduplicated. + - A burst of one hundred writes is then one refetch per showing query, not one hundred. - **The watch table** lives on the server strand, next to the connection scopes `closeConnection` already reclaims. A closed connection drops its watches. @@ -269,21 +312,29 @@ The interpreter's query cache applies a change the same way whether it came from | Change | From a reply (the caller's own) | From an event | |---|---|---| -| `updates R` with keys | Patch: each entry holding `R` merges the new records into the rows with those keys, using the existing `patch` merge and the catalog's `listFields` for a list inside a record. No refetch. | Each entry holding `R` whose value contains one of the keys is marked stale, and refetched if mounted. | +| `updates R` with keys | Patch: each entry holding `R` merges the new records into the rows with those keys, at the paths and by the key field the catalog gives (§5). It uses the existing `patch` merge, and the catalog's `listFields` for a list inside a record. No refetch. | Each entry holding `R` whose value contains one of the keys is marked stale, and refetched if mounted. | | `changes R` | Each entry holding `R` is marked stale, and refetched if mounted. | The same. | -| `changesInstance` | Each entry read through the same instance is marked stale, and refetched if mounted. | The same, for the shared instance named. | +| `changesInstance` | Each entry read through the same instance is marked stale, and refetched if mounted. | The same, for the instance named: the private one this connection owns, or a keyed shared one it watches. | | `resync` | none | Every watched entry is marked stale, and refetched if mounted. | +- **A change during a fetch is kept.** A stale mark on an entry whose fetch is pending sets `refetchAfter`. When the + fetch settles, the entry stays stale and fetches once more, even if it succeeded. Otherwise the success would clear + the mark, and a change committed after the fetch read would be lost. This replaces today's unconditional + `stale = false` on settle, and it is what makes the watch-before-fetch argument in §7 hold. - **Every reply applies its effects.** The single hook is `session.actionSucceeded`, which every success already goes through: mutations, and form, collection and wizard submits (`aliasSubmits`). It gains the reply's JSON, so `updates` can patch. A `latest` run whose reply is discarded still applies its effects; only its value is dropped. An offline replay's reply reaches the offline worker, not a mutation. Its effects reach the cache as an event, because the caller is not excluded (§8). +- **Duplicates cost at most one extra fetch.** The caller sees each change twice: once from its reply and once as an + event. + - On an idle entry, the second mark lands on an entry already stale or already fetching. While fetching, it sets + `refetchAfter` (above) and costs one extra fetch. + - That fetch is the price of never losing a change. Telling a duplicate from a genuine second change would need the + event to name the call it came from, and §15 defers that. - **Why other clients refetch rather than receive values.** The new values were computed for the caller. Another user may see different fields (`secretFields`) or may not be allowed the record at all. A refetch goes through the receiver's own authorised read. Pushing values to other receivers is the first item in §15. -- **Duplicates are harmless.** The caller sees each change twice, once from its reply and once as an event. Marking - an entry that is already stale, or already refetching, is a no-op, so a duplicate costs nothing. - **Batching.** One event, or one reply's effects, is applied in one reactive batch, so a screen updates in one frame. - **`optimistic` is unchanged.** It still shows a guess before the reply, and the reply's patch confirms or corrects @@ -296,14 +347,19 @@ With effects declared, a document needs no refresh declarations. **At load, in `ui/1`, additively.** The loader already has the catalog, and it reports: -- **refused: a query whose action declares effects.** A query must read. This is what makes the default safe: two - clients cannot refetch each other in a ring, because a fetch never produces a change. It is new in what it - refuses, so it applies only to actions that declare effects, which no existing action does. +- **refused: a query whose action declares effects** (anything other than nothing declared). A query must read, so a + fetch never produces a change and two clients cannot refetch each other through their queries. It is new in what + it refuses, so it applies only to actions that declare effects, which no existing action does. +- **warning, watch feedback:** a document `watch` whose `on` reads a query's value, whose `source` is `any` (the + default), and whose `do` runs a mutation with effects. Another client's change refetches the query, the watch + fires, and the mutation's change reaches the other client, which may do the same. Such a watch should use + `"source": "user"`. The loader cannot prove a ring, so this is a warning, and the claim above is limited to + queries. - **warning, redundant:** an `invalidates`, `patch` or `refreshOn` that the mutation's action's effects already cover, so a document can migrate one screen at a time. -- **warning, uncovered:** a mutation whose action declares no effects, with no `invalidates` or `patch` of its own - and no query naming it in `refreshOn`. After it succeeds, nothing on any screen updates. It may genuinely change - nothing that is shown, so this is only a warning, but it points at actions still to annotate. +- **warning, uncovered:** a mutation whose action declares nothing (not even `none`), with no `invalidates` or + `patch` of its own and no query naming it in `refreshOn`. After it succeeds, nothing on any screen updates. + `effects::none` silences it for actions that attach or open, such as `OpenSample` and `OpenBoard`. **In `ui/2`, the removal.** `invalidates`, `patch` and `refreshOn` are removed from mutations and queries. Removing keys is not additive, so it is a new vocabulary version under spec 5 §12. A server that keeps `ui/1` documents for @@ -324,12 +380,14 @@ its golden files (`tests/ui/golden/`) change with it. An agent acts on the application through the same dispatch as everything else, so this design is what makes its changes visible. -- **The MCP adapter** is a server-side component. It opens a connection scope with `openConnection()` for each - agent session, and sends each tool call through `RemoteServer::handle(msg, reply, cid)` as an ordinary `execute`. - That keeps every check the GUI gets: authentication and the principal, `authorize` and `authorizeInstance`, - limits, the journal, cancellation and the in-flight count. It lists the actions the principal may run (the - catalog's `permitted`, with their JSON shapes) as MCP tools, and it offers `ui-evaluate` as a dry run. The agent - may commit directly. +- **The MCP adapter** is a server-side component. + - It opens a connection scope with `openConnection()` for each agent session, and sends each tool call through + `RemoteServer::handle(msg, reply, cid)` as an ordinary `execute`. + - That keeps every check the GUI gets: authentication and the principal, `authorize` and `authorizeInstance`, + limits, the journal, cancellation and the in-flight count. + - It lists the actions the principal may run (the catalog's `permitted`, with their JSON shapes) as MCP tools, and + offers `ui-evaluate` as a dry run. + - The agent may commit directly. - **Fan-out treats the agent's connection like any other,** so every open GUI showing the changed records updates through §8 and §9. - **Model access and keys are not part of this design.** The adapter does not call a model provider. An external @@ -338,39 +396,46 @@ changes visible. ## 12. Security -- **Each watch carries a principal.** A connection has no principal of its own: the server verifies one on every - envelope. The watch table therefore stores, with each watch, the verified principal of the `ui-watch` envelope - that created it. +- **Each watch carries a session context.** A connection has no principal of its own: the server verifies one on + every envelope. The watch table therefore stores, with each watch, the session context (principal and token) of the + `watch` envelope that created it, and the verified principal. - **A later envelope on the connection with a different verified principal drops all of the connection's - watches.** That covers signing in as someone else, and a client that keeps old watches across a sign-out. The - client's cache is already cleared on a principal change (spec 5 §5), and it watches again. - - Fan-out re-authorises each watch at send time, with its stored principal and that token's expiry. An expired - token drops the watch. The client watches again with its next envelope, which carries a fresh token. + watches.** The comparison is made in `stampVerifiedPrincipal`, which every kind passes through, given the + envelope's `cid`. That covers signing in as someone else, and a client that keeps old watches across a sign-out. + The client's cache is already cleared on a principal change (spec 5 §5), and it watches again. + - **Fan-out re-authorises each watch at send time.** It runs `authenticate` on the stored context, which fails for + an expired token, then `authorize` for the action that produced the change, or `authorizeInstance` for an + instance topic. A failure drops the watch. The client watches again with its next envelope, which carries a + fresh token. - **What a watch may name.** - A record type: only when the principal may run at least one action whose `records` contain it. Otherwise the `ok` reply lists it as refused, and the client logs it. - An instance topic: only when `authorizeInstance` admits the principal to that instance. -- **An event reveals that records of `R` changed, and their keys.** Keys are ids of records the receiver could +- **An event reveals that records of `R` changed, and their keys.** The keys are ids of records the receiver could list through some action returning `R`, but not necessarily the same ones. For a model that filters rows per - principal, the record type is registered with `morph::record::privateKeys`. Its changes then reach principals other - than the caller's without keys, and every entry holding `R` refetches. That costs more and reveals nothing. + principal, the record type is registered with `morph::record::privateKeys`. Its changes then reach principals + other than the caller's without keys, and every entry holding `R` refetches. That costs more and reveals nothing. - **Values never travel to other connections** (§9), so `secretFields` and row-level filtering stay on the read path, where they are enforced today. -- **Volume.** Coalescing bounds events to 20 a second per connection, and the outbound cap (§6) bounds what a slow - reader can queue. The query-must-read rule (§10) removes the only way the design could feed itself. +- **Volume.** + - Coalescing bounds events to 20 a second per connection. + - The outbound cap (§6) bounds what a slow reader can queue. + - The query-must-read rule (§10) removes the way a fetch could feed itself. ## 13. Tests Every test below must fail if the feature does nothing, so each names the change that should break it. - **Records and effects (core).** - - Registration extracts raw and wrapped keys from an `R`, a `vector` and a nested member. - - `updates` over a result with no `R`, and a `derivedFrom` cycle, both fail registration. - - `records(A)` lists the result's record types. + - Registration extracts raw and wrapped keys, and their paths, from an `R`, a `vector` and a nested member. + - Registration order across translation units does not change `records(A)`. + - `updates` over a result with no `R`, `updates` on a keyless `R`, and a `derivedFrom` cycle all fail when + the phase closes. + - A translation unit that does not see `BRIDGE_REGISTER_EFFECTS` compiles and links with one that does. - *Mutation: drop the reflection walk; key extraction fails.* - **Channel (core, net, qt).** - - One conformance case runs on both transports, on `SimulatedRemoteBackend` and on the local backend: a posted - event reaches the handler. + - One conformance case runs on both transports and on `SimulatedRemoteBackend`: a posted event reaches the + handler. - A connection that did not offer `events` receives none. - An event is not taken for a reply. - Overflow produces exactly one `resync`. @@ -379,21 +444,30 @@ Every test below must fail if the feature does nothing, so each names the change - Two connections watch `R`, and an execute from A posts to both. - An unwatched type posts nothing, and a burst of 100 is one event. - A watch made under principal P is dropped by an envelope under Q. + - A watch whose token has expired is dropped at send time. - A watch on a type the principal cannot read is refused. + - A `changesInstance` on a private instance reaches only its owning connection. - An execute that times out at the client but succeeds still reaches the caller as an event. - *Mutation: skip re-authorisation; the principal case fails.* - **Client cache (ui).** A recording-backend screen with a list query and no refresh declarations: - - An own `updates` reply patches the row without a fetch, and the fetch counter stays flat. - - A `changes` event refetches it. + - An own `updates` reply patches the row, matched by a non-`id` key field, without a fetch, and the fetch counter + stays flat. + - A `changes` event, delivered through the local backend's test seam, refetches it. + - A change marked while a fetch is pending causes exactly one more fetch after it settles. - A submit (not a mutation) applies its effects. - A discarded `latest` reply still applies its effects. - `changesInstance` refetches only entries of that instance. - - *Mutation: skip the cache step in `actionSucceeded`; the shown row keeps its old value.* -- **Documents (ui).** A query whose action declares effects is refused at load. A redundant `invalidates` warns. An - uncovered mutation warns. -- **End to end (examples).** One rung runs two clients against one server. Client A's mutation updates client B's - mounted screen, with no `refreshOn` and no `refreshEvery` in either document. *Control: remove the action's - `BRIDGE_REGISTER_EFFECTS`; B does not update.* + - *Mutation: restore the unconditional `stale = false`; the mid-flight case fails.* +- **Documents (ui).** + - A query whose action declares effects is refused at load. + - A redundant `invalidates` warns. + - An uncovered mutation warns, and `effects::none` silences it. + - The watch-feedback shape warns. +- **End to end.** + - A harness in `tests/` runs two generic-client sessions over the `net` transport against one server. + - Client A's mutation updates client B's mounted screen, with no `refreshOn` and no `refreshEvery` in either + document. + - *Control: remove the action's `BRIDGE_REGISTER_EFFECTS`; B does not update.* ## 14. Delivery @@ -402,33 +476,41 @@ Each part updates the `docs/spec/` pages named in its row, and keeps every new p | # | Part | Layer | Needs | Spec pages | |---|---|---|---|---| -| 1 | Records and effects: `BRIDGE_REGISTER_RECORD`, keys raw and wrapped, `keyless`, `derivedFrom`, `privateKeys`; `BRIDGE_REGISTER_EFFECTS`, `ActionEffects`, `ActionTraits::effects()`; `records(A)`; deriving changes from a result | core | none | `core/registry.md` | -| 2 | Server-to-client channel: the `event` kind, `setConnectionWriter` and `post`, the outbound cap and `resync`, both transports, `SimulatedRemoteBackend`, `setEventHandler`, in-process delivery in the local backend | core, net, qt | none | `core/wire.md`, `core/backend.md`, `security.md` | -| 3 | Catalog and negotiation: `ActionInfo::effects` and `records`, local lookup and `ui-catalog`; `events` in `ui-hello` | ui | 1 | `ui/service.md`, `ui/document.md` | -| 4 | Watching and fan-out: `ui-watch` and `ui-unwatch`, the watch table with principals, fan-out in `dispatchExecute`, `derivedFrom` expansion, coalescing, re-authorisation, `privateKeys` | core, ui | 1, 2, 3 | `core/backend.md`, `ui/service.md`, `security.md` | -| 5 | Client cache: `actionSucceeded` carries the reply and applies effects; events and `resync`; topic reference counting; watch before fetch; reconnect order | ui | 2, 3 (4 for its remote tests) | `ui/interpreter.md`, spec 5 §5 | -| 6 | Document checks: query-must-read refused; redundant and uncovered warnings | ui | 3, 5 | `ui/document.md` | -| 7 | Examples: records and effects in the seven rungs that use refresh declarations, their declarations removed in `ui/1` documents where the warnings say so, and the two-client end-to-end test | examples | 5, 6 | the rungs' READMEs | -| 8 | `ui/2`: remove `invalidates`, `patch` and `refreshOn`; `optimistic` keeps its own shape; the authoring layer and golden files emit `ui/2`; the server keeps serving `ui/1` | ui | 7 | `ui/document.md`, `ui/author.md`, spec 5 §5 and §12 | -| 9 | MCP adapter: a connection scope per agent session, tools from `permitted`, calls through `RemoteServer::handle` | net, ui | 4 | a new `ui/agent.md` | - -**Ready now:** parts 1 and 2. They are independent: part 1 is `registry.hpp` and a new record header, part 2 is -`wire.hpp`, `remote.hpp`, `backend.hpp` and the transports. **Then:** part 3, then parts 4 and 5 side by side. Part 5 -uses the local backend's in-process delivery for its tests until part 4 lands. Part 9 can run beside anything after -part 4. +| 1 | **Records and effects.** `BRIDGE_REGISTER_RECORD`, raw and wrapped keys, `keyless`, `derivedFrom`, `privateKeys`; `BRIDGE_REGISTER_EFFECTS` as a registrar; the result walker in the action registration; matching when the phase closes; `records(A)` with paths; deriving changes from a result | core | none | `core/registry.md` | +| 2 | **Server-to-client channel.** The `event` kind, `setConnectionWriter` and `post`, the outbound cap and `resync`, both transports, `SimulatedRemoteBackend`, `setEventHandler`, the local backend's `deliverEvent` seam | core, net, qt | none | `core/wire.md`, `core/backend.md`, `security.md` | +| 3 | **Catalog and wire shapes.** `ActionInfo::effects` and `records` (key field, paths), local lookup and `ui-catalog`; `events` in `ui-hello`; the `watch`/`unwatch` and `changes` body shapes, defined here so parts 4 and 5 share them | ui, core (`wire.hpp` shapes) | 1 | `ui/service.md`, `ui/document.md`, `core/wire.md` | +| 4 | **Watching and fan-out.** Core `watch`/`unwatch` kinds in `RemoteServer` with the `cid`; the watch table with session contexts; the principal comparison in `stampVerifiedPrincipal`; fan-out in `dispatchExecute`, with private instances to their owner; `derivedFrom` expansion, coalescing, re-authorisation, `privateKeys` | core | 1, 2, 3 | `core/backend.md`, `core/shared_instances.md`, `security.md` | +| 5 | **Client cache.** `actionSucceeded` carries the reply and applies effects; patch by key field and path; `refetchAfter`; events and `resync`; topic reference counting; watch before fetch; the watch resend in `ReconnectCoordinator`'s `bindContext` | ui, offline glue | 2, 3 (4 for its remote tests) | `ui/interpreter.md`, `offline/offline.md`, `ui/host.md`, spec 5 §5 | +| 6 | **Document checks.** Query-must-read refused; watch-feedback, redundant and uncovered warnings | ui | 3, 5 | `ui/document.md` | +| 7 | **End-to-end harness and examples.** The two-session harness in `tests/`. Records and effects in the seven rungs that use refresh declarations (`none` on attach-style actions), with their declarations removed in `ui/1` documents where the warnings say so | tests, examples | 5, 6 | the rungs' READMEs, `testing_strategy.md` | +| 8 | **`ui/2`.** Remove `invalidates`, `patch` and `refreshOn`; `optimistic` keeps its own shape; the authoring layer and golden files emit `ui/2`; the server keeps serving `ui/1` | ui | 7 | `ui/document.md`, `ui/author.md`, spec 5 §5 and §12 | +| 9 | **MCP adapter.** A connection scope per agent session, tools from `permitted`, calls through `RemoteServer::handle` | net, ui | 4 | a new `ui/agent.md` | + +**Ready now: parts 1 and 2.** They are independent. Part 1 is `registry.hpp` and a new records header. Part 2 is +`wire.hpp`'s event kind, `remote.hpp`'s writer and `post`, `backend.hpp` and the transports. + +**Then part 3, then parts 4 and 5 side by side.** +- They share only the shapes part 3 defines. +- Part 4 is in `remote.hpp`; part 5 is in the interpreter and the offline glue. +- Part 5 tests its event path through the local backend's seam until part 4 lands. + +Part 9 can run beside anything after part 4. ## 15. Out of scope - **Pushing values to other connections.** Receivers refetch through their own authorised read. Sending the new record, filtered per receiver, would save that round trip. It needs per-receiver field filtering on the server, and it is the first follow-up once refetch traffic has been measured. +- **Suppressing the caller's duplicate.** An event could name the call that caused it, so the caller skips the change + it already applied from the reply. That saves the one extra fetch §9 accepts, and is deferred until it is measured. - **Per-key and per-field watching.** Watching is by record type, and the event carries keys. Field-level effects would let a client skip refetches for fields a screen does not show. That is deferred until a measurement shows refetches cost something. - **Effects computed at run time,** such as a change list a model returns. Declared effects with `derivedFrom` cover the examples' cases. A run-time list would have to keep the rule that a model does not know the framework exists. - **Push for non-UI clients.** Negotiation is in `ui-hello`. A plain `Client` that never sends it receives no events. - Offering events in the core `hello` would need that request to carry capabilities, which it does not today. + The core `hello` reply already lists capabilities, but its request carries none, so a plain client has no way to + offer events. Adding that is a separate change to `hello`. - **An assistant built into the client, and where its model keys live.** §11 covers agents acting through the server only. - **Push for anything other than changes:** progress of a long action, notifications. The `event` envelope has a From eb9f945aea115ac80734e0541d2dcd1d17c6ad15 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Fri, 16 Oct 2026 19:33:48 +0200 Subject: [PATCH 04/68] =?UTF-8?q?docs:=20spec=208=20=E2=80=94=20address=20?= =?UTF-8?q?the=20third=20review=20round?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 --- .../specs/2026-10-16-server-push-design.md | 140 +++++++++++++----- 1 file changed, 99 insertions(+), 41 deletions(-) diff --git a/docs/superpowers/specs/2026-10-16-server-push-design.md b/docs/superpowers/specs/2026-10-16-server-push-design.md index 88e412178..162363c42 100644 --- a/docs/superpowers/specs/2026-10-16-server-push-design.md +++ b/docs/superpowers/specs/2026-10-16-server-push-design.md @@ -157,12 +157,16 @@ BRIDGE_REGISTER_EFFECTS(OpenSample, morph::effects::none) ``` - **`BRIDGE_REGISTER_EFFECTS(A, effects...)`** expands to a static registrar that records, under - `ActionTraits::typeId()`, the effects as data (`{kind, record type?}`). It must appear where `ActionTraits` - is visible: after `BRIDGE_REGISTER_ACTION`, or after `BRIDGE_DECLARE_ACTION` in the same header. It defines no - specialisation, so a translation unit that does not see it is not inconsistent with one that does. -- **Client-only builds** (`MORPH_CLIENT_ONLY`) compile the registrars too; they hold no model code. A pure remote - client does not consult them: it takes effects and records from `ui-catalog` (§5), as it takes every other - action fact. + `ActionTraits::typeId()`, the effects as data (`{kind, record type?}`). + - It must appear where `ActionTraits` is visible. Its place is beside the action's registration: after + `BRIDGE_REGISTER_ACTION`, or after `BRIDGE_REGISTER_ACTION_SOURCE` in the `.cpp`, for an action split with + `BRIDGE_DECLARE_ACTION`. In a header it would run once per translation unit, which is the static-initialisation + cost `BRIDGE_DECLARE_ACTION` exists to avoid. + - It is idempotent per action, like `registerActionOnce`: registering the same effects twice is a no-op. Registering + different effects for one action is refused when the phase closes. + - It defines no specialisation, so a translation unit that does not see it is not inconsistent with one that does. +- **Client-only builds** (`MORPH_CLIENT_ONLY`) compile the registrars out, as they do every action registrar. A + pure remote client takes effects and records from `ui-catalog` (§5), as it takes every other action fact. - **`updates` finds the records in the result.** The result may be an `R`, a `std::vector`, or an aggregate whose members, found by reflection, hold an `R` or a `std::vector`. `R` must have a key. If no `R` is found, or `R` has no key, startup fails when the registration phase closes, not when the action first runs. Bank's @@ -184,11 +188,26 @@ BRIDGE_REGISTER_EFFECTS(OpenSample, morph::effects::none) ### What each action's result holds -Each action's existing registration also records a reflection walker over its `Result` type, which needs no record -to be registered yet. When the registration phase closes, the walker's type identities are matched against the -record registry, giving `records(A)`. For each record, it gives the **paths** at which the result holds it and -whether it is keyed. A cached query holds exactly the records of its action. `ListResults` returns -`ListResultsResult{results: vector}`, so its entries hold `Lims_Result` at path `results`. +Each action's existing registration (`BRIDGE_DETAIL_ACTION_REGISTRARS`, which every form of the macro runs) also +records a reflection walker over `ActionTraits::Result`. That is the unwrapped type, so a `Task` handler +contributes `R`. The walker needs no record to be registered yet. When the registration phase closes, the walker's +type identities are matched against the record registry, giving `records(A)`. For each record, it gives the +**paths** at which the result holds it and whether it is keyed. A cached query holds exactly the records of its +action. `ListResults` returns `ListResultsResult{results: vector}`, so its entries hold `Lims_Result` at +path `results`. + +- **What the walker visits.** + - The result itself; a `std::vector`, `std::optional` or `std::map` value of a visited type. + - The members of a reflectable aggregate. Their JSON names are taken from `glz::reflect`'s keys, which are the + names the codec writes. + - It recurses into members, depth-first, so a record inside a record (a row list inside a header) has a path such + as `header.rows`. It stops at a registered record, so a record's own members are not searched for further + records. It stops at a type already on its path, so recursive types end. +- **What it skips:** `void`, scalars, strings, and any type it cannot reflect, such as a type with a custom + `glz::meta` codec or an anonymous-namespace struct. A skipped type contributes no records. An action whose result + is skipped holds none. `updates` on such an action fails when the phase closes, because `R` is not found. +- **The registrars are compiled out of client-only builds.** `MORPH_CLIENT_ONLY` empties every registrar, these + included. That is consistent with a pure remote client taking these facts from `ui-catalog`. ## 5. The catalog @@ -225,12 +244,19 @@ The wire gains its first server-initiated message. transport's own thread, as replies are queued today. - The size cap (`kMaxEnvelopeBytes`) applies. - A write failure drops the connection's watches, as a disconnect does. -- **The outbound cap.** Events are queued per connection, up to 256 pending events or 1 MiB. - - On overflow, the queue is replaced by one `resync` event, meaning "everything you watch may have changed", and - the client marks every watched entry stale. - - Overflow itself closes nothing. A transport's own rules still apply: the `net` writer closes a peer that - stalls past `sendTimeout`, as it does for replies. - - This is the only outbound limit the transports gain. +- **The outbound bound.** The server cannot see what a transport still holds unsent: replies and events are handed + to the transport's outbox and forgotten. The design bounds what it can see, and names the transport's rule for the + rest. + - **The writer reports its backlog.** `setConnectionWriter(cid, write)` takes a writer that returns the bytes + still queued for that connection after the write. Both transports already own that queue (the `net` connection's + frame queue, Qt's socket buffer), so the writer reads it. + - **Events, not replies, give way.** When the reported backlog exceeds 1 MiB, the server stops posting events to + that connection and records that it owes it a `resync`. When a later write reports the backlog under 256 KiB, it + sends one `resync` event, meaning "everything you watch may have changed". The client marks every watched entry + stale. Replies are never held back. + - **A peer that stops reading** is closed by the transport's own rule: the `net` writer closes a peer that stalls + past `sendTimeout`, as it does today for replies. The Qt transport gains the same timeout in part 2, so both + behave alike. - **The client side.** `IBackend` gains `setEventHandler(fn)`, delivered on the bridge's executor: - the Qt WebSocket backend, the `net` socket backend and `SimulatedRemoteBackend`, which the core transport tests use, call it for each `event`; @@ -246,15 +272,18 @@ A client watches the topics its cache holds, so the server sends each connection - **Topics.** - A record type, for every record its entries hold. - - `(model type, instance id)` for every entry read through a **keyed shared** instance whose action holds no - records, which is the case for working-copy reads. The instance id is the binding's `currentId`. + - `(model type, primary key)` for every entry read through a **keyed shared** instance whose action holds no + records, which is the case for working-copy reads. The primary key is the one the alias binds with: the + interpreter already binds a keyed alias as `bind(type, Shared, key)` from its token, so the key is known before + the first fetch. The server resolves it through its instance directory (`DirectoryKey{typeId, primary}`). The + binding's `currentId` cannot serve here, because it is 0 until that first fetch has bound the alias. - A private instance needs no watch: its owner is known to the server (§8). A keyless `"shared"` alias names a stateless model, which has no instance state to change. - **The kinds.** `watch` and `unwatch` are **core** kinds, handled by `RemoteServer` on its strand with the connection's `cid`, because the watch table belongs to the connection and fan-out reads it in the execute path. - Core can validate them: after part 1, records are a core registry, and instance topics are checked with `authorizeInstance`. - - The body is `{records: [...], instances: [{modelType, instanceId}]}`. The `ok` reply lists any topic refused + - The body is `{records: [...], instances: [{modelType, primary}]}`. The `ok` reply lists any topic refused (§12). - The UI layer sends them; it does not handle them. - **Lifetime.** A topic is watched while at least one cache entry holds it, mounted or kept (spec 5 §5: for @@ -266,10 +295,17 @@ A client watches the topics its cache holds, so the server sends each connection table before the fetch reads anything: - a write that commits before the fetch reads is in the fetched value; - one that commits after produces an event, which §9 keeps even if the fetch is still in flight. -- **Reconnect.** The watch set is resent in the `bindContext` step of the offline reconnect sequence (`activate` → - `bindContext` → `replay`), after `ui-hello` has negotiated again and before the offline queue replays. The client - then marks every entry stale, as spec 6 §9 already does for a backend switch. Events missed while disconnected are - not replayed; the stale marks cover them. The replay's own changes arrive as events (§8). +- **Reconnect.** Both transports reconnect on their own, so the UI layer cannot rely on being told by an offline + coordinator. The hosting layer (`ui/host.hpp`) installs the backend's `setConnectHandler`, which runs on every + successful connect, the first included. On each connect it: + 1. sends `ui-hello` again, so the new connection negotiates `events`; + 2. sends the whole watch set; + 3. marks every entry stale, as spec 6 §9 already does for a backend switch. + + These are ordinary requests on the new connection, so they go ahead of any execute sent after them. That + includes an offline replay, which reaches the cache as events (§8). In an application that also runs + `ReconnectCoordinator`, its `bindContext` step runs before `replay`, and the connect handler has already run by + then. Events missed while disconnected are not replayed; the stale marks cover them. - **Granularity.** Record topics are per type, not per key. Per-key watching would make the server track every key every client shows, and those keys change as lists scroll. The event carries keys, and the client filters on them (§9). @@ -321,11 +357,17 @@ The interpreter's query cache applies a change the same way whether it came from fetch settles, the entry stays stale and fetches once more, even if it succeeded. Otherwise the success would clear the mark, and a change committed after the fetch read would be lost. This replaces today's unconditional `stale = false` on settle, and it is what makes the watch-before-fetch argument in §7 hold. +- **A patch on a fetching entry also refetches.** A reply's `updates` patch that lands on an entry whose fetch is + pending is applied and also sets `refetchAfter`. Otherwise that fetch, which may have read before the write + committed, would settle over the patch. Remotely the caller's duplicate event would repair this too, but local + mode has no events, so the patch path must not rely on one. - **Every reply applies its effects.** The single hook is `session.actionSucceeded`, which every success already goes through: mutations, and form, collection and wizard submits (`aliasSubmits`). It gains the reply's JSON, so - `updates` can patch. A `latest` run whose reply is discarded still applies its effects; only its value is - dropped. An offline replay's reply reaches the offline worker, not a mutation. Its effects reach the cache as an - event, because the caller is not excluded (§8). + `updates` can patch. + - **A `latest` run whose reply is discarded** applies its effects as marks only: each `updates` is treated as + `changes`. Its values are older than the reply a newer run already applied, so it must never patch. + - **An offline replay's reply** reaches the offline worker, not a mutation. Its effects reach the cache as an event, + because the caller is not excluded (§8). - **Duplicates cost at most one extra fetch.** The caller sees each change twice: once from its reply and once as an event. - On an idle entry, the second mark lands on an entry already stale or already fetching. While fetching, it sets @@ -400,13 +442,19 @@ changes visible. every envelope. The watch table therefore stores, with each watch, the session context (principal and token) of the `watch` envelope that created it, and the verified principal. - **A later envelope on the connection with a different verified principal drops all of the connection's - watches.** The comparison is made in `stampVerifiedPrincipal`, which every kind passes through, given the - envelope's `cid`. That covers signing in as someone else, and a client that keeps old watches across a sign-out. + watches.** The comparison is made in `stampVerifiedPrincipal`, given the envelope's `cid`. Every kind that + authorises passes through it; `hello`, which carries no session, does not. That covers signing in as someone else, and a client that keeps old watches across a sign-out. The client's cache is already cleared on a principal change (spec 5 §5), and it watches again. - - **Fan-out re-authorises each watch at send time.** It runs `authenticate` on the stored context, which fails for - an expired token, then `authorize` for the action that produced the change, or `authorizeInstance` for an - instance topic. A failure drops the watch. The client watches again with its next envelope, which carries a - fresh token. + - **Fan-out re-authorises each watch at send time, by the rule that admitted it.** + - It runs `authenticate` on the stored context, which fails for an expired token. + - For a record topic, it then requires again that the principal may run some action whose `records` hold the + type, the same rule as below. It does not check the action that wrote the change: a role that may list + results but not capture them still receives the change. + - For an instance topic, it runs `authorizeInstance`. + + A failure drops the watch. The client watches again with its next envelope, which carries a fresh token. The + "may run some action holding `R`" answer is cached per principal and record type, and refreshed when that + principal's token is. - **What a watch may name.** - A record type: only when the principal may run at least one action whose `records` contain it. Otherwise the `ok` reply lists it as refused, and the client logs it. @@ -419,7 +467,8 @@ changes visible. where they are enforced today. - **Volume.** - Coalescing bounds events to 20 a second per connection. - - The outbound cap (§6) bounds what a slow reader can queue. + - The backlog bound (§6) holds events back from a slow reader and replaces them with one `resync`, and the + transport's `sendTimeout` closes a reader that stops. - The query-must-read rule (§10) removes the way a fetch could feed itself. ## 13. Tests @@ -429,6 +478,8 @@ Every test below must fail if the feature does nothing, so each names the change - **Records and effects (core).** - Registration extracts raw and wrapped keys, and their paths, from an `R`, a `vector` and a nested member. - Registration order across translation units does not change `records(A)`. + - The walker finds a record nested inside a record, skips a type it cannot reflect, and ends on a recursive type. + - Registering the same effects twice is a no-op; different effects for one action fail when the phase closes. - `updates` over a result with no `R`, `updates` on a keyless `R`, and a `derivedFrom` cycle all fail when the phase closes. - A translation unit that does not see `BRIDGE_REGISTER_EFFECTS` compiles and links with one that does. @@ -438,7 +489,8 @@ Every test below must fail if the feature does nothing, so each names the change handler. - A connection that did not offer `events` receives none. - An event is not taken for a reply. - - Overflow produces exactly one `resync`. + - A writer reporting a backlog over 1 MiB holds events back, and exactly one `resync` follows when it drains. + Replies are not held back. - *Mutation: route events through reply matching; the routing case fails.* - **Watching and fan-out (core).** - Two connections watch `R`, and an execute from A posts to both. @@ -446,6 +498,8 @@ Every test below must fail if the feature does nothing, so each names the change - A watch made under principal P is dropped by an envelope under Q. - A watch whose token has expired is dropped at send time. - A watch on a type the principal cannot read is refused. + - A principal who may list `R` but not run the writer still receives the change. + - An instance topic named by primary key reaches a client whose alias has not fetched yet. - A `changesInstance` on a private instance reaches only its owning connection. - An execute that times out at the client but succeeds still reaches the caller as an event. - *Mutation: skip re-authorisation; the principal case fails.* @@ -455,7 +509,10 @@ Every test below must fail if the feature does nothing, so each names the change - A `changes` event, delivered through the local backend's test seam, refetches it. - A change marked while a fetch is pending causes exactly one more fetch after it settles. - A submit (not a mutation) applies its effects. - - A discarded `latest` reply still applies its effects. + - A discarded `latest` reply marks entries stale and patches nothing, so a newer reply's row is kept. + - A reply's `updates` patch on an entry whose fetch is pending causes one more fetch, in local mode. + - After the transport reconnects, the client sends `ui-hello` and its watch set again before any execute, and + events arrive on the new connection. - `changesInstance` refetches only entries of that instance. - *Mutation: restore the unconditional `stale = false`; the mid-flight case fails.* - **Documents (ui).** @@ -477,11 +534,11 @@ Each part updates the `docs/spec/` pages named in its row, and keeps every new p | # | Part | Layer | Needs | Spec pages | |---|---|---|---|---| | 1 | **Records and effects.** `BRIDGE_REGISTER_RECORD`, raw and wrapped keys, `keyless`, `derivedFrom`, `privateKeys`; `BRIDGE_REGISTER_EFFECTS` as a registrar; the result walker in the action registration; matching when the phase closes; `records(A)` with paths; deriving changes from a result | core | none | `core/registry.md` | -| 2 | **Server-to-client channel.** The `event` kind, `setConnectionWriter` and `post`, the outbound cap and `resync`, both transports, `SimulatedRemoteBackend`, `setEventHandler`, the local backend's `deliverEvent` seam | core, net, qt | none | `core/wire.md`, `core/backend.md`, `security.md` | -| 3 | **Catalog and wire shapes.** `ActionInfo::effects` and `records` (key field, paths), local lookup and `ui-catalog`; `events` in `ui-hello`; the `watch`/`unwatch` and `changes` body shapes, defined here so parts 4 and 5 share them | ui, core (`wire.hpp` shapes) | 1 | `ui/service.md`, `ui/document.md`, `core/wire.md` | +| 2 | **Server-to-client channel.** The `event` kind, `setConnectionWriter` with a backlog-reporting writer, `post`, holding events back and `resync`, both transports (and `sendTimeout` for the Qt one), `SimulatedRemoteBackend`, `setEventHandler`, the local backend's `deliverEvent` seam | core, net, qt | none | `core/wire.md`, `core/backend.md`, `security.md` | +| 3 | **Catalog and wire shapes.** `ActionInfo::effects` and `records` (key field, paths), local lookup and `ui-catalog`; `events` in `ui-hello`; the `watch`/`unwatch` and `changes` body shapes, defined here so parts 4 and 5 share them | ui, core (`wire.hpp` shapes) | 1, 2 | `ui/service.md`, `ui/document.md`, `core/wire.md` | | 4 | **Watching and fan-out.** Core `watch`/`unwatch` kinds in `RemoteServer` with the `cid`; the watch table with session contexts; the principal comparison in `stampVerifiedPrincipal`; fan-out in `dispatchExecute`, with private instances to their owner; `derivedFrom` expansion, coalescing, re-authorisation, `privateKeys` | core | 1, 2, 3 | `core/backend.md`, `core/shared_instances.md`, `security.md` | -| 5 | **Client cache.** `actionSucceeded` carries the reply and applies effects; patch by key field and path; `refetchAfter`; events and `resync`; topic reference counting; watch before fetch; the watch resend in `ReconnectCoordinator`'s `bindContext` | ui, offline glue | 2, 3 (4 for its remote tests) | `ui/interpreter.md`, `offline/offline.md`, `ui/host.md`, spec 5 §5 | -| 6 | **Document checks.** Query-must-read refused; watch-feedback, redundant and uncovered warnings | ui | 3, 5 | `ui/document.md` | +| 5 | **Client cache.** `actionSucceeded` carries the reply and applies effects; patch by key field and path; `refetchAfter`, set by a stale mark or a patch on a fetching entry; discarded `latest` replies as marks; events and `resync`; topic reference counting; watch before fetch; the connect handler in `ui/host.hpp` that re-sends `ui-hello` and the watch set | ui | 2, 3 (4 for its remote tests) | `ui/interpreter.md`, `ui/host.md`, `offline/offline.md`, spec 5 §5 | +| 6 | **Document checks.** Query-must-read refused; watch-feedback, redundant and uncovered warnings | ui | 3 | `ui/document.md` | | 7 | **End-to-end harness and examples.** The two-session harness in `tests/`. Records and effects in the seven rungs that use refresh declarations (`none` on attach-style actions), with their declarations removed in `ui/1` documents where the warnings say so | tests, examples | 5, 6 | the rungs' READMEs, `testing_strategy.md` | | 8 | **`ui/2`.** Remove `invalidates`, `patch` and `refreshOn`; `optimistic` keeps its own shape; the authoring layer and golden files emit `ui/2`; the server keeps serving `ui/1` | ui | 7 | `ui/document.md`, `ui/author.md`, spec 5 §5 and §12 | | 9 | **MCP adapter.** A connection scope per agent session, tools from `permitted`, calls through `RemoteServer::handle` | net, ui | 4 | a new `ui/agent.md` | @@ -489,7 +546,8 @@ Each part updates the `docs/spec/` pages named in its row, and keeps every new p **Ready now: parts 1 and 2.** They are independent. Part 1 is `registry.hpp` and a new records header. Part 2 is `wire.hpp`'s event kind, `remote.hpp`'s writer and `post`, `backend.hpp` and the transports. -**Then part 3, then parts 4 and 5 side by side.** +**Then part 3,** after part 2, because both change `wire.hpp`. **Then parts 4 and 5 side by side, with part 6 +beside them.** - They share only the shapes part 3 defines. - Part 4 is in `remote.hpp`; part 5 is in the interpreter and the offline glue. - Part 5 tests its event path through the local backend's seam until part 4 lands. From 35039b01a34e71e358b92b8b2383c098b1a887cc Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Fri, 16 Oct 2026 19:41:38 +0200 Subject: [PATCH 05/68] =?UTF-8?q?docs:=20spec=208=20=E2=80=94=20address=20?= =?UTF-8?q?the=20fourth=20review=20round?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 --- .../specs/2026-10-16-server-push-design.md | 104 ++++++++++++------ 1 file changed, 70 insertions(+), 34 deletions(-) diff --git a/docs/superpowers/specs/2026-10-16-server-push-design.md b/docs/superpowers/specs/2026-10-16-server-push-design.md index 162363c42..89733aeeb 100644 --- a/docs/superpowers/specs/2026-10-16-server-push-design.md +++ b/docs/superpowers/specs/2026-10-16-server-push-design.md @@ -61,7 +61,7 @@ other client can see that working copy. | Fact | Where | |---|---| | The query cache is per client session, keyed by `(model type, instance, action, canonical body)`. `invalidates`, `refreshOn` and `{"refetch"}` mark entries stale. | spec 5 §5 "The query cache"; `include/morph/ui/interpreter.hpp` (the cache key) | -| A successful fetch clears an entry's `stale` mark unconditionally, and a refetch is skipped while a fetch is pending. | `include/morph/ui/interpreter.hpp` (`settleSuccess`, `refetchCurrent`) | +| A successful fetch clears an entry's `stale` mark unconditionally, and a refetch is skipped while a fetch is pending. | `include/morph/ui/interpreter.hpp` (`settleReply`, `refetchCurrent`) | | `refreshOn` matches by **model type, not instance**, because two private instances of a database-backed model read the same rows. | spec 5 §5 "Queries" | | An alias is `"private"`, keyless `"shared"` (a stateless model), or keyed by an instance expression. | spec 5 §5 "Models" | | `patch` merges a reply into a cached row found by its `id` field (`rowId`). `optimistic` applies the same shape before the reply. `latest` applies only the newest reply. | spec 5 §5 "Mutations"; `include/morph/ui/interpreter.hpp` | @@ -203,8 +203,9 @@ path `results`. - It recurses into members, depth-first, so a record inside a record (a row list inside a header) has a path such as `header.rows`. It stops at a registered record, so a record's own members are not searched for further records. It stops at a type already on its path, so recursive types end. -- **What it skips:** `void`, scalars, strings, and any type it cannot reflect, such as a type with a custom - `glz::meta` codec or an anonymous-namespace struct. A skipped type contributes no records. An action whose result +- **What it skips:** `void`, scalars, strings, and any type it cannot reflect: a type whose `glz::meta` replaces its + codec (`custom_read`/`custom_write`, or a `value` mapping), or an anonymous-namespace struct. A `glz::meta` that only + renames members is reflected, and its renamed keys are the paths. A skipped type contributes no records. An action whose result is skipped holds none. `updates` on such an action fails when the phase closes, because `R` is not found. - **The registrars are compiled out of client-only builds.** `MORPH_CLIENT_ONLY` empties every registrar, these included. That is consistent with a pure remote client taking these facts from `ui-catalog`. @@ -247,13 +248,20 @@ The wire gains its first server-initiated message. - **The outbound bound.** The server cannot see what a transport still holds unsent: replies and events are handed to the transport's outbox and forgotten. The design bounds what it can see, and names the transport's rule for the rest. - - **The writer reports its backlog.** `setConnectionWriter(cid, write)` takes a writer that returns the bytes - still queued for that connection after the write. Both transports already own that queue (the `net` connection's - frame queue, Qt's socket buffer), so the writer reads it. - - **Events, not replies, give way.** When the reported backlog exceeds 1 MiB, the server stops posting events to - that connection and records that it owes it a `resync`. When a later write reports the backlog under 256 KiB, it - sends one `resync` event, meaning "everything you watch may have changed". The client marks every watched entry - stale. Replies are never held back. + - **Each transport counts its backlog.** Writes are asynchronous: the `net` writer posts the frame to its loop, + and Qt queues it to the socket's thread. Neither queue can be read from the server strand. So each transport + keeps a per-connection atomic byte counter: + - incremented when its writer accepts a frame, a reply or an event; + - decremented on the loop or socket thread once the frame is written (the `net` `outbox` emptying, Qt's + `bytesWritten`). + + `setConnectionWriter(cid, write, backlog, onDrained)` hands the server the writer, a read of that counter, and a + drain callback that the transport calls when the counter falls below 256 KiB. + - **Events, not replies, give way.** When `backlog()` exceeds 1 MiB, the server stops posting events to that + connection and records that it owes it a `resync`. When the transport calls `onDrained`, the server sends one + `resync` event, meaning "everything you watch may have changed", and the client marks every watched entry stale. + The drain callback, not a later write, triggers it, so an idle client that only watches still receives it. + Replies are never held back. - **A peer that stops reading** is closed by the transport's own rule: the `net` writer closes a peer that stalls past `sendTimeout`, as it does today for replies. The Qt transport gains the same timeout in part 2, so both behave alike. @@ -277,9 +285,15 @@ A client watches the topics its cache holds, so the server sends each connection interpreter already binds a keyed alias as `bind(type, Shared, key)` from its token, so the key is known before the first fetch. The server resolves it through its instance directory (`DirectoryKey{typeId, primary}`). The binding's `currentId` cannot serve here, because it is 0 until that first fetch has bound the alias. + - **An instance watch is stored by key, not resolved.** The watch is sent before the batch's fetches, so the + instance may not exist on the server yet: no client has attached it. The server therefore stores the topic as + `(modelType, primary)` and never resolves it to an instance id. At fan-out, it matches the executing instance's + directory key, which every keyed instance holds from construction. An instance that never comes to exist simply + never matches. - A private instance needs no watch: its owner is known to the server (§8). A keyless `"shared"` alias names a stateless model, which has no instance state to change. -- **The kinds.** `watch` and `unwatch` are **core** kinds, handled by `RemoteServer` on its strand with the +- **The kinds.** The `watch` and `unwatch` wire kinds (unrelated to a document's `watch` declarations) are **core** + kinds, handled by `RemoteServer` on its strand with the connection's `cid`, because the watch table belongs to the connection and fan-out reads it in the execute path. - Core can validate them: after part 1, records are a core registry, and instance topics are checked with `authorizeInstance`. @@ -296,16 +310,27 @@ A client watches the topics its cache holds, so the server sends each connection - a write that commits before the fetch reads is in the fetched value; - one that commits after produces an event, which §9 keeps even if the fetch is still in flight. - **Reconnect.** Both transports reconnect on their own, so the UI layer cannot rely on being told by an offline - coordinator. The hosting layer (`ui/host.hpp`) installs the backend's `setConnectHandler`, which runs on every - successful connect, the first included. On each connect it: - 1. sends `ui-hello` again, so the new connection negotiates `events`; - 2. sends the whole watch set; - 3. marks every entry stale, as spec 6 §9 already does for a backend switch. - - These are ordinary requests on the new connection, so they go ahead of any execute sent after them. That - includes an offline replay, which reaches the cache as events (§8). In an application that also runs - `ReconnectCoordinator`, its `bindContext` step runs before `replay`, and the connect handler has already run by - then. Events missed while disconnected are not replayed; the stale marks cover them. + coordinator. + - **Who is told.** The remote `AppSource` (`ui/frontend.hpp`, spec 6) builds the backend, so it installs the + backend's `setConnectHandler`. That handler runs on every successful connect, the first included, on the + transport's thread. It only posts to the owner. On the owner, the source calls the session's new `reconnected()` + entry point, which `runApp` wires when it opens the source. + - The Qt WebSocket backend already fires the handler. + - The `net` `SocketBackend` does not override it today. Part 2 adds the override, fired from `onConnected()` on + its loop, the first connect included. + - **What `reconnected()` does,** on the owner: + 1. sends `ui-hello` again, so the new connection negotiates `events`; + 2. sends the whole watch set; + 3. marks every entry stale, as spec 6 §9 already does for a backend switch. Entries with a fetch in flight get + `refetchAfter`. + - **What it does not promise is order.** Both transports replay the requests queued during the outage as they + reconnect, before a handler posted to the owner can send anything. Executes filed during the outage therefore + reach the server **before** this `ui-hello` and watch set, and their changes may produce no event for this + client. Correctness rests on step 3 instead. Every entry is stale, and every fetch still in flight refetches once + it settles. The first fetches after the connect then run behind the watch set, so nothing committed before them + is missed, and anything committed after them produces an event. + - An offline replay through `ReconnectCoordinator` runs after this, and its changes reach the cache as events (§8). + Events missed while disconnected are not replayed; the stale marks cover them. - **Granularity.** Record topics are per type, not per key. Per-key watching would make the server track every key every client shows, and those keys change as lists scroll. The event carries keys, and the client filters on them (§9). @@ -392,7 +417,7 @@ With effects declared, a document needs no refresh declarations. - **refused: a query whose action declares effects** (anything other than nothing declared). A query must read, so a fetch never produces a change and two clients cannot refetch each other through their queries. It is new in what it refuses, so it applies only to actions that declare effects, which no existing action does. -- **warning, watch feedback:** a document `watch` whose `on` reads a query's value, whose `source` is `any` (the +- **warning, watch feedback:** a `watch` declaration (spec 5 §6, not the `watch` wire kind of §7) whose `on` reads a query's value, whose `source` is `any` (the default), and whose `do` runs a mutation with effects. Another client's change refetches the query, the watch fires, and the mutation's change reaches the other client, which may do the same. Such a watch should use `"source": "user"`. The loader cannot prove a ring, so this is a warning, and the claim above is limited to @@ -443,14 +468,17 @@ changes visible. `watch` envelope that created it, and the verified principal. - **A later envelope on the connection with a different verified principal drops all of the connection's watches.** The comparison is made in `stampVerifiedPrincipal`, given the envelope's `cid`. Every kind that - authorises passes through it; `hello`, which carries no session, does not. That covers signing in as someone else, and a client that keeps old watches across a sign-out. + authorises passes through it; `hello`, which carries no session, does not. + - **The baseline** is the verified principal of the first stamped envelope after the connection opens. An envelope + whose token fails `authenticate` is stamped with no principal, so it counts as different: the watches drop, and + the client sends them again with its next, freshly signed, envelope. That covers signing in as someone else, and a client that keeps old watches across a sign-out. The client's cache is already cleared on a principal change (spec 5 §5), and it watches again. - **Fan-out re-authorises each watch at send time, by the rule that admitted it.** - It runs `authenticate` on the stored context, which fails for an expired token. - For a record topic, it then requires again that the principal may run some action whose `records` hold the type, the same rule as below. It does not check the action that wrote the change: a role that may list results but not capture them still receives the change. - - For an instance topic, it runs `authorizeInstance`. + - For an instance topic, it re-runs the admission check for instance topics below. A failure drops the watch. The client watches again with its next envelope, which carries a fresh token. The "may run some action holding `R`" answer is cached per principal and record type, and refreshed when that @@ -458,7 +486,12 @@ changes visible. - **What a watch may name.** - A record type: only when the principal may run at least one action whose `records` contain it. Otherwise the `ok` reply lists it as refused, and the client logs it. - - An instance topic: only when `authorizeInstance` admits the principal to that instance. + - An instance topic: only when `authorizeInstance(ctx, modelType, {}, 0, "")` admits the principal. The instance + may not exist yet, so the call carries no instance id. A shared instance is recorded with an empty owner, so + ownership gives no protection here. What protects the topic is the same `authorize` rule as for an execute on + that model type: the principal must be allowed some action of `modelType`. The record rule above is applied in + the same way. An event on an instance topic carries no data, only "refetch", so it reveals no more than the + principal could learn by fetching. - **An event reveals that records of `R` changed, and their keys.** The keys are ids of records the receiver could list through some action returning `R`, but not necessarily the same ones. For a model that filters rows per principal, the record type is registered with `morph::record::privateKeys`. Its changes then reach principals @@ -487,10 +520,11 @@ Every test below must fail if the feature does nothing, so each names the change - **Channel (core, net, qt).** - One conformance case runs on both transports and on `SimulatedRemoteBackend`: a posted event reaches the handler. + - Both remote backends fire the connect handler on the first connect and on every reconnect. - A connection that did not offer `events` receives none. - An event is not taken for a reply. - - A writer reporting a backlog over 1 MiB holds events back, and exactly one `resync` follows when it drains. - Replies are not held back. + - A backlog over 1 MiB holds events back, and exactly one `resync` follows when the transport reports the drain, + on a connection that sends nothing in between. Replies are not held back. - *Mutation: route events through reply matching; the routing case fails.* - **Watching and fan-out (core).** - Two connections watch `R`, and an execute from A posts to both. @@ -499,7 +533,8 @@ Every test below must fail if the feature does nothing, so each names the change - A watch whose token has expired is dropped at send time. - A watch on a type the principal cannot read is refused. - A principal who may list `R` but not run the writer still receives the change. - - An instance topic named by primary key reaches a client whose alias has not fetched yet. + - An instance topic named by primary key, watched before any client attached the instance, is stored, and an + execute on the instance created later reaches the watcher. - A `changesInstance` on a private instance reaches only its owning connection. - An execute that times out at the client but succeeds still reaches the caller as an event. - *Mutation: skip re-authorisation; the principal case fails.* @@ -511,8 +546,9 @@ Every test below must fail if the feature does nothing, so each names the change - A submit (not a mutation) applies its effects. - A discarded `latest` reply marks entries stale and patches nothing, so a newer reply's row is kept. - A reply's `updates` patch on an entry whose fetch is pending causes one more fetch, in local mode. - - After the transport reconnects, the client sends `ui-hello` and its watch set again before any execute, and - events arrive on the new connection. + - After the transport reconnects, the client sends `ui-hello` and its watch set again before any execute filed + after the connect. Every entry is marked stale, and events arrive on the new connection. + - An entry whose fetch was queued during the outage and settles after the reconnect refetches once. - `changesInstance` refetches only entries of that instance. - *Mutation: restore the unconditional `stale = false`; the mid-flight case fails.* - **Documents (ui).** @@ -534,12 +570,12 @@ Each part updates the `docs/spec/` pages named in its row, and keeps every new p | # | Part | Layer | Needs | Spec pages | |---|---|---|---|---| | 1 | **Records and effects.** `BRIDGE_REGISTER_RECORD`, raw and wrapped keys, `keyless`, `derivedFrom`, `privateKeys`; `BRIDGE_REGISTER_EFFECTS` as a registrar; the result walker in the action registration; matching when the phase closes; `records(A)` with paths; deriving changes from a result | core | none | `core/registry.md` | -| 2 | **Server-to-client channel.** The `event` kind, `setConnectionWriter` with a backlog-reporting writer, `post`, holding events back and `resync`, both transports (and `sendTimeout` for the Qt one), `SimulatedRemoteBackend`, `setEventHandler`, the local backend's `deliverEvent` seam | core, net, qt | none | `core/wire.md`, `core/backend.md`, `security.md` | -| 3 | **Catalog and wire shapes.** `ActionInfo::effects` and `records` (key field, paths), local lookup and `ui-catalog`; `events` in `ui-hello`; the `watch`/`unwatch` and `changes` body shapes, defined here so parts 4 and 5 share them | ui, core (`wire.hpp` shapes) | 1, 2 | `ui/service.md`, `ui/document.md`, `core/wire.md` | +| 2 | **Server-to-client channel.** The `event` kind; `setConnectionWriter(cid, write, backlog, onDrained)` with a per-connection atomic byte counter in each transport; `post`; holding events back and `resync` on drain; `sendTimeout` for the Qt transport; `SocketBackend::setConnectHandler`, fired from `onConnected()`; `SimulatedRemoteBackend`; `setEventHandler`; the local backend's `deliverEvent` seam | core, net, qt | none | `core/wire.md`, `core/backend.md`, `security.md` | +| 3 | **Catalog and wire shapes.** `ActionInfo::effects` and `records` (key field, paths), local lookup and `ui-catalog`; `events` in `ui-hello`; the `watch`/`unwatch` and `changes` body shapes, defined here so parts 4 and 5 share them | ui, core (`wire.hpp` shapes) | 1, 2 | `ui/service.md`, `ui/document.md`, `core/wire.md`, spec 5 §15 (the push kind is core `event`, not `ui-event`) | | 4 | **Watching and fan-out.** Core `watch`/`unwatch` kinds in `RemoteServer` with the `cid`; the watch table with session contexts; the principal comparison in `stampVerifiedPrincipal`; fan-out in `dispatchExecute`, with private instances to their owner; `derivedFrom` expansion, coalescing, re-authorisation, `privateKeys` | core | 1, 2, 3 | `core/backend.md`, `core/shared_instances.md`, `security.md` | -| 5 | **Client cache.** `actionSucceeded` carries the reply and applies effects; patch by key field and path; `refetchAfter`, set by a stale mark or a patch on a fetching entry; discarded `latest` replies as marks; events and `resync`; topic reference counting; watch before fetch; the connect handler in `ui/host.hpp` that re-sends `ui-hello` and the watch set | ui | 2, 3 (4 for its remote tests) | `ui/interpreter.md`, `ui/host.md`, `offline/offline.md`, spec 5 §5 | +| 5 | **Client cache.** `actionSucceeded` carries the reply and applies effects; patch by key field and path; `refetchAfter`, set by a stale mark or a patch on a fetching entry; discarded `latest` replies as marks; events and `resync`; topic reference counting; watch before fetch; `Session::reconnected()`, and the remote `AppSource` installing the connect handler and posting to it | ui | 2, 3 (4 for its remote tests) | `ui/interpreter.md`, `ui/frontend.md`, `offline/offline.md`, spec 5 §5 | | 6 | **Document checks.** Query-must-read refused; watch-feedback, redundant and uncovered warnings | ui | 3 | `ui/document.md` | -| 7 | **End-to-end harness and examples.** The two-session harness in `tests/`. Records and effects in the seven rungs that use refresh declarations (`none` on attach-style actions), with their declarations removed in `ui/1` documents where the warnings say so | tests, examples | 5, 6 | the rungs' READMEs, `testing_strategy.md` | +| 7 | **End-to-end harness and examples.** The two-session harness in `tests/`. Records and effects in the seven rungs that use refresh declarations (`none` on attach-style actions), with their declarations removed in `ui/1` documents where the warnings say so | tests, examples | 4, 5, 6 | the rungs' READMEs, `testing_strategy.md` | | 8 | **`ui/2`.** Remove `invalidates`, `patch` and `refreshOn`; `optimistic` keeps its own shape; the authoring layer and golden files emit `ui/2`; the server keeps serving `ui/1` | ui | 7 | `ui/document.md`, `ui/author.md`, spec 5 §5 and §12 | | 9 | **MCP adapter.** A connection scope per agent session, tools from `permitted`, calls through `RemoteServer::handle` | net, ui | 4 | a new `ui/agent.md` | From 1c64354092fd05718cb5f4aaaccc5430a668bfd5 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Fri, 16 Oct 2026 19:49:09 +0200 Subject: [PATCH 06/68] =?UTF-8?q?docs:=20spec=208=20=E2=80=94=20address=20?= =?UTF-8?q?the=20fifth=20review=20round?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 --- .../specs/2026-10-16-server-push-design.md | 63 ++++++++++++++----- 1 file changed, 46 insertions(+), 17 deletions(-) diff --git a/docs/superpowers/specs/2026-10-16-server-push-design.md b/docs/superpowers/specs/2026-10-16-server-push-design.md index 89733aeeb..3b8ae2451 100644 --- a/docs/superpowers/specs/2026-10-16-server-push-design.md +++ b/docs/superpowers/specs/2026-10-16-server-push-design.md @@ -257,6 +257,10 @@ The wire gains its first server-initiated message. `setConnectionWriter(cid, write, backlog, onDrained)` hands the server the writer, a read of that counter, and a drain callback that the transport calls when the counter falls below 256 KiB. + - The counter counts the bytes of the encoded envelopes handed to the writer, and is decremented by the same + amount when that frame has been written. It never uses a transport's framed byte counts, so it cannot drift. + - `onDrained` runs on the transport's thread. It only posts to the server strand, where the owed `resync` is sent + and the flag cleared. - **Events, not replies, give way.** When `backlog()` exceeds 1 MiB, the server stops posting events to that connection and records that it owes it a `resync`. When the transport calls `onDrained`, the server sends one `resync` event, meaning "everything you watch may have changed", and the client marks every watched entry stale. @@ -288,7 +292,8 @@ A client watches the topics its cache holds, so the server sends each connection - **An instance watch is stored by key, not resolved.** The watch is sent before the batch's fetches, so the instance may not exist on the server yet: no client has attached it. The server therefore stores the topic as `(modelType, primary)` and never resolves it to an instance id. At fan-out, it matches the executing instance's - directory key, which every keyed instance holds from construction. An instance that never comes to exist simply + directory key (`Instance::sharedKey`), which a keyed instance holds from the moment it is keyed: at construction, + or when an `assign` promotes it. An instance that never comes to exist simply never matches. - A private instance needs no watch: its owner is known to the server (§8). A keyless `"shared"` alias names a stateless model, which has no instance state to change. @@ -297,7 +302,8 @@ A client watches the topics its cache holds, so the server sends each connection connection's `cid`, because the watch table belongs to the connection and fan-out reads it in the execute path. - Core can validate them: after part 1, records are a core registry, and instance topics are checked with `authorizeInstance`. - - The body is `{records: [...], instances: [{modelType, primary}]}`. The `ok` reply lists any topic refused + - The body is `{records: [...], instances: [{modelType, primary}], replace?}`. Without `replace` it adds topics. + `unwatch` removes them. The `ok` reply lists any topic refused (§12). - The UI layer sends them; it does not handle them. - **Lifetime.** A topic is watched while at least one cache entry holds it, mounted or kept (spec 5 §5: for @@ -311,24 +317,46 @@ A client watches the topics its cache holds, so the server sends each connection - one that commits after produces an event, which §9 keeps even if the fetch is still in flight. - **Reconnect.** Both transports reconnect on their own, so the UI layer cannot rely on being told by an offline coordinator. - - **Who is told.** The remote `AppSource` (`ui/frontend.hpp`, spec 6) builds the backend, so it installs the - backend's `setConnectHandler`. That handler runs on every successful connect, the first included, on the - transport's thread. It only posts to the owner. On the owner, the source calls the session's new `reconnected()` - entry point, which `runApp` wires when it opens the source. - - The Qt WebSocket backend already fires the handler. - - The `net` `SocketBackend` does not override it today. Part 2 adds the override, fired from `onConnected()` on - its loop, the first connect included. + - **Who is told: library seams, not a particular source.** In the library, `AppSource` (`ui/frontend.hpp`) is an + interface. The concrete remote source that builds the backend lives in the generic client + (`examples/common/app`), as does the shell that builds the `Session`. So the library names the seams, and a + source and a shell only connect them: + - **`AppSource::onConnected(listener)`**, a new virtual with an empty default for local sources. The source calls + every listener on the owner after each successful connect, the first included. A remote source implements it + by installing the backend's `setConnectHandler`, which runs on the transport's thread, and posting to the + owner. + - **The source owns the backend's single `setConnectHandler` slot.** Anything else that wants connects, such as a + connection-status indicator, subscribes through `onConnected`, so the slot is never taken twice. + - **`Session::reconnected()`**, new. The shell that builds the `Session` registers it with + `source.onConnected(...)`. That is one line in whichever shell builds the session; in this repository, the + generic client's shell. + - **The `watch` verb.** `Session` has no request channel of its own. Its `Dispatcher` already carries `evaluate`, + built over the source's exchange by `evaluatorOver(exchange, session, cbExec)`. It gains `watch`, built the same + way by a new `watcherOver(exchange, cbExec)`. + - The Qt WebSocket backend already fires `setConnectHandler`. The `net` `SocketBackend` does not override it + today; part 2 adds the override, fired from `onConnected()` on its loop, the first connect included. - **What `reconnected()` does,** on the owner: 1. sends `ui-hello` again, so the new connection negotiates `events`; 2. sends the whole watch set; 3. marks every entry stale, as spec 6 §9 already does for a backend switch. Entries with a fetch in flight get `refetchAfter`. - - **What it does not promise is order.** Both transports replay the requests queued during the outage as they - reconnect, before a handler posted to the owner can send anything. Executes filed during the outage therefore - reach the server **before** this `ui-hello` and watch set, and their changes may produce no event for this - client. Correctness rests on step 3 instead. Every entry is stale, and every fetch still in flight refetches once - it settles. The first fetches after the connect then run behind the watch set, so nothing committed before them - is missed, and anything committed after them produces an event. + - **What it does not promise is order, and why that is safe.** What the transports do today: + - An `execute` made while disconnected is rejected at once with `DisconnectedError`, and one in flight at the + drop is cancelled. No fetch is queued across an outage. + - `request()` envelopes made while disconnected (the `ui-*` kinds, and `watch`/`unwatch` changes) are held, and + replayed when the connection opens, before the posted handler can run. + - Executes the owner files **between the connect and `reconnected()` running** (a user's mutation, a + `refreshEvery` tick, calls released by the bridge's rebinds) reach the server before the new `ui-hello` and + watch set. A fetch among them may read before a change commits that produces no event for this client. + + Correctness rests on step 3. Every entry is stale, and an entry whose fetch is in flight gets `refetchAfter`. + Every refetch that follows is filed after the watch set, on the same ordered channel: `request()` and `execute()` + share one queue per connection on both transports. A change committed before such a refetch reads is in its + value, and one committed after produces an event. Nothing is missed for good. + - **The full watch set replaces.** The `watch` sent in step 2 carries `replace: true`: the connection's watches + become exactly that set. Watch changes held during the outage and replayed ahead of it are therefore superseded. + The server accepts `watch` before `ui-hello` on a connection; it stores the watches but posts events only once + `events` is negotiated. - An offline replay through `ReconnectCoordinator` runs after this, and its changes reach the cache as events (§8). Events missed while disconnected are not replayed; the stale marks cover them. - **Granularity.** Record topics are per type, not per key. Per-key watching would make the server track every key @@ -548,7 +576,8 @@ Every test below must fail if the feature does nothing, so each names the change - A reply's `updates` patch on an entry whose fetch is pending causes one more fetch, in local mode. - After the transport reconnects, the client sends `ui-hello` and its watch set again before any execute filed after the connect. Every entry is marked stale, and events arrive on the new connection. - - An entry whose fetch was queued during the outage and settles after the reconnect refetches once. + - A fetch filed after the connect but before `reconnected()` runs, and settling after it, refetches once. + - Watch changes held during the outage and replayed before the full set are superseded by it (`replace: true`). - `changesInstance` refetches only entries of that instance. - *Mutation: restore the unconditional `stale = false`; the mid-flight case fails.* - **Documents (ui).** @@ -573,7 +602,7 @@ Each part updates the `docs/spec/` pages named in its row, and keeps every new p | 2 | **Server-to-client channel.** The `event` kind; `setConnectionWriter(cid, write, backlog, onDrained)` with a per-connection atomic byte counter in each transport; `post`; holding events back and `resync` on drain; `sendTimeout` for the Qt transport; `SocketBackend::setConnectHandler`, fired from `onConnected()`; `SimulatedRemoteBackend`; `setEventHandler`; the local backend's `deliverEvent` seam | core, net, qt | none | `core/wire.md`, `core/backend.md`, `security.md` | | 3 | **Catalog and wire shapes.** `ActionInfo::effects` and `records` (key field, paths), local lookup and `ui-catalog`; `events` in `ui-hello`; the `watch`/`unwatch` and `changes` body shapes, defined here so parts 4 and 5 share them | ui, core (`wire.hpp` shapes) | 1, 2 | `ui/service.md`, `ui/document.md`, `core/wire.md`, spec 5 §15 (the push kind is core `event`, not `ui-event`) | | 4 | **Watching and fan-out.** Core `watch`/`unwatch` kinds in `RemoteServer` with the `cid`; the watch table with session contexts; the principal comparison in `stampVerifiedPrincipal`; fan-out in `dispatchExecute`, with private instances to their owner; `derivedFrom` expansion, coalescing, re-authorisation, `privateKeys` | core | 1, 2, 3 | `core/backend.md`, `core/shared_instances.md`, `security.md` | -| 5 | **Client cache.** `actionSucceeded` carries the reply and applies effects; patch by key field and path; `refetchAfter`, set by a stale mark or a patch on a fetching entry; discarded `latest` replies as marks; events and `resync`; topic reference counting; watch before fetch; `Session::reconnected()`, and the remote `AppSource` installing the connect handler and posting to it | ui | 2, 3 (4 for its remote tests) | `ui/interpreter.md`, `ui/frontend.md`, `offline/offline.md`, spec 5 §5 | +| 5 | **Client cache and reconnect seams.** `actionSucceeded` carries the reply and applies effects; patch by key field and path; `refetchAfter`, set by a stale mark or a patch on a fetching entry; discarded `latest` replies as marks; events and `resync`; topic reference counting; watch before fetch; `Session::reconnected()`; `AppSource::onConnected` with an empty default; `watcherOver` and `Dispatcher::watch`; the generic client's remote source and shell wired to them (`examples/common/app`) | ui, examples/common | 2, 3 (4 for its remote tests) | `ui/interpreter.md`, `ui/frontend.md`, `ui/service.md`, `offline/offline.md`, spec 5 §5 | | 6 | **Document checks.** Query-must-read refused; watch-feedback, redundant and uncovered warnings | ui | 3 | `ui/document.md` | | 7 | **End-to-end harness and examples.** The two-session harness in `tests/`. Records and effects in the seven rungs that use refresh declarations (`none` on attach-style actions), with their declarations removed in `ui/1` documents where the warnings say so | tests, examples | 4, 5, 6 | the rungs' READMEs, `testing_strategy.md` | | 8 | **`ui/2`.** Remove `invalidates`, `patch` and `refreshOn`; `optimistic` keeps its own shape; the authoring layer and golden files emit `ui/2`; the server keeps serving `ui/1` | ui | 7 | `ui/document.md`, `ui/author.md`, spec 5 §5 and §12 | From 229c22d94b0453cb862f1cd0aebea4458fe6c777 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Fri, 16 Oct 2026 19:56:47 +0200 Subject: [PATCH 07/68] =?UTF-8?q?docs:=20spec=208=20=E2=80=94=20address=20?= =?UTF-8?q?the=20sixth=20review=20round?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 --- .../specs/2026-10-16-server-push-design.md | 35 ++++++++++++------- 1 file changed, 22 insertions(+), 13 deletions(-) diff --git a/docs/superpowers/specs/2026-10-16-server-push-design.md b/docs/superpowers/specs/2026-10-16-server-push-design.md index 3b8ae2451..32bf34715 100644 --- a/docs/superpowers/specs/2026-10-16-server-push-design.md +++ b/docs/superpowers/specs/2026-10-16-server-push-design.md @@ -237,6 +237,10 @@ The wire gains its first server-initiated message. and spec 5 §15 names it as the place to negotiate this. A server sends events only on a connection whose `ui-hello` offered them and that it accepted. A client that never offers sees today's protocol unchanged. Negotiation is per connection, so a reconnect negotiates again. +- **How core learns the result.** `ui-hello` is a UI service kind, and `IEnvelopeService::handle` receives no + connection id. It gains a third parameter, `ConnectionFacts&`, with a default overload forwarding to today's + two-parameter form. `RemoteServer` passes the facts of the envelope's `cid`. The UI service sets + `facts.events = true` when it accepts the offer. Core reads the flag at fan-out. Core never parses a `ui-*` body. - **The envelope.** `kind: "event"`, `callId: 0`, `body: {topic, payload}`. For this design the topic is `changes`. A client routes it by kind before reply matching, so an event is never taken for an orphaned reply. - **Writing to a connection.** `RemoteServer` gains `setConnectionWriter(cid, write)`, the transport's writer for a @@ -330,15 +334,20 @@ A client watches the topics its cache holds, so the server sends each connection - **`Session::reconnected()`**, new. The shell that builds the `Session` registers it with `source.onConnected(...)`. That is one line in whichever shell builds the session; in this repository, the generic client's shell. - - **The `watch` verb.** `Session` has no request channel of its own. Its `Dispatcher` already carries `evaluate`, - built over the source's exchange by `evaluatorOver(exchange, session, cbExec)`. It gains `watch`, built the same - way by a new `watcherOver(exchange, cbExec)`. + - **The `watch` verb.** `Session` owns the cache, so it does the topic reference counting, and it needs its own + route to the server: `SessionOptions` gains a `watcher`, a functor `(body) -> Completion`. The + shell builds it with a new `watcherOver(exchange, currentSession, cbExec)`, alongside `evaluatorOver`. Unlike + the evaluator, whose session is fixed when it is built, it takes `currentSession`, a function returning the + source's session **now**. A `watch` is sent with whatever session the user is signed in with at that moment. + Requests through the exchange are not stamped with the backend's session, so an unsigned `watch` would be + refused (§12). `Dispatcher` is unchanged, so its test doubles need nothing. - The Qt WebSocket backend already fires `setConnectHandler`. The `net` `SocketBackend` does not override it today; part 2 adds the override, fired from `onConnected()` on its loop, the first connect included. - - **What `reconnected()` does,** on the owner: - 1. sends `ui-hello` again, so the new connection negotiates `events`; - 2. sends the whole watch set; - 3. marks every entry stale, as spec 6 §9 already does for a backend switch. Entries with a fetch in flight get + - **What happens on each connect,** on the owner: + 1. the remote source's own listener sends `ui-hello` again, so the new connection negotiates `events`, then + calls the other listeners, `Session::reconnected()` among them; + 2. `reconnected()` sends the whole watch set; + 3. `reconnected()` marks every entry stale, as spec 6 §9 already does for a backend switch. Entries with a fetch in flight get `refetchAfter`. - **What it does not promise is order, and why that is safe.** What the transports do today: - An `execute` made while disconnected is rejected at once with `DisconnectedError`, and one in flight at the @@ -346,7 +355,7 @@ A client watches the topics its cache holds, so the server sends each connection - `request()` envelopes made while disconnected (the `ui-*` kinds, and `watch`/`unwatch` changes) are held, and replayed when the connection opens, before the posted handler can run. - Executes the owner files **between the connect and `reconnected()` running** (a user's mutation, a - `refreshEvery` tick, calls released by the bridge's rebinds) reach the server before the new `ui-hello` and + `refreshEvery` tick, a timed replay of queued mutations, calls released by the bridge's rebinds) reach the server before the new `ui-hello` and watch set. A fetch among them may read before a change commits that produces no event for this client. Correctness rests on step 3. Every entry is stale, and an entry whose fetch is in flight gets `refetchAfter`. @@ -574,8 +583,8 @@ Every test below must fail if the feature does nothing, so each names the change - A submit (not a mutation) applies its effects. - A discarded `latest` reply marks entries stale and patches nothing, so a newer reply's row is kept. - A reply's `updates` patch on an entry whose fetch is pending causes one more fetch, in local mode. - - After the transport reconnects, the client sends `ui-hello` and its watch set again before any execute filed - after the connect. Every entry is marked stale, and events arrive on the new connection. + - After the transport reconnects, the client sends `ui-hello` and its watch set again before any refetch that + `reconnected()` files. Every entry is marked stale, and events arrive on the new connection. - A fetch filed after the connect but before `reconnected()` runs, and settling after it, refetches once. - Watch changes held during the outage and replayed before the full set are superseded by it (`replace: true`). - `changesInstance` refetches only entries of that instance. @@ -599,10 +608,10 @@ Each part updates the `docs/spec/` pages named in its row, and keeps every new p | # | Part | Layer | Needs | Spec pages | |---|---|---|---|---| | 1 | **Records and effects.** `BRIDGE_REGISTER_RECORD`, raw and wrapped keys, `keyless`, `derivedFrom`, `privateKeys`; `BRIDGE_REGISTER_EFFECTS` as a registrar; the result walker in the action registration; matching when the phase closes; `records(A)` with paths; deriving changes from a result | core | none | `core/registry.md` | -| 2 | **Server-to-client channel.** The `event` kind; `setConnectionWriter(cid, write, backlog, onDrained)` with a per-connection atomic byte counter in each transport; `post`; holding events back and `resync` on drain; `sendTimeout` for the Qt transport; `SocketBackend::setConnectHandler`, fired from `onConnected()`; `SimulatedRemoteBackend`; `setEventHandler`; the local backend's `deliverEvent` seam | core, net, qt | none | `core/wire.md`, `core/backend.md`, `security.md` | -| 3 | **Catalog and wire shapes.** `ActionInfo::effects` and `records` (key field, paths), local lookup and `ui-catalog`; `events` in `ui-hello`; the `watch`/`unwatch` and `changes` body shapes, defined here so parts 4 and 5 share them | ui, core (`wire.hpp` shapes) | 1, 2 | `ui/service.md`, `ui/document.md`, `core/wire.md`, spec 5 §15 (the push kind is core `event`, not `ui-event`) | +| 2 | **Server-to-client channel.** The `event` kind; `setConnectionWriter(cid, write, backlog, onDrained)` with a per-connection atomic byte counter in each transport; `post`; holding events back and `resync` on drain; `sendTimeout` for the Qt transport; `SocketBackend::setConnectHandler`, fired in `onConnected()` after the held requests are flushed and before the reconnect handler is posted; `SimulatedRemoteBackend`; `setEventHandler`; the local backend's `deliverEvent` seam | core, net, qt | none | `core/wire.md`, `core/backend.md`, `security.md` | +| 3 | **Catalog and wire shapes.** `ActionInfo::effects` and `records` (key field, paths), local lookup and `ui-catalog`; `events` in `ui-hello`, and `ConnectionFacts` on `IEnvelopeService::handle`; the `watch`/`unwatch` and `changes` body shapes, defined here so parts 4 and 5 share them | ui, core (`wire.hpp` shapes) | 1, 2 | `ui/service.md`, `ui/document.md`, `core/wire.md`, spec 5 §15 (the push kind is core `event`, not `ui-event`) | | 4 | **Watching and fan-out.** Core `watch`/`unwatch` kinds in `RemoteServer` with the `cid`; the watch table with session contexts; the principal comparison in `stampVerifiedPrincipal`; fan-out in `dispatchExecute`, with private instances to their owner; `derivedFrom` expansion, coalescing, re-authorisation, `privateKeys` | core | 1, 2, 3 | `core/backend.md`, `core/shared_instances.md`, `security.md` | -| 5 | **Client cache and reconnect seams.** `actionSucceeded` carries the reply and applies effects; patch by key field and path; `refetchAfter`, set by a stale mark or a patch on a fetching entry; discarded `latest` replies as marks; events and `resync`; topic reference counting; watch before fetch; `Session::reconnected()`; `AppSource::onConnected` with an empty default; `watcherOver` and `Dispatcher::watch`; the generic client's remote source and shell wired to them (`examples/common/app`) | ui, examples/common | 2, 3 (4 for its remote tests) | `ui/interpreter.md`, `ui/frontend.md`, `ui/service.md`, `offline/offline.md`, spec 5 §5 | +| 5 | **Client cache and reconnect seams.** `actionSucceeded` carries the reply and applies effects; patch by key field and path; `refetchAfter`, set by a stale mark or a patch on a fetching entry; discarded `latest` replies as marks; events and `resync`; topic reference counting; watch before fetch; `Session::reconnected()`; `AppSource::onConnected` with an empty default; `watcherOver` and `SessionOptions::watcher`; the generic client's remote source and shell wired to them (`examples/common/app`) | ui, examples/common | 2, 3 (4 for its remote tests) | `ui/interpreter.md`, `ui/frontend.md`, `ui/service.md`, `offline/offline.md`, spec 5 §5 | | 6 | **Document checks.** Query-must-read refused; watch-feedback, redundant and uncovered warnings | ui | 3 | `ui/document.md` | | 7 | **End-to-end harness and examples.** The two-session harness in `tests/`. Records and effects in the seven rungs that use refresh declarations (`none` on attach-style actions), with their declarations removed in `ui/1` documents where the warnings say so | tests, examples | 4, 5, 6 | the rungs' READMEs, `testing_strategy.md` | | 8 | **`ui/2`.** Remove `invalidates`, `patch` and `refreshOn`; `optimistic` keeps its own shape; the authoring layer and golden files emit `ui/2`; the server keeps serving `ui/1` | ui | 7 | `ui/document.md`, `ui/author.md`, spec 5 §5 and §12 | From e349152a39600a45558d0a0b549466beb600e394 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Fri, 16 Oct 2026 20:03:17 +0200 Subject: [PATCH 08/68] =?UTF-8?q?docs:=20spec=208=20=E2=80=94=20address=20?= =?UTF-8?q?the=20seventh=20review=20round's=20minor=20findings?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 --- .../specs/2026-10-16-server-push-design.md | 24 +++++++++++++------ 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/docs/superpowers/specs/2026-10-16-server-push-design.md b/docs/superpowers/specs/2026-10-16-server-push-design.md index 32bf34715..19a26ca7a 100644 --- a/docs/superpowers/specs/2026-10-16-server-push-design.md +++ b/docs/superpowers/specs/2026-10-16-server-push-design.md @@ -241,6 +241,11 @@ The wire gains its first server-initiated message. connection id. It gains a third parameter, `ConnectionFacts&`, with a default overload forwarding to today's two-parameter form. `RemoteServer` passes the facts of the envelope's `cid`. The UI service sets `facts.events = true` when it accepts the offer. Core reads the flag at fan-out. Core never parses a `ui-*` body. + - A message handled without a connection (`handle(msg, reply)`, `cid` 0) gets throwaway facts with + `events = false`. + - `IEnvelopeService` keeps its two-parameter `handle` pure. The three-parameter one is a non-pure virtual whose + default calls it. A service that overrides the new one adds `using IEnvelopeService::handle;`, as the + `-Woverloaded-virtual` build requires. - **The envelope.** `kind: "event"`, `callId: 0`, `body: {topic, payload}`. For this design the topic is `changes`. A client routes it by kind before reply matching, so an event is never taken for an orphaned reply. - **Writing to a connection.** `RemoteServer` gains `setConnectionWriter(cid, write)`, the transport's writer for a @@ -336,16 +341,21 @@ A client watches the topics its cache holds, so the server sends each connection generic client's shell. - **The `watch` verb.** `Session` owns the cache, so it does the topic reference counting, and it needs its own route to the server: `SessionOptions` gains a `watcher`, a functor `(body) -> Completion`. The - shell builds it with a new `watcherOver(exchange, currentSession, cbExec)`, alongside `evaluatorOver`. Unlike - the evaluator, whose session is fixed when it is built, it takes `currentSession`, a function returning the - source's session **now**. A `watch` is sent with whatever session the user is signed in with at that moment. + remote source builds it with a new `watcherOver(exchange, currentSession, cbExec)`, as it builds the evaluator, + because the source holds the exchange. The shell sets `sessionOptions.watcher = source.watcher()`. The watcher + takes `currentSession`, a function returning the source's session **now**, rather than a session fixed when it + is built. + - When `SessionOptions::watcher` is unset, as for a local source with no `RemoteServer`, the session still + counts topics and sends nothing. Local mode is covered by the effects every reply applies (§9). A `watch` is sent with whatever session the user is signed in with at that moment. Requests through the exchange are not stamped with the backend's session, so an unsigned `watch` would be refused (§12). `Dispatcher` is unchanged, so its test doubles need nothing. - The Qt WebSocket backend already fires `setConnectHandler`. The `net` `SocketBackend` does not override it today; part 2 adds the override, fired from `onConnected()` on its loop, the first connect included. - **What happens on each connect,** on the owner: - 1. the remote source's own listener sends `ui-hello` again, so the new connection negotiates `events`, then - calls the other listeners, `Session::reconnected()` among them; + 1. on a **reconnect** (not the first connect, whose `ui-hello` `open()` has already filed), the remote source's + own listener sends `ui-hello` again, so the new connection negotiates `events`, then calls the other + listeners, `Session::reconnected()` among them. It sends only the `ui-hello` handshake, not a bundle reload: + the documents did not change because the connection dropped; 2. `reconnected()` sends the whole watch set; 3. `reconnected()` marks every entry stale, as spec 6 §9 already does for a backend switch. Entries with a fetch in flight get `refetchAfter`. @@ -609,7 +619,7 @@ Each part updates the `docs/spec/` pages named in its row, and keeps every new p |---|---|---|---|---| | 1 | **Records and effects.** `BRIDGE_REGISTER_RECORD`, raw and wrapped keys, `keyless`, `derivedFrom`, `privateKeys`; `BRIDGE_REGISTER_EFFECTS` as a registrar; the result walker in the action registration; matching when the phase closes; `records(A)` with paths; deriving changes from a result | core | none | `core/registry.md` | | 2 | **Server-to-client channel.** The `event` kind; `setConnectionWriter(cid, write, backlog, onDrained)` with a per-connection atomic byte counter in each transport; `post`; holding events back and `resync` on drain; `sendTimeout` for the Qt transport; `SocketBackend::setConnectHandler`, fired in `onConnected()` after the held requests are flushed and before the reconnect handler is posted; `SimulatedRemoteBackend`; `setEventHandler`; the local backend's `deliverEvent` seam | core, net, qt | none | `core/wire.md`, `core/backend.md`, `security.md` | -| 3 | **Catalog and wire shapes.** `ActionInfo::effects` and `records` (key field, paths), local lookup and `ui-catalog`; `events` in `ui-hello`, and `ConnectionFacts` on `IEnvelopeService::handle`; the `watch`/`unwatch` and `changes` body shapes, defined here so parts 4 and 5 share them | ui, core (`wire.hpp` shapes) | 1, 2 | `ui/service.md`, `ui/document.md`, `core/wire.md`, spec 5 §15 (the push kind is core `event`, not `ui-event`) | +| 3 | **Catalog and wire shapes.** `ActionInfo::effects` and `records` (key field, paths), local lookup and `ui-catalog`; `events` in `ui-hello`, and `ConnectionFacts` on `IEnvelopeService::handle`; the `watch`/`unwatch` and `changes` body shapes, defined here so parts 4 and 5 share them | ui, core (`wire.hpp` shapes, `remote.hpp` `ConnectionFacts`) | 1, 2 | `ui/service.md`, `ui/document.md`, `core/wire.md`, spec 5 §15 (the push kind is core `event`, not `ui-event`) | | 4 | **Watching and fan-out.** Core `watch`/`unwatch` kinds in `RemoteServer` with the `cid`; the watch table with session contexts; the principal comparison in `stampVerifiedPrincipal`; fan-out in `dispatchExecute`, with private instances to their owner; `derivedFrom` expansion, coalescing, re-authorisation, `privateKeys` | core | 1, 2, 3 | `core/backend.md`, `core/shared_instances.md`, `security.md` | | 5 | **Client cache and reconnect seams.** `actionSucceeded` carries the reply and applies effects; patch by key field and path; `refetchAfter`, set by a stale mark or a patch on a fetching entry; discarded `latest` replies as marks; events and `resync`; topic reference counting; watch before fetch; `Session::reconnected()`; `AppSource::onConnected` with an empty default; `watcherOver` and `SessionOptions::watcher`; the generic client's remote source and shell wired to them (`examples/common/app`) | ui, examples/common | 2, 3 (4 for its remote tests) | `ui/interpreter.md`, `ui/frontend.md`, `ui/service.md`, `offline/offline.md`, spec 5 §5 | | 6 | **Document checks.** Query-must-read refused; watch-feedback, redundant and uncovered warnings | ui | 3 | `ui/document.md` | @@ -620,7 +630,7 @@ Each part updates the `docs/spec/` pages named in its row, and keeps every new p **Ready now: parts 1 and 2.** They are independent. Part 1 is `registry.hpp` and a new records header. Part 2 is `wire.hpp`'s event kind, `remote.hpp`'s writer and `post`, `backend.hpp` and the transports. -**Then part 3,** after part 2, because both change `wire.hpp`. **Then parts 4 and 5 side by side, with part 6 +**Then part 3,** after part 2, because both change `wire.hpp` and `remote.hpp`. **Then parts 4 and 5 side by side, with part 6 beside them.** - They share only the shapes part 3 defines. - Part 4 is in `remote.hpp`; part 5 is in the interpreter and the offline glue. From 01ff972a1f04553a3d6947bf76a115b4354ce29c Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 09:44:41 +0200 Subject: [PATCH 09/68] docs: spec 8 follows master's three-rung ladder and explorer; one PR, with its implementation plan The examples the spec cites are bookmarks, bank and lims; ledger's derived report becomes bank's SpendingReport. Matching runs on the first registry read, since a release build's latch closes only on request. The explorer shows effects and records. The MCP adapter's shape is fixed. Delivery is one PR in nine ordered parts, planned in docs/superpowers/plans. Co-Authored-By: Claude Opus 5.5 --- .../plans/2026-10-18-server-push.md | 768 ++++++++++++++++++ .../specs/2026-10-16-server-push-design.md | 75 +- 2 files changed, 824 insertions(+), 19 deletions(-) create mode 100644 docs/superpowers/plans/2026-10-18-server-push.md diff --git a/docs/superpowers/plans/2026-10-18-server-push.md b/docs/superpowers/plans/2026-10-18-server-push.md new file mode 100644 index 000000000..3a26e1c43 --- /dev/null +++ b/docs/superpowers/plans/2026-10-18-server-push.md @@ -0,0 +1,768 @@ +# Server push (spec 8) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Actions declare once what they change. Every reply and every server event updates the client cache from +those declarations. Documents stop declaring refreshes, and an agent's calls through the server update every open +GUI. All of it lands in PR #990. + +**Architecture:** Two core registries (records, effects) are matched lazily on first read and give each action its +`effects` and its `records` with key fields and paths. `RemoteServer` gains per-connection writers, a watch table +keyed by connection, and fan-out in the execute path. The UI interpreter's cache applies a change the same way +whether it came from a reply (`actionSucceeded`) or from an `event`. The loader checks documents against effects, +and `ui/2` removes the refresh keys. A transport-free MCP adapter drives the server through an ordinary connection +scope. + +**Tech Stack:** C++23, glaze (JSON and reflection), Catch2 v3, CMake + Ninja; `morph::net` (POSIX) and Qt6 +WebSockets transports. + +**Spec:** `docs/superpowers/specs/2026-10-16-server-push-design.md` (section numbers below, such as §7, are the +spec's). + +## Global Constraints + +- Every part keeps the branch building, with `morph_tests`, `morph_net_tests`, `morph_qt_tests` and the three + rungs' unit and GUI tests green. Commit at the end of each task, never with a red suite. +- Every new public symbol has complete Doxygen (`@param`/`@tparam`/`@return`). The Docs workflow runs with + `WARN_AS_ERROR = FAIL_ON_WARNINGS`. +- Comments and `docs/` state what the code does now and why: no history, no issue numbers, no commit hashes + (AGENTS.md). +- Before changing a public type or subsystem, read its `docs/spec/` page, and update that page in the same task. +- Numbers from the spec, verbatim: backlog hold-back above **1 MiB** (`1 << 20`), drain below **256 KiB** + (`256 << 10`); coalescing window **50 ms** or **256 keys**; watch lifetime follows the cache's `keepUnused` + (**5 minutes**); MCP unknown method is JSON-RPC error **`-32601`**. +- Wire names, verbatim: envelope kind `event` (`callId: 0`, body `{topic, payload}`), topics `changes` and + `resync`; core kinds `watch` and `unwatch`, body `{records, instances: [{modelType, primary}], replace?}`; + `ui-hello` gains `events`; `ActionInfo` gains `effects: [{kind, record?}]` with kind + `updates | changes | changesInstance | none`, and `records: [{record, keyField?, paths}]`. +- `MORPH_CLIENT_ONLY` builds compile out every new registrar, as they do the action registrars. +- `ActionTraits` is not changed. No new specialisation of a trait that `BRIDGE_REGISTER_ACTION` defines. +- Do not run clang-tidy locally (it is too slow here); fix its findings from CI's clang-tidy-diff log. +- Run one local build at a time; Ninja already uses every core. + +## Review Focus + +1. **A result that holds a record inside `std::optional` or a `std::map` value**, such as lims's optional nested + views. The walker must give a path (`header.rows`), and a reply patch must find rows there. Pinned in Task 2 + (`walker_paths_through_optional_and_map`) and Task 19 (`reply_patch_at_nested_path`). +2. **Sign-out, or a token that expires, while watches are live.** No event may reach the connection under the old + principal. Pinned in Task 15 (`unauthenticated_envelope_drops_watches`) and Task 16 + (`expired_watch_dropped_at_send`). +3. **A screen unmounted while its entry's fetch is in flight and an event arrives.** The entry is kept (unused), + marked stale, and not refetched until something mounts it again. Pinned in Task 20 + (`event_on_unmounted_entry_marks_only`). +4. **A slow reader during a write burst.** Replies keep flowing, events stop, and exactly one `resync` follows the + drain. Pinned in Task 7 (`backlog_holds_events_not_replies`) and Task 17 (`burst_over_backlog_yields_one_resync`). +5. **Two clients whose documents both `watch` a query and run a mutation with effects.** The loader warns, and the + example rungs contain no such watch. Pinned in Task 23 (`watch_feedback_warns`) and Task 25 (no warnings + from loading each rung's bundle). + +## Build and run + +```bash +cmake -S . -B build/push -G Ninja -DCMAKE_BUILD_TYPE=Debug -DMORPH_BUILD_TESTS=ON -DMORPH_BUILD_NET=ON \ + -DMORPH_BUILD_QT=ON -DMORPH_BUILD_LADDER=ON -DMORPH_BUILD_BANK_EXAMPLE=ON +cmake --build build/push +build/push/tests/morph_tests "[push]" # this feature's core and ui cases +build/push/tests/net/morph_net_tests "[push]" +ctest --test-dir build/push --output-on-failure # whole suite, at the end of each part +``` + +Every new Catch2 case is tagged `[push]` plus its part's tag: `[push][records]`, `[push][channel]`, +`[push][catalog]`, `[push][watch]`, `[push][cache]`, `[push][documents]`, `[push][explorer]`, `[push][e2e]`, +`[push][ui2]`, `[push][agent]`. New `tests/*.cpp` files are added to `morph_tests` in `tests/CMakeLists.txt`, and +`tests/net/*.cpp` files to `morph_net_tests`. + +## File structure + +| File | Responsibility | +|---|---| +| `include/morph/core/records.hpp` (new) | `BRIDGE_REGISTER_RECORD`, `morph::record::{key, keyless, derivedFrom, privateKeys}`, the record registry | +| `include/morph/core/detail/result_walker.hpp` (new) | Reflection walk of `ActionTraits::Result` into `{type, path}` slots | +| `include/morph/core/effects.hpp` (new) | `BRIDGE_REGISTER_EFFECTS`, `morph::effects::*`, the effects registry, lazy matching, `effectsOf`, `recordsOf`, `EffectsRegistrationError` | +| `include/morph/core/changes.hpp` (new) | `Changes` and `deriveChanges`, from an action's result JSON | +| `include/morph/core/wire.hpp` | `event` kind, `ChangesBody`, `WatchBody`, `WatchReply` | +| `include/morph/core/remote.hpp` | `ConnectionFacts`, `ConnectionWriter`, `setConnectionWriter`, `post`, `watch`/`unwatch`, fan-out, principal baseline | +| `include/morph/core/detail/watch_table.hpp` (new) | Per-connection watches with session contexts; matching a change to recipients | +| `include/morph/core/detail/change_coalescer.hpp` (new) | Per-connection 50 ms / 256-key merge | +| `include/morph/core/backend.hpp` | `IBackend::setEventHandler`, `LocalBackend::deliverEvent` | +| `include/morph/net/detail/ws_connection.hpp`, `include/morph/net/socket_server.hpp`, `include/morph/net/socket_backend.hpp` | byte counter, drain callback, event routing, `setConnectHandler` | +| `include/morph/qt/qt_websocket_server.hpp`, `src/qt/qt_websocket_server.cpp`, `src/qt/qt_websocket_backend.cpp` | the same for Qt, plus `sendTimeout` | +| `include/morph/ui/document.hpp` | `ActionInfo::effects`, `ActionInfo::records`, `Document::warnings`, the four checks, `ui/2` | +| `include/morph/ui/bundle.hpp`, `include/morph/ui/service.hpp` | `events` in `ui-hello`; the UI service sets `ConnectionFacts::events` | +| `include/morph/ui/author.hpp` | `CatalogFacts::action` fills effects and records; emits `ui/2` | +| `include/morph/ui/interpreter.hpp` | `refetchAfter`, effects on replies, events, topic counting, `watcher`, `reconnected()` | +| `include/morph/ui/service_client.hpp` | `watcherOver` | +| `include/morph/ui/frontend.hpp` | `AppSource::onConnected` | +| `include/morph/ui/explorer.hpp` | effects and records rows | +| `include/morph/ui/agent.hpp` (new) | `McpAdapter`, `serveStdio` | +| `examples/common/app/app_source.*`, `client_shell.*` | wiring of the seams | +| `examples/{bookmarks,bank,lims}` | records, effects, documents without refresh declarations | + +--- + +## Part 1: Records and effects (core) + +### Task 1: The record registry + +**Files:** +- Create: `include/morph/core/records.hpp` +- Test: `tests/test_push_records.cpp` + +**Interfaces:** +- Produces: + - `namespace morph::record`: `template struct key {};` `struct keyless {};` + `template struct derivedFrom {};` `struct privateKeys {};` + - `struct morph::record::RecordInfo { std::string name; std::type_index type; std::optional keyField; std::vector derivedFrom; bool privateKeys = false; bool keyIsString = false; };` + - `morph::record::detail::registerRecordOnce(std::string_view name) noexcept -> bool`, asserting the + registration latch like `registerActionOnce`. + - `morph::record::detail::recordRegistry() -> std::vector const&`, raw and unvalidated (Task 3 + validates it). + - `BRIDGE_REGISTER_RECORD(T, NAME, ...)`, empty under `MORPH_CLIENT_ONLY`. + +- [ ] **Step 1: Write the failing tests** (`[push][records]`) + - `record_default_key_is_id`: register `struct Row { std::int64_t id; std::string name; }` as `"T_Row"`. Its + `RecordInfo` has `keyField == "id"`. + - `record_named_key_member`: `key<&Row2::code>` gives `keyField == "code"`, the glaze JSON key. + - `record_wrapped_key`: a member of a strong-id type that satisfies `WrappedModelKey` is accepted, and + `keyIsString` follows its raw type. + - `record_keyless_and_derived`: `keyless, derivedFrom` gives no `keyField` and `derivedFrom == {typeid(Row)}`. + - `record_private_keys`: the flag is set. +- [ ] **Step 2: Run** `build/push/tests/morph_tests "[records]"`. Expected: compile failure (header missing). +- [ ] **Step 3: Implement** `records.hpp`. The default key is the member reflected under the name `id` + (`glz::reflect::keys`). A type with no `id`, no `key<>` and no `keyless` is recorded with no key field. Task 3 + rejects that only when `updates` names it. Static registrars follow `MORPH_DETAIL_REGISTER_ACTION_LOCAL`. +- [ ] **Step 4: Run** the tests. Expected: PASS. +- [ ] **Step 5: Commit** `core: a registry of record types, with keys, derivation and private keys` + +### Task 2: The result walker + +**Files:** +- Create: `include/morph/core/detail/result_walker.hpp` +- Modify: `include/morph/core/registry.hpp` (`BRIDGE_DETAIL_ACTION_REGISTRARS` gains a third registrar) +- Test: `tests/test_push_records.cpp` + +**Interfaces:** +- Produces: + - `struct morph::model::detail::ResultSlot { std::type_index type; std::string path; };` + - `template std::vector walkResult();`. Every visited aggregate is a slot, so matching can + later stop at registered records. Walking stops at a type already on the current path. + - `morph::model::detail::registerActionResultOnce(std::string_view actionId) noexcept -> bool` stores + `walkResult::Result>()` by action id. `morph::model::detail::actionResultSlots()` gives the + map. + +- [ ] **Step 1: Write the failing tests** + - `walker_result_itself`: for `Row`, a slot `{Row, ""}`. + - `walker_vector_member`: for `struct ListResult { std::vector rows; }`, a slot `{Row, "rows"}`. + - `walker_paths_through_optional_and_map`: `std::optional
header` whose `Header` holds `std::vector + rows` gives `{Row, "header.rows"}`. A `std::map` member gives `{Row, "byName"}`. + - `walker_skips_custom_codec`: a type with `glz::meta` `custom_read` gives no slot beneath it. + - `walker_ends_on_recursive_type`: `struct Node { std::vector children; }` terminates. + - `walker_renamed_member_uses_wire_name`: a `glz::meta` rename gives the renamed path. + - `action_registration_records_result_slots`: a test action registered with `BRIDGE_REGISTER_ACTION` has its + slots in `actionResultSlots()` under its id. *Mutation: remove the registrar line; this case fails.* +- [ ] **Step 2: Run** `"[records]"`. Expected: FAIL. +- [ ] **Step 3: Implement** with `glz::reflect` over reflectable aggregates. Paths join with `.`; vector, optional + and map values add no segment. Add the registrar to `BRIDGE_DETAIL_ACTION_REGISTRARS` through a new + `MORPH_DETAIL_REGISTER_RESULT_LOCAL(A)`, which is empty under `MORPH_CLIENT_ONLY`. +- [ ] **Step 4: Run** `"[records]"`, then the whole `morph_tests`, because every action registration now runs the + walker. Expected: PASS. +- [ ] **Step 5: Commit** `core: an action's registration records where its result holds which types` + +### Task 3: Effects, matching and validation + +**Files:** +- Create: `include/morph/core/effects.hpp` +- Modify: `docs/spec/core/registry.md` (new sections "Records" and "Effects") +- Create: `tests/compile_checks/effects_unseen_a.cpp`, `tests/compile_checks/effects_unseen_b.cpp` (one TU sees + `BRIDGE_REGISTER_EFFECTS`, the other does not; linked into one executable `morph_effects_unseen_links`) +- Test: `tests/test_push_effects.cpp` + +**Interfaces:** +- Consumes: Tasks 1 and 2. +- Produces: + - `namespace morph::effects`: `template struct updates {};` `template struct changes {};` + `struct changesInstance {};` `struct none {};` + - `enum class morph::effects::Kind : std::uint8_t { Updates, Changes, ChangesInstance, None };` + - `struct morph::effects::Effect { Kind kind; std::string record; };` (`record` is empty for the instance kinds) + - `struct morph::effects::ActionRecord { std::string record; std::optional keyField; std::vector paths; };` + - `[[nodiscard]] std::optional> morph::effects::effectsOf(std::string_view actionId);` + (`nullopt` means nothing is declared: the action reads) + - `[[nodiscard]] std::vector morph::effects::recordsOf(std::string_view actionId);` + - `[[nodiscard]] morph::record::RecordInfo const* morph::effects::recordNamed(std::string_view name);` + - `struct morph::model::EffectsRegistrationError : std::logic_error` + - `BRIDGE_REGISTER_EFFECTS(A, ...)` + +- [ ] **Step 1: Write the failing tests** + - `records_of_list_action`: an action returning `ListResult` gives + `recordsOf == {{"T_Row", "id", {"rows"}}}`. + - `registration_order_irrelevant`: records and effects registered in two TUs, with the record's TU after the + action's (`tests/push_records_late.cpp` linked into `morph_tests`), give the same `recordsOf`. + - `effects_twice_same_is_noop` and `effects_conflict_throws`: the conflict throws `EffectsRegistrationError` on + the first `effectsOf`, and its `what()` names the action. + - `updates_without_record_in_result_throws`, `updates_on_keyless_throws`, `derived_cycle_throws`, + `duplicate_record_name_throws`: each throws, and one registry holding all four lists all four in `what()`. + - `unregistered_action_reads`: `effectsOf("NoSuch")` is `nullopt`. + - Validation tests run against a registry built in the test (`detail::resolveForTesting(records, slots, effects)`), + so the process registry stays valid for the rest of the suite. +- [ ] **Step 2: Run** `"[push][records]"`. Expected: FAIL. +- [ ] **Step 3: Implement.** `effectsOf`, `recordsOf` and `recordNamed` call `detail::resolved()`, which uses + `std::call_once` and calls `noteRegistryRead(true)`. It matches every action's slots against registered records, + stopping at the first registered record on each path, and caches the result or the error. The error is rethrown + on every later read. Registrars store `{kind, std::type_index}` and are idempotent per action, as + `registerActionOnce` is. +- [ ] **Step 4: Run** the tests and build `morph_effects_unseen_links`. Expected: PASS, links. +- [ ] **Step 5: Commit** `core: actions declare their effects; matched and validated on first read` + +### Task 4: Deriving changes from a result + +**Files:** +- Create: `include/morph/core/changes.hpp` +- Test: `tests/test_push_effects.cpp` + +**Interfaces:** +- Consumes: Task 3. +- Produces: + - `struct morph::effects::RecordChange { std::string record; std::optional> keys; };` + (`keys` absent means the whole type changed) + - `struct morph::effects::InstanceChange { std::string modelType; std::uint64_t instance = 0; std::string primary; };` + (`primary` is the directory key's text for a keyed shared instance, empty for a private one) + - `struct morph::effects::Changes { std::vector records; std::vector instances; [[nodiscard]] bool empty() const; };` + - `[[nodiscard]] Changes morph::effects::deriveChanges(std::string_view actionId, std::string_view resultJson, InstanceChange const& ranOn);` + +- [ ] **Step 1: Write the failing tests** + - `derive_updates_reads_keys_at_paths`: `updates` over `{"rows":[{"id":1},{"id":2}]}` gives + `{"T_Row", [1, 2]}`. + - `derive_changes_has_no_keys`. + - `derive_expands_derived_from`: `changes` also yields the keyless `{"T_Report"}` derived from it. + - `derive_changes_instance_uses_ran_on`. + - `derive_reads_nothing_for_reader`: an undeclared action gives `empty()`. + - *Mutation: return no keys from `updates`; the first case fails.* +- [ ] **Step 2: Run** `"[push][records]"`. Expected: FAIL. +- [ ] **Step 3: Implement** by parsing `resultJson` with `glz::generic` and walking each `ActionRecord` path; the + key value is copied as raw JSON. +- [ ] **Step 4: Run.** Expected: PASS. +- [ ] **Step 5: Commit** `core: derive an execute's changes from its effects and its result` + +--- + +## Part 2: The server-to-client channel + +### Task 5: The `event` kind and client-side routing + +**Files:** +- Modify: `include/morph/core/wire.hpp`, `include/morph/core/backend.hpp`, `include/morph/core/detail/reply_router.hpp` +- Modify: `docs/spec/core/wire.md` +- Test: `tests/test_push_channel.cpp` + +**Interfaces:** +- Produces: + - `inline constexpr std::string_view morph::wire::kEventKind = "event";` + - `[[nodiscard]] Envelope morph::wire::makeEvent(std::string_view topic, std::string payloadJson);` + (`callId` 0, body `{"topic":…, "payload":…}`) + - `struct morph::wire::EventBody { std::string topic; glz::raw_json payload; };` + - `virtual void IBackend::setEventHandler(std::function handler, exec::IExecutor* exec);` + (default: ignore). Delivered on @p exec. + - `void LocalBackend::deliverEvent(wire::EventBody event);` (test seam) +- [ ] **Step 1: Write the failing tests** (`[push][channel]`) + - `reply_router_routes_event_before_call_match`: a router with a pending `callId 0`, given an event envelope, + calls the event handler and leaves the pending call unresolved. *Mutation: route events through reply + matching; this case fails.* + - `local_backend_deliver_event_reaches_handler_on_exec`. +- [ ] **Step 2: Run** `"[channel]"`. Expected: FAIL. +- [ ] **Step 3: Implement.** +- [ ] **Step 4: Run.** Expected: PASS. +- [ ] **Step 5: Commit** `core: a server-initiated event kind, routed before reply matching` + +### Task 6: `RemoteServer` writers, `post` and the backlog + +**Files:** +- Modify: `include/morph/core/remote.hpp`, `docs/spec/core/backend.md`, `docs/spec/security.md` +- Test: `tests/test_push_channel.cpp` + +**Interfaces:** +- Produces: + - `struct morph::backend::ConnectionWriter { std::function write; std::function backlog; };` + - `void RemoteServer::setConnectionWriter(ConnectionId cid, ConnectionWriter writer);` and + `void RemoteServer::connectionDrained(ConnectionId cid);` (what the transport's `onDrained` calls; it posts to + the strand) + - `void RemoteServer::post(ConnectionId cid, std::string envelope);` (any thread; above the size cap it is + dropped and counted) + - `inline constexpr std::size_t kEventBacklogHigh = std::size_t{1} << 20;` + `inline constexpr std::size_t kEventBacklogLow = std::size_t{256} << 10;` + - The writer is dropped in `closeConnectionHere`. A throwing `write` drops the connection's watches (Part 4 adds + those). +- [ ] **Step 1: Write the failing tests** + - `post_reaches_connection_writer`. + - `backlog_holds_events_not_replies`: a writer whose `backlog()` reads 2 MiB receives replies to executes and no + posted event. After `connectionDrained`, it receives exactly one `resync` event, with nothing else sent in + between. + - `post_after_close_is_dropped`. +- [ ] **Step 2: Run** `"[channel]"`. Expected: FAIL. +- [ ] **Step 3: Implement.** `post` hops to the server strand, checks `backlog() > kEventBacklogHigh`, and either + writes or sets `owesResync`. `connectionDrained` sends `makeEvent("resync", "{}")` when `owesResync` is set. +- [ ] **Step 4: Run.** Expected: PASS. +- [ ] **Step 5: Commit** `core: RemoteServer writes events to a connection, and holds them back from a slow reader` + +### Task 7: The `net` transport + +**Files:** +- Modify: `include/morph/net/detail/ws_connection.hpp`, `include/morph/net/socket_server.hpp`, `include/morph/net/socket_backend.hpp` +- Test: `tests/net/test_push_net.cpp` + +**Interfaces:** +- Produces: + - `LoopConnection` gains `std::atomic pending` and `std::function onDrained`. The amount + added is the size of the encoded envelope. The writer subtracts the same amount after the frame's last chunk is + written. `onDrained` fires when `pending` falls below `kEventBacklogLow` after having been at or above it. + - `SocketServer`'s `accepted` calls `server->setConnectionWriter(cid, …)`, and wires `onDrained` to + `server->connectionDrained(cid)`. + - `SocketBackend::setConnectHandler` override, fired in `onConnected()` after held requests are flushed and before + the reconnect handler is posted, the first connect included. `setEventHandler` override. +- [ ] **Step 1: Write the failing tests** (`[push][channel]`) + - `net_posted_event_reaches_backend_handler`. + - `net_connect_handler_fires_on_first_and_reconnect`: the server is restarted on the same port, and the handler + has fired twice. + - `net_counter_returns_to_zero`: after 100 replies are written, `pending == 0`. +- [ ] **Step 2: Run** `build/push/tests/net/morph_net_tests "[push]"`. Expected: FAIL. +- [ ] **Step 3: Implement.** +- [ ] **Step 4: Run.** Expected: PASS. +- [ ] **Step 5: Commit** `net: per-connection byte backlog, drain callback, events and the connect handler` + +### Task 8: The Qt transport + +**Files:** +- Modify: `include/morph/qt/qt_websocket_server.hpp`, `src/qt/qt_websocket_server.cpp`, `src/qt/qt_websocket_backend.cpp`, `include/morph/qt/qt_websocket_backend.hpp` +- Test: `tests/qt/test_qt_websocket.cpp` + +**Interfaces:** +- Produces: the same counter, decremented in `QWebSocket::bytesWritten` by the envelope sizes it completes (keep + a FIFO of sizes); `QtServerConfig::sendTimeout` (default 30 s, as `net`), closing a peer whose oldest pending + frame is older; `QtWebSocketBackend::setEventHandler`. +- [ ] **Step 1: Write the failing tests:** `qt_posted_event_reaches_handler`, `qt_connect_handler_fires_on_reconnect`, + `qt_stalled_peer_closed_after_send_timeout` (a 200 ms timeout and a peer that never reads). +- [ ] **Step 2: Run** `build/push/tests/qt/morph_qt_tests "[push]"`. Expected: FAIL. +- [ ] **Step 3: Implement.** +- [ ] **Step 4: Run.** Expected: PASS. +- [ ] **Step 5: Commit** `qt: per-connection backlog, events, and a send timeout like net's` + +### Task 9: The simulated backend and one conformance case + +**Files:** +- Modify: `include/morph/core/remote.hpp` (`SimulatedRemoteBackend`) +- Test: `tests/test_push_channel.cpp`, `tests/net/test_push_net.cpp`, `tests/qt/test_qt_websocket.cpp` +- Modify: `docs/spec/core/backend.md` + +- [ ] **Step 1: Write the failing test.** `channel_conformance(make)` is a helper in `tests/push_conformance.hpp`, + instantiated by each suite. It checks: a posted event reaches the handler; it is not taken for a reply; a + connection's events arrive in posting order. +- [ ] **Step 2: Run** all three suites with `"[push][channel]"`. Expected: the simulated one fails. +- [ ] **Step 3: Implement** `SimulatedRemoteBackend` registering a writer for its connection and routing events. +- [ ] **Step 4: Run.** Expected: PASS on all three. +- [ ] **Step 5: Commit** `core: SimulatedRemoteBackend carries events; one conformance case on every transport` + +--- + +## Part 3: Catalog and wire shapes + +### Task 10: `ActionInfo::effects` and `records` + +**Files:** +- Modify: `include/morph/ui/document.hpp`, `include/morph/ui/author.hpp`, `include/morph/ui/bundle.hpp` (glz meta) +- Modify: `docs/spec/ui/document.md`, `docs/spec/ui/service.md` +- Test: `tests/test_push_catalog.cpp` + +**Interfaces:** +- Consumes: `effectsOf`, `recordsOf` (Task 3). +- Produces: + - `struct morph::ui::ActionEffect { std::string kind; std::optional record; };` + - `struct morph::ui::ActionRecordInfo { std::string record; std::optional keyField; std::vector paths; };` + - `ActionInfo` gains `std::optional> effects;` (absent: reads) and + `std::vector records;` + - `CatalogFacts::action()` fills both. +- [ ] **Step 1: Write the failing tests** (`[push][catalog]`): `catalog_facts_carry_effects_and_records`; + `ui_catalog_reply_round_trips_effects` (encode a `CatalogReply`, read it back, and the fields are equal); + `old_reply_without_fields_reads_as_reads` (a JSON reply with no `effects` gives `nullopt`). +- [ ] **Step 2: Run** `"[catalog]"`. Expected: FAIL. +- [ ] **Step 3: Implement.** +- [ ] **Step 4: Run.** Expected: PASS. +- [ ] **Step 5: Commit** `ui: the catalog carries each action's effects and records` + +### Task 11: `events` in `ui-hello`, and `ConnectionFacts` + +**Files:** +- Modify: `include/morph/core/remote.hpp`, `include/morph/ui/bundle.hpp`, `include/morph/ui/service.hpp`, `include/morph/ui/service_client.hpp` +- Modify: `docs/spec/ui/service.md`, `docs/spec/core/backend.md` ("Envelope services") +- Test: `tests/test_push_catalog.cpp`, `tests/test_remote_envelope_service.cpp` + +**Interfaces:** +- Produces: + - `struct morph::backend::ConnectionFacts { bool events = false; };` + - `virtual std::string IEnvelopeService::handle(wire::Envelope const&, session::IAuthorizer const&, ConnectionFacts& facts)` + is non-pure, and its default forwards to the two-parameter `handle`. + - `HelloRequest::events` (`bool`, default false) and `HelloReply::events`. + - `[[nodiscard]] bool RemoteServer::eventsNegotiated(ConnectionId cid) const;` (server strand only) + - The generic UI client's `ui-hello` offers `events: true`. +- [ ] **Step 1: Write the failing tests:** `hello_with_events_sets_connection_fact`; `hello_without_events_leaves_it_false`; + `unscoped_message_gets_throwaway_facts`; `two_param_service_still_compiles_and_serves` (the existing test + service, unchanged). +- [ ] **Step 2: Run** `"[catalog]"`. Expected: FAIL. +- [ ] **Step 3: Implement.** +- [ ] **Step 4: Run** the tests, then `morph_tests "[ui-service]"` to catch regressions. Expected: PASS. +- [ ] **Step 5: Commit** `core, ui: negotiate events in ui-hello; core learns it through ConnectionFacts` + +### Task 12: The watch and changes bodies + +**Files:** +- Modify: `include/morph/core/wire.hpp`, `docs/spec/core/wire.md`, and spec 5 §15 in its `docs/spec` page (the push + kind is core `event`, not `ui-event`) +- Test: `tests/test_push_catalog.cpp` + +**Interfaces:** +- Produces: + - `struct morph::wire::InstanceTopic { std::string modelType; std::string primary; std::uint64_t instance = 0; };` + - `struct morph::wire::WatchBody { std::vector records; std::vector instances; bool replace = false; };` + - `struct morph::wire::WatchReply { std::vector refusedRecords; std::vector refusedInstances; };` + - `struct morph::wire::ChangesBody { std::vector records; std::vector instances; };` + (the payload of a `changes` event) + - Readers that apply the bounds of `kMaxEnvelopeBytes`: `readWatchBody(std::string_view) -> std::expected`, and the same for the other two. +- [ ] **Step 1: Write the failing tests:** each body round-trips; an over-long record list is refused. +- [ ] **Step 2–4:** Run, implement, run. Expected: PASS. +- [ ] **Step 5: Commit** `core: the watch, unwatch and changes bodies` + +--- + +## Part 4: Watching and fan-out (core) + +### Task 13: The watch table and the `watch`/`unwatch` kinds + +**Files:** +- Create: `include/morph/core/detail/watch_table.hpp` +- Modify: `include/morph/core/remote.hpp` (`dispatchEnvelope` branches) +- Modify: `docs/spec/core/backend.md`, `docs/spec/core/shared_instances.md` +- Test: `tests/test_push_watch.cpp` + +**Interfaces:** +- Produces: + - `struct detail::Watch { session::Context context; std::string principal; };` + - `class detail::WatchTable { void add(ConnectionId, wire::WatchBody const&, Watch); void remove(ConnectionId, wire::WatchBody const&); void dropConnection(ConnectionId); [[nodiscard]] std::vector watchersOf(std::string_view record) const; [[nodiscard]] std::vector watchersOf(std::string_view modelType, std::string_view primary) const; Watch const* watchOf(ConnectionId, std::string_view record) const; };` + - Admission: a record only when the principal is authorised for at least one action whose `recordsOf` holds it + (cached per principal and record). An instance only when `authorizeInstance(ctx, modelType, {}, 0, "")` admits + it. The reply is `ok` with a `WatchReply`. +- [ ] **Step 1: Write the failing tests** (`[push][watch]`): `watch_record_admitted`; `watch_on_unreadable_type_refused`; + `instance_watch_stored_before_instance_exists`; `replace_supersedes`; `close_connection_drops_watches`; + `watch_before_hello_is_stored`. +- [ ] **Step 2: Run** `"[watch]"`. Expected: FAIL. +- [ ] **Step 3: Implement.** +- [ ] **Step 4: Run.** Expected: PASS. +- [ ] **Step 5: Commit** `core: connections watch record types and keyed instances` + +### Task 14: Fan-out in the execute path + +**Files:** +- Modify: `include/morph/core/remote.hpp` (`finishRemote`, `RemoteRun` gains `cid` and the instance's + `sharedKey`/`mid`) +- Test: `tests/test_push_watch.cpp` + +**Interfaces:** +- Consumes: `deriveChanges` (Task 4), `WatchTable` (Task 13), `eventsNegotiated` (Task 11), `post` (Task 6). +- Produces: `void RemoteServer::fanOut(effects::Changes changes, ConnectionId caller);` on the server strand. It converts + each `InstanceChange` to a `wire::InstanceTopic` for the event body. It converts each `InstanceChange` to a `wire::InstanceTopic` for the event body. +- [ ] **Step 1: Write the failing tests:** `two_watchers_both_receive_caller_included`; `unwatched_type_posts_nothing`; + `unnegotiated_connection_receives_nothing`; `private_instance_change_only_to_owner`; + `keyed_instance_created_later_reaches_watcher`; `timed_out_call_still_reaches_caller_as_event` (`executeTimeout` + 10 ms, the handler sleeps 50 ms); `failed_execute_posts_nothing`. +- [ ] **Step 2: Run.** Expected: FAIL. +- [ ] **Step 3: Implement.** In `finishRemote`, on success only: derive the changes on the model strand, before + `complete`, and post `fanOut` to the server strand. The reply path is unchanged. +- [ ] **Step 4: Run** the tests, then `"[remote]"`. Expected: PASS. +- [ ] **Step 5: Commit** `core: a successful execute tells every watching connection what changed` + +### Task 15: The principal baseline per connection + +**Files:** +- Modify: `include/morph/core/remote.hpp` (`stampVerifiedPrincipal(env, cid)`; every call site passes `cid`) +- Modify: `docs/spec/security.md` +- Test: `tests/test_push_watch.cpp` + +- [ ] **Step 1: Write the failing tests:** `envelope_under_other_principal_drops_watches`; + `unauthenticated_envelope_drops_watches`; `hello_does_not_reset_baseline`. +- [ ] **Step 2–4:** Run, implement, run. Expected: PASS. +- [ ] **Step 5: Commit** `core: a connection's watches drop when its principal changes` + +### Task 16: Re-authorisation and private keys at send time + +**Files:** +- Modify: `include/morph/core/remote.hpp`, `include/morph/core/detail/watch_table.hpp` +- Test: `tests/test_push_watch.cpp` + +- [ ] **Step 1: Write the failing tests:** `expired_watch_dropped_at_send` (an authoriser whose `authenticate` stops + accepting the token); `lister_not_writer_still_receives`; `private_keys_stripped_for_other_principals_kept_for_caller`. + *Mutation: skip re-authorisation; the expiry case fails.* +- [ ] **Step 2–4:** Run, implement, run. Expected: PASS. +- [ ] **Step 5: Commit** `core: fan-out re-authorises each watch by the rule that admitted it` + +### Task 17: Coalescing + +**Files:** +- Create: `include/morph/core/detail/change_coalescer.hpp` +- Modify: `include/morph/core/remote.hpp` +- Test: `tests/test_push_watch.cpp` + +**Interfaces:** +- Produces: `class detail::ChangeCoalescer { void add(effects::Changes const&); [[nodiscard]] std::size_t keyCount() const; [[nodiscard]] wire::ChangesBody take(); };`. + A `changes` absorbs that record's `updates` keys, and instance topics are deduplicated. The server flushes after + 50 ms, through the server's `TimeoutScheduler`, or at 256 keys. +- [ ] **Step 1: Write the failing tests:** `burst_of_100_is_one_event`; `changes_absorbs_updates_keys`; + `256_keys_flush_without_waiting`; `burst_over_backlog_yields_one_resync`. +- [ ] **Step 2–4:** Run, implement, run. Expected: PASS. +- [ ] **Step 5: Commit** `core: coalesce a connection's changes for 50 ms or 256 keys` + +--- + +## Part 5: The client cache and reconnect seams + +### Task 18: `refetchAfter` + +**Files:** +- Modify: `include/morph/ui/interpreter.hpp` (`CacheEntry`, `Session::settleReply`, `QueryRuntime::refetchCurrent`) +- Modify: `docs/spec/ui/interpreter.md` +- Test: `tests/test_push_cache.cpp` + +**Interfaces:** +- Produces: `CacheEntry::refetchAfter` (`bool`); `void Session::markStale(detail::CacheEntry&)` sets + `stale = true` and, while `pending`, `refetchAfter = true`. `settleReply` keeps the entry stale and fetches once + more when `refetchAfter` was set. +- [ ] **Step 1: Write the failing test:** `stale_mark_during_fetch_fetches_once_more`. Use a `RecordingBackend` + screen with one list query and a held reply: invalidate, then release the reply. The fetch counter goes from 1 + to 2, and stays at 2 after the second reply. *Mutation: restore the unconditional `stale = false`; this case + fails.* +- [ ] **Step 2–4:** Run `"[cache]"`, implement, run. Expected: PASS. +- [ ] **Step 5: Commit** `ui: a change marked during a fetch is kept until the next fetch` + +### Task 19: Replies apply their effects + +**Files:** +- Modify: `include/morph/ui/interpreter.hpp` (`Session::actionSucceeded`, the mutation run, `ScreenCore::aliasSubmits`, `CacheEntry` gains `instanceToken` and a `currentId` reader) +- Test: `tests/test_push_cache.cpp` + +**Interfaces:** +- Produces: + - `enum class ReplyUse : std::uint8_t { Applied, Discarded };` + - `void Session::actionSucceeded(std::string const& typeId, std::string const& actionId, std::string const& replyJson, ReplyUse use, std::string const& instanceToken);` + It keeps today's `refreshOn` behaviour and adds the effects from `_options.catalog.action(actionId)->effects`. + - `void Session::applyChanges(wire::ChangesBody const& changes, ChangeSource source);` with + `enum class ChangeSource : std::uint8_t { Reply, Event };`. The reply path patches `updates` by key field and + path. The event path marks only the entries whose value holds one of the keys. Both run in one + `runtime.batch`. +- [ ] **Step 1: Write the failing tests:** + - `own_updates_reply_patches_by_key_field_without_fetch`: the record's key field is `code`, not `id`. The row + changes and the fetch counter stays flat. + - `reply_patch_at_nested_path`: the path is `header.rows`. + - `changes_reply_refetches_holding_entries`. + - `submit_applies_effects`: a form submit runs through `aliasSubmits`. + - `discarded_latest_reply_marks_only`: a newer reply's row is kept. + - `patch_on_fetching_entry_refetches_once`. + - `changes_instance_refetches_only_that_instance`. +- [ ] **Step 2–4:** Run, implement, run. Expected: PASS. +- [ ] **Step 5: Commit** `ui: every reply applies its action's effects to the cache` + +### Task 20: Events and `resync` + +**Files:** +- Modify: `include/morph/ui/interpreter.hpp` +- Test: `tests/test_push_cache.cpp` + +**Interfaces:** +- Produces: `void Session::deliverEvent(wire::EventBody const& event);` (`changes` goes to + `applyChanges(…, Event)`; `resync` marks every watched entry stale; anything else is ignored). +- [ ] **Step 1: Write the failing tests:** `changes_event_via_local_seam_refetches` (through + `LocalBackend::deliverEvent`, wired to `deliverEvent`); `resync_marks_all_watched`; + `event_on_unmounted_entry_marks_only`; `event_keys_filter_entries_without_them`. +- [ ] **Step 2–4:** Run, implement, run. Expected: PASS. +- [ ] **Step 5: Commit** `ui: the cache applies change events and resync` + +### Task 21: Topic counting, the watcher, and watch before fetch + +**Files:** +- Modify: `include/morph/ui/interpreter.hpp` (`SessionOptions::watcher`, `Session::reconnected`), `include/morph/ui/service_client.hpp` (`watcherOver`) +- Modify: `docs/spec/ui/interpreter.md`, `docs/spec/ui/service.md`, `docs/spec/offline/offline.md` +- Test: `tests/test_push_cache.cpp` + +**Interfaces:** +- Produces: + - `using Watcher = std::function(std::string const& kind, std::string const& bodyJson)>;` + and `SessionOptions::watcher`. + - `[[nodiscard]] Watcher watcherOver(Exchange exchange, std::function currentSession, exec::IExecutor* cbExec);` + - `void Session::reconnected();` sends the whole set with `replace: true`, marks every entry stale, and sets + `refetchAfter` on the pending ones. + - Reference counts are per record name and per `(modelType, primary)`. A count leaving zero queues a `watch`, and + one returning to zero queues an `unwatch`. The counts change on entry creation and eviction. Queued changes go + out as one message before the batch's fetches: `Session::fetch` flushes the queue first. +- [ ] **Step 1: Write the failing tests:** `watch_sent_before_first_fetch` (the recorder's order shows `watch`, then + `execute`); `one_watch_message_per_batch`; `unwatch_after_eviction`; `no_watcher_counts_and_sends_nothing`; + `reconnected_sends_replace_and_marks_stale`; `fetch_filed_before_reconnected_refetches_once`. +- [ ] **Step 2–4:** Run, implement, run. Expected: PASS. +- [ ] **Step 5: Commit** `ui: the session watches what its cache holds, before it fetches` + +### Task 22: `AppSource::onConnected` and the generic client + +**Files:** +- Modify: `include/morph/ui/frontend.hpp`, `docs/spec/ui/frontend.md` +- Modify: `examples/common/app/app_source.{hpp,cpp}`, `examples/common/app/client_shell.cpp` +- Test: `examples/common/app/tests/test_sources.cpp` + +**Interfaces:** +- Produces: + - `virtual void AppSource::onConnected(std::function listener) {}` (empty default) + - The remote source owns the backend's `setConnectHandler`. On a reconnect, it re-sends `ui-hello` with + `events: true`, then calls its listeners on the owner. It installs `setEventHandler` and forwards events to the + session. It exposes `watcher()`. + - The shell sets `sessionOptions.watcher = source.watcher()` and calls + `source.onConnected([&] { session.reconnected(); })`. +- [ ] **Step 1: Write the failing test:** `remote_source_rehellos_then_rewatches_on_reconnect`, over the `net` + transport against an in-process server that is dropped and reopened. Its order is `ui-hello`, then `watch` + (replace), then the refetches. +- [ ] **Step 2–4:** Run the app tests, implement, run. Expected: PASS. +- [ ] **Step 5: Commit** `ui, examples: the generic client re-negotiates and re-watches on every reconnect` + +--- + +## Part 6: Document checks + +### Task 23: The query-must-read rule and the three warnings + +**Files:** +- Modify: `include/morph/ui/document.hpp`, `docs/spec/ui/document.md` +- Test: `tests/test_push_documents.cpp` + +**Interfaces:** +- Produces: + - `enum class LoadWarningKind : std::uint8_t { WatchFeedback, Redundant, Uncovered };` + - `struct LoadWarning { LoadWarningKind kind; std::string path; std::string text; };` + - `Document::warnings` (`std::vector`). The bundle loader and `ScreenRegistry` log each warning + where they already log a refusal. +- [ ] **Step 1: Write the failing tests** (`[push][documents]`): `query_of_action_with_effects_refused`; + `redundant_invalidates_warns`; `redundant_refresh_on_warns`; `uncovered_mutation_warns`; + `effects_none_silences_uncovered`; `watch_feedback_warns`; `watch_source_user_does_not_warn`. +- [ ] **Step 2–4:** Run, implement, run. Expected: PASS. +- [ ] **Step 5: Commit** `ui: documents are checked against action effects` + +--- + +## Part 7: Examples, the explorer, end to end + +### Task 24: The explorer's effects and records rows + +**Files:** +- Modify: `include/morph/ui/explorer.hpp` (`catalogFacts`), `docs/spec/ui/explorer.md` +- Test: `tests/test_ui_explorer.cpp` + +- [ ] **Step 1: Write the failing tests** (`[push][explorer]`): `facts_show_changes_record` gives the cells + `"effects"`, `"changes T_Row"`; `facts_show_reads_when_undeclared` gives `"effects"`, `"reads"`; + `facts_show_records_with_key_and_path` gives `"records"`, `"T_Row by id at rows"`. The wording follows spec §5. +- [ ] **Step 2–4:** Run, implement, run. Expected: PASS. +- [ ] **Step 5: Commit** `ui: the explorer shows what an action changes and which records it returns` + +### Task 25: The three rungs declare records and effects + +**Files:** +- Modify: bookmarks (`examples/bookmarks/include/bookmarks/models/*.hpp`, `screens/bookmarks_screens.cpp`); + bank (`examples/bank/include/bank/models/*.hpp`, `examples/bank/memory/include/bank/models/*.hpp`, + `screens/bank_screens.cpp`); lims (`examples/lims/include/lims/models/*.hpp`, `screens/lims_screens.cpp`); + the three rung READMEs, `docs/spec/testing_strategy.md` +- Test: each rung's existing unit and GUI tests, plus one test per rung, `_bundle_loads_without_warnings` + +- [ ] **Step 1: Write the failing tests:** each rung's `bundle_loads_without_warnings` loads every document and + asserts `warnings.empty()`. It fails now on `uncovered`. +- [ ] **Step 2: Run** each rung's tests. Expected: the new case fails, the rest pass. +- [ ] **Step 3: Declare records and effects** beside each registration. Bank declares them in both model sets. + Use the spec's names: `Bank_Account`, `Bank_Transaction`, `Bank_SpendingReport` (`derivedFrom`), + `Lims_Result`, and `none` on `OpenSample`. Then remove the 9 `invalidates`, the 5 `refreshOn` and the 1 `patch` + that the redundant warnings name. Keep `onSuccess` commands that do something other than refresh. +- [ ] **Step 4: Run** every rung's unit and GUI tests (`ctest --test-dir build/push -L ladder`). Expected: all + PASS, no warnings, and a `grep` for `invalidates|refreshOn|\.patch` in `examples/*/screens` finds nothing. +- [ ] **Step 5: Commit** one commit per rung: `examples(): actions declare their effects; screens drop refresh declarations` + +### Task 26: The two-client harness + +**Files:** +- Create: `tests/net/test_push_two_clients.cpp` +- Modify: `docs/spec/testing_strategy.md` + +- [ ] **Step 1: Write the test** (`[push][e2e]`). One `RemoteServer` and `SocketServer` run with test models + `ListRows` and `EditRow` (`changes`), and two `SocketBackend`s, each with a `ui::Session` wired through + `watcherOver` and `setEventHandler`. Each mounts one screen with a list query and no refresh declarations. + Client A runs the mutation, and B's fetch counter and value change within 2 s, with no `refreshEvery`. +- [ ] **Step 2: Run the control.** Comment out the `BRIDGE_REGISTER_EFFECTS` line. Expected: B does not update, and + the test fails. Restore the line. +- [ ] **Step 3: Run.** Expected: PASS. +- [ ] **Step 4: Commit** `tests: one client's mutation updates another client's screen over net` + +--- + +## Part 8: `ui/2` + +### Task 27: Remove the refresh keys in a new vocabulary + +**Files:** +- Modify: `include/morph/ui/document.hpp` (`kVocabulary` stays `"ui/1"`; add `kVocabulary2 = "ui/2"`; the parser + accepts both and refuses `invalidates`, `patch` and `refreshOn` under `ui/2`; `optimistic` gets its own + definition with the `{query, key, with}` shape) +- Modify: `include/morph/ui/author.hpp` (emits `ui/2`, drops `.invalidates`, `.patch` and `.refreshOn`), + `include/morph/ui/service.hpp` (serves both vocabularies), `tests/ui/golden/*.json` +- Modify: `docs/spec/ui/document.md`, `docs/spec/ui/author.md`, spec 5 §5 and §12 +- Test: `tests/test_push_ui2.cpp`, `tests/test_ui_author.cpp` + +- [ ] **Step 1: Write the failing tests** (`[push][ui2]`): `ui2_refuses_invalidates`; `ui2_refuses_refresh_on`; + `ui2_keeps_optimistic`; `ui1_document_still_loads_with_refresh_on`; `author_emits_ui2`. +- [ ] **Step 2–4:** Run, implement, regenerate the goldens with the existing golden-update path, run all + `morph_tests`, and run the rung suites. Expected: PASS. +- [ ] **Step 5: Commit** `ui: ui/2 removes invalidates, patch and refreshOn; ui/1 documents still load` + +--- + +## Part 9: The MCP adapter + +### Task 28: `McpAdapter` + +**Files:** +- Create: `include/morph/ui/agent.hpp`, `docs/spec/ui/agent.md` +- Test: `tests/test_push_agent.cpp` + +**Interfaces:** +- Consumes: `RemoteServer::openConnection`, `handle(msg, reply, cid)`, `CatalogReply::permitted`, the `schemas` kind. +- Produces: + - `struct McpOptions { session::Context session; std::string serverName = "morph"; };` + - `class McpAdapter { public: McpAdapter(std::shared_ptr server, McpOptions options); ~McpAdapter(); [[nodiscard]] async::Completion> handle(std::string message); };` + The destructor closes its connection. + - `void serveStdio(McpAdapter& adapter, std::istream& in, std::ostream& out);` +- [ ] **Step 1: Write the failing tests** (`[push][agent]`): `initialize_answers_capabilities`; + `tools_list_is_permitted_plus_evaluate`; `tools_call_runs_and_returns_result_text`; + `refused_call_is_error_and_posts_nothing`; `call_reaches_watching_gui_connection_as_event`; + `unknown_method_is_32601`; `notification_gets_no_reply`; `stdio_round_trip` (over `std::stringstream`). +- [ ] **Step 2–4:** Run, implement, run. Expected: PASS. +- [ ] **Step 5: Commit** `ui: an MCP adapter runs agent tool calls through an ordinary connection` + +### Task 29: Bank serves MCP on stdio + +**Files:** +- Modify: `examples/bank/src/server/main.cpp`, `examples/bank/README.md` +- Test: `examples/bank/tests/` (one case: the server binary with `--mcp-stdio` answers `tools/list` on a pipe) + +- [ ] **Step 1–4:** Write the test, run it (FAIL), add the option (the token comes from `MORPH_MCP_TOKEN`), run it + (PASS). +- [ ] **Step 5: Commit** `examples(bank): --mcp-stdio serves the MCP adapter beside the WebSocket server` + +--- + +## Task 30: Whole-branch verification and the PR + +- [ ] **Step 1:** `ctest --test-dir build/push --output-on-failure`. Expected: 100% pass. Record the totals. +- [ ] **Step 2:** Build the docs with the command in AGENTS.md "CI notes". Expected: no Doxygen warnings. +- [ ] **Step 3:** Build with `-DMORPH_CLIENT_ONLY=ON` (`morph_client_only_*` targets). Expected: builds; no + registrar is emitted. +- [ ] **Step 4:** `grep -rnE 'invalidates|refreshOn' examples --include='*.cpp' | grep -v /tests/` gives no code + occurrence. +- [ ] **Step 5:** Push, update PR #990's title and body to describe the implementation (`/update-pr`), and drive CI + to green, fixing clang-tidy findings from CI's log. diff --git a/docs/superpowers/specs/2026-10-16-server-push-design.md b/docs/superpowers/specs/2026-10-16-server-push-design.md index 19a26ca7a..2bb502d24 100644 --- a/docs/superpowers/specs/2026-10-16-server-push-design.md +++ b/docs/superpowers/specs/2026-10-16-server-push-design.md @@ -81,7 +81,9 @@ other client can see that working copy. | A key may be raw (integral or `std::string`) or a strong id wrapping one (`WrappedModelKey`). | `include/morph/core/model_key.hpp` | | Offline replay runs as reconnect → activate → `bindContext` → `replay`. | `docs/spec/offline/offline.md`, `include/morph/offline/reconnect_coordinator.hpp` | | Within a vocabulary version, changes are additive. | spec 5 §12 "Versions" | -| The examples hold 21 `invalidates` and 13 `refreshOn` occurrences outside their tests, in seven rungs. | `grep` over `examples/`, at `a55374f5` | +| The registration latch closes on the first read of a process-level registry only in a debug build. A release build closes it only when the application calls `closeRegistrationPhase()`, which is `noexcept`. | `include/morph/core/registry.hpp` | +| The app explorer shows, for one action, its catalog facts (`explorer_facts`): whether it sends a credential, its calculated, secret and list fields, and its payload fingerprint. | `include/morph/ui/explorer.hpp`, `docs/spec/ui/explorer.md` | +| The ladder has three rungs: bookmarks, bank and lims (`examples/rungs.txt`). Outside their tests they hold 9 `invalidates` (bookmarks 2, bank 5, lims 2), 5 `refreshOn` and 1 `patch` (lims), and no `refreshEvery`. Bank registers its actions in two model sets, `examples/bank/include` and `examples/bank/memory/include`. | `grep` over `examples/`, at `e70025cc` | ## 3. Concepts @@ -89,7 +91,7 @@ other client can see that working copy. optionally a key member: `ResultView` keyed by its `ResultId`, `AccountInfo` keyed by `id`. A record type shown through several models is one record type: `CustomerModel::ListAccounts` and `AccountModel::GetAccount` both return `AccountInfo`. -- **Derived record type.** A record computed from others: ledger's `GetBudgetReportResult` from transactions. It +- **Derived record type.** A record computed from others: bank's `SpendingReport` from transactions (`TxnInfo`). It is keyless, and it declares what it is derived from (§4), so a change to its sources reaches it without every writer naming it. - **Effect.** What a successful action did. One of: @@ -122,13 +124,22 @@ Records and effects are matched by **runtime type identity** (`std::type_index`) closes. The order of `BRIDGE_REGISTER_RECORD`, `BRIDGE_REGISTER_ACTION` and `BRIDGE_REGISTER_EFFECTS` across translation units therefore does not matter. +**When matching runs.** The latch cannot run it: a release build closes the latch only when the application asks, +and `closeRegistrationPhase()` is `noexcept`. Matching therefore runs once (`std::call_once`), on the first read of +the record or effects registry, which also closes the latch as any registry read does. `RemoteServer`'s constructor +and the catalog facts (`ui::CatalogFacts::action`, which local mode and the server's `ui-catalog` both build from) +each make that read, so a server or a local client fails at startup. A +failure throws `morph::model::EffectsRegistrationError` (a `std::logic_error`) naming every problem found, not only +the first, and every later read throws the same error. "Refused when the phase closes" below means this. + ### Records ```cpp BRIDGE_REGISTER_RECORD(bank::dto::AccountInfo, "Bank_Account") // key: member `id` BRIDGE_REGISTER_RECORD(lims::ResultView, "Lims_Result") // key: `id`, a ResultId -BRIDGE_REGISTER_RECORD(ledger::dto::GetBudgetReportResult, "Ledger_BudgetReport", - morph::record::keyless, morph::record::derivedFrom) +BRIDGE_REGISTER_RECORD(bank::dto::TxnInfo, "Bank_Transaction") // key: member `id` +BRIDGE_REGISTER_RECORD(bank::dto::SpendingReport, "Bank_SpendingReport", + morph::record::keyless, morph::record::derivedFrom) ``` - **The key** defaults to a member named `id`, the field the interpreter's `patch` already identifies rows by. It @@ -228,6 +239,12 @@ the record types its entries hold. An `evaluate` cache entry (`ui-evaluate`, spec 5 §7) holds no records and is never watched: an evaluation depends on its body, not on stored data. +**The app explorer shows both.** Its Facts section (`explorer_facts`) gains two rows from the same `ActionInfo`: +"effects", listing each effect as `changes Bank_Account`, `updates Lims_Result`, `changes its instance` or `none`, +and "reads" when nothing is declared; and "records", listing each record with its key field and paths, such as +`Lims_Result by id at results`. A person deciding why a screen did or did not update reads the answer there, without +the source. + ## 6. The server-to-client channel The wire gains its first server-initiated message. @@ -473,7 +490,7 @@ With effects declared, a document needs no refresh declarations. cover, so a document can migrate one screen at a time. - **warning, uncovered:** a mutation whose action declares nothing (not even `none`), with no `invalidates` or `patch` of its own and no query naming it in `refreshOn`. After it succeeds, nothing on any screen updates. - `effects::none` silences it for actions that attach or open, such as `OpenSample` and `OpenBoard`. + `effects::none` silences it for actions that attach or open, such as lims's `OpenSample`. **In `ui/2`, the removal.** `invalidates`, `patch` and `refreshOn` are removed from mutations and queries. Removing keys is not additive, so it is a new vocabulary version under spec 5 §12. A server that keeps `ui/1` documents for @@ -502,6 +519,21 @@ changes visible. - It lists the actions the principal may run (the catalog's `permitted`, with their JSON shapes) as MCP tools, and offers `ui-evaluate` as a dry run. - The agent may commit directly. +- **The adapter's shape.** + - `morph::ui::McpAdapter` (`include/morph/ui/agent.hpp`) speaks MCP's JSON-RPC 2.0 and knows no transport: + `handle(message)` takes one JSON-RPC message and completes with its reply, or with nothing for a notification. + It answers `initialize`, `tools/list` and `tools/call`, and refuses every other method with JSON-RPC error + `-32601`. + - `serveStdio(adapter, in, out)` drives it over newline-delimited streams, which is MCP's stdio transport. An + agent such as Claude Code launches the server process with an option that turns it on (bank's server: + `--mcp-stdio`), and the process serves its WebSocket clients as before. An HTTP transport is out of scope (§15). + - **Identity.** The deployer constructs the adapter with the `session::Context` its envelopes carry, for example + from a token in the environment. The server authenticates it on every envelope, as it does a GUI's. + - **Tools.** One tool per permitted action, named by the action id, whose `inputSchema` is the action's schema + (`schemas`). A keyed model's tool takes the instance key as the argument `instance`, and the adapter attaches + the shared instance by it. Any other model runs on one private instance per model type, registered on the + adapter's connection at first use. A tool named `evaluate`, taking `{action, body}`, is the dry run. + - A tool call's reply is the action's result JSON as text content, or its `err` message with `isError: true`. - **Fan-out treats the agent's connection like any other,** so every open GUI showing the changed records updates through §8 and §9. - **Model access and keys are not part of this design.** The adapter does not call a model provider. An external @@ -604,6 +636,13 @@ Every test below must fail if the feature does nothing, so each names the change - A redundant `invalidates` warns. - An uncovered mutation warns, and `effects::none` silences it. - The watch-feedback shape warns. +- **Explorer (ui).** An action with `changes` shows `changes R` in `explorer_facts`, and one with nothing + declared shows `reads`. *Mutation: drop the effects row; the case fails.* +- **Agents (ui).** + - `tools/list` lists exactly the permitted actions, plus `evaluate`. + - A `tools/call` of a writer through the adapter reaches a GUI connection watching its record as an event. + - A call the authoriser refuses answers `isError: true` and posts nothing. + - *Control: the adapter's connection is not watched by anyone; no event is posted.* - **End to end.** - A harness in `tests/` runs two generic-client sessions over the `net` transport against one server. - Client A's mutation updates client B's mounted screen, with no `refreshOn` and no `refreshEvery` in either @@ -612,8 +651,11 @@ Every test below must fail if the feature does nothing, so each names the change ## 14. Delivery -Each part lands as its own pull request to `master`. Parts with no dependency between them can run side by side. -Each part updates the `docs/spec/` pages named in its row, and keeps every new public symbol's Doxygen complete. +All nine parts land in **one pull request**, the one that carries this design, in the order below. Each part is a +run of commits that leaves the branch building, with its unit and GUI tests green, so a bisect never lands on a +broken commit and review can read the branch part by part. Each part updates the `docs/spec/` pages named in its +row, and keeps every new public symbol's Doxygen complete. The implementation plan is +`docs/superpowers/plans/2026-10-18-server-push.md`. | # | Part | Layer | Needs | Spec pages | |---|---|---|---|---| @@ -623,20 +665,13 @@ Each part updates the `docs/spec/` pages named in its row, and keeps every new p | 4 | **Watching and fan-out.** Core `watch`/`unwatch` kinds in `RemoteServer` with the `cid`; the watch table with session contexts; the principal comparison in `stampVerifiedPrincipal`; fan-out in `dispatchExecute`, with private instances to their owner; `derivedFrom` expansion, coalescing, re-authorisation, `privateKeys` | core | 1, 2, 3 | `core/backend.md`, `core/shared_instances.md`, `security.md` | | 5 | **Client cache and reconnect seams.** `actionSucceeded` carries the reply and applies effects; patch by key field and path; `refetchAfter`, set by a stale mark or a patch on a fetching entry; discarded `latest` replies as marks; events and `resync`; topic reference counting; watch before fetch; `Session::reconnected()`; `AppSource::onConnected` with an empty default; `watcherOver` and `SessionOptions::watcher`; the generic client's remote source and shell wired to them (`examples/common/app`) | ui, examples/common | 2, 3 (4 for its remote tests) | `ui/interpreter.md`, `ui/frontend.md`, `ui/service.md`, `offline/offline.md`, spec 5 §5 | | 6 | **Document checks.** Query-must-read refused; watch-feedback, redundant and uncovered warnings | ui | 3 | `ui/document.md` | -| 7 | **End-to-end harness and examples.** The two-session harness in `tests/`. Records and effects in the seven rungs that use refresh declarations (`none` on attach-style actions), with their declarations removed in `ui/1` documents where the warnings say so | tests, examples | 4, 5, 6 | the rungs' READMEs, `testing_strategy.md` | +| 7 | **End-to-end harness, examples and the explorer.** The two-session harness in `tests/`. Records and effects in the three rungs: bookmarks, bank (both model sets) and lims, with `none` on attach-style actions; their 9 `invalidates`, 5 `refreshOn` and 1 `patch` removed from `ui/1` documents where the warnings say so. The explorer's effects and records rows | tests, examples, ui | 4, 5, 6 | the rungs' READMEs, `testing_strategy.md`, `ui/explorer.md` | | 8 | **`ui/2`.** Remove `invalidates`, `patch` and `refreshOn`; `optimistic` keeps its own shape; the authoring layer and golden files emit `ui/2`; the server keeps serving `ui/1` | ui | 7 | `ui/document.md`, `ui/author.md`, spec 5 §5 and §12 | -| 9 | **MCP adapter.** A connection scope per agent session, tools from `permitted`, calls through `RemoteServer::handle` | net, ui | 4 | a new `ui/agent.md` | - -**Ready now: parts 1 and 2.** They are independent. Part 1 is `registry.hpp` and a new records header. Part 2 is -`wire.hpp`'s event kind, `remote.hpp`'s writer and `post`, `backend.hpp` and the transports. - -**Then part 3,** after part 2, because both change `wire.hpp` and `remote.hpp`. **Then parts 4 and 5 side by side, with part 6 -beside them.** -- They share only the shapes part 3 defines. -- Part 4 is in `remote.hpp`; part 5 is in the interpreter and the offline glue. -- Part 5 tests its event path through the local backend's seam until part 4 lands. +| 9 | **MCP adapter.** `McpAdapter` over a connection scope, tools from `permitted`, calls through `RemoteServer::handle`, `serveStdio`, bank's `--mcp-stdio` | ui, examples | 4 | a new `ui/agent.md` | -Part 9 can run beside anything after part 4. +**Order in the branch:** 1, 2, 3, 4, 5, 6, 7, 8, 9. Parts 1 and 2 are independent and 6 needs only 3, but one +branch is written in one order. Part 9 comes after 8 so that the agent test runs against the examples' final +documents. ## 15. Out of scope @@ -650,6 +685,8 @@ Part 9 can run beside anything after part 4. refetches cost something. - **Effects computed at run time,** such as a change list a model returns. Declared effects with `derivedFrom` cover the examples' cases. A run-time list would have to keep the rule that a model does not know the framework exists. +- **An HTTP transport for the MCP adapter.** The adapter is transport-free and is served over stdio (§11). MCP's + streamable HTTP transport would let an agent reach a running server it did not launch. - **Push for non-UI clients.** Negotiation is in `ui-hello`. A plain `Client` that never sends it receives no events. The core `hello` reply already lists capabilities, but its request carries none, so a plain client has no way to offer events. Adding that is a separate change to `hello`. From eadecffc15256617e7d82561af0e3a052830de65 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 11:27:24 +0200 Subject: [PATCH 10/68] tests: table tests use std::iota, which every libc++ has std::ranges::iota is missing from Apple's libc++ as it is from Emscripten's, so the table suite did not compile on macOS. The library already uses std::iota for the same reason. Co-Authored-By: Claude Opus 5.5 --- tests/table/test_table_filter.cpp | 6 ++++-- tests/table/test_table_sort.cpp | 15 ++++++++++----- tests/table/test_table_view_change.cpp | 12 ++++++++---- 3 files changed, 22 insertions(+), 11 deletions(-) diff --git a/tests/table/test_table_filter.cpp b/tests/table/test_table_filter.cpp index 77c9980ff..59240177c 100644 --- a/tests/table/test_table_filter.cpp +++ b/tests/table/test_table_filter.cpp @@ -55,7 +55,8 @@ struct Filtered { pointers.push_back(&key); } Rows rows(snapshot->rowCount()); - std::ranges::iota(rows, 0U); + // NOLINTNEXTLINE(modernize-use-ranges): std::ranges::iota is missing from libc++. + std::iota(rows.begin(), rows.end(), 0U); FilterRun run{*compiled, pointers, rows}; static_cast(run.run(kNoDeadline, ::core::async::StopToken{})); return std::move(run).take(); @@ -273,7 +274,8 @@ TEST_CASE("table: a filter run yields at a deadline and keeps candidate order", auto keys = KeyColumn::forColumn(source->columns()[0], services, cells.size()); buildKeys(*source->snapshot(), 0, source->columns()[0], services, 0, cells.size(), keys); Rows reversed(cells.size()); - std::ranges::iota(std::views::reverse(reversed), 0U); + // NOLINTNEXTLINE(modernize-use-ranges): std::ranges::iota is missing from libc++. + std::iota(reversed.rbegin(), reversed.rend(), 0U); FilterRun run{*compiled, {&keys}, reversed}; std::size_t steps = 0; while (!run.run(std::chrono::steady_clock::now(), ::core::async::StopToken{})) { diff --git a/tests/table/test_table_sort.cpp b/tests/table/test_table_sort.cpp index eb59e25fe..6ef735924 100644 --- a/tests/table/test_table_sort.cpp +++ b/tests/table/test_table_sort.cpp @@ -53,7 +53,8 @@ struct Sorted { } } std::vector rows(snapshot->rowCount()); - std::ranges::iota(rows, 0U); + // NOLINTNEXTLINE(modernize-use-ranges): std::ranges::iota is missing from libc++. + std::iota(rows.begin(), rows.end(), 0U); return sortRows(std::move(rows), RowOrder{std::move(resolved)}); } }; @@ -235,7 +236,8 @@ TEST_CASE("table: a stop request ends a merge sort early, and a later run finish auto keys = KeyColumn::forColumn(source->columns()[0], services, snapshot->rowCount()); buildKeys(*snapshot, 0, source->columns()[0], services, 0, snapshot->rowCount(), keys); std::vector rows(snapshot->rowCount()); - std::ranges::iota(rows, 0U); + // NOLINTNEXTLINE(modernize-use-ranges): std::ranges::iota is missing from libc++. + std::iota(rows.begin(), rows.end(), 0U); ChunkedMergeSort sort{rows, RowOrder{{{.keys = &keys, .dir = kAsc}}}}; ::core::async::StopSource stopSource; // NOLINT(misc-const-correctness): request_stop() is non-const @@ -261,7 +263,8 @@ TEST_CASE("table: a past deadline yields between chunks", "[table][sort]") { auto keys = KeyColumn::forColumn(source->columns()[0], services, snapshot->rowCount()); buildKeys(*snapshot, 0, source->columns()[0], services, 0, snapshot->rowCount(), keys); std::vector rows(snapshot->rowCount()); - std::ranges::iota(rows, 0U); + // NOLINTNEXTLINE(modernize-use-ranges): std::ranges::iota is missing from libc++. + std::iota(rows.begin(), rows.end(), 0U); ChunkedMergeSort sort{rows, RowOrder{{{.keys = &keys, .dir = kAsc}}}}; std::size_t steps = 0; while (!sort.run(std::chrono::steady_clock::now(), ::core::async::StopToken{})) { @@ -323,7 +326,8 @@ TEST_CASE("table: a merge pass of narrow runs looks at the clock", "[table][sort auto keys = KeyColumn::forColumn(source->columns()[0], services, kRows); buildKeys(*snapshot, 0, source->columns()[0], services, 0, kRows, keys); std::vector order(kRows); - std::ranges::iota(order, 0U); + // NOLINTNEXTLINE(modernize-use-ranges): std::ranges::iota is missing from libc++. + std::iota(order.begin(), order.end(), 0U); ChunkedMergeSort sort{std::move(order), RowOrder{{{.keys = &keys, .dir = kAsc}}}}; std::size_t largestStep = 0; bool done = false; @@ -402,7 +406,8 @@ TEST_CASE("table: a merge sort is done only once a run returned true", "[table][ auto keys = KeyColumn::forColumn(source->columns()[0], services, cells.size()); buildKeys(*source->snapshot(), 0, source->columns()[0], services, 0, cells.size(), keys); Rows rows(cells.size()); - std::ranges::iota(rows, 0U); + // NOLINTNEXTLINE(modernize-use-ranges): std::ranges::iota is missing from libc++. + std::iota(rows.begin(), rows.end(), 0U); RowOrder const order{{{.keys = &keys, .dir = kAsc}}}; CHECK_FALSE(order.empty()); CHECK(RowOrder{}.empty()); diff --git a/tests/table/test_table_view_change.cpp b/tests/table/test_table_view_change.cpp index 08fd05c1f..c14442e24 100644 --- a/tests/table/test_table_view_change.cpp +++ b/tests/table/test_table_view_change.cpp @@ -92,7 +92,8 @@ TEST_CASE("table: an updated row that moves is one move and one change", "[table TEST_CASE("table: a reversal moves all but one row", "[table][viewchange]") { Ids before(50); - std::ranges::iota(before, 0U); + // NOLINTNEXTLINE(modernize-use-ranges): std::ranges::iota is missing from libc++. + std::iota(before.begin(), before.end(), 0U); Ids after(before.rbegin(), before.rend()); auto const change = diff(before, after); CHECK(count(change, Kind::Moved) == 49); @@ -105,7 +106,8 @@ TEST_CASE("table: random reorders with inserts and removes replay exactly", "[ta for (int round = 0; round < 300; ++round) { auto const universe = static_cast(1 + (rng() % 60)); Ids pool(universe); - std::ranges::iota(pool, 0U); + // NOLINTNEXTLINE(modernize-use-ranges): std::ranges::iota is missing from libc++. + std::iota(pool.begin(), pool.end(), 0U); std::ranges::shuffle(pool, rng); Ids before(pool.begin(), pool.begin() + static_cast(rng() % (universe + 1))); std::ranges::shuffle(pool, rng); @@ -132,14 +134,16 @@ TEST_CASE("table: random reorders with inserts and removes replay exactly", "[ta TEST_CASE("table: a change over the threshold is one reset", "[table][viewchange]") { Ids before(100); - std::ranges::iota(before, 0U); + // NOLINTNEXTLINE(modernize-use-ranges): std::ranges::iota is missing from libc++. + std::iota(before.begin(), before.end(), 0U); Ids after(before.rbegin(), before.rend()); auto const change = diff(before, after, {}, 10); CHECK(change.isReset()); CHECK(replay(before, change, after) == after); Ids full(100); - std::ranges::iota(full, 0U); + // NOLINTNEXTLINE(modernize-use-ranges): std::ranges::iota is missing from libc++. + std::iota(full.begin(), full.end(), 0U); Ids sparse; sparse.reserve(50); for (std::uint32_t i = 0; i < 100; i += 2) { From 9add7e6064f3e5389c12a113ac51ad3870edd9a7 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 11:30:54 +0200 Subject: [PATCH 11/68] core: a registry of record types, with keys, derivation and private keys Co-Authored-By: Claude Opus 5.5 --- CMakeLists.txt | 1 + docs/spec/core/registry.md | 21 ++- include/morph/core/records.hpp | 232 +++++++++++++++++++++++++++++++++ tests/CMakeLists.txt | 1 + tests/test_push_records.cpp | 112 ++++++++++++++++ 5 files changed, 366 insertions(+), 1 deletion(-) create mode 100644 include/morph/core/records.hpp create mode 100644 tests/test_push_records.cpp diff --git a/CMakeLists.txt b/CMakeLists.txt index 096701d9b..d6507e0cd 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -453,6 +453,7 @@ target_sources(morph include/morph/core/timeout_scheduler.hpp include/morph/core/model.hpp include/morph/core/registry.hpp + include/morph/core/records.hpp include/morph/core/backend.hpp include/morph/core/envelope_transport.hpp include/morph/core/remote.hpp diff --git a/docs/spec/core/registry.md b/docs/spec/core/registry.md index 6c62ecbab..dd2c7293f 100644 --- a/docs/spec/core/registry.md +++ b/docs/spec/core/registry.md @@ -1643,4 +1643,23 @@ testing obligation, not a compile-time guarantee. originate in the `ActionTraits` JSON functions. - **Concurrency and lifetimes** — the static-init-then-read-only discipline in [Thread safety](#thread-safety) and the process-lifetime singleton ownership in - [Limitations](#limitations). \ No newline at end of file + [Limitations](#limitations). +## Record registry + +`include/morph/core/records.hpp` holds a registry of *record types*, the rows a +server can describe changes to. `BRIDGE_REGISTER_RECORD(T, NAME, options...)` is a +static registrar like `BRIDGE_REGISTER_ACTION`'s: it asserts the registration +latch and compiles out under `MORPH_CLIENT_ONLY`. Each call appends one +`morph::record::RecordInfo` (wire name, `std::type_index`, key field, sources a +record is derived from, `privateKeys`, `keyIsString`) to a process-level vector, +read through `detail::recordRegistry()`. + +- The key defaults to the member glaze reflects as `id`; `key<&T::m>` names + another member (its glaze JSON key is recorded), `keyless` declares none. A + type with no `id`, no `key<>` and no `keyless` is recorded with no key field. +- The key member is a `ModelKey`, raw or a `WrappedModelKey` strong id. + `keyIsString` follows the raw type. +- The registry is raw: duplicate names or types, `derivedFrom` cycles and keyless + records that an effect names are not checked at registration. +- Record types need external linkage: glaze reflects member names through an + external-linkage object, so a record in an anonymous namespace does not compile. diff --git a/include/morph/core/records.hpp b/include/morph/core/records.hpp new file mode 100644 index 000000000..9336bf8fe --- /dev/null +++ b/include/morph/core/records.hpp @@ -0,0 +1,232 @@ +// SPDX-License-Identifier: Apache-2.0 + +#pragma once +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "../detail/reflected_member.hpp" +#include "model_key.hpp" +#include "registry.hpp" + +/// @file +/// The registry of record types: the rows a server can push changes about. +/// +/// A record is a plain reflected struct identified by a key member. Registration +/// is a static registrar, like an action's, so it is checked by the same +/// registration-phase latch. The registry stores what was declared; cross-entry +/// validation (duplicate names, derivation cycles, keyless records that an effect +/// names) happens when the registry is read for matching, not here. + +namespace morph::record { + +/// @brief Option for `BRIDGE_REGISTER_RECORD`: the record's key is the named member instead of `id`. +/// @tparam Member Pointer to a data member of the record, e.g. `&Row::code`. +template +struct key {}; + +/// @brief Option for `BRIDGE_REGISTER_RECORD`: the record has no key. +struct keyless {}; + +/// @brief Option for `BRIDGE_REGISTER_RECORD`: a change to any of @p S is a change to this record type. +/// @tparam S The source record types. +template +struct derivedFrom {}; + +/// @brief Option for `BRIDGE_REGISTER_RECORD`: the record's keys must not reach other principals. +struct privateKeys {}; + +/// @brief What one `BRIDGE_REGISTER_RECORD` declared. +struct RecordInfo { + /// The wire name of the record type. + std::string name; + /// The record's C++ type; records and effects are matched by this identity. + std::type_index type; + /// The glaze JSON name of the key member; empty for a keyless record or one without an `id` member. + std::optional keyField; + /// The source types this record is derived from. + std::vector derivedFrom; + /// True when the record's keys must not reach other principals. + bool privateKeys = false; + /// True when the key's raw type is `std::string`; false when it is integral or there is no key. + bool keyIsString = false; +}; + +namespace detail { + +/// @brief Detects a `key<>` option and exposes its member pointer. +/// @tparam T An option. +template +struct KeyOption { + static constexpr bool value = false; +}; +template +struct KeyOption> { + static constexpr bool value = true; + static constexpr auto member = Member; +}; + +/// @brief Detects a `derivedFrom<>` option. +/// @tparam T An option. +template +struct IsDerivedFrom : std::false_type {}; +template +struct IsDerivedFrom> : std::true_type {}; + +/// @brief The type of the data member a member pointer names. +/// @tparam M A pointer-to-data-member type. +template +struct MemberType; +template +struct MemberType { + using type = Member; +}; + +/// @brief Whether a key member's raw type is `std::string`, looking through a wrapped strong id. +/// @tparam K The key member's type. +template +constexpr bool keyTypeIsString() { + using Plain = std::remove_cvref_t; + if constexpr (model::WrappedModelKey) { + return std::is_same_v, std::string>; + } else { + return std::is_same_v; + } +} + +/// @brief The reflected name of a data member, found by its address in a value-initialised object. +/// @tparam Class The record type. +/// @tparam Member The member's type. +/// @param member The member. +/// @return The glaze JSON key of the member, or empty when glaze does not reflect it. +template +[[nodiscard]] std::string memberKeyName(Member Class::* member) { + Class const probe{}; + void const* const target = std::addressof(probe.*member); + std::string found; + [&](std::index_sequence) { + auto visit = [&]() { + decltype(auto) value = morph::detail::reflectedMember(probe); + if constexpr (std::is_reference_v) { + if (static_cast(std::addressof(value)) == target) { + found = std::string{glz::reflect::keys[I]}; + } + } + }; + (visit.template operator()(), ...); + }(std::make_index_sequence::size>{}); + return found; +} + +/// @brief The index of the reflected member named `id`, or `size` when there is none. +/// @tparam T The record type. +template +constexpr std::size_t idIndex() { + constexpr auto keys = glz::reflect::keys; + for (std::size_t i = 0; i < keys.size(); ++i) { + if (keys[i] == "id") { + return i; + } + } + return keys.size(); +} + +/// @brief Fills the key fields of @p info from the options. +/// @tparam T The record type. +/// @tparam Opts The options. +/// @param info The entry being built. +template +void applyKey(RecordInfo& info) { + constexpr bool keyless_ = (std::is_same_v || ...); + constexpr int explicitKeys = (static_cast(KeyOption::value) + ... + 0); + static_assert(explicitKeys <= 1, "BRIDGE_REGISTER_RECORD: at most one key<> option"); + static_assert(!(keyless_ && explicitKeys > 0), "BRIDGE_REGISTER_RECORD: keyless and key<> contradict"); + if constexpr (keyless_) { + return; + } else if constexpr (explicitKeys == 1) { + auto const apply = [](RecordInfo& target) { + if constexpr (KeyOption::value) { + using Raw = typename MemberType::member)>>::type; + static_assert(model::ModelKey>, + "BRIDGE_REGISTER_RECORD: a key member must be a ModelKey"); + target.keyField = memberKeyName(KeyOption::member); + target.keyIsString = keyTypeIsString(); + } + }; + (apply.template operator()(info), ...); + } else if constexpr (constexpr std::size_t index = idIndex(); index < glz::reflect::size) { + using Raw = std::remove_cvref_t(std::declval()))>; + static_assert(model::ModelKey, "BRIDGE_REGISTER_RECORD: the `id` member must be a ModelKey"); + info.keyField = "id"; + info.keyIsString = keyTypeIsString(); + } +} + +/// @brief The process-level record list, appended to by static registrars. +/// @return The mutable list. +inline std::vector& mutableRecordRegistry() { + static std::vector entries; + return entries; +} + +/// @brief The records declared so far, raw and unvalidated. +/// @return The entries in registration order. +inline std::vector const& recordRegistry() { + return mutableRecordRegistry(); +} + +/// @brief Static-init helper for `BRIDGE_REGISTER_RECORD`. +/// @tparam T The record type. +/// @tparam Opts Any of `key<>`, `keyless`, `derivedFrom<>`, `privateKeys`. +/// @param name The record's wire name. +/// @return Always true, so the result can initialise a namespace-scope variable. +template +inline bool registerRecordOnce(std::string_view name) noexcept { + // NOLINTNEXTLINE(misc-static-assert,cert-dcl03-c) + assert(!::morph::model::registrationPhaseClosed() && + "registerRecordOnce: registration after the registration phase closed. The process-level " + "registries are unsynchronised and are read-only once dispatch begins -- registering now " + "races their internals against concurrent lookups. Load and register plugin modules before " + "the first dispatch."); + RecordInfo info{.name = std::string{name}, .type = std::type_index{typeid(T)}, .keyField = {}, .derivedFrom = {}}; + applyKey(info); + info.privateKeys = (std::is_same_v || ... || false); + auto const addSources = [](RecordInfo& target) { + if constexpr (IsDerivedFrom::value) { + [](RecordInfo& out, derivedFrom) { + (out.derivedFrom.emplace_back(typeid(S)), ...); + }(target, O{}); + } + }; + (addSources.template operator()(info), ...); + mutableRecordRegistry().push_back(std::move(info)); + return true; +} + +} // namespace detail +} // namespace morph::record + +/// @def BRIDGE_REGISTER_RECORD +/// @brief Registers a record type under a wire name, with optional key, derivation and privacy options. +/// +/// The key defaults to the member reflected as `id`. Options are `morph::record::key<&T::m>`, +/// `morph::record::keyless`, `morph::record::derivedFrom` and `morph::record::privateKeys`. Expands to +/// nothing under `MORPH_CLIENT_ONLY`. +/// @param T The record type. +/// @param NAME The record's wire name. +/// @param ... The options. +#ifdef MORPH_CLIENT_ONLY +#define BRIDGE_REGISTER_RECORD(T, NAME, ...) +#else +#define BRIDGE_REGISTER_RECORD(T, NAME, ...) \ + namespace { \ + [[maybe_unused]] const bool BRIDGE_DETAIL_CAT(bridge_record_reg_, __COUNTER__) = \ + morph::record::detail::registerRecordOnce(NAME); \ + } +#endif diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 4b08301a5..f4e0ce23c 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -22,6 +22,7 @@ add_executable(morph_tests test_executor_scope.cpp test_io_loop.cpp test_example.cpp + test_push_records.cpp test_executor_extra.cpp test_strand.cpp test_completion.cpp diff --git a/tests/test_push_records.cpp b/tests/test_push_records.cpp new file mode 100644 index 000000000..e8ff74b27 --- /dev/null +++ b/tests/test_push_records.cpp @@ -0,0 +1,112 @@ +#include +#include +#include +#include +#include + +// Glaze reflects member names through an external-linkage object, so these types cannot sit in an +// anonymous namespace. +namespace push_records_test { + +struct Row { + std::int64_t id{}; + std::string name; +}; + +struct Row2 { + std::int64_t id{}; + std::string code; +}; + +struct SkuId { + std::string raw; + SkuId() = default; + explicit SkuId(std::string value) : raw{std::move(value)} {} + [[nodiscard]] bool hasValue() const { return !raw.empty(); } + [[nodiscard]] std::string const& operator*() const { return raw; } +}; + +struct Sku { + SkuId id; + int qty{}; +}; + +struct Report { + int total{}; +}; + +struct Secret { + std::int64_t id{}; +}; + +struct NoId { + int value{}; +}; + +} // namespace push_records_test + +using namespace push_records_test; + +BRIDGE_REGISTER_RECORD(Row, "T_Row") +BRIDGE_REGISTER_RECORD(Row2, "T_Row2", morph::record::key<&Row2::code>) +BRIDGE_REGISTER_RECORD(Sku, "T_Sku") +BRIDGE_REGISTER_RECORD(Report, "T_Report", morph::record::keyless, morph::record::derivedFrom) +BRIDGE_REGISTER_RECORD(Secret, "T_Secret", morph::record::privateKeys) +BRIDGE_REGISTER_RECORD(NoId, "T_NoId") + +namespace { + +morph::record::RecordInfo const* find(std::string_view name) { + for (auto const& info : morph::record::detail::recordRegistry()) { + if (info.name == name) { + return &info; + } + } + return nullptr; +} + +} // namespace + +TEST_CASE("record_default_key_is_id", "[push][records]") { + auto const* info = find("T_Row"); + REQUIRE(info != nullptr); + REQUIRE(info->type == std::type_index{typeid(Row)}); + REQUIRE(info->keyField == "id"); + REQUIRE_FALSE(info->keyIsString); + REQUIRE_FALSE(info->privateKeys); + REQUIRE(info->derivedFrom.empty()); +} + +TEST_CASE("record_named_key_member", "[push][records]") { + auto const* info = find("T_Row2"); + REQUIRE(info != nullptr); + REQUIRE(info->keyField == "code"); + REQUIRE(info->keyIsString); +} + +TEST_CASE("record_wrapped_key", "[push][records]") { + auto const* info = find("T_Sku"); + REQUIRE(info != nullptr); + REQUIRE(info->keyField == "id"); + REQUIRE(info->keyIsString); +} + +TEST_CASE("record_keyless_and_derived", "[push][records]") { + auto const* info = find("T_Report"); + REQUIRE(info != nullptr); + REQUIRE_FALSE(info->keyField.has_value()); + REQUIRE(info->derivedFrom == std::vector{std::type_index{typeid(Row)}}); +} + +TEST_CASE("record_private_keys", "[push][records]") { + auto const* info = find("T_Secret"); + REQUIRE(info != nullptr); + REQUIRE(info->privateKeys); + REQUIRE(info->keyField == "id"); +} + +TEST_CASE("record_without_id_has_no_key", "[push][records]") { + auto const* info = find("T_NoId"); + REQUIRE(info != nullptr); + REQUIRE_FALSE(info->keyField.has_value()); +} From aa2b4fdf5c21aeef0d0fa7b96f0b887fc094a1ae Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 11:33:03 +0200 Subject: [PATCH 12/68] core: complete the record registry's documentation Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/registry.md | 39 +++++++++++++++++----------------- include/morph/core/records.hpp | 12 ++++++++++- 2 files changed, 31 insertions(+), 20 deletions(-) diff --git a/docs/spec/core/registry.md b/docs/spec/core/registry.md index dd2c7293f..4aa0524c5 100644 --- a/docs/spec/core/registry.md +++ b/docs/spec/core/registry.md @@ -1608,6 +1608,26 @@ testing obligation, not a compile-time guarantee. for an action that is registered. A concept constraining `Sharing` to the two tags would turn that runtime surprise into a compile error; none exists today. +## Record registry + +`include/morph/core/records.hpp` holds a registry of *record types*, the rows a +server can describe changes to. `BRIDGE_REGISTER_RECORD(T, NAME, options...)` is a +static registrar like `BRIDGE_REGISTER_ACTION`'s: it asserts the registration +latch and compiles out under `MORPH_CLIENT_ONLY`. Each call appends one +`morph::record::RecordInfo` (wire name, `std::type_index`, key field, sources a +record is derived from, `privateKeys`, `keyIsString`) to a process-level vector, +read through `detail::recordRegistry()`. + +- The key defaults to the member glaze reflects as `id`; `key<&T::m>` names + another member (its glaze JSON key is recorded), `keyless` declares none. A + type with no `id`, no `key<>` and no `keyless` is recorded with no key field. +- The key member is a `ModelKey`, raw or a `WrappedModelKey` strong id. + `keyIsString` follows the raw type. +- The registry is raw: duplicate names or types, `derivedFrom` cycles and keyless + records that an effect names are not checked at registration. +- Record types need external linkage: glaze reflects member names through an + external-linkage object, so a record in an anonymous namespace does not compile. + ## Cross-references - **[bridge.md](bridge.md)** — defines `BridgeHandler`, `Bridge`, and the @@ -1644,22 +1664,3 @@ testing obligation, not a compile-time guarantee. - **Concurrency and lifetimes** — the static-init-then-read-only discipline in [Thread safety](#thread-safety) and the process-lifetime singleton ownership in [Limitations](#limitations). -## Record registry - -`include/morph/core/records.hpp` holds a registry of *record types*, the rows a -server can describe changes to. `BRIDGE_REGISTER_RECORD(T, NAME, options...)` is a -static registrar like `BRIDGE_REGISTER_ACTION`'s: it asserts the registration -latch and compiles out under `MORPH_CLIENT_ONLY`. Each call appends one -`morph::record::RecordInfo` (wire name, `std::type_index`, key field, sources a -record is derived from, `privateKeys`, `keyIsString`) to a process-level vector, -read through `detail::recordRegistry()`. - -- The key defaults to the member glaze reflects as `id`; `key<&T::m>` names - another member (its glaze JSON key is recorded), `keyless` declares none. A - type with no `id`, no `key<>` and no `keyless` is recorded with no key field. -- The key member is a `ModelKey`, raw or a `WrappedModelKey` strong id. - `keyIsString` follows the raw type. -- The registry is raw: duplicate names or types, `derivedFrom` cycles and keyless - records that an effect names are not checked at registration. -- Record types need external linkage: glaze reflects member names through an - external-linkage object, so a record in an anonymous namespace does not compile. diff --git a/include/morph/core/records.hpp b/include/morph/core/records.hpp index 9336bf8fe..df98d7ac3 100644 --- a/include/morph/core/records.hpp +++ b/include/morph/core/records.hpp @@ -1,6 +1,7 @@ // SPDX-License-Identifier: Apache-2.0 #pragma once +#include #include #include #include @@ -64,11 +65,14 @@ namespace detail { /// @tparam T An option. template struct KeyOption { + /// True for a `key<>` option. static constexpr bool value = false; }; template struct KeyOption> { + /// True for a `key<>` option. static constexpr bool value = true; + /// The member pointer the option names. static constexpr auto member = Member; }; @@ -90,6 +94,7 @@ struct MemberType { /// @brief Whether a key member's raw type is `std::string`, looking through a wrapped strong id. /// @tparam K The key member's type. +/// @return True when the raw key type is `std::string`. template constexpr bool keyTypeIsString() { using Plain = std::remove_cvref_t; @@ -126,6 +131,7 @@ template /// @brief The index of the reflected member named `id`, or `size` when there is none. /// @tparam T The record type. +/// @return The index, or `glz::reflect::keys.size()` when no member is named `id`. template constexpr std::size_t idIndex() { constexpr auto keys = glz::reflect::keys; @@ -155,7 +161,11 @@ void applyKey(RecordInfo& info) { using Raw = typename MemberType::member)>>::type; static_assert(model::ModelKey>, "BRIDGE_REGISTER_RECORD: a key member must be a ModelKey"); - target.keyField = memberKeyName(KeyOption::member); + auto name = memberKeyName(KeyOption::member); + assert(!name.empty() && "BRIDGE_REGISTER_RECORD: key<> names a member glaze does not reflect"); + if (!name.empty()) { + target.keyField = std::move(name); + } target.keyIsString = keyTypeIsString(); } }; From 404c61d784d7cdc2c7e013bf0ec0f5a516f35c6a Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 12:14:42 +0200 Subject: [PATCH 13/68] core: an action's registration records where its result holds which types Co-Authored-By: Claude Opus 5.5 --- CMakeLists.txt | 1 + docs/spec/core/registry.md | 52 +++++- include/morph/core/detail/result_walker.hpp | 185 ++++++++++++++++++++ include/morph/core/registry.hpp | 47 ++++- tests/test_push_records.cpp | 143 +++++++++++++++ 5 files changed, 422 insertions(+), 6 deletions(-) create mode 100644 include/morph/core/detail/result_walker.hpp diff --git a/CMakeLists.txt b/CMakeLists.txt index d6507e0cd..a729cb7c0 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -602,6 +602,7 @@ target_sources(morph include/morph/core/detail/owner_probe.hpp include/morph/core/detail/reply_router.hpp include/morph/core/detail/subscription_registry.hpp + include/morph/core/detail/result_walker.hpp include/morph/forms/detail/session_common.hpp include/morph/forms/detail/schema_name.hpp include/morph/forms/engine/detail/text.hpp diff --git a/docs/spec/core/registry.md b/docs/spec/core/registry.md index 4aa0524c5..b325298c0 100644 --- a/docs/spec/core/registry.md +++ b/docs/spec/core/registry.md @@ -41,6 +41,8 @@ without knowing their concrete types. - [Failure modes](#failure-modes) - [Limitations](#limitations) - [Cross-references](#cross-references) +- [Record registry](#record-registry) +- [Action result slots](#action-result-slots) ## Overview @@ -91,7 +93,8 @@ namespace scope: — which has external visibility to the type system; and 2. one or more **file-scope initialiser objects** (`[[maybe_unused]] const bool` in an anonymous namespace) whose initialisation runs - `registerModelOnce` / `registerActionOnce` / `registerActionExecutorOnce`. + `registerModelOnce` / `registerActionOnce` / `registerActionResultOnce` / + `registerActionExecutorOnce`. `BRIDGE_REGISTER_VALIDATOR` emits **only** item 1 — an `ActionValidator` specialisation. It performs no static-init registration (there is no singleton @@ -883,19 +886,20 @@ construction; nothing in the type system says so (see ### Static-init helpers -Three `detail` functions serve as static-init helpers that the macros call: +Four `detail` functions serve as static-init helpers that the macros call: | Function | Purpose | |---|---| | `registerModelOnce(modelId)` | Registers a model factory with `ModelRegistryFactory::instance()`. Returns `true` so it can be assigned to a `const bool` in an anonymous namespace. | | `registerActionOnce(modelId, actionId)` | Registers a runner with `ActionDispatcher::instance()`. Returns `true`. | +| `registerActionResultOnce(actionId)` | Walks `ActionTraits::Result` and stores its slots under the action id (see [Action result slots](#action-result-slots)). Idempotent per id. | | `registerActionExecutorOnce(modelId, actionId)` | Registers with `ActionExecuteRegistry::instance()`. Only declared in `registry.hpp`; defined in `bridge.hpp` to avoid a `registry.hpp` → `bridge.hpp` include cycle. | The process-level singletons are returned by `defaultDispatcher()` and `defaultRegistry()` (both are `inline` functions with function-local `static` variables). -All three helpers `assert` in a debug build that the +All four helpers `assert` in a debug build that the [registration-phase latch](#the-registration-phase-latch) is still open, which is what turns the thread-safety precondition below from advice into a diagnosable failure. @@ -980,6 +984,11 @@ Expands to: `detail::registerActionOnce(morph::model::ModelTraits::typeId(), NAME)` (the model-id argument is the model's registered `typeId()`, not a raw string). See [`MORPH_CLIENT_ONLY`](#morph_client_only--suppressing-model-owning-registrars). +- Unless `MORPH_CLIENT_ONLY` is defined: a `[[maybe_unused]] const bool` in an + anonymous namespace calling + `detail::registerActionResultOnce(morph::model::ActionTraits::typeId())`, + which records where the result holds which reflected aggregates (see + [Action result slots](#action-result-slots)). - A `[[maybe_unused]] const bool` in an anonymous namespace calling `detail::registerActionExecutorOnce(morph::model::ModelTraits::typeId(), NAME)` — always emitted, `MORPH_CLIENT_ONLY` or not. @@ -1246,6 +1255,11 @@ constructor and `execute()` are declared but never defined anywhere in the link succeeds when built with `MORPH_CLIENT_ONLY` defined, and fails to link (both symbols genuinely referenced) when built without it. +**The result-slot registrar is suppressed as well.** `registerActionResultOnce` +references no model code, so linking is not the reason: the slots are facts a +process that hosts models serves, and a pure client takes them from the +server's catalog. + **Must be carried on the `morph` target's `INTERFACE`, never per-consumer.** The macro changes which registrars a model header emits; two translation units disagreeing about it — one linking `registerModelOnce`'s closure, the @@ -1664,3 +1678,35 @@ read through `detail::recordRegistry()`. - **Concurrency and lifetimes** — the static-init-then-read-only discipline in [Thread safety](#thread-safety) and the process-lifetime singleton ownership in [Limitations](#limitations). + +## Action result slots + +`include/morph/core/detail/result_walker.hpp` walks an action's result type +and lists every reflected aggregate it can hold, with the JSON path to each. +`BRIDGE_DETAIL_ACTION_REGISTRARS`, which every form of `BRIDGE_REGISTER_ACTION` +and `BRIDGE_REGISTER_ACTION_SOURCE` runs, calls +`detail::registerActionResultOnce(typeId)`, which stores +`detail::walkResult::Result>()` under the action id in +`detail::actionResultSlots()`. `Result` is the unwrapped type, so a `Task` +handler contributes `R`. + +- **A slot** is `{std::type_index type, std::string path}`. Every visited + aggregate is one, the result itself at path `""`, so matching against the + record registry can later stop at the first registered record on a path. The + walker does not read the record registry. +- **Visited:** the result; the value of a `std::vector`, `std::optional` or + `std::map` (adds no path segment); the members of a type glaze writes as an + object, either by pure reflection or through a `glz::meta` object mapping. + Path segments are `glz::reflect::keys`, the names the codec writes, so a + renamed member contributes its wire name. Segments join with `.`. +- **Skipped,** with nothing recorded beneath them: `void`, scalars, strings, + any other container, a type whose `glz::meta` sets `custom_read` or + `custom_write` or maps `value` to something other than an object, and a + pure-reflection type in an anonymous namespace (glaze cannot read its member + names, since that needs an external-linkage object of the type). +- **Recursion ends** at a type already on the current path; that occurrence is + not a slot. +- **Cost.** The walk is compile-time recursion over member types, pruned to + members that can hold an aggregate, so scalar members instantiate nothing. At + static initialisation it runs once per action id: a registration repeated in + another translation unit finds the id and does not walk again. diff --git a/include/morph/core/detail/result_walker.hpp b/include/morph/core/detail/result_walker.hpp new file mode 100644 index 000000000..26bfd1826 --- /dev/null +++ b/include/morph/core/detail/result_walker.hpp @@ -0,0 +1,185 @@ +// SPDX-License-Identifier: Apache-2.0 + +#pragma once + +/// @file core/detail/result_walker.hpp +/// @brief Where an action's result holds which reflected types. +/// +/// Every action registration walks its result type once and stores, under the +/// action id, each reflected aggregate the result can hold and the JSON path to +/// it. The walk needs no record to be registered: it reports every aggregate, +/// and matching those types against the record registry happens when that +/// registry is first read. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../../detail/reflected_member.hpp" + +namespace morph::model::detail { + +/// @brief One place an action's result holds a reflected aggregate. +struct ResultSlot { + /// The aggregate's C++ type. + std::type_index type; + /// The JSON path to it: member names joined with `.`, empty for the result itself. + /// A `std::vector`, `std::optional` or `std::map` value adds no segment. + std::string path; +}; + +/// @brief Every action's result slots, by action id. +using ActionResultSlotMap = std::map, std::less<>>; + +namespace result_walker { + +/// @brief The value type a container the walker looks through holds; `void` for any other type. +/// @tparam T The type being walked. +template +struct Element { + /// The contained value type. + using type = void; +}; +template +struct Element> { + /// The contained value type. + using type = T; +}; +template +struct Element> { + /// The contained value type. + using type = T; +}; +template +struct Element> { + /// The contained value type. + using type = V; +}; + +/// @brief Whether a type's name places it in an anonymous namespace. +/// +/// Glaze reads a pure-reflection type's member names through an +/// external-linkage object, which a type without linkage cannot have, so such a +/// type cannot be reflected at all. +/// @tparam T A class type. +/// @return True when the type, or a template argument of it, has no linkage. +template +consteval bool hasAnonymousName() { + constexpr std::string_view name = glz::type_name; + return name.contains("(anonymous namespace)") || name.contains("{anonymous}") || + name.contains("`anonymous namespace'"); +} + +/// @brief A type whose members the walker visits: one glaze writes as an object, member by member. +/// +/// A `glz::meta` that replaces the codec (`custom_read`/`custom_write`, or a +/// `value` mapping) leaves the JSON shape unknown, so such a type is skipped. A +/// `glz::meta` object mapping, renamed members included, is visited. +template +concept Object = std::is_class_v && !glz::custom_read && !glz::custom_write && + (glz::glaze_object_t || (glz::reflectable && !hasAnonymousName())); + +/// @brief Whether walking a type can yield a slot, so members that cannot are not walked at all. +/// @tparam T The type being walked. +/// @return True for an `Object`, or a container the walker looks through whose value type can. +template +consteval bool mayHoldSlots() { + if constexpr (!std::is_void_v::type>) { + return mayHoldSlots::type>>(); + } else { + return Object; + } +} + +/// @brief The state of one walk. +struct Walk { + /// The slots found so far. + std::vector found; + /// The aggregates on the current path, which end the walk where they recur. + std::vector onPath; +}; + +template +void visit(Walk& walk, std::string const& path); + +/// @brief Visits each member of an `Object` that can yield a slot. +/// @tparam T The aggregate. +/// @tparam I Its reflected member indices. +/// @param walk The walk's state. +/// @param path The aggregate's own path. +template +void visitMembers(Walk& walk, std::string const& path, std::index_sequence /*indices*/) { + auto const member = [&]() { + using Member = std::remove_cvref_t(std::declval()))>; + if constexpr (mayHoldSlots()) { + constexpr std::string_view key = glz::reflect::keys[Index]; + visit(walk, path.empty() ? std::string{key} : path + "." + std::string{key}); + } + }; + (member.template operator()(), ...); +} + +/// @brief Visits a value of type @p T at @p path. +/// @tparam T The type being walked. +/// @param walk The walk's state. +/// @param path Where the value sits in the result. +template +void visit(Walk& walk, std::string const& path) { + if constexpr (!std::is_void_v::type>) { + visit::type>>(walk, path); + } else if constexpr (Object) { + std::type_index const self{typeid(T)}; + if (std::ranges::find(walk.onPath, self) != walk.onPath.end()) { + return; + } + walk.found.push_back(ResultSlot{.type = self, .path = path}); + walk.onPath.push_back(self); + visitMembers(walk, path, std::make_index_sequence::size>{}); + walk.onPath.pop_back(); + } +} + +} // namespace result_walker + +/// @brief Every reflected aggregate a value of type @p R can hold, and the path to each. +/// +/// The result itself, a `std::vector`, `std::optional` or `std::map` value of a +/// visited type, and the members of a reflected aggregate are visited, +/// depth-first. Member names are glaze's reflected keys, which are the names +/// the codec writes. `void`, scalars, strings, types whose `glz::meta` replaces +/// the codec, and pure-reflection types in an anonymous namespace are skipped. +/// The walk stops at a type already on its path, so recursive types end. +/// @tparam R The result type, `void` included. +/// @return The slots in visiting order. +template +[[nodiscard]] std::vector walkResult() { + result_walker::Walk walk; + if constexpr (!std::is_void_v) { + if constexpr (result_walker::mayHoldSlots>()) { + result_walker::visit>(walk, std::string{}); + } + } + return std::move(walk.found); +} + +/// @brief The process-level map of action result slots, filled by static registrars. +/// @return The mutable map. +inline ActionResultSlotMap& mutableActionResultSlots() { + static ActionResultSlotMap entries; + return entries; +} + +/// @brief Every registered action's result slots, by action id. +/// @return The map, unmatched against records. +inline ActionResultSlotMap const& actionResultSlots() { return mutableActionResultSlots(); } + +} // namespace morph::model::detail diff --git a/include/morph/core/registry.hpp b/include/morph/core/registry.hpp index bbfae7f29..efbcf9357 100644 --- a/include/morph/core/registry.hpp +++ b/include/morph/core/registry.hpp @@ -24,6 +24,7 @@ #include "../forms/field_errors.hpp" #include "../forms/forms.hpp" +#include "detail/result_walker.hpp" #include "model.hpp" #include "payload_schema.hpp" #include "profiler.hpp" @@ -1494,6 +1495,30 @@ inline bool registerActionOnce(std::string_view modelId, std::string_view action return true; } +/// @brief Static-init helper for every action registration: records where the +/// action's result holds which reflected aggregates. +/// +/// Walks `ActionTraits::Result`, the unwrapped type, so a `Task` +/// handler contributes `R`. Idempotent per action id: a registration repeated +/// in another translation unit does not walk again. +/// @tparam Action The action type; `ActionTraits` must be visible. +/// @param actionId The action's registered type-id. +/// @return Always true, so the result can initialise a namespace-scope variable. +template +inline bool registerActionResultOnce(std::string_view actionId) noexcept { + // NOLINTNEXTLINE(misc-static-assert,cert-dcl03-c) + assert(!::morph::model::registrationPhaseClosed() && + "registerActionResultOnce: registration after the registration phase closed. The process-level " + "registries are unsynchronised and are read-only once dispatch begins -- registering now " + "races their internals against concurrent lookups. Load and register plugin modules before " + "the first dispatch."); + auto& entries = mutableActionResultSlots(); + if (!entries.contains(actionId)) { + entries.emplace(std::string{actionId}, walkResult::Result>()); + } + return true; +} + /// @brief Static-init helper for `BRIDGE_REGISTER_ACTION`'s generic-execute registration. /// /// Forward declaration only; the definition is in `bridge.hpp` (after `ActionExecuteRegistry` @@ -1585,6 +1610,10 @@ void actionSourceRegistrationRequired() {} /// (for `ActionTraits::Result`, via `decltype`) — never its definition — so /// it is unaffected and always emitted. /// +/// The result-slot registrar (`registerActionResultOnce`) references no model +/// code, but it is suppressed too: it records facts only a process that hosts +/// models serves, and a pure client takes them from the server's catalog. +/// /// Defined on the `morph` target's INTERFACE via the `MORPH_CLIENT_ONLY` /// CMake option, never per translation unit: two TUs disagreeing on whether a /// model registers itself would violate ODR. See docs/spec/core/registry.md, @@ -1611,6 +1640,7 @@ void actionSourceRegistrationRequired() {} #ifdef MORPH_CLIENT_ONLY #define MORPH_DETAIL_REGISTER_MODEL_LOCAL(M, NAME) #define MORPH_DETAIL_REGISTER_ACTION_LOCAL(M, A, NAME) +#define MORPH_DETAIL_REGISTER_RESULT_LOCAL(A) #define MORPH_DETAIL_REQUIRE_SOURCE_LOCAL(PREFIX, ...) #define MORPH_DETAIL_DEFINE_SOURCE_LOCAL(...) #else @@ -1629,6 +1659,11 @@ void actionSourceRegistrationRequired() {} [[maybe_unused]] const bool BRIDGE_DETAIL_CAT(bridge_action_reg_, __COUNTER__) = \ morph::model::detail::registerActionOnce(morph::model::ModelTraits::typeId(), NAME); \ } +#define MORPH_DETAIL_REGISTER_RESULT_LOCAL(A) \ + namespace { \ + [[maybe_unused]] const bool BRIDGE_DETAIL_CAT(bridge_action_result_reg_, __COUNTER__) = \ + morph::model::detail::registerActionResultOnce(morph::model::ActionTraits::typeId()); \ + } #define MORPH_DETAIL_REQUIRE_SOURCE_LOCAL(PREFIX, ...) \ extern template void __VA_ARGS__; \ namespace { \ @@ -1890,13 +1925,18 @@ void actionSourceRegistrationRequired() {} }; // clang-format on -/// @brief The two registrar blocks `BRIDGE_DETAIL_ACTION_TRAITS_BODY` appends +/// @brief The registrar blocks `BRIDGE_DETAIL_ACTION_TRAITS_BODY` appends /// after `BRIDGE_DETAIL_ACTION_TRAITS_ONLY_BODY`'s trait /// specialisation, factored out so `BRIDGE_REGISTER_ACTION_SOURCE` can -/// emit exactly the same two registrars from a `.cpp`, without +/// emit exactly the same registrars from a `.cpp`, without /// repeating the trait specialisation `BRIDGE_DECLARE_ACTION` already /// placed in the header. /// +/// There are three: the dispatcher's runner (`registerActionOnce`), the +/// result's slots (`registerActionResultOnce`), both suppressed under +/// `MORPH_CLIENT_ONLY`, and the generic executor +/// (`registerActionExecutorOnce`), always emitted. +/// /// Reads `morph::model::ActionTraits::typeId()` rather than taking a /// `NAME` parameter, so a `BRIDGE_DECLARE_ACTION`/`BRIDGE_REGISTER_ACTION_SOURCE` /// pair cannot register under a different string than the one @p A's traits @@ -1908,6 +1948,7 @@ void actionSourceRegistrationRequired() {} // clang-format off -- public macro surface; see CONTRIBUTING.md, "Formatting/linting". #define BRIDGE_DETAIL_ACTION_REGISTRARS(M, A) \ MORPH_DETAIL_REGISTER_ACTION_LOCAL(M, A, morph::model::ActionTraits::typeId()) \ + MORPH_DETAIL_REGISTER_RESULT_LOCAL(A) \ namespace { \ [[maybe_unused]] const bool BRIDGE_DETAIL_CAT(bridge_action_exec_reg_, __COUNTER__) = \ morph::model::detail::registerActionExecutorOnce(morph::model::ModelTraits::typeId(), \ @@ -1920,7 +1961,7 @@ void actionSourceRegistrationRequired() {} /// `BRIDGE_REGISTER_ACTION_FOR_CLIENT_5` expand to alike, parameterised /// only on how `Result` is spelled. Composes /// `BRIDGE_DETAIL_ACTION_TRAITS_ONLY_BODY` (the type) with -/// `BRIDGE_DETAIL_ACTION_REGISTRARS` (the two registrars) -- the same +/// `BRIDGE_DETAIL_ACTION_REGISTRARS` (the registrars) -- the same /// two pieces `BRIDGE_DECLARE_ACTION`/`BRIDGE_REGISTER_ACTION_SOURCE` /// emit separately, in a header and a `.cpp` respectively. /// diff --git a/tests/test_push_records.cpp b/tests/test_push_records.cpp index e8ff74b27..07d942226 100644 --- a/tests/test_push_records.cpp +++ b/tests/test_push_records.cpp @@ -1,8 +1,14 @@ +#include #include #include +#include +#include +#include #include +#include #include #include +#include // Glaze reflects member names through an external-linkage object, so these types cannot sit in an // anonymous namespace. @@ -43,8 +49,70 @@ struct NoId { int value{}; }; +struct ListResult { + std::vector rows; +}; + +struct Header { + std::vector rows; +}; + +struct Nested { + std::optional
header; + std::map byName; +}; + +// Its codec is replaced, so the walker cannot know what its JSON holds. +struct Opaque { + std::vector rows; +}; + +// A `value` mapping replaces the object codec as well. +struct Mapped { + std::vector rows; +}; + +struct HoldsOpaque { + Opaque opaque; + Mapped mapped; + int count{}; +}; + +struct Node { + std::vector children; + std::vector rows; +}; + +struct Renamed { + std::vector items; +}; + +struct ListRows { + int limit{}; +}; + +struct WalkerModel { + ListResult execute(ListRows const& /*action*/) { return {}; } +}; + } // namespace push_records_test +template <> +struct glz::meta { + static constexpr bool custom_read = true; + static constexpr bool custom_write = true; +}; + +template <> +struct glz::meta { + static constexpr auto value = &push_records_test::Mapped::rows; +}; + +template <> +struct glz::meta { + static constexpr auto value = glz::object("entries", &push_records_test::Renamed::items); +}; + using namespace push_records_test; BRIDGE_REGISTER_RECORD(Row, "T_Row") @@ -54,6 +122,9 @@ BRIDGE_REGISTER_RECORD(Report, "T_Report", morph::record::keyless, morph::record BRIDGE_REGISTER_RECORD(Secret, "T_Secret", morph::record::privateKeys) BRIDGE_REGISTER_RECORD(NoId, "T_NoId") +BRIDGE_REGISTER_MODEL(WalkerModel, "T_WalkerModel") +BRIDGE_REGISTER_ACTION(WalkerModel, ListRows, "T_WalkerListRows") + namespace { morph::record::RecordInfo const* find(std::string_view name) { @@ -65,6 +136,17 @@ morph::record::RecordInfo const* find(std::string_view name) { return nullptr; } +// Glaze cannot read this type's member names, so the walker must skip it rather than fail to compile. +struct Hidden { + std::vector rows; +}; + +using morph::model::detail::ResultSlot; + +bool hasSlot(std::vector const& slots, std::type_index type, std::string_view path) { + return std::ranges::any_of(slots, [&](ResultSlot const& slot) { return slot.type == type && slot.path == path; }); +} + } // namespace TEST_CASE("record_default_key_is_id", "[push][records]") { @@ -110,3 +192,64 @@ TEST_CASE("record_without_id_has_no_key", "[push][records]") { REQUIRE(info != nullptr); REQUIRE_FALSE(info->keyField.has_value()); } + +TEST_CASE("walker_result_itself", "[push][records]") { + auto const slots = morph::model::detail::walkResult(); + REQUIRE(hasSlot(slots, typeid(Row), "")); + REQUIRE(slots.size() == 1); +} + +TEST_CASE("walker_vector_member", "[push][records]") { + auto const slots = morph::model::detail::walkResult(); + REQUIRE(hasSlot(slots, typeid(ListResult), "")); + REQUIRE(hasSlot(slots, typeid(Row), "rows")); + REQUIRE(slots.size() == 2); +} + +TEST_CASE("walker_paths_through_optional_and_map", "[push][records]") { + auto const slots = morph::model::detail::walkResult(); + REQUIRE(hasSlot(slots, typeid(Header), "header")); + REQUIRE(hasSlot(slots, typeid(Row), "header.rows")); + REQUIRE(hasSlot(slots, typeid(Row), "byName")); +} + +TEST_CASE("walker_skips_custom_codec", "[push][records]") { + auto const slots = morph::model::detail::walkResult(); + REQUIRE(hasSlot(slots, typeid(HoldsOpaque), "")); + REQUIRE(slots.size() == 1); + REQUIRE(morph::model::detail::walkResult().empty()); + REQUIRE(morph::model::detail::walkResult().empty()); +} + +TEST_CASE("walker_skips_anonymous_namespace_type", "[push][records]") { + REQUIRE(morph::model::detail::walkResult().empty()); + REQUIRE(morph::model::detail::walkResult>().empty()); +} + +TEST_CASE("walker_skips_scalars_strings_and_void", "[push][records]") { + REQUIRE(morph::model::detail::walkResult().empty()); + REQUIRE(morph::model::detail::walkResult().empty()); + REQUIRE(morph::model::detail::walkResult().empty()); + REQUIRE(morph::model::detail::walkResult>().empty()); +} + +TEST_CASE("walker_ends_on_recursive_type", "[push][records]") { + auto const slots = morph::model::detail::walkResult(); + REQUIRE(hasSlot(slots, typeid(Node), "")); + REQUIRE(hasSlot(slots, typeid(Row), "rows")); + REQUIRE(slots.size() == 2); +} + +TEST_CASE("walker_renamed_member_uses_wire_name", "[push][records]") { + auto const slots = morph::model::detail::walkResult(); + REQUIRE(hasSlot(slots, typeid(Row), "entries")); + REQUIRE_FALSE(hasSlot(slots, typeid(Row), "items")); +} + +TEST_CASE("action_registration_records_result_slots", "[push][records]") { + auto const& all = morph::model::detail::actionResultSlots(); + auto const found = all.find("T_WalkerListRows"); + REQUIRE(found != all.end()); + REQUIRE(hasSlot(found->second, typeid(ListResult), "")); + REQUIRE(hasSlot(found->second, typeid(Row), "rows")); +} From cd8047796149845fb6539380838f4a141e740631 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 12:27:45 +0200 Subject: [PATCH 14/68] core: a result slot remembers its action type, and a second type under one id is a conflict Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/registry.md | 22 ++++++++---- include/morph/core/detail/result_walker.hpp | 32 +++++++++++++++-- include/morph/core/registry.hpp | 23 ++++++++---- tests/test_push_records.cpp | 39 +++++++++++++++++++-- 4 files changed, 99 insertions(+), 17 deletions(-) diff --git a/docs/spec/core/registry.md b/docs/spec/core/registry.md index b325298c0..385ce5060 100644 --- a/docs/spec/core/registry.md +++ b/docs/spec/core/registry.md @@ -904,9 +904,9 @@ All four helpers `assert` in a debug build that the is what turns the thread-safety precondition below from advice into a diagnosable failure. -#### Why all three are `noexcept` while allocating +#### Why all four are `noexcept` while allocating -Each of the three builds `std::string` keys and grows a map, so each can throw +Each of the four builds `std::string` keys and grows a map, so each can throw `std::bad_alloc` through a `noexcept` boundary and call `std::terminate`. That is recorded in [Failure modes](#failure-modes) and is **deliberate, not an oversight**. @@ -1200,7 +1200,7 @@ Expands to `template <> struct morph::model::ActionValidator { static bool re ## `MORPH_CLIENT_ONLY` — suppressing model-owning registrars A pure client — one that dispatches every action to a remote peer and never -constructs a model locally — has no use for two of the three registrars +constructs a model locally — has no use for two of the registrars `BRIDGE_REGISTER_MODEL`/`BRIDGE_REGISTER_ACTION` normally emit: - `registerModelOnce` stores a factory (`[] { return ModelFactory::create(); }`) @@ -1419,12 +1419,12 @@ correctly under `MORPH_CLIENT_ONLY`. | Macro | Arguments | Generates | |---|---|---| | `BRIDGE_REGISTER_MODEL` | `(M, NAME)` | `ModelTraits` specialisation + static-init factory registration. | -| `BRIDGE_REGISTER_ACTION` | `(M, A, NAME, ...)` | `ActionTraits` specialisation + static-init dispatcher and executor registration. Optional 4th arg: `Loggable`. `Result` deduced from `decltype(M::execute(A))`, requiring `M` complete. | +| `BRIDGE_REGISTER_ACTION` | `(M, A, NAME, ...)` | `ActionTraits` specialisation + static-init dispatcher, result-slot and executor registration. Optional 4th arg: `Loggable`. `Result` deduced from `decltype(M::execute(A))`, requiring `M` complete. | | `BRIDGE_REGISTER_ACTION_FOR_CLIENT` | `(M, A, RESULT, NAME, ...)` | Same as `BRIDGE_REGISTER_ACTION`, except `Result` is the explicitly-named `RESULT` argument — `M` may be forward-declared only. See ["a header seam for `MORPH_CLIENT_ONLY`"](#bridge_register_action_for_client--a-header-seam-for-morph_client_only). | | `BRIDGE_DECLARE_MODEL` | `(M, NAME)` | `ModelTraits` specialisation + a link-time canary requiring `BRIDGE_REGISTER_MODEL_SOURCE(M)` somewhere in the link. No registration. See ["Moving a registrar out of the header"](#moving-a-registrar-out-of-the-header). | | `BRIDGE_REGISTER_MODEL_SOURCE` | `(M)` | Static-init factory registration for a `M` already declared via `BRIDGE_DECLARE_MODEL`. Reads `NAME` back off `ModelTraits::typeId()`. | | `BRIDGE_DECLARE_ACTION` | `(M, A, NAME, ...)` | `ActionTraits` specialisation + a link-time canary requiring `BRIDGE_REGISTER_ACTION_SOURCE(M, A)` somewhere in the link. Optional 4th arg: `Loggable`. No registration. | -| `BRIDGE_REGISTER_ACTION_SOURCE` | `(M, A)` | Static-init dispatcher and executor registration for an `A` already declared via `BRIDGE_DECLARE_ACTION`. Reads `NAME` back off `ActionTraits::typeId()`. | +| `BRIDGE_REGISTER_ACTION_SOURCE` | `(M, A)` | Static-init dispatcher, result-slot and executor registration for an `A` already declared via `BRIDGE_DECLARE_ACTION`. Reads `NAME` back off `ActionTraits::typeId()`. | | `BRIDGE_REGISTER_VALIDATOR` | `(A, FN)` | `ActionValidator` specialisation + custom predicate. | ### Detail helpers @@ -1462,7 +1462,7 @@ correctly under `MORPH_CLIENT_ONLY`. | `coalesce` defaults to `false` | **Every execution is a distinct, permanent fact** | The right default for anything resembling a business event. Only actions where only the latest occurrence should survive a checkpoint (e.g. a form-field edit fired repeatedly via `morph::flows::FlowSession::set`) opt in. | | `ActionDispatcher` keeps one record per pair, not four maps | **`ActionEntry` in a single `unordered_map`** | The four sub-maps were keyed identically and filled by one function, so the lockstep was real but unstated; and `RemoteServer::handle` reached three of them per request. One record makes the invariant structural and the three reads one table each. | | Enforcing the "no registration after `main`" precondition | **A debug-build latch and an `assert`, not a mutex** | Documented alone the constraint is unenforceable, and a violation's only symptom is intermittent map corruption. A latch closed by the first singleton read costs nothing in a release build and turns the `dlopen` scenario into an abort with a message. Synchronising the registries instead would put a lock on a per-request read path to legalise a startup-only operation — a trade nobody has measured. | -| `register*Once` stays `noexcept` | **Keep it, and record why** | The only caller is a namespace-scope initialiser, where an escaping exception already calls `std::terminate` ([basic.start.dynamic]). Removing `noexcept` changes nothing about an OOM at static init and adds a dead unwind path. See ["Why all three are `noexcept` while allocating"](#why-all-three-are-noexcept-while-allocating). | +| `register*Once` stays `noexcept` | **Keep it, and record why** | The only caller is a namespace-scope initialiser, where an escaping exception already calls `std::terminate` ([basic.start.dynamic]). Removing `noexcept` changes nothing about an OOM at static init and adds a dead unwind path. See ["Why all four are `noexcept` while allocating"](#why-all-four-are-noexcept-while-allocating). | | Registry lookups are heterogeneous | **Transparent `PairKeyHash`/`PairKeyEqual`, `find(PairKeyView)`** | Every caller already holds `string_view`s; materialising the stored `pair` to hash it allocated for any id past the SSO buffer, measured at 2 allocations per lookup. The transparent hash routes both overloads through one `string_view` body so lookup and stored hashes cannot drift apart — a drift that would report a registered action as unknown with no diagnostic. | ## Thread safety @@ -1552,7 +1552,7 @@ and separately `noteRegistryRead` emptied). | `dispatch` when the decoded action fails `ActionValidator::ready(...)` | Throws `morph::model::ValidationError` (a `std::runtime_error` subclass) **before** `Model::execute` runs — the action is never executed. Actions with no validator (the common case) are unaffected: `ready()` defaults to `true`. | `ActionDispatcher::registerAction`'s runner | | `create` with an unknown model id | Throws `std::runtime_error("unknown model type: …")` at runtime. | `ModelRegistryFactory::create` | | `coalesce` for an unknown pair | Does **not** throw — defaults to `false` (every entry kept). | `ActionDispatcher::coalesce` | -| Allocation failure inside a `register*Once` helper during static init | `registerModelOnce` / `registerActionOnce` (and `registerActionExecutorOnce`) are declared `noexcept` yet allocate (they build `std::string` keys and grow the map). An OOM there raises an exception through a `noexcept` boundary, which calls `std::terminate` — the process aborts during static init. This is the intended outcome rather than an accepted defect: the same OOM without `noexcept` terminates anyway, because the caller is the dynamic initialiser of a non-local variable. See ["Why all three are `noexcept` while allocating"](#why-all-three-are-noexcept-while-allocating). | `registry.hpp` | +| Allocation failure inside a `register*Once` helper during static init | `registerModelOnce` / `registerActionOnce` (and `registerActionResultOnce`, `registerActionExecutorOnce`) are declared `noexcept` yet allocate (they build `std::string` keys and grow the map). An OOM there raises an exception through a `noexcept` boundary, which calls `std::terminate` — the process aborts during static init. This is the intended outcome rather than an accepted defect: the same OOM without `noexcept` terminates anyway, because the caller is the dynamic initialiser of a non-local variable. See ["Why all four are `noexcept` while allocating"](#why-all-four-are-noexcept-while-allocating). | `registry.hpp` | | A `register*Once` call after the registration phase closes (e.g. a `dlopen`ed module registering once dispatch has begun) | **Debug build:** the `assert` fires and the process aborts with a message naming this hazard. **Release build:** unchanged — undefined behaviour, racing the map's internals against concurrent `find`s, with no diagnostic. The latch detects the violation; it does not make it safe. | `registry.hpp`, `bridge.hpp` — see ["The registration-phase latch"](#the-registration-phase-latch) | | `BRIDGE_DECLARE_MODEL`/`BRIDGE_DECLARE_ACTION` used with no matching `BRIDGE_REGISTER_MODEL_SOURCE`/`BRIDGE_REGISTER_ACTION_SOURCE` anywhere in the same link | **Fails to link** — an unresolved external symbol naming `modelSourceRegistrationRequired` / `actionSourceRegistrationRequired`, before the program ever runs. Contrast the row above: this is the one registration mistake in this table that is *not* deferred to runtime. See ["The link-time canary"](#the-link-time-canary-closing-bridge_declare_s-own-new-hazard). | `registry.hpp` | @@ -1690,6 +1690,14 @@ and `BRIDGE_REGISTER_ACTION_SOURCE` runs, calls `detail::actionResultSlots()`. `Result` is the unwrapped type, so a `Task` handler contributes `R`. +- **Keyed by action id, with the action type kept.** An entry is + `{std::type_index action, std::vector resultSlots}`. The + registry pins an action id per model, not per process, so two action types + may register under one id. A registration repeated with the same type is a + no-op. One with a different type keeps the first type's slots and lists the + id once in `detail::actionResultConflicts()`. Matching refuses a listed id, + since which result a cached query holds would otherwise depend on link + order. Recording rather than throwing keeps the registrar `noexcept`. - **A slot** is `{std::type_index type, std::string path}`. Every visited aggregate is one, the result itself at path `""`, so matching against the record registry can later stop at the first registered record on a path. The diff --git a/include/morph/core/detail/result_walker.hpp b/include/morph/core/detail/result_walker.hpp index 26bfd1826..c81062267 100644 --- a/include/morph/core/detail/result_walker.hpp +++ b/include/morph/core/detail/result_walker.hpp @@ -37,8 +37,21 @@ struct ResultSlot { std::string path; }; -/// @brief Every action's result slots, by action id. -using ActionResultSlotMap = std::map, std::less<>>; +/// @brief What one action id's registration recorded about its result. +/// +/// The action type is kept beside the slots because an action id is unique per +/// model, not per process: two action types may register under one id. The +/// first registration's slots are kept, and a later registration with a +/// different action type is listed in `actionResultConflicts()` instead. +struct ActionResult { + /// The action type that registered under the id first. + std::type_index action; + /// The places that action's result holds a reflected aggregate. + std::vector resultSlots; +}; + +/// @brief Every action's result, by action id. +using ActionResultSlotMap = std::map>; namespace result_walker { @@ -182,4 +195,19 @@ inline ActionResultSlotMap& mutableActionResultSlots() { /// @return The map, unmatched against records. inline ActionResultSlotMap const& actionResultSlots() { return mutableActionResultSlots(); } +/// @brief The process-level list of action ids registered by more than one action type. +/// @return The mutable list. +inline std::vector& mutableActionResultConflicts() { + static std::vector ids; + return ids; +} + +/// @brief The action ids that a second, different action type registered under. +/// +/// Each id appears once. Its entry in `actionResultSlots()` holds the first +/// type's slots; matching refuses the id, since which result a cached query +/// holds would depend on link order. +/// @return The ids in the order the conflicts were found. +inline std::vector const& actionResultConflicts() { return mutableActionResultConflicts(); } + } // namespace morph::model::detail diff --git a/include/morph/core/registry.hpp b/include/morph/core/registry.hpp index efbcf9357..14f56a7dc 100644 --- a/include/morph/core/registry.hpp +++ b/include/morph/core/registry.hpp @@ -1499,8 +1499,10 @@ inline bool registerActionOnce(std::string_view modelId, std::string_view action /// action's result holds which reflected aggregates. /// /// Walks `ActionTraits::Result`, the unwrapped type, so a `Task` -/// handler contributes `R`. Idempotent per action id: a registration repeated -/// in another translation unit does not walk again. +/// handler contributes `R`. Idempotent per action type: a registration repeated +/// in another translation unit does not walk again. A different action type +/// under an id already taken keeps the first type's slots and lists the id in +/// `actionResultConflicts()`. /// @tparam Action The action type; `ActionTraits` must be visible. /// @param actionId The action's registered type-id. /// @return Always true, so the result can initialise a namespace-scope variable. @@ -1512,10 +1514,19 @@ inline bool registerActionResultOnce(std::string_view actionId) noexcept { "registries are unsynchronised and are read-only once dispatch begins -- registering now " "races their internals against concurrent lookups. Load and register plugin modules before " "the first dispatch."); + std::type_index const action{typeid(Action)}; auto& entries = mutableActionResultSlots(); - if (!entries.contains(actionId)) { - entries.emplace(std::string{actionId}, walkResult::Result>()); + if (auto const existing = entries.find(actionId); existing != entries.end()) { + if (existing->second.action != action) { + auto& conflicts = mutableActionResultConflicts(); + if (std::ranges::find(conflicts, actionId) == conflicts.end()) { + conflicts.emplace_back(actionId); + } + } + return true; } + entries.emplace(std::string{actionId}, + ActionResult{.action = action, .resultSlots = walkResult::Result>()}); return true; } @@ -1604,7 +1615,7 @@ void actionSourceRegistrationRequired() {} /// build still forces the linker to resolve them, pulling in implementations /// (a database driver, a native UI framework, an OS-specific API) the client /// target may have no link path for at all, and will never call regardless. -/// The third registrar (`registerActionExecutorOnce`, `bridge.hpp`) routes +/// The executor registrar (`registerActionExecutorOnce`, `bridge.hpp`) routes /// through `Bridge::executeVia` -> `IBackend::execute` (fully abstract) and /// needs only the action's JSON codecs plus `Model::execute`'s *declaration* /// (for `ActionTraits::Result`, via `decltype`) — never its definition — so @@ -1956,7 +1967,7 @@ void actionSourceRegistrationRequired() {} } // clang-format on -/// @brief Shared body for `ActionTraits`'s specialisation plus its two +/// @brief Shared body for `ActionTraits`'s specialisation plus its /// registrars -- everything `BRIDGE_REGISTER_ACTION_4` and /// `BRIDGE_REGISTER_ACTION_FOR_CLIENT_5` expand to alike, parameterised /// only on how `Result` is spelled. Composes diff --git a/tests/test_push_records.cpp b/tests/test_push_records.cpp index 07d942226..755b965ce 100644 --- a/tests/test_push_records.cpp +++ b/tests/test_push_records.cpp @@ -91,8 +91,13 @@ struct ListRows { int limit{}; }; +struct CountRows { + int limit{}; +}; + struct WalkerModel { ListResult execute(ListRows const& /*action*/) { return {}; } + Header execute(CountRows const& /*action*/) { return {}; } }; } // namespace push_records_test @@ -124,6 +129,17 @@ BRIDGE_REGISTER_RECORD(NoId, "T_NoId") BRIDGE_REGISTER_MODEL(WalkerModel, "T_WalkerModel") BRIDGE_REGISTER_ACTION(WalkerModel, ListRows, "T_WalkerListRows") +BRIDGE_REGISTER_ACTION(WalkerModel, CountRows, "T_WalkerCountRows") + +// Another model may register a different action type under an id already taken. +namespace { +[[maybe_unused]] bool const walkerConflictFirst = + morph::model::detail::registerActionResultOnce("T_WalkerConflict"); +[[maybe_unused]] bool const walkerConflictSecond = + morph::model::detail::registerActionResultOnce("T_WalkerConflict"); +[[maybe_unused]] bool const walkerSameFirst = morph::model::detail::registerActionResultOnce("T_WalkerSame"); +[[maybe_unused]] bool const walkerSameSecond = morph::model::detail::registerActionResultOnce("T_WalkerSame"); +} // namespace namespace { @@ -208,9 +224,11 @@ TEST_CASE("walker_vector_member", "[push][records]") { TEST_CASE("walker_paths_through_optional_and_map", "[push][records]") { auto const slots = morph::model::detail::walkResult(); + REQUIRE(hasSlot(slots, typeid(Nested), "")); REQUIRE(hasSlot(slots, typeid(Header), "header")); REQUIRE(hasSlot(slots, typeid(Row), "header.rows")); REQUIRE(hasSlot(slots, typeid(Row), "byName")); + REQUIRE(slots.size() == 4); } TEST_CASE("walker_skips_custom_codec", "[push][records]") { @@ -250,6 +268,23 @@ TEST_CASE("action_registration_records_result_slots", "[push][records]") { auto const& all = morph::model::detail::actionResultSlots(); auto const found = all.find("T_WalkerListRows"); REQUIRE(found != all.end()); - REQUIRE(hasSlot(found->second, typeid(ListResult), "")); - REQUIRE(hasSlot(found->second, typeid(Row), "rows")); + REQUIRE(found->second.action == std::type_index{typeid(ListRows)}); + REQUIRE(hasSlot(found->second.resultSlots, typeid(ListResult), "")); + REQUIRE(hasSlot(found->second.resultSlots, typeid(Row), "rows")); +} + +TEST_CASE("action_result_second_type_under_one_id_is_a_conflict", "[push][records]") { + auto const& all = morph::model::detail::actionResultSlots(); + auto const& conflicts = morph::model::detail::actionResultConflicts(); + auto const conflict = all.find("T_WalkerConflict"); + REQUIRE(conflict != all.end()); + REQUIRE(conflict->second.action == std::type_index{typeid(ListRows)}); + REQUIRE(hasSlot(conflict->second.resultSlots, typeid(ListResult), "")); + REQUIRE_FALSE(hasSlot(conflict->second.resultSlots, typeid(Header), "")); + REQUIRE(std::ranges::count(conflicts, std::string{"T_WalkerConflict"}) == 1); + + auto const same = all.find("T_WalkerSame"); + REQUIRE(same != all.end()); + REQUIRE(std::ranges::find(conflicts, std::string{"T_WalkerSame"}) == conflicts.end()); + REQUIRE(std::ranges::find(conflicts, std::string{"T_WalkerListRows"}) == conflicts.end()); } From cb281df3c0f1fbf53b887ee66f686a82cb5b55e1 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 12:38:04 +0200 Subject: [PATCH 15/68] core: actions declare their effects; matched and validated on first read Co-Authored-By: Claude Opus 5.5 --- CMakeLists.txt | 1 + docs/spec/core/registry.md | 61 +- include/morph/core/effects.hpp | 538 ++++++++++++++++++ tests/CMakeLists.txt | 13 +- tests/compile_checks/effects_unseen_a.cpp | 22 + tests/compile_checks/effects_unseen_b.cpp | 10 + .../compile_checks/effects_unseen_shared.hpp | 35 ++ tests/push_records_late.cpp | 11 + tests/push_records_late.hpp | 22 + tests/test_push_effects.cpp | 301 ++++++++++ tests/test_push_records.cpp | 22 +- 11 files changed, 1029 insertions(+), 7 deletions(-) create mode 100644 include/morph/core/effects.hpp create mode 100644 tests/compile_checks/effects_unseen_a.cpp create mode 100644 tests/compile_checks/effects_unseen_b.cpp create mode 100644 tests/compile_checks/effects_unseen_shared.hpp create mode 100644 tests/push_records_late.cpp create mode 100644 tests/push_records_late.hpp create mode 100644 tests/test_push_effects.cpp diff --git a/CMakeLists.txt b/CMakeLists.txt index a729cb7c0..12d25a3e2 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -454,6 +454,7 @@ target_sources(morph include/morph/core/model.hpp include/morph/core/registry.hpp include/morph/core/records.hpp + include/morph/core/effects.hpp include/morph/core/backend.hpp include/morph/core/envelope_transport.hpp include/morph/core/remote.hpp diff --git a/docs/spec/core/registry.md b/docs/spec/core/registry.md index 385ce5060..2a1c3c92b 100644 --- a/docs/spec/core/registry.md +++ b/docs/spec/core/registry.md @@ -43,6 +43,7 @@ without knowing their concrete types. - [Cross-references](#cross-references) - [Record registry](#record-registry) - [Action result slots](#action-result-slots) +- [Effects](#effects) ## Overview @@ -1638,7 +1639,8 @@ read through `detail::recordRegistry()`. - The key member is a `ModelKey`, raw or a `WrappedModelKey` strong id. `keyIsString` follows the raw type. - The registry is raw: duplicate names or types, `derivedFrom` cycles and keyless - records that an effect names are not checked at registration. + records that an effect names are not checked at registration. They are + checked on the first read through `morph::effects` (see [Effects](#effects)). - Record types need external linkage: glaze reflects member names through an external-linkage object, so a record in an anonymous namespace does not compile. @@ -1718,3 +1720,60 @@ handler contributes `R`. members that can hold an aggregate, so scalar members instantiate nothing. At static initialisation it runs once per action id: a registration repeated in another translation unit finds the id and does not walk again. + +## Effects + +`include/morph/core/effects.hpp` records what each action changes and answers, +for every action, which records its result holds. + +- **Declaring.** `BRIDGE_REGISTER_EFFECTS(A, effects...)` is a static registrar + storing `{Kind, std::type_index?}` per effect under `ActionTraits::typeId()` + in `detail::mutableEffectsRegistry()`. The effects are + `morph::effects::updates`, `changes`, `changesInstance` and `none`. It + asserts the registration latch, compiles out under `MORPH_CLIENT_ONLY`, and + defines no specialisation, so a translation unit that does not see it is + consistent with one that does (the `morph_effects_unseen_links` probe links + two such halves). It belongs beside the action's registration, not in a + header, where it would run once per translation unit. +- **Idempotent per action.** Each action id keeps its distinct declarations; + one equal to a stored one, in any order, is not added again. Two distinct + declarations are a conflict, recorded rather than thrown so the registrar + stays `noexcept`. +- **Lazy resolution.** Nothing is matched at registration, so the order of + `BRIDGE_REGISTER_RECORD`, `BRIDGE_REGISTER_ACTION` and + `BRIDGE_REGISTER_EFFECTS` across translation units does not matter. The + first call to `effectsOf`, `recordsOf`, `recordNamed` or `actionsHolding` + resolves the three process registries once, under `std::call_once`. Every + call is a process-registry read (`noteRegistryRead(true)`), so it closes the + latch in a debug build. Resolution cannot run at the latch: + `closeRegistrationPhase()` is `noexcept`, and a release build closes it only + when the application asks. +- **Matching.** For each action, its result slots are taken in path order, and a + slot is kept when its type is a registered record and no kept slot's path + contains it (a path contains itself and every path below it; `""` contains + all). So each path stops at its first registered record, and a record's own + members are never searched for further records. Kept slots are grouped by + record into `ActionRecord{record, keyField, paths}`, giving `recordsOf`. + `actionsHolding(record)` is the inverse: the action ids, in id order, whose + `recordsOf` names the record. +- **Validation.** Resolution refuses, all problems at once: + - an action id registered by two action types (`actionResultConflicts()`); + - an action with two different effects declarations; + - `updates` or `changes` where `R` is not a registered record; + - `updates` where `R` has no key, or where the action's `recordsOf` does + not name `R`; + - a type registered as a record twice, or one name registered for two types; + - a `derivedFrom` cycle among registered records, a record derived from + itself included. A source that is not a registered record is ignored. +- **Failure.** A refusal throws `morph::model::EffectsRegistrationError`, a + `std::logic_error` whose `what()` lists every problem, one per line, each + naming the action or record. The error is kept and the same error is thrown + by every later read, so no read ever sees a partly valid registry. +- **Reads.** `effectsOf(id)` is `nullopt` for an action that declared nothing: + it reads. `effectsOf` gives each record as its wire name, empty for + `changesInstance` and `none`. `recordNamed(name)` points into the resolved + copy of the record registry, valid for the life of the process. +- **Testing.** `detail::resolveForTesting(records, slots, effects, + slotConflicts)` runs the same resolution over registries a test builds, so an + invalid configuration can be exercised without making the process registry + invalid for the rest of the suite. diff --git a/include/morph/core/effects.hpp b/include/morph/core/effects.hpp new file mode 100644 index 000000000..19abb5dc1 --- /dev/null +++ b/include/morph/core/effects.hpp @@ -0,0 +1,538 @@ +// SPDX-License-Identifier: Apache-2.0 + +#pragma once +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "detail/result_walker.hpp" +#include "records.hpp" +#include "registry.hpp" + +/// @file +/// What an action changes, and which records its result holds. +/// +/// `BRIDGE_REGISTER_EFFECTS` records an action's effects as data under its id. +/// Nothing is matched or checked at registration, so the order of record, +/// action and effects registrations across translation units does not matter. +/// The first read through `effectsOf`, `recordsOf`, `recordNamed` or +/// `actionsHolding` matches every action's result slots against the record +/// registry and validates the whole configuration, once. A configuration with +/// problems throws `morph::model::EffectsRegistrationError` naming all of them, +/// on that read and on every later one. + +namespace morph::model { + +/// @brief Thrown on a read of the effects or record registry when the registered configuration is invalid. +/// +/// Lists every problem found, one per line, not only the first. Matching runs on +/// the first read, not when the registration phase closes, because +/// `closeRegistrationPhase()` is `noexcept` and a release build closes it only +/// when the application asks. +struct EffectsRegistrationError : std::logic_error { + /// @brief Constructs the error from the problems found. + /// @param problems One sentence per problem. + explicit EffectsRegistrationError(std::vector problems) + : std::logic_error(message(problems)), _problems(std::move(problems)) {} + + /// @brief The problems found, in the order they were found. + /// @return One sentence per problem. + [[nodiscard]] std::vector const& problems() const noexcept { return _problems; } + +private: + static std::string message(std::vector const& problems) { + std::string text = "morph: the registered records and effects are invalid:"; + for (auto const& problem : problems) { + text += "\n - "; + text += problem; + } + return text; + } + + std::vector _problems; +}; + +} // namespace morph::model + +namespace morph::effects { + +/// @brief Effect: the action edits existing rows of record @p R, and its result carries them. +/// @tparam R A keyed record type that the action's result holds. +template +struct updates {}; + +/// @brief Effect: the action may create, remove or change rows of record @p R, which its result need not carry. +/// @tparam R A record type. +template +struct changes {}; + +/// @brief Effect: the state of the instance the action ran on changed. +struct changesInstance {}; + +/// @brief Effect: the action changes nothing, declared explicitly. +struct none {}; + +/// @brief The kind of one declared effect. +enum class Kind : std::uint8_t { + Updates, ///< `updates` + Changes, ///< `changes` + ChangesInstance, ///< `changesInstance` + None, ///< `none` +}; + +/// @brief One effect of an action, with its record resolved to a wire name. +struct Effect { + /// What the effect is. + Kind kind; + /// The record's wire name; empty for `ChangesInstance` and `None`. + std::string record; + + /// @brief Memberwise equality. + /// @param lhs One effect. + /// @param rhs The other. + /// @return True when kind and record match. + friend bool operator==(Effect const& lhs, Effect const& rhs) = default; +}; + +/// @brief A record an action's result holds, and where. +struct ActionRecord { + /// The record's wire name. + std::string record; + /// The record's key field, empty for a keyless record. + std::optional keyField; + /// The JSON paths at which the result holds the record; `""` is the result itself. + std::vector paths; + + /// @brief Memberwise equality. + /// @param lhs One entry. + /// @param rhs The other. + /// @return True when record, key field and paths match. + friend bool operator==(ActionRecord const& lhs, ActionRecord const& rhs) = default; +}; + +namespace detail { + +/// @brief One effect as a registrar records it: the record still a type, not yet a name. +struct DeclaredEffect { + /// What the effect is. + Kind kind; + /// The record type, for `Updates` and `Changes`. + std::optional record; + + /// @brief Memberwise equality. + /// @param lhs One effect. + /// @param rhs The other. + /// @return True when kind and record match. + friend bool operator==(DeclaredEffect const& lhs, DeclaredEffect const& rhs) = default; +}; + +/// @brief Every distinct effects declaration, by action id. More than one for an id is a conflict. +using EffectsMap = std::map>, std::less<>>; + +/// @brief Maps an effect tag type to its `DeclaredEffect`. +/// @tparam E An effect tag. +template +struct EffectOf { + static_assert(sizeof(E) == 0, + "BRIDGE_REGISTER_EFFECTS: an effect is updates, changes, changesInstance or none"); +}; +template +struct EffectOf> { + /// @brief The declared effect. + /// @return `Updates` on @p R. + static DeclaredEffect value() { return {.kind = Kind::Updates, .record = std::type_index{typeid(R)}}; } +}; +template +struct EffectOf> { + /// @brief The declared effect. + /// @return `Changes` on @p R. + static DeclaredEffect value() { return {.kind = Kind::Changes, .record = std::type_index{typeid(R)}}; } +}; +template <> +struct EffectOf { + /// @brief The declared effect. + /// @return `ChangesInstance`. + static DeclaredEffect value() { return {.kind = Kind::ChangesInstance, .record = std::nullopt}; } +}; +template <> +struct EffectOf { + /// @brief The declared effect. + /// @return `None`. + static DeclaredEffect value() { return {.kind = Kind::None, .record = std::nullopt}; } +}; + +/// @brief Adds one declaration for an action, unless an equal one is already there. +/// +/// Two declarations are equal when they hold the same effects in any order, so +/// one repeated in several translation units is stored once. +/// @param map The registry to add to. +/// @param actionId The action's id. +/// @param declared The effects. +inline void addEffects(EffectsMap& map, std::string_view actionId, std::vector declared) { + auto entry = map.find(actionId); + if (entry == map.end()) { + entry = map.emplace(std::string{actionId}, std::vector>{}).first; + } + auto& declarations = entry->second; + bool const known = std::ranges::any_of(declarations, [&](std::vector const& existing) { + return std::ranges::is_permutation(existing, declared); + }); + if (!known) { + declarations.push_back(std::move(declared)); + } +} + +/// @brief The process-level effects registry, appended to by static registrars. +/// @return The mutable map. +inline EffectsMap& mutableEffectsRegistry() { + static EffectsMap entries; + return entries; +} + +/// @brief Static-init helper for `BRIDGE_REGISTER_EFFECTS`. +/// @tparam Action The action type; `ActionTraits` must be visible. +/// @tparam E The effect tags. +/// @return Always true, so the result can initialise a namespace-scope variable. +template +inline bool registerEffectsOnce() noexcept { + static_assert(sizeof...(E) > 0, "BRIDGE_REGISTER_EFFECTS: declare at least one effect"); + // NOLINTNEXTLINE(misc-static-assert,cert-dcl03-c) + assert(!::morph::model::registrationPhaseClosed() && + "registerEffectsOnce: registration after the registration phase closed. The process-level " + "registries are unsynchronised and are read-only once dispatch begins -- registering now " + "races their internals against concurrent lookups. Load and register plugin modules before " + "the first dispatch."); + addEffects(mutableEffectsRegistry(), ::morph::model::ActionTraits::typeId(), + std::vector{EffectOf::value()...}); + return true; +} + +/// @brief The matched and validated registries: what every read is answered from. +struct Resolved { + /// @brief The records, as registered. + std::vector records; + /// @brief Each action's effects, by action id; actions that declared none are absent. + std::map, std::less<>> effects; + /// @brief Each action's records, by action id; actions whose result holds none are absent. + std::map, std::less<>> actionRecords; + /// @brief The ids of the actions holding each record, by record name, in id order. + std::map, std::less<>> holding; + + /// @brief See `morph::effects::effectsOf`. + /// @param actionId The action's id. + /// @return Its effects, or `nullopt` when it declared none. + [[nodiscard]] std::optional> effectsOf(std::string_view actionId) const { + if (auto const found = effects.find(actionId); found != effects.end()) { + return found->second; + } + return std::nullopt; + } + + /// @brief See `morph::effects::recordsOf`. + /// @param actionId The action's id. + /// @return The records its result holds. + [[nodiscard]] std::vector recordsOf(std::string_view actionId) const { + if (auto const found = actionRecords.find(actionId); found != actionRecords.end()) { + return found->second; + } + return {}; + } + + /// @brief See `morph::effects::recordNamed`. + /// @param name A record's wire name. + /// @return The record, or null. + [[nodiscard]] record::RecordInfo const* recordNamed(std::string_view name) const { + auto const found = std::ranges::find(records, name, &record::RecordInfo::name); + return found == records.end() ? nullptr : &*found; + } + + /// @brief See `morph::effects::actionsHolding`. + /// @param record A record's wire name. + /// @return The ids of the actions whose result holds it. + [[nodiscard]] std::vector actionsHolding(std::string_view record) const { + if (auto const found = holding.find(record); found != holding.end()) { + return found->second; + } + return {}; + } +}; + +/// @brief Whether @p prefix is @p path or a JSON path that contains it. +/// @param prefix A shorter path; `""` contains every path. +/// @param path The path. +/// @return True when @p path is at or below @p prefix. +[[nodiscard]] inline bool pathWithin(std::string_view prefix, std::string_view path) { + if (prefix.empty() || prefix == path) { + return true; + } + return path.size() > prefix.size() && path.starts_with(prefix) && path[prefix.size()] == '.'; +} + +/// @brief Appends a problem for every `derivedFrom` cycle among the records. +/// @param records The records, as registered. +/// @param indexOf Each record type's first registration. +/// @param problems Where to append. +inline void findDerivationCycles(std::vector const& records, + std::map const& indexOf, + std::vector& problems) { + enum class Mark : std::uint8_t { Unvisited, OnPath, Done }; + std::vector marks(records.size(), Mark::Unvisited); + std::vector path; + auto const visit = [&](auto const& self, std::size_t index) -> void { + marks[index] = Mark::OnPath; + path.push_back(index); + for (auto const& source : records[index].derivedFrom) { + auto const found = indexOf.find(source); + if (found == indexOf.end()) { + continue; + } + if (marks[found->second] == Mark::OnPath) { + std::string chain; + auto const start = std::ranges::find(path, found->second); + for (auto step = start; step != path.end(); ++step) { + chain += records[*step].name + " -> "; + } + chain += records[found->second].name; + problems.push_back("record `" + records[found->second].name + "` is derived from itself: " + chain); + } else if (marks[found->second] == Mark::Unvisited) { + self(self, found->second); + } + } + path.pop_back(); + marks[index] = Mark::Done; + }; + for (auto const& [type, index] : indexOf) { + if (marks[index] == Mark::Unvisited) { + visit(visit, index); + } + } +} + +/// @brief Matches action results against records and validates the configuration. +/// @param records The records, as registered. +/// @param slots Every action's result slots, by action id. +/// @param declared Every action's effects declarations, by action id. +/// @param slotConflicts Action ids registered by more than one action type. +/// @return The resolved registries. +/// @throws morph::model::EffectsRegistrationError listing every problem, when there is one. +[[nodiscard]] inline Resolved resolve(std::vector records, + model::detail::ActionResultSlotMap const& slots, EffectsMap const& declared, + std::vector const& slotConflicts) { + std::vector problems; + + std::map indexOf; + std::map nameIndex; + for (std::size_t index = 0; index < records.size(); ++index) { + auto const& info = records[index]; + if (auto const [held, added] = indexOf.emplace(info.type, index); !added) { + problems.push_back(std::string{"type "} + info.type.name() + " is registered as a record twice (`" + + records[held->second].name + "` and `" + info.name + "`)"); + } + if (auto const [held, added] = nameIndex.emplace(info.name, index); !added) { + if (records[held->second].type != info.type) { + problems.push_back("record name `" + info.name + "` is registered for two types (" + + records[held->second].type.name() + " and " + info.type.name() + ")"); + } + } + } + findDerivationCycles(records, indexOf, problems); + + for (auto const& id : slotConflicts) { + problems.push_back("action `" + id + "` is registered by two action types; give each its own id"); + } + + Resolved resolved; + for (auto const& [actionId, result] : slots) { + std::vector sorted; + sorted.reserve(result.resultSlots.size()); + for (auto const& slot : result.resultSlots) { + sorted.push_back(&slot); + } + // A prefix sorts before every path below it, so a record is always kept before what it contains. + std::ranges::stable_sort(sorted, std::less<>{}, [](auto const* slot) { return std::string_view{slot->path}; }); + std::vector kept; + std::vector matched; + for (auto const* slot : sorted) { + auto const found = indexOf.find(slot->type); + if (found == indexOf.end()) { + continue; + } + if (std::ranges::any_of(kept, [&](std::string_view prefix) { return pathWithin(prefix, slot->path); })) { + continue; + } + kept.emplace_back(slot->path); + auto const& info = records[found->second]; + auto entry = std::ranges::find(matched, info.name, &ActionRecord::record); + if (entry == matched.end()) { + matched.push_back(ActionRecord{.record = info.name, .keyField = info.keyField, .paths = {}}); + entry = std::prev(matched.end()); + } + entry->paths.push_back(slot->path); + } + if (!matched.empty()) { + for (auto const& entry : matched) { + resolved.holding[entry.record].push_back(actionId); + } + resolved.actionRecords.emplace(actionId, std::move(matched)); + } + } + + for (auto const& [actionId, declarations] : declared) { + if (declarations.size() > 1) { + problems.push_back("action `" + actionId + "` declares different effects in two registrations"); + continue; + } + std::vector effects; + for (auto const& effect : declarations.front()) { + if (!effect.record) { + effects.push_back(Effect{.kind = effect.kind, .record = {}}); + continue; + } + char const* const kindName = effect.kind == Kind::Updates ? "updates" : "changes"; + auto const found = indexOf.find(*effect.record); + if (found == indexOf.end()) { + problems.push_back("action `" + actionId + "` declares " + kindName + "<" + effect.record->name() + + ">, which is not a registered record"); + continue; + } + auto const& info = records[found->second]; + if (effect.kind == Kind::Updates) { + if (!info.keyField) { + problems.push_back("action `" + actionId + "` declares updates on record `" + info.name + + "`, which has no key"); + } + auto const held = resolved.actionRecords.find(actionId); + if (held == resolved.actionRecords.end() || + std::ranges::find(held->second, info.name, &ActionRecord::record) == held->second.end()) { + problems.push_back("action `" + actionId + "` declares updates on record `" + info.name + + "`, but its result holds no `" + info.name + "`"); + } + } + effects.push_back(Effect{.kind = effect.kind, .record = info.name}); + } + resolved.effects.emplace(actionId, std::move(effects)); + } + + if (!problems.empty()) { + throw model::EffectsRegistrationError(std::move(problems)); + } + resolved.records = std::move(records); + return resolved; +} + +/// @brief Resolves a registry built by a test, leaving the process registry untouched. +/// @param records The records. +/// @param slots The action result slots, by action id. +/// @param declared The effects declarations, by action id. +/// @param slotConflicts Action ids registered by more than one action type. +/// @return The resolved registries. +/// @throws morph::model::EffectsRegistrationError listing every problem, when there is one. +[[nodiscard]] inline Resolved resolveForTesting(std::vector records, + model::detail::ActionResultSlotMap const& slots, + EffectsMap const& declared, + std::vector const& slotConflicts = {}) { + return resolve(std::move(records), slots, declared, slotConflicts); +} + +/// @brief The process registries, matched and validated on the first call. +/// +/// Every call is a read of a process-level registry and closes the +/// registration latch in a debug build. The first call resolves, under +/// `std::call_once`; an invalid configuration's error is kept and thrown again +/// on every call. +/// @return The resolved registries. +/// @throws morph::model::EffectsRegistrationError when the registered configuration is invalid. +[[nodiscard]] inline Resolved const& resolved() { + model::detail::noteRegistryRead(true); + struct State { + std::once_flag once; + std::optional value; + std::optional error; + }; + static State state; + std::call_once(state.once, [] { + try { + state.value.emplace(resolve(record::detail::recordRegistry(), model::detail::actionResultSlots(), + mutableEffectsRegistry(), model::detail::actionResultConflicts())); + } catch (model::EffectsRegistrationError const& error) { + state.error.emplace(error); + } + }); + if (state.error) { + throw *state.error; + } + return *state.value; +} + +} // namespace detail + +/// @brief The effects an action declared. +/// @param actionId The action's id. +/// @return Its effects, or `nullopt` when it declared none: the action reads. +/// @throws morph::model::EffectsRegistrationError when the registered configuration is invalid. +[[nodiscard]] inline std::optional> effectsOf(std::string_view actionId) { + return detail::resolved().effectsOf(actionId); +} + +/// @brief The records an action's result holds, with the paths to each. +/// +/// Each path stops at the first registered record on it, so the members of a +/// record are not searched for further records. +/// @param actionId The action's id. +/// @return One entry per record, in the order their first path sorts; empty for an unknown action. +/// @throws morph::model::EffectsRegistrationError when the registered configuration is invalid. +[[nodiscard]] inline std::vector recordsOf(std::string_view actionId) { + return detail::resolved().recordsOf(actionId); +} + +/// @brief A registered record, by wire name. +/// @param name The record's wire name. +/// @return The record, valid for the life of the process, or null when no record has that name. +/// @throws morph::model::EffectsRegistrationError when the registered configuration is invalid. +[[nodiscard]] inline record::RecordInfo const* recordNamed(std::string_view name) { + return detail::resolved().recordNamed(name); +} + +/// @brief The actions whose result holds a record: those whose `recordsOf` names it. +/// @param record The record's wire name. +/// @return The action ids, in id order; empty for a record no result holds. +/// @throws morph::model::EffectsRegistrationError when the registered configuration is invalid. +[[nodiscard]] inline std::vector actionsHolding(std::string_view record) { + return detail::resolved().actionsHolding(record); +} + +} // namespace morph::effects + +/// @def BRIDGE_REGISTER_EFFECTS +/// @brief Declares what action @p A changes: any of `morph::effects::updates`, `changes`, +/// `changesInstance` and `none`. +/// +/// Place it beside the action's registration, where `ActionTraits` is visible. It defines no +/// specialisation, so a translation unit that does not see it is consistent with one that does. Repeating +/// the same effects is a no-op; different effects for one action are refused on the first read. Expands to +/// nothing under `MORPH_CLIENT_ONLY`. +/// @param A The action type. +/// @param ... The effects. +#ifdef MORPH_CLIENT_ONLY +#define BRIDGE_REGISTER_EFFECTS(A, ...) +#else +#define BRIDGE_REGISTER_EFFECTS(A, ...) \ + namespace { \ + [[maybe_unused]] const bool BRIDGE_DETAIL_CAT(bridge_effects_reg_, __COUNTER__) = \ + morph::effects::detail::registerEffectsOnce(); \ + } +#endif diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index f4e0ce23c..f00fbb0cc 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -23,6 +23,8 @@ add_executable(morph_tests test_io_loop.cpp test_example.cpp test_push_records.cpp + test_push_effects.cpp + push_records_late.cpp test_executor_extra.cpp test_strand.cpp test_completion.cpp @@ -587,6 +589,14 @@ add_test(NAME morph_duplicate_model_id -P ${CMAKE_CURRENT_SOURCE_DIR}/compile_checks/expect_message.cmake) set_tests_properties(morph_duplicate_model_id PROPERTIES TIMEOUT 60) +# An action's effects registered in one translation unit and not seen by another that registers the same action: +# BRIDGE_REGISTER_EFFECTS defines no specialisation, so the program links, and both halves read the same effects. +add_executable(morph_effects_unseen_links compile_checks/effects_unseen_a.cpp compile_checks/effects_unseen_b.cpp) +target_link_libraries(morph_effects_unseen_links PRIVATE morph_guard_probe_deps) +morph_suppress_test_dialogs(morph_effects_unseen_links) +add_test(NAME morph_effects_unseen_links COMMAND morph_effects_unseen_links) +set_tests_properties(morph_effects_unseen_links PROPERTIES TIMEOUT 60) + # The negative probe stays a configure-time try_compile(): the link failure is # the assertion. Its scratch project cannot link core-cpp either, so a failure # alone would prove nothing; the check is that the linker named @@ -706,7 +716,8 @@ set_tests_properties(morph_client_only_facade PROPERTIES TIMEOUT 60) # program linking them fails with the sanitizer runtime undefined. if(DEFINED AF_SANITIZER) foreach(_morph_guard_probe IN ITEMS morph_client_only_guard_links morph_client_only_runtime_throw - morph_client_only_facade morph_declare_source_canary) + morph_client_only_facade morph_declare_source_canary + morph_effects_unseen_links) apply_sanitizers(${_morph_guard_probe} ${AF_SANITIZER}) endforeach() unset(_morph_guard_probe) diff --git a/tests/compile_checks/effects_unseen_a.cpp b/tests/compile_checks/effects_unseen_a.cpp new file mode 100644 index 000000000..ebb57e28a --- /dev/null +++ b/tests/compile_checks/effects_unseen_a.cpp @@ -0,0 +1,22 @@ +// SPDX-License-Identifier: Apache-2.0 +// +// Half of a program in which only one translation unit sees an action's +// BRIDGE_REGISTER_EFFECTS. It defines no specialisation, so the two halves link +// into one program and read the same effects. + +#include +#include "effects_unseen_shared.hpp" + +BRIDGE_REGISTER_RECORD(effects_unseen::Row, "Unseen_Row") +BRIDGE_REGISTER_EFFECTS(effects_unseen::EditRows, morph::effects::updates) + +int main() { + std::vector const expected{{morph::effects::Kind::Updates, "Unseen_Row"}}; + auto const seenHere = morph::effects::effectsOf("Unseen_EditRows"); + auto const seenFromB = effects_unseen::effectsSeenFromB(); + if (seenHere != expected || seenFromB != expected) { + std::puts("effects differ between the translation units, or were not registered"); + return 1; + } + return 0; +} diff --git a/tests/compile_checks/effects_unseen_b.cpp b/tests/compile_checks/effects_unseen_b.cpp new file mode 100644 index 000000000..b6a7e7d58 --- /dev/null +++ b/tests/compile_checks/effects_unseen_b.cpp @@ -0,0 +1,10 @@ +// SPDX-License-Identifier: Apache-2.0 +// +// The half of morph_effects_unseen_links that registers the action but never +// sees its BRIDGE_REGISTER_EFFECTS. + +#include "effects_unseen_shared.hpp" + +std::optional> effects_unseen::effectsSeenFromB() { + return morph::effects::effectsOf("Unseen_EditRows"); +} diff --git a/tests/compile_checks/effects_unseen_shared.hpp b/tests/compile_checks/effects_unseen_shared.hpp new file mode 100644 index 000000000..f09e2fa8c --- /dev/null +++ b/tests/compile_checks/effects_unseen_shared.hpp @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: Apache-2.0 +// +// An action registered in a header, as an application's model header would. Both +// effects_unseen_a.cpp and effects_unseen_b.cpp include it; only the first also +// sees the action's BRIDGE_REGISTER_EFFECTS. +#pragma once + +#include +#include +#include +#include +#include +#include + +namespace effects_unseen { + +struct Row { + std::int64_t id{}; +}; + +struct EditRows { + int count{}; +}; + +struct RowsModel { + std::vector execute(EditRows const& /*action*/) { return {}; } +}; + +/// The effects of `Unseen_EditRows` as read from effects_unseen_b.cpp, which does not see their registration. +std::optional> effectsSeenFromB(); + +} // namespace effects_unseen + +BRIDGE_REGISTER_MODEL(effects_unseen::RowsModel, "Unseen_RowsModel") +BRIDGE_REGISTER_ACTION(effects_unseen::RowsModel, effects_unseen::EditRows, "Unseen_EditRows") diff --git a/tests/push_records_late.cpp b/tests/push_records_late.cpp new file mode 100644 index 000000000..4f4386995 --- /dev/null +++ b/tests/push_records_late.cpp @@ -0,0 +1,11 @@ +// SPDX-License-Identifier: Apache-2.0 +// +// Registers a record after the action that returns it and the effects that name +// it: listed after test_push_effects.cpp in morph_tests, so its static +// initialiser runs later. Matching is lazy, so the order must not matter. + +#include "push_records_late.hpp" + +#include + +BRIDGE_REGISTER_RECORD(push_records_late::LateRow, "T_LateRow") diff --git a/tests/push_records_late.hpp b/tests/push_records_late.hpp new file mode 100644 index 000000000..0d74c1c31 --- /dev/null +++ b/tests/push_records_late.hpp @@ -0,0 +1,22 @@ +// SPDX-License-Identifier: Apache-2.0 +// +// A record type whose registration sits in a translation unit linked after the +// action that returns it (push_records_late.cpp, after test_push_effects.cpp). +#pragma once + +#include +#include +#include + +namespace push_records_late { + +struct LateRow { + std::int64_t id{}; + std::string label; +}; + +struct LateList { + std::vector rows; +}; + +} // namespace push_records_late diff --git a/tests/test_push_effects.cpp b/tests/test_push_effects.cpp new file mode 100644 index 000000000..37f12aa35 --- /dev/null +++ b/tests/test_push_effects.cpp @@ -0,0 +1,301 @@ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "push_records_late.hpp" + +// Glaze reflects member names through an external-linkage object, so these types cannot sit in an +// anonymous namespace. +namespace push_effects_test { + +struct Item { + std::int64_t id{}; + std::string name; +}; + +struct ItemList { + std::vector items; +}; + +struct EditItem { + std::int64_t id{}; +}; + +struct LoadItems { + int limit{}; +}; + +struct OpenItems { + int limit{}; +}; + +struct ListLate { + int limit{}; +}; + +struct EffectsModel { + ItemList execute(EditItem const& /*action*/) { return {}; } + ItemList execute(LoadItems const& /*action*/) { return {}; } + ItemList execute(OpenItems const& /*action*/) { return {}; } + push_records_late::LateList execute(ListLate const& /*action*/) { return {}; } +}; + +} // namespace push_effects_test + +using namespace push_effects_test; +namespace effects = morph::effects; + +BRIDGE_REGISTER_RECORD(Item, "T_EItem") + +BRIDGE_REGISTER_MODEL(EffectsModel, "T_EffectsModel") +BRIDGE_REGISTER_ACTION(EffectsModel, EditItem, "T_EffectsEdit") +BRIDGE_REGISTER_EFFECTS(EditItem, effects::updates, effects::changes) +// The same declaration again, as a header included by two translation units would give. +BRIDGE_REGISTER_EFFECTS(EditItem, effects::updates, effects::changes) +BRIDGE_REGISTER_ACTION(EffectsModel, LoadItems, "T_EffectsLoad") +BRIDGE_REGISTER_EFFECTS(LoadItems, effects::changesInstance) +BRIDGE_REGISTER_ACTION(EffectsModel, OpenItems, "T_EffectsOpen") +BRIDGE_REGISTER_EFFECTS(OpenItems, effects::none) + +// The record this action holds and its effects name is registered in push_records_late.cpp, whose +// static initialisers run after this file's. +BRIDGE_REGISTER_ACTION(EffectsModel, ListLate, "T_EffectsListLate") +BRIDGE_REGISTER_EFFECTS(ListLate, effects::updates) + +namespace { + +// Whether the late record was already registered when this file's registrars ran. The ordering test +// proves something only when it was not. +bool const lateRecordSeenEarly = + std::ranges::any_of(morph::record::detail::recordRegistry(), + [](morph::record::RecordInfo const& info) { return info.name == "T_LateRow"; }); + +// Types used only for their identity in registries built by the tests. +struct VRow {}; +struct VKeyless {}; +struct VOther {}; +struct VList {}; +struct VHeader {}; +struct VInner {}; +struct VCycleA {}; +struct VCycleB {}; +struct VActionA {}; +struct VActionB {}; + +using morph::effects::Kind; +using morph::effects::detail::DeclaredEffect; +using morph::effects::detail::EffectsMap; +using morph::model::detail::ActionResult; +using morph::model::detail::ActionResultSlotMap; +using morph::model::detail::ResultSlot; +using morph::record::RecordInfo; + +RecordInfo record(std::string name, std::type_index type, std::optional keyField = "id", + std::vector derivedFrom = {}) { + return RecordInfo{.name = std::move(name), + .type = type, + .keyField = std::move(keyField), + .derivedFrom = std::move(derivedFrom), + .privateKeys = false, + .keyIsString = false}; +} + +ActionResult result(std::vector slots) { + return ActionResult{.action = typeid(VActionA), .resultSlots = std::move(slots)}; +} + +DeclaredEffect updatesOf(std::type_index type) { return DeclaredEffect{.kind = Kind::Updates, .record = type}; } +DeclaredEffect changesOf(std::type_index type) { return DeclaredEffect{.kind = Kind::Changes, .record = type}; } + +// Fails the test when resolving does not throw; returns the message when it does. +std::string refusal(std::vector records, ActionResultSlotMap slots, EffectsMap declared, + std::vector slotConflicts = {}) { + try { + static_cast(morph::effects::detail::resolveForTesting(std::move(records), std::move(slots), + std::move(declared), std::move(slotConflicts))); + } catch (morph::model::EffectsRegistrationError const& error) { + return error.what(); + } + FAIL("resolving did not throw EffectsRegistrationError"); + return {}; +} + +} // namespace + +// ── The process registry ───────────────────────────────────────────────────── + +TEST_CASE("records_of_list_action", "[push][records]") { + auto const records = effects::recordsOf("T_WalkerListRows"); + REQUIRE(records == std::vector{{"T_Row", "id", {"rows"}}}); +} + +TEST_CASE("registration_order_irrelevant", "[push][records]") { + // Static initialisers ran in link order: this file's before push_records_late.cpp's. + REQUIRE_FALSE(lateRecordSeenEarly); + REQUIRE(effects::recordsOf("T_EffectsListLate") == + std::vector{{"T_LateRow", "id", {"rows"}}}); + REQUIRE(effects::effectsOf("T_EffectsListLate") == + std::vector{{effects::Kind::Updates, "T_LateRow"}}); +} + +TEST_CASE("effects_declared_are_read_back", "[push][records]") { + REQUIRE(effects::effectsOf("T_EffectsEdit") == + std::vector{{effects::Kind::Updates, "T_EItem"}, {effects::Kind::Changes, "T_EItem"}}); + REQUIRE(effects::effectsOf("T_EffectsLoad") == std::vector{{effects::Kind::ChangesInstance, ""}}); + REQUIRE(effects::effectsOf("T_EffectsOpen") == std::vector{{effects::Kind::None, ""}}); + REQUIRE(effects::recordsOf("T_EffectsEdit") == std::vector{{"T_EItem", "id", {"items"}}}); +} + +TEST_CASE("unregistered_action_reads", "[push][records]") { + REQUIRE_FALSE(effects::effectsOf("NoSuch").has_value()); + REQUIRE(effects::recordsOf("NoSuch").empty()); + // Registered, but nothing declared: the action reads. + REQUIRE_FALSE(effects::effectsOf("T_WalkerListRows").has_value()); +} + +TEST_CASE("record_named_finds_registered_records", "[push][records]") { + auto const* row = effects::recordNamed("T_Row"); + REQUIRE(row != nullptr); + REQUIRE(row->keyField == "id"); + REQUIRE(effects::recordNamed("T_NoSuchRecord") == nullptr); +} + +TEST_CASE("actions_holding_a_record", "[push][records]") { + auto const holding = effects::actionsHolding("T_EItem"); + REQUIRE(std::ranges::find(holding, std::string{"T_EffectsEdit"}) != holding.end()); + REQUIRE(std::ranges::find(holding, std::string{"T_EffectsLoad"}) != holding.end()); + REQUIRE(std::ranges::find(holding, std::string{"T_WalkerListRows"}) == holding.end()); + REQUIRE(effects::actionsHolding("T_NoSuchRecord").empty()); +} + +TEST_CASE("reading_effects_closes_the_registration_phase", "[push][records]") { +#ifndef NDEBUG + morph::model::detail::reopenRegistrationPhaseForTesting(); + static_cast(effects::effectsOf("T_EffectsEdit")); + REQUIRE(morph::model::registrationPhaseClosed()); +#endif +} + +// ── Registries built by the test ───────────────────────────────────────────── + +TEST_CASE("matching_stops_at_first_record_on_a_path", "[push][records]") { + ActionResultSlotMap slots; + slots.emplace("A_Nested", result({{typeid(VList), ""}, + {typeid(VHeader), "header"}, + {typeid(VRow), "header.rows"}, + {typeid(VInner), "header.rows.inner"}, + {typeid(VRow), "loose"}})); + slots.emplace("A_Root", result({{typeid(VHeader), ""}, {typeid(VRow), "rows"}})); + auto const resolved = morph::effects::detail::resolveForTesting( + {record("V_Row", typeid(VRow)), record("V_Inner", typeid(VInner)), record("V_Header", typeid(VHeader), {})}, + std::move(slots), {}); + + REQUIRE(resolved.recordsOf("A_Nested") == std::vector{ + {"V_Header", std::nullopt, {"header"}}, {"V_Row", "id", {"loose"}}}); + REQUIRE(resolved.recordsOf("A_Root") == + std::vector{{"V_Header", std::nullopt, {""}}}); + REQUIRE(resolved.actionsHolding("V_Row") == std::vector{"A_Nested"}); + REQUIRE(resolved.actionsHolding("V_Header") == std::vector{"A_Nested", "A_Root"}); + REQUIRE(resolved.actionsHolding("V_Inner").empty()); +} + +TEST_CASE("effects_twice_same_is_noop", "[push][records]") { + EffectsMap declared; + morph::effects::detail::addEffects(declared, "A_List", {updatesOf(typeid(VRow)), changesOf(typeid(VRow))}); + morph::effects::detail::addEffects(declared, "A_List", {updatesOf(typeid(VRow)), changesOf(typeid(VRow))}); + REQUIRE(declared.at("A_List").size() == 1); + + ActionResultSlotMap slots; + slots.emplace("A_List", result({{typeid(VList), ""}, {typeid(VRow), "rows"}})); + auto const resolved = morph::effects::detail::resolveForTesting({record("V_Row", typeid(VRow))}, std::move(slots), + std::move(declared)); + REQUIRE(resolved.effectsOf("A_List") == + std::vector{{Kind::Updates, "V_Row"}, {Kind::Changes, "V_Row"}}); +} + +TEST_CASE("effects_conflict_throws", "[push][records]") { + EffectsMap declared; + morph::effects::detail::addEffects(declared, "A_Conflicted", {changesOf(typeid(VRow))}); + morph::effects::detail::addEffects(declared, "A_Conflicted", {DeclaredEffect{.kind = Kind::None, .record = {}}}); + REQUIRE(declared.at("A_Conflicted").size() == 2); + auto const message = refusal({record("V_Row", typeid(VRow))}, {}, std::move(declared)); + REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("`A_Conflicted`")); + REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("different effects")); +} + +TEST_CASE("action_id_held_by_two_action_types_throws", "[push][records]") { + ActionResultSlotMap slots; + slots.emplace("A_Shared", result({{typeid(VRow), ""}})); + auto const message = refusal({record("V_Row", typeid(VRow))}, std::move(slots), {}, {"A_Shared"}); + REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("`A_Shared`")); + REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("two action types")); +} + +TEST_CASE("updates_without_record_in_result_throws", "[push][records]") { + ActionResultSlotMap slots; + slots.emplace("A_Counts", result({{typeid(VList), ""}})); + EffectsMap declared; + morph::effects::detail::addEffects(declared, "A_Counts", {updatesOf(typeid(VRow))}); + auto const message = refusal({record("V_Row", typeid(VRow))}, std::move(slots), std::move(declared)); + REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("`A_Counts`")); + REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("result holds no `V_Row`")); +} + +TEST_CASE("updates_on_keyless_throws", "[push][records]") { + ActionResultSlotMap slots; + slots.emplace("A_Report", result({{typeid(VKeyless), ""}})); + EffectsMap declared; + morph::effects::detail::addEffects(declared, "A_Report", {updatesOf(typeid(VKeyless))}); + auto const message = + refusal({record("V_Keyless", typeid(VKeyless), std::nullopt)}, std::move(slots), std::move(declared)); + REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("`A_Report`")); + REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("`V_Keyless`, which has no key")); +} + +TEST_CASE("effect_on_unregistered_type_throws", "[push][records]") { + EffectsMap declared; + morph::effects::detail::addEffects(declared, "A_Stray", {changesOf(typeid(VOther))}); + auto const message = refusal({record("V_Row", typeid(VRow))}, {}, std::move(declared)); + REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("`A_Stray`")); + REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("not a registered record")); +} + +TEST_CASE("derived_cycle_throws", "[push][records]") { + auto const message = + refusal({record("V_CycleA", typeid(VCycleA), std::nullopt, {typeid(VCycleB)}), + record("V_CycleB", typeid(VCycleB), std::nullopt, {typeid(VCycleA)}), record("V_Row", typeid(VRow))}, + {}, {}); + REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("V_CycleA -> V_CycleB -> V_CycleA")); +} + +TEST_CASE("duplicate_record_name_throws", "[push][records]") { + auto const byName = refusal({record("V_Row", typeid(VRow)), record("V_Row", typeid(VOther))}, {}, {}); + REQUIRE_THAT(byName, Catch::Matchers::ContainsSubstring("record name `V_Row` is registered for two types")); + auto const byType = refusal({record("V_Row", typeid(VRow)), record("V_Row2", typeid(VRow))}, {}, {}); + REQUIRE_THAT(byType, Catch::Matchers::ContainsSubstring("registered as a record twice (`V_Row` and `V_Row2`)")); +} + +TEST_CASE("every_problem_is_listed", "[push][records]") { + ActionResultSlotMap slots; + slots.emplace("A_Counts", result({{typeid(VList), ""}})); + slots.emplace("A_Report", result({{typeid(VKeyless), ""}})); + EffectsMap declared; + morph::effects::detail::addEffects(declared, "A_Counts", {updatesOf(typeid(VRow))}); + morph::effects::detail::addEffects(declared, "A_Report", {updatesOf(typeid(VKeyless))}); + auto const message = refusal( + {record("V_Row", typeid(VRow)), record("V_Keyless", typeid(VKeyless), std::nullopt), + record("V_CycleA", typeid(VCycleA), std::nullopt, {typeid(VCycleA)}), record("V_Row", typeid(VOther))}, + std::move(slots), std::move(declared)); + REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("result holds no `V_Row`")); + REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("`V_Keyless`, which has no key")); + REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("V_CycleA -> V_CycleA")); + REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("record name `V_Row` is registered for two types")); +} diff --git a/tests/test_push_records.cpp b/tests/test_push_records.cpp index 755b965ce..392728b5e 100644 --- a/tests/test_push_records.cpp +++ b/tests/test_push_records.cpp @@ -131,12 +131,7 @@ BRIDGE_REGISTER_MODEL(WalkerModel, "T_WalkerModel") BRIDGE_REGISTER_ACTION(WalkerModel, ListRows, "T_WalkerListRows") BRIDGE_REGISTER_ACTION(WalkerModel, CountRows, "T_WalkerCountRows") -// Another model may register a different action type under an id already taken. namespace { -[[maybe_unused]] bool const walkerConflictFirst = - morph::model::detail::registerActionResultOnce("T_WalkerConflict"); -[[maybe_unused]] bool const walkerConflictSecond = - morph::model::detail::registerActionResultOnce("T_WalkerConflict"); [[maybe_unused]] bool const walkerSameFirst = morph::model::detail::registerActionResultOnce("T_WalkerSame"); [[maybe_unused]] bool const walkerSameSecond = morph::model::detail::registerActionResultOnce("T_WalkerSame"); } // namespace @@ -274,6 +269,23 @@ TEST_CASE("action_registration_records_result_slots", "[push][records]") { } TEST_CASE("action_result_second_type_under_one_id_is_a_conflict", "[push][records]") { + // Another model may register a different action type under an id already taken. Matching refuses such + // an id, so the conflict is registered here and removed again: the process registry must stay valid for + // every other case that reads it. + bool const wasClosed = morph::model::registrationPhaseClosed(); + morph::model::detail::reopenRegistrationPhaseForTesting(); + static_cast(morph::model::detail::registerActionResultOnce("T_WalkerConflict")); + static_cast(morph::model::detail::registerActionResultOnce("T_WalkerConflict")); + if (wasClosed) { + morph::model::closeRegistrationPhase(); + } + struct Restore { + ~Restore() { + morph::model::detail::mutableActionResultSlots().erase("T_WalkerConflict"); + std::erase(morph::model::detail::mutableActionResultConflicts(), std::string{"T_WalkerConflict"}); + } + } const restore; + auto const& all = morph::model::detail::actionResultSlots(); auto const& conflicts = morph::model::detail::actionResultConflicts(); auto const conflict = all.find("T_WalkerConflict"); From cbd944536e2c27852352a724fd3d0bcc8863801f Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 12:43:26 +0200 Subject: [PATCH 16/68] core: identical record registrations are one registration; stricter effect validation Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/registry.md | 23 +++++- .../specs/2026-10-16-server-push-design.md | 5 +- include/morph/core/effects.hpp | 79 ++++++++++++++----- tests/test_push_effects.cpp | 43 +++++++++- 4 files changed, 126 insertions(+), 24 deletions(-) diff --git a/docs/spec/core/registry.md b/docs/spec/core/registry.md index 2a1c3c92b..a878ff2b5 100644 --- a/docs/spec/core/registry.md +++ b/docs/spec/core/registry.md @@ -1641,6 +1641,8 @@ read through `detail::recordRegistry()`. - The registry is raw: duplicate names or types, `derivedFrom` cycles and keyless records that an effect names are not checked at registration. They are checked on the first read through `morph::effects` (see [Effects](#effects)). + There, an identical registration repeated is one registration, so a record + may be registered in a model header. - Record types need external linkage: glaze reflects member names through an external-linkage object, so a record in an anonymous namespace does not compile. @@ -1758,17 +1760,32 @@ for every action, which records its result holds. `recordsOf` names the record. - **Validation.** Resolution refuses, all problems at once: - an action id registered by two action types (`actionResultConflicts()`); - - an action with two different effects declarations; + - an action with two different effects declarations. A declaration is a + set, so a repeated effect counts once and order does not matter; + - `none` together with any other effect in one declaration; - `updates` or `changes` where `R` is not a registered record; - `updates` where `R` has no key, or where the action's `recordsOf` does not name `R`; - - a type registered as a record twice, or one name registered for two types; + - conflicting record registrations: one type under two names, one name on + two types, or one type registered twice with different options (key, + sources, `privateKeys`). An identical registration repeated is one + registration, as `registerActionOnce` treats a repeated action, so + `BRIDGE_REGISTER_RECORD` may sit in a model header. `BRIDGE_REGISTER_EFFECTS` + may not: it is also idempotent, but in a header it runs once per + translation unit for nothing; + - a keyed record with `derivedFrom`: a key-level change to a source does not + say which derived rows moved, so a derived record is keyless; - a `derivedFrom` cycle among registered records, a record derived from - itself included. A source that is not a registered record is ignored. + itself included, found in registration order so the message is the same on + every platform. A source that is not a registered record is ignored. - **Failure.** A refusal throws `morph::model::EffectsRegistrationError`, a `std::logic_error` whose `what()` lists every problem, one per line, each naming the action or record. The error is kept and the same error is thrown by every later read, so no read ever sees a partly valid registry. +- **Registrations after the first read are not seen.** Resolution runs once; + a record, action or effects registration made after it (which a debug build + already asserts against, since the read closed the latch) is not in any + later answer. - **Reads.** `effectsOf(id)` is `nullopt` for an action that declared nothing: it reads. `effectsOf` gives each record as its wire name, empty for `changesInstance` and `none`. `recordNamed(name)` points into the resolved diff --git a/docs/superpowers/specs/2026-10-16-server-push-design.md b/docs/superpowers/specs/2026-10-16-server-push-design.md index 2bb502d24..c106496a0 100644 --- a/docs/superpowers/specs/2026-10-16-server-push-design.md +++ b/docs/superpowers/specs/2026-10-16-server-push-design.md @@ -149,8 +149,9 @@ BRIDGE_REGISTER_RECORD(bank::dto::SpendingReport, "Bank_SpendingReport", derives a change (§8). A derived type is keyless, because a key-level change to a source does not say which derived rows moved. - **`morph::record::privateKeys`** marks a type whose keys must not reach other principals (§12). -- `derivedFrom` cycles, and two registrations of one type or one name, are refused when the registration phase - closes. +- `derivedFrom` cycles and conflicting registrations (one type under two names, one name on two types, or one type + with different options) are refused when the registration phase closes. An identical registration repeated, as a + record registered in a header gives once per translation unit, is one registration. ### Effects diff --git a/include/morph/core/effects.hpp b/include/morph/core/effects.hpp index 19abb5dc1..349c35bee 100644 --- a/include/morph/core/effects.hpp +++ b/include/morph/core/effects.hpp @@ -177,8 +177,9 @@ struct EffectOf { /// @brief Adds one declaration for an action, unless an equal one is already there. /// -/// Two declarations are equal when they hold the same effects in any order, so -/// one repeated in several translation units is stored once. +/// A declaration is a set: a repeated effect is stored once, and two +/// declarations holding the same effects in any order are equal, so one +/// repeated in several translation units is stored once. /// @param map The registry to add to. /// @param actionId The action's id. /// @param declared The effects. @@ -187,12 +188,18 @@ inline void addEffects(EffectsMap& map, std::string_view actionId, std::vector>{}).first; } + std::vector distinct; + for (auto& effect : declared) { + if (std::ranges::find(distinct, effect) == distinct.end()) { + distinct.push_back(std::move(effect)); + } + } auto& declarations = entry->second; bool const known = std::ranges::any_of(declarations, [&](std::vector const& existing) { - return std::ranges::is_permutation(existing, declared); + return std::ranges::is_permutation(existing, distinct); }); if (!known) { - declarations.push_back(std::move(declared)); + declarations.push_back(std::move(distinct)); } } @@ -223,7 +230,7 @@ inline bool registerEffectsOnce() noexcept { /// @brief The matched and validated registries: what every read is answered from. struct Resolved { - /// @brief The records, as registered. + /// @brief The records, one entry per registered type, in registration order. std::vector records; /// @brief Each action's effects, by action id; actions that declared none are absent. std::map, std::less<>> effects; @@ -282,8 +289,18 @@ struct Resolved { return path.size() > prefix.size() && path.starts_with(prefix) && path[prefix.size()] == '.'; } +/// @brief Whether two registrations of one record type declared the same record. +/// @param lhs One registration. +/// @param rhs The other. +/// @return True when name, key, sources and privacy all match. +[[nodiscard]] inline bool sameRegistration(record::RecordInfo const& lhs, record::RecordInfo const& rhs) { + return lhs.name == rhs.name && lhs.type == rhs.type && lhs.keyField == rhs.keyField && + lhs.derivedFrom == rhs.derivedFrom && lhs.privateKeys == rhs.privateKeys && + lhs.keyIsString == rhs.keyIsString; +} + /// @brief Appends a problem for every `derivedFrom` cycle among the records. -/// @param records The records, as registered. +/// @param records The records, one registration per type. /// @param indexOf Each record type's first registration. /// @param problems Where to append. inline void findDerivationCycles(std::vector const& records, @@ -315,7 +332,8 @@ inline void findDerivationCycles(std::vector const& records, path.pop_back(); marks[index] = Mark::Done; }; - for (auto const& [type, index] : indexOf) { + // Registration order, so the cycle a message names does not depend on how `type_index` sorts. + for (std::size_t index = 0; index < records.size(); ++index) { if (marks[index] == Mark::Unvisited) { visit(visit, index); } @@ -334,19 +352,37 @@ inline void findDerivationCycles(std::vector const& records, std::vector const& slotConflicts) { std::vector problems; + // An identical registration, as a record registered in a header gives once per translation unit, is + // one registration. Any other second registration of a type or a name is a conflict. + std::vector distinct; + for (auto& info : records) { + auto const sameType = std::ranges::find(distinct, info.type, &record::RecordInfo::type); + if (sameType != distinct.end()) { + if (sameType->name != info.name) { + problems.push_back(std::string{"type "} + info.type.name() + " is registered under two record names (`" + + sameType->name + "` and `" + info.name + "`)"); + } else if (!sameRegistration(*sameType, info)) { + problems.push_back("record `" + info.name + "` is registered twice with different options"); + } + continue; + } + auto const sameName = std::ranges::find(distinct, info.name, &record::RecordInfo::name); + if (sameName != distinct.end()) { + problems.push_back("record name `" + info.name + "` is registered for two types (" + + sameName->type.name() + " and " + info.type.name() + ")"); + continue; + } + distinct.push_back(std::move(info)); + } + records = std::move(distinct); + std::map indexOf; - std::map nameIndex; for (std::size_t index = 0; index < records.size(); ++index) { auto const& info = records[index]; - if (auto const [held, added] = indexOf.emplace(info.type, index); !added) { - problems.push_back(std::string{"type "} + info.type.name() + " is registered as a record twice (`" + - records[held->second].name + "` and `" + info.name + "`)"); - } - if (auto const [held, added] = nameIndex.emplace(info.name, index); !added) { - if (records[held->second].type != info.type) { - problems.push_back("record name `" + info.name + "` is registered for two types (" + - records[held->second].type.name() + " and " + info.type.name() + ")"); - } + indexOf.emplace(info.type, index); + if (info.keyField && !info.derivedFrom.empty()) { + problems.push_back("record `" + info.name + + "` is derived from other records but has a key; a derived record is keyless"); } } findDerivationCycles(records, indexOf, problems); @@ -396,8 +432,15 @@ inline void findDerivationCycles(std::vector const& records, problems.push_back("action `" + actionId + "` declares different effects in two registrations"); continue; } + auto const& declaration = declarations.front(); + if (declaration.size() > 1 && std::ranges::any_of(declaration, [](DeclaredEffect const& effect) { + return effect.kind == Kind::None; + })) { + problems.push_back("action `" + actionId + "` declares none together with other effects"); + continue; + } std::vector effects; - for (auto const& effect : declarations.front()) { + for (auto const& effect : declaration) { if (!effect.record) { effects.push_back(Effect{.kind = effect.kind, .record = {}}); continue; diff --git a/tests/test_push_effects.cpp b/tests/test_push_effects.cpp index 37f12aa35..2516670bd 100644 --- a/tests/test_push_effects.cpp +++ b/tests/test_push_effects.cpp @@ -280,7 +280,48 @@ TEST_CASE("duplicate_record_name_throws", "[push][records]") { auto const byName = refusal({record("V_Row", typeid(VRow)), record("V_Row", typeid(VOther))}, {}, {}); REQUIRE_THAT(byName, Catch::Matchers::ContainsSubstring("record name `V_Row` is registered for two types")); auto const byType = refusal({record("V_Row", typeid(VRow)), record("V_Row2", typeid(VRow))}, {}, {}); - REQUIRE_THAT(byType, Catch::Matchers::ContainsSubstring("registered as a record twice (`V_Row` and `V_Row2`)")); + REQUIRE_THAT(byType, Catch::Matchers::ContainsSubstring("registered under two record names (`V_Row` and `V_Row2`)")); +} + +TEST_CASE("identical_record_registrations_are_one", "[push][records]") { + ActionResultSlotMap slots; + slots.emplace("A_List", result({{typeid(VList), ""}, {typeid(VRow), "rows"}})); + auto const resolved = morph::effects::detail::resolveForTesting( + {record("V_Row", typeid(VRow)), record("V_Row", typeid(VRow))}, std::move(slots), {}); + REQUIRE(resolved.records.size() == 1); + REQUIRE(resolved.recordNamed("V_Row") != nullptr); + REQUIRE(resolved.recordsOf("A_List") == std::vector{{"V_Row", "id", {"rows"}}}); +} + +TEST_CASE("record_same_type_different_options_throws", "[push][records]") { + auto const message = refusal({record("V_Row", typeid(VRow)), record("V_Row", typeid(VRow), "code")}, {}, {}); + REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("record `V_Row` is registered twice with different options")); +} + +TEST_CASE("keyed_derived_record_throws", "[push][records]") { + auto const message = + refusal({record("V_Row", typeid(VRow)), record("V_Header", typeid(VHeader), "id", {typeid(VRow)})}, {}, {}); + REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("record `V_Header` is derived from other records but has a key")); +} + +TEST_CASE("effect_none_with_others_throws", "[push][records]") { + EffectsMap declared; + morph::effects::detail::addEffects(declared, "A_Mixed", + {changesOf(typeid(VRow)), DeclaredEffect{.kind = Kind::None, .record = {}}}); + auto const message = refusal({record("V_Row", typeid(VRow))}, {}, std::move(declared)); + REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("action `A_Mixed` declares none together with other effects")); +} + +TEST_CASE("repeated_effect_in_declaration_is_one", "[push][records]") { + EffectsMap declared; + morph::effects::detail::addEffects(declared, "A_List", {updatesOf(typeid(VRow)), updatesOf(typeid(VRow))}); + morph::effects::detail::addEffects(declared, "A_List", {updatesOf(typeid(VRow))}); + REQUIRE(declared.at("A_List").size() == 1); + ActionResultSlotMap slots; + slots.emplace("A_List", result({{typeid(VRow), ""}})); + auto const resolved = morph::effects::detail::resolveForTesting({record("V_Row", typeid(VRow))}, std::move(slots), + std::move(declared)); + REQUIRE(resolved.effectsOf("A_List") == std::vector{{Kind::Updates, "V_Row"}}); } TEST_CASE("every_problem_is_listed", "[push][records]") { From dde59f556026f86090ee0f079077c3e97b38117d Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 12:49:27 +0200 Subject: [PATCH 17/68] core: derive an execute's changes from its effects and its result Co-Authored-By: Claude Opus 5.5 --- CMakeLists.txt | 2 + include/morph/core/change_types.hpp | 46 ++++++ include/morph/core/changes.hpp | 237 ++++++++++++++++++++++++++++ tests/test_push_effects.cpp | 146 +++++++++++++++++ 4 files changed, 431 insertions(+) create mode 100644 include/morph/core/change_types.hpp create mode 100644 include/morph/core/changes.hpp diff --git a/CMakeLists.txt b/CMakeLists.txt index 12d25a3e2..fb7729cbb 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -455,6 +455,8 @@ target_sources(morph include/morph/core/registry.hpp include/morph/core/records.hpp include/morph/core/effects.hpp + include/morph/core/change_types.hpp + include/morph/core/changes.hpp include/morph/core/backend.hpp include/morph/core/envelope_transport.hpp include/morph/core/remote.hpp diff --git a/include/morph/core/change_types.hpp b/include/morph/core/change_types.hpp new file mode 100644 index 000000000..9ff1d3ba9 --- /dev/null +++ b/include/morph/core/change_types.hpp @@ -0,0 +1,46 @@ +// SPDX-License-Identifier: Apache-2.0 + +#pragma once +#include +#include +#include +#include +#include + +/// @file +/// The plain data a derived change is made of. This header includes nothing of the registry or the +/// forms layer, so the wire types can use it. + +namespace morph::effects { + +/// @brief A change to the rows of one record type. +struct RecordChange { + /// The record's wire name. + std::string record; + /// The keys of the rows that changed, each as raw JSON; absent when the whole type changed. + std::optional> keys; +}; + +/// @brief A change to the state of one model instance. +struct InstanceChange { + /// The model's registered type name. + std::string modelType; + /// The instance id. + std::uint64_t instance = 0; + /// The directory key's text for a keyed shared instance; empty for a private one. + std::string primary; +}; + +/// @brief What one successful execute changed. +struct Changes { + /// Record topics, one entry per record type. + std::vector records; + /// Instance topics. + std::vector instances; + + /// @brief Whether nothing changed. + /// @return True when there are no record and no instance changes. + [[nodiscard]] bool empty() const { return records.empty() && instances.empty(); } +}; + +} // namespace morph::effects diff --git a/include/morph/core/changes.hpp b/include/morph/core/changes.hpp new file mode 100644 index 000000000..28f3445a0 --- /dev/null +++ b/include/morph/core/changes.hpp @@ -0,0 +1,237 @@ +// SPDX-License-Identifier: Apache-2.0 + +#pragma once +#include +#include +#include +#include +#include +#include +#include +#include + +#include "change_types.hpp" +#include "effects.hpp" + +/// @file +/// Deriving an execute's changes from the action's declared effects and its result. + +namespace morph::effects { + +namespace detail { + +/// The JSON tree a result is parsed into. It keeps integers exact, so a 64-bit key is copied as written. +using ResultJson = glz::generic_u64; + +/// @brief Appends the key of one record value to @p out. +/// @param value A JSON value expected to be one record. +/// @param keyField The record's key member. +/// @param out Where to append the key's raw JSON. +inline void collectKey(ResultJson const& value, std::string_view keyField, std::vector& out) { + if (!value.is_object()) { + return; + } + auto const& members = value.get_object(); + auto const key = members.find(keyField); + if (key == members.end()) { + return; + } + if (auto text = glz::write_json(key->second)) { + out.emplace_back(std::move(*text)); + } +} + +/// @brief Appends the keys of the records held at a value that the path has reached. +/// +/// An object is one record when it has the key member, and otherwise a map whose values are records. +/// @param value The value at the end of a path. +/// @param keyField The record's key member. +/// @param out Where to append. +inline void collectKeysAtLeaf(ResultJson const& value, std::string_view keyField, std::vector& out) { + if (value.is_array()) { + for (auto const& element : value.get_array()) { + collectKeysAtLeaf(element, keyField, out); + } + } else if (value.is_object()) { + auto const& members = value.get_object(); + if (members.find(keyField) != members.end()) { + collectKey(value, keyField, out); + } else { + for (auto const& [name, entry] : members) { + collectKey(entry, keyField, out); + } + } + } +} + +/// @brief Walks the remaining path segments from @p value, then collects the keys found there. +/// +/// A vector or optional adds no segment, so an array is entered without consuming one. A map adds +/// none either, so an object that lacks the next segment is searched through its values. +/// @param value The current value. +/// @param path The dot-joined member names; `""` is the value itself. +/// @param keyField The record's key member. +/// @param out Where to append. +inline void collectKeysAtPath(ResultJson const& value, std::string_view path, std::string_view keyField, + std::vector& out) { + if (path.empty()) { + collectKeysAtLeaf(value, keyField, out); + return; + } + if (value.is_array()) { + for (auto const& element : value.get_array()) { + collectKeysAtPath(element, path, keyField, out); + } + return; + } + if (!value.is_object()) { + return; + } + auto const dot = path.find('.'); + auto const segment = path.substr(0, dot); + auto const rest = dot == std::string_view::npos ? std::string_view{} : path.substr(dot + 1); + auto const& members = value.get_object(); + if (auto const next = members.find(segment); next != members.end()) { + collectKeysAtPath(next->second, rest, keyField, out); + return; + } + for (auto const& [name, entry] : members) { + collectKeysAtPath(entry, path, keyField, out); + } +} + +/// @brief Adds a change for a record, merging with one already there. +/// +/// A whole-type change absorbs keys, and keys are not repeated. +/// @param changes The changes so far. +/// @param record The record's wire name. +/// @param keys The changed keys, or `nullopt` for the whole type. +inline void addRecordChange(Changes& changes, std::string const& record, + std::optional> keys) { + auto entry = std::ranges::find(changes.records, record, &RecordChange::record); + if (entry == changes.records.end()) { + changes.records.push_back(RecordChange{.record = record, .keys = std::move(keys)}); + return; + } + if (!keys) { + entry->keys.reset(); + return; + } + if (!entry->keys) { + return; + } + for (auto& key : *keys) { + auto const same = [&](glz::raw_json const& have) { return have.str == key.str; }; + if (std::ranges::none_of(*entry->keys, same)) { + entry->keys->push_back(std::move(key)); + } + } +} + +/// @brief Adds a whole-type change for every record derived, directly or through others, from @p source. +/// @param resolved The resolved registries. +/// @param source A changed record's wire name. +/// @param changes Where to add. +inline void addDerivedChanges(Resolved const& resolved, std::string const& source, Changes& changes) { + auto const* const root = resolved.recordNamed(source); + if (root == nullptr) { + return; + } + std::vector reached{root->type}; + // A record is visited once, which also ends a walk over records that share a source. + for (std::size_t next = 0; next < reached.size(); ++next) { + for (auto const& candidate : resolved.records) { + if (std::ranges::find(candidate.derivedFrom, reached[next]) == candidate.derivedFrom.end() || + std::ranges::find(reached, candidate.type) != reached.end()) { + continue; + } + reached.push_back(candidate.type); + addRecordChange(changes, candidate.name, std::nullopt); + } + } +} + +/// @brief See `morph::effects::deriveChanges`, over given registries. +/// @param resolved The resolved registries. +/// @param actionId The action's id. +/// @param resultJson The action's result as JSON. +/// @param ranOn The instance the action ran on. +/// @return The changes. +[[nodiscard]] inline Changes deriveChangesFrom(Resolved const& resolved, std::string_view actionId, + std::string_view resultJson, InstanceChange const& ranOn) { + Changes changes; + auto const declared = resolved.effectsOf(actionId); + if (!declared) { + return changes; + } + std::optional parsed; + bool parseTried = false; + for (auto const& effect : *declared) { + switch (effect.kind) { + case Kind::None: + break; + case Kind::ChangesInstance: + if (std::ranges::none_of(changes.instances, [&](InstanceChange const& have) { + return have.modelType == ranOn.modelType && have.instance == ranOn.instance && + have.primary == ranOn.primary; + })) { + changes.instances.push_back(ranOn); + } + break; + case Kind::Changes: + addRecordChange(changes, effect.record, std::nullopt); + addDerivedChanges(resolved, effect.record, changes); + break; + case Kind::Updates: { + if (!parseTried) { + parseTried = true; + ResultJson tree; + if (!glz::read_json(tree, resultJson)) { + parsed = std::move(tree); + } + } + auto const* const info = resolved.recordNamed(effect.record); + // A result that does not parse, or a record without a key, cannot name rows: the whole type + // is reported, which over-notifies but never misses. + if (!parsed || info == nullptr || !info->keyField) { + addRecordChange(changes, effect.record, std::nullopt); + } else { + std::vector keys; + for (auto const& held : resolved.recordsOf(actionId)) { + if (held.record != effect.record) { + continue; + } + for (auto const& path : held.paths) { + collectKeysAtPath(*parsed, path, *info->keyField, keys); + } + } + addRecordChange(changes, effect.record, std::move(keys)); + } + addDerivedChanges(resolved, effect.record, changes); + break; + } + default: + break; + } + } + return changes; +} + +} // namespace detail + +/// @brief The changes a successful execute made, from the action's declared effects and its result. +/// +/// `updates` gives the keys of the `R` rows found in the result, `changes` the whole type, and both +/// also every record derived from `R`, directly or through others, as a whole-type change. +/// `changesInstance` gives @p ranOn. An action that declared nothing, or only `none`, gives no changes. +/// @param actionId The action's id. +/// @param resultJson The action's result as JSON. +/// @param ranOn The instance the action ran on. +/// @return The changes; empty for a reader. +/// @throws morph::model::EffectsRegistrationError when the registered configuration is invalid. +[[nodiscard]] inline Changes deriveChanges(std::string_view actionId, std::string_view resultJson, + InstanceChange const& ranOn) { + return detail::deriveChangesFrom(detail::resolved(), actionId, resultJson, ranOn); +} + +} // namespace morph::effects diff --git a/tests/test_push_effects.cpp b/tests/test_push_effects.cpp index 2516670bd..799cd59f9 100644 --- a/tests/test_push_effects.cpp +++ b/tests/test_push_effects.cpp @@ -3,6 +3,7 @@ #include #include #include +#include #include #include #include @@ -340,3 +341,148 @@ TEST_CASE("every_problem_is_listed", "[push][records]") { REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("V_CycleA -> V_CycleA")); REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("record name `V_Row` is registered for two types")); } + +// ── Deriving changes ───────────────────────────────────────────────────────── + +namespace { + +using morph::effects::Changes; +using morph::effects::InstanceChange; +using morph::effects::RecordChange; + +struct DTxn {}; +struct DReport {}; +struct DSummary {}; +struct DNamed {}; + +std::vector keyTexts(RecordChange const& change) { + std::vector texts; + for (auto const& key : *change.keys) { + texts.push_back(key.str); + } + return texts; +} + +// Records: D_Row (keyed), D_Txn (keyed), D_Report derived from D_Txn, D_Summary derived from D_Report. +morph::effects::detail::Resolved derivation(EffectsMap declared) { + ActionResultSlotMap slots; + slots.emplace("A_Rows", result({{typeid(VList), ""}, {typeid(VRow), "rows"}})); + slots.emplace("A_Root", result({{typeid(VRow), ""}})); + slots.emplace("A_Deep", result({{typeid(VList), ""}, {typeid(VRow), "header.rows"}})); + slots.emplace("A_Named", result({{typeid(VList), ""}, {typeid(DNamed), "byName"}})); + return morph::effects::detail::resolveForTesting( + {record("D_Row", typeid(VRow)), record("D_Txn", typeid(DTxn)), + record("D_Report", typeid(DReport), std::nullopt, {typeid(DTxn)}), + record("D_Summary", typeid(DSummary), std::nullopt, {typeid(DReport)}), record("D_Header", typeid(VHeader), {}), + record("D_Named", typeid(DNamed), "code")}, + std::move(slots), declared); +} + +EffectsMap declare(std::string id, std::vector effects) { + EffectsMap declared; + morph::effects::detail::addEffects(declared, id, std::move(effects)); + return declared; +} + +InstanceChange const ranOnInstance{.modelType = "M", .instance = 7, .primary = "k"}; + +} // namespace + +TEST_CASE("derive_updates_reads_keys_at_paths", "[push][records]") { + auto const resolved = derivation(declare("A_Rows", {updatesOf(typeid(VRow))})); + auto const changes = morph::effects::detail::deriveChangesFrom( + resolved, "A_Rows", R"({"rows":[{"id":1,"n":"a"},{"id":2},{"n":"no key"}]})", ranOnInstance); + REQUIRE(changes.instances.empty()); + REQUIRE(changes.records.size() == 1); + REQUIRE(changes.records[0].record == "D_Row"); + REQUIRE(changes.records[0].keys.has_value()); + REQUIRE(keyTexts(changes.records[0]) == std::vector{"1", "2"}); +} + +TEST_CASE("derive_updates_reads_every_result_shape", "[push][records]") { + auto const rootResolved = derivation(declare("A_Root", {updatesOf(typeid(VRow))})); + auto const one = morph::effects::detail::deriveChangesFrom(rootResolved, "A_Root", R"({"id":9007199254740993})", + ranOnInstance); + REQUIRE(keyTexts(one.records.at(0)) == std::vector{"9007199254740993"}); + + auto const absent = morph::effects::detail::deriveChangesFrom(rootResolved, "A_Root", "null", ranOnInstance); + REQUIRE(absent.records.at(0).keys.has_value()); + REQUIRE(absent.records.at(0).keys->empty()); + + auto const deepResolved = derivation(declare("A_Deep", {updatesOf(typeid(VRow))})); + auto const deep = morph::effects::detail::deriveChangesFrom( + deepResolved, "A_Deep", R"({"header":{"rows":[{"id":"x"},{"id":"y"}]}})", ranOnInstance); + REQUIRE(keyTexts(deep.records.at(0)) == std::vector{"\"x\"", "\"y\""}); + + auto const namedResolved = derivation(declare("A_Named", {updatesOf(typeid(DNamed))})); + auto const map = morph::effects::detail::deriveChangesFrom( + namedResolved, "A_Named", R"({"byName":{"a":{"code":"c1"},"b":{"code":"c2"}}})", ranOnInstance); + REQUIRE(keyTexts(map.records.at(0)) == std::vector{"\"c1\"", "\"c2\""}); +} + +TEST_CASE("derive_updates_with_unparseable_result_reports_the_type", "[push][records]") { + auto const resolved = derivation(declare("A_Rows", {updatesOf(typeid(VRow))})); + auto const changes = morph::effects::detail::deriveChangesFrom(resolved, "A_Rows", "{not json", ranOnInstance); + REQUIRE(changes.records.size() == 1); + REQUIRE_FALSE(changes.records[0].keys.has_value()); +} + +TEST_CASE("derive_changes_has_no_keys", "[push][records]") { + auto const resolved = derivation(declare("A_Rows", {changesOf(typeid(VRow))})); + auto const changes = + morph::effects::detail::deriveChangesFrom(resolved, "A_Rows", R"({"rows":[{"id":1}]})", ranOnInstance); + REQUIRE(changes.records.size() == 1); + REQUIRE(changes.records[0].record == "D_Row"); + REQUIRE_FALSE(changes.records[0].keys.has_value()); +} + +TEST_CASE("derive_changes_absorbs_updates_keys", "[push][records]") { + auto const resolved = derivation(declare("A_Rows", {updatesOf(typeid(VRow)), changesOf(typeid(VRow))})); + auto const changes = + morph::effects::detail::deriveChangesFrom(resolved, "A_Rows", R"({"rows":[{"id":1}]})", ranOnInstance); + REQUIRE(changes.records.size() == 1); + REQUIRE_FALSE(changes.records[0].keys.has_value()); +} + +TEST_CASE("derive_expands_derived_from", "[push][records]") { + auto const resolved = derivation(declare("A_Rows", {changesOf(typeid(DTxn))})); + auto const changes = morph::effects::detail::deriveChangesFrom(resolved, "A_Rows", "{}", ranOnInstance); + REQUIRE(changes.records.size() == 3); + REQUIRE(changes.records[0].record == "D_Txn"); + // A derived record of a derived record is reached too, and neither carries keys. + REQUIRE(changes.records[1].record == "D_Report"); + REQUIRE(changes.records[2].record == "D_Summary"); + REQUIRE_FALSE(changes.records[1].keys.has_value()); + REQUIRE_FALSE(changes.records[2].keys.has_value()); +} + +TEST_CASE("derive_changes_instance_uses_ran_on", "[push][records]") { + auto const resolved = derivation(declare("A_Rows", {DeclaredEffect{.kind = Kind::ChangesInstance, .record = {}}})); + auto const changes = morph::effects::detail::deriveChangesFrom(resolved, "A_Rows", "{}", ranOnInstance); + REQUIRE(changes.records.empty()); + REQUIRE(changes.instances.size() == 1); + REQUIRE(changes.instances[0].modelType == "M"); + REQUIRE(changes.instances[0].instance == 7); + REQUIRE(changes.instances[0].primary == "k"); +} + +TEST_CASE("derive_reads_nothing_for_reader", "[push][records]") { + auto const resolved = derivation(declare("A_Other", {changesOf(typeid(VRow))})); + REQUIRE(morph::effects::detail::deriveChangesFrom(resolved, "A_Rows", "{}", ranOnInstance).empty()); + auto const none = derivation(declare("A_Rows", {DeclaredEffect{.kind = Kind::None, .record = {}}})); + REQUIRE(morph::effects::detail::deriveChangesFrom(none, "A_Rows", "{}", ranOnInstance).empty()); + // The process registry: a registered action that declared nothing reads. + REQUIRE(morph::effects::deriveChanges("T_WalkerListRows", "{}", ranOnInstance).empty()); +} + +TEST_CASE("derive_from_process_registry", "[push][records]") { + auto const changes = + morph::effects::deriveChanges("T_EffectsEdit", R"({"items":[{"id":5,"name":"a"}]})", ranOnInstance); + // updates and changes together: the whole type. + REQUIRE(changes.records.size() == 1); + REQUIRE(changes.records[0].record == "T_EItem"); + REQUIRE_FALSE(changes.records[0].keys.has_value()); + auto const instance = morph::effects::deriveChanges("T_EffectsLoad", "{}", ranOnInstance); + REQUIRE(instance.instances.size() == 1); + REQUIRE(instance.records.empty()); +} From 0a1dac7ae252b6b7d07aa3b4d70d2bb489f006bd Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 13:28:59 +0200 Subject: [PATCH 18/68] core: a server-initiated event kind, routed before reply matching Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/wire.md | 27 +++++++ include/morph/core/backend.hpp | 70 +++++++++++++++++ include/morph/core/detail/reply_router.hpp | 30 ++++++++ include/morph/core/wire.hpp | 33 ++++++++ tests/CMakeLists.txt | 1 + tests/test_push_channel.cpp | 88 ++++++++++++++++++++++ 6 files changed, 249 insertions(+) create mode 100644 tests/test_push_channel.cpp diff --git a/docs/spec/core/wire.md b/docs/spec/core/wire.md index 1fd1eda40..c27c11337 100644 --- a/docs/spec/core/wire.md +++ b/docs/spec/core/wire.md @@ -15,6 +15,7 @@ carries all request and reply variants, discriminated by a `kind` string field. - [Cancelling a call](#cancelling-a-call) - [Serving action schemas](#serving-action-schemas) - [The UI service's kinds](#the-ui-services-kinds) +- [The `event` kind](#the-event-kind) - [Action-evolution policy](#action-evolution-policy) - [API reference](#api-reference) - [Design decisions](#design-decisions) @@ -41,6 +42,7 @@ their `kind` needs and leave the rest as default-constructed values. | `"ui-hello"` | request | The UI service's handshake: what the client speaks; replies with the application, the vocabularies its bundle needs and the manifest digest. Answered only by a server configured with the UI service. See [The UI service's kinds](#the-ui-services-kinds). | `body`, `session` | | `"ui-catalog"` | request | The catalog filtered for the principal, its permitted actions and the load-time facts. UI service only. | `session` | | `"ui-fetch"` | request | A bundle file by its content hash; replies with the file's exact bytes in `body`. UI service only. | `body`, `session` | +| `"event"` | server-initiated | An unprompted notification on a connection that negotiated events. Never a reply to anything. See [The `event` kind](#the-event-kind). | `callId` (always `0`), `body` (`{topic, payload}`) | | `"ok"` | reply | Server success. | `callId`, `body` (serialized result, or — for a `"hello"` reply — the server's `ProtocolRange`), `modelId` (for register-replies) | | `"err"` | reply | Server failure. | `callId`, `message` | @@ -678,6 +680,30 @@ peer that predates the handshake. An old client never sends these kinds, so a server that has the UI service answers it as before; its `hello` reply is byte-for-byte the one a server without the service gives. +## The `event` kind + +`"event"` is the wire's one server-initiated kind: every other kind is a +request or the reply to one. Its `callId` is `0` and its `body` is an +`EventBody`, `{"topic": , "payload": }`; `payload` is +carried verbatim (`glz::raw_json`), so the wire layer never interprets it. +`makeEvent(topic, payloadJson)` builds one; an empty `payloadJson` is sent as +`null`. + +**Routed before reply matching.** `callId` 0 names no request, so an event that +reached reply matching would be counted as an orphaned reply. A client backend +therefore tests each decoded envelope with `backend::detail::routeEvent` +(`core/detail/reply_router.hpp`) first: an event goes to the handler installed +by `IBackend::setEventHandler(handler, exec)`, posted to `exec`, and is never +matched to a call. An event with no handler, or whose body does not decode, is +dropped. A pending call is never resolved by an event. + +**Order.** Events on one connection are delivered in the order they arrived. No +order is promised between an event and any reply. + +`LocalBackend` has no server and produces no events; `deliverEvent(EventBody)` +posts one to the installed handler, so the client's event path can be tested +without a transport. + ## Action-evolution policy The passive forward-compat contract (`error_on_unknown_keys = false` on @@ -838,6 +864,7 @@ client. | `makeDeregister` | `Envelope makeDeregister(uint64_t modelId)` | | `makeHello` | `Envelope makeHello(uint32_t protocolVersion = kProtocolVersion)` | | `makeCancel` | `Envelope makeCancel(uint64_t targetCallId)` | +| `makeEvent` | `Envelope makeEvent(std::string_view topic, std::string payloadJson)` | | `makeOk` | `Envelope makeOk(uint64_t callId = 0, std::string body = {}, uint64_t modelId = 0)` | | `makeErr` | `Envelope makeErr(std::string message, uint64_t callId = 0)` | diff --git a/include/morph/core/backend.hpp b/include/morph/core/backend.hpp index ca337f3c4..355ea0920 100644 --- a/include/morph/core/backend.hpp +++ b/include/morph/core/backend.hpp @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -31,6 +32,7 @@ #include "profiler.hpp" #include "registry.hpp" #include "strand.hpp" +#include "wire.hpp" namespace morph::backend { @@ -504,6 +506,28 @@ struct IBackend { (void)exec; } + /// @brief Installs a callback invoked for each server-initiated event the + /// backend receives. + /// + /// A transport backend decodes an `event` envelope, routes it here before + /// reply matching, and posts the handler to @p exec, so the handler is an + /// ordinary task on the installer's owner. Events on one connection are + /// delivered in the order they arrived. + /// + /// Default implementation: store-and-ignore. A backend with no server + /// channel never invokes it. + /// @param handler Callable posted to @p exec once per event. Pass + /// `nullptr` to clear. + /// @param exec Executor the handler is posted to. Borrowed: it must + /// outlive the installation. May be null only when + /// @p handler is. + // NOLINTNEXTLINE(performance-unnecessary-value-param): the base discards it; overrides take it by value and move it in. + virtual void setEventHandler(std::function handler, + ::morph::exec::IExecutor* exec) { + (void)handler; + (void)exec; + } + /// @brief Installs a callback invoked on every successful connect, including the first. /// /// `setReconnectHandler` deliberately skips the first connect (there is @@ -873,6 +897,16 @@ class SynchronousBackendAdapter : public detail::IBackend { }); } + /// @brief Queues the installation on the wrapped backend. + /// @param handler Callable posted to @p exec once per event; `nullptr` clears. + /// @param exec Executor the handler is posted to. + void setEventHandler(std::function handler, + ::morph::exec::IExecutor* exec) override { + post([handler = std::move(handler), exec](detail::IBackend& inner) mutable { + inner.setEventHandler(std::move(handler), exec); + }); + } + /// @brief Queues the installation on the wrapped backend. /// @param handler Callable invoked after every successful connect; `nullptr` clears. void setConnectHandler(const std::function& handler) override { @@ -1087,6 +1121,38 @@ class LocalBackend : public detail::IBackend { }); } + /// @brief Installs the event handler `deliverEvent` posts to. + /// @param handler Callable posted to @p exec once per delivered event; `nullptr` clears. + /// @param exec Executor the handler is posted to. Borrowed: it must outlive the installation. + void setEventHandler(std::function handler, + ::morph::exec::IExecutor* exec) override { + std::lock_guard lock{_eventMutex}; + _eventHandler = handler ? std::make_shared>(std::move(handler)) + : nullptr; + _eventExec = exec; + } + + /// @brief Test seam: delivers @p event as if a server had sent it. + /// + /// A local backend has no server and produces no events itself; this lets + /// a test drive the client's event path without one. Thread-safe. The + /// handler is posted to the executor given to `setEventHandler`; with no + /// handler installed the event is dropped. + /// @param event The event to deliver. + void deliverEvent(::morph::wire::EventBody event) { + std::shared_ptr> handler; + ::morph::exec::IExecutor* exec = nullptr; + { + std::lock_guard lock{_eventMutex}; + handler = _eventHandler; + exec = _eventExec; + } + if (!handler || exec == nullptr) { + return; + } + exec->post([handler = std::move(handler), event = std::move(event)]() mutable { (*handler)(std::move(event)); }); + } + LocalBackend(const LocalBackend&) = delete; LocalBackend& operator=(const LocalBackend&) = delete; LocalBackend(LocalBackend&&) = delete; @@ -1749,6 +1815,10 @@ class LocalBackend : public detail::IBackend { // The Task runs `cancelPending` and the destructor stop. Weak, so a // finished run is not kept. Owner-only. std::vector> _taskRuns; + // The event handler and its executor. `deliverEvent` may be called from any thread, so both sit behind a mutex. + std::mutex _eventMutex; + std::shared_ptr> _eventHandler; + ::morph::exec::IExecutor* _eventExec = nullptr; std::size_t _taskRunsCompactAt = kPendingCompactFloor; // Concurrent in-flight executes, for the executeInFlight metric. A // shared_ptr (not a plain atomic member) so strand tasks hold their own diff --git a/include/morph/core/detail/reply_router.hpp b/include/morph/core/detail/reply_router.hpp index 460ffebc9..1784e131a 100644 --- a/include/morph/core/detail/reply_router.hpp +++ b/include/morph/core/detail/reply_router.hpp @@ -3,6 +3,8 @@ #pragma once #include #include +#include +#include #include #include #include @@ -81,6 +83,34 @@ enum class ExecuteReplyKind : std::uint8_t { return ExecuteReplyKind::Error; } +/// @brief Routes a decoded incoming envelope that is a server-initiated +/// event, before any reply matching. +/// +/// A client backend calls this first on each envelope it decodes: an `event` +/// carries `callId` 0, which names no request, so letting it reach reply +/// matching would count it as an orphaned reply. An event whose body does not +/// decode is consumed and dropped, not handed on, for the same reason. +/// +/// @param env Decoded incoming envelope. +/// @param handler Receives the event; may be empty, in which case the event +/// is consumed and dropped. +/// @return `true` if @p env is an event (the caller must not match it to a +/// call), `false` for any other kind. +[[nodiscard]] inline bool routeEvent(const ::morph::wire::Envelope& env, + const std::function& handler) { + if (env.kind != ::morph::wire::kEventKind) { + return false; + } + if (handler) { + ::morph::wire::EventBody body; + constexpr glz::opts lenient{.error_on_unknown_keys = false}; + if (!glz::read(body, env.body)) { + handler(std::move(body)); + } + } + return true; +} + /// @brief The map of in-flight calls, keyed by call id, owned by one executor. /// /// @tparam PendingT Backend-specific per-call state (the `Completion` state, diff --git a/include/morph/core/wire.hpp b/include/morph/core/wire.hpp index 977281012..81c71a295 100644 --- a/include/morph/core/wire.hpp +++ b/include/morph/core/wire.hpp @@ -75,6 +75,10 @@ inline constexpr std::uint32_t kProtocolVersion = 1; /// see `makeHello()` and `interpretHelloReply()`. A peer /// predating negotiation answers `err "unknown envelope kind: /// hello"`, which is how a legacy server is detected. +/// - `"event"` — server-initiated notification, sent unprompted on a +/// connection that negotiated events. `callId` is `0`; `body` +/// is an `EventBody`. A client routes it by kind before reply +/// matching; see `makeEvent()`. /// - `"cancel"` — client asks the server to stop one of its own in-flight /// `execute` calls. Uses `cancelCallId` (the execute's /// `callId`) and `session`; its own `callId` correlates the @@ -607,6 +611,35 @@ inline Envelope makeErr(std::string message, uint64_t callId = 0) { return env; } +/// @brief The envelope kind of a server-initiated event. +inline constexpr std::string_view kEventKind = "event"; + +/// @brief The decoded `body` of an `event` envelope. +struct EventBody { + /// @brief What the event is about (`changes`, `resync`). + std::string topic; + /// @brief The topic's payload: a JSON value, carried verbatim. + glz::raw_json payload; +}; + +/// @brief Builds an `event` envelope: `callId` 0, `body` = `{"topic":…,"payload":…}`. +/// +/// @param topic Event topic. +/// @param payloadJson The payload as a JSON value; embedded verbatim, so it +/// must be valid JSON (an empty string is sent as `null`). +/// @return The event envelope. +[[nodiscard]] inline Envelope makeEvent(std::string_view topic, std::string payloadJson) { + Envelope env; + env.kind = std::string{kEventKind}; + env.callId = 0; + if (payloadJson.empty()) { + payloadJson = "null"; + } + EventBody body{std::string{topic}, glz::raw_json{std::move(payloadJson)}}; + env.body = glz::write_json(body).value_or(std::string{}); + return env; +} + /// @brief The `err` reply message `RemoteServer` sends when /// `LimitPolicy::executeTimeout` fires server-side (see /// `RemoteServer::dispatchExecute`'s `_executeTimeouts` path). diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index f00fbb0cc..c3f9f07d0 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -24,6 +24,7 @@ add_executable(morph_tests test_example.cpp test_push_records.cpp test_push_effects.cpp + test_push_channel.cpp push_records_late.cpp test_executor_extra.cpp test_strand.cpp diff --git a/tests/test_push_channel.cpp b/tests/test_push_channel.cpp new file mode 100644 index 000000000..f4c3bd02b --- /dev/null +++ b/tests/test_push_channel.cpp @@ -0,0 +1,88 @@ +// SPDX-License-Identifier: Apache-2.0 + +// The server-initiated `event` kind: its builder, its routing ahead of reply +// matching, and the local backend's delivery seam. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using ::morph::backend::detail::PendingCallTable; +using ::morph::backend::detail::routeEvent; + +struct InlineExec final : ::morph::exec::IExecutor { + int posted = 0; + void post(std::function task) override { + ++posted; + task(); + } +}; + +} // namespace + +TEST_CASE("make_event_round_trips_through_encode_and_route", "[push][channel]") { + auto env = ::morph::wire::makeEvent("changes", R"({"a":[1,2]})"); + REQUIRE(env.kind == "event"); + REQUIRE(env.callId == 0); + auto decoded = ::morph::wire::decode(::morph::wire::encode(env)); + + std::vector<::morph::wire::EventBody> got; + REQUIRE(routeEvent(decoded, [&](::morph::wire::EventBody e) { got.push_back(std::move(e)); })); + REQUIRE(got.size() == 1); + CHECK(got[0].topic == "changes"); + CHECK(got[0].payload.str == R"({"a":[1,2]})"); +} + +TEST_CASE("reply_router_routes_event_before_call_match", "[push][channel]") { + PendingCallTable table; + table.insert(0, 7); + + int calls = 0; + auto const handler = [&](::morph::wire::EventBody) { ++calls; }; + CHECK(routeEvent(::morph::wire::makeEvent("resync", "null"), handler)); + CHECK(calls == 1); + CHECK(table.size() == 1); // the pending callId 0 is untouched + + // A reply is not an event: the caller proceeds to reply matching. + CHECK_FALSE(routeEvent(::morph::wire::makeOk(0), handler)); + CHECK(calls == 1); +} + +TEST_CASE("reply_router_drops_malformed_event_and_event_without_handler", "[push][channel]") { + ::morph::wire::Envelope bad; + bad.kind = "event"; + bad.body = "not json"; + int calls = 0; + CHECK(routeEvent(bad, [&](::morph::wire::EventBody) { ++calls; })); + CHECK(calls == 0); + CHECK(routeEvent(::morph::wire::makeEvent("changes", "{}"), {})); +} + +TEST_CASE("local_backend_deliver_event_reaches_handler_on_exec", "[push][channel]") { + InlineExec pool; + InlineExec cb; + ::morph::backend::LocalBackend backend{pool}; + + backend.deliverEvent({"changes", {"{}"}}); // no handler: dropped + CHECK(cb.posted == 0); + + std::vector topics; + backend.setEventHandler([&](::morph::wire::EventBody e) { topics.push_back(e.topic); }, &cb); + backend.deliverEvent({"changes", {"{}"}}); + backend.deliverEvent({"resync", {"null"}}); + CHECK(cb.posted == 2); + CHECK(topics == std::vector{"changes", "resync"}); + + backend.setEventHandler(nullptr, nullptr); + backend.deliverEvent({"changes", {"{}"}}); + CHECK(topics.size() == 2); +} From b3b5e43a656c54f67fe7f19c74fef957dfbee585 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 13:52:25 +0200 Subject: [PATCH 19/68] core: RemoteServer writes events to a connection, and holds them back from a slow reader Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/backend.md | 60 ++++- docs/spec/core/observability.md | 4 +- docs/spec/security.md | 11 + .../specs/2026-10-16-server-push-design.md | 23 +- include/morph/core/observability.hpp | 3 + include/morph/core/remote.hpp | 179 ++++++++++++++ tests/test_push_channel.cpp | 226 +++++++++++++++++- 7 files changed, 491 insertions(+), 15 deletions(-) diff --git a/docs/spec/core/backend.md b/docs/spec/core/backend.md index 4465a7abc..b450a84ec 100644 --- a/docs/spec/core/backend.md +++ b/docs/spec/core/backend.md @@ -46,6 +46,7 @@ and react to backend changes. - [`PayloadCompleteness` — enforcing the action-evolution policy](#payloadcompleteness--enforcing-the-action-evolution-policy) - [`LimitPolicy` — opt-in resource limits](#limitpolicy--opt-in-resource-limits) - [Connection scopes](#connection-scopes) +- [Writing to a connection](#writing-to-a-connection) - [`SimulatedRemoteBackend` — adapter for testing](#simulatedremotebackend--adapter-for-testing) - [`QtWebSocketBackend` — client-side WebSocket transport](#qtwebsocketbackend--client-side-websocket-transport) - [`QtWebSocketServer` — server-side WebSocket transport](#qtwebsocketserver--server-side-websocket-transport) @@ -982,8 +983,9 @@ nothing for a caller that never uses it. - `closeConnection(cid)` posts, to the server strand, the erasure of every model still recorded in `cid`'s scope (its directory record and the per-instance connection entry) exactly as the `deregister` path does, then drops the scope - itself. Passing `0`, an unknown `cid`, or a `cid` already closed is a no-op — - idempotent by construction. + itself, and the connection's writer with it (see + [Writing to a connection](#writing-to-a-connection)). Passing `0`, an unknown + `cid`, or a `cid` already closed is a no-op — idempotent by construction. - **`closeConnection` does not consult `IAuthorizer`.** It is server-side housekeeping triggered by the transport observing its own connection close, not a caller-attributed action — synthesising a `deregister` envelope @@ -1008,6 +1010,55 @@ nothing for a caller that never uses it. - `SimulatedRemoteBackend` keeps using the unscoped path (its "connection" is the process itself) — it is unaffected by connection scopes. +### Writing to a connection + +A reply goes back through the `reply` functor its own request carried. What the +server sends unasked — an `event` envelope (see [wire.md](wire.md)) — needs a +way to reach a connection with no request in hand, so a transport hands the +server one per connection. + +- `ConnectionWriter{write, backlog}` is that handle. `write(std::string)` + queues one encoded envelope on the transport's own thread, as a reply is + queued; `backlog()` reads how many bytes of encoded envelopes, replies and + events alike, the transport has accepted for the connection and not yet + written. Both run on the server strand and must not block. +- `setConnectionWriter(cid, writer)` posts to the server strand and keeps the + writer only while `cid`'s scope is open: a writer for `0`, an unknown `cid` + or a closed one is dropped, so a transport racing its own disconnect cannot + leave a writer nothing removes. It replaces any writer `cid` already had; an + empty `write` removes it. `closeConnection(cid)` drops it. The writer lives + next to the connection scopes, in a `cid → {writer, owesResync}` map that + only the strand touches. +- `post(cid, envelope)` is callable from any thread. On the calling thread, an + envelope larger than `wire::kMaxEnvelopeBytes` is dropped, logged at `warn` + and counted as `Metric::eventOversize` — the peer would refuse to decode it. + Otherwise it is posted to the strand, where it is written, or dropped if the + connection has no writer. Envelopes one thread posts to one connection are + written in that order. +- **Events, not replies, give way to a slow reader.** The server cannot see a + transport's outbox; it sees only `backlog()`. When `backlog()` exceeds + `kEventBacklogHigh` (1 MiB) at a `post`, the envelope is dropped and the + connection is marked as owed a `resync`. Every later `post` to it is dropped + too, even once the backlog has fallen, until the transport calls + `connectionDrained(cid)`; the server then sends one + `makeEvent("resync", "{}")` — "everything you watch may have changed" — and + clears the mark. So the `resync` is the next event the connection receives + after the first one it missed, and no event it receives is older than one it + missed. A drain with nothing owed sends nothing. +- `connectionDrained(cid)` is what a transport calls when the backlog falls + below `kEventBacklogLow` (256 KiB). It is callable from any thread — the + transport's loop or socket thread in practice — and only posts to the strand. + The drain, not a later write, sends the `resync`, so a connection that only + listens still receives it. The gap between the two bounds keeps a reader + hovering near one of them from flipping between held back and drained on + every frame. +- The backlog is counted by the transport, because only it can see what it + still holds; replies count toward it, and are never held back. A peer that + stops reading altogether is the transport's to close, by its send timeout. +- A `write` that throws is caught on the strand and logged at `warn`; the + server keeps serving and the connection keeps its writer until the transport + closes it. + ### Graceful shutdown (`beginShutdown()` / `drainedWithin()`) `beginShutdown()` enters shutdown: every subsequent `register`, `attach` and @@ -1970,7 +2021,10 @@ server: each call is a loop task, so the loop serialises them. | `handle(msg, reply, cid)` | Like `handle(msg, reply)`, additionally attributing any `register` in `msg` to connection `cid`'s scope. `cid == 0` behaves exactly like the two-argument overload. | | `handleInline(msg)` | Sync: runs the control envelope on the server strand — posted and waited for, or inline when already on it — and returns the reply JSON. **Rejects `execute`** — returns an `err` reply without dispatching, because an `execute` reply is produced asynchronously after this call returns. Needs a pool thread other than the caller's free to run the strand. Unscoped. | | `openConnection()` | Returns a fresh non-zero `ConnectionId` at once; its empty scope is opened on the server strand, before the caller's next `handle()`. | -| `closeConnection(cid)` | Posts: erases every model still recorded in `cid`'s scope (as `deregister` would) and drops the scope. `cid == 0`, unknown, or already-closed is a no-op — idempotent. Bypasses `IAuthorizer` by design. | +| `closeConnection(cid)` | Posts: erases every model still recorded in `cid`'s scope (as `deregister` would) and drops the scope and its writer. `cid == 0`, unknown, or already-closed is a no-op — idempotent. Bypasses `IAuthorizer` by design. | +| `setConnectionWriter(cid, writer)` | Posts: keeps `writer` (`ConnectionWriter{write, backlog}`) for `cid` while its scope is open; replaces an earlier one; an empty `write` removes it. `0`, unknown or closed `cid`: dropped. | +| `post(cid, envelope)` | Any thread. Above `wire::kMaxEnvelopeBytes`: dropped and counted (`Metric::eventOversize`). On the strand: written through `cid`'s writer, or held back — dropped, with one `resync` owed — while `backlog()` exceeds `kEventBacklogHigh` or a `resync` is still owed. No writer: dropped. | +| `connectionDrained(cid)` | Any thread; posts. Sends the owed `resync` event, if any, and lets events flow again. What a transport calls when the backlog falls below `kEventBacklogLow`. | | `payloadCompleteness()` | The `ServerConfig::payloadCompleteness` given at construction. | | `strand()` | The server strand, as an `exec::IExecutor`: `runningOn(server.strand())` is true inside every task on the server's state. | | `health(replyExec)` | `[[nodiscard]] Completion health(IExecutor& replyExec)` — readiness/liveModels/inFlight, answered on the server strand; delivered on `replyExec`, the executor the caller attaches from (borrowed: it must outlive the delivery). See [observability.md](observability.md). | diff --git a/docs/spec/core/observability.md b/docs/spec/core/observability.md index 96ae5b835..360daf1c1 100644 --- a/docs/spec/core/observability.md +++ b/docs/spec/core/observability.md @@ -25,7 +25,7 @@ if the seam did not exist. `RemoteServer` additionally exposes a ## Metrics -`Metric` is a closed `enum class` of nine observation kinds: +`Metric` is a closed `enum class` of ten observation kinds: | Enumerator | Kind | Meaning | |---|---|---| @@ -38,6 +38,7 @@ if the seam did not exist. `RemoteServer` additionally exposes a | `reconnectAttempts` | counter | `ReconnectCoordinator::onOnline`'s `tryReconnect` attempts. | | `reconnectOutcome` | counter | One per `onOnline()` call, tagged by outcome. | | `queueOverflow` | counter | An `IOfflineQueue::enqueue()` call rejected because the queue was at its configured `maxDepth()` (see `docs/spec/offline/offline.md`). | +| `eventOversize` | counter | An envelope handed to `RemoteServer::post` dropped, unsent, because it exceeds `wire::kMaxEnvelopeBytes` (see [backend.md](backend.md), "Writing to a connection"). | Each observation is a `MetricEvent{metric, value, tags}`; `tags` is a `std::span>` carrying @@ -116,6 +117,7 @@ a deployment's transport (e.g. `QtWebSocketServer`) is expected to expose | `SyncWorker::run()` | `queueDepth` (once, at drain) | — | | `ReconnectCoordinator::onOnline()` | `reconnectAttempts` (per attempt), `reconnectOutcome` (once, tagged `outcome`) | — | | `InMemoryOfflineQueue::enqueue`, `FileOfflineQueue::enqueue`, `SqliteOfflineQueue::enqueue` | `queueOverflow` (once per rejected call, at the queue's configured `maxDepth()`) | — | +| `RemoteServer::post` | `eventOversize` (once per dropped envelope, on the calling thread) | — | `registerCount`/`deregisterCount` count every call, not just successful ones — an unauthorized or malformed `register` still increments it, so the counter diff --git a/docs/spec/security.md b/docs/spec/security.md index 0d456e408..fb23bb624 100644 --- a/docs/spec/security.md +++ b/docs/spec/security.md @@ -34,6 +34,17 @@ application. Be explicit about the boundary: `wire::kMaxEnvelopeBytes` (8 MiB) before parsing, bounding a single message's allocation and parse cost (see [wire.md](core/wire.md)). This is a coarse per-message backstop only — not a rate limit, timeout, or inner-`body` bound. +- A bound on what a slow reader costs in server-sent events. `RemoteServer::post` + holds events back from a connection whose transport reports more than + `kEventBacklogHigh` (1 MiB) written to it and not yet sent, and owes it a + single `resync` event instead, sent when the transport reports the backlog + below `kEventBacklogLow` (256 KiB). However many events a peer fails to read, + the server queues none of them for it; what the transport already holds is + the transport's to bound. Replies are never held back, and a peer that stops + reading altogether is closed by its transport's send timeout. An event larger + than `wire::kMaxEnvelopeBytes` is dropped unsent. The writer and the backlog + read are transport code inside the trust boundary, like `closeConnection` (see + [backend.md](core/backend.md), "Writing to a connection"). - A negotiated protocol-version handshake (`wire::kind == "hello"`), opt-in and exchanged before any `execute`, so an incompatible peer is refused with a clear diagnostic instead of failing per-request later (see diff --git a/docs/superpowers/specs/2026-10-16-server-push-design.md b/docs/superpowers/specs/2026-10-16-server-push-design.md index c106496a0..dca21a5d5 100644 --- a/docs/superpowers/specs/2026-10-16-server-push-design.md +++ b/docs/superpowers/specs/2026-10-16-server-push-design.md @@ -266,8 +266,9 @@ The wire gains its first server-initiated message. `-Woverloaded-virtual` build requires. - **The envelope.** `kind: "event"`, `callId: 0`, `body: {topic, payload}`. For this design the topic is `changes`. A client routes it by kind before reply matching, so an event is never taken for an orphaned reply. -- **Writing to a connection.** `RemoteServer` gains `setConnectionWriter(cid, write)`, the transport's writer for a - connection already opened with `openConnection()`, and `post(cid, envelope)`. +- **Writing to a connection.** `RemoteServer` gains `setConnectionWriter(cid, writer)`, which takes the transport's + `ConnectionWriter{write, backlog}` for a connection already opened with `openConnection()`, `post(cid, envelope)`, + and `connectionDrained(cid)`. - `post` is callable from any thread. It hands the encoded envelope to the writer, which queues it on the transport's own thread, as replies are queued today. - The size cap (`kMaxEnvelopeBytes`) applies. @@ -282,16 +283,18 @@ The wire gains its first server-initiated message. - decremented on the loop or socket thread once the frame is written (the `net` `outbox` emptying, Qt's `bytesWritten`). - `setConnectionWriter(cid, write, backlog, onDrained)` hands the server the writer, a read of that counter, and a - drain callback that the transport calls when the counter falls below 256 KiB. + `setConnectionWriter(cid, writer)` hands the server the writer and a read of that counter, as + `ConnectionWriter{write, backlog}`. When the counter falls below 256 KiB, the transport calls + `RemoteServer::connectionDrained(cid)`; the server holds no callback into the transport. - The counter counts the bytes of the encoded envelopes handed to the writer, and is decremented by the same amount when that frame has been written. It never uses a transport's framed byte counts, so it cannot drift. - - `onDrained` runs on the transport's thread. It only posts to the server strand, where the owed `resync` is sent - and the flag cleared. + - `connectionDrained` is called on the transport's thread. It only posts to the server strand, where the owed + `resync` is sent and the flag cleared. - **Events, not replies, give way.** When `backlog()` exceeds 1 MiB, the server stops posting events to that - connection and records that it owes it a `resync`. When the transport calls `onDrained`, the server sends one - `resync` event, meaning "everything you watch may have changed", and the client marks every watched entry stale. - The drain callback, not a later write, triggers it, so an idle client that only watches still receives it. + connection and records that it owes it a `resync`. When the transport calls `connectionDrained`, the server sends + one `resync` event, meaning "everything you watch may have changed", and the client marks every watched entry + stale. + The drain call, not a later write, triggers it, so an idle client that only watches still receives it. Replies are never held back. - **A peer that stops reading** is closed by the transport's own rule: the `net` writer closes a peer that stalls past `sendTimeout`, as it does today for replies. The Qt transport gains the same timeout in part 2, so both @@ -661,7 +664,7 @@ row, and keeps every new public symbol's Doxygen complete. The implementation pl | # | Part | Layer | Needs | Spec pages | |---|---|---|---|---| | 1 | **Records and effects.** `BRIDGE_REGISTER_RECORD`, raw and wrapped keys, `keyless`, `derivedFrom`, `privateKeys`; `BRIDGE_REGISTER_EFFECTS` as a registrar; the result walker in the action registration; matching when the phase closes; `records(A)` with paths; deriving changes from a result | core | none | `core/registry.md` | -| 2 | **Server-to-client channel.** The `event` kind; `setConnectionWriter(cid, write, backlog, onDrained)` with a per-connection atomic byte counter in each transport; `post`; holding events back and `resync` on drain; `sendTimeout` for the Qt transport; `SocketBackend::setConnectHandler`, fired in `onConnected()` after the held requests are flushed and before the reconnect handler is posted; `SimulatedRemoteBackend`; `setEventHandler`; the local backend's `deliverEvent` seam | core, net, qt | none | `core/wire.md`, `core/backend.md`, `security.md` | +| 2 | **Server-to-client channel.** The `event` kind; `setConnectionWriter(cid, writer)` and `connectionDrained(cid)` with a per-connection atomic byte counter in each transport; `post`; holding events back and `resync` on drain; `sendTimeout` for the Qt transport; `SocketBackend::setConnectHandler`, fired in `onConnected()` after the held requests are flushed and before the reconnect handler is posted; `SimulatedRemoteBackend`; `setEventHandler`; the local backend's `deliverEvent` seam | core, net, qt | none | `core/wire.md`, `core/backend.md`, `security.md` | | 3 | **Catalog and wire shapes.** `ActionInfo::effects` and `records` (key field, paths), local lookup and `ui-catalog`; `events` in `ui-hello`, and `ConnectionFacts` on `IEnvelopeService::handle`; the `watch`/`unwatch` and `changes` body shapes, defined here so parts 4 and 5 share them | ui, core (`wire.hpp` shapes, `remote.hpp` `ConnectionFacts`) | 1, 2 | `ui/service.md`, `ui/document.md`, `core/wire.md`, spec 5 §15 (the push kind is core `event`, not `ui-event`) | | 4 | **Watching and fan-out.** Core `watch`/`unwatch` kinds in `RemoteServer` with the `cid`; the watch table with session contexts; the principal comparison in `stampVerifiedPrincipal`; fan-out in `dispatchExecute`, with private instances to their owner; `derivedFrom` expansion, coalescing, re-authorisation, `privateKeys` | core | 1, 2, 3 | `core/backend.md`, `core/shared_instances.md`, `security.md` | | 5 | **Client cache and reconnect seams.** `actionSucceeded` carries the reply and applies effects; patch by key field and path; `refetchAfter`, set by a stale mark or a patch on a fetching entry; discarded `latest` replies as marks; events and `resync`; topic reference counting; watch before fetch; `Session::reconnected()`; `AppSource::onConnected` with an empty default; `watcherOver` and `SessionOptions::watcher`; the generic client's remote source and shell wired to them (`examples/common/app`) | ui, examples/common | 2, 3 (4 for its remote tests) | `ui/interpreter.md`, `ui/frontend.md`, `ui/service.md`, `offline/offline.md`, spec 5 §5 | diff --git a/include/morph/core/observability.hpp b/include/morph/core/observability.hpp index 68c6c35d0..88a8352e9 100644 --- a/include/morph/core/observability.hpp +++ b/include/morph/core/observability.hpp @@ -33,6 +33,9 @@ enum class Metric : std::uint8_t { /// @brief Counter: an `IOfflineQueue::enqueue()` call rejected because the /// queue was at its configured `maxDepth()`. queueOverflow, + /// @brief Counter: an envelope handed to `RemoteServer::post` dropped + /// because it exceeds `wire::kMaxEnvelopeBytes`. + eventOversize, }; /// @brief One metric observation delivered to the installed `MetricSink`. diff --git a/include/morph/core/remote.hpp b/include/morph/core/remote.hpp index 184871770..ea5791dee 100644 --- a/include/morph/core/remote.hpp +++ b/include/morph/core/remote.hpp @@ -184,6 +184,33 @@ class OpaqueIdGenerator { /// values are minted by `RemoteServer::openConnection()`. using ConnectionId = std::uint64_t; +/// @brief A transport's outbound side of one connection, for what the server +/// sends unasked: events. +/// +/// Handed to `RemoteServer::setConnectionWriter`. Both callables run on the +/// server strand, so neither may block: `write` queues the frame on the +/// transport's own thread, as a reply is queued. +struct ConnectionWriter { + /// @brief Queues one encoded envelope for the connection. May throw when + /// the connection can no longer be written; the server catches it. + std::function write; + /// @brief Bytes of encoded envelopes handed to the transport for this + /// connection, replies and events alike, and not yet written. + /// Empty reads as `0`. + std::function backlog; +}; + +/// @brief Backlog above which `RemoteServer::post` holds a connection's events +/// back and owes it one `resync` instead: 1 MiB. +inline constexpr std::size_t kEventBacklogHigh = std::size_t{1} << 20; + +/// @brief Backlog below which a transport reports a connection drained, by +/// calling `RemoteServer::connectionDrained`: 256 KiB. +/// +/// Well under `kEventBacklogHigh`, so a reader hovering at the bound does not +/// flip between held back and drained on every frame. +inline constexpr std::size_t kEventBacklogLow = std::size_t{256} << 10; + /// @brief Snapshot of a `RemoteServer`'s current health. struct HealthStatus { /// @brief `true` if the server currently accepts and dispatches new work. @@ -576,6 +603,89 @@ class RemoteServer : public std::enable_shared_from_this { _strand.postTask([self = shared_from_this(), cid] { self->closeConnectionHere(cid); }); } + /// @brief Hands the server the transport's writer for connection @p cid, + /// through which `post` reaches it. + /// + /// Replaces any writer @p cid already has. Kept only while the connection + /// is open: a writer for `0`, for an unknown `cid` or for one already + /// closed is dropped, and `closeConnection(cid)` drops it. A writer whose + /// `write` is empty removes the connection's writer. + /// + /// Callable from any thread; runs on the server strand, after an + /// `openConnection()` and before any `post` the same thread makes next. + /// @param cid Connection, as returned by `openConnection()`. + /// @param writer The connection's writer and its backlog read. + void setConnectionWriter(ConnectionId cid, ConnectionWriter writer) { + if (cid == 0) { + return; + } + _strand.postTask([self = shared_from_this(), cid, writer = std::move(writer)]() mutable { + self->noteOwner("RemoteServer::setConnectionWriter"); + if (!self->_connectionScopes.contains(cid)) { + return; + } + if (!writer.write) { + self->_writers.erase(cid); + return; + } + self->_writers.insert_or_assign(cid, ConnectionOutbox{.writer = std::move(writer)}); + }); + } + + /// @brief Writes @p envelope to connection @p cid, unless its reader has + /// fallen behind. + /// + /// On the server strand, while the connection's `backlog()` exceeds + /// `kEventBacklogHigh`, the envelope is dropped and the connection is owed + /// one `resync` event, which `connectionDrained(cid)` sends. Until then + /// every later envelope is dropped too, so the `resync` is the next thing + /// the connection is sent after the hold-back began. Replies do not pass + /// through here and are never held back. + /// + /// Dropped as well: an envelope larger than `wire::kMaxEnvelopeBytes` + /// (counted as `Metric::eventOversize`), and one for a connection with no + /// writer — `0`, unknown, or closed. A `write` that throws is caught on + /// the strand. + /// + /// Callable from any thread. Envelopes one thread posts to one connection + /// are written in that order. + /// @param cid Connection to write to. + /// @param envelope The encoded envelope (via `wire::encode`). + void post(ConnectionId cid, std::string envelope) { + if (envelope.size() > ::morph::wire::kMaxEnvelopeBytes) { + ::morph::observe::detail::emitMetric(::morph::observe::Metric::eventOversize, 1.0); + ::morph::log::logWarn("[post] envelope for connection {} dropped: {} bytes exceeds {}", cid, + envelope.size(), ::morph::wire::kMaxEnvelopeBytes); + return; + } + if (cid == 0) { + return; + } + _strand.postTask([self = shared_from_this(), cid, envelope = std::move(envelope)]() mutable { + self->postHere(cid, std::move(envelope)); + }); + } + + /// @brief Tells the server that connection @p cid's backlog has fallen + /// below `kEventBacklogLow`. + /// + /// What a transport calls when it has written enough of what it holds. If + /// `post` held an event back since the last drain, the connection is sent + /// one `resync` event (`wire::makeEvent("resync", "{}")`), meaning + /// everything it watches may have changed, and events flow again. + /// Otherwise nothing is sent. The drain, not a later write, sends it, so a + /// connection that only listens still receives it. + /// + /// Callable from any thread, the transport's included; it only posts to + /// the server strand. + /// @param cid Connection whose backlog drained. + void connectionDrained(ConnectionId cid) { + if (cid == 0) { + return; + } + _strand.postTask([self = shared_from_this(), cid] { self->connectionDrainedHere(cid); }); + } + /// @brief The server strand, as an executor. /// /// What `runningOn(server.strand())` asks about: true inside every task the @@ -678,6 +788,13 @@ class RemoteServer : public std::enable_shared_from_this { std::string error; }; + /// One connection's writer, and whether `post` has held an event back + /// from it since the last drain. + struct ConnectionOutbox { + ConnectionWriter writer; + bool owesResync{false}; + }; + /// @brief Rejects a configuration whose protocol range is empty. /// @param config The configuration to check. /// @return @p config, unchanged. @@ -746,6 +863,7 @@ class RemoteServer : public std::enable_shared_from_this { /// @param cid Connection scope to close. void closeConnectionHere(ConnectionId cid) { noteOwner("RemoteServer::closeConnection"); + _writers.erase(cid); auto scopeIter = _connectionScopes.find(cid); if (scopeIter == _connectionScopes.end()) { return; @@ -761,6 +879,64 @@ class RemoteServer : public std::enable_shared_from_this { _connectionScopes.erase(scopeIter); } + /// @brief Body of `post`. On the server strand. + /// @param cid Connection to write to. + /// @param envelope The encoded envelope, within the size cap. + void postHere(ConnectionId cid, std::string envelope) { + noteOwner("RemoteServer::post"); + auto found = _writers.find(cid); + if (found == _writers.end()) { + return; + } + ConnectionOutbox& outbox = found->second; + if (outbox.owesResync) { + return; + } + if (outbox.writer.backlog && outbox.writer.backlog() > kEventBacklogHigh) { + outbox.owesResync = true; + return; + } + writeTo(cid, outbox, std::move(envelope)); + } + + /// @brief Body of `connectionDrained`. On the server strand. + /// @param cid Connection whose backlog drained. + void connectionDrainedHere(ConnectionId cid) { + noteOwner("RemoteServer::connectionDrained"); + auto found = _writers.find(cid); + if (found == _writers.end() || !found->second.owesResync) { + return; + } + found->second.owesResync = false; + writeTo(cid, found->second, ::morph::wire::encode(::morph::wire::makeEvent("resync", "{}"))); + } + + /// @brief Hands @p envelope to @p outbox's writer, catching what it + /// throws. On the server strand. + /// @param cid Connection the writer belongs to. + /// @param outbox The connection's writer state. + /// @param envelope The encoded envelope. + void writeTo(ConnectionId cid, ConnectionOutbox& outbox, std::string envelope) { + try { + outbox.writer.write(std::move(envelope)); + } catch (const std::exception& exc) { + writeFailed(cid, exc.what()); + } catch (...) { + writeFailed(cid, "unknown exception"); + } + } + + /// @brief What a failed write to connection @p cid does. On the server strand. + /// + /// The connection keeps its writer: the transport reports the connection + /// gone through `closeConnection`, which drops it. + /// @param cid Connection whose write threw. + /// @param reason The exception's message. + void writeFailed(ConnectionId cid, std::string_view reason) { + noteOwner("RemoteServer::writeFailed"); + ::morph::log::logWarn("[post] write to connection {} failed: {}", cid, reason); + } + /// @brief Registers a drain waiter, or answers it at once. On the server strand. /// @param deadline Longest time to wait for the in-flight count to reach zero. /// @param promise Settled `true` at zero, `false` at the deadline. @@ -1831,6 +2007,9 @@ class RemoteServer : public std::enable_shared_from_this { std::unordered_map> _connectionScopes; + // The writer of each open connection that has one, set by + // `setConnectionWriter` and dropped with the connection's scope. + std::unordered_map _writers; /// Names one cancellable call: the connection it arrived on and the /// `callId` its client chose. struct CancelKey { diff --git a/tests/test_push_channel.cpp b/tests/test_push_channel.cpp index f4c3bd02b..3a02df904 100644 --- a/tests/test_push_channel.cpp +++ b/tests/test_push_channel.cpp @@ -1,19 +1,32 @@ // SPDX-License-Identifier: Apache-2.0 // The server-initiated `event` kind: its builder, its routing ahead of reply -// matching, and the local backend's delivery seam. +// matching, the local backend's delivery seam, and `RemoteServer` writing +// events to a connection and holding them back from a slow reader. +#include #include +#include #include #include +#include +#include #include #include #include +#include +#include +#include #include +#include +#include #include +#include #include #include +#include "test_support.hpp" + namespace { using ::morph::backend::detail::PendingCallTable; @@ -86,3 +99,214 @@ TEST_CASE("local_backend_deliver_event_reaches_handler_on_exec", "[push][channel backend.deliverEvent({"changes", {"{}"}}); CHECK(topics.size() == 2); } + +// ── RemoteServer: writing events to a connection ──────────────────────────── + +struct PcBumpAction {}; +struct PcCounterModel { + int count = 0; + int execute(const PcBumpAction&) { return ++count; } +}; + +template <> +struct morph::model::ModelTraits { + static constexpr std::string_view typeId() { return "PC_CounterModel"; } +}; +template <> +struct morph::model::ActionTraits { + using Result = int; + static constexpr std::string_view typeId() { return "PC_BumpAction"; } + static std::string toJson(const PcBumpAction&) { return "{}"; } + static PcBumpAction fromJson(std::string_view) { return {}; } + static std::string resultToJson(const int& res) { return std::to_string(res); } + static int resultFromJson(std::string_view json) { return std::stoi(std::string{json}); } +}; + +namespace { + +struct PcEnv { + ::morph::model::detail::ActionDispatcher dispatcher; + ::morph::model::detail::ModelRegistryFactory registry; +}; + +PcEnv& pcEnv() { + static PcEnv env = [] { + PcEnv made; + made.registry.registerModel("PC_CounterModel"); + made.dispatcher.registerAction("PC_CounterModel", "PC_BumpAction"); + return made; + }(); + return env; +} + +/// What a transport put on one connection's wire, replies and events alike, +/// in the order it was handed them. +struct FakeWire { + std::mutex mtx; + std::vector sent; + std::atomic backlog{0}; + + void push(std::string frame) { + std::scoped_lock const lock{mtx}; + sent.push_back(std::move(frame)); + } + std::vector snapshot() { + std::scoped_lock const lock{mtx}; + return sent; + } + ::morph::backend::ConnectionWriter writer() { + return {.write = [this](std::string frame) { push(std::move(frame)); }, + .backlog = [this] { return backlog.load(); }}; + } +}; + +/// Runs every task already posted to the server strand before returning. +void flush(::morph::backend::RemoteServer& server) { + std::latch done{1}; + server.strand().post([&done] { done.count_down(); }); + done.wait(); +} + +std::string encodedEvent(std::string_view topic, std::string payload) { + return ::morph::wire::encode(::morph::wire::makeEvent(topic, std::move(payload))); +} + +} // namespace + +TEST_CASE("post_reaches_connection_writer", "[push][channel]") { + ::morph::exec::ThreadPoolExecutor pool{2}; + auto server = std::make_shared<::morph::backend::RemoteServer>(pool, pcEnv().dispatcher, pcEnv().registry); + FakeWire wire; + auto const cid = server->openConnection(); + server->setConnectionWriter(cid, wire.writer()); + + auto const first = encodedEvent("changes", R"({"n":1})"); + auto const second = encodedEvent("changes", R"({"n":2})"); + server->post(cid, first); + server->post(cid, second); + flush(*server); + + CHECK(wire.snapshot() == std::vector{first, second}); +} + +TEST_CASE("backlog_holds_events_not_replies", "[push][channel]") { + ::morph::exec::ThreadPoolExecutor pool{2}; + auto server = std::make_shared<::morph::backend::RemoteServer>(pool, pcEnv().dispatcher, pcEnv().registry); + FakeWire wire; + wire.backlog = std::size_t{2} << 20; + auto const cid = server->openConnection(); + server->setConnectionWriter(cid, wire.writer()); + auto const reply = [&wire](std::string frame) { wire.push(std::move(frame)); }; + + server->post(cid, encodedEvent("changes", "{}")); + server->handle(::morph::wire::encode(::morph::wire::makeRegister("PC_CounterModel")), reply, cid); + REQUIRE(::morph::testing::waitUntil([&] { return wire.snapshot().size() == 1; })); + auto const registered = ::morph::wire::decode(wire.snapshot().front()); + REQUIRE(registered.kind == "ok"); + + ::morph::wire::Envelope exec; + exec.kind = "execute"; + exec.callId = 7; + exec.modelId = registered.modelId; + exec.modelType = "PC_CounterModel"; + exec.actionType = "PC_BumpAction"; + exec.body = "{}"; + server->handle(::morph::wire::encode(exec), reply, cid); + server->post(cid, encodedEvent("changes", "{}")); + REQUIRE(::morph::testing::waitUntil([&] { return wire.snapshot().size() == 2; })); + server->post(cid, encodedEvent("changes", "{}")); + flush(*server); + + auto held = wire.snapshot(); + REQUIRE(held.size() == 2); + for (const auto& frame : held) { + CHECK(::morph::wire::decode(frame).kind == "ok"); + } + + // Below the bound but not yet drained: the resync is still owed, so an + // event now would arrive ahead of it. + wire.backlog = 0; + server->post(cid, encodedEvent("changes", "{}")); + flush(*server); + REQUIRE(wire.snapshot().size() == 2); + + server->connectionDrained(cid); + flush(*server); + auto const drained = wire.snapshot(); + REQUIRE(drained.size() == 3); + CHECK(drained[2] == encodedEvent("resync", "{}")); + + // The resync is owed once: a second drain sends nothing, and events flow again. + server->connectionDrained(cid); + auto const after = encodedEvent("changes", R"({"n":3})"); + server->post(cid, after); + flush(*server); + auto const resumed = wire.snapshot(); + REQUIRE(resumed.size() == 4); + CHECK(resumed[3] == after); +} + +TEST_CASE("post_after_close_is_dropped", "[push][channel]") { + ::morph::exec::ThreadPoolExecutor pool{2}; + auto server = std::make_shared<::morph::backend::RemoteServer>(pool, pcEnv().dispatcher, pcEnv().registry); + FakeWire wire; + auto const cid = server->openConnection(); + server->setConnectionWriter(cid, wire.writer()); + server->closeConnection(cid); + server->post(cid, encodedEvent("changes", "{}")); + flush(*server); + CHECK(wire.snapshot().empty()); + + // A writer handed over for a connection already closed is not kept. + server->setConnectionWriter(cid, wire.writer()); + server->post(cid, encodedEvent("changes", "{}")); + server->post(0, encodedEvent("changes", "{}")); + flush(*server); + CHECK(wire.snapshot().empty()); +} + +TEST_CASE("post_above_envelope_cap_is_dropped_and_counted", "[push][channel]") { + ::morph::observe::ScopedObserveOverride const restore; + std::atomic oversize{0}; + ::morph::observe::setMetricSink([&oversize](const ::morph::observe::MetricEvent& evt) { + if (evt.metric == ::morph::observe::Metric::eventOversize) { + oversize.fetch_add(1); + } + }); + ::morph::exec::ThreadPoolExecutor pool{2}; + auto server = std::make_shared<::morph::backend::RemoteServer>(pool, pcEnv().dispatcher, pcEnv().registry); + FakeWire wire; + auto const cid = server->openConnection(); + server->setConnectionWriter(cid, wire.writer()); + + server->post(cid, std::string(::morph::wire::kMaxEnvelopeBytes + 1, 'x')); + auto const fits = encodedEvent("changes", "{}"); + server->post(cid, fits); + flush(*server); + + CHECK(oversize.load() == 1); + CHECK(wire.snapshot() == std::vector{fits}); +} + +TEST_CASE("throwing_connection_writer_does_not_escape_the_server", "[push][channel]") { + ::morph::exec::ThreadPoolExecutor pool{2}; + auto server = std::make_shared<::morph::backend::RemoteServer>(pool, pcEnv().dispatcher, pcEnv().registry); + std::atomic attempts{0}; + auto const cid = server->openConnection(); + server->setConnectionWriter(cid, {.write = + [&attempts](const std::string&) { + attempts.fetch_add(1); + throw std::runtime_error("socket gone"); + }, + .backlog = [] { return std::size_t{0}; }}); + server->post(cid, encodedEvent("changes", "{}")); + server->post(cid, encodedEvent("changes", "{}")); + flush(*server); + CHECK(attempts.load() == 2); + + // The strand is still serving. + ::morph::testing::WaitReply hello; + server->handle(::morph::wire::encode(::morph::wire::makeHello()), std::ref(hello), cid); + REQUIRE(hello.await()); + CHECK(hello.env.kind == "ok"); +} From 0dfdc2c6cca6ba6a5c4d91e06e526d39049fc719 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 14:13:15 +0200 Subject: [PATCH 20/68] net: per-connection byte backlog, drain callback, events and the connect handler SocketServer hands RemoteServer a ConnectionWriter for each accepted connection. Every frame handed to it, replies and events alike, is counted on the connection's atomic backlog by the size of its encoded envelope, and released on the loop once its last chunk is written or it is dropped; a release that takes the backlog below kEventBacklogLow calls connectionDrained. SocketBackend routes event envelopes before reply matching and posts them to the setEventHandler executor, and runs the setConnectHandler handler after each connect, the first included. RemoteServer keeps an owed resync when a connection's writer is replaced, and catches a throwing backlog() read like a throwing write. Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/backend.md | 82 +++- include/morph/core/remote.hpp | 31 +- include/morph/net/detail/ws_connection.hpp | 85 +++- include/morph/net/socket_backend.hpp | 78 +++- include/morph/net/socket_server.hpp | 85 +++- tests/net/CMakeLists.txt | 1 + tests/net/test_push_net.cpp | 483 +++++++++++++++++++++ tests/test_push_channel.cpp | 56 +++ 8 files changed, 846 insertions(+), 55 deletions(-) create mode 100644 tests/net/test_push_net.cpp diff --git a/docs/spec/core/backend.md b/docs/spec/core/backend.md index b450a84ec..61bc9eb77 100644 --- a/docs/spec/core/backend.md +++ b/docs/spec/core/backend.md @@ -182,11 +182,12 @@ clears either. The reconnect handler differs on purpose: it re-binds the bridge's handlers, which is owner work, so the backend posts it to the executor it was installed with. -`QtWebSocketBackend` is currently the only backend that overrides either: -its `connected`/`disconnected` `QWebSocket` signal slots invoke -`_connectHandler`/`_disconnectHandler` (if installed) at the same points -they already invoke `_reconnectHandler`/schedule a reconnect — see that -section below. +`QtWebSocketBackend` overrides both: its `connected`/`disconnected` +`QWebSocket` signal slots invoke `_connectHandler`/`_disconnectHandler` (if +installed) at the same points they already invoke `_reconnectHandler`/schedule +a reconnect — see that section below. `morph::net::SocketBackend` overrides +`setConnectHandler`, run on its I/O loop after each connect (see +[Server-initiated events](#server-initiated-events-over-the-socket-transport)). ## Session propagation to control envelopes @@ -1025,7 +1026,9 @@ server one per connection. - `setConnectionWriter(cid, writer)` posts to the server strand and keeps the writer only while `cid`'s scope is open: a writer for `0`, an unknown `cid` or a closed one is dropped, so a transport racing its own disconnect cannot - leave a writer nothing removes. It replaces any writer `cid` already had; an + leave a writer nothing removes. It replaces any writer `cid` already had, + keeping a `resync` the connection is owed — the held-back events were the + connection's, not the old writer's; an empty `write` removes it. `closeConnection(cid)` drops it. The writer lives next to the connection scopes, in a `cid → {writer, owesResync}` map that only the strand touches. @@ -1055,9 +1058,9 @@ server one per connection. - The backlog is counted by the transport, because only it can see what it still holds; replies count toward it, and are never held back. A peer that stops reading altogether is the transport's to close, by its send timeout. -- A `write` that throws is caught on the strand and logged at `warn`; the - server keeps serving and the connection keeps its writer until the transport - closes it. +- A `write` or a `backlog()` read that throws is caught on the strand and + logged at `warn`, and the envelope is not written; the server keeps serving + and the connection keeps its writer until the transport closes it. ### Graceful shutdown (`beginShutdown()` / `drainedWithin()`) @@ -1570,8 +1573,8 @@ itself. | Component | Cross-thread surface | Everything else | |---|---|---| -| `SocketBackend` | `execute`, `bindModel`, `promoteModel`, `deregisterModel`, `cancelPending` post; `waitForConnected` posts a waiter and blocks the caller; the synchronous control verbs post their frame and wait on a future for the reply; `setSession`, `setReconnectHandler` post | socket, both pending tables, call-id counter, session, reconnect handler, reconnect delay and backoff timer, connect waiters: loop only | -| `SocketServer` | `listen()`, `close()` run on the loop and wait; `port()` is atomic; a `RemoteServer` reply is posted from the replying thread | listener, connections, writers: loop only | +| `SocketBackend` | `execute`, `bindModel`, `promoteModel`, `deregisterModel`, `cancelPending` post; `waitForConnected` posts a waiter and blocks the caller; the synchronous control verbs post their frame and wait on a future for the reply; `setSession`, `setReconnectHandler`, `setConnectHandler`, `setEventHandler` post | socket, both pending tables, call-id counter, session, reconnect, connect and event handlers, reconnect delay and backoff timer, connect waiters: loop only | +| `SocketServer` | `listen()`, `close()` run on the loop and wait; `port()` is atomic; a `RemoteServer` reply is posted from the replying thread, an event from the server strand, each counted on the connection's atomic backlog first | listener, connections, writers: loop only | The flows are `core::async::Task`s spawned on the loop. `SocketBackend` runs one attempt flow per connection attempt — dial through `core::net::connect` @@ -1583,6 +1586,61 @@ drains its queued frames one 64 KiB chunk at a time; a loop timer armed around each chunk's write (`sendTimeout`) closes the socket if the write makes no progress, which ends the connection like any other disconnect. +### Server-initiated events over the socket transport + +The server side counts each connection's backlog; the client side routes +`event` envelopes and reports each connect. See +[Writing to a connection](#writing-to-a-connection) for what `RemoteServer` +does with both. + +- **One writer per connection.** `SocketServer` opens the connection's scope + when it accepts it and at once hands `RemoteServer` a `ConnectionWriter` for + it. `write` encodes the envelope as a frame on the server strand and posts it + to the loop, as a reply's frame is posted from the replying thread; it + throws only when the loop is gone. Neither callable keeps the connection + alive: they hold a weak reference to it, and the server drops them when the + connection's scope closes. +- **The backlog counts encoded envelopes.** `LoopConnection::pending` is an + atomic count of the bytes of encoded envelopes handed to the connection — + replies and events alike — and not yet written. The size is added on the + handing thread, before the frame is posted, so a `backlog()` read on the + strand already covers a frame still on its way to the loop. The same size is + subtracted on the loop when the writer has written the frame's last chunk, + or when the frame is dropped: a connection closed with frames queued, or one + that is closed by the time the frame arrives. Framed byte counts are never + used, so the add and the subtract are always the same number and the count + cannot drift. The counter is held by `shared_ptr`, so a thread off the loop + keeps the counter without keeping the connection, whose socket must be + released on the loop. Frames the transport sends for itself (the handshake + response, Pong, the Close echo) are not counted. +- **The drain call.** When a written frame takes the backlog from at least + `kEventBacklogLow` to below it, the loop calls + `RemoteServer::connectionDrained(cid)`, judging the crossing from the value + its own subtraction replaced. A connection that is closed or closing is not + reported. Frames handed over meanwhile on other threads can lift the count + again and produce a second report; the server's drain handling sends nothing + when no `resync` is owed, so that is harmless. +- **A peer that stops reading** is closed by `sendTimeout`, as before: the + backlog bounds only what the server sends unasked. +- **Events are routed before reply matching.** `SocketBackend` hands every + decoded envelope to `routeEvent` before it looks in either pending table, so + an `event` is never taken for a reply, whatever its `callId`. The handler + installed by `setEventHandler` is stored on the loop and posted to its + executor once per event, in arrival order; with none installed, events are + dropped. +- **The connect handler** is stored on the loop and run there after every + connect, the first included: after the requests held while the socket was + down have been sent, and before the reconnect handler is posted. One that + throws is logged, and the connection carries on. + +`tests/net/test_push_net.cpp` pins it: an event posted to a connection reaches +the client's handler; an event carrying a pending call's own `callId` reaches +the handler and leaves the call to its real reply; the connect handler fires on +the first connect and on a reconnect, ahead of the reconnect handler; 100 +replies written return the backlog to zero with one drain report; and a reader +that stalls behind 8 MiB of replies gets no event posted meanwhile, then one +`resync` once it has read them. + `morph::net` keeps its own WebSocket framing (`detail/ws_frame.hpp`) and handshake text (`detail/ws_handshake.hpp`); only the socket underneath is core-cpp's. `detail/ws_connection.hpp` holds the loop-side pieces both roles @@ -2022,7 +2080,7 @@ server: each call is a loop task, so the loop serialises them. | `handleInline(msg)` | Sync: runs the control envelope on the server strand — posted and waited for, or inline when already on it — and returns the reply JSON. **Rejects `execute`** — returns an `err` reply without dispatching, because an `execute` reply is produced asynchronously after this call returns. Needs a pool thread other than the caller's free to run the strand. Unscoped. | | `openConnection()` | Returns a fresh non-zero `ConnectionId` at once; its empty scope is opened on the server strand, before the caller's next `handle()`. | | `closeConnection(cid)` | Posts: erases every model still recorded in `cid`'s scope (as `deregister` would) and drops the scope and its writer. `cid == 0`, unknown, or already-closed is a no-op — idempotent. Bypasses `IAuthorizer` by design. | -| `setConnectionWriter(cid, writer)` | Posts: keeps `writer` (`ConnectionWriter{write, backlog}`) for `cid` while its scope is open; replaces an earlier one; an empty `write` removes it. `0`, unknown or closed `cid`: dropped. | +| `setConnectionWriter(cid, writer)` | Posts: keeps `writer` (`ConnectionWriter{write, backlog}`) for `cid` while its scope is open; replaces an earlier one, keeping an owed `resync`; an empty `write` removes it. `0`, unknown or closed `cid`: dropped. | | `post(cid, envelope)` | Any thread. Above `wire::kMaxEnvelopeBytes`: dropped and counted (`Metric::eventOversize`). On the strand: written through `cid`'s writer, or held back — dropped, with one `resync` owed — while `backlog()` exceeds `kEventBacklogHigh` or a `resync` is still owed. No writer: dropped. | | `connectionDrained(cid)` | Any thread; posts. Sends the owed `resync` event, if any, and lets events flow again. What a transport calls when the backlog falls below `kEventBacklogLow`. | | `payloadCompleteness()` | The `ServerConfig::payloadCompleteness` given at construction. | diff --git a/include/morph/core/remote.hpp b/include/morph/core/remote.hpp index ea5791dee..a249148b6 100644 --- a/include/morph/core/remote.hpp +++ b/include/morph/core/remote.hpp @@ -196,7 +196,8 @@ struct ConnectionWriter { std::function write; /// @brief Bytes of encoded envelopes handed to the transport for this /// connection, replies and events alike, and not yet written. - /// Empty reads as `0`. + /// Empty reads as `0`. May throw; the server catches it and writes + /// nothing. std::function backlog; }; @@ -606,7 +607,8 @@ class RemoteServer : public std::enable_shared_from_this { /// @brief Hands the server the transport's writer for connection @p cid, /// through which `post` reaches it. /// - /// Replaces any writer @p cid already has. Kept only while the connection + /// Replaces any writer @p cid already has, keeping a `resync` the + /// connection is owed. Kept only while the connection /// is open: a writer for `0`, for an unknown `cid` or for one already /// closed is dropped, and `closeConnection(cid)` drops it. A writer whose /// `write` is empty removes the connection's writer. @@ -628,7 +630,9 @@ class RemoteServer : public std::enable_shared_from_this { self->_writers.erase(cid); return; } - self->_writers.insert_or_assign(cid, ConnectionOutbox{.writer = std::move(writer)}); + // A resync already owed stays owed: the events it stands for were + // held back from the connection, not from the writer it had then. + self->_writers[cid].writer = std::move(writer); }); } @@ -644,8 +648,8 @@ class RemoteServer : public std::enable_shared_from_this { /// /// Dropped as well: an envelope larger than `wire::kMaxEnvelopeBytes` /// (counted as `Metric::eventOversize`), and one for a connection with no - /// writer — `0`, unknown, or closed. A `write` that throws is caught on - /// the strand. + /// writer — `0`, unknown, or closed. A `write` or `backlog` that throws + /// is caught on the strand, and the envelope is not written. /// /// Callable from any thread. Envelopes one thread posts to one connection /// are written in that order. @@ -892,7 +896,17 @@ class RemoteServer : public std::enable_shared_from_this { if (outbox.owesResync) { return; } - if (outbox.writer.backlog && outbox.writer.backlog() > kEventBacklogHigh) { + std::size_t backlog = 0; + try { + backlog = outbox.writer.backlog ? outbox.writer.backlog() : 0; + } catch (const std::exception& exc) { + writeFailed(cid, exc.what()); + return; + } catch (...) { + writeFailed(cid, "unknown exception"); + return; + } + if (backlog > kEventBacklogHigh) { outbox.owesResync = true; return; } @@ -926,11 +940,12 @@ class RemoteServer : public std::enable_shared_from_this { } } - /// @brief What a failed write to connection @p cid does. On the server strand. + /// @brief What a failed write to connection @p cid, or a failed read of + /// its backlog, does. On the server strand. /// /// The connection keeps its writer: the transport reports the connection /// gone through `closeConnection`, which drops it. - /// @param cid Connection whose write threw. + /// @param cid Connection whose `write` or `backlog` threw. /// @param reason The exception's message. void writeFailed(ConnectionId cid, std::string_view reason) { noteOwner("RemoteServer::writeFailed"); diff --git a/include/morph/net/detail/ws_connection.hpp b/include/morph/net/detail/ws_connection.hpp index ebd0282c1..47e054fb2 100644 --- a/include/morph/net/detail/ws_connection.hpp +++ b/include/morph/net/detail/ws_connection.hpp @@ -3,12 +3,14 @@ #pragma once #include #include +#include #include #include #include #include #include #include +#include #include #include #include @@ -17,6 +19,7 @@ #include #include "../../core/profiler.hpp" +#include "../../core/remote.hpp" #include "ws_handshake.hpp" /// @file @@ -27,10 +30,20 @@ /// Everything here runs on the loop's thread and is touched nowhere else; the /// `LoopConnection` itself is held by `shared_ptr` from each flow that uses it, /// so a flow that resumes after its owner has let go of the connection still -/// finds the socket object alive. +/// finds the socket object alive. The one exception is the backlog counter, +/// `pending`, which other threads add to and read. namespace morph::net::detail { +/// @brief One whole frame waiting for the writer. +struct OutboundFrame { + /// @brief The encoded WebSocket frame. + std::string bytes; + /// @brief What the frame added to `LoopConnection::pending`: the size of + /// the envelope it carries, or `0` for a frame that is not counted. + std::size_t counted{0}; +}; + /// @brief A connected socket on the loop, plus its outgoing frame queue. struct LoopConnection { /// @param eventLoop The loop the socket is registered with. @@ -53,7 +66,23 @@ struct LoopConnection { /// @brief Bound on one chunk's write; see `writerFlow`. std::chrono::milliseconds sendTimeout; /// @brief Whole frames waiting for the writer, oldest first. - std::deque outbox; + std::deque outbox; + /// @brief Bytes of encoded envelopes handed to this connection and not yet + /// written: its backlog, as `RemoteServer` reads it. + /// + /// Added to by whoever hands over a frame, on any thread, before the frame + /// reaches the loop, so the count covers frames still on their way to the + /// outbox. Subtracted on the loop by the same amount once the frame's last + /// chunk is written, or once the frame is dropped. Counting envelopes, not + /// framed bytes, makes the add and the subtract the same number, so the + /// count cannot drift. Held by `shared_ptr` so a thread off the loop can + /// keep the counter without keeping the connection, whose socket must be + /// released on the loop. + std::shared_ptr> const pending{std::make_shared>(0)}; + /// @brief Called on the loop when a written frame takes `pending` from at + /// least `backend::kEventBacklogLow` to below it. Not called once + /// the connection is closed or closing. Empty means nobody is told. + std::function onDrained; /// @brief Whether a writer flow is running. bool writing{false}; /// @brief Whether the socket has been closed. @@ -62,17 +91,45 @@ struct LoopConnection { bool closeWhenFlushed{false}; }; +/// @brief Takes @p bytes off @p conn's backlog, and calls `onDrained` if +/// that takes it below `backend::kEventBacklogLow`. +/// +/// The crossing is judged from the value this subtraction replaced, so of +/// several releases that pass the bound together exactly one reports it. +/// Frames handed over meanwhile on other threads may lift the count again; +/// the server's drain handling is idempotent, so a second report is harmless. +/// @param conn The connection. +/// @param bytes What the frame added when it was handed over. +inline void releaseBacklog(LoopConnection& conn, std::size_t bytes) { + if (bytes == 0) { + return; + } + std::size_t const before = conn.pending->fetch_sub(bytes); + bool const crossed = + before >= ::morph::backend::kEventBacklogLow && before - bytes < ::morph::backend::kEventBacklogLow; + if (crossed && !conn.closed && !conn.closeWhenFlushed && conn.onDrained) { + conn.onDrained(); + } +} + /// @brief Closes @p conn's socket now, dropping any unwritten frames. /// /// A read or write parked on it resumes with `NetErrorCode::Cancelled` in a -/// later drain step of the loop, which is how the flows learn of it. +/// later drain step of the loop, which is how the flows learn of it. The +/// dropped frames leave the backlog; a frame the writer holds leaves it when +/// the writer lets go of it. /// @param conn The connection; idempotent. inline void closeConnection(LoopConnection& conn) noexcept { if (conn.closed) { return; } conn.closed = true; + std::size_t dropped = 0; + for (auto const& frame : conn.outbox) { + dropped += frame.counted; + } conn.outbox.clear(); + conn.pending->fetch_sub(dropped); conn.socket->close(); } @@ -89,13 +146,16 @@ inline void closeOnDeadline(void* connPtr) { closeConnection(*static_cast writerFlow(std::shared_ptr conn) { while (!conn->closed && !conn->outbox.empty()) { - std::string const frame = std::move(conn->outbox.front()); + OutboundFrame const next = std::move(conn->outbox.front()); conn->outbox.pop_front(); + std::string const& frame = next.bytes; std::size_t offset = 0; while (offset < frame.size() && !conn->closed) { std::size_t const length = std::min(kWriteChunk, frame.size() - offset); @@ -113,6 +173,7 @@ inline ::core::async::Task writerFlow(std::shared_ptr conn } offset += *written; } + releaseBacklog(*conn, next.counted); } conn->writing = false; if (conn->closeWhenFlushed) { @@ -121,18 +182,22 @@ inline ::core::async::Task writerFlow(std::shared_ptr conn } /// @brief Queues @p frame on @p conn and starts the writer if it is idle. -/// @param conn The connection. -/// @param frame One complete, encoded WebSocket frame. -/// @return `false` if the connection is closed and the frame was dropped. -inline bool enqueueFrame(std::shared_ptr const& conn, std::string frame) { +/// @param conn The connection. +/// @param frame One complete, encoded WebSocket frame. +/// @param counted What the caller added to `conn->pending` for this frame; +/// `0` for a frame that is not counted. +/// @return `false` if the connection is closed and the frame was dropped, its +/// count released with it. +inline bool enqueueFrame(std::shared_ptr const& conn, std::string frame, std::size_t counted = 0) { // The send side's zone is here and not in `writerFlow`: a zone must not // stay open across a `co_await`, where the loop runs other work on the // same thread before the writer resumes. MORPH_ZONE("ws::enqueueFrame"); if (conn->closed || conn->closeWhenFlushed) { + releaseBacklog(*conn, counted); return false; } - conn->outbox.push_back(std::move(frame)); + conn->outbox.push_back(OutboundFrame{.bytes = std::move(frame), .counted = counted}); if (!conn->writing) { conn->writing = true; conn->loop.spawn(writerFlow(conn)); diff --git a/include/morph/net/socket_backend.hpp b/include/morph/net/socket_backend.hpp index 7e5df069d..4fc3117c5 100644 --- a/include/morph/net/socket_backend.hpp +++ b/include/morph/net/socket_backend.hpp @@ -76,14 +76,15 @@ struct SocketBackendConfig { /// Mirrors `morph::qt::QtWebSocketBackend`'s observable behavior /// (`deregisterModel` fire-and-forget, `execute` and `bindModel` asynchronous /// and callId-multiplexed, `DisconnectedError`/reconnect semantics). Its -/// socket, its pending calls, its session, its reconnect handler and its -/// reconnect state machine live on an `exec::IoLoop`, which it shares with -/// every other socket, timer and probe the application builds on that loop. +/// socket, its pending calls, its session, its handlers and its reconnect +/// state machine live on an `exec::IoLoop`, which it shares with every other +/// socket, timer and probe the application builds on that loop. /// /// Cross-thread surface: `execute`, `bindModel`, `promoteModel`, `request`, -/// `deregisterModel`, `cancelPending`, `setSession` and `setReconnectHandler` -/// post to the loop and return, so calls from one thread take effect in the -/// order they were made; `waitForConnected` posts a waiter and blocks the +/// `deregisterModel`, `cancelPending`, `setSession`, `setReconnectHandler`, +/// `setConnectHandler` and `setEventHandler` post to the loop and return, so +/// calls from one thread take effect in the order they were made; +/// `waitForConnected` posts a waiter and blocks the /// caller, never the loop; the synchronous control verbs (`registerModel`, /// `registerModelWithContext`, `assignPrimary`, `listInstances`) issue the same /// posted request and wait for its reply, never on the loop's own thread. @@ -481,6 +482,43 @@ class SocketBackend : public ::morph::backend::detail::IBackend, public ::morph: }); } + /// @brief Installs the handler run after every successful connect, the + /// first included. + /// + /// Stored on the loop, and run there: after the requests held while the + /// socket was down have been sent, and before the reconnect handler is + /// posted. A handler that throws is logged; the connection carries on. + /// @param handler Callable run on the backend's I/O loop. Pass `nullptr` + /// to clear. + void setConnectHandler(const std::function& handler) override { + _loop->post([core = _core, handler] { + core->note("SocketBackend::setConnectHandler"); + core->connectHandler = handler; + }); + } + + /// @brief Installs the handler each server-initiated event is posted to. + /// See `IBackend::setEventHandler`. + /// + /// Stored on the loop. An `event` envelope is recognised by its kind + /// before any reply matching, so whatever its `callId`, it never settles a + /// call. Events are posted to @p exec in the order they arrived. + /// @param handler Callable posted to @p exec once per event. Pass + /// `nullptr` to clear. + /// @param exec Executor the handler is posted to. Borrowed: it must + /// outlive the installation. May be null only when + /// @p handler is. + void setEventHandler(std::function handler, + ::morph::exec::IExecutor* exec) override { + _loop->post([core = _core, handler = std::move(handler), exec]() mutable { + core->note("SocketBackend::setEventHandler"); + core->eventHandler = + handler ? std::make_shared>(std::move(handler)) + : nullptr; + core->eventExec = exec; + }); + } + /// @brief Installs the session stamped onto every control envelope this /// backend subsequently builds (`register`, `registerShared`, /// `attach`, `assign`, `deregister`, `instances`). See @@ -930,6 +968,15 @@ class SocketBackend : public ::morph::backend::detail::IBackend, public ::morph: for (auto& [env, pending] : std::exchange(queuedControls, {})) { fileControl(std::move(env), std::move(pending)); } + if (connectHandler) { + try { + connectHandler(); + } catch (const std::exception& exc) { + ::morph::log::logWarn("[SocketBackend] connect handler threw: {}", exc.what()); + } catch (...) { + ::morph::log::logWarn("[SocketBackend] connect handler threw a non-standard exception"); + } + } if (isReconnect && reconnectHandler && reconnectExec != nullptr) { // Posted, never run here: the handler re-registers through // `bindModel`, whose replies this loop delivers. @@ -986,6 +1033,8 @@ class SocketBackend : public ::morph::backend::detail::IBackend, public ::morph: } connectWaiters.clear(); reconnectHandler = nullptr; + connectHandler = nullptr; + eventHandler = nullptr; cancelAll(std::make_exception_ptr(::morph::backend::DisconnectedError{})); } @@ -1007,6 +1056,17 @@ class SocketBackend : public ::morph::backend::detail::IBackend, public ::morph: return; } note("SocketBackend::reply"); + // By kind first: an event's `callId` is not a reply's, so it must + // never reach the tables below. + if (::morph::backend::detail::routeEvent(env, [this](::morph::wire::EventBody event) { + if (eventHandler && eventExec != nullptr) { + eventExec->post([handler = eventHandler, event = std::move(event)]() mutable { + (*handler)(std::move(event)); + }); + } + })) { + return; + } auto pending = executes.take(env.callId); if (!pending) { // Not an execute: it may be a control reply for a `bindModel`/ @@ -1168,6 +1228,12 @@ class SocketBackend : public ::morph::backend::detail::IBackend, public ::morph: /// Posted to `reconnectExec` after every reconnect; cleared by `close()`. std::function reconnectHandler; ::morph::exec::IExecutor* reconnectExec{nullptr}; + /// Run on the loop after every connect; cleared by `close()`. + std::function connectHandler; + /// Posted to `eventExec` for each `event` envelope; cleared by `close()`. + /// Shared, so each posted delivery holds it without copying it. + std::shared_ptr> eventHandler; + ::morph::exec::IExecutor* eventExec{nullptr}; bool everConnected{false}; bool closed{false}; /// No further connect will be attempted: a bind made now is rejected, diff --git a/include/morph/net/socket_server.hpp b/include/morph/net/socket_server.hpp index 1ae076cc8..0387382cf 100644 --- a/include/morph/net/socket_server.hpp +++ b/include/morph/net/socket_server.hpp @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -284,15 +285,71 @@ class SocketServer { if (server == nullptr) { return; } - auto client = - std::make_shared(Client{.conn = std::make_shared<::morph::net::detail::LoopConnection>( - loop.loop(), std::move(socket), cfg.sendTimeout), - .cid = server->openConnection(), - .done = false}); + auto const cid = server->openConnection(); + auto conn = std::make_shared<::morph::net::detail::LoopConnection>(loop.loop(), std::move(socket), + cfg.sendTimeout); + // Weak: the connection belongs to this core, and is told nothing + // once `~SocketServer` has cut the core's tie to the server. + conn->onDrained = [weak = weak_from_this(), cid] { + if (auto const self = weak.lock(); self && self->server != nullptr) { + self->server->connectionDrained(cid); + } + }; + auto client = std::make_shared(Client{.conn = std::move(conn), .cid = cid, .done = false}); clients.push_back(client); + server->setConnectionWriter(cid, writerFor(client)); loop.loop().spawn(clientFlow(shared_from_this(), client)); } + /// The writer `RemoteServer::post` reaches @p client through, and its + /// backlog read. Neither keeps the connection: the server may hold + /// them until it learns the connection is gone. + [[nodiscard]] ::morph::backend::ConnectionWriter writerFor(std::shared_ptr const& client) const { + std::weak_ptr weak = client; + auto const counter = client->conn->pending; + return {.write = + [loopHandle = loop.weak(), weak, counter](std::string envelope) { + if (!sendEnvelope(loopHandle, weak, counter, envelope)) { + throw std::runtime_error("the connection's I/O loop is gone"); + } + }, + .backlog = [counter] { return counter->load(); }}; + } + + /// Counts @p envelope against the connection's backlog, then posts + /// its frame to the loop, to be queued on the connection if it is + /// still open. Callable from any thread: replies arrive on the + /// server's pool, events on its strand. Encoded on the calling thread, + /// so the loop only queues it. + /// @return `false` if the loop is gone, the frame dropped and its + /// count released. + static bool sendEnvelope(::morph::exec::IoLoop::Weak const& loopHandle, std::weak_ptr const& weak, + std::shared_ptr> const& counter, + std::string const& envelope) { + std::string frame = + ::morph::net::detail::encodeWsFrame(::morph::net::detail::WsOpcode::kText, envelope, /*mask=*/false); + std::size_t const counted = envelope.size(); + // Before the post, so a backlog read on the server strand already + // covers a frame still on its way to the loop. + counter->fetch_add(counted); + bool const posted = loopHandle.post([weak, counter, counted, frame = std::move(frame)]() mutable { + auto const target = weak.lock(); + if (!target) { + counter->fetch_sub(counted); + return; + } + ::morph::exec::detail::noteOwner("SocketServer::reply", target->conn->loop, + ::morph::exec::runningOn(target->conn->loop)); + // A finished client's connection is closed or closing, so this + // drops the frame and releases its count. + static_cast(::morph::net::detail::enqueueFrame(target->conn, std::move(frame), counted)); + }); + if (!posted) { + counter->fetch_sub(counted); + } + return posted; + } + /// Drops a listener that can no longer accept, so `port()` reads `0` /// and a later `listen()` can bind again. Connections already accepted /// keep running: only the listening socket is gone. @@ -422,24 +479,14 @@ class SocketServer { /// Hands one request to the server. Its reply comes back on whichever /// thread produces it — the server's pool, usually — and is posted - /// here, to be queued on this connection if it is still open. + /// here, to be queued on this connection if it is still open. It + /// counts against the connection's backlog like an event. void dispatch(std::shared_ptr const& client, std::string const& payload) const { std::weak_ptr const weak = client; server->handle( payload, - [loopHandle = loop.weak(), weak](const std::string& reply) { - // Encoded on the replying thread, so the loop only queues it. - std::string frame = ::morph::net::detail::encodeWsFrame(::morph::net::detail::WsOpcode::kText, - reply, /*mask=*/false); - static_cast(loopHandle.post([weak, frame = std::move(frame)]() mutable { - auto const target = weak.lock(); - if (!target || target->done) { - return; - } - ::morph::exec::detail::noteOwner("SocketServer::reply", target->conn->loop, - ::morph::exec::runningOn(target->conn->loop)); - static_cast(::morph::net::detail::enqueueFrame(target->conn, std::move(frame))); - })); + [loopHandle = loop.weak(), weak, counter = client->conn->pending](const std::string& reply) { + static_cast(sendEnvelope(loopHandle, weak, counter, reply)); }, client->cid); } diff --git a/tests/net/CMakeLists.txt b/tests/net/CMakeLists.txt index c312de9e6..589a41876 100644 --- a/tests/net/CMakeLists.txt +++ b/tests/net/CMakeLists.txt @@ -7,6 +7,7 @@ add_executable(morph_net_tests test_handshake_over_socket.cpp test_socket_server.cpp test_socket_backend.cpp + test_push_net.cpp ) target_link_libraries(morph_net_tests diff --git a/tests/net/test_push_net.cpp b/tests/net/test_push_net.cpp new file mode 100644 index 000000000..fe9654b62 --- /dev/null +++ b/tests/net/test_push_net.cpp @@ -0,0 +1,483 @@ +// SPDX-License-Identifier: Apache-2.0 + +// The `net` transport carrying server-initiated events: the server side's +// per-connection byte backlog and drain call, and the client side's event +// routing and connect handler. + +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../test_support.hpp" + +// External linkage: glaze reflection and the BRIDGE_REGISTER_* macros need it. +// NOLINTBEGIN(misc-use-internal-linkage) +struct PnEcho { + int value = 0; +}; +struct PnBig { + int size = 0; +}; +struct PnModel { + int execute(PnEcho action) { return action.value; } + // NOLINTNEXTLINE(modernize-return-braced-init-list): braces would pick the initializer_list constructor. + std::string execute(PnBig action) { return std::string(static_cast(action.size), 'x'); } +}; +// NOLINTEND(misc-use-internal-linkage) + +BRIDGE_REGISTER_MODEL(PnModel, "PnModel") +BRIDGE_REGISTER_ACTION(PnModel, PnEcho, "PnEcho") +BRIDGE_REGISTER_ACTION(PnModel, PnBig, "PnBig") + +namespace { + +using namespace std::chrono_literals; + +/// Runs each task inline on the posting thread, and counts them. +struct CountingExec final : ::morph::exec::IExecutor { + std::atomic posted{0}; + void post(std::function task) override { + posted.fetch_add(1); + task(); + } +}; + +/// What an event handler saw, from whichever thread delivered it. +struct EventLog { + std::mutex mtx; + std::vector<::morph::wire::EventBody> events; + + std::function handler() { + return [this](::morph::wire::EventBody body) { + std::scoped_lock const lock{mtx}; + events.push_back(std::move(body)); + }; + } + std::size_t size() { + std::scoped_lock const lock{mtx}; + return events.size(); + } + std::vector<::morph::wire::EventBody> snapshot() { + std::scoped_lock const lock{mtx}; + return events; + } +}; + +/// The connection id a fresh `SocketServer` connection to @p server will get: +/// `RemoteServer` numbers connections in order. +::morph::backend::ConnectionId nextConnectionId(::morph::backend::RemoteServer& server) { + auto const probe = server.openConnection(); + server.closeConnection(probe); + return probe + 1; +} + +std::string wsUrl(std::uint16_t port) { return "ws://127.0.0.1:" + std::to_string(static_cast(port)); } + +/// A raw WebSocket client against a real `SocketServer`. +class RawClient { +public: + explicit RawClient(std::uint16_t port, int receiveBuffer = 0) : _reader{/*expectMasked=*/false} { + _socket = ::morph::net::detail::TcpSocket::connect("127.0.0.1", port, 2000ms); + if (receiveBuffer > 0) { + ::setsockopt(_socket.nativeHandle(), SOL_SOCKET, SO_RCVBUF, &receiveBuffer, sizeof(receiveBuffer)); + } + ::morph::net::detail::ParsedWsUrl const url{.host = "127.0.0.1", .port = port, .path = "/"}; + _reader.feed(::morph::net::detail::performClientHandshake(_socket, url)); + } + + void send(const ::morph::wire::Envelope& env) { + std::string const frame = ::morph::net::detail::encodeWsFrame(::morph::net::detail::WsOpcode::kText, + ::morph::wire::encode(env), /*mask=*/true); + _socket.sendAll(frame.data(), frame.size()); + } + + ::morph::wire::Envelope receive() { + for (;;) { + if (auto frame = _reader.tryExtractFrame()) { + return ::morph::wire::decode(frame->payload); + } + std::array buf{}; + std::size_t const got = _socket.recvSome(buf.data(), buf.size()); + if (got == 0) { + throw std::runtime_error("RawClient: peer closed"); + } + _reader.feed(std::string_view{buf.data(), got}); + } + } + + void boundReceives(std::chrono::milliseconds timeout) { static_cast(_socket.setRecvTimeout(timeout)); } + +private: + ::morph::net::detail::TcpSocket _socket; + ::morph::net::detail::WsFrameReader _reader; +}; + +/// Closes and drops a test's `LoopConnection` on its loop when the test ends, +/// passed or failed: its socket must be released there. +struct LoopConnectionGuard { + ::morph::exec::IoLoop& io; + std::shared_ptr<::morph::net::detail::LoopConnection>& conn; + LoopConnectionGuard(::morph::exec::IoLoop& loop, std::shared_ptr<::morph::net::detail::LoopConnection>& held) + : io{loop}, conn{held} {} + LoopConnectionGuard(const LoopConnectionGuard&) = delete; + LoopConnectionGuard& operator=(const LoopConnectionGuard&) = delete; + LoopConnectionGuard(LoopConnectionGuard&&) = delete; + LoopConnectionGuard& operator=(LoopConnectionGuard&&) = delete; + ~LoopConnectionGuard() { + io.runAndWait([this] { + if (conn) { + ::morph::net::detail::closeConnection(*conn); + conn.reset(); + } + }); + } +}; + +::morph::wire::Envelope executeEnvelope(std::uint64_t callId, std::uint64_t modelId, std::string_view action, + std::string body) { + ::morph::wire::Envelope env; + env.kind = "execute"; + env.callId = callId; + env.modelId = modelId; + env.modelType = "PnModel"; + env.actionType = std::string{action}; + env.body = std::move(body); + return env; +} + +} // namespace + +TEST_CASE("net_posted_event_reaches_backend_handler", "[push][channel]") { + ::morph::exec::ThreadPoolExecutor pool{2}; + auto server = std::make_shared<::morph::backend::RemoteServer>(pool); + ::morph::net::SocketServer wsServer{*server, 0}; + REQUIRE(wsServer.listen()); + auto const cid = nextConnectionId(*server); + + CountingExec exec; + EventLog log; + ::morph::net::SocketBackend backend{wsUrl(wsServer.port())}; + backend.setEventHandler(log.handler(), &exec); + REQUIRE(backend.waitForConnected()); + + server->post(cid, ::morph::wire::encode(::morph::wire::makeEvent("changes", R"({"n":1})"))); + server->post(cid, ::morph::wire::encode(::morph::wire::makeEvent("resync", "{}"))); + REQUIRE(::morph::testing::waitUntil([&] { return log.size() == 2; })); + + auto const got = log.snapshot(); + CHECK(got[0].topic == "changes"); + CHECK(got[0].payload.str == R"({"n":1})"); + CHECK(got[1].topic == "resync"); + CHECK(exec.posted.load() == 2); + + // Cleared: a later event is dropped, and the connection keeps working. + backend.setEventHandler(nullptr, nullptr); + server->post(cid, ::morph::wire::encode(::morph::wire::makeEvent("changes", "{}"))); + CHECK(backend.registerModel("PnModel", nullptr).v != 0U); + CHECK(log.size() == 2); +} + +TEST_CASE("net_event_is_routed_before_reply_matching", "[push][channel]") { + // A fake server, so an event can carry the very callId a pending execute + // was filed under: only routing by kind first keeps it off that call. + auto listener = ::morph::net::detail::TcpSocket::listen(0); + ::morph::net::SocketBackend backend{wsUrl(listener.boundPort())}; + auto peer = listener.accept(); + ::morph::net::detail::WsFrameReader reader{/*expectMasked=*/true}; + reader.feed(::morph::net::detail::performServerHandshake(peer)); + REQUIRE(backend.waitForConnected()); + + CountingExec exec; + EventLog log; + backend.setEventHandler(log.handler(), &exec); + + ::morph::backend::detail::ActionCall call; + call.modelTypeId = "PnModel"; + call.actionTypeId = "PnEcho"; + call.serializeAction = [](const void*) { return std::string{"{}"}; }; + call.deserializeResult = [](std::string_view body) -> std::shared_ptr { + return std::make_shared(std::stoi(std::string{body})); + }; + ::morph::exec::MainThreadExecutor owner; + std::atomic result{-1}; + std::atomic failed{false}; + backend.execute(::morph::exec::detail::ModelId{1}, std::move(call), &owner) + .then([&](const std::shared_ptr& value) { result.store(*std::static_pointer_cast(value)); }) + .onError([&](const std::exception_ptr&) { failed.store(true); }); + + ::morph::wire::Envelope request; + for (;;) { + if (auto frame = reader.tryExtractFrame()) { + request = ::morph::wire::decode(frame->payload); + break; + } + std::array buf{}; + std::size_t const got = peer.recvSome(buf.data(), buf.size()); + REQUIRE(got != 0); + reader.feed(std::string_view{buf.data(), got}); + } + REQUIRE(request.kind == "execute"); + REQUIRE(request.callId != 0); + + auto sendText = [&peer](const ::morph::wire::Envelope& env) { + std::string const frame = ::morph::net::detail::encodeWsFrame(::morph::net::detail::WsOpcode::kText, + ::morph::wire::encode(env), /*mask=*/false); + peer.sendAll(frame.data(), frame.size()); + }; + auto sameId = ::morph::wire::makeEvent("changes", R"({"which":"same"})"); + sameId.callId = request.callId; + sendText(sameId); + sendText(::morph::wire::makeEvent("changes", R"({"which":"zero"})")); + REQUIRE(::morph::testing::waitUntil([&] { return log.size() == 2; })); + for (int i = 0; i < 5; ++i) { + owner.runFor(10ms); + } + CHECK(result.load() == -1); + CHECK_FALSE(failed.load()); + + // The call is still pending, and its own reply settles it. + sendText(::morph::wire::makeOk(request.callId, "7")); + for (int i = 0; i < 200 && result.load() == -1 && !failed.load(); ++i) { + owner.runFor(10ms); + } + CHECK(result.load() == 7); + CHECK_FALSE(failed.load()); +} + +TEST_CASE("net_connect_handler_fires_on_first_and_reconnect", "[push][channel]") { + ::morph::exec::ThreadPoolExecutor pool{2}; + auto server = std::make_shared<::morph::backend::RemoteServer>(pool); + + // Learn a free port, then leave it closed: the backend starts retrying + // before anything listens, so its handlers are installed before the first + // connect can happen. + std::uint16_t port = 0; + { + ::morph::net::SocketServer probe{*server, 0}; + REQUIRE(probe.listen()); + port = probe.port(); + } + + ::morph::net::SocketBackend::Config cfg; + cfg.initialReconnectDelay = 20ms; + cfg.maxReconnectDelay = 100ms; + ::morph::net::SocketBackend backend{wsUrl(port), cfg}; + + std::mutex mtx; + std::vector seen; + auto record = [&](std::string what) { + std::scoped_lock const lock{mtx}; + seen.push_back(std::move(what)); + }; + auto snapshot = [&] { + std::scoped_lock const lock{mtx}; + return seen; + }; + CountingExec reconnectExec; + backend.setConnectHandler([&] { record("connect"); }); + backend.setReconnectHandler([&] { record("reconnect"); }, &reconnectExec); + + { + ::morph::net::SocketServer first{*server, port}; + REQUIRE(first.listen()); + REQUIRE(::morph::testing::waitUntil([&] { return snapshot().size() == 1; })); + } + REQUIRE(::morph::testing::waitUntil([&] { return !backend.waitForConnected(0ms); })); + + ::morph::net::SocketServer second{*server, port}; + REQUIRE(second.listen()); + REQUIRE(::morph::testing::waitUntil([&] { return snapshot().size() == 3; })); + // The connect handler runs before the reconnect handler is posted. + CHECK(snapshot() == std::vector{"connect", "connect", "reconnect"}); +} + +TEST_CASE("net_counter_returns_to_zero", "[push][channel]") { + // One connection's backlog, end to end on a socket pair: 100 replies, + // counted as they are handed over, written, and read by the peer. + std::array fds{}; + REQUIRE(::socketpair(AF_UNIX, SOCK_STREAM, 0, fds.data()) == 0); + REQUIRE(::fcntl(fds[0], F_SETFL, ::fcntl(fds[0], F_GETFL) | O_NONBLOCK) == 0); + + ::morph::exec::IoLoop io; + std::shared_ptr<::morph::net::detail::LoopConnection> conn; + LoopConnectionGuard const guard{io, conn}; + std::atomic drained{0}; + io.runAndWait([&] { + auto socket = ::core::net::adoptFd(io.loop(), fds[0]); + REQUIRE(socket); + conn = std::make_shared<::morph::net::detail::LoopConnection>(io.loop(), std::move(*socket), 5000ms); + conn->onDrained = [&drained] { drained.fetch_add(1); }; + }); + + // 100 replies of ~8 KiB: about 800 KiB in all, so the backlog passes + // `kEventBacklogLow` on the way up and crosses it once on the way down. + constexpr int kReplies = 100; + std::vector envelopes; + std::size_t total = 0; + for (int i = 0; i < kReplies; ++i) { + envelopes.push_back(::morph::wire::encode( + ::morph::wire::makeOk(static_cast(i + 1), '"' + std::string(8192, 'r') + '"'))); + total += envelopes.back().size(); + } + REQUIRE(total >= ::morph::backend::kEventBacklogLow); + for (auto const& envelope : envelopes) { + conn->pending->fetch_add(envelope.size()); + } + CHECK(conn->pending->load() == total); + for (auto& envelope : envelopes) { + std::size_t const counted = envelope.size(); + std::string frame = ::morph::net::detail::encodeWsFrame(::morph::net::detail::WsOpcode::kText, envelope, + /*mask=*/false); + io.post([conn, frame = std::move(frame), counted]() mutable { + static_cast(::morph::net::detail::enqueueFrame(conn, std::move(frame), counted)); + }); + } + + ::morph::net::detail::WsFrameReader reader{/*expectMasked=*/false}; + int frames = 0; + std::array buf{}; + while (frames < kReplies) { + if (auto frame = reader.tryExtractFrame()) { + ++frames; + continue; + } + auto const got = ::read(fds[1], buf.data(), buf.size()); + REQUIRE(got > 0); + reader.feed(std::string_view{buf.data(), static_cast(got)}); + } + + REQUIRE(::morph::testing::waitUntil([&] { return conn->pending->load() == 0; })); + CHECK(drained.load() == 1); + ::close(fds[1]); +} + +TEST_CASE("net_closing_a_connection_releases_what_it_still_held", "[push][channel]") { + std::array fds{}; + REQUIRE(::socketpair(AF_UNIX, SOCK_STREAM, 0, fds.data()) == 0); + REQUIRE(::fcntl(fds[0], F_SETFL, ::fcntl(fds[0], F_GETFL) | O_NONBLOCK) == 0); + + ::morph::exec::IoLoop io; + std::shared_ptr<::morph::net::detail::LoopConnection> conn; + LoopConnectionGuard const guard{io, conn}; + std::atomic drained{0}; + io.runAndWait([&] { + auto socket = ::core::net::adoptFd(io.loop(), fds[0]); + REQUIRE(socket); + conn = std::make_shared<::morph::net::detail::LoopConnection>(io.loop(), std::move(*socket), 5000ms); + conn->onDrained = [&drained] { drained.fetch_add(1); }; + // Nothing reads the peer: most of 4 MiB stays queued. + for (int i = 0; i < 64; ++i) { + std::string const frame(64 * 1024, 'q'); + conn->pending->fetch_add(frame.size()); + static_cast(::morph::net::detail::enqueueFrame(conn, frame, frame.size())); + } + }); + CHECK(conn->pending->load() > ::morph::backend::kEventBacklogHigh); + + io.runAndWait([&] { ::morph::net::detail::closeConnection(*conn); }); + REQUIRE(::morph::testing::waitUntil([&] { return conn->pending->load() == 0; })); + // A closed connection is not reported drained: nothing can be sent on it. + CHECK(drained.load() == 0); + + // A frame handed over after the close is released at once. + io.runAndWait([&] { + conn->pending->fetch_add(10); + CHECK_FALSE(::morph::net::detail::enqueueFrame(conn, std::string(10, 'z'), 10)); + CHECK(conn->pending->load() == 0); + }); + ::close(fds[1]); +} + +TEST_CASE("net_replies_written_leave_no_backlog", "[push][channel]") { + // 100 replies of 12 KiB through a real `SocketServer`: 1.2 MiB, past the + // hold-back bound. Once all are read, an event still flows, which it + // would not if the server had kept counting what it had written. + ::morph::exec::ThreadPoolExecutor pool{2}; + auto server = std::make_shared<::morph::backend::RemoteServer>(pool); + ::morph::net::SocketServer wsServer{*server, 0}; + REQUIRE(wsServer.listen()); + auto const cid = nextConnectionId(*server); + + RawClient client{wsServer.port()}; + client.boundReceives(10s); + client.send(::morph::wire::makeRegister("PnModel")); + auto const reg = client.receive(); + REQUIRE(reg.kind == "ok"); + + constexpr int kReplies = 100; + for (int i = 0; i < kReplies; ++i) { + client.send(executeEnvelope(static_cast(i + 1), reg.modelId, "PnBig", R"({"size":12288})")); + } + for (int i = 0; i < kReplies; ++i) { + REQUIRE(client.receive().kind == "ok"); + } + REQUIRE(::morph::testing::awaitAnswer([&](auto& owner) { return server->drainedWithin(10s, owner); })); + + auto const event = ::morph::wire::encode(::morph::wire::makeEvent("changes", R"({"after":"replies"})")); + server->post(cid, event); + auto const got = client.receive(); + CHECK(got.kind == "event"); + CHECK(::morph::wire::encode(got) == event); +} + +TEST_CASE("net_backlog_holds_events_and_drain_sends_resync", "[push][channel]") { + // A reader that stops reading while 8 MiB of replies are queued for it: + // an event posted then is held back, and once the reader catches up the + // drain sends one `resync` in its place. + ::morph::exec::ThreadPoolExecutor pool{2}; + auto server = std::make_shared<::morph::backend::RemoteServer>(pool); + ::morph::net::SocketServer wsServer{*server, 0}; + REQUIRE(wsServer.listen()); + auto const cid = nextConnectionId(*server); + + RawClient client{wsServer.port(), /*receiveBuffer=*/2048}; + client.boundReceives(20s); + client.send(::morph::wire::makeRegister("PnModel")); + auto const reg = client.receive(); + REQUIRE(reg.kind == "ok"); + + constexpr int kReplies = 64; + for (int i = 0; i < kReplies; ++i) { + client.send(executeEnvelope(static_cast(i + 1), reg.modelId, "PnBig", R"({"size":131072})")); + } + // Every reply has been handed to the transport once none is in flight. + REQUIRE(::morph::testing::awaitAnswer([&](auto& owner) { return server->drainedWithin(20s, owner); })); + server->post(cid, ::morph::wire::encode(::morph::wire::makeEvent("changes", R"({"held":true})"))); + + for (int i = 0; i < kReplies; ++i) { + auto const reply = client.receive(); + REQUIRE(reply.kind == "ok"); + } + auto const next = client.receive(); + CHECK(next.kind == "event"); + CHECK(::morph::wire::encode(next) == ::morph::wire::encode(::morph::wire::makeEvent("resync", "{}"))); +} diff --git a/tests/test_push_channel.cpp b/tests/test_push_channel.cpp index 3a02df904..5fc3ea8a7 100644 --- a/tests/test_push_channel.cpp +++ b/tests/test_push_channel.cpp @@ -4,6 +4,7 @@ // matching, the local backend's delivery seam, and `RemoteServer` writing // events to a connection and holding them back from a slow reader. +#include #include #include #include @@ -14,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -310,3 +312,57 @@ TEST_CASE("throwing_connection_writer_does_not_escape_the_server", "[push][chann REQUIRE(hello.await()); CHECK(hello.env.kind == "ok"); } + +TEST_CASE("replacing_a_connection_writer_keeps_an_owed_resync", "[push][channel]") { + ::morph::exec::ThreadPoolExecutor pool{2}; + auto server = std::make_shared<::morph::backend::RemoteServer>(pool, pcEnv().dispatcher, pcEnv().registry); + FakeWire slow; + slow.backlog = std::size_t{2} << 20; + auto const cid = server->openConnection(); + server->setConnectionWriter(cid, slow.writer()); + server->post(cid, encodedEvent("changes", "{}")); + flush(*server); + REQUIRE(slow.snapshot().empty()); + + // The transport hands over a new writer while the resync is owed: what + // was held back is still owed, and the drain sends it on the new writer. + FakeWire fresh; + server->setConnectionWriter(cid, fresh.writer()); + server->post(cid, encodedEvent("changes", "{}")); + server->connectionDrained(cid); + flush(*server); + CHECK(slow.snapshot().empty()); + CHECK(fresh.snapshot() == std::vector{encodedEvent("resync", "{}")}); +} + +TEST_CASE("throwing_backlog_read_is_caught_like_a_throwing_write", "[push][channel]") { + std::mutex logMtx; + std::vector lines; + ::morph::log::ScopedLoggerOverride const guard{[&](::morph::log::LogLevel, std::string_view msg) { + std::scoped_lock const lock{logMtx}; + lines.emplace_back(msg); + }}; + ::morph::exec::ThreadPoolExecutor pool{2}; + auto server = std::make_shared<::morph::backend::RemoteServer>(pool, pcEnv().dispatcher, pcEnv().registry); + std::atomic writes{0}; + auto const cid = server->openConnection(); + server->setConnectionWriter(cid, {.write = [&writes](const std::string&) { writes.fetch_add(1); }, + .backlog = []() -> std::size_t { throw std::runtime_error("counter gone"); }}); + server->post(cid, encodedEvent("changes", "{}")); + flush(*server); + CHECK(writes.load() == 0); + { + std::scoped_lock const lock{logMtx}; + auto const reported = std::ranges::count_if(lines, [cid](const std::string& line) { + return line.find("write to connection " + std::to_string(cid) + " failed: counter gone") != + std::string::npos; + }); + CHECK(reported == 1); + } + + // The strand is still serving. + ::morph::testing::WaitReply hello; + server->handle(::morph::wire::encode(::morph::wire::makeHello()), std::ref(hello), cid); + REQUIRE(hello.await()); + CHECK(hello.env.kind == "ok"); +} From ab1a1c53657ba1a6a96ab52e362623d39badd619 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 14:21:27 +0200 Subject: [PATCH 21/68] net: a connection gets its writer after the handshake; backlog constants in their own header SocketServer hands RemoteServer a connection's writer only once the 101 response is queued, so an event posted to a connection still in its handshake is dropped instead of reaching the peer ahead of the 101. ConnectionWriter and the backlog bounds move to core/connection_writer.hpp, so the shared connection code no longer compiles RemoteServer. Co-Authored-By: Claude Opus 5.5 --- CMakeLists.txt | 1 + docs/spec/core/backend.md | 6 +- include/morph/core/connection_writer.hpp | 46 ++++++++++++++ include/morph/core/remote.hpp | 29 +-------- include/morph/net/detail/ws_connection.hpp | 2 +- include/morph/net/socket_server.hpp | 7 ++- tests/net/test_push_net.cpp | 71 +++++++++++++++++++++- 7 files changed, 128 insertions(+), 34 deletions(-) create mode 100644 include/morph/core/connection_writer.hpp diff --git a/CMakeLists.txt b/CMakeLists.txt index fb7729cbb..cc907ba11 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -459,6 +459,7 @@ target_sources(morph include/morph/core/changes.hpp include/morph/core/backend.hpp include/morph/core/envelope_transport.hpp + include/morph/core/connection_writer.hpp include/morph/core/remote.hpp include/morph/core/bridge.hpp include/morph/core/wire.hpp diff --git a/docs/spec/core/backend.md b/docs/spec/core/backend.md index 61bc9eb77..9973e162e 100644 --- a/docs/spec/core/backend.md +++ b/docs/spec/core/backend.md @@ -1594,8 +1594,10 @@ The server side counts each connection's backlog; the client side routes does with both. - **One writer per connection.** `SocketServer` opens the connection's scope - when it accepts it and at once hands `RemoteServer` a `ConnectionWriter` for - it. `write` encodes the envelope as a frame on the server strand and posts it + when it accepts it, and hands `RemoteServer` a `ConnectionWriter` for it only + once the handshake's `101` response is queued. A frame queued earlier would + reach the peer ahead of the `101` and break the handshake; until then the + connection has no writer, so what the server posts to it is dropped. `write` encodes the envelope as a frame on the server strand and posts it to the loop, as a reply's frame is posted from the replying thread; it throws only when the loop is gone. Neither callable keeps the connection alive: they hold a weak reference to it, and the server drops them when the diff --git a/include/morph/core/connection_writer.hpp b/include/morph/core/connection_writer.hpp new file mode 100644 index 000000000..9cac185e7 --- /dev/null +++ b/include/morph/core/connection_writer.hpp @@ -0,0 +1,46 @@ +// SPDX-License-Identifier: Apache-2.0 + +#pragma once +#include +#include +#include + +/// @file +/// @brief What a transport hands `RemoteServer` to write to one connection, +/// and the backlog bounds both sides apply. +/// +/// Kept apart from `remote.hpp` so a transport's connection code, which both +/// the client and the server role share, can use the bounds without compiling +/// the server. + +namespace morph::backend { + +/// @brief A transport's outbound side of one connection, for what the server +/// sends unasked: events. +/// +/// Handed to `RemoteServer::setConnectionWriter`. Both callables run on the +/// server strand, so neither may block: `write` queues the frame on the +/// transport's own thread, as a reply is queued. +struct ConnectionWriter { + /// @brief Queues one encoded envelope for the connection. May throw when + /// the connection can no longer be written; the server catches it. + std::function write; + /// @brief Bytes of encoded envelopes handed to the transport for this + /// connection, replies and events alike, and not yet written. + /// Empty reads as `0`. May throw; the server catches it and writes + /// nothing. + std::function backlog; +}; + +/// @brief Backlog above which `RemoteServer::post` holds a connection's events +/// back and owes it one `resync` instead: 1 MiB. +inline constexpr std::size_t kEventBacklogHigh = std::size_t{1} << 20; + +/// @brief Backlog below which a transport reports a connection drained, by +/// calling `RemoteServer::connectionDrained`: 256 KiB. +/// +/// Well under `kEventBacklogHigh`, so a reader hovering at the bound does not +/// flip between held back and drained on every frame. +inline constexpr std::size_t kEventBacklogLow = std::size_t{256} << 10; + +} // namespace morph::backend diff --git a/include/morph/core/remote.hpp b/include/morph/core/remote.hpp index a249148b6..76df6094a 100644 --- a/include/morph/core/remote.hpp +++ b/include/morph/core/remote.hpp @@ -27,6 +27,7 @@ #include "../journal/action_log.hpp" #include "../session/session.hpp" #include "completion.hpp" +#include "connection_writer.hpp" #include "detail/owner_probe.hpp" #include "detail/reply_router.hpp" #include "logger.hpp" @@ -184,34 +185,6 @@ class OpaqueIdGenerator { /// values are minted by `RemoteServer::openConnection()`. using ConnectionId = std::uint64_t; -/// @brief A transport's outbound side of one connection, for what the server -/// sends unasked: events. -/// -/// Handed to `RemoteServer::setConnectionWriter`. Both callables run on the -/// server strand, so neither may block: `write` queues the frame on the -/// transport's own thread, as a reply is queued. -struct ConnectionWriter { - /// @brief Queues one encoded envelope for the connection. May throw when - /// the connection can no longer be written; the server catches it. - std::function write; - /// @brief Bytes of encoded envelopes handed to the transport for this - /// connection, replies and events alike, and not yet written. - /// Empty reads as `0`. May throw; the server catches it and writes - /// nothing. - std::function backlog; -}; - -/// @brief Backlog above which `RemoteServer::post` holds a connection's events -/// back and owes it one `resync` instead: 1 MiB. -inline constexpr std::size_t kEventBacklogHigh = std::size_t{1} << 20; - -/// @brief Backlog below which a transport reports a connection drained, by -/// calling `RemoteServer::connectionDrained`: 256 KiB. -/// -/// Well under `kEventBacklogHigh`, so a reader hovering at the bound does not -/// flip between held back and drained on every frame. -inline constexpr std::size_t kEventBacklogLow = std::size_t{256} << 10; - /// @brief Snapshot of a `RemoteServer`'s current health. struct HealthStatus { /// @brief `true` if the server currently accepts and dispatches new work. diff --git a/include/morph/net/detail/ws_connection.hpp b/include/morph/net/detail/ws_connection.hpp index 47e054fb2..70faa6940 100644 --- a/include/morph/net/detail/ws_connection.hpp +++ b/include/morph/net/detail/ws_connection.hpp @@ -18,8 +18,8 @@ #include #include +#include "../../core/connection_writer.hpp" #include "../../core/profiler.hpp" -#include "../../core/remote.hpp" #include "ws_handshake.hpp" /// @file diff --git a/include/morph/net/socket_server.hpp b/include/morph/net/socket_server.hpp index 0387382cf..2c8ea2159 100644 --- a/include/morph/net/socket_server.hpp +++ b/include/morph/net/socket_server.hpp @@ -297,7 +297,6 @@ class SocketServer { }; auto client = std::make_shared(Client{.conn = std::move(conn), .cid = cid, .done = false}); clients.push_back(client); - server->setConnectionWriter(cid, writerFor(client)); loop.loop().spawn(clientFlow(shared_from_this(), client)); } @@ -408,6 +407,12 @@ class SocketServer { static_cast(::morph::net::detail::enqueueFrame( connection, ::morph::net::detail::buildServerHandshakeResponse(request.key))); leftover = std::move(header->leftover); + // Only now: a frame the server posted earlier would be + // queued ahead of the 101 and break the handshake. Until + // here the server has no writer, and drops what it posts. + if (self->server != nullptr) { + self->server->setConnectionWriter(client->cid, self->writerFor(client)); + } } catch (const std::exception&) { // NOLINT(bugprone-empty-catch) // A malformed Upgrade request: the connection ends below. } diff --git a/tests/net/test_push_net.cpp b/tests/net/test_push_net.cpp index fe9654b62..28a09cd7f 100644 --- a/tests/net/test_push_net.cpp +++ b/tests/net/test_push_net.cpp @@ -5,6 +5,7 @@ // routing and connect handler. #include +#include #include #include @@ -30,6 +31,7 @@ #include #include #include +#include #include #include #include @@ -449,10 +451,75 @@ TEST_CASE("net_replies_written_leave_no_backlog", "[push][channel]") { CHECK(::morph::wire::encode(got) == event); } +TEST_CASE("net_event_posted_during_the_handshake_is_dropped", "[push][channel]") { + // Connection ids are sequential, so a fan-out can name a connection still + // in its handshake. Nothing may reach the peer ahead of the `101`. + ::morph::exec::ThreadPoolExecutor pool{2}; + auto server = std::make_shared<::morph::backend::RemoteServer>(pool); + ::morph::net::SocketServer wsServer{*server, 0}; + REQUIRE(wsServer.listen()); + std::uint16_t const port = wsServer.port(); + + // Find the accepted connection's id without guessing when the accept ran: + // probes take ids in sequence, and the one a probe skips is the + // connection's. + auto previous = nextConnectionId(*server) - 1; + auto socket = ::morph::net::detail::TcpSocket::connect("127.0.0.1", port, 2000ms); + ::morph::backend::ConnectionId cid = 0; + REQUIRE(::morph::testing::waitUntil([&] { + auto const probe = server->openConnection(); + server->closeConnection(probe); + if (probe != previous + 1) { + cid = previous + 1; + return true; + } + previous = probe; + return false; + })); + + server->post(cid, ::morph::wire::encode(::morph::wire::makeEvent("changes", R"({"when":"handshake"})"))); + pollfd early{.fd = socket.nativeHandle(), .events = POLLIN, .revents = 0}; + CHECK(::poll(&early, 1, 300) == 0); // nothing is sent before the Upgrade request + + ::morph::net::detail::ParsedWsUrl const url{.host = "127.0.0.1", .port = port, .path = "/"}; + std::string const key = ::morph::net::detail::generateClientKey(); + std::string const upgrade = ::morph::net::detail::buildClientHandshakeRequest(url, key); + socket.sendAll(upgrade.data(), upgrade.size()); + REQUIRE(socket.setRecvTimeout(5000ms)); + std::string received; + std::array buf{}; + while (received.find("\r\n\r\n") == std::string::npos) { + std::size_t const got = socket.recvSome(buf.data(), buf.size()); + REQUIRE(got != 0); + received.append(buf.data(), got); + } + CHECK(received.starts_with("HTTP/1.1 101")); + + // The pre-handshake event never arrives: the first frame is the one + // posted after the handshake. + auto const after = ::morph::wire::encode(::morph::wire::makeEvent("changes", R"({"when":"after"})")); + server->post(cid, after); + ::morph::net::detail::WsFrameReader reader{/*expectMasked=*/false}; + reader.feed(received.substr(received.find("\r\n\r\n") + 4)); + std::optional<::morph::net::detail::WsFrame> frame; + while (!(frame = reader.tryExtractFrame())) { + std::size_t const got = socket.recvSome(buf.data(), buf.size()); + REQUIRE(got != 0); + reader.feed(std::string_view{buf.data(), got}); + } + CHECK(frame->payload == after); +} + TEST_CASE("net_backlog_holds_events_and_drain_sends_resync", "[push][channel]") { - // A reader that stops reading while 8 MiB of replies are queued for it: + // A reader that stops reading while 16 MiB of replies are queued for it: // an event posted then is held back, and once the reader catches up the // drain sends one `resync` in its place. + // + // Assumes the kernel's loopback buffers (the server's send buffer and the + // client's receive window, shrunk to 2 KiB) absorb far less than 15 MiB, + // so more than `kEventBacklogHigh` is still in the transport's backlog + // when the event is posted. Were the kernel to take nearly everything, + // the event would be delivered and this test would fail, not pass. ::morph::exec::ThreadPoolExecutor pool{2}; auto server = std::make_shared<::morph::backend::RemoteServer>(pool); ::morph::net::SocketServer wsServer{*server, 0}; @@ -465,7 +532,7 @@ TEST_CASE("net_backlog_holds_events_and_drain_sends_resync", "[push][channel]") auto const reg = client.receive(); REQUIRE(reg.kind == "ok"); - constexpr int kReplies = 64; + constexpr int kReplies = 128; for (int i = 0; i < kReplies; ++i) { client.send(executeEnvelope(static_cast(i + 1), reg.modelId, "PnBig", R"({"size":131072})")); } From e15681ceb8b0027daf340ce17045789e1330e974 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 14:37:52 +0200 Subject: [PATCH 22/68] qt: per-connection backlog, events, and a send timeout like net's QtWebSocketServer hands RemoteServer a ConnectionWriter for each connection as it is accepted (the upgrade is already complete by then). Every frame written to a connection, replies and events alike, counts toward its atomic backlog by the size of its encoded envelope; a FIFO of each frame's end position in the socket's written-byte count releases it from bytesWritten, and a release that crosses below kEventBacklogLow calls connectionDrained. QtWebSocketServerConfig::sendTimeout (default 30 s, as net) aborts a peer whose oldest unwritten frame has outlived it. QtWebSocketBackend routes event envelopes before reply matching and posts them to the setEventHandler executor. Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/backend.md | 55 +++++ docs/spec/security.md | 18 +- include/morph/qt/qt_websocket_backend.hpp | 22 ++ include/morph/qt/qt_websocket_server.hpp | 114 +++++++++ src/qt/qt_websocket_backend.cpp | 26 ++ src/qt/qt_websocket_server.cpp | 228 ++++++++++++++++-- tests/qt/test_qt_websocket.cpp | 276 ++++++++++++++++++++++ 7 files changed, 719 insertions(+), 20 deletions(-) diff --git a/docs/spec/core/backend.md b/docs/spec/core/backend.md index 9973e162e..2c7f64c76 100644 --- a/docs/spec/core/backend.md +++ b/docs/spec/core/backend.md @@ -1437,6 +1437,58 @@ client socket was destroyed before the reply is ready, the `QPointer` is null an the reply is silently dropped. A malformed frame produces an `err` reply from the `RemoteServer` and does not disconnect the client or affect other clients. +**Server-initiated events.** `QtWebSocketServer` carries events the way the +`net` transport does (see +[Server-initiated events over the socket transport](#server-initiated-events-over-the-socket-transport) +and [Writing to a connection](#writing-to-a-connection)): + +- **One writer per connection,** handed to `RemoteServer::setConnectionWriter` + in `onNewConnection`, right after `openConnection()`. + `QWebSocketServer::newConnection` fires only once the upgrade is complete, + so no frame the writer queues can precede the handshake response. + `closeConnection(cid)`, on disconnect or `close()`, drops it. `write` counts + the envelope and queues it to the Qt thread through + `QMetaObject::invokeMethod`, as a reply is queued; it throws only when there + is no `QCoreApplication`. Neither callable keeps the socket or the server + alive: both are held by `QPointer`, and a frame that arrives after either is + gone is dropped and its count released. +- **The backlog counts encoded envelopes.** Each connection has an atomic + count of the bytes of encoded envelopes handed to it — replies, rejection + replies and events alike — and not yet written. The size is added on the + handing thread before the frame is queued, so a `backlog()` read on the + strand already covers a frame still on its way to the socket. Once + `sendTextMessage` has taken the frame, the connection records where in the + socket's running count of written bytes the frame ends (its framed size: + `QWebSocket` splits a message into `outgoingFrameSize()` frames, each with + an unmasked 2-, 4- or 10-byte header) in a FIFO, next to the envelope size + it counted. `QWebSocket::bytesWritten` advances the running count and + releases, by its envelope size, every frame it completes. The running count + starts below zero by what the socket still held at accept — the handshake + response — so those bytes are not taken for a frame's. When the socket + reports nothing left to write, every frame still in the FIFO is released, so + a misjudged framed size cannot strand a count. Over TLS, `bytesWritten` + reports encrypted bytes, which run ahead of the framed ones, so a frame can + be released a little before its last byte is out; never after. A connection + that goes away releases whatever it still held. +- **The drain call.** A release that takes the backlog from at least + `kEventBacklogLow` to below it calls `RemoteServer::connectionDrained(cid)` + on the Qt thread, judged from the value its own subtraction replaced; a + socket no longer connected is not reported. +- **A peer that stops reading** is aborted by `sendTimeout`: a per-connection + one-shot timer armed for the oldest unwritten frame aborts the socket — an + abort, since a close frame would queue behind what the peer is not reading — + once that frame is older than `sendTimeout`, logging at `warn`. The + disconnect that follows reclaims the connection's scope as any other does. + +`tests/qt/test_qt_websocket.cpp` pins it (`[push][channel]`): an event posted +to a connection reaches the client's handler; an event carrying a pending +call's own `callId` reaches the handler and leaves the call to its real reply; +the connect handler fires on the first connect and on a reconnect, ahead of +the reconnect handler; a 2 MiB event holds the next one back, and the drain +after it is written sends one `resync`, after which events flow again; and a +peer that completes the upgrade and never reads is aborted no sooner than a +200 ms `sendTimeout`. + **Multi-client / concurrency.** One server serves many clients; each client registers its own model instances on the shared `RemoteServer`, so per-client model state is isolated. Because `handle()` posts to the pool, replies for @@ -1483,6 +1535,7 @@ reason as `QtWebSocketBackendConfig`) bounds per-connection resource usage: | `messagesPerSecond` | `0` (unbounded) | A per-connection token bucket (capacity = `messagesPerSecond`, refilled continuously). A frame that finds an empty bucket is refused — it never reaches `RemoteServer` — and answered with an `err "rate limited"` addressed to that frame's own `callId`. Not queued, and the connection is not closed. | | `handshakeTimeout` | `0` (disabled) | A one-shot timer per connection; if no frame arrives before it fires, the socket is closed. Cancelled on the first frame. Because `QWebSocketServer::newConnection()` only fires after the WS (and TLS, in `SecureMode`) opening handshake completes, this in practice bounds time-to-first-frame after that point, not the handshake itself. | | `idleTimeout` | `0` (disabled) | A shared ~1-second housekeeping sweep closes any connection whose last frame is older than `idleTimeout`; the actual close can lag the configured value by up to the sweep interval. | +| `sendTimeout` | `30000` ms, as `morph::net` | A per-connection one-shot timer aborts a connection whose oldest frame handed to the socket is still unwritten after `sendTimeout` (see "Server-initiated events" above). `0` disables it. | | `bindAddress` | `QHostAddress::LocalHost` | The address `listen()` binds to (see "Flow" above). | | `allowPlaintextExposure` | `false` | Deliberate opt-out of the exposure guard: set `true` only to knowingly serve plaintext on a non-loopback `bindAddress` (see "Flow" above). | @@ -2145,6 +2198,7 @@ server: each call is a loop task, so the loop serialises them. | `setReconnectHandler(handler)` | Stores the handler; invoked on the Qt thread after every *subsequent* connect. `nullptr` clears. | | `setConnectHandler(handler)` | Stores the handler; invoked on the Qt thread after every successful connect, first included. `nullptr` clears. | | `setDisconnectHandler(handler)` | Stores the handler; invoked on the Qt thread whenever the socket drops, before reconnect scheduling. `nullptr` clears. | +| `setEventHandler(handler, exec)` | Stores the handler under a mutex (callable from any thread). `onTextMessage` hands every decoded envelope to `routeEvent` before any reply matching, so an `event` never settles a call, whatever its `callId`; each is posted to `exec` in arrival order. `nullptr` clears; with none installed, events are dropped. | ### `QtWebSocketServerConfig` (namespace `morph::qt`) @@ -2155,6 +2209,7 @@ server: each call is a loop task, so the loop serialises them. | `messagesPerSecond` | `std::size_t` | `0` (unbounded) | | `handshakeTimeout` | `std::chrono::milliseconds` | `0` (disabled) | | `idleTimeout` | `std::chrono::milliseconds` | `0` (disabled) | +| `sendTimeout` | `std::chrono::milliseconds` | `30000` | | `bindAddress` | `QHostAddress` | `QHostAddress::LocalHost` | | `allowPlaintextExposure` | `bool` | `false` | diff --git a/docs/spec/security.md b/docs/spec/security.md index fb23bb624..a6eab8d6e 100644 --- a/docs/spec/security.md +++ b/docs/spec/security.md @@ -610,8 +610,13 @@ because it needs no Qt gets none of it. bounds connection count (`maxConnections`), per-frame size (`maxMessageBytes`, defaulting to the wire-layer `kMaxEnvelopeBytes` cap), per-connection message rate (`messagesPerSecond`, token bucket), and handshake/idle time - (`handshakeTimeout`/`idleTimeout`). All default to unbounded/disabled, so an - unconfigured server behaves exactly as before. See + (`handshakeTimeout`/`idleTimeout`). All default to unbounded/disabled. +- **A peer that stops reading is closed.** `QtWebSocketServerConfig::sendTimeout` + (default 30 s, as `morph::net`'s) aborts a connection whose oldest frame + handed to the socket is still unwritten after it, so a stalled reader cannot + hold replies and events on the server indefinitely. Each connection's + backlog — encoded envelopes handed to the socket and not yet written — is + what `RemoteServer::post` reads to hold events back from a slow reader. See [backend.md](core/backend.md#qtwebsocketserver--server-side-websocket-transport). ### The Qt-free reference transport (`morph::net`) @@ -727,9 +732,9 @@ responsibility: `maxInFlightExecutes`) and the Qt transport's `QtWebSocketServerConfig` (`maxConnections`, `maxMessageBytes`, `messagesPerSecond`, `handshakeTimeout`/`idleTimeout`) cover every gap this bullet used to call out - **on the Qt transport**. Both default to unbounded/off, so **installing - neither changes anything** — a deployer exposing `RemoteServer` publicly - should configure both. On `morph::net` only the `LimitPolicy` half exists: + **on the Qt transport**. Both default to unbounded/off — save the Qt + transport's `sendTimeout` (30 s), which closes a peer that stops reading — so + a deployer exposing `RemoteServer` publicly should configure both. On `morph::net` only the `LimitPolicy` half exists: `SocketServerConfig` has no connection, size, rate or timeout limits at all (above). See [backend.md](core/backend.md#limitpolicy--opt-in-resource-limits). @@ -821,7 +826,8 @@ contrast the "Transport security" section above describes. strand result and `TimeoutError` surfacing through `SimulatedRemoteBackend`); `tests/qt/test_qt_websocket.cpp` covers `QtWebSocketServerConfig` (`maxConnections`, `maxMessageBytes`, `messagesPerSecond`, -`handshakeTimeout`/`idleTimeout`) and their composition with `LimitPolicy`. +`handshakeTimeout`/`idleTimeout`, `sendTimeout`) and their composition with +`LimitPolicy`. `examples/vetted_hmac/test_libsodium_adapter.cpp` and `test_openssl_adapter.cpp` (built only with `MORPH_BUILD_HMAC_EXAMPLES=ON` and diff --git a/include/morph/qt/qt_websocket_backend.hpp b/include/morph/qt/qt_websocket_backend.hpp index 3c9e7fe21..bcfd7864d 100644 --- a/include/morph/qt/qt_websocket_backend.hpp +++ b/include/morph/qt/qt_websocket_backend.hpp @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -366,6 +367,22 @@ class QtWebSocketBackend : public ::morph::backend::detail::IBackend, public ::m /// drops. Pass `nullptr` to clear. void setDisconnectHandler(const std::function& handler) override; + /// @brief Installs the handler each server-initiated event is posted to. + /// See `IBackend::setEventHandler`. + /// + /// An `event` envelope is recognised by its kind before any reply + /// matching, so whatever its `callId`, it never settles a call. Events are + /// posted to @p exec in the order they arrived. Callable from any thread: + /// the handler is held under a mutex, and an event already being delivered + /// keeps the handler it was read with. + /// @param handler Callable posted to @p exec once per event. Pass + /// `nullptr` to clear. + /// @param exec Executor the handler is posted to. Borrowed: it must + /// outlive the installation. May be null only when + /// @p handler is. + void setEventHandler(std::function handler, + ::morph::exec::IExecutor* exec) override; + /// @brief Installs the session stamped onto every control envelope this /// backend subsequently builds (`register`, `registerShared`, /// `attach`, `assign`, `deregister`). See `IBackend::setSession`. @@ -476,6 +493,11 @@ class QtWebSocketBackend : public ::morph::backend::detail::IBackend, public ::m ::morph::exec::IExecutor* _reconnectExec{nullptr}; std::function _connectHandler; std::function _disconnectHandler; + /// Guards `_eventHandler` and `_eventExec`, which `setEventHandler` may + /// write from any thread while the socket's thread reads them. + std::mutex _eventMutex; + std::shared_ptr> _eventHandler; + ::morph::exec::IExecutor* _eventExec{nullptr}; ::morph::session::Context _session; /// Set by `negotiateProtocolVersion`: re-send `"hello"` on each connect. bool _negotiate{false}; diff --git a/include/morph/qt/qt_websocket_server.hpp b/include/morph/qt/qt_websocket_server.hpp index 44c5bac15..b99ab81c2 100644 --- a/include/morph/qt/qt_websocket_server.hpp +++ b/include/morph/qt/qt_websocket_server.hpp @@ -3,16 +3,22 @@ #pragma once #include #include +#include #ifndef QT_NO_SSL #include #endif #include #include #include +#include #include +#include +#include +#include #include #include #include +#include #include namespace morph::qt { @@ -66,6 +72,16 @@ struct QtWebSocketServerConfig { /// actual close can lag the configured value by up to that sweep interval. std::chrono::milliseconds idleTimeout{0}; + /// @brief Time a frame handed to a connection may stay unwritten before + /// the connection is closed. `0` = disabled. + /// + /// A peer that stops reading leaves the oldest frame queued on its socket; + /// once that frame is older than this, the socket is aborted, so the + /// connection is retired — its models reclaimed, its events no longer + /// written — instead of holding frames it will never read. The same bound, + /// with the same default, as `morph::net::SocketServerConfig::sendTimeout`. + std::chrono::milliseconds sendTimeout{30000}; + /// @brief Address `listen()` binds to. Default `QHostAddress::LocalHost`. QHostAddress bindAddress = QHostAddress::LocalHost; @@ -100,6 +116,16 @@ struct QtWebSocketServerConfig { /// the constructor, and `cfg.allowPlaintextExposure` is `false`. See /// `QtWebSocketServerConfig` and security.md's "Transport security" section. /// +/// @par Server-initiated events +/// Each connection is handed to `RemoteServer::setConnectionWriter` as it is +/// accepted, so `RemoteServer::post` reaches it. Every frame the server writes +/// to a connection, replies and events alike, counts toward that connection's +/// backlog by the size of its encoded envelope until the socket has written +/// it; when the backlog falls below `backend::kEventBacklogLow`, the server is +/// told through `RemoteServer::connectionDrained`. A connection whose oldest +/// unwritten frame is older than `QtWebSocketServerConfig::sendTimeout` is +/// aborted. +/// /// @par Usage /// Call `listen()` to start accepting connections, and `port()` to discover the /// bound port (useful when @p port was 0 — let the OS assign a free port). @@ -194,7 +220,25 @@ class QtWebSocketServer : public QObject { /// @brief Qt slot called when a client disconnects. Q_SLOT void onDisconnected(); + /// @brief Qt slot called when a client's socket has written @p bytes. + /// + /// Releases from the connection's backlog every frame those bytes + /// complete, and re-arms its send timeout for the oldest frame left. + /// @param bytes Bytes the socket just wrote. + Q_SLOT void onBytesWritten(qint64 bytes); + private: + /// @brief One frame handed to a socket and not yet written. + struct UnwrittenFrame { + /// @brief Position, in the socket's running count of written bytes, + /// at which this frame's last byte is written. + qint64 end = 0; + /// @brief What the frame added to the backlog: its envelope's size. + std::size_t counted = 0; + /// @brief When the frame was handed to the socket; drives `sendTimeout`. + std::chrono::steady_clock::time_point queuedAt; + }; + /// @brief Per-connection state tracked between accept and disconnect. struct ClientState { /// @brief The connection this state belongs to (for symmetry with the map key; unused for lookup). @@ -217,8 +261,78 @@ class QtWebSocketServer : public QObject { /// client at accept time, so its disconnect can reclaim every model /// it registered (see `RemoteServer::closeConnection`). ::morph::backend::ConnectionId cid = 0; + + /// @brief Bytes of encoded envelopes handed to this connection and not + /// yet written: the backlog `RemoteServer` reads. + /// + /// Added to on whichever thread hands over a frame, before the frame is + /// queued to the Qt thread, so the count covers frames still on their + /// way to the socket. Held by `shared_ptr` so the writer, kept on the + /// server strand, can read it without reaching this state. + std::shared_ptr> backlog = std::make_shared>(0); + + /// @brief Frames on the socket and not yet written, oldest first. + std::deque unwritten; + + /// @brief Running count of bytes the socket has written, as reported by + /// `QWebSocket::bytesWritten`, less what it held at accept. + qint64 written = 0; + + /// @brief One-shot timer enforcing `sendTimeout` on the oldest + /// unwritten frame; `nullptr` if `sendTimeout == 0`. + QTimer* sendTimer = nullptr; }; + /// @brief Counts @p envelope against the connection's backlog, then + /// queues it to the Qt thread to be written to @p socket. + /// + /// Callable from any thread: replies arrive on the server's pool, events + /// on its strand. Dropped, its count released, if the server or the + /// socket is gone by the time it reaches the Qt thread. + /// @param server This server, held weakly. + /// @param socket The connection's socket, held weakly. + /// @param backlog The connection's backlog counter. + /// @param envelope The encoded envelope. + /// @return `false` if there is no Qt application to queue to; the + /// envelope is dropped and its count released. + static bool queueEnvelope(QPointer const& server, QPointer const& socket, + std::shared_ptr> const& backlog, std::string envelope); + + /// @brief Writes @p envelope, already counted, to @p socket and tracks it + /// until the socket has written it. On the Qt thread. + /// @param socket A tracked socket. + /// @param state @p socket's state. + /// @param envelope The encoded envelope. + void writeCounted(QWebSocket* socket, ClientState& state, std::string const& envelope); + + /// @brief Counts @p envelope against @p socket's backlog and writes it. + /// On the Qt thread. + /// @param socket A tracked socket. + /// @param state @p socket's state. + /// @param envelope The encoded envelope. + void writeNow(QWebSocket* socket, ClientState& state, std::string const& envelope); + + /// @brief Takes @p bytes off @p state's backlog, and tells the server if + /// that takes it below `backend::kEventBacklogLow`. + /// @param state The connection. + /// @param bytes What a frame added when it was handed over. + void releaseBacklog(ClientState const& state, std::size_t bytes); + + /// @brief Arms @p state's send timer for its oldest unwritten frame, or + /// stops it when none is left. + /// @param state The connection. + void armSendTimer(ClientState& state) const; + + /// @brief Aborts @p socket if its oldest unwritten frame has outlived + /// `sendTimeout`; otherwise re-arms the timer. + /// @param socket The connection's socket. + void onSendTimer(QWebSocket* socket); + + /// @brief Releases every frame @p state still holds and drops its timers. + /// Called when the connection goes away. + /// @param state The connection. + static void forgetConnection(ClientState& state); + /// @brief Refills @p state's token bucket for elapsed time, then consumes one /// token if available. /// @param state Per-connection state to update. diff --git a/src/qt/qt_websocket_backend.cpp b/src/qt/qt_websocket_backend.cpp index ad2994f1b..1c2fb8401 100644 --- a/src/qt/qt_websocket_backend.cpp +++ b/src/qt/qt_websocket_backend.cpp @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -488,6 +489,14 @@ void QtWebSocketBackend::setConnectHandler(const std::function& handler) void QtWebSocketBackend::setDisconnectHandler(const std::function& handler) { _disconnectHandler = handler; } +void QtWebSocketBackend::setEventHandler(std::function handler, + ::morph::exec::IExecutor* exec) { + std::scoped_lock const lock{_eventMutex}; + _eventHandler = + handler ? std::make_shared>(std::move(handler)) : nullptr; + _eventExec = exec; +} + void QtWebSocketBackend::setSession(::morph::session::Context session) { _session = std::move(session); } void QtWebSocketBackend::scheduleReconnect() { @@ -599,6 +608,23 @@ void QtWebSocketBackend::onTextMessage(const QString& message) { std::runtime_error("protocol error: server sent a message that is not a valid envelope"))); return; } + // Before any reply matching: an event names no request, whatever its callId. + if (::morph::backend::detail::routeEvent(env, [this](::morph::wire::EventBody event) { + std::shared_ptr> handler; + ::morph::exec::IExecutor* exec = nullptr; + { + std::scoped_lock const lock{_eventMutex}; + handler = _eventHandler; + exec = _eventExec; + } + if (handler && exec != nullptr) { + exec->post([handler = std::move(handler), event = std::move(event)]() mutable { + (*handler)(std::move(event)); + }); + } + })) { + return; + } routeReply(env); } diff --git a/src/qt/qt_websocket_server.cpp b/src/qt/qt_websocket_server.cpp index 65daeee3a..f46ab0c80 100644 --- a/src/qt/qt_websocket_server.cpp +++ b/src/qt/qt_websocket_server.cpp @@ -9,15 +9,48 @@ #include #include #include +#include #include #include #include +#include +#include #include #include #include namespace morph::qt { +namespace { + +/// @brief Bytes a server-side `QWebSocket` puts on the socket for a text +/// message whose UTF-8 payload is @p payload bytes. +/// +/// `QWebSocket` splits a message into frames of at most @p maxFrame payload +/// bytes, each with its own header; a server's frames are unmasked, so a +/// header is 2 bytes plus a 2- or 8-byte extended length. +/// @param payload The message's UTF-8 length. +/// @param maxFrame The socket's `outgoingFrameSize()`. +/// @return The framed size. +qint64 framedSize(qint64 payload, quint64 maxFrame) { + auto const header = [](qint64 size) -> qint64 { + if (size < 126) { + return 2; + } + return size <= 0xFFFF ? 4 : 10; + }; + auto const cap = static_cast(std::max(maxFrame, 1)); + qint64 const whole = payload / cap; + qint64 const rest = payload % cap; + qint64 total = payload + (whole * header(cap)); + if (rest != 0 || whole == 0) { + total += header(rest); + } + return total; +} + +} // namespace + QtWebSocketServer::QtWebSocketServer(::morph::backend::RemoteServer& server, quint16 port, #ifndef QT_NO_SSL std::optional tls, @@ -80,6 +113,7 @@ void QtWebSocketServer::close() { state.handshakeTimer->stop(); state.handshakeTimer->deleteLater(); } + forgetConnection(state); socket->abort(); socket->deleteLater(); } @@ -182,6 +216,7 @@ void QtWebSocketServer::onNewConnection() { } connect(socket, &QWebSocket::textMessageReceived, this, &QtWebSocketServer::onTextMessage); connect(socket, &QWebSocket::disconnected, this, &QtWebSocketServer::onDisconnected); + connect(socket, &QWebSocket::bytesWritten, this, &QtWebSocketServer::onBytesWritten); ClientState state; state.socket = socket; @@ -212,7 +247,176 @@ void QtWebSocketServer::onNewConnection() { state.handshakeTimer = timer; } - _clients.emplace(socket, state); + // The handshake response may still be on the socket; its bytes are + // written ahead of any frame, so they start the count below zero. + state.written = -socket->bytesToWrite(); + if (_cfg.sendTimeout.count() > 0) { + // NOLINTNEXTLINE(cppcoreguidelines-owning-memory) — QObject parent owns this + auto* timer = new QTimer(this); + timer->setSingleShot(true); + connect(timer, &QTimer::timeout, this, [this, socket] { onSendTimer(socket); }); + state.sendTimer = timer; + } + + auto const cid = state.cid; + auto const backlog = state.backlog; + _clients.emplace(socket, std::move(state)); + + // QWebSocketServer hands over a socket whose upgrade is already complete, + // so a frame written from here on cannot precede the handshake response. + QPointer const self{this}; + QPointer const weakSocket{socket}; + _server.setConnectionWriter( + cid, ::morph::backend::ConnectionWriter{ + .write = + [self, weakSocket, backlog](std::string envelope) { + if (!queueEnvelope(self, weakSocket, backlog, std::move(envelope))) { + throw std::runtime_error("no Qt application to write the connection from"); + } + }, + .backlog = [backlog] { return backlog->load(); }}); +} + +bool QtWebSocketServer::queueEnvelope(QPointer const& server, QPointer const& socket, + std::shared_ptr> const& backlog, + std::string envelope) { + auto* app = QCoreApplication::instance(); + if (app == nullptr) { + return false; + } + std::size_t const counted = envelope.size(); + // Before the queueing, so a backlog read on the server strand already + // covers a frame still on its way to the Qt thread. + backlog->fetch_add(counted); + bool const queued = QMetaObject::invokeMethod( + app, + [server, socket, backlog, counted, envelope = std::move(envelope)]() { + if (server && socket) { + auto iter = server->_clients.find(socket.data()); + if (iter != server->_clients.end()) { + server->writeCounted(socket.data(), iter->second, envelope); + return; + } + } + backlog->fetch_sub(counted); + }, + Qt::QueuedConnection); + if (!queued) { + backlog->fetch_sub(counted); + } + return queued; +} + +void QtWebSocketServer::writeNow(QWebSocket* socket, ClientState& state, std::string const& envelope) { + state.backlog->fetch_add(envelope.size()); + writeCounted(socket, state, envelope); +} + +void QtWebSocketServer::writeCounted(QWebSocket* socket, ClientState& state, std::string const& envelope) { + qint64 const payload = socket->sendTextMessage(QString::fromStdString(envelope)); + if (payload <= 0) { + // Not connected any more: nothing reached the socket. + releaseBacklog(state, envelope.size()); + return; + } + // The frame ends where the socket's written count will stand once its + // last byte is out: after the frame ahead of it, or after whatever the + // socket has written so far. `bytesWritten` reports framed bytes, so the + // position is framed too; only the envelope's size is counted. + qint64 const base = state.unwritten.empty() ? state.written : state.unwritten.back().end; + state.unwritten.push_back(UnwrittenFrame{.end = base + framedSize(payload, socket->outgoingFrameSize()), + .counted = envelope.size(), + .queuedAt = std::chrono::steady_clock::now()}); + if (state.unwritten.size() == 1) { + armSendTimer(state); + } +} + +void QtWebSocketServer::releaseBacklog(ClientState const& state, std::size_t bytes) { + if (bytes == 0) { + return; + } + // Judged from the value this subtraction replaced, so of several releases + // that pass the bound together exactly one reports it. A second report, + // after other threads lift the count again, is harmless: the server sends + // nothing when no resync is owed. + std::size_t const before = state.backlog->fetch_sub(bytes); + bool const crossed = + before >= ::morph::backend::kEventBacklogLow && before - bytes < ::morph::backend::kEventBacklogLow; + if (crossed && state.socket != nullptr && state.socket->state() == QAbstractSocket::ConnectedState) { + _server.connectionDrained(state.cid); + } +} + +void QtWebSocketServer::onBytesWritten(qint64 bytes) { + auto* socket = qobject_cast(sender()); + if (socket == nullptr) { + return; + } + auto iter = _clients.find(socket); + if (iter == _clients.end()) { + return; + } + ClientState& state = iter->second; + state.written += bytes; + // An empty socket buffer means every frame handed to it is out, whatever + // the positions say: a frame whose framed size was misjudged is still + // released, rather than holding its bytes and tripping `sendTimeout`. + bool const allWritten = socket->bytesToWrite() == 0; + bool released = false; + while (!state.unwritten.empty() && (allWritten || state.unwritten.front().end <= state.written)) { + std::size_t const counted = state.unwritten.front().counted; + state.unwritten.pop_front(); + releaseBacklog(state, counted); + released = true; + } + if (released) { + armSendTimer(state); + } +} + +void QtWebSocketServer::armSendTimer(ClientState& state) const { + if (state.sendTimer == nullptr) { + return; + } + if (state.unwritten.empty()) { + state.sendTimer->stop(); + return; + } + auto const due = state.unwritten.front().queuedAt + _cfg.sendTimeout; + auto const remaining = std::chrono::ceil(due - std::chrono::steady_clock::now()); + state.sendTimer->start(std::max(std::chrono::milliseconds{0}, remaining)); +} + +void QtWebSocketServer::onSendTimer(QWebSocket* socket) { + auto iter = _clients.find(socket); + if (iter == _clients.end() || iter->second.unwritten.empty()) { + return; + } + ClientState& state = iter->second; + if (std::chrono::steady_clock::now() - state.unwritten.front().queuedAt < _cfg.sendTimeout) { + armSendTimer(state); + return; + } + ::morph::log::logWarn("[QtWebSocketServer] connection {} closed: send timeout, {} bytes unwritten after {} ms", + state.cid, state.backlog->load(), _cfg.sendTimeout.count()); + // An abort, not a close: a close frame would queue behind what the peer + // is not reading. It may disconnect synchronously, erasing `state`. + socket->abort(); +} + +void QtWebSocketServer::forgetConnection(ClientState& state) { + if (state.sendTimer != nullptr) { + state.sendTimer->stop(); + state.sendTimer->deleteLater(); + state.sendTimer = nullptr; + } + std::size_t dropped = 0; + for (auto const& frame : state.unwritten) { + dropped += frame.counted; + } + state.unwritten.clear(); + state.backlog->fetch_sub(dropped); } bool QtWebSocketServer::consumeToken(ClientState& state) const { @@ -264,8 +468,8 @@ void QtWebSocketServer::onTextMessage(const QString& message) { }; if (std::cmp_greater(utf8.size(), _cfg.maxMessageBytes)) { - socket->sendTextMessage(QString::fromStdString( - ::morph::wire::encode(::morph::wire::makeErr("message exceeds maxMessageBytes", peekedCallId())))); + writeNow(socket, state, + ::morph::wire::encode(::morph::wire::makeErr("message exceeds maxMessageBytes", peekedCallId()))); return; } @@ -279,23 +483,18 @@ void QtWebSocketServer::onTextMessage(const QString& message) { // the same reply-without-full-decode pattern the maxMessageBytes // branch above uses. The connection stays open: rate limiting throttles // a client, it does not evict one. - socket->sendTextMessage( - QString::fromStdString(::morph::wire::encode(::morph::wire::makeErr("rate limited", peekedCallId())))); + writeNow(socket, state, ::morph::wire::encode(::morph::wire::makeErr("rate limited", peekedCallId()))); return; } + // A reply counts toward the connection's backlog like an event, and is + // never held back. + QPointer const self{this}; QPointer const weakSocket{socket}; _server.handle( message.toStdString(), - [weakSocket](const std::string& reply) { - QMetaObject::invokeMethod( - QCoreApplication::instance(), - [weakSocket, reply]() { - if (weakSocket) { - weakSocket->sendTextMessage(QString::fromStdString(reply)); - } - }, - Qt::QueuedConnection); + [self, weakSocket, backlog = state.backlog](const std::string& reply) { + static_cast(queueEnvelope(self, weakSocket, backlog, reply)); }, state.cid); } @@ -319,6 +518,7 @@ void QtWebSocketServer::onDisconnected() { iter->second.handshakeTimer->stop(); iter->second.handshakeTimer->deleteLater(); } + forgetConnection(iter->second); _clients.erase(iter); socket->deleteLater(); } diff --git a/tests/qt/test_qt_websocket.cpp b/tests/qt/test_qt_websocket.cpp index f3f956b66..396f3c3ad 100644 --- a/tests/qt/test_qt_websocket.cpp +++ b/tests/qt/test_qt_websocket.cpp @@ -10,8 +10,10 @@ #include #include #include +#include #include #include +#include #include #include #include @@ -2971,3 +2973,277 @@ TEST_CASE("morph::qt::QtWebSocketBackend: cancelPending rejects a request still backend.cancelPending(std::make_exception_ptr(std::runtime_error{"cancelled before connect"})); CHECK_THROWS_WITH(awaitOn(cbExec, std::move(held)), "cancelled before connect"); } + +// ── Server-initiated events over the Qt transport ─────────────────────────── +namespace { + +/// Runs each task inline on the posting thread, and counts them. +struct PushCountingExec final : morph::exec::IExecutor { + std::atomic posted{0}; + void post(std::function task) override { + posted.fetch_add(1); + task(); + } +}; + +/// What an event handler saw. +struct PushEventLog { + std::mutex mtx; + std::vector events; + + std::function handler() { + return [this](morph::wire::EventBody body) { + std::scoped_lock const lock{mtx}; + events.push_back(std::move(body)); + }; + } + std::size_t size() { + std::scoped_lock const lock{mtx}; + return events.size(); + } + std::vector snapshot() { + std::scoped_lock const lock{mtx}; + return events; + } +}; + +/// The connection id the next `QtWebSocketServer` connection to @p server gets: +/// `RemoteServer` numbers connections in order. +morph::backend::ConnectionId nextQtConnectionId(morph::backend::RemoteServer& server) { + auto const probe = server.openConnection(); + server.closeConnection(probe); + return probe + 1; +} + +/// A JSON string payload of @p bytes characters. +std::string bigPayload(std::size_t bytes) { return "\"" + std::string(bytes, 'x') + "\""; } + +} // namespace + +TEST_CASE("qt_posted_event_reaches_handler", "[push][channel]") { + ensureApp(); + morph::exec::ThreadPoolExecutor serverPool{2}; + auto server = std::make_shared(serverPool); + morph::qt::QtWebSocketServer wsServer{*server, 0}; + REQUIRE(wsServer.listen()); + auto const cid = nextQtConnectionId(*server); + + PushCountingExec exec; + PushEventLog log; + morph::qt::QtWebSocketBackend backend{QUrl{QString("ws://127.0.0.1:%1").arg(wsServer.port())}}; + backend.setEventHandler(log.handler(), &exec); + REQUIRE(backend.waitForConnected()); + + server->post(cid, morph::wire::encode(morph::wire::makeEvent("changes", R"({"n":1})"))); + server->post(cid, morph::wire::encode(morph::wire::makeEvent("resync", "{}"))); + pumpUntil([&] { return log.size() == 2; }, 300); + REQUIRE(log.size() == 2); + + auto const got = log.snapshot(); + CHECK(got[0].topic == "changes"); + CHECK(got[0].payload.str == R"({"n":1})"); + CHECK(got[1].topic == "resync"); + CHECK(exec.posted.load() == 2); + + // Cleared: a later event is dropped, and the connection keeps working. + backend.setEventHandler(nullptr, nullptr); + server->post(cid, morph::wire::encode(morph::wire::makeEvent("changes", "{}"))); + morph::exec::MainThreadExecutor cbExec; + CHECK(registerPrivate(backend, cbExec, "WsEchoModel").v != 0U); + CHECK(log.size() == 2); +} + +TEST_CASE("qt_event_is_routed_before_reply_matching", "[push][channel]") { + ensureApp(); + // A fake server, so an event can carry the very callId a pending execute + // was filed under: only routing by kind first keeps it off that call. + QWebSocketServer fake{QStringLiteral("fake"), QWebSocketServer::NonSecureMode}; + REQUIRE(fake.listen(QHostAddress::LocalHost, 0)); + morph::qt::QtWebSocketBackend backend{QUrl{QString("ws://127.0.0.1:%1").arg(fake.serverPort())}}; + REQUIRE(backend.waitForConnected()); + pumpUntil([&] { return fake.hasPendingConnections(); }, 100); + std::unique_ptr peer{fake.nextPendingConnection()}; + REQUIRE(peer != nullptr); + std::vector received; + QObject::connect(peer.get(), &QWebSocket::textMessageReceived, + [&](const QString& msg) { received.push_back(msg.toStdString()); }); + + PushCountingExec exec; + PushEventLog log; + backend.setEventHandler(log.handler(), &exec); + + morph::backend::detail::ActionCall call; + call.modelTypeId = "WsEchoModel"; + call.actionTypeId = "WsEchoAction"; + call.serializeAction = [](const void*) { return std::string{"{}"}; }; + call.deserializeResult = [](std::string_view body) -> std::shared_ptr { + return std::make_shared(std::stoi(std::string{body})); + }; + morph::exec::MainThreadExecutor owner; + std::atomic result{-1}; + std::atomic failed{false}; + backend.execute(morph::exec::detail::ModelId{1}, std::move(call), &owner) + .then([&](const std::shared_ptr& value) { result.store(*std::static_pointer_cast(value)); }) + .onError([&](const std::exception_ptr&) { failed.store(true); }); + + pumpUntil([&] { return !received.empty(); }, 200); + REQUIRE(received.size() == 1); + auto const request = morph::wire::decode(received.front()); + REQUIRE(request.kind == "execute"); + REQUIRE(request.callId != 0); + + auto sameId = morph::wire::makeEvent("changes", R"({"which":"same"})"); + sameId.callId = request.callId; + peer->sendTextMessage(QString::fromStdString(morph::wire::encode(sameId))); + peer->sendTextMessage( + QString::fromStdString(morph::wire::encode(morph::wire::makeEvent("changes", R"({"which":"zero"})")))); + pumpUntil( + [&] { + owner.drain(); + return log.size() == 2; + }, + 200); + REQUIRE(log.size() == 2); + pumpUntil([&] { + owner.drain(); + return false; + }, 5); + CHECK(result.load() == -1); + CHECK_FALSE(failed.load()); + + // The call is still pending, and its own reply settles it. + peer->sendTextMessage(QString::fromStdString(morph::wire::encode(morph::wire::makeOk(request.callId, "7")))); + pumpUntil( + [&] { + owner.drain(); + return result.load() != -1 || failed.load(); + }, + 200); + CHECK(result.load() == 7); + CHECK_FALSE(failed.load()); +} + +TEST_CASE("qt_connect_handler_fires_on_reconnect", "[push][channel]") { + ensureApp(); + morph::exec::ThreadPoolExecutor serverPool{2}; + auto server = std::make_shared(serverPool); + + quint16 port = 0; + std::vector seen; + PushCountingExec reconnectExec; + std::unique_ptr backend; + { + morph::qt::QtWebSocketServer first{*server, 0}; + REQUIRE(first.listen()); + port = first.port(); + backend = std::make_unique( + QUrl{QString("ws://127.0.0.1:%1").arg(port)}, + morph::qt::QtWebSocketBackend::Config{.initialReconnectDelay = std::chrono::milliseconds{20}, + .maxReconnectDelay = std::chrono::milliseconds{100}}); + // Installed before the Qt loop runs, so before the first connect. + backend->setConnectHandler([&] { seen.emplace_back("connect"); }); + backend->setReconnectHandler([&] { seen.emplace_back("reconnect"); }, &reconnectExec); + REQUIRE(backend->waitForConnected()); + pumpUntil([&] { return seen.size() == 1; }, 100); + REQUIRE(seen == std::vector{"connect"}); + } + pumpUntil([] { return false; }, 10); + + morph::qt::QtWebSocketServer second{*server, port}; + REQUIRE(second.listen()); + pumpUntil([&] { return seen.size() == 3; }, 300); + // The connect handler runs before the reconnect handler is posted. + CHECK(seen == std::vector{"connect", "connect", "reconnect"}); +} + +TEST_CASE("qt_stalled_peer_closed_after_send_timeout", "[push][channel]") { + ensureApp(); + morph::exec::ThreadPoolExecutor serverPool{2}; + auto server = std::make_shared(serverPool); + morph::qt::QtWebSocketServerConfig cfg; + cfg.sendTimeout = std::chrono::milliseconds{200}; + morph::qt::QtWebSocketServer wsServer{*server, 0, std::nullopt, cfg}; + REQUIRE(wsServer.listen()); + auto const cid = nextQtConnectionId(*server); + + std::mutex logMtx; + std::vector> logged; + morph::log::ScopedLoggerOverride guard{[&](morph::log::LogLevel /*level*/, std::string_view msg) { + std::scoped_lock const lock{logMtx}; + logged.emplace_back(std::chrono::steady_clock::now(), + std::string{msg}); + }, + morph::log::LogLevel::debug}; + auto findLogged = [&](std::string_view needle) -> std::optional { + std::scoped_lock const lock{logMtx}; + for (auto const& [when, msg] : logged) { + if (msg.find(needle) != std::string::npos) { + return when; + } + } + return std::nullopt; + }; + + // A peer that completes the upgrade, then never reads again: a small + // kernel receive buffer and a one-byte Qt read buffer leave the server's + // frame stuck in its own write buffer. + QTcpSocket peer; + peer.setSocketOption(QAbstractSocket::ReceiveBufferSizeSocketOption, 4096); + peer.connectToHost(QHostAddress::LocalHost, wsServer.port()); + pumpUntil([&] { return peer.state() == QAbstractSocket::ConnectedState; }, 100); + REQUIRE(peer.state() == QAbstractSocket::ConnectedState); + peer.write(QString("GET / HTTP/1.1\r\nHost: 127.0.0.1:%1\r\nUpgrade: websocket\r\nConnection: Upgrade\r\n" + "Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\nSec-WebSocket-Version: 13\r\n\r\n") + .arg(wsServer.port()) + .toUtf8()); + QByteArray response; + pumpUntil( + [&] { + response += peer.readAll(); + return response.contains("\r\n\r\n"); + }, + 200); + REQUIRE(response.startsWith("HTTP/1.1 101")); + peer.setReadBufferSize(1); + + auto const posted = std::chrono::steady_clock::now(); + server->post(cid, morph::wire::encode(morph::wire::makeEvent("changes", bigPayload(std::size_t{4} << 20)))); + pumpUntil([&] { return findLogged("send timeout").has_value() && findLogged("disconnected").has_value(); }, 300); + + auto const closedAt = findLogged("send timeout"); + REQUIRE(closedAt.has_value()); + CHECK(*closedAt - posted >= std::chrono::milliseconds{200}); + CHECK(findLogged("disconnected").has_value()); +} + +TEST_CASE("qt_backlog_drains_and_sends_resync", "[push][channel]") { + ensureApp(); + morph::exec::ThreadPoolExecutor serverPool{2}; + auto server = std::make_shared(serverPool); + morph::qt::QtWebSocketServer wsServer{*server, 0}; + REQUIRE(wsServer.listen()); + auto const cid = nextQtConnectionId(*server); + + PushCountingExec exec; + PushEventLog log; + morph::qt::QtWebSocketBackend backend{QUrl{QString("ws://127.0.0.1:%1").arg(wsServer.port())}}; + backend.setEventHandler(log.handler(), &exec); + REQUIRE(backend.waitForConnected()); + + // The first event alone lifts the backlog past kEventBacklogHigh, so the + // second is held back and owed as a resync, sent only once the first has + // been written and the backlog reported drained. + server->post(cid, morph::wire::encode(morph::wire::makeEvent("changes", bigPayload(std::size_t{2} << 20)))); + server->post(cid, morph::wire::encode(morph::wire::makeEvent("changes", R"({"held":true})"))); + pumpUntil([&] { return log.size() == 2; }, 500); + REQUIRE(log.size() == 2); + + // Events flow again after the resync. + server->post(cid, morph::wire::encode(morph::wire::makeEvent("changes", R"({"after":true})"))); + pumpUntil([&] { return log.size() == 3; }, 300); + auto const got = log.snapshot(); + REQUIRE(got.size() == 3); + CHECK(got[0].topic == "changes"); + CHECK(got[1].topic == "resync"); + CHECK(got[2].payload.str == R"({"after":true})"); +} From c8c96ead40b1a8665c60a665735f86be1c20a285 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 14:52:10 +0200 Subject: [PATCH 23/68] qt: the backlog over TLS counts encrypted bytes Over wss:// QWebSocket::bytesToWrite() counts only plaintext not yet encrypted, which Qt encrypts eagerly, so the server released a frame long before its ciphertext was sent: a stalled TLS reader got every event and never tripped sendTimeout. A frame's release position is now measured when it is handed over -- bytes written so far plus everything the socket stack holds, the QSslSocket's encryptedBytesToWrite() included -- in a small detail::OutboundLedger, and "nothing left to write" means nothing in either buffer. The QSslSocket is found among the QWebSocketServer's descendants by the connection's endpoints. Tests cover a stalled reader held back and resynced over ws and wss, the send timeout over wss, and the ledger's release by position; the cleared-handler check no longer passes vacuously. Co-Authored-By: Claude Opus 5.5 --- CMakeLists.txt | 1 + docs/spec/core/backend.md | 47 ++-- include/morph/qt/detail/outbound_ledger.hpp | 92 ++++++++ include/morph/qt/qt_websocket_server.hpp | 52 +++-- src/qt/qt_websocket_server.cpp | 118 +++++----- tests/qt/test_qt_websocket.cpp | 245 +++++++++++++++++++- 6 files changed, 449 insertions(+), 106 deletions(-) create mode 100644 include/morph/qt/detail/outbound_ledger.hpp diff --git a/CMakeLists.txt b/CMakeLists.txt index cc907ba11..46f24ddf0 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -1047,6 +1047,7 @@ if(MORPH_BUILD_QT) include/morph/qt/qt_tls.hpp include/morph/qt/qt_websocket_server.hpp include/morph/qt/qt_websocket_backend.hpp + include/morph/qt/detail/outbound_ledger.hpp ) # Compiled implementation (QObject subclass needs MOC via a .cpp file) diff --git a/docs/spec/core/backend.md b/docs/spec/core/backend.md index 2c7f64c76..2d3d7b835 100644 --- a/docs/spec/core/backend.md +++ b/docs/spec/core/backend.md @@ -1457,19 +1457,31 @@ and [Writing to a connection](#writing-to-a-connection)): replies and events alike — and not yet written. The size is added on the handing thread before the frame is queued, so a `backlog()` read on the strand already covers a frame still on its way to the socket. Once - `sendTextMessage` has taken the frame, the connection records where in the - socket's running count of written bytes the frame ends (its framed size: - `QWebSocket` splits a message into `outgoingFrameSize()` frames, each with - an unmasked 2-, 4- or 10-byte header) in a FIFO, next to the envelope size - it counted. `QWebSocket::bytesWritten` advances the running count and - releases, by its envelope size, every frame it completes. The running count - starts below zero by what the socket still held at accept — the handshake - response — so those bytes are not taken for a frame's. When the socket - reports nothing left to write, every frame still in the FIFO is released, so - a misjudged framed size cannot strand a count. Over TLS, `bytesWritten` - reports encrypted bytes, which run ahead of the framed ones, so a frame can - be released a little before its last byte is out; never after. A connection - that goes away releases whatever it still held. + `sendTextMessage` has taken the frame, `detail::OutboundLedger` records, + next to the envelope size it counted, where in the socket's running count of + written bytes the frame's last byte falls: the bytes written so far plus + every byte the socket stack still holds, the frame's own included. The + position is measured, not computed from a frame layout, so bytes the socket + queued for itself (the handshake response, a Pong) are covered, and each + position is taken afresh, so no error accumulates. `QWebSocket::bytesWritten` + advances the count and releases, by its envelope size, every frame it + completes; when the stack holds nothing, every frame is released. + - **Over TLS** `QWebSocket::bytesWritten` reports encrypted bytes sent, and + `QWebSocket::bytesToWrite()` only the plaintext not yet encrypted. Qt + encrypts eagerly into the TLS socket's own buffer, so that plaintext count + reaches zero while megabytes of ciphertext are still unsent; the bytes held + therefore also count that buffer, `QSslSocket::encryptedBytesToWrite()`. + `QWebSocket` does not expose its `QSslSocket`: a `SecureMode` + `QWebSocketServer` accepts through a `QSslServer` it owns, which parents + each accepted socket, so the server finds it among its descendants by the + connection's two endpoints. A connection whose TLS socket is not found is + logged at `warn`, and its backlog then counts only plaintext not yet + encrypted, so a slow reader on it is not held back. + - Plaintext waiting to be encrypted is counted one-for-one, not with its + record overhead, so over TLS a frame can be released before its last byte + is sent by at most the overhead on the plaintext still unencrypted when it + was handed over. On a plain socket the position is exact. + - A connection that goes away releases whatever it still held. - **The drain call.** A release that takes the backlog from at least `kEventBacklogLow` to below it calls `RemoteServer::connectionDrained(cid)` on the Qt thread, judged from the value its own subtraction replaced; a @@ -1485,9 +1497,12 @@ to a connection reaches the client's handler; an event carrying a pending call's own `callId` reaches the handler and leaves the call to its real reply; the connect handler fires on the first connect and on a reconnect, ahead of the reconnect handler; a 2 MiB event holds the next one back, and the drain -after it is written sends one `resync`, after which events flow again; and a -peer that completes the upgrade and never reads is aborted no sooner than a -200 ms `sendTimeout`. +after it is written sends one `resync`, after which events flow again; a peer +that completes the upgrade and then stops reading, on `ws://` and on +`wss://`, has the event after a 7 MiB one held back and receives a `resync` in +its place once it reads again; such a peer is aborted no sooner than a 200 ms +`sendTimeout`, on both; and the ledger releases a frame once the count reaches +its position while later frames are still held. **Multi-client / concurrency.** One server serves many clients; each client registers its own model instances on the shared `RemoteServer`, so per-client diff --git a/include/morph/qt/detail/outbound_ledger.hpp b/include/morph/qt/detail/outbound_ledger.hpp new file mode 100644 index 000000000..5d53379fd --- /dev/null +++ b/include/morph/qt/detail/outbound_ledger.hpp @@ -0,0 +1,92 @@ +// SPDX-License-Identifier: Apache-2.0 + +#pragma once +#include +#include +#include +#include +#include + +/// @file +/// @brief Which frames handed to a socket it has written, told from the +/// socket's own byte counts. + +namespace morph::qt::detail { + +/// @brief Frames handed to one socket and not yet written, oldest first, each +/// with the position in the socket's running count of written bytes at +/// which its last byte is out. +/// +/// A frame's position is taken when it is handed over: the bytes written so +/// far, plus every byte the socket still holds, the frame's own included. So +/// it is measured, not computed from a frame layout, and covers whatever the +/// socket queued for itself ahead of it. Over TLS the bytes held must include +/// the ciphertext not yet sent as well as the plaintext not yet encrypted; +/// plaintext is held unencrypted only briefly, so a frame can be released +/// ahead of its last byte by at most the record overhead on what was then +/// unencrypted, and the error does not accumulate: each position is taken +/// afresh. +/// +/// Not synchronised: owned by the socket's thread. +class OutboundLedger { +public: + /// @brief Records a frame just handed to the socket. + /// @param pending Bytes the socket holds after taking the frame, + /// the frame's own included. + /// @param counted What the frame added to the connection's backlog. + /// @param queuedAt When the frame was handed over. + void push(qint64 pending, std::size_t counted, std::chrono::steady_clock::time_point queuedAt) { + _frames.push_back(Frame{.end = _written + pending, .counted = counted, .queuedAt = queuedAt}); + } + + /// @brief Advances the written count and releases every frame it completes. + /// @param bytes Bytes the socket just reported written. + /// @param pending Bytes the socket still holds after writing them. `0` + /// releases every frame, whatever its position. + /// @return The sum of the released frames' `counted`. + [[nodiscard]] std::size_t written(qint64 bytes, qint64 pending) { + _written += bytes; + std::size_t released = 0; + while (!_frames.empty() && (pending == 0 || _frames.front().end <= _written)) { + released += _frames.front().counted; + _frames.pop_front(); + } + return released; + } + + /// @brief Forgets every frame, as when the connection goes away. + /// @return The sum of the forgotten frames' `counted`. + [[nodiscard]] std::size_t clear() { + std::size_t dropped = 0; + for (auto const& frame : _frames) { + dropped += frame.counted; + } + _frames.clear(); + return dropped; + } + + /// @brief When the oldest unwritten frame was handed over. + /// @return Its time, or `std::nullopt` when every frame is written. + [[nodiscard]] std::optional oldest() const { + if (_frames.empty()) { + return std::nullopt; + } + return _frames.front().queuedAt; + } + +private: + /// @brief One frame handed to the socket and not yet written. + struct Frame { + /// @brief Written count at which the frame's last byte is out. + qint64 end = 0; + /// @brief What the frame added to the backlog: its envelope's size. + std::size_t counted = 0; + /// @brief When the frame was handed over. + std::chrono::steady_clock::time_point queuedAt; + }; + + std::deque _frames; + qint64 _written = 0; +}; + +} // namespace morph::qt::detail diff --git a/include/morph/qt/qt_websocket_server.hpp b/include/morph/qt/qt_websocket_server.hpp index b99ab81c2..2899639f0 100644 --- a/include/morph/qt/qt_websocket_server.hpp +++ b/include/morph/qt/qt_websocket_server.hpp @@ -5,6 +5,9 @@ #include #include #ifndef QT_NO_SSL +#include +#endif +#ifndef QT_NO_SSL #include #endif #include @@ -13,10 +16,10 @@ #include #include #include -#include #include #include #include +#include #include #include #include @@ -107,8 +110,10 @@ struct QtWebSocketServerConfig { /// /// @par Resource limits /// Pass a `QtWebSocketServerConfig` to bound connection count, per-frame size, -/// per-connection message rate, and handshake/idle time. All fields default to -/// unbounded, except `maxMessageBytes`, which defaults to the wire-layer cap. +/// per-connection message rate, handshake/idle time, and how long a frame may +/// stay unwritten. All fields default to unbounded, except `maxMessageBytes`, +/// which defaults to the wire-layer cap, and `sendTimeout`, which defaults to +/// 30 s. /// /// @par Bind address & plaintext-exposure guard /// `listen()` refuses — returns `false` and logs at `morph::log::LogLevel::error` @@ -228,17 +233,6 @@ class QtWebSocketServer : public QObject { Q_SLOT void onBytesWritten(qint64 bytes); private: - /// @brief One frame handed to a socket and not yet written. - struct UnwrittenFrame { - /// @brief Position, in the socket's running count of written bytes, - /// at which this frame's last byte is written. - qint64 end = 0; - /// @brief What the frame added to the backlog: its envelope's size. - std::size_t counted = 0; - /// @brief When the frame was handed to the socket; drives `sendTimeout`. - std::chrono::steady_clock::time_point queuedAt; - }; - /// @brief Per-connection state tracked between accept and disconnect. struct ClientState { /// @brief The connection this state belongs to (for symmetry with the map key; unused for lookup). @@ -271,12 +265,18 @@ class QtWebSocketServer : public QObject { /// server strand, can read it without reaching this state. std::shared_ptr> backlog = std::make_shared>(0); - /// @brief Frames on the socket and not yet written, oldest first. - std::deque unwritten; + /// @brief Frames on the socket and not yet written, released as + /// `QWebSocket::bytesWritten` reports them out. + detail::OutboundLedger unwritten; - /// @brief Running count of bytes the socket has written, as reported by - /// `QWebSocket::bytesWritten`, less what it held at accept. - qint64 written = 0; +#ifndef QT_NO_SSL + /// @brief The TLS socket under `socket`, on a `SecureMode` server. + /// + /// `QWebSocket` reports encrypted bytes written but holds back only + /// the plaintext it has not yet encrypted; what is encrypted and still + /// unsent is on this socket, so a frame's position must count it. + QPointer tls; +#endif /// @brief One-shot timer enforcing `sendTimeout` on the oldest /// unwritten frame; `nullptr` if `sendTimeout == 0`. @@ -328,6 +328,20 @@ class QtWebSocketServer : public QObject { /// @param socket The connection's socket. void onSendTimer(QWebSocket* socket); + /// @brief Bytes @p state's socket stack still holds unsent: the + /// `QWebSocket`'s own, plus, over TLS, the ciphertext not yet sent. + /// @param state The connection. + /// @return The bytes held. + static qint64 pendingBytes(ClientState const& state); + +#ifndef QT_NO_SSL + /// @brief Finds the `QSslSocket` a `SecureMode` server accepted + /// @p socket over. + /// @param socket The upgraded connection. + /// @return The TLS socket, or null when there is none or it cannot be found. + [[nodiscard]] QSslSocket* findTlsSocket(QWebSocket const* socket) const; +#endif + /// @brief Releases every frame @p state still holds and drops its timers. /// Called when the connection goes away. /// @param state The connection. diff --git a/src/qt/qt_websocket_server.cpp b/src/qt/qt_websocket_server.cpp index f46ab0c80..f9ee38346 100644 --- a/src/qt/qt_websocket_server.cpp +++ b/src/qt/qt_websocket_server.cpp @@ -21,36 +21,6 @@ namespace morph::qt { -namespace { - -/// @brief Bytes a server-side `QWebSocket` puts on the socket for a text -/// message whose UTF-8 payload is @p payload bytes. -/// -/// `QWebSocket` splits a message into frames of at most @p maxFrame payload -/// bytes, each with its own header; a server's frames are unmasked, so a -/// header is 2 bytes plus a 2- or 8-byte extended length. -/// @param payload The message's UTF-8 length. -/// @param maxFrame The socket's `outgoingFrameSize()`. -/// @return The framed size. -qint64 framedSize(qint64 payload, quint64 maxFrame) { - auto const header = [](qint64 size) -> qint64 { - if (size < 126) { - return 2; - } - return size <= 0xFFFF ? 4 : 10; - }; - auto const cap = static_cast(std::max(maxFrame, 1)); - qint64 const whole = payload / cap; - qint64 const rest = payload % cap; - qint64 total = payload + (whole * header(cap)); - if (rest != 0 || whole == 0) { - total += header(rest); - } - return total; -} - -} // namespace - QtWebSocketServer::QtWebSocketServer(::morph::backend::RemoteServer& server, quint16 port, #ifndef QT_NO_SSL std::optional tls, @@ -247,9 +217,16 @@ void QtWebSocketServer::onNewConnection() { state.handshakeTimer = timer; } - // The handshake response may still be on the socket; its bytes are - // written ahead of any frame, so they start the count below zero. - state.written = -socket->bytesToWrite(); +#ifndef QT_NO_SSL + if (_wsServer.secureMode() == QWebSocketServer::SecureMode) { + state.tls = findTlsSocket(socket); + if (state.tls.isNull()) { + ::morph::log::logWarn("[QtWebSocketServer] connection {}: TLS socket not found; its backlog counts " + "only plaintext not yet encrypted, so a slow reader is not held back", + state.cid); + } + } +#endif if (_cfg.sendTimeout.count() > 0) { // NOLINTNEXTLINE(cppcoreguidelines-owning-memory) — QObject parent owns this auto* timer = new QTimer(this); @@ -313,21 +290,14 @@ void QtWebSocketServer::writeNow(QWebSocket* socket, ClientState& state, std::st } void QtWebSocketServer::writeCounted(QWebSocket* socket, ClientState& state, std::string const& envelope) { - qint64 const payload = socket->sendTextMessage(QString::fromStdString(envelope)); - if (payload <= 0) { + if (socket->sendTextMessage(QString::fromStdString(envelope)) <= 0) { // Not connected any more: nothing reached the socket. releaseBacklog(state, envelope.size()); return; } - // The frame ends where the socket's written count will stand once its - // last byte is out: after the frame ahead of it, or after whatever the - // socket has written so far. `bytesWritten` reports framed bytes, so the - // position is framed too; only the envelope's size is counted. - qint64 const base = state.unwritten.empty() ? state.written : state.unwritten.back().end; - state.unwritten.push_back(UnwrittenFrame{.end = base + framedSize(payload, socket->outgoingFrameSize()), - .counted = envelope.size(), - .queuedAt = std::chrono::steady_clock::now()}); - if (state.unwritten.size() == 1) { + bool const wasIdle = !state.unwritten.oldest().has_value(); + state.unwritten.push(pendingBytes(state), envelope.size(), std::chrono::steady_clock::now()); + if (wasIdle) { armSendTimer(state); } } @@ -358,19 +328,9 @@ void QtWebSocketServer::onBytesWritten(qint64 bytes) { return; } ClientState& state = iter->second; - state.written += bytes; - // An empty socket buffer means every frame handed to it is out, whatever - // the positions say: a frame whose framed size was misjudged is still - // released, rather than holding its bytes and tripping `sendTimeout`. - bool const allWritten = socket->bytesToWrite() == 0; - bool released = false; - while (!state.unwritten.empty() && (allWritten || state.unwritten.front().end <= state.written)) { - std::size_t const counted = state.unwritten.front().counted; - state.unwritten.pop_front(); - releaseBacklog(state, counted); - released = true; - } - if (released) { + std::size_t const released = state.unwritten.written(bytes, pendingBytes(state)); + if (released != 0) { + releaseBacklog(state, released); armSendTimer(state); } } @@ -379,22 +339,27 @@ void QtWebSocketServer::armSendTimer(ClientState& state) const { if (state.sendTimer == nullptr) { return; } - if (state.unwritten.empty()) { + auto const oldest = state.unwritten.oldest(); + if (!oldest) { state.sendTimer->stop(); return; } - auto const due = state.unwritten.front().queuedAt + _cfg.sendTimeout; + auto const due = *oldest + _cfg.sendTimeout; auto const remaining = std::chrono::ceil(due - std::chrono::steady_clock::now()); state.sendTimer->start(std::max(std::chrono::milliseconds{0}, remaining)); } void QtWebSocketServer::onSendTimer(QWebSocket* socket) { auto iter = _clients.find(socket); - if (iter == _clients.end() || iter->second.unwritten.empty()) { + if (iter == _clients.end()) { return; } ClientState& state = iter->second; - if (std::chrono::steady_clock::now() - state.unwritten.front().queuedAt < _cfg.sendTimeout) { + auto const oldest = state.unwritten.oldest(); + if (!oldest) { + return; + } + if (std::chrono::steady_clock::now() - *oldest < _cfg.sendTimeout) { armSendTimer(state); return; } @@ -411,14 +376,35 @@ void QtWebSocketServer::forgetConnection(ClientState& state) { state.sendTimer->deleteLater(); state.sendTimer = nullptr; } - std::size_t dropped = 0; - for (auto const& frame : state.unwritten) { - dropped += frame.counted; + state.backlog->fetch_sub(state.unwritten.clear()); +} + +qint64 QtWebSocketServer::pendingBytes(ClientState const& state) { + qint64 pending = state.socket->bytesToWrite(); +#ifndef QT_NO_SSL + if (!state.tls.isNull()) { + pending += state.tls->encryptedBytesToWrite(); } - state.unwritten.clear(); - state.backlog->fetch_sub(dropped); +#endif + return pending; } +#ifndef QT_NO_SSL +QSslSocket* QtWebSocketServer::findTlsSocket(QWebSocket const* socket) const { + // `QWebSocket` does not expose the socket it wraps. A `SecureMode` + // `QWebSocketServer` accepts through a `QSslServer` it owns, which parents + // each socket it accepts, so the socket is among the server's + // descendants; the connection's two endpoints pick it out. + for (auto* candidate : _wsServer.findChildren()) { + if (candidate->peerPort() == socket->peerPort() && candidate->peerAddress() == socket->peerAddress() && + candidate->localPort() == socket->localPort() && candidate->localAddress() == socket->localAddress()) { + return candidate; + } + } + return nullptr; +} +#endif + bool QtWebSocketServer::consumeToken(ClientState& state) const { if (_cfg.messagesPerSecond == 0) { return true; // unbounded (today's behavior) diff --git a/tests/qt/test_qt_websocket.cpp b/tests/qt/test_qt_websocket.cpp index 396f3c3ad..99b20aa53 100644 --- a/tests/qt/test_qt_websocket.cpp +++ b/tests/qt/test_qt_websocket.cpp @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -33,6 +34,7 @@ #include #include #include +#include #include #include #include @@ -3015,6 +3017,11 @@ morph::backend::ConnectionId nextQtConnectionId(morph::backend::RemoteServer& se return probe + 1; } +/// An event payload size a stalled peer cannot take in: more than the kernel's +/// socket buffers absorb (macOS grows a send buffer to 4 MiB), so most of it +/// stays on the server's socket, yet under the 8 MiB envelope cap. +constexpr std::size_t kStallBytes = std::size_t{7} << 20; + /// A JSON string payload of @p bytes characters. std::string bigPayload(std::size_t bytes) { return "\"" + std::string(bytes, 'x') + "\""; } @@ -3045,11 +3052,18 @@ TEST_CASE("qt_posted_event_reaches_handler", "[push][channel]") { CHECK(got[1].topic == "resync"); CHECK(exec.posted.load() == 2); - // Cleared: a later event is dropped, and the connection keeps working. + // Cleared: a later event is dropped. A handler installed after it sees + // only the marker posted after it, which also proves the dropped event + // had arrived: one connection's events arrive in order. backend.setEventHandler(nullptr, nullptr); - server->post(cid, morph::wire::encode(morph::wire::makeEvent("changes", "{}"))); - morph::exec::MainThreadExecutor cbExec; - CHECK(registerPrivate(backend, cbExec, "WsEchoModel").v != 0U); + server->post(cid, morph::wire::encode(morph::wire::makeEvent("changes", R"({"dropped":true})"))); + pumpUntil([] { return false; }, 10); + PushEventLog after; + backend.setEventHandler(after.handler(), &exec); + server->post(cid, morph::wire::encode(morph::wire::makeEvent("changes", R"({"marker":true})"))); + pumpUntil([&] { return after.size() == 1; }, 300); + REQUIRE(after.size() == 1); + CHECK(after.snapshot()[0].payload.str == R"({"marker":true})"); CHECK(log.size() == 2); } @@ -3207,7 +3221,7 @@ TEST_CASE("qt_stalled_peer_closed_after_send_timeout", "[push][channel]") { peer.setReadBufferSize(1); auto const posted = std::chrono::steady_clock::now(); - server->post(cid, morph::wire::encode(morph::wire::makeEvent("changes", bigPayload(std::size_t{4} << 20)))); + server->post(cid, morph::wire::encode(morph::wire::makeEvent("changes", bigPayload(kStallBytes)))); pumpUntil([&] { return findLogged("send timeout").has_value() && findLogged("disconnected").has_value(); }, 300); auto const closedAt = findLogged("send timeout"); @@ -3247,3 +3261,224 @@ TEST_CASE("qt_backlog_drains_and_sends_resync", "[push][channel]") { CHECK(got[1].topic == "resync"); CHECK(got[2].payload.str == R"({"after":true})"); } + +namespace { + +/// A WebSocket peer driven by hand over a raw `QTcpSocket` or `QSslSocket`: +/// it completes the upgrade, and then reads only when told to, so a test can +/// stall it with the server's frames piling up behind it. +class RawWsPeer { +public: + enum class Transport : std::uint8_t { Plain, Tls }; + + RawWsPeer(quint16 port, Transport transport) { + if (transport == Transport::Tls) { + auto ssl = std::make_unique(); + ssl->setPeerVerifyMode(QSslSocket::VerifyNone); + ssl->setSocketOption(QAbstractSocket::ReceiveBufferSizeSocketOption, 4096); + ssl->connectToHostEncrypted(QStringLiteral("127.0.0.1"), port); + auto* raw = ssl.get(); + pumpUntil([raw] { return raw->isEncrypted(); }, 300); + _socket = std::move(ssl); + } else { + _socket = std::make_unique(); + _socket->setSocketOption(QAbstractSocket::ReceiveBufferSizeSocketOption, 4096); + _socket->connectToHost(QHostAddress::LocalHost, port); + auto* raw = _socket.get(); + pumpUntil([raw] { return raw->state() == QAbstractSocket::ConnectedState; }, 100); + } + REQUIRE(_socket->state() == QAbstractSocket::ConnectedState); + _socket->write(QString("GET / HTTP/1.1\r\nHost: 127.0.0.1:%1\r\nUpgrade: websocket\r\nConnection: Upgrade\r\n" + "Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\nSec-WebSocket-Version: 13\r\n\r\n") + .arg(port) + .toUtf8()); + pumpUntil( + [this] { + _buffer += _socket->readAll(); + return _buffer.contains("\r\n\r\n"); + }, + 300); + REQUIRE(_buffer.startsWith("HTTP/1.1 101")); + _buffer.remove(0, _buffer.indexOf("\r\n\r\n") + 4); + } + + /// Stops reading: what the server sends piles up in the kernel and then + /// in the server's own socket buffer. + void stall() { _socket->setReadBufferSize(1); } + + /// Reads again, without limit. + void resume() { _socket->setReadBufferSize(0); } + + /// Pumps and reads until @p count whole text messages have arrived. + std::vector readMessages(std::size_t count, int maxIterations = 500) { + pumpUntil( + [&] { + _buffer += _socket->readAll(); + parse(); + return _messages.size() >= count; + }, + maxIterations); + return _messages; + } + +private: + /// Moves every whole frame out of `_buffer`, assembling text messages. + void parse() { + for (;;) { + if (_buffer.size() < 2) { + return; + } + auto const* data = reinterpret_cast(_buffer.constData()); + bool const fin = (data[0] & 0x80U) != 0; + unsigned const opcode = data[0] & 0x0FU; + std::uint64_t length = data[1] & 0x7FU; + qsizetype header = 2; + if (length == 126) { + if (_buffer.size() < 4) { + return; + } + length = (std::uint64_t{data[2]} << 8U) | data[3]; + header = 4; + } else if (length == 127) { + if (_buffer.size() < 10) { + return; + } + length = 0; + for (int i = 0; i < 8; ++i) { + length = (length << 8U) | data[2 + i]; + } + header = 10; + } + auto const total = header + static_cast(length); + if (_buffer.size() < total) { + return; + } + if (opcode == 0x1 || opcode == 0x0) { + _partial.append(_buffer.constData() + header, static_cast(length)); + if (fin) { + _messages.push_back(std::move(_partial)); + _partial.clear(); + } + } + _buffer.remove(0, total); + } + } + + std::unique_ptr _socket; + QByteArray _buffer; + std::string _partial; + std::vector _messages; +}; + +/// A Qt server, plain or TLS, with the given send timeout. +std::unique_ptr makePushServer(morph::backend::RemoteServer& server, + RawWsPeer::Transport transport, + std::chrono::milliseconds sendTimeout) { + morph::qt::QtWebSocketServerConfig cfg; + cfg.sendTimeout = sendTimeout; + std::optional tls; + if (transport == RawWsPeer::Transport::Tls) { + tls = makeServerTlsConfig(); + } + auto ws = std::make_unique(server, quint16{0}, tls, cfg); + REQUIRE(ws->listen()); + return ws; +} + +} // namespace + +TEST_CASE("qt_stalled_reader_is_held_back_then_resynced", "[push][channel]") { + ensureApp(); + auto const transport = GENERATE(RawWsPeer::Transport::Plain, RawWsPeer::Transport::Tls); + CAPTURE(transport == RawWsPeer::Transport::Tls ? "wss" : "ws"); + morph::exec::ThreadPoolExecutor serverPool{2}; + auto server = std::make_shared(serverPool); + auto const wsServer = makePushServer(*server, transport, std::chrono::milliseconds{0}); + auto const cid = nextQtConnectionId(*server); + RawWsPeer peer{wsServer->port(), transport}; + peer.stall(); + + // An event the peer does not read: the backlog stays above + // kEventBacklogHigh, so the next event is held back and owed as a resync. + server->post(cid, morph::wire::encode(morph::wire::makeEvent("changes", bigPayload(kStallBytes)))); + pumpUntil([] { return false; }, 30); + server->post(cid, morph::wire::encode(morph::wire::makeEvent("changes", R"({"held":true})"))); + pumpUntil([] { return false; }, 10); + + peer.resume(); + auto const got = peer.readMessages(2); + REQUIRE(got.size() >= 2); + CHECK(morph::wire::decode(got[0]).kind == "event"); + auto const second = morph::wire::decode(got[1]); + UNSCOPED_INFO("second message: " << got[1].substr(0, 120)); + CHECK(second.body.find(R"("topic":"resync")") != std::string::npos); + CHECK(second.body.find("held") == std::string::npos); +} + +TEST_CASE("qt_stalled_peer_closed_after_send_timeout_wss", "[push][channel]") { + ensureApp(); + morph::exec::ThreadPoolExecutor serverPool{2}; + auto server = std::make_shared(serverPool); + auto const wsServer = makePushServer(*server, RawWsPeer::Transport::Tls, std::chrono::milliseconds{200}); + auto const cid = nextQtConnectionId(*server); + + std::mutex logMtx; + std::vector> logged; + morph::log::ScopedLoggerOverride guard{[&](morph::log::LogLevel /*level*/, std::string_view msg) { + std::scoped_lock const lock{logMtx}; + logged.emplace_back(std::chrono::steady_clock::now(), + std::string{msg}); + }, + morph::log::LogLevel::debug}; + auto findLogged = [&](std::string_view needle) -> std::optional { + std::scoped_lock const lock{logMtx}; + for (auto const& [when, msg] : logged) { + if (msg.find(needle) != std::string::npos) { + return when; + } + } + return std::nullopt; + }; + + RawWsPeer peer{wsServer->port(), RawWsPeer::Transport::Tls}; + peer.stall(); + auto const posted = std::chrono::steady_clock::now(); + server->post(cid, morph::wire::encode(morph::wire::makeEvent("changes", bigPayload(kStallBytes)))); + pumpUntil([&] { return findLogged("send timeout").has_value() && findLogged("disconnected").has_value(); }, 300); + + auto const closedAt = findLogged("send timeout"); + REQUIRE(closedAt.has_value()); + CHECK(*closedAt - posted >= std::chrono::milliseconds{200}); + CHECK(findLogged("disconnected").has_value()); +} + +TEST_CASE("qt_ledger_releases_a_frame_once_its_last_byte_is_written", "[push][channel]") { + morph::qt::detail::OutboundLedger ledger; + auto const t0 = std::chrono::steady_clock::now(); + auto const t1 = t0 + std::chrono::milliseconds{5}; + // 30 bytes the socket queued for itself, then two frames behind them. + ledger.push(/*pending=*/130, /*counted=*/90, t0); + ledger.push(/*pending=*/330, /*counted=*/190, t1); + CHECK(ledger.oldest() == t0); + + // Short of the first frame's end, with bytes still held: nothing released. + CHECK(ledger.written(129, 201) == 0); + // Its last byte out while the second frame is still held: the first + // alone is released, by what it counted, not by the bytes written. + CHECK(ledger.written(1, 200) == 90); + CHECK(ledger.oldest() == t1); + CHECK(ledger.written(150, 50) == 0); + + // Nothing left on the socket: everything is out, whatever the positions. + CHECK(ledger.written(10, 0) == 190); + CHECK_FALSE(ledger.oldest().has_value()); + + // A position taken later counts from the bytes written so far. + ledger.push(/*pending=*/40, /*counted=*/30, t1); + CHECK(ledger.written(39, 1) == 0); + CHECK(ledger.written(1, 7) == 30); + + ledger.push(/*pending=*/100, /*counted=*/70, t1); + CHECK(ledger.clear() == 70); + CHECK_FALSE(ledger.oldest().has_value()); +} From a3980e341a76c76a747e2df4b20cc0074288cd54 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 15:09:56 +0200 Subject: [PATCH 24/68] core: SimulatedRemoteBackend carries events; one conformance case on every transport Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/backend.md | 21 ++++ include/morph/core/remote.hpp | 93 +++++++++++++- tests/net/test_push_net.cpp | 19 +++ tests/push_conformance.hpp | 214 +++++++++++++++++++++++++++++++++ tests/qt/test_qt_websocket.cpp | 22 ++++ tests/test_push_channel.cpp | 16 +++ 6 files changed, 384 insertions(+), 1 deletion(-) create mode 100644 tests/push_conformance.hpp diff --git a/docs/spec/core/backend.md b/docs/spec/core/backend.md index 2d3d7b835..adbad2f12 100644 --- a/docs/spec/core/backend.md +++ b/docs/spec/core/backend.md @@ -1195,6 +1195,27 @@ connection is gone" moment to hook here — so a caller that wants the scope reclaimed calls `server.closeConnection(cid)` explicitly (or lets the server itself be destroyed). +**Events.** The scoped constructor also makes the backend its connection's event +receiver: it hands the server a `ConnectionWriter` for the `ConnectionId`, and +`setEventHandler(handler, exec)` posts each `event` envelope the server writes +to `exec`, in the order the server posted them. The writer's `backlog` always +reads `0`: an in-process hand-off queues nothing, so `RemoteServer::post` never +holds an event back from it and never owes it a `resync`. The writer replaces +any writer the connection already had, so a second backend on the same scope +takes the events over. It shares its handler state with the backend rather than +pointing at it, so the server can keep posting after the backend is destroyed: +the destructor clears the handler and later envelopes are dropped. The +unscoped constructor has no connection, so it receives no events and +`setEventHandler` on it stores nothing. + +One conformance case, `channelConformance` in `tests/push_conformance.hpp`, runs +on this backend, `SocketBackend` and `QtWebSocketBackend` (`[push][channel]`). +Each suite hands it a factory for a server, a connected backend and the +connection's id. It checks that a posted event reaches the handler, that an +event carrying the `callId` of a call in flight reaches the handler and leaves +that call to its real reply, and that 50 events on one connection arrive in +the order they were posted. + ## `QtWebSocketBackend` — client-side WebSocket transport `morph::qt::QtWebSocketBackend` is the concrete `IBackend` that talks to a diff --git a/include/morph/core/remote.hpp b/include/morph/core/remote.hpp index 76df6094a..b0e30bb14 100644 --- a/include/morph/core/remote.hpp +++ b/include/morph/core/remote.hpp @@ -2065,6 +2065,12 @@ class RemoteServer : public std::enable_shared_from_this { /// Its pending list and session belong to the caller's owner (`setOwner`) and /// are touched only there, without a lock; the server's reply callback touches /// only the completion it settles. +/// +/// A backend scoped to a connection also receives the server's events for it +/// (`setEventHandler`): it registers a `ConnectionWriter` whose `write` hands +/// each envelope to the handler, in the order the server posted them, with a +/// `backlog` that always reads `0` -- an in-process hand-off holds nothing +/// back, so the server never withholds an event or owes a `resync`. class SimulatedRemoteBackend : public detail::IBackend { public: /// @brief Constructs the backend targeting @p server, unscoped. @@ -2104,8 +2110,33 @@ class SimulatedRemoteBackend : public detail::IBackend { /// the simulated connection should be reclaimed. /// @param server The `RemoteServer` instance to forward calls to. Borrowed, /// on the same terms as the constructor above. + /// + /// The backend also becomes @p cid's event receiver: it gives the server a + /// `ConnectionWriter` for @p cid, replacing any writer the connection had, + /// so a second backend on the same scope takes the events over. /// @param cid Connection scope, as returned by `server.openConnection()`. - SimulatedRemoteBackend(RemoteServer& server MORPH_LIFETIMEBOUND, ConnectionId cid) : _server{server}, _cid{cid} {} + SimulatedRemoteBackend(RemoteServer& server MORPH_LIFETIMEBOUND, ConnectionId cid) + : _server{server}, _cid{cid}, _inbox{std::make_shared()} { + if (_cid == 0) { + return; + } + _server.setConnectionWriter( + _cid, ConnectionWriter{.write = [inbox = _inbox](std::string frame) { inbox->receive(frame); }, + .backlog = [] { return std::size_t{0}; }}); + } + + /// @brief Stops delivering events: the server may still post to the + /// connection, and those envelopes are dropped. + ~SimulatedRemoteBackend() override { + if (_inbox) { + _inbox->set(nullptr, nullptr); + } + } + + SimulatedRemoteBackend(const SimulatedRemoteBackend&) = delete; + SimulatedRemoteBackend& operator=(const SimulatedRemoteBackend&) = delete; + SimulatedRemoteBackend(SimulatedRemoteBackend&&) = delete; + SimulatedRemoteBackend& operator=(SimulatedRemoteBackend&&) = delete; /// @brief Registers the model type on the server and returns its assigned id. /// @@ -2361,6 +2392,25 @@ class SimulatedRemoteBackend : public detail::IBackend { } } + /// @brief Installs the handler each server-initiated event is posted to. + /// See `IBackend::setEventHandler`. + /// + /// Only a backend scoped to a connection (the `ConnectionId` constructor) + /// receives events; an unscoped one has no connection to post to. Events + /// are posted to @p exec in the order the server posted them. An `event` + /// envelope is recognised by its kind, so it never settles a call. + /// @param handler Callable posted to @p exec once per event. Pass + /// `nullptr` to clear. + /// @param exec Executor the handler is posted to. Borrowed: it must + /// outlive the installation. May be null only when + /// @p handler is. + void setEventHandler(std::function handler, + ::morph::exec::IExecutor* exec) override { + if (_inbox) { + _inbox->set(std::move(handler), exec); + } + } + /// @brief Installs the session stamped onto every control envelope this /// backend subsequently builds (`register`, `registerShared`, /// `attach`, `assign`, `deregister`). See `IBackend::setSession`. @@ -2371,6 +2421,45 @@ class SimulatedRemoteBackend : public detail::IBackend { } private: + /// @brief The event handler and the entry point the connection's writer + /// calls, shared with that writer so the server can outlive the backend. + /// + /// The writer runs on the server strand and `setEventHandler` on the + /// backend's owner, so both sit behind a mutex. + struct EventInbox { + /// @brief Replaces the handler and its executor. + /// @param next New handler; empty clears it. + /// @param target Executor it is posted to. + void set(std::function next, ::morph::exec::IExecutor* target) { + std::lock_guard lock{mutex}; + handler = next ? std::make_shared>(std::move(next)) : nullptr; + exec = target; + } + + /// @brief Decodes one envelope the server wrote and posts an event to the handler. + /// @param frame The encoded envelope. Anything that is not a well-formed event is dropped. + void receive(const std::string& frame) { + std::shared_ptr> current; + ::morph::exec::IExecutor* target = nullptr; + { + std::lock_guard lock{mutex}; + current = handler; + target = exec; + } + if (!current || target == nullptr) { + return; + } + // By kind: an event's `callId` is not a reply's. + static_cast(detail::routeEvent(::morph::wire::decode(frame), [&](::morph::wire::EventBody event) { + target->post([current, event = std::move(event)]() mutable { (*current)(std::move(event)); }); + })); + } + + std::mutex mutex; + std::shared_ptr> handler; + ::morph::exec::IExecutor* exec = nullptr; + }; + /// @brief Checks, in a debug build, that the caller is on the owner the /// backend was given. Nothing to check before `setOwner`. /// @param site Name of the calling body. @@ -2413,6 +2502,8 @@ class SimulatedRemoteBackend : public detail::IBackend { // constructed with a ConnectionId from server.openConnection(). Threaded // through every handle()/handleInline() call this backend makes. ConnectionId _cid{0}; + // Null for an unscoped backend, which has no connection to receive on. + std::shared_ptr _inbox; // The owner every verb is called from, once `setOwner` has named it. The // two fields below are touched only there. std::optional<::morph::exec::detail::OwnerAffinity> _affinity; diff --git a/tests/net/test_push_net.cpp b/tests/net/test_push_net.cpp index 28a09cd7f..324ad663e 100644 --- a/tests/net/test_push_net.cpp +++ b/tests/net/test_push_net.cpp @@ -39,6 +39,7 @@ #include #include +#include "../push_conformance.hpp" #include "../test_support.hpp" // External linkage: glaze reflection and the BRIDGE_REGISTER_* macros need it. @@ -548,3 +549,21 @@ TEST_CASE("net_backlog_holds_events_and_drain_sends_resync", "[push][channel]") CHECK(next.kind == "event"); CHECK(::morph::wire::encode(next) == ::morph::wire::encode(::morph::wire::makeEvent("resync", "{}"))); } + +TEST_CASE("net_channel_conformance", "[push][channel]") { + ::morph::testing::push::channelConformance([](const auto& body) { + ::morph::exec::ThreadPoolExecutor pool{2}; + auto server = std::make_shared<::morph::backend::RemoteServer>(pool); + ::morph::net::SocketServer wsServer{*server, 0}; + REQUIRE(wsServer.listen()); + auto const cid = nextConnectionId(*server); + ::morph::net::SocketBackend backend{wsUrl(wsServer.port())}; + REQUIRE(backend.waitForConnected()); + ::morph::testing::push::ChannelFixture fixture{ + .server = server.get(), + .cid = cid, + .backend = &backend, + .waitFor = [](const std::function& done) { return ::morph::testing::waitUntil(done); }}; + body(fixture); + }); +} diff --git a/tests/push_conformance.hpp b/tests/push_conformance.hpp new file mode 100644 index 000000000..e7d61e130 --- /dev/null +++ b/tests/push_conformance.hpp @@ -0,0 +1,214 @@ +// SPDX-License-Identifier: Apache-2.0 + +#pragma once + +// One channel conformance case, instantiated by each remote backend's suite +// (simulated, `net`, Qt). It states what the server-to-client channel promises +// a backend, independent of the transport that carries it: a posted event +// reaches the handler, is never taken for a reply, and arrives in the order it +// was posted. +// +// Include this from one translation unit per test executable: it registers the +// model its pending-call check drives. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +// External linkage: glaze reflection and the BRIDGE_REGISTER_* macros need it. +// NOLINTBEGIN(misc-use-internal-linkage) +struct PcfGateAction { + int value = 0; +}; +/// Open while the model's action may finish; closed, the action holds, so a +/// call stays pending for as long as a test needs it. +inline std::atomic pcfGateOpen{true}; +struct PcfGateModel { + int execute(PcfGateAction /*action*/) { + auto const deadline = std::chrono::steady_clock::now() + std::chrono::seconds(20); + while (!pcfGateOpen.load() && std::chrono::steady_clock::now() < deadline) { + std::this_thread::sleep_for(std::chrono::milliseconds(1)); + } + return 7; + } +}; +// NOLINTEND(misc-use-internal-linkage) + +BRIDGE_REGISTER_MODEL(PcfGateModel, "PcfGateModel") +BRIDGE_REGISTER_ACTION(PcfGateModel, PcfGateAction, "PcfGateAction") + +namespace morph::testing::push { + +/// What a suite hands the conformance case: a server, a backend connected to +/// it, and the id of the connection the server posts to. +struct ChannelFixture { + /// The server that posts. It serves `PcfGateModel` from the global registry. + ::morph::backend::RemoteServer* server = nullptr; + /// The connection @p backend is on, as `server` numbers it. + ::morph::backend::ConnectionId cid = 0; + /// The client backend under test, already connected. + ::morph::backend::detail::IBackend* backend = nullptr; + /// Runs the transport until @p done holds or the suite's deadline passes. + /// @return Whether @p done held. + std::function& done)> waitFor; +}; + +/// Builds a fixture, runs the body it is given, then tears the fixture down. +using FixtureFactory = std::function& body)>; + +namespace detail { + +/// Runs each task inline on the posting thread. +struct InlineExec final : ::morph::exec::IExecutor { + void post(std::function task) override { task(); } +}; + +/// What an event handler saw, from whichever thread delivered it. +struct Events { + std::mutex mtx; + std::vector<::morph::wire::EventBody> seen; + + std::function handler() { + return [this](::morph::wire::EventBody body) { + std::scoped_lock const lock{mtx}; + seen.push_back(std::move(body)); + }; + } + std::size_t size() { + std::scoped_lock const lock{mtx}; + return seen.size(); + } + std::vector<::morph::wire::EventBody> snapshot() { + std::scoped_lock const lock{mtx}; + return seen; + } +}; + +/// Closes the gate on construction and opens it again on destruction, so a +/// failing check cannot leave a server action holding. +struct GateClosed { + GateClosed() { pcfGateOpen.store(false); } + ~GateClosed() { pcfGateOpen.store(true); } + GateClosed(const GateClosed&) = delete; + GateClosed& operator=(const GateClosed&) = delete; +}; + +inline std::string eventJson(std::size_t n) { return "{\"n\":" + std::to_string(n) + "}"; } + +} // namespace detail + +/// @brief The channel promises every remote backend keeps, checked through +/// the transport @p make builds. +/// +/// Three checks, each on a fresh fixture: a posted event reaches the handler; +/// an event carrying the call id of a call still in flight reaches the handler +/// and does not settle that call, whose real reply still does; and the events +/// of one connection arrive in the order they were posted. +/// @param make Builds the transport and runs the body it is handed. +inline void channelConformance(const FixtureFactory& make) { + SECTION("a posted event reaches the handler") { + make([](ChannelFixture& fx) { + detail::InlineExec exec; + detail::Events log; + fx.backend->setEventHandler(log.handler(), &exec); + + fx.server->post(fx.cid, ::morph::wire::encode(::morph::wire::makeEvent("changes", R"({"n":1})"))); + REQUIRE(fx.waitFor([&] { return log.size() == 1; })); + auto const got = log.snapshot(); + CHECK(got[0].topic == "changes"); + CHECK(got[0].payload.str == R"({"n":1})"); + }); + } + + SECTION("an event is not taken for a reply") { + make([](ChannelFixture& fx) { + detail::InlineExec exec; + detail::Events log; + fx.backend->setEventHandler(log.handler(), &exec); + + // Completions settle on their owner, which is drained from the + // thread that waits. + ::morph::exec::MainThreadExecutor owner; + auto const waitFor = [&fx, &owner](const std::function& done) { + return fx.waitFor([&] { + owner.drain(); + return done(); + }); + }; + + std::atomic modelId{0}; + fx.backend->bindModel({.typeId = "PcfGateModel"}, owner).then([&](::morph::exec::detail::ModelId mid) { + modelId.store(mid.v); + }); + REQUIRE(waitFor([&] { return modelId.load() != 0; })); + + detail::GateClosed const held; + ::morph::backend::detail::ActionCall call; + call.modelTypeId = "PcfGateModel"; + call.actionTypeId = "PcfGateAction"; + call.serializeAction = [](const void*) { return std::string{"{}"}; }; + call.deserializeResult = [](std::string_view body) -> std::shared_ptr { + return std::make_shared(std::stoi(std::string{body})); + }; + std::atomic result{-1}; + std::atomic failed{false}; + fx.backend->execute(::morph::exec::detail::ModelId{modelId.load()}, std::move(call), &owner) + .then([&](const std::shared_ptr& value) { result.store(*std::static_pointer_cast(value)); }) + .onError([&](const std::exception_ptr&) { failed.store(true); }); + + // The call's id is the transport's to choose, so the events carry + // every id a first call could have been filed under. + constexpr std::size_t kIds = 8; + for (std::size_t id = 0; id <= kIds; ++id) { + auto env = ::morph::wire::makeEvent("changes", detail::eventJson(id)); + env.callId = id; + fx.server->post(fx.cid, ::morph::wire::encode(env)); + } + REQUIRE(waitFor([&] { return log.size() == kIds + 1; })); + CHECK(result.load() == -1); + CHECK_FALSE(failed.load()); + + // The call is still pending, and its own reply settles it. + pcfGateOpen.store(true); + REQUIRE(waitFor([&] { return result.load() != -1 || failed.load(); })); + CHECK(result.load() == 7); + CHECK_FALSE(failed.load()); + }); + } + + SECTION("events on one connection arrive in posting order") { + make([](ChannelFixture& fx) { + detail::InlineExec exec; + detail::Events log; + fx.backend->setEventHandler(log.handler(), &exec); + + constexpr std::size_t kEvents = 50; + for (std::size_t n = 0; n < kEvents; ++n) { + fx.server->post(fx.cid, ::morph::wire::encode(::morph::wire::makeEvent("changes", detail::eventJson(n)))); + } + REQUIRE(fx.waitFor([&] { return log.size() == kEvents; })); + auto const got = log.snapshot(); + for (std::size_t n = 0; n < kEvents; ++n) { + CHECK(got[n].payload.str == detail::eventJson(n)); + } + }); + } +} + +} // namespace morph::testing::push diff --git a/tests/qt/test_qt_websocket.cpp b/tests/qt/test_qt_websocket.cpp index 99b20aa53..3df49199c 100644 --- a/tests/qt/test_qt_websocket.cpp +++ b/tests/qt/test_qt_websocket.cpp @@ -49,6 +49,7 @@ #include #include +#include "../push_conformance.hpp" #include "../test_support.hpp" // ── Shared QCoreApplication ────────────────────────────────────────────────── @@ -3482,3 +3483,24 @@ TEST_CASE("qt_ledger_releases_a_frame_once_its_last_byte_is_written", "[push][ch CHECK(ledger.clear() == 70); CHECK_FALSE(ledger.oldest().has_value()); } + +TEST_CASE("qt_channel_conformance", "[push][channel]") { + ensureApp(); + morph::testing::push::channelConformance([](const auto& body) { + morph::exec::ThreadPoolExecutor serverPool{2}; + auto server = std::make_shared(serverPool); + morph::qt::QtWebSocketServer wsServer{*server, 0}; + REQUIRE(wsServer.listen()); + auto const cid = nextQtConnectionId(*server); + morph::qt::QtWebSocketBackend backend{QUrl{QString("ws://127.0.0.1:%1").arg(wsServer.port())}}; + REQUIRE(backend.waitForConnected()); + morph::testing::push::ChannelFixture fixture{.server = server.get(), + .cid = cid, + .backend = &backend, + .waitFor = [](const std::function& done) { + pumpUntil(done, 300); + return done(); + }}; + body(fixture); + }); +} diff --git a/tests/test_push_channel.cpp b/tests/test_push_channel.cpp index 5fc3ea8a7..86db47d76 100644 --- a/tests/test_push_channel.cpp +++ b/tests/test_push_channel.cpp @@ -27,6 +27,7 @@ #include #include +#include "push_conformance.hpp" #include "test_support.hpp" namespace { @@ -366,3 +367,18 @@ TEST_CASE("throwing_backlog_read_is_caught_like_a_throwing_write", "[push][chann REQUIRE(hello.await()); CHECK(hello.env.kind == "ok"); } + +TEST_CASE("simulated_backend_channel_conformance", "[push][channel]") { + ::morph::testing::push::channelConformance([](const auto& body) { + ::morph::exec::ThreadPoolExecutor pool{2}; + auto server = std::make_shared<::morph::backend::RemoteServer>(pool); + auto const cid = server->openConnection(); + ::morph::backend::SimulatedRemoteBackend backend{*server, cid}; + ::morph::testing::push::ChannelFixture fixture{ + .server = server.get(), + .cid = cid, + .backend = &backend, + .waitFor = [](const std::function& done) { return ::morph::testing::waitUntil(done); }}; + body(fixture); + }); +} From 92eca962ad9d9c7b185346811e28326d77822804 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 15:40:09 +0200 Subject: [PATCH 25/68] ui: the catalog carries each action's effects and records Co-Authored-By: Claude Opus 5.5 --- docs/spec/ui/document.md | 9 ++- docs/spec/ui/service.md | 3 +- include/morph/ui/author.hpp | 48 ++++++++++++- include/morph/ui/document.hpp | 36 ++++++++++ tests/CMakeLists.txt | 1 + tests/test_push_catalog.cpp | 130 ++++++++++++++++++++++++++++++++++ 6 files changed, 224 insertions(+), 3 deletions(-) create mode 100644 tests/test_push_catalog.cpp diff --git a/docs/spec/ui/document.md b/docs/spec/ui/document.md index f9f0cae0e..99711d616 100644 --- a/docs/spec/ui/document.md +++ b/docs/spec/ui/document.md @@ -228,7 +228,7 @@ A node list is an array of nodes or one node. `nodeKinds()` lists the kinds. `ui::Catalog` holds two lookups the caller supplies: `model(typeId)` → `ModelInfo{stateless, keyed}` and `action(actionId)` → `ActionInfo{modelType, calculatedFields, secretFields, -credential, listFields}`. In local mode the caller answers them in-process from the model registry; on the +credential, listFields, effects, records}`. In local mode the caller answers them in-process from the model registry; on the server, registration answers them from the same registry; a remote client answers them from the UI service's `ui-catalog` reply. With them, a document is also refused when a model type or action is unknown, an `instance` expression names a model that is not keyed, an action belongs to another @@ -238,6 +238,13 @@ the server calculates, or a mutation that sends a credential or returns a secret that hold a list; a `patch` or `optimistic` of such a query is then refused without an `at` (it could never find a list) or with an `at` that is not one of them. When it is absent, the catalog does not know the result's shape, and a patch that finds no list is reported by the interpreter when it runs. +`effects` and `records` are what the core registries hold for the action: `effects` is the list of +`{kind, record?}` the action declared, `kind` one of `updates`, `changes`, `changesInstance` and `none`, `record` the +record's wire name for the first two; it is absent when the action declared nothing, which makes the action a read. +`records` lists the records the action's result holds as `{record, keyField?, paths}`, empty when it holds none. They +are read from the registries where `CatalogFacts::action` records the action, so an invalid effects and records +configuration (`EffectsRegistrationError`) fails at that call, at startup. No load-time rule reads them yet; they +are what a client's watch is derived from. A lookup left empty skips its rules; the document is otherwise checked in full. `stateless` decides an alias's default instance, which is the interpreter's. A third lookup, `schema(actionId)`, answers an action's schema JSON; it is not a load-time rule but what the interpreter builds a `form` node's diff --git a/docs/spec/ui/service.md b/docs/spec/ui/service.md index b023640fc..b398e7fa4 100644 --- a/docs/spec/ui/service.md +++ b/docs/spec/ui/service.md @@ -109,7 +109,8 @@ For each registered entry, in registration order: - `models` and `actions` carry the `ui::Catalog` facts (`ModelInfo`, `ActionInfo`) the registry's catalog gave, for the documents the principal may fetch: the app document, and every listed screen that is available or public. They are absent when the registry had no lookup, which leaves the client's lookup empty and skips its - rules, as on the server. `CatalogReply::facts()` turns them into the lookups `loadDocument` takes. + rules, as on the server. `ActionInfo` carries each action's `effects` and `records` as optional keys: a reply + without `effects` reads as no declared effects (a read), one without `records` as none. `CatalogReply::facts()` turns them into the lookups `loadDocument` takes. - `views` and `wizards` carry the forms documents a `collection` or `wizard` node builds from, keyed by the action the node names, for the same documents. The registry takes them from its catalog's `view` and `wizard` lookups, for every action a document names. The author builders put them there diff --git a/include/morph/ui/author.hpp b/include/morph/ui/author.hpp index fc8b07169..b2c0755a4 100644 --- a/include/morph/ui/author.hpp +++ b/include/morph/ui/author.hpp @@ -22,6 +22,7 @@ #include #include +#include "../core/effects.hpp" #include "../core/model_key.hpp" #include "../core/registry.hpp" #include "../detail/fixed_string.hpp" @@ -113,6 +114,45 @@ template } } +/// @brief The catalog's form of an action's declared effects. +/// @param actionId The action's id. +/// @return Its effects, or `std::nullopt` when it declared none: the action reads. +/// @throws morph::model::EffectsRegistrationError when the registered records and effects are invalid. +[[nodiscard]] inline std::optional> effectsInfo(std::string_view actionId) { + auto const declared = ::morph::effects::effectsOf(actionId); + if (!declared) { + return std::nullopt; + } + std::vector out; + out.reserve(declared->size()); + for (auto const& effect : *declared) { + std::string_view kind = "none"; + if (effect.kind == ::morph::effects::Kind::Updates) { + kind = "updates"; + } else if (effect.kind == ::morph::effects::Kind::Changes) { + kind = "changes"; + } else if (effect.kind == ::morph::effects::Kind::ChangesInstance) { + kind = "changesInstance"; + } + out.push_back(ActionEffect{.kind = std::string{kind}, + .record = effect.record.empty() ? std::nullopt + : std::optional{effect.record}}); + } + return out; +} + +/// @brief The catalog's form of the records an action's result holds. +/// @param actionId The action's id. +/// @return One entry per record; empty when the result holds none. +/// @throws morph::model::EffectsRegistrationError when the registered records and effects are invalid. +[[nodiscard]] inline std::vector recordsInfo(std::string_view actionId) { + std::vector out; + for (auto const& held : ::morph::effects::recordsOf(actionId)) { + out.push_back(ActionRecordInfo{.record = held.record, .keyField = held.keyField, .paths = held.paths}); + } + return out; +} + } // namespace detail /// @brief The catalog facts the builders gathered from the model and action types a screen names. @@ -151,11 +191,15 @@ struct CatalogFacts { /// @brief Records an action as one of a model type's. /// @tparam Model The model type that runs it. /// @tparam Action The action type; `::morph::model::ActionTraits` names its id. + /// @throws morph::model::EffectsRegistrationError when the registered records and effects are invalid: the + /// action's effects and records are read from the process registry here, so a misconfigured registry + /// fails at the start, not on a client's first watch. template requires Runs void action() { model(); - actions.emplace(std::string{::morph::model::ActionTraits::typeId()}, + auto const actionId = std::string{::morph::model::ActionTraits::typeId()}; + actions.emplace(actionId, ActionInfo{ .modelType = std::string{::morph::model::ModelTraits::typeId()}, .calculatedFields = ActionFacts::calculatedFields(), @@ -163,6 +207,8 @@ struct CatalogFacts { .credential = ActionFacts::credential, .listFields = detail::listFields<::morph::model::HandlerResultT< decltype(std::declval().execute(std::declval()))>>(), + .effects = detail::effectsInfo(actionId), + .records = detail::recordsInfo(actionId), }); } diff --git a/include/morph/ui/document.hpp b/include/morph/ui/document.hpp index 2ae8fff95..6c8ab91ee 100644 --- a/include/morph/ui/document.hpp +++ b/include/morph/ui/document.hpp @@ -44,6 +44,36 @@ struct ModelInfo { bool keyed = false; }; +/// @brief One effect an action declares, as the catalog states it. +struct ActionEffect { + /// @brief What the effect is: `updates`, `changes`, `changesInstance` or `none`. + std::string kind; + /// @brief The wire name of the record the effect names; absent for `changesInstance` and `none`. + std::optional record; + + /// @brief Memberwise equality. + /// @param lhs One effect. + /// @param rhs The other. + /// @return True when kind and record match. + friend bool operator==(ActionEffect const& lhs, ActionEffect const& rhs) = default; +}; + +/// @brief A record an action's result holds, as the catalog states it. +struct ActionRecordInfo { + /// @brief The record's wire name. + std::string record; + /// @brief The record's key field; absent for a keyless record. + std::optional keyField; + /// @brief The JSON paths at which the result holds the record; `""` is the result itself. + std::vector paths; + + /// @brief Memberwise equality. + /// @param lhs One entry. + /// @param rhs The other. + /// @return True when record, key field and paths match. + friend bool operator==(ActionRecordInfo const& lhs, ActionRecordInfo const& rhs) = default; +}; + /// @brief What the catalog says about an action. struct ActionInfo { /// @brief The registered type id of the model the action belongs to. @@ -59,6 +89,12 @@ struct ActionInfo { /// `optimistic` of a query of the action may name as its `at`. Empty when the catalog does not say, or the /// result is not a record. std::optional> listFields; + /// @brief What the action declares it changes. Absent when it declared nothing, which makes it a read; present + /// and empty is not produced by the registry. + std::optional> effects; + /// @brief The records the action's result holds, with the paths to each. Empty when it holds none or the + /// catalog does not say. + std::vector records; }; /// @brief The facts load-time validation needs from the application's catalog. Local mode answers them from the diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index c3f9f07d0..b67e5579f 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -24,6 +24,7 @@ add_executable(morph_tests test_example.cpp test_push_records.cpp test_push_effects.cpp + test_push_catalog.cpp test_push_channel.cpp push_records_late.cpp test_executor_extra.cpp diff --git a/tests/test_push_catalog.cpp b/tests/test_push_catalog.cpp new file mode 100644 index 000000000..ea069b354 --- /dev/null +++ b/tests/test_push_catalog.cpp @@ -0,0 +1,130 @@ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +// Glaze reflects member names through an external-linkage object, so these types cannot sit in an +// anonymous namespace. +namespace push_catalog_test { + +struct Entry { + std::int64_t id{}; + std::string name; +}; + +struct EntryList { + std::vector entries; +}; + +struct EditEntry { + std::int64_t id{}; +}; + +struct ListEntries { + int limit{}; +}; + +struct ReopenEntry { + std::int64_t id{}; +}; + +struct PingEntry { + int unused{}; +}; + +struct CatalogModel { + EntryList execute(EditEntry const& /*action*/) { return {}; } + EntryList execute(ListEntries const& /*action*/) { return {}; } + EntryList execute(ReopenEntry const& /*action*/) { return {}; } + int execute(PingEntry const& /*action*/) { return 0; } +}; + +} // namespace push_catalog_test + +using namespace push_catalog_test; +namespace effects = morph::effects; + +BRIDGE_REGISTER_RECORD(Entry, "T_CatalogEntry") +BRIDGE_REGISTER_MODEL(CatalogModel, "T_CatalogModel") +BRIDGE_REGISTER_ACTION(CatalogModel, EditEntry, "T_CatalogEdit") +BRIDGE_REGISTER_EFFECTS(EditEntry, effects::updates, effects::changes) +BRIDGE_REGISTER_ACTION(CatalogModel, ListEntries, "T_CatalogList") +BRIDGE_REGISTER_ACTION(CatalogModel, ReopenEntry, "T_CatalogReopen") +BRIDGE_REGISTER_EFFECTS(ReopenEntry, effects::changesInstance) +BRIDGE_REGISTER_ACTION(CatalogModel, PingEntry, "T_CatalogPing") +BRIDGE_REGISTER_EFFECTS(PingEntry, effects::none) + +namespace { + +morph::ui::author::CatalogFacts registeredFacts() { + morph::ui::author::CatalogFacts facts; + facts.action(); + facts.action(); + facts.action(); + facts.action(); + return facts; +} + +} // namespace + +TEST_CASE("catalog_facts_carry_effects_and_records", "[push][catalog]") { + auto const facts = registeredFacts(); + + auto const& edit = facts.actions.at("T_CatalogEdit"); + REQUIRE(edit.effects.has_value()); + REQUIRE(*edit.effects == std::vector{{"updates", "T_CatalogEntry"}, + {"changes", "T_CatalogEntry"}}); + REQUIRE(edit.records.size() == 1U); + CHECK(edit.records.front().record == "T_CatalogEntry"); + CHECK(edit.records.front().keyField == std::optional{"id"}); + CHECK(edit.records.front().paths == std::vector{"entries"}); + + CHECK(*facts.actions.at("T_CatalogReopen").effects == + std::vector{{"changesInstance", std::nullopt}}); + CHECK(*facts.actions.at("T_CatalogPing").effects == std::vector{{"none", std::nullopt}}); + CHECK(facts.actions.at("T_CatalogPing").records.empty()); + + // A read declares no effects but still holds the record. + auto const& list = facts.actions.at("T_CatalogList"); + CHECK_FALSE(list.effects.has_value()); + CHECK(list.records == edit.records); + + // The catalog's lookup serves the same facts. + auto const found = facts.catalog().action("T_CatalogEdit"); + REQUIRE(found); + CHECK(found->effects == edit.effects); +} + +TEST_CASE("ui_catalog_reply_round_trips_effects", "[push][catalog]") { + morph::ui::CatalogReply reply; + reply.actions = registeredFacts().actions; + + std::string text; + REQUIRE_FALSE(glz::write_json(reply, text)); + auto const parsed = morph::ui::parseCatalogReply(text); + REQUIRE(parsed); + REQUIRE(parsed->actions.has_value()); + + auto const& edit = parsed->actions->at("T_CatalogEdit"); + CHECK(edit.effects == reply.actions->at("T_CatalogEdit").effects); + CHECK(edit.records == reply.actions->at("T_CatalogEdit").records); + CHECK_FALSE(parsed->actions->at("T_CatalogList").effects.has_value()); + CHECK(text.find(R"("effects":[{"kind":"updates","record":"T_CatalogEntry"})") != std::string::npos); + CHECK(text.find(R"({"kind":"changesInstance"})") != std::string::npos); +} + +TEST_CASE("old_reply_without_fields_reads_as_reads", "[push][catalog]") { + auto const parsed = + morph::ui::parseCatalogReply(R"({"entries":[],"permitted":[],"actions":{"Old_Get":{"modelType":"Old"}}})"); + REQUIRE(parsed); + auto const& info = parsed->actions->at("Old_Get"); + CHECK_FALSE(info.effects.has_value()); + CHECK(info.records.empty()); +} From 7c8899b240e91c83577a67f1e4992727294a1adc Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 15:57:42 +0200 Subject: [PATCH 26/68] core, ui: negotiate events in ui-hello; core learns it through ConnectionFacts IEnvelopeService gains a non-pure three-parameter handle taking the connection's ConnectionFacts, whose default forwards to the pure two-parameter one. RemoteServer passes the facts of the envelope's cid, kept beside the connection scopes on the server strand and dropped by closeConnection; a message with no open connection gets throwaway facts. eventsNegotiated(cid) reads the flag, server strand only. HelloRequest and HelloReply gain an additive `events`. UiService accepts an offer on a hello it answers ok and sets facts.events; the generic client's ui-hello (clientHello) offers it. The in-repo two-parameter services add `using IEnvelopeService::handle;`: GCC's -Woverloaded-virtual at level 2 flags the hidden overload. Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/backend.md | 30 +++++++- docs/spec/ui/service.md | 20 ++++- examples/common/app/app_source.cpp | 15 ++-- examples/common/app/app_source.hpp | 4 + include/morph/core/remote.hpp | 76 +++++++++++++++++- include/morph/ui/bundle.hpp | 5 ++ include/morph/ui/service.hpp | 29 ++++++- tests/net/test_socket_backend.cpp | 2 + tests/qt/test_qt_websocket.cpp | 2 + tests/test_push_catalog.cpp | 102 +++++++++++++++++++++++++ tests/test_remote_envelope_service.cpp | 93 ++++++++++++++++++++++ 11 files changed, 365 insertions(+), 13 deletions(-) diff --git a/docs/spec/core/backend.md b/docs/spec/core/backend.md index adbad2f12..4bdffe13c 100644 --- a/docs/spec/core/backend.md +++ b/docs/spec/core/backend.md @@ -888,6 +888,33 @@ ships; it answers `ui-hello`, `ui-catalog` and `ui-fetch`. empty one, never the client's claim. - **Authorising is the service's.** `handle(request, authorizer)` is given the server's `IAuthorizer`; core makes no decision about a kind it does not know. +- **Connection facts.** What the server calls is the three-parameter + `handle(request, authorizer, facts)`, where `facts` is the `ConnectionFacts` + of the connection the envelope arrived on. It is how core learns what a + service negotiated without parsing that service's bodies: today the one fact + is `events`, which the UI service sets when a `ui-hello` offers events and it + accepts ([ui/service.md](../ui/service.md#the-kinds)), and which core reads + at fan-out through `eventsNegotiated(cid)`. + - The three-parameter `handle` is a non-pure virtual whose default forwards to + the two-parameter one, which stays pure. A service that learns nothing + about connections overrides only the two-parameter one, and so adds + `using IEnvelopeService::handle;`: GCC's `-Woverloaded-virtual`, which the + strict build enables at level 2, flags a derived class that overrides some + overloads of a virtual and hides the rest. Clang does not, so only the GCC + build catches its omission. A service that overrides both needs no `using`. + - The facts sit beside the connection scopes in a `cid → ConnectionFacts` map + only the server strand touches, are created on the first envelope a + service handles for an open connection, and are dropped by + `closeConnection(cid)`. A reconnect is a new connection and starts from + `events = false`. + - A message with no open connection behind it — `handle(msg, reply)`, + `handleInline(msg)`, or a `cid` never opened or already closed — is handed + throwaway facts, reset to `events = false` before each such message and + kept by nothing, so a fact can only be recorded where the connection that + negotiated it can be found again. + - `eventsNegotiated(cid) const` is server-strand only (a debug build + asserts it): it is read where fan-out runs, on the strand, and the map + has no lock. It is `false` for `0`, an unknown `cid` or a closed one. - **Synchronous, on the server strand.** `handle` returns the encoded reply and must not block. A throw becomes an `err` reply carrying its message, through the same catch every other kind uses. @@ -2170,10 +2197,11 @@ server: each call is a loop task, so the loop serialises them. | `handle(msg, reply, cid)` | Like `handle(msg, reply)`, additionally attributing any `register` in `msg` to connection `cid`'s scope. `cid == 0` behaves exactly like the two-argument overload. | | `handleInline(msg)` | Sync: runs the control envelope on the server strand — posted and waited for, or inline when already on it — and returns the reply JSON. **Rejects `execute`** — returns an `err` reply without dispatching, because an `execute` reply is produced asynchronously after this call returns. Needs a pool thread other than the caller's free to run the strand. Unscoped. | | `openConnection()` | Returns a fresh non-zero `ConnectionId` at once; its empty scope is opened on the server strand, before the caller's next `handle()`. | -| `closeConnection(cid)` | Posts: erases every model still recorded in `cid`'s scope (as `deregister` would) and drops the scope and its writer. `cid == 0`, unknown, or already-closed is a no-op — idempotent. Bypasses `IAuthorizer` by design. | +| `closeConnection(cid)` | Posts: erases every model still recorded in `cid`'s scope (as `deregister` would) and drops the scope, its writer and its `ConnectionFacts`. `cid == 0`, unknown, or already-closed is a no-op — idempotent. Bypasses `IAuthorizer` by design. | | `setConnectionWriter(cid, writer)` | Posts: keeps `writer` (`ConnectionWriter{write, backlog}`) for `cid` while its scope is open; replaces an earlier one, keeping an owed `resync`; an empty `write` removes it. `0`, unknown or closed `cid`: dropped. | | `post(cid, envelope)` | Any thread. Above `wire::kMaxEnvelopeBytes`: dropped and counted (`Metric::eventOversize`). On the strand: written through `cid`'s writer, or held back — dropped, with one `resync` owed — while `backlog()` exceeds `kEventBacklogHigh` or a `resync` is still owed. No writer: dropped. | | `connectionDrained(cid)` | Any thread; posts. Sends the owed `resync` event, if any, and lets events flow again. What a transport calls when the backlog falls below `kEventBacklogLow`. | +| `eventsNegotiated(cid) const` | Server strand only (debug-asserted). Whether `cid` negotiated events: the `ConnectionFacts::events` a service set for it. `false` for `0`, an unknown `cid` or a closed one. | | `payloadCompleteness()` | The `ServerConfig::payloadCompleteness` given at construction. | | `strand()` | The server strand, as an `exec::IExecutor`: `runningOn(server.strand())` is true inside every task on the server's state. | | `health(replyExec)` | `[[nodiscard]] Completion health(IExecutor& replyExec)` — readiness/liveModels/inFlight, answered on the server strand; delivered on `replyExec`, the executor the caller attaches from (borrowed: it must outlive the delivery). See [observability.md](observability.md). | diff --git a/docs/spec/ui/service.md b/docs/spec/ui/service.md index b398e7fa4..d114ee635 100644 --- a/docs/spec/ui/service.md +++ b/docs/spec/ui/service.md @@ -36,7 +36,7 @@ the kinds are in the wire kind table ([`../core/wire.md`](../core/wire.md#the-ui | Kind | Request `body` | `ok` reply `body` | |---|---|---| -| `ui-hello` | `HelloRequest`: `{vocabularies, qt, components, hostComponents, hostCommands, hostValues}` | `HelloReply`: `{application, version, vocabularies, digest}` | +| `ui-hello` | `HelloRequest`: `{vocabularies, qt, components, hostComponents, hostCommands, hostValues, events?}` | `HelloReply`: `{application, version, vocabularies, digest, events?}` | | `ui-catalog` | empty | `CatalogReply`: `{entries, permitted, models?, actions?, views?, wizards?}` | | `ui-fetch` | `FetchRequest`: `{hash}` | the file's exact bytes | | `ui-evaluate` | the action's JSON body; `modelType` and `actionType` name it | `EvaluateReply`: `{body, errors, ready}` | @@ -47,6 +47,24 @@ service sees it. `ui-hello` refuses a client that does not speak every vocabular `err "missing vocabulary: [, …]"`; the bundle always needs `expr/1` and `ui/1`, and whatever the registry adds (`requireVocabulary`). `hello` and `kProtocolVersion` are unchanged. +**Negotiating events.** `ui-hello` is where a connection negotiates server-sent `event` envelopes, since the generic +client already opens with it, on the same connection as its executes. `HelloRequest::events: true` offers them; +the service accepts every offer on a hello it answers `ok`, says so with `HelloReply::events: true`, and records it +by setting `ConnectionFacts::events` on the connection the hello arrived on +([core/backend.md, "Envelope services"](../core/backend.md#envelope-services)). Core reads that flag; it never parses +a `ui-*` body. Both keys are additive and default to `false`, so a client that never offers sees the protocol it +saw before, and an old client reading a new reply ignores the key. + +- A hello that does not offer events leaves the connection's flag as it is, so a later hello on the same connection, + such as a development reload's poll, does not withdraw what the first negotiated. The flag goes with the + connection; a reconnect is a new connection and negotiates again. +- A refused hello (`unauthorized`, `missing vocabulary`, a body it cannot read) negotiates nothing. +- A hello handled with no connection behind it is still answered `events: true` when it offers them, and the + acceptance is recorded nowhere: there is no connection to send events on. +- `UiService` therefore overrides both `handle` overloads; the two-parameter one answers with throwaway facts. +- The generic client (`examples/common/app`'s `clientHello()`) offers events on every `ui-hello` it sends. `UiClient` + sends the `HelloRequest` it was given, unchanged. + ## Content addressing - A **content hash** is `sha256:` and the 64 lower-case hexadecimal digits of the SHA-256 digest diff --git a/examples/common/app/app_source.cpp b/examples/common/app/app_source.cpp index 88403ff10..37c3100e7 100644 --- a/examples/common/app/app_source.cpp +++ b/examples/common/app/app_source.cpp @@ -126,6 +126,13 @@ std::vector clientVocabularies() { return {std::string{ui::kVocabulary}, std::string{ui::kExprVocabulary}}; } +ui::HelloRequest clientHello() { + ui::HelloRequest hello; + hello.vocabularies = clientVocabularies(); + hello.events = true; + return hello; +} + // ── ClientSource ──────────────────────────────────────────────────────────── ui::Dispatcher& ClientSource::dispatcher() { @@ -341,9 +348,7 @@ void ClientSource::open(ui::AppContext& context, void ClientSource::reload(std::shared_ptr const& alive, std::function)> done) { - ui::HelloRequest hello; - hello.vocabularies = clientVocabularies(); - ui::UiClient const client{_exchange, std::move(hello), _store}; + ui::UiClient const client{_exchange, clientHello(), _store}; client.open(_session, [this, alive, done = std::move(done)](std::expected opened) mutable { if (!*alive) { @@ -368,9 +373,7 @@ void ClientSource::poll(std::function changed)> ch return; } _polling = true; - ui::HelloRequest hello; - hello.vocabularies = clientVocabularies(); - ui::UiClient const client{_exchange, std::move(hello), _store}; + ui::UiClient const client{_exchange, clientHello(), _store}; client.poll(_session, _loaded->hello.digest, [this, alive = _alive, changed = std::move(changed)]( std::expected, ui::ConnectError> polled) mutable { diff --git a/examples/common/app/app_source.hpp b/examples/common/app/app_source.hpp index 9cfa70217..df033ae20 100644 --- a/examples/common/app/app_source.hpp +++ b/examples/common/app/app_source.hpp @@ -82,6 +82,10 @@ struct LocalApp { /// @brief The vocabularies the generic client speaks. [[nodiscard]] std::vector clientVocabularies(); +/// @brief The `ui-hello` the generic client opens with, and sends again on each poll: its vocabularies, and the +/// offer of server-sent events on the connection. +[[nodiscard]] ui::HelloRequest clientHello(); + /// @brief An `AppSource` that also hands the shell what it mounts screens with, once it delivered a bundle. /// /// Opened, it holds a bridge over its backend, owned by the context's executor, and the verified bundle with its diff --git a/include/morph/core/remote.hpp b/include/morph/core/remote.hpp index b0e30bb14..950b812cd 100644 --- a/include/morph/core/remote.hpp +++ b/include/morph/core/remote.hpp @@ -4,6 +4,7 @@ #include #include #include +#include #include #include #include @@ -203,6 +204,19 @@ struct HealthStatus { using LogProvider = std::function(std::string_view modelType, std::string_view contextKey)>; +/// @brief What a `RemoteServer` knows about one connection that a service, not +/// core, decides: handed to `IEnvelopeService::handle` with each envelope. +/// +/// A service sets a fact while answering an envelope it serves, and core reads +/// it later without parsing that service's bodies. The facts live as long as +/// the connection; a message handled without a connection (`cid` 0) is handed +/// fresh facts that are dropped after its reply. +struct ConnectionFacts { + /// @brief Whether the connection negotiated server-sent events: its + /// `ui-hello` offered them and the UI service accepted. + bool events = false; +}; + /// @brief Envelope kinds a `RemoteServer` answers through an extension, such as /// the UI service's `ui-*` kinds. /// @@ -239,6 +253,28 @@ class IEnvelopeService { /// `request.callId`. [[nodiscard]] virtual std::string handle(const ::morph::wire::Envelope& request, const ::morph::session::IAuthorizer& authorizer) = 0; + + /// @brief Answers one envelope of a kind `serves` accepted, with the facts + /// of the connection it arrived on. + /// + /// What the server calls. A service that learns something about the + /// connection, as the UI service learns that `ui-hello` negotiated events, + /// overrides this one and records it in @p facts; the default forwards to + /// the two-parameter `handle` and leaves @p facts alone. A service + /// overriding only one of the two adds `using IEnvelopeService::handle;`, + /// so the other is not hidden. Same thread and error rules as the + /// two-parameter `handle`. + /// @param request The decoded envelope. + /// @param authorizer The server's authoriser. + /// @param facts The connection's facts, which the service may set; + /// fresh ones for a message handled without a connection. + /// @return The encoded reply envelope (`wire::encode`), addressed to + /// `request.callId`. + [[nodiscard]] virtual std::string handle(const ::morph::wire::Envelope& request, + const ::morph::session::IAuthorizer& authorizer, ConnectionFacts& facts) { + static_cast(facts); + return handle(request, authorizer); + } }; /// @brief Everything about a `RemoteServer` that is configured rather than @@ -663,6 +699,21 @@ class RemoteServer : public std::enable_shared_from_this { _strand.postTask([self = shared_from_this(), cid] { self->connectionDrainedHere(cid); }); } + /// @brief Whether connection @p cid negotiated server-sent events. + /// + /// Server strand only: call it from a task on `strand()`; a debug build + /// asserts that. The answer is what the UI service recorded when the + /// connection's `ui-hello` offered events, and stays until the connection + /// closes; a reconnect is a new connection and negotiates again. + /// @param cid Connection, as returned by `openConnection()`. + /// @return `true` once the connection negotiated events; `false` for `0`, + /// an unknown connection or a closed one. + [[nodiscard]] bool eventsNegotiated(ConnectionId cid) const { + assert(_strand.runningHere() && "RemoteServer::eventsNegotiated runs on the server strand"); + auto const found = _facts.find(cid); + return found != _facts.end() && found->second.events; + } + /// @brief The server strand, as an executor. /// /// What `runningOn(server.strand())` asks about: true inside every task the @@ -841,6 +892,7 @@ class RemoteServer : public std::enable_shared_from_this { void closeConnectionHere(ConnectionId cid) { noteOwner("RemoteServer::closeConnection"); _writers.erase(cid); + _facts.erase(cid); auto scopeIter = _connectionScopes.find(cid); if (scopeIter == _connectionScopes.end()) { return; @@ -856,6 +908,22 @@ class RemoteServer : public std::enable_shared_from_this { _connectionScopes.erase(scopeIter); } + /// @brief The facts a service records for connection @p cid. On the server + /// strand. + /// + /// Fresh facts for `0`, an unknown `cid` or one already closed, so what a + /// service sets there outlives no reply: a fact is kept only where the + /// connection that negotiated it can be found again. + /// @param cid Connection the envelope arrived on. + /// @return The connection's facts, or throwaway ones. + [[nodiscard]] ConnectionFacts& factsOf(ConnectionId cid) { + if (cid == 0 || !_connectionScopes.contains(cid)) { + _throwawayFacts = ConnectionFacts{}; + return _throwawayFacts; + } + return _facts[cid]; + } + /// @brief Body of `post`. On the server strand. /// @param cid Connection to write to. /// @param envelope The encoded envelope, within the size cap. @@ -1603,7 +1671,7 @@ class RemoteServer : public std::enable_shared_from_this { // the service decides on the verified principal, never on the // client's claim. stampVerifiedPrincipal(env); - reply(service->handle(env, *_authorizer)); + reply(service->handle(env, *_authorizer, factsOf(cid))); } else { reply(::morph::wire::encode(::morph::wire::makeErr("unknown envelope kind: " + env.kind, env.callId))); } @@ -1998,6 +2066,12 @@ class RemoteServer : public std::enable_shared_from_this { // The writer of each open connection that has one, set by // `setConnectionWriter` and dropped with the connection's scope. std::unordered_map _writers; + // What services recorded about each open connection (`factsOf`), dropped + // with the connection's scope. + std::unordered_map _facts; + // Handed to a service for a message with no open connection behind it; + // reset before each such message. + ConnectionFacts _throwawayFacts; /// Names one cancellable call: the connection it arrived on and the /// `callId` its client chose. struct CancelKey { diff --git a/include/morph/ui/bundle.hpp b/include/morph/ui/bundle.hpp index efb7c3dba..e6604299f 100644 --- a/include/morph/ui/bundle.hpp +++ b/include/morph/ui/bundle.hpp @@ -170,6 +170,8 @@ struct HelloRequest { std::map> hostCommands; /// @brief The host values the client has, by name, with their versions. std::map> hostValues; + /// @brief Whether the client offers to take server-sent `event` envelopes on this connection. + bool events = false; }; /// @brief The body of a `ui-hello` reply. @@ -182,6 +184,9 @@ struct HelloReply { std::vector vocabularies; /// @brief The manifest's content hash. std::string digest; + /// @brief Whether the server accepted the client's offer of events: it may send `event` envelopes on this + /// connection. + bool events = false; }; /// @brief The body of a `ui-evaluate` reply: what `ActionDispatcher::evaluate` answered. diff --git a/include/morph/ui/service.hpp b/include/morph/ui/service.hpp index e780b7ec4..83fe68496 100644 --- a/include/morph/ui/service.hpp +++ b/include/morph/ui/service.hpp @@ -711,14 +711,30 @@ class UiService final : public ::morph::backend::IEnvelopeService { (kind == kFileKind && _files != nullptr); } - /// @brief Answers one request. + /// @brief Answers one request with no connection behind it: a `ui-hello` offer of events is accepted in the + /// reply, and recorded nowhere. /// @param request The envelope, its principal verified. /// @param authorizer The server's authoriser. /// @return The encoded reply. [[nodiscard]] std::string handle(::morph::wire::Envelope const& request, ::morph::session::IAuthorizer const& authorizer) override { + ::morph::backend::ConnectionFacts facts; + return handle(request, authorizer, facts); + } + + /// @brief Answers one request; a `ui-hello` that offers events and is answered `ok` sets `facts.events`. + /// + /// A hello that does not offer them leaves the facts as they are, so a later hello on the same connection, such + /// as a development reload's poll, does not withdraw what the first negotiated. + /// @param request The envelope, its principal verified. + /// @param authorizer The server's authoriser. + /// @param facts The facts of the connection the request arrived on. + /// @return The encoded reply. + [[nodiscard]] std::string handle(::morph::wire::Envelope const& request, + ::morph::session::IAuthorizer const& authorizer, + ::morph::backend::ConnectionFacts& facts) override { if (request.kind == kHelloKind) { - return hello(request, authorizer); + return hello(request, authorizer, facts); } if (request.kind == kCatalogKind) { return catalog(request, authorizer); @@ -753,7 +769,8 @@ class UiService final : public ::morph::backend::IEnvelopeService { } [[nodiscard]] std::string hello(::morph::wire::Envelope const& request, - ::morph::session::IAuthorizer const& authorizer) const { + ::morph::session::IAuthorizer const& authorizer, + ::morph::backend::ConnectionFacts& facts) const { if (!gate(authorizer, request, kHelloAction)) { return err(request, "unauthorized"); } @@ -770,10 +787,14 @@ class UiService final : public ::morph::backend::IEnvelopeService { if (!missing.empty()) { return err(request, "missing vocabulary: " + missing); } + if (client->events) { + facts.events = true; + } return ok(request, detail::writeMessage(HelloReply{.application = _bundle->application.id, .version = _bundle->application.version, .vocabularies = _bundle->vocabularies, - .digest = _bundle->digest})); + .digest = _bundle->digest, + .events = client->events})); } /// @brief Evaluates the action body @p request carries (spec 5 §7): decoded, reconciled and recomputed, never diff --git a/tests/net/test_socket_backend.cpp b/tests/net/test_socket_backend.cpp index b624cee7f..d2294a064 100644 --- a/tests/net/test_socket_backend.cpp +++ b/tests/net/test_socket_backend.cpp @@ -2812,6 +2812,8 @@ namespace { /// Answers `probe-ok` with the kind it saw and `probe-err` with an `err`. class ProbeService final : public morph::backend::IEnvelopeService { public: + using IEnvelopeService::handle; + [[nodiscard]] bool serves(std::string_view kind) const override { return kind.starts_with("probe-"); } [[nodiscard]] std::string handle(morph::wire::Envelope const& request, diff --git a/tests/qt/test_qt_websocket.cpp b/tests/qt/test_qt_websocket.cpp index 3df49199c..5eba8e7cb 100644 --- a/tests/qt/test_qt_websocket.cpp +++ b/tests/qt/test_qt_websocket.cpp @@ -2890,6 +2890,8 @@ namespace { /// Answers `probe-ok` with the token the request carried and `probe-err` with an `err`. class ProbeService final : public morph::backend::IEnvelopeService { public: + using IEnvelopeService::handle; + [[nodiscard]] bool serves(std::string_view kind) const override { return kind.starts_with("probe-"); } [[nodiscard]] std::string handle(morph::wire::Envelope const& request, diff --git a/tests/test_push_catalog.cpp b/tests/test_push_catalog.cpp index ea069b354..dfb5638a9 100644 --- a/tests/test_push_catalog.cpp +++ b/tests/test_push_catalog.cpp @@ -1,15 +1,23 @@ #include #include #include +#include +#include #include #include +#include #include +#include +#include #include #include +#include #include #include #include +#include "ui_service_support.hpp" + // Glaze reflects member names through an external-linkage object, so these types cannot sit in an // anonymous namespace. namespace push_catalog_test { @@ -72,6 +80,45 @@ morph::ui::author::CatalogFacts registeredFacts() { return facts; } +/// A server answering the UI service's kinds over the fixture bundle, every principal allowed to say hello. +struct HelloRig { + morph::exec::ThreadPoolExecutor pool{2}; + std::shared_ptr server; + + HelloRig() { + morph::backend::ServerConfig config; + config.services.push_back(std::make_shared(uiservice::bundle())); + server = std::make_shared(pool, uiservice::authorizer(), config); + } + + /// Sends `ui-hello` with @p body on @p cid and reads the reply. + morph::ui::HelloReply hello(morph::backend::ConnectionId cid, std::string const& body) { + morph::wire::Envelope env; + env.kind = std::string{morph::ui::kHelloKind}; + env.callId = 1; + env.body = body; + morph::wire::Envelope const reply = morph::wire::decode(server->handleInline(morph::wire::encode(env), cid)); + REQUIRE(reply.kind == "ok"); + auto parsed = morph::ui::parseHelloReply(reply.body); + REQUIRE(parsed); + return *parsed; + } + + /// Whether @p cid negotiated events, read on the server strand. + bool negotiated(morph::backend::ConnectionId cid) { + bool answer = false; + std::latch done{1}; + server->strand().post([&] { + answer = server->eventsNegotiated(cid); + done.count_down(); + }); + done.wait(); + return answer; + } +}; + +constexpr char const* kSpeaksAll = R"("vocabularies":["ui/1","expr/1","ui-theme/1"])"; + } // namespace TEST_CASE("catalog_facts_carry_effects_and_records", "[push][catalog]") { @@ -128,3 +175,58 @@ TEST_CASE("old_reply_without_fields_reads_as_reads", "[push][catalog]") { CHECK_FALSE(info.effects.has_value()); CHECK(info.records.empty()); } + +TEST_CASE("hello_with_events_sets_connection_fact", "[push][catalog]") { + HelloRig rig; + auto const cid = rig.server->openConnection(); + auto const other = rig.server->openConnection(); + REQUIRE_FALSE(rig.negotiated(cid)); + + auto const reply = rig.hello(cid, std::string{"{"} + kSpeaksAll + R"(,"events":true})"); + CHECK(reply.events); + CHECK(rig.negotiated(cid)); + CHECK_FALSE(rig.negotiated(other)); + + SECTION("a later hello without the offer does not withdraw it") { + CHECK_FALSE(rig.hello(cid, std::string{"{"} + kSpeaksAll + "}").events); + CHECK(rig.negotiated(cid)); + } + SECTION("a reconnect negotiates again") { + rig.server->closeConnection(cid); + CHECK_FALSE(rig.negotiated(cid)); + } + SECTION("the service sets the fact it is handed") { + morph::ui::UiService service{uiservice::bundle()}; + morph::backend::ConnectionFacts facts; + morph::wire::Envelope env; + env.kind = std::string{morph::ui::kHelloKind}; + env.body = std::string{"{"} + kSpeaksAll + R"(,"events":true})"; + static_cast(service.handle(env, *uiservice::authorizer(), facts)); + CHECK(facts.events); + } +} + +TEST_CASE("hello_without_events_leaves_it_false", "[push][catalog]") { + HelloRig rig; + auto const cid = rig.server->openConnection(); + + SECTION("no offer") { CHECK_FALSE(rig.hello(cid, std::string{"{"} + kSpeaksAll + "}").events); } + SECTION("an explicit refusal of the offer") { + CHECK_FALSE(rig.hello(cid, std::string{"{"} + kSpeaksAll + R"(,"events":false})").events); + } + CHECK_FALSE(rig.negotiated(cid)); +} + +TEST_CASE("hello_events_round_trip_on_the_wire", "[push][catalog]") { + auto const request = morph::ui::parseHelloRequest(R"({"vocabularies":["ui/1"],"events":true})"); + REQUIRE(request); + CHECK(request->events); + auto const old = morph::ui::parseHelloRequest(R"({"vocabularies":["ui/1"]})"); + REQUIRE(old); + CHECK_FALSE(old->events); + + std::string const digest = "sha256:" + std::string(64, '0'); + auto const oldReply = morph::ui::parseHelloReply(R"({"application":"a","digest":")" + digest + R"("})"); + REQUIRE(oldReply); + CHECK_FALSE(oldReply->events); +} diff --git a/tests/test_remote_envelope_service.cpp b/tests/test_remote_envelope_service.cpp index 47181cbd7..f6e39e6f8 100644 --- a/tests/test_remote_envelope_service.cpp +++ b/tests/test_remote_envelope_service.cpp @@ -6,6 +6,7 @@ // nobody serves is still `unknown envelope kind`. #include +#include #include #include #include @@ -25,6 +26,8 @@ using morph::wire::Envelope; /// Answers `echo-*` kinds with the principal and kind it saw; throws on `echo-throw`. class EchoService final : public morph::backend::IEnvelopeService { public: + using IEnvelopeService::handle; + [[nodiscard]] bool serves(std::string_view kind) const override { return kind.starts_with("echo-") || kind == "execute" || kind == "hello"; } @@ -60,6 +63,41 @@ struct TokenAuthorizer final : morph::session::IAuthorizer { } }; +/// Records whether each envelope arrived with `events` already negotiated, then +/// negotiates it: what a service that learns a connection fact looks like. +class FactsService final : public morph::backend::IEnvelopeService { +public: + using IEnvelopeService::handle; + + [[nodiscard]] bool serves(std::string_view kind) const override { return kind == "facts-hello"; } + + [[nodiscard]] std::string handle(Envelope const& request, + morph::session::IAuthorizer const& /*authorizer*/) override { + return encode(morph::wire::makeOk(request.callId)); + } + + [[nodiscard]] std::string handle(Envelope const& request, morph::session::IAuthorizer const& authorizer, + morph::backend::ConnectionFacts& facts) override { + seen.push_back(facts.events); + facts.events = true; + return handle(request, authorizer); + } + + std::vector seen; +}; + +/// Whether @p cid negotiated events, read on the server strand. +bool negotiated(morph::backend::RemoteServer& server, morph::backend::ConnectionId cid) { + bool answer = false; + std::latch done{1}; + server.strand().post([&] { + answer = server.eventsNegotiated(cid); + done.count_down(); + }); + done.wait(); + return answer; +} + Envelope request(std::string kind, std::uint64_t callId) { Envelope env; env.kind = std::move(kind); @@ -138,3 +176,58 @@ TEST_CASE("RemoteServer: a null service is refused at construction", "[remote][s config.services.push_back(nullptr); REQUIRE_THROWS_AS(std::make_shared(pool, config), std::invalid_argument); } + +TEST_CASE("unscoped_message_gets_throwaway_facts", "[push][catalog]") { + morph::exec::ThreadPoolExecutor pool{2}; + auto service = std::make_shared(); + morph::backend::ServerConfig config; + config.services.push_back(service); + auto server = std::make_shared(pool, config); + + REQUIRE(decode(server->handleInline(encode(request("facts-hello", 1)))).kind == "ok"); + REQUIRE(decode(server->handleInline(encode(request("facts-hello", 2)))).kind == "ok"); + // Each unscoped message starts from fresh facts: what the first one set is not kept. + CHECK(service->seen == std::vector{false, false}); + CHECK_FALSE(negotiated(*server, 0)); + + SECTION("a connection keeps its facts until it closes, and only its own") { + auto const first = server->openConnection(); + auto const second = server->openConnection(); + static_cast(server->handleInline(encode(request("facts-hello", 3)), first)); + static_cast(server->handleInline(encode(request("facts-hello", 4)), first)); + static_cast(server->handleInline(encode(request("facts-hello", 5)), second)); + CHECK(service->seen == std::vector{false, false, false, true, false}); + CHECK(negotiated(*server, first)); + CHECK(negotiated(*server, second)); + + server->closeConnection(first); + CHECK_FALSE(negotiated(*server, first)); + CHECK(negotiated(*server, second)); + } + SECTION("an unknown connection has negotiated nothing") { CHECK_FALSE(negotiated(*server, 4242)); } +} + +TEST_CASE("two_param_service_still_compiles_and_serves", "[push][catalog]") { + morph::exec::ThreadPoolExecutor pool{2}; + auto service = std::make_shared(); + morph::backend::ServerConfig config; + config.services.push_back(service); + auto server = std::make_shared(pool, std::make_shared(), config); + + // Through a connection, the server calls the three-parameter handle; its default reaches the service's own. + auto const cid = server->openConnection(); + Envelope env = request("echo-ping", 7); + env.session.token = "alice"; + Envelope const reply = decode(server->handleInline(encode(env), cid)); + REQUIRE(reply.kind == "ok"); + CHECK(reply.body == "verified:alice|echo-ping"); + CHECK(service->seen == std::vector{"echo-ping"}); + CHECK_FALSE(negotiated(*server, cid)); + + // Called directly, the default forwards and leaves the facts alone. + morph::backend::ConnectionFacts facts; + morph::backend::IEnvelopeService& base = *service; + Envelope const direct = decode(base.handle(request("echo-pong", 8), TokenAuthorizer{}, facts)); + CHECK(direct.body == "|echo-pong"); + CHECK_FALSE(facts.events); +} From 0a8439bbf0bd767eea13d32cca26d08c8903b2b1 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 16:38:23 +0200 Subject: [PATCH 27/68] core: the watch, unwatch and changes bodies Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/wire.md | 30 +++ .../specs/2026-10-07-ui-document-design.md | 4 +- .../specs/2026-10-08-app-hosting-design.md | 2 +- .../specs/2026-10-16-server-push-design.md | 5 +- include/morph/core/remote.hpp | 5 +- include/morph/core/wire.hpp | 188 ++++++++++++++++++ tests/test_push_catalog.cpp | 99 +++++++++ 7 files changed, 326 insertions(+), 7 deletions(-) diff --git a/docs/spec/core/wire.md b/docs/spec/core/wire.md index c27c11337..b3d303ec3 100644 --- a/docs/spec/core/wire.md +++ b/docs/spec/core/wire.md @@ -704,6 +704,33 @@ order is promised between an event and any reply. posts one to the installed handler, so the client's event path can be tested without a transport. +## The watch, unwatch and changes bodies + +A connection that negotiated events chooses what it hears about with two core kinds, `"watch"` and +`"unwatch"` (`kWatchKind`, `kUnwatchKind`). Both carry a `WatchBody`: +`{"records": […], "instances": [{"modelType", "primary", "instance"?}…], "replace"?: bool}`. +`replace` is meaningful on `watch` only: the connection's whole watch set becomes the body instead of +growing by it. An `InstanceTopic` names an instance by type and directory key text (`primary`, empty for a +private instance); `instance` is the id and is 0 in a request that names the topic by type and primary. + +The `ok` reply to a `watch` carries a `WatchReply`, `{"refusedRecords": […], "refusedInstances": […]}`, +naming the topics the server does not publish or the caller may not read; both lists are empty when +nothing was refused. + +The payload of a `"changes"` event (`kChangesTopic`) is a `ChangesBody`: +`{"records": [{"record", "keys"?}…], "instances": [{"modelType", "primary", "instance"}…]}`. A record entry +without `keys` means the whole record type changed; `records` reuses `effects::RecordChange` from +`core/change_types.hpp`, which this header includes alone of the effects layer. A `"resync"` event +(`kResyncTopic`) tells the client events were withheld and it must re-read what it shows. + +**Readers.** `readWatchBody`, `readWatchReply` and `readChangesBody` return +`std::expected<…, std::string>`. They refuse input longer than `kMaxEnvelopeBytes`, text that is not the +body, more than `kMaxWatchTopics` (4096) records or instances in one list, more than `kMaxChangeKeys` +(4096) keys in one record entry, and any record name, model type or primary longer than +`kMaxTopicNameBytes` (1024). Unknown keys are ignored, so a newer peer may add fields. The list bounds sit +well above the coalescer's 256-key flush and any real watch set, so only a malformed or hostile peer meets +them. + ## Action-evolution policy The passive forward-compat contract (`error_on_unknown_keys = false` on @@ -865,6 +892,9 @@ client. | `makeHello` | `Envelope makeHello(uint32_t protocolVersion = kProtocolVersion)` | | `makeCancel` | `Envelope makeCancel(uint64_t targetCallId)` | | `makeEvent` | `Envelope makeEvent(std::string_view topic, std::string payloadJson)` | +| `readWatchBody` | `std::expected readWatchBody(std::string_view)` | +| `readWatchReply` | `std::expected readWatchReply(std::string_view)` | +| `readChangesBody` | `std::expected readChangesBody(std::string_view)` | | `makeOk` | `Envelope makeOk(uint64_t callId = 0, std::string body = {}, uint64_t modelId = 0)` | | `makeErr` | `Envelope makeErr(std::string message, uint64_t callId = 0)` | diff --git a/docs/superpowers/specs/2026-10-07-ui-document-design.md b/docs/superpowers/specs/2026-10-07-ui-document-design.md index 38316fe4a..fb225903f 100644 --- a/docs/superpowers/specs/2026-10-07-ui-document-design.md +++ b/docs/superpowers/specs/2026-10-07-ui-document-design.md @@ -884,7 +884,7 @@ places, none of which changes the document: children a document declares is the `arrange` point. - **Server push.** A document learns of a change by its own calls, by `refreshOn` over its own dispatch and by polling with `refreshEvery`. A server-initiated message (pushed progress, another - client's change) needs a wire kind the protocol does not have; it is a candidate for `ui/2`, as a - `ui-event` kind negotiated by `ui-hello`. + client's change) needs a wire kind the protocol does not have; it is carried by the core + `event` kind, which `ui-hello` negotiates per connection. - **Server-side composite actions.** Committing several models in one transaction is not part of this design; a dialog that must commit atomically runs one server action (§5). diff --git a/docs/superpowers/specs/2026-10-08-app-hosting-design.md b/docs/superpowers/specs/2026-10-08-app-hosting-design.md index 9629ca146..6f47822c2 100644 --- a/docs/superpowers/specs/2026-10-08-app-hosting-design.md +++ b/docs/superpowers/specs/2026-10-08-app-hosting-design.md @@ -815,7 +815,7 @@ A screen can open another and receive an answer: document polls it with `refreshEvery` and `when: state is running`, and shows `progress` and `steps`. A `watch` on `state` runs the follow-up when it becomes `done` or `failed`. - Pushing progress needs a server-initiated message the wire does not have; it is not part of - `ui/1`. A later vocabulary version may add a `ui-event` kind negotiated by `ui-hello`. + `ui/1`. Server push is the core `event` kind; a `ui-hello` negotiates it per connection. ## 19. Settings and messages diff --git a/docs/superpowers/specs/2026-10-16-server-push-design.md b/docs/superpowers/specs/2026-10-16-server-push-design.md index dca21a5d5..c3d890991 100644 --- a/docs/superpowers/specs/2026-10-16-server-push-design.md +++ b/docs/superpowers/specs/2026-10-16-server-push-design.md @@ -262,8 +262,9 @@ The wire gains its first server-initiated message. - A message handled without a connection (`handle(msg, reply)`, `cid` 0) gets throwaway facts with `events = false`. - `IEnvelopeService` keeps its two-parameter `handle` pure. The three-parameter one is a non-pure virtual whose - default calls it. A service that overrides the new one adds `using IEnvelopeService::handle;`, as the - `-Woverloaded-virtual` build requires. + default calls it. A service that overrides only one of the two + overloads adds `using IEnvelopeService::handle;`, as the `-Woverloaded-virtual` build requires; one that + overrides both needs none. - **The envelope.** `kind: "event"`, `callId: 0`, `body: {topic, payload}`. For this design the topic is `changes`. A client routes it by kind before reply matching, so an event is never taken for an orphaned reply. - **Writing to a connection.** `RemoteServer` gains `setConnectionWriter(cid, writer)`, which takes the transport's diff --git a/include/morph/core/remote.hpp b/include/morph/core/remote.hpp index 950b812cd..fcabe6361 100644 --- a/include/morph/core/remote.hpp +++ b/include/morph/core/remote.hpp @@ -684,7 +684,7 @@ class RemoteServer : public std::enable_shared_from_this { /// /// What a transport calls when it has written enough of what it holds. If /// `post` held an event back since the last drain, the connection is sent - /// one `resync` event (`wire::makeEvent("resync", "{}")`), meaning + /// one `resync` event (`wire::makeEvent(wire::kResyncTopic, "{}")`), meaning /// everything it watches may have changed, and events flow again. /// Otherwise nothing is sent. The drain, not a later write, sends it, so a /// connection that only listens still receives it. @@ -963,7 +963,8 @@ class RemoteServer : public std::enable_shared_from_this { return; } found->second.owesResync = false; - writeTo(cid, found->second, ::morph::wire::encode(::morph::wire::makeEvent("resync", "{}"))); + writeTo(cid, found->second, + ::morph::wire::encode(::morph::wire::makeEvent(::morph::wire::kResyncTopic, "{}"))); } /// @brief Hands @p envelope to @p outbox's writer, catching what it diff --git a/include/morph/core/wire.hpp b/include/morph/core/wire.hpp index 81c71a295..adfcbab96 100644 --- a/include/morph/core/wire.hpp +++ b/include/morph/core/wire.hpp @@ -7,7 +7,9 @@ #include #include #include +#include #include +#include #include #include #include @@ -15,6 +17,7 @@ #include #include "../session/session.hpp" +#include "change_types.hpp" #include "profiler.hpp" namespace morph::wire { @@ -640,6 +643,191 @@ struct EventBody { return env; } +/// @brief The `changes` event topic: a `ChangesBody` payload naming what changed. +inline constexpr std::string_view kChangesTopic = "changes"; + +/// @brief The `resync` event topic: events were withheld, so a client re-reads what it shows. +inline constexpr std::string_view kResyncTopic = "resync"; + +/// @brief The envelope kind that adds topics to a connection's watch set (or replaces it). +inline constexpr std::string_view kWatchKind = "watch"; + +/// @brief The envelope kind that removes topics from a connection's watch set. +inline constexpr std::string_view kUnwatchKind = "unwatch"; + +/// @brief The most topics (records or instances) one `watch`, `unwatch` or `changes` body may list. +/// +/// A watch set is one entry per record type and open instance of a screen, and the coalescer flushes +/// well below this, so a body past it is a malformed or hostile peer rather than a large application. +inline constexpr std::size_t kMaxWatchTopics = 4096; + +/// @brief The most keys one record entry of a `changes` body may list. +/// +/// The server flushes a coalescing window at 256 keys; 4096 leaves room for several windows merged by +/// one writer without letting a peer force an unbounded allocation. +inline constexpr std::size_t kMaxChangeKeys = 4096; + +/// @brief The longest record name, model type or primary key text in a watch or changes body. +inline constexpr std::size_t kMaxTopicNameBytes = 1024; + +/// @brief One instance topic: a model type, its directory key text and its instance id. +struct InstanceTopic { + /// @brief The model's registered type name. + std::string modelType; + /// @brief The directory key's text for a keyed shared instance; empty for a private one. + std::string primary; + /// @brief The instance id; 0 where the topic is named by type and primary alone (a request). + std::uint64_t instance = 0; +}; + +/// @brief The body of a `watch` or `unwatch` request. +struct WatchBody { + /// @brief Record wire names to watch (or stop watching). + std::vector records; + /// @brief Instance topics to watch (or stop watching). + std::vector instances; + /// @brief For `watch`: replace the connection's whole watch set instead of adding to it. + bool replace = false; +}; + +/// @brief The body of the `ok` reply to a `watch`: the topics the server refused. +struct WatchReply { + /// @brief Record names the server does not publish or the caller may not read. + std::vector refusedRecords; + /// @brief Instance topics the server refused. + std::vector refusedInstances; +}; + +/// @brief The payload of a `changes` event. +struct ChangesBody { + /// @brief Record changes; a record with no `keys` changed as a whole. + std::vector records; + /// @brief Instance changes. + std::vector instances; +}; + +namespace detail { + +template +[[nodiscard]] inline std::expected readBody(std::string_view json, std::string_view what) { + if (json.size() > kMaxEnvelopeBytes) { + return std::unexpected(std::string{what} + " exceeds " + std::to_string(kMaxEnvelopeBytes) + " bytes"); + } + T out{}; + static constexpr glz::opts kLenient{.null_terminated = false, .error_on_unknown_keys = false}; + if (auto ec = glz::read(out, json)) { + return std::unexpected(std::string{what} + ": " + glz::format_error(ec, json)); + } + return out; +} + +[[nodiscard]] inline std::optional checkTopicName(std::string_view name, std::string_view what) { + if (name.size() > kMaxTopicNameBytes) { + return std::string{what} + " is longer than " + std::to_string(kMaxTopicNameBytes) + " bytes"; + } + return std::nullopt; +} + +[[nodiscard]] inline std::optional checkNames(const std::vector& names, + std::string_view what) { + if (names.size() > kMaxWatchTopics) { + return std::string{what} + " list more than " + std::to_string(kMaxWatchTopics) + " topics"; + } + for (const auto& name : names) { + if (auto reason = checkTopicName(name, what)) { + return reason; + } + } + return std::nullopt; +} + +[[nodiscard]] inline std::optional checkInstances(const std::vector& topics, + std::string_view what) { + if (topics.size() > kMaxWatchTopics) { + return std::string{what} + " list more than " + std::to_string(kMaxWatchTopics) + " topics"; + } + for (const auto& topic : topics) { + if (auto reason = checkTopicName(topic.modelType, what)) { + return reason; + } + if (auto reason = checkTopicName(topic.primary, what)) { + return reason; + } + } + return std::nullopt; +} + +} // namespace detail + +/// @brief Reads the body of a `watch` or `unwatch` request. +/// +/// Refused: input longer than `kMaxEnvelopeBytes`, text that is not a `WatchBody`, more than +/// `kMaxWatchTopics` records or instances, or a name longer than `kMaxTopicNameBytes`. Unknown keys are +/// ignored. +/// @param json The envelope `body`. +/// @return The body, or the reason it was refused. +[[nodiscard]] inline std::expected readWatchBody(std::string_view json) { + auto body = detail::readBody(json, "the watch body"); + if (!body) { + return body; + } + if (auto reason = detail::checkNames(body->records, "a record name")) { + return std::unexpected(*reason); + } + if (auto reason = detail::checkInstances(body->instances, "an instance topic")) { + return std::unexpected(*reason); + } + return body; +} + +/// @brief Reads the body of the `ok` reply to a `watch`. +/// +/// Bounded as `readWatchBody` is. +/// @param json The envelope `body`. +/// @return The reply, or the reason it was refused. +[[nodiscard]] inline std::expected readWatchReply(std::string_view json) { + auto body = detail::readBody(json, "the watch reply"); + if (!body) { + return body; + } + if (auto reason = detail::checkNames(body->refusedRecords, "a record name")) { + return std::unexpected(*reason); + } + if (auto reason = detail::checkInstances(body->refusedInstances, "an instance topic")) { + return std::unexpected(*reason); + } + return body; +} + +/// @brief Reads the payload of a `changes` event. +/// +/// Refused: input longer than `kMaxEnvelopeBytes`, text that is not a `ChangesBody`, more than +/// `kMaxWatchTopics` records or instances, more than `kMaxChangeKeys` keys in one record, or a name longer +/// than `kMaxTopicNameBytes`. +/// @param json The event's `payload`. +/// @return The payload, or the reason it was refused. +[[nodiscard]] inline std::expected readChangesBody(std::string_view json) { + auto body = detail::readBody(json, "the changes payload"); + if (!body) { + return body; + } + if (body->records.size() > kMaxWatchTopics) { + return std::unexpected("a record list more than " + std::to_string(kMaxWatchTopics) + " topics"); + } + for (const auto& change : body->records) { + if (auto reason = detail::checkTopicName(change.record, "a record name")) { + return std::unexpected(*reason); + } + if (change.keys && change.keys->size() > kMaxChangeKeys) { + return std::unexpected("a record's keys list more than " + std::to_string(kMaxChangeKeys) + " keys"); + } + } + if (auto reason = detail::checkInstances(body->instances, "an instance topic")) { + return std::unexpected(*reason); + } + return body; +} + /// @brief The `err` reply message `RemoteServer` sends when /// `LimitPolicy::executeTimeout` fires server-side (see /// `RemoteServer::dispatchExecute`'s `_executeTimeouts` path). diff --git a/tests/test_push_catalog.cpp b/tests/test_push_catalog.cpp index dfb5638a9..038ff3378 100644 --- a/tests/test_push_catalog.cpp +++ b/tests/test_push_catalog.cpp @@ -230,3 +230,102 @@ TEST_CASE("hello_events_round_trip_on_the_wire", "[push][catalog]") { REQUIRE(oldReply); CHECK_FALSE(oldReply->events); } + +TEST_CASE("watch_body_round_trips_and_is_bounded", "[push][catalog]") { + morph::wire::WatchBody body; + body.records = {"Entry", "Tag"}; + body.instances = {{"Editor", "k1", 0}, {"Editor", "", 7}}; + body.replace = true; + auto const text = glz::write_json(body).value_or(""); + auto const back = morph::wire::readWatchBody(text); + REQUIRE(back); + CHECK(back->records == body.records); + REQUIRE(back->instances.size() == 2); + CHECK(back->instances[0].primary == "k1"); + CHECK(back->instances[1].instance == 7); + CHECK(back->replace); + + SECTION("absent fields default; unknown keys are ignored") { + auto const bare = morph::wire::readWatchBody(R"({"records":["A"],"future":1})"); + REQUIRE(bare); + CHECK_FALSE(bare->replace); + CHECK(bare->instances.empty()); + } + SECTION("an over-long record list is refused") { + morph::wire::WatchBody big; + big.records.assign(morph::wire::kMaxWatchTopics + 1, "R"); + CHECK_FALSE(morph::wire::readWatchBody(glz::write_json(big).value_or(""))); + big.records.pop_back(); + CHECK(morph::wire::readWatchBody(glz::write_json(big).value_or(""))); + } + SECTION("an over-long instance list or name is refused") { + morph::wire::WatchBody big; + big.instances.assign(morph::wire::kMaxWatchTopics + 1, {"M", "", 0}); + CHECK_FALSE(morph::wire::readWatchBody(glz::write_json(big).value_or(""))); + morph::wire::WatchBody longName; + longName.records = {std::string(morph::wire::kMaxTopicNameBytes + 1, 'x')}; + CHECK_FALSE(morph::wire::readWatchBody(glz::write_json(longName).value_or(""))); + } + SECTION("garbage and oversized input are refused") { + CHECK_FALSE(morph::wire::readWatchBody("not json")); + CHECK_FALSE(morph::wire::readWatchBody(std::string(morph::wire::kMaxEnvelopeBytes + 1, ' '))); + } +} + +TEST_CASE("watch_reply_round_trips", "[push][catalog]") { + morph::wire::WatchReply reply; + reply.refusedRecords = {"Secret"}; + reply.refusedInstances = {{"Editor", "p", 3}}; + auto const back = morph::wire::readWatchReply(glz::write_json(reply).value_or("")); + REQUIRE(back); + CHECK(back->refusedRecords == reply.refusedRecords); + REQUIRE(back->refusedInstances.size() == 1); + CHECK(back->refusedInstances[0].instance == 3); + CHECK(morph::wire::readWatchReply("{}")); + + morph::wire::WatchReply big; + big.refusedRecords.assign(morph::wire::kMaxWatchTopics + 1, "R"); + CHECK_FALSE(morph::wire::readWatchReply(glz::write_json(big).value_or(""))); +} + +TEST_CASE("changes_body_round_trips_and_is_bounded", "[push][catalog]") { + auto const text = R"({"records":[{"record":"Entry","keys":[1,"b"]},{"record":"Tag"}],)" + R"("instances":[{"modelType":"Editor","primary":"p","instance":9}]})"; + auto const back = morph::wire::readChangesBody(text); + REQUIRE(back); + REQUIRE(back->records.size() == 2); + REQUIRE(back->records[0].keys); + CHECK(back->records[0].keys->size() == 2); + CHECK_FALSE(back->records[1].keys); + REQUIRE(back->instances.size() == 1); + CHECK(back->instances[0].instance == 9); + + SECTION("it is the payload makeEvent carries") { + auto const env = morph::wire::makeEvent(morph::wire::kChangesTopic, text); + morph::wire::EventBody event; + REQUIRE_FALSE(glz::read_json(event, env.body)); + CHECK(event.topic == "changes"); + CHECK(morph::wire::readChangesBody(event.payload.str)); + } + SECTION("too many keys in one record is refused") { + std::string keys = "["; + for (std::size_t i = 0; i <= morph::wire::kMaxChangeKeys; ++i) { + keys += (i ? ",1" : "1"); + } + keys += "]"; + CHECK_FALSE(morph::wire::readChangesBody(R"({"records":[{"record":"E","keys":)" + keys + "}]}")); + } + SECTION("too many records is refused") { + std::string records = "["; + for (std::size_t i = 0; i <= morph::wire::kMaxWatchTopics; ++i) { + records += (i ? R"(,{"record":"R"})" : R"({"record":"R"})"); + } + records += "]"; + CHECK_FALSE(morph::wire::readChangesBody(R"({"records":)" + records + "}")); + } + SECTION("the kind and topic names are the wire names") { + CHECK(morph::wire::kWatchKind == "watch"); + CHECK(morph::wire::kUnwatchKind == "unwatch"); + CHECK(morph::wire::kResyncTopic == "resync"); + } +} From ee9fd2e039a169a0b41716e66fe1176c040ddd1b Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 16:50:08 +0200 Subject: [PATCH 28/68] core: a change never carries more keys than a client accepts Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/wire.md | 11 ++++++----- include/morph/core/change_types.hpp | 8 ++++++++ include/morph/core/changes.hpp | 6 ++++++ include/morph/core/wire.hpp | 24 ++++++++++-------------- tests/test_push_catalog.cpp | 12 +++++++++++- tests/test_push_effects.cpp | 22 ++++++++++++++++++++++ 6 files changed, 63 insertions(+), 20 deletions(-) diff --git a/docs/spec/core/wire.md b/docs/spec/core/wire.md index b3d303ec3..0427a8e8f 100644 --- a/docs/spec/core/wire.md +++ b/docs/spec/core/wire.md @@ -725,11 +725,12 @@ without `keys` means the whole record type changed; `records` reuses `effects::R **Readers.** `readWatchBody`, `readWatchReply` and `readChangesBody` return `std::expected<…, std::string>`. They refuse input longer than `kMaxEnvelopeBytes`, text that is not the -body, more than `kMaxWatchTopics` (4096) records or instances in one list, more than `kMaxChangeKeys` -(4096) keys in one record entry, and any record name, model type or primary longer than -`kMaxTopicNameBytes` (1024). Unknown keys are ignored, so a newer peer may add fields. The list bounds sit -well above the coalescer's 256-key flush and any real watch set, so only a malformed or hostile peer meets -them. +body, more than `kMaxWatchTopics` (4096) records or instances in one list, more than `effects::kMaxChangeKeys` +(4096, `core/change_types.hpp`) keys in one record entry, and any record name, model type or primary longer +than `kMaxTopicNameBytes` (1024); the message says which one. Unknown keys are ignored, so a newer peer may +add fields. The bounds keep a peer-supplied list from forcing an unbounded allocation and sit above any +real watch set. The producer honours the key bound too: `deriveChanges` reports a record whose keys would +exceed it as a whole-type change (keys absent), so a change is never refused for its size. ## Action-evolution policy diff --git a/include/morph/core/change_types.hpp b/include/morph/core/change_types.hpp index 9ff1d3ba9..61632b2bd 100644 --- a/include/morph/core/change_types.hpp +++ b/include/morph/core/change_types.hpp @@ -1,6 +1,7 @@ // SPDX-License-Identifier: Apache-2.0 #pragma once +#include #include #include #include @@ -13,6 +14,13 @@ namespace morph::effects { +/// @brief The most keys one `RecordChange` carries. +/// +/// The wire readers refuse a record entry with more keys, so a producer never emits one: a change that +/// would name more rows is reported as a whole-type change (keys absent), which over-notifies but never +/// misses. The bound keeps one peer-supplied list from forcing an unbounded allocation. +inline constexpr std::size_t kMaxChangeKeys = 4096; + /// @brief A change to the rows of one record type. struct RecordChange { /// The record's wire name. diff --git a/include/morph/core/changes.hpp b/include/morph/core/changes.hpp index 28f3445a0..4d11f24c2 100644 --- a/include/morph/core/changes.hpp +++ b/include/morph/core/changes.hpp @@ -214,6 +214,12 @@ inline void addDerivedChanges(Resolved const& resolved, std::string const& sourc break; } } + // A record naming more rows than a reader accepts is reported whole: it over-notifies, never misses. + for (auto& change : changes.records) { + if (change.keys && change.keys->size() > kMaxChangeKeys) { + change.keys.reset(); + } + } return changes; } diff --git a/include/morph/core/wire.hpp b/include/morph/core/wire.hpp index adfcbab96..f257cea73 100644 --- a/include/morph/core/wire.hpp +++ b/include/morph/core/wire.hpp @@ -6,8 +6,8 @@ #include #include #include -#include #include +#include #include #include #include @@ -657,17 +657,13 @@ inline constexpr std::string_view kUnwatchKind = "unwatch"; /// @brief The most topics (records or instances) one `watch`, `unwatch` or `changes` body may list. /// -/// A watch set is one entry per record type and open instance of a screen, and the coalescer flushes -/// well below this, so a body past it is a malformed or hostile peer rather than a large application. +/// A watch set is one entry per record type and open instance of a screen, so a list past this is a +/// malformed or hostile peer rather than a large application, and the bound keeps its allocation finite. inline constexpr std::size_t kMaxWatchTopics = 4096; -/// @brief The most keys one record entry of a `changes` body may list. -/// -/// The server flushes a coalescing window at 256 keys; 4096 leaves room for several windows merged by -/// one writer without letting a peer force an unbounded allocation. -inline constexpr std::size_t kMaxChangeKeys = 4096; - /// @brief The longest record name, model type or primary key text in a watch or changes body. +/// +/// The most keys one record entry of a `changes` body may list is `effects::kMaxChangeKeys`. inline constexpr std::size_t kMaxTopicNameBytes = 1024; /// @brief One instance topic: a model type, its directory key text and its instance id. @@ -747,10 +743,10 @@ template return std::string{what} + " list more than " + std::to_string(kMaxWatchTopics) + " topics"; } for (const auto& topic : topics) { - if (auto reason = checkTopicName(topic.modelType, what)) { + if (auto reason = checkTopicName(topic.modelType, "a model type")) { return reason; } - if (auto reason = checkTopicName(topic.primary, what)) { + if (auto reason = checkTopicName(topic.primary, "a primary key")) { return reason; } } @@ -802,7 +798,7 @@ template /// @brief Reads the payload of a `changes` event. /// /// Refused: input longer than `kMaxEnvelopeBytes`, text that is not a `ChangesBody`, more than -/// `kMaxWatchTopics` records or instances, more than `kMaxChangeKeys` keys in one record, or a name longer +/// `kMaxWatchTopics` records or instances, more than `effects::kMaxChangeKeys` keys in one record, or a name longer /// than `kMaxTopicNameBytes`. /// @param json The event's `payload`. /// @return The payload, or the reason it was refused. @@ -818,8 +814,8 @@ template if (auto reason = detail::checkTopicName(change.record, "a record name")) { return std::unexpected(*reason); } - if (change.keys && change.keys->size() > kMaxChangeKeys) { - return std::unexpected("a record's keys list more than " + std::to_string(kMaxChangeKeys) + " keys"); + if (change.keys && change.keys->size() > effects::kMaxChangeKeys) { + return std::unexpected("a record's keys list more than " + std::to_string(effects::kMaxChangeKeys) + " keys"); } } if (auto reason = detail::checkInstances(body->instances, "an instance topic")) { diff --git a/tests/test_push_catalog.cpp b/tests/test_push_catalog.cpp index 038ff3378..8db5674b8 100644 --- a/tests/test_push_catalog.cpp +++ b/tests/test_push_catalog.cpp @@ -307,9 +307,19 @@ TEST_CASE("changes_body_round_trips_and_is_bounded", "[push][catalog]") { CHECK(event.topic == "changes"); CHECK(morph::wire::readChangesBody(event.payload.str)); } + SECTION("exactly the key bound is accepted") { + std::string keys = "["; + for (std::size_t i = 0; i < morph::effects::kMaxChangeKeys; ++i) { + keys += (i ? ",1" : "1"); + } + keys += "]"; + auto const at = morph::wire::readChangesBody(R"({"records":[{"record":"E","keys":)" + keys + "}]}"); + REQUIRE(at); + CHECK(at->records[0].keys->size() == morph::effects::kMaxChangeKeys); + } SECTION("too many keys in one record is refused") { std::string keys = "["; - for (std::size_t i = 0; i <= morph::wire::kMaxChangeKeys; ++i) { + for (std::size_t i = 0; i <= morph::effects::kMaxChangeKeys; ++i) { keys += (i ? ",1" : "1"); } keys += "]"; diff --git a/tests/test_push_effects.cpp b/tests/test_push_effects.cpp index 799cd59f9..4e8d6dfba 100644 --- a/tests/test_push_effects.cpp +++ b/tests/test_push_effects.cpp @@ -399,6 +399,28 @@ TEST_CASE("derive_updates_reads_keys_at_paths", "[push][records]") { REQUIRE(keyTexts(changes.records[0]) == std::vector{"1", "2"}); } +TEST_CASE("derive_updates_collapses_past_the_key_bound", "[push][records]") { + auto const resolved = derivation(declare("A_Rows", {updatesOf(typeid(VRow))})); + auto const rowsJson = [](std::size_t count) { + std::string json = R"({"rows":[)"; + for (std::size_t i = 0; i < count; ++i) { + json += (i == 0 ? "" : ",") + std::string{R"({"id":)"} + std::to_string(i + 1) + "}"; + } + return json + "]}"; + }; + auto const atBound = morph::effects::detail::deriveChangesFrom( + resolved, "A_Rows", rowsJson(morph::effects::kMaxChangeKeys), ranOnInstance); + REQUIRE(atBound.records.size() == 1); + REQUIRE(atBound.records[0].keys.has_value()); + CHECK(atBound.records[0].keys->size() == morph::effects::kMaxChangeKeys); + + auto const pastBound = morph::effects::detail::deriveChangesFrom( + resolved, "A_Rows", rowsJson(morph::effects::kMaxChangeKeys + 1), ranOnInstance); + REQUIRE(pastBound.records.size() == 1); + CHECK(pastBound.records[0].record == "D_Row"); + CHECK_FALSE(pastBound.records[0].keys.has_value()); +} + TEST_CASE("derive_updates_reads_every_result_shape", "[push][records]") { auto const rootResolved = derivation(declare("A_Root", {updatesOf(typeid(VRow))})); auto const one = morph::effects::detail::deriveChangesFrom(rootResolved, "A_Root", R"({"id":9007199254740993})", From 4ee5617cb39ea66445b6f21b271b3156dda40417 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 17:12:31 +0200 Subject: [PATCH 29/68] core: effects.hpp names no identifier after a Qt keyword `slots` is a macro wherever Qt keywords are enabled, so a parameter of that name broke every Qt translation unit that includes effects.hpp. Co-Authored-By: Claude Opus 5.5 --- include/morph/core/effects.hpp | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/include/morph/core/effects.hpp b/include/morph/core/effects.hpp index 349c35bee..2f4a82e10 100644 --- a/include/morph/core/effects.hpp +++ b/include/morph/core/effects.hpp @@ -342,13 +342,13 @@ inline void findDerivationCycles(std::vector const& records, /// @brief Matches action results against records and validates the configuration. /// @param records The records, as registered. -/// @param slots Every action's result slots, by action id. +/// @param resultSlots Every action's result slots, by action id. /// @param declared Every action's effects declarations, by action id. /// @param slotConflicts Action ids registered by more than one action type. /// @return The resolved registries. /// @throws morph::model::EffectsRegistrationError listing every problem, when there is one. [[nodiscard]] inline Resolved resolve(std::vector records, - model::detail::ActionResultSlotMap const& slots, EffectsMap const& declared, + model::detail::ActionResultSlotMap const& resultSlots, EffectsMap const& declared, std::vector const& slotConflicts) { std::vector problems; @@ -392,7 +392,7 @@ inline void findDerivationCycles(std::vector const& records, } Resolved resolved; - for (auto const& [actionId, result] : slots) { + for (auto const& [actionId, result] : resultSlots) { std::vector sorted; sorted.reserve(result.resultSlots.size()); for (auto const& slot : result.resultSlots) { @@ -479,16 +479,16 @@ inline void findDerivationCycles(std::vector const& records, /// @brief Resolves a registry built by a test, leaving the process registry untouched. /// @param records The records. -/// @param slots The action result slots, by action id. +/// @param resultSlots The action result slots, by action id. /// @param declared The effects declarations, by action id. /// @param slotConflicts Action ids registered by more than one action type. /// @return The resolved registries. /// @throws morph::model::EffectsRegistrationError listing every problem, when there is one. [[nodiscard]] inline Resolved resolveForTesting(std::vector records, - model::detail::ActionResultSlotMap const& slots, + model::detail::ActionResultSlotMap const& resultSlots, EffectsMap const& declared, std::vector const& slotConflicts = {}) { - return resolve(std::move(records), slots, declared, slotConflicts); + return resolve(std::move(records), resultSlots, declared, slotConflicts); } /// @brief The process registries, matched and validated on the first call. From 4d8d21021b7e63e323af6f60f43abf14f08be804 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 17:48:10 +0200 Subject: [PATCH 30/68] core: connections watch record types and keyed instances Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/backend.md | 63 ++++- docs/spec/core/registry.md | 6 + docs/spec/core/shared_instances.md | 19 ++ include/morph/core/connection_writer.hpp | 9 + include/morph/core/detail/watch_table.hpp | 203 ++++++++++++++ include/morph/core/registry.hpp | 20 ++ include/morph/core/remote.hpp | 124 ++++++++- tests/CMakeLists.txt | 1 + tests/test_push_watch.cpp | 324 ++++++++++++++++++++++ 9 files changed, 758 insertions(+), 11 deletions(-) create mode 100644 include/morph/core/detail/watch_table.hpp create mode 100644 tests/test_push_watch.cpp diff --git a/docs/spec/core/backend.md b/docs/spec/core/backend.md index 4bdffe13c..186f3177b 100644 --- a/docs/spec/core/backend.md +++ b/docs/spec/core/backend.md @@ -47,6 +47,7 @@ and react to backend changes. - [`LimitPolicy` — opt-in resource limits](#limitpolicy--opt-in-resource-limits) - [Connection scopes](#connection-scopes) - [Writing to a connection](#writing-to-a-connection) +- [Watches](#watches) - [`SimulatedRemoteBackend` — adapter for testing](#simulatedremotebackend--adapter-for-testing) - [`QtWebSocketBackend` — client-side WebSocket transport](#qtwebsocketbackend--client-side-websocket-transport) - [`QtWebSocketServer` — server-side WebSocket transport](#qtwebsocketserver--server-side-websocket-transport) @@ -626,6 +627,8 @@ external reference is dropped. | `execute` | `modelId`, `modelType`, `actionType`, `body`, `session` | `ok` with `body` or `err` | See the execute flow below. | | `hello` | `protocolVersion` | `ok` with `body` = `ProtocolRange`, or `err "protocol version unsupported"` | Protocol-version negotiation, exchanged once per connection before any `register`/`execute`. Carries no `session` and is not authorized — orthogonal to `IAuthorizer`. The range's `capabilities` lists `"cancel"`. See [wire.md](wire.md#protocol-version-negotiation). | | `cancel` | `cancelCallId`, `session` | `ok` (the cancel's own `callId`), always | Stamps the verified principal, then requests stop on the run of the `execute` filed under `cancelCallId` **on the same connection**, if the cancel's verified principal is the execute's and the cancel passes the execute's own `authorize` and `authorizeInstance`. Anything else — unknown, finished, another connection's, another principal's, refused, unscoped — is the same `ok` and changes nothing. See [wire.md](wire.md#cancelling-a-call). | +| `watch` | `body` = `WatchBody`, `session` | `ok` with `body` = `WatchReply`, or `err` | Stamps the verified principal, then adds the admitted topics to the connection's watch set, or makes the set exactly them with `replace: true`. Topics not admitted are listed in the reply, not refused with an `err`. `err` for `cid == 0`, a closed connection, or a body `wire::readWatchBody` refuses. See [Watches](#watches). | +| `unwatch` | `body` = `WatchBody`, `session` | `ok` with an empty `WatchReply`, or `err` | Stamps the verified principal, then removes the listed topics from the connection's watch set. `err` as for `watch`. | | `schemas` | `typeId`, `session` | `ok` with `body` = `{actionType: schema}`, or `err` | Empty `typeId` → `err "schemas requires a typeId"` (checked before authorization). Authenticates, then consults `_authorizer->authorize(env.session, typeId, {})` — the same type-level read hook `instances` uses; denied → `err "unauthorized"`. Answers from `ActionDispatcher::schemasJson(typeId)`; a type with no registered actions yields `{}`, not an error. See [Serving action schemas](#serving-action-schemas). | **Execute flow.** Admission runs on the server strand (`dispatchExecute`), the @@ -1087,7 +1090,62 @@ server one per connection. stops reading altogether is the transport's to close, by its send timeout. - A `write` or a `backlog()` read that throws is caught on the strand and logged at `warn`, and the envelope is not written; the server keeps serving - and the connection keeps its writer until the transport closes it. + and the connection keeps its writer until the transport closes it. Its + watches are dropped at once ([Watches](#watches)). + +### Watches + +A connection tells the server which topics its client shows, so that what a +write changes is sent only where it can be shown. The `watch` and `unwatch` +kinds are core kinds: `RemoteServer` handles them on its strand, with the +connection's `cid`, because the watch table belongs to the connection and the +execute path reads it. The UI layer sends them; nothing on the server outside +`RemoteServer` handles them. + +- **Topics.** A record type, by its wire name; and a keyed shared instance, by + `(modelType, primary)`. An instance topic is stored by key and never resolved + to an instance id: the watch is sent before the fetch that may create the + instance, so the instance may not exist yet. One that never comes to exist + simply never matches. +- **The table.** `detail::WatchTable` (`core/detail/watch_table.hpp`) holds + each connection's record and instance topics. Each watch keeps a + `detail::Watch{context, principal}`: the session of the `watch` envelope that + created it and the principal verified from it, because a connection has no + principal of its own. Watching a topic again keeps the newer one. + `watchersOf(record)` and `watchersOf(modelType, primary)` answer the + connections watching a topic, ascending; `watchOf(cid, record)` the watch + itself. `RemoteServer::watchTable()` exposes it read-only, on the strand. +- **Admission.** Each topic of a `watch` is admitted or refused on its own; the + `ok` reply's `WatchReply` lists the refused ones, and the admitted ones are + stored. + - A record type is admitted when the verified principal may run at least one + action whose result holds it: for each action id + `effects::actionsHolding(record)` returns, and each model type + `ActionDispatcher::modelsOf(actionId)` registers it under, + `authorize(session, modelType, actionId)` — the call an `execute` of that + action is admitted on. The answer for a registered record is cached per + verified principal and record in the table, and a principal's answers are + forgotten once none of its watches is left. A caller with no verified + principal is judged afresh each time: an authorizer that does not + authenticate may still tell such callers apart by their token. A name no record has is refused without + caching, so a peer cannot grow the cache with names it makes up. + - An instance topic is admitted when + `authorizeInstance(session, modelType, {}, 0, "")` admits it: no instance + id, because the instance may not exist, and no owner, because a shared + instance has none ([shared_instances.md](shared_instances.md#watching-a-keyed-instance)). +- **`replace`.** A `watch` with `replace: true` makes the connection's watch + set exactly its admitted topics. A client sends its whole set this way after a + reconnect, which supersedes any watch changes replayed ahead of it. +- **Before `ui-hello`.** A `watch` on a connection that has not negotiated + events is stored all the same; events are posted to it only once it has. +- **Unscoped is refused.** `cid == 0` has no connection whose watch set a + message could change, so both kinds answer `err` there. A connection already + closed answers `err "connection closed"`, so nothing is stored that no close + would remove. +- **What drops watches.** `closeConnection(cid)` drops all of `cid`'s watches, + as does a failed `write` or `backlog()` read on its writer: a connection that + cannot be written to is sent no events, and its client sends the whole set + again when it reconnects. ### Graceful shutdown (`beginShutdown()` / `drainedWithin()`) @@ -2197,10 +2255,11 @@ server: each call is a loop task, so the loop serialises them. | `handle(msg, reply, cid)` | Like `handle(msg, reply)`, additionally attributing any `register` in `msg` to connection `cid`'s scope. `cid == 0` behaves exactly like the two-argument overload. | | `handleInline(msg)` | Sync: runs the control envelope on the server strand — posted and waited for, or inline when already on it — and returns the reply JSON. **Rejects `execute`** — returns an `err` reply without dispatching, because an `execute` reply is produced asynchronously after this call returns. Needs a pool thread other than the caller's free to run the strand. Unscoped. | | `openConnection()` | Returns a fresh non-zero `ConnectionId` at once; its empty scope is opened on the server strand, before the caller's next `handle()`. | -| `closeConnection(cid)` | Posts: erases every model still recorded in `cid`'s scope (as `deregister` would) and drops the scope, its writer and its `ConnectionFacts`. `cid == 0`, unknown, or already-closed is a no-op — idempotent. Bypasses `IAuthorizer` by design. | +| `closeConnection(cid)` | Posts: erases every model still recorded in `cid`'s scope (as `deregister` would) and drops the scope, its writer, its `ConnectionFacts` and its watches. `cid == 0`, unknown, or already-closed is a no-op — idempotent. Bypasses `IAuthorizer` by design. | | `setConnectionWriter(cid, writer)` | Posts: keeps `writer` (`ConnectionWriter{write, backlog}`) for `cid` while its scope is open; replaces an earlier one, keeping an owed `resync`; an empty `write` removes it. `0`, unknown or closed `cid`: dropped. | | `post(cid, envelope)` | Any thread. Above `wire::kMaxEnvelopeBytes`: dropped and counted (`Metric::eventOversize`). On the strand: written through `cid`'s writer, or held back — dropped, with one `resync` owed — while `backlog()` exceeds `kEventBacklogHigh` or a `resync` is still owed. No writer: dropped. | | `connectionDrained(cid)` | Any thread; posts. Sends the owed `resync` event, if any, and lets events flow again. What a transport calls when the backlog falls below `kEventBacklogLow`. | +| `watchTable() const` | Server strand only (debug-asserted). The `detail::WatchTable` of every open connection's watches, read-only. See [Watches](#watches). | | `eventsNegotiated(cid) const` | Server strand only (debug-asserted). Whether `cid` negotiated events: the `ConnectionFacts::events` a service set for it. `false` for `0`, an unknown `cid` or a closed one. | | `payloadCompleteness()` | The `ServerConfig::payloadCompleteness` given at construction. | | `strand()` | The server strand, as an `exec::IExecutor`: `runningOn(server.strand())` is true inside every task on the server's state. | diff --git a/docs/spec/core/registry.md b/docs/spec/core/registry.md index a878ff2b5..a438836d7 100644 --- a/docs/spec/core/registry.md +++ b/docs/spec/core/registry.md @@ -619,6 +619,7 @@ class ActionDispatcher { std::string schemasJson(std::string_view modelId) const; const std::vector* requiredFieldsFor(std::string_view modelId, std::string_view actionId) const; + std::vector modelsOf(std::string_view actionId) const; static ActionDispatcher& instance(); }; ``` @@ -689,6 +690,11 @@ class ActionDispatcher { required*. Read by `RemoteServer`'s opt-in `PayloadCompleteness::RequireDeclaredFields` gate, so the rule the server enforces is by construction the rule the schema published. +- `modelsOf` returns the model type-ids an action id is registered under, + sorted; empty for an unregistered action. It scans every registered pair. + `RemoteServer` reads it when admitting a record watch, which knows an action + only by the id `morph::effects::actionsHolding` returns and must authorize it + against its model type ([backend.md, "Watches"](backend.md#watches)). #### `ActionDescription` — what `registerAction` files for the wire diff --git a/docs/spec/core/shared_instances.md b/docs/spec/core/shared_instances.md index 2b1553e2b..3efefb244 100644 --- a/docs/spec/core/shared_instances.md +++ b/docs/spec/core/shared_instances.md @@ -13,6 +13,7 @@ - [Wire protocol changes](#wire-protocol-changes) - [Register-or-attach and attach go through the bind rule](#register-or-attach-and-attach-go-through-the-bind-rule) - [Ownership and authorization](#ownership-and-authorization) +- [Watching a keyed instance](#watching-a-keyed-instance) - [Lifetime and the A7 connection-scope change](#lifetime-and-the-a7-connection-scope-change) - [API reference](#api-reference) - [Design decisions](#design-decisions) @@ -437,6 +438,24 @@ An application that needs per-instance ownership on a shared model must enforce it inside the model, from `Context::principal`, which is the same advice `security.md` already gives for security-critical checks. +## Watching a keyed instance + +A client watches a keyed shared instance it reads, so that a write to it can be +announced to every connection that shows it. The `watch` topic is +`(modelType, primary)`, the key the alias binds with, and `RemoteServer` stores +it by that key without looking it up in the directory: the watch is sent before +the fetch that may create the instance, so the instance may not exist yet. It +matches whichever instance holds that directory key, from the moment the +instance is keyed, by construction or by `assign`. A key no instance comes to +hold never matches. + +Admission is `authorizeInstance(session, modelType, {}, 0, "")`: no instance +id, because there may be none, and an empty owner, because a shared instance is +ownerless. Ownership therefore protects nothing here; what protects the topic +is the authorizer's rule for the model type, as it is for an execute on a +shared instance. A private instance needs no watch, since the server knows its +owner. See [backend.md, "Watches"](backend.md#watches). + ## Lifetime and the A7 connection-scope change This changed shipped A7 behaviour, which nothing in the §A–§E program did. It was diff --git a/include/morph/core/connection_writer.hpp b/include/morph/core/connection_writer.hpp index 9cac185e7..cceb5673f 100644 --- a/include/morph/core/connection_writer.hpp +++ b/include/morph/core/connection_writer.hpp @@ -2,6 +2,7 @@ #pragma once #include +#include #include #include @@ -15,6 +16,14 @@ namespace morph::backend { +/// @brief Opaque id a transport uses to scope model registrations to one +/// connection so `RemoteServer::closeConnection` can reclaim them. +/// +/// `0` is reserved and means *unscoped* — the meaning today's two-argument +/// `RemoteServer::handle()`/`handleInline()` calls always have. Non-zero +/// values are minted by `RemoteServer::openConnection()`. +using ConnectionId = std::uint64_t; + /// @brief A transport's outbound side of one connection, for what the server /// sends unasked: events. /// diff --git a/include/morph/core/detail/watch_table.hpp b/include/morph/core/detail/watch_table.hpp new file mode 100644 index 000000000..3c71321a7 --- /dev/null +++ b/include/morph/core/detail/watch_table.hpp @@ -0,0 +1,203 @@ +// SPDX-License-Identifier: Apache-2.0 + +#pragma once +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../../session/session.hpp" +#include "../connection_writer.hpp" +#include "../wire.hpp" + +/// @file +/// @brief The topics each connection watches, as `RemoteServer` keeps them. +/// +/// Holds only what the server decided: admission, which needs the record and +/// action registries and the authorizer, happens in the server, which hands +/// this table the admitted topics. + +namespace morph::backend::detail { + +/// @brief What one watched topic was admitted on. +/// +/// A connection has no principal of its own, so each watch keeps the session +/// of the `watch` envelope that created it and the principal verified from +/// that session. Fan-out re-authorises the watch on them at send time. +struct Watch { + /// @brief The `watch` envelope's session, its principal already the verified one. + ::morph::session::Context context; + /// @brief The principal the server verified for that envelope; empty when none was. + std::string principal; +}; + +/// @brief Each connection's watched record types and keyed instances. On the +/// server strand: not synchronised. +/// +/// A record topic is stored by record name. An instance topic is stored by +/// `(modelType, primary)` and never resolved to an instance id, because the +/// watch is sent before the fetch that may create the instance. Watching a +/// topic the connection already watches keeps the newer `Watch`. +/// +/// The table also caches, per principal and record, whether the principal may +/// watch the record, so a connection's repeated watches do not re-run the +/// admission rule. A principal's answers are forgotten once none of its +/// watches is left, so a principal that watches again is judged afresh. +class WatchTable { +public: + /// @brief Adds @p body's topics to @p cid's watch set, or, when + /// `body.replace` is set, makes the set exactly @p body's topics. + /// @param cid Connection the topics are watched on. + /// @param body The admitted topics; `instance` ids are ignored. + /// @param watch What the topics were admitted on. + void add(ConnectionId cid, ::morph::wire::WatchBody const& body, Watch watch) { + if (body.replace) { + _connections.erase(cid); + } + if (body.records.empty() && body.instances.empty()) { + forgetIdleAdmissions(); + return; + } + Connection& connection = _connections[cid]; + for (auto const& record : body.records) { + connection.records.insert_or_assign(record, watch); + } + for (auto const& topic : body.instances) { + connection.instances.insert_or_assign(InstanceKey{topic.modelType, topic.primary}, watch); + } + forgetIdleAdmissions(); + } + + /// @brief Removes @p body's topics from @p cid's watch set. A topic not + /// watched there is ignored. + /// @param cid Connection the topics were watched on. + /// @param body The topics; `replace` and `instance` ids are ignored. + void remove(ConnectionId cid, ::morph::wire::WatchBody const& body) { + auto found = _connections.find(cid); + if (found == _connections.end()) { + return; + } + for (auto const& record : body.records) { + found->second.records.erase(record); + } + for (auto const& topic : body.instances) { + found->second.instances.erase(InstanceKey{topic.modelType, topic.primary}); + } + if (found->second.records.empty() && found->second.instances.empty()) { + _connections.erase(found); + } + forgetIdleAdmissions(); + } + + /// @brief Drops every watch @p cid holds. + /// @param cid Connection that closed, failed a write, or changed principal. + void dropConnection(ConnectionId cid) { + if (_connections.erase(cid) != 0) { + forgetIdleAdmissions(); + } + } + + /// @brief The connections watching record type @p record. + /// @param record A record's wire name. + /// @return The connection ids, ascending. + [[nodiscard]] std::vector watchersOf(std::string_view record) const { + std::vector watchers; + for (auto const& [cid, connection] : _connections) { + if (connection.records.contains(record)) { + watchers.push_back(cid); + } + } + return watchers; + } + + /// @brief The connections watching the keyed instance `(modelType, primary)`. + /// @param modelType The instance's registered model type. + /// @param primary The instance's directory key text. + /// @return The connection ids, ascending. + [[nodiscard]] std::vector watchersOf(std::string_view modelType, std::string_view primary) const { + InstanceKey const key{std::string{modelType}, std::string{primary}}; + std::vector watchers; + for (auto const& [cid, connection] : _connections) { + if (connection.instances.contains(key)) { + watchers.push_back(cid); + } + } + return watchers; + } + + /// @brief What @p cid's watch on record type @p record was admitted on. + /// @param cid Connection. + /// @param record A record's wire name. + /// @return The watch, valid until the table next changes, or null when + /// @p cid does not watch @p record. + [[nodiscard]] Watch const* watchOf(ConnectionId cid, std::string_view record) const { + auto const connection = _connections.find(cid); + if (connection == _connections.end()) { + return nullptr; + } + auto const found = connection->second.records.find(record); + return found == connection->second.records.end() ? nullptr : &found->second; + } + + /// @brief The cached answer to whether @p principal may watch @p record. + /// @param principal A verified principal; empty for none. + /// @param record A record's wire name. + /// @return The answer, or `nullopt` when none is cached. + [[nodiscard]] std::optional admission(std::string_view principal, std::string_view record) const { + auto const byPrincipal = _admissions.find(principal); + if (byPrincipal == _admissions.end()) { + return std::nullopt; + } + auto const found = byPrincipal->second.find(record); + if (found == byPrincipal->second.end()) { + return std::nullopt; + } + return found->second; + } + + /// @brief Caches whether @p principal may watch @p record. + /// @param principal A verified principal; empty for none. + /// @param record A registered record's wire name. + /// @param admitted The answer. + void noteAdmission(std::string_view principal, std::string_view record, bool admitted) { + auto byPrincipal = _admissions.find(principal); + if (byPrincipal == _admissions.end()) { + byPrincipal = _admissions.emplace(std::string{principal}, Answers{}).first; + } + byPrincipal->second.insert_or_assign(std::string{record}, admitted); + } + +private: + using InstanceKey = std::pair; + using Answers = std::map>; + + /// One connection's watches, by topic. + struct Connection { + std::map> records; + std::map instances; + }; + + /// Forgets the cached answers of every principal no watch carries. + void forgetIdleAdmissions() { + std::erase_if(_admissions, [this](auto const& entry) { + for (auto const& [cid, connection] : _connections) { + auto const carries = [&](auto const& topic) { return topic.second.principal == entry.first; }; + if (std::ranges::any_of(connection.records, carries) || + std::ranges::any_of(connection.instances, carries)) { + return false; + } + } + return true; + }); + } + + // Ordered by connection id, so `watchersOf` answers in ascending order. + std::map _connections; + std::map> _admissions; +}; + +} // namespace morph::backend::detail diff --git a/include/morph/core/registry.hpp b/include/morph/core/registry.hpp index 14f56a7dc..302748b40 100644 --- a/include/morph/core/registry.hpp +++ b/include/morph/core/registry.hpp @@ -1250,6 +1250,26 @@ class ActionDispatcher { } } + /// @brief The model type-ids @p actionId is registered under. + /// + /// An action id names one action, but the pair registered is + /// `(model, action)`, so a server that knows only the action id, as a + /// record watch's admission does, asks here which model types to + /// authorize it against. + /// + /// @param actionId Action type-id. + /// @return The model type-ids, sorted; empty for an unregistered action. + [[nodiscard]] std::vector modelsOf(std::string_view actionId) const { + std::vector models; + for (const auto& [key, entry] : _actions) { + if (key.second == actionId) { + models.push_back(key.first); + } + } + std::ranges::sort(models); + return models; + } + /// @brief Returns the process-level singleton dispatcher. static ActionDispatcher& instance() { return defaultDispatcher(); } diff --git a/include/morph/core/remote.hpp b/include/morph/core/remote.hpp index fcabe6361..c93a3bee7 100644 --- a/include/morph/core/remote.hpp +++ b/include/morph/core/remote.hpp @@ -31,6 +31,8 @@ #include "connection_writer.hpp" #include "detail/owner_probe.hpp" #include "detail/reply_router.hpp" +#include "detail/watch_table.hpp" +#include "effects.hpp" #include "logger.hpp" #include "observability.hpp" #include "owner_strand.hpp" @@ -178,14 +180,6 @@ class OpaqueIdGenerator { } // namespace detail -/// @brief Opaque id a transport uses to scope model registrations to one -/// connection so `RemoteServer::closeConnection` can reclaim them. -/// -/// `0` is reserved and means *unscoped* — the meaning today's two-argument -/// `RemoteServer::handle()`/`handleInline()` calls always have. Non-zero -/// values are minted by `RemoteServer::openConnection()`. -using ConnectionId = std::uint64_t; - /// @brief Snapshot of a `RemoteServer`'s current health. struct HealthStatus { /// @brief `true` if the server currently accepts and dispatches new work. @@ -714,6 +708,17 @@ class RemoteServer : public std::enable_shared_from_this { return found != _facts.end() && found->second.events; } + /// @brief The topics each open connection watches. + /// + /// Server strand only: call it from a task on `strand()`; a debug build + /// asserts that. Filled by the `watch` and `unwatch` kinds, and emptied for + /// a connection when it closes or a write to it fails. + /// @return The watch table, valid for the server's life. + [[nodiscard]] detail::WatchTable const& watchTable() const { + assert(_strand.runningHere() && "RemoteServer::watchTable runs on the server strand"); + return _watches; + } + /// @brief The server strand, as an executor. /// /// What `runningOn(server.strand())` asks about: true inside every task the @@ -893,6 +898,7 @@ class RemoteServer : public std::enable_shared_from_this { noteOwner("RemoteServer::closeConnection"); _writers.erase(cid); _facts.erase(cid); + _watches.dropConnection(cid); auto scopeIter = _connectionScopes.find(cid); if (scopeIter == _connectionScopes.end()) { return; @@ -986,12 +992,104 @@ class RemoteServer : public std::enable_shared_from_this { /// its backlog, does. On the server strand. /// /// The connection keeps its writer: the transport reports the connection - /// gone through `closeConnection`, which drops it. + /// gone through `closeConnection`, which drops it. Its watches go now: a + /// connection that cannot be written to is sent no events, and the client + /// sends its whole watch set again when it reconnects. /// @param cid Connection whose `write` or `backlog` threw. /// @param reason The exception's message. void writeFailed(ConnectionId cid, std::string_view reason) { noteOwner("RemoteServer::writeFailed"); ::morph::log::logWarn("[post] write to connection {} failed: {}", cid, reason); + _watches.dropConnection(cid); + } + + /// @brief Answers a `watch` or `unwatch` envelope. On the server strand, + /// after @p env's principal is stamped. + /// + /// Refused with an `err`: `cid == 0`, since an unscoped message has no + /// connection whose watch set it could change; a connection already + /// closed; and a body `readWatchBody` refuses. Otherwise `ok` with a + /// `WatchReply` listing the topics not admitted, which `unwatch` never has. + /// @param env The envelope, its principal the verified one. + /// @param reply Reply sink, invoked once. + /// @param cid Connection the envelope arrived on. + void handleWatch(const ::morph::wire::Envelope& env, const std::function& reply, + ConnectionId cid) { + noteOwner("RemoteServer::watch"); + if (cid == 0) { + reply(::morph::wire::encode(::morph::wire::makeErr(env.kind + " needs a connection", env.callId))); + return; + } + if (!_connectionScopes.contains(cid)) { + reply(::morph::wire::encode(::morph::wire::makeErr("connection closed", env.callId))); + return; + } + auto body = ::morph::wire::readWatchBody(env.body); + if (!body) { + reply(::morph::wire::encode(::morph::wire::makeErr(body.error(), env.callId))); + return; + } + ::morph::wire::WatchReply refused; + if (env.kind == ::morph::wire::kUnwatchKind) { + _watches.remove(cid, *body); + } else { + ::morph::wire::WatchBody admitted{.records = {}, .instances = {}, .replace = body->replace}; + for (auto& record : body->records) { + (mayWatchRecord(env.session, record) ? admitted.records : refused.refusedRecords) + .push_back(std::move(record)); + } + for (auto& topic : body->instances) { + // No instance id and no owner: the instance may not exist yet, + // and a shared one has no owner, so what admits the topic is + // the authorizer's rule for the model type. + bool const allowed = _authorizer->authorizeInstance(env.session, topic.modelType, {}, 0, {}); + (allowed ? admitted.instances : refused.refusedInstances).push_back(std::move(topic)); + } + _watches.add(cid, admitted, detail::Watch{.context = env.session, .principal = env.session.principal}); + } + std::string json; + (void)glz::write_json(refused, json); + reply(::morph::wire::encode(::morph::wire::makeOk(env.callId, std::move(json)))); + } + + /// @brief Whether @p session's principal may watch record type @p record: + /// whether it may run at least one action whose result holds it. + /// On the server strand. + /// + /// The answer for a registered record is cached in the watch table per + /// verified principal. A caller with no verified principal is judged + /// afresh each time, since the authorizer may tell such callers apart by + /// something other than a principal. A name no record has is refused + /// without caching, so a peer cannot grow the cache with names it makes up. + /// @param session The `watch` envelope's session, its principal the verified one. + /// @param record The record's wire name. + /// @return True when the record may be watched. + /// @throws morph::model::EffectsRegistrationError when the registered + /// records and effects are invalid. + [[nodiscard]] bool mayWatchRecord(const ::morph::session::Context& session, std::string_view record) { + bool const cacheable = !session.principal.empty(); + if (auto const cached = cacheable ? _watches.admission(session.principal, record) : std::nullopt) { + return *cached; + } + if (::morph::effects::recordNamed(record) == nullptr) { + return false; + } + bool admitted = false; + for (auto const& actionId : ::morph::effects::actionsHolding(record)) { + for (auto const& modelType : _dispatcher.modelsOf(actionId)) { + if (_authorizer->authorize(session, modelType, actionId)) { + admitted = true; + break; + } + } + if (admitted) { + break; + } + } + if (cacheable) { + _watches.noteAdmission(session.principal, record, admitted); + } + return admitted; } /// @brief Registers a drain waiter, or answers it at once. On the server strand. @@ -1667,6 +1765,11 @@ class RemoteServer : public std::enable_shared_from_this { body); reply(::morph::wire::encode(::morph::wire::makeOk(env.callId, std::move(body)))); } + } else if (env.kind == ::morph::wire::kWatchKind || env.kind == ::morph::wire::kUnwatchKind) { + // Stamped first: the watch keeps the verified principal, and + // admission decides on it. + stampVerifiedPrincipal(env); + handleWatch(env, reply, cid); } else if (IEnvelopeService* const service = serviceFor(env.kind); service != nullptr) { // Stamped first, as for every built-in kind that authorises: // the service decides on the verified principal, never on the @@ -2073,6 +2176,9 @@ class RemoteServer : public std::enable_shared_from_this { // Handed to a service for a message with no open connection behind it; // reset before each such message. ConnectionFacts _throwawayFacts; + // What each open connection watches, filled by `watch`/`unwatch` and + // dropped with the connection's scope or on a failed write. + detail::WatchTable _watches; /// Names one cancellable call: the connection it arrived on and the /// `callId` its client chose. struct CancelKey { diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index b67e5579f..f630285c1 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -26,6 +26,7 @@ add_executable(morph_tests test_push_effects.cpp test_push_catalog.cpp test_push_channel.cpp + test_push_watch.cpp push_records_late.cpp test_executor_extra.cpp test_strand.cpp diff --git a/tests/test_push_watch.cpp b/tests/test_push_watch.cpp new file mode 100644 index 000000000..64f508bbd --- /dev/null +++ b/tests/test_push_watch.cpp @@ -0,0 +1,324 @@ +// SPDX-License-Identifier: Apache-2.0 + +// The server's per-connection watch table and the core `watch`/`unwatch` +// kinds: what a watch may name, how it is stored, and what drops it. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +// Glaze reflects member names through an external-linkage object, so these types cannot sit in an +// anonymous namespace. +namespace push_watch_test { + +struct Row { + std::int64_t id{}; + std::string name; +}; + +struct RowList { + std::vector rows; +}; + +struct ListRows { + int limit{}; +}; + +struct SecretRow { + std::int64_t id{}; +}; + +struct SecretList { + std::vector rows; +}; + +struct ListSecrets { + int limit{}; +}; + +struct RowModel { + RowList execute(ListRows const& /*action*/) { return {}; } +}; + +struct SecretModel { + SecretList execute(ListSecrets const& /*action*/) { return {}; } +}; + +} // namespace push_watch_test + +using namespace push_watch_test; + +BRIDGE_REGISTER_RECORD(Row, "T_WatchRow") +BRIDGE_REGISTER_RECORD(SecretRow, "T_WatchSecret") +BRIDGE_REGISTER_MODEL(RowModel, "T_WatchRowModel") +BRIDGE_REGISTER_ACTION(RowModel, ListRows, "T_WatchListRows") +BRIDGE_REGISTER_MODEL(SecretModel, "T_WatchSecretModel") +BRIDGE_REGISTER_ACTION(SecretModel, ListSecrets, "T_WatchListSecrets") + +namespace { + +using ::morph::backend::ConnectionId; +using ::morph::backend::RemoteServer; + +constexpr std::string_view kSecretModel = "T_WatchSecretModel"; + +/// Verifies a session by its token, which is the principal itself, and refuses +/// every action and instance of the secret model. +struct SecretGuard final : ::morph::session::IAuthorizer { + mutable std::atomic authorizeCalls{0}; + + [[nodiscard]] bool authorize(const ::morph::session::Context& /*ctx*/, std::string_view modelType, + std::string_view /*actionType*/) const override { + authorizeCalls.fetch_add(1); + return modelType != kSecretModel; + } + [[nodiscard]] std::optional authenticate(const ::morph::session::Context& ctx) const override { + if (ctx.token.empty()) { + return std::nullopt; + } + return ctx.token; + } + [[nodiscard]] bool authorizeInstance(const ::morph::session::Context& /*ctx*/, std::string_view modelType, + std::string_view /*actionType*/, std::uint64_t /*modelId*/, + std::string_view /*ownerPrincipal*/) const override { + return modelType != kSecretModel; + } +}; + +struct Rig { + ::morph::exec::ThreadPoolExecutor pool{2}; + std::shared_ptr guard = std::make_shared(); + std::shared_ptr server = std::make_shared(pool, guard); + + /// Sends @p kind with @p body on @p cid, signed as @p principal, and returns the reply. + ::morph::wire::Envelope send(ConnectionId cid, std::string_view kind, ::morph::wire::WatchBody const& body, + std::string const& principal = "alice") { + ::morph::wire::Envelope env; + env.kind = std::string{kind}; + env.callId = 7; + env.session.token = principal; + env.body = glz::write_json(body).value_or(std::string{}); + return ::morph::wire::decode(server->handleInline(::morph::wire::encode(env), cid)); + } + + /// Sends a `watch` and returns its decoded `ok` reply. + ::morph::wire::WatchReply watch(ConnectionId cid, ::morph::wire::WatchBody const& body, + std::string const& principal = "alice") { + auto const reply = send(cid, ::morph::wire::kWatchKind, body, principal); + REQUIRE(reply.kind == "ok"); + auto parsed = ::morph::wire::readWatchReply(reply.body); + REQUIRE(parsed); + return *parsed; + } + + /// Runs @p read on the server strand and returns what it returned. + template + auto onStrand(Fn read) { + std::optionalwatchTable()))> answer; + std::latch done{1}; + server->strand().post([&] { + answer.emplace(read(server->watchTable())); + done.count_down(); + }); + done.wait(); + return std::move(*answer); + } + + std::vector recordWatchers(std::string_view record) { + return onStrand([record](auto const& table) { return table.watchersOf(record); }); + } + + std::vector instanceWatchers(std::string_view modelType, std::string_view primary) { + return onStrand([modelType, primary](auto const& table) { return table.watchersOf(modelType, primary); }); + } +}; + +::morph::wire::WatchBody records(std::vector names, bool replace = false) { + return {.records = std::move(names), .instances = {}, .replace = replace}; +} + +::morph::wire::WatchBody instance(std::string modelType, std::string primary) { + return {.records = {}, .instances = {{.modelType = std::move(modelType), .primary = std::move(primary)}}}; +} + +} // namespace + +TEST_CASE("watch_record_admitted", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + + auto const reply = rig.watch(cid, records({"T_WatchRow"})); + CHECK(reply.refusedRecords.empty()); + CHECK(reply.refusedInstances.empty()); + CHECK(rig.recordWatchers("T_WatchRow") == std::vector{cid}); + + // The watch keeps the session it arrived with and the principal verified from it. + auto const stored = rig.onStrand([cid](auto const& table) -> std::optional> { + auto const* watch = table.watchOf(cid, "T_WatchRow"); + if (watch == nullptr) { + return std::nullopt; + } + return std::pair{watch->principal, watch->context.token}; + }); + REQUIRE(stored); + CHECK(stored->first == "alice"); + CHECK(stored->second == "alice"); + + // A record no registered record has is refused, as is a record no action's result holds... and + // `unwatch` takes the admitted one away again. + auto const unknown = rig.watch(cid, records({"T_WatchNoSuchRecord"})); + CHECK(unknown.refusedRecords == std::vector{"T_WatchNoSuchRecord"}); + auto const unwatched = rig.send(cid, ::morph::wire::kUnwatchKind, records({"T_WatchRow"})); + CHECK(unwatched.kind == "ok"); + CHECK(rig.recordWatchers("T_WatchRow").empty()); +} + +TEST_CASE("watch_on_unreadable_type_refused", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + + ::morph::wire::WatchBody body = records({"T_WatchSecret", "T_WatchRow"}); + body.instances = {{.modelType = std::string{kSecretModel}, .primary = "k1"}, + {.modelType = "T_WatchRowModel", .primary = "k1"}}; + auto const reply = rig.watch(cid, body); + + CHECK(reply.refusedRecords == std::vector{"T_WatchSecret"}); + REQUIRE(reply.refusedInstances.size() == 1U); + CHECK(reply.refusedInstances.front().modelType == kSecretModel); + CHECK(reply.refusedInstances.front().primary == "k1"); + CHECK(rig.recordWatchers("T_WatchSecret").empty()); + CHECK(rig.instanceWatchers(kSecretModel, "k1").empty()); + CHECK(rig.recordWatchers("T_WatchRow") == std::vector{cid}); + CHECK(rig.instanceWatchers("T_WatchRowModel", "k1") == std::vector{cid}); +} + +TEST_CASE("instance_watch_stored_before_instance_exists", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + + // No client has registered or attached `T_WatchRowModel` under "never-made": the topic is kept by + // its key alone. + auto const reply = rig.watch(cid, instance("T_WatchRowModel", "never-made")); + CHECK(reply.refusedInstances.empty()); + CHECK(rig.instanceWatchers("T_WatchRowModel", "never-made") == std::vector{cid}); + CHECK(rig.instanceWatchers("T_WatchRowModel", "other").empty()); +} + +TEST_CASE("replace_supersedes", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + auto const other = rig.server->openConnection(); + + ::morph::wire::WatchBody first = records({"T_WatchRow"}); + first.instances = {{.modelType = "T_WatchRowModel", .primary = "a"}}; + (void)rig.watch(cid, first); + (void)rig.watch(other, records({"T_WatchRow"})); + + ::morph::wire::WatchBody full = instance("T_WatchRowModel", "b"); + full.replace = true; + auto const reply = rig.watch(cid, full); + CHECK(reply.refusedInstances.empty()); + + // The connection's set is exactly the body; another connection's is untouched. + CHECK(rig.recordWatchers("T_WatchRow") == std::vector{other}); + CHECK(rig.instanceWatchers("T_WatchRowModel", "a").empty()); + CHECK(rig.instanceWatchers("T_WatchRowModel", "b") == std::vector{cid}); + + // Without `replace`, a watch adds. + (void)rig.watch(cid, records({"T_WatchRow"})); + CHECK(rig.recordWatchers("T_WatchRow") == std::vector{cid, other}); + CHECK(rig.instanceWatchers("T_WatchRowModel", "b") == std::vector{cid}); +} + +TEST_CASE("close_connection_drops_watches", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + auto const other = rig.server->openConnection(); + ::morph::wire::WatchBody body = records({"T_WatchRow"}); + body.instances = {{.modelType = "T_WatchRowModel", .primary = "a"}}; + (void)rig.watch(cid, body); + (void)rig.watch(other, records({"T_WatchRow"})); + + rig.server->closeConnection(cid); + + CHECK(rig.recordWatchers("T_WatchRow") == std::vector{other}); + CHECK(rig.instanceWatchers("T_WatchRowModel", "a").empty()); + + // A closed connection takes no new watches. + auto const late = rig.send(cid, ::morph::wire::kWatchKind, records({"T_WatchRow"})); + CHECK(late.kind == "err"); + CHECK(rig.recordWatchers("T_WatchRow") == std::vector{other}); +} + +TEST_CASE("watch_before_hello_is_stored", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + + (void)rig.watch(cid, records({"T_WatchRow"})); + + CHECK_FALSE(rig.onStrand([&](auto const& /*table*/) { return rig.server->eventsNegotiated(cid); })); + CHECK(rig.recordWatchers("T_WatchRow") == std::vector{cid}); +} + +TEST_CASE("write_failure_drops_watches", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + rig.server->setConnectionWriter(cid, {.write = [](std::string const&) { throw std::runtime_error("socket gone"); }, + .backlog = [] { return std::size_t{0}; }}); + (void)rig.watch(cid, records({"T_WatchRow"})); + REQUIRE(rig.recordWatchers("T_WatchRow") == std::vector{cid}); + + rig.server->post(cid, ::morph::wire::encode(::morph::wire::makeEvent(::morph::wire::kChangesTopic, "{}"))); + + CHECK(rig.recordWatchers("T_WatchRow").empty()); +} + +TEST_CASE("unscoped_watch_refused", "[push][watch]") { + Rig rig; + + auto const reply = rig.send(ConnectionId{0}, ::morph::wire::kWatchKind, records({"T_WatchRow"})); + CHECK(reply.kind == "err"); + auto const unwatch = rig.send(ConnectionId{0}, ::morph::wire::kUnwatchKind, records({"T_WatchRow"})); + CHECK(unwatch.kind == "err"); + CHECK(rig.recordWatchers("T_WatchRow").empty()); +} + +TEST_CASE("record_admission_cached_per_principal", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + + (void)rig.watch(cid, records({"T_WatchRow"})); + int const afterFirst = rig.guard->authorizeCalls.load(); + REQUIRE(afterFirst > 0); + (void)rig.watch(cid, records({"T_WatchRow"})); + CHECK(rig.guard->authorizeCalls.load() == afterFirst); + + // A caller with no verified principal is judged every time. + auto const anonymous = rig.server->openConnection(); + (void)rig.watch(anonymous, records({"T_WatchRow"}), ""); + int const afterAnonymous = rig.guard->authorizeCalls.load(); + CHECK(afterAnonymous > afterFirst); + (void)rig.watch(anonymous, records({"T_WatchRow"}), ""); + CHECK(rig.guard->authorizeCalls.load() > afterAnonymous); + + // Once the principal's watches are gone, so is its cached answer. + rig.server->closeConnection(cid); + auto const again = rig.server->openConnection(); + (void)rig.watch(again, records({"T_WatchRow"})); + CHECK(rig.guard->authorizeCalls.load() > afterAnonymous + 1); +} From 9c0a671e9336e037970cba68153607529678024f Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 18:21:58 +0200 Subject: [PATCH 31/68] core: watch admission keys on the token, checks instance types, and caps a connection's watches Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/backend.md | 48 +++-- docs/spec/core/registry.md | 10 ++ docs/spec/core/shared_instances.md | 8 +- docs/spec/core/wire.md | 3 +- include/morph/core/connection_writer.hpp | 6 +- include/morph/core/detail/watch_table.hpp | 204 ++++++++++++++++------ include/morph/core/registry.hpp | 26 +++ include/morph/core/remote.hpp | 119 +++++++++---- include/morph/core/wire.hpp | 3 +- tests/test_push_watch.cpp | 172 +++++++++++++++++- 10 files changed, 480 insertions(+), 119 deletions(-) diff --git a/docs/spec/core/backend.md b/docs/spec/core/backend.md index 186f3177b..709e444b4 100644 --- a/docs/spec/core/backend.md +++ b/docs/spec/core/backend.md @@ -1114,25 +1114,45 @@ execute path reads it. The UI layer sends them; nothing on the server outside principal of its own. Watching a topic again keeps the newer one. `watchersOf(record)` and `watchersOf(modelType, primary)` answer the connections watching a topic, ascending; `watchOf(cid, record)` the watch - itself. `RemoteServer::watchTable()` exposes it read-only, on the strand. -- **Admission.** Each topic of a `watch` is admitted or refused on its own; the + itself; `topicCount(cid)` how many topics `cid` watches. `RemoteServer::watchTable()` exposes it read-only, on the strand. +- **Admission.** Each topic of a `watch` is admitted or refused on its own, + on the principal `stampVerifiedPrincipal` verified for the envelope; the `ok` reply's `WatchReply` lists the refused ones, and the admitted ones are - stored. + stored. A topic listed twice in one body is judged, and listed, once. - A record type is admitted when the verified principal may run at least one action whose result holds it: for each action id `effects::actionsHolding(record)` returns, and each model type `ActionDispatcher::modelsOf(actionId)` registers it under, - `authorize(session, modelType, actionId)` — the call an `execute` of that - action is admitted on. The answer for a registered record is cached per - verified principal and record in the table, and a principal's answers are - forgotten once none of its watches is left. A caller with no verified - principal is judged afresh each time: an authorizer that does not - authenticate may still tell such callers apart by their token. A name no record has is refused without - caching, so a peer cannot grow the cache with names it makes up. - - An instance topic is admitted when - `authorizeInstance(session, modelType, {}, 0, "")` admits it: no instance - id, because the instance may not exist, and no owner, because a shared - instance has none ([shared_instances.md](shared_instances.md#watching-a-keyed-instance)). + `authorize(session, modelType, actionId)`. Which actions hold a record + comes from the process effects registry; which model types an action + belongs to comes from this server's dispatcher. A name no record has is + refused. + - An instance topic is admitted when three things hold: its model type is + registered with this server (`ModelRegistryFactory::contains`); the + principal may run at least one of that model's actions + (`authorize(session, modelType, actionId)` over + `ActionDispatcher::actionsOf(modelType)`); and + `authorizeInstance(session, modelType, {}, 0, "")` admits it. The last + carries no instance id, because the instance may not exist, and no owner, + because a shared instance has none + ([shared_instances.md](shared_instances.md#watching-a-keyed-instance)). + The first two matter because `authorizeInstance` allows everything by + default. + - **The cache.** The `authorize` answer for a record type, and for a model + type's actions, is cached in the table per verified principal **and + token**: two tokens of one principal, such as a scoped agent token and a + full one, may carry different claims. Each `(principal, token)` counts the + topics watched with it, and its answers are forgotten when the count + returns to zero. A caller with no verified principal is judged afresh each + time, since an authorizer that does not authenticate may still tell such + callers apart by their token. A name no record has, and a model type not + registered, are refused without caching, so a peer cannot grow the cache + with names it makes up. +- **The cap.** One connection watches at most `wire::kMaxWatchTopics` (4096) + topics, records and instances together. A topic the connection does not + already watch is refused, and listed in the reply, while it is at the cap; an + `unwatch` makes room again. Every topic one `watch` admits shares one copy of + its session. - **`replace`.** A `watch` with `replace: true` makes the connection's watch set exactly its admitted topics. A client sends its whole set this way after a reconnect, which supersedes any watch changes replayed ahead of it. diff --git a/docs/spec/core/registry.md b/docs/spec/core/registry.md index a438836d7..9f1cc9ff2 100644 --- a/docs/spec/core/registry.md +++ b/docs/spec/core/registry.md @@ -620,6 +620,7 @@ class ActionDispatcher { const std::vector* requiredFieldsFor(std::string_view modelId, std::string_view actionId) const; std::vector modelsOf(std::string_view actionId) const; + std::vector actionsOf(std::string_view modelId) const; static ActionDispatcher& instance(); }; ``` @@ -695,6 +696,10 @@ class ActionDispatcher { `RemoteServer` reads it when admitting a record watch, which knows an action only by the id `morph::effects::actionsHolding` returns and must authorize it against its model type ([backend.md, "Watches"](backend.md#watches)). +- `actionsOf` returns the action type-ids registered under a model type-id, + sorted; empty for a model with none. `RemoteServer` reads it when admitting + an instance watch, which a principal may hold only when it may run one of the + model's actions. #### `ActionDescription` — what `registerAction` files for the wire @@ -752,10 +757,15 @@ class ModelRegistryFactory { void registerModel(std::string_view modelId, Factory factory); std::unique_ptr create(std::string_view modelId, std::string_view primary = {}); + bool contains(std::string_view modelId) const; static ModelRegistryFactory& instance(); }; ``` +- `contains` answers whether a model type is registered, without creating + one. `RemoteServer` refuses an instance watch on a model type it does not + have. + - `create` throws `std::runtime_error` for unknown model types. The optional @p primary parameter is the instance's stable primary key (empty for an anonymous instance); after building the holder, `create` calls diff --git a/docs/spec/core/shared_instances.md b/docs/spec/core/shared_instances.md index 3efefb244..b189c5a0b 100644 --- a/docs/spec/core/shared_instances.md +++ b/docs/spec/core/shared_instances.md @@ -449,11 +449,13 @@ matches whichever instance holds that directory key, from the moment the instance is keyed, by construction or by `assign`. A key no instance comes to hold never matches. -Admission is `authorizeInstance(session, modelType, {}, 0, "")`: no instance +Admission requires the model type to be registered, the principal to be +allowed at least one of the model's actions by `authorize`, and +`authorizeInstance(session, modelType, {}, 0, "")` to admit it: no instance id, because there may be none, and an empty owner, because a shared instance is ownerless. Ownership therefore protects nothing here; what protects the topic -is the authorizer's rule for the model type, as it is for an execute on a -shared instance. A private instance needs no watch, since the server knows its +is the authorizer's rule for the model type's actions, as it is for an execute +on a shared instance. A private instance needs no watch, since the server knows its owner. See [backend.md, "Watches"](backend.md#watches). ## Lifetime and the A7 connection-scope change diff --git a/docs/spec/core/wire.md b/docs/spec/core/wire.md index 0427a8e8f..d51fe9814 100644 --- a/docs/spec/core/wire.md +++ b/docs/spec/core/wire.md @@ -729,7 +729,8 @@ body, more than `kMaxWatchTopics` (4096) records or instances in one list, more (4096, `core/change_types.hpp`) keys in one record entry, and any record name, model type or primary longer than `kMaxTopicNameBytes` (1024); the message says which one. Unknown keys are ignored, so a newer peer may add fields. The bounds keep a peer-supplied list from forcing an unbounded allocation and sit above any -real watch set. The producer honours the key bound too: `deriveChanges` reports a record whose keys would +real watch set. `RemoteServer` applies `kMaxWatchTopics` to a connection's whole watch set too +([backend.md, "Watches"](backend.md#watches)). The producer honours the key bound too: `deriveChanges` reports a record whose keys would exceed it as a whole-type change (keys absent), so a change is never refused for its size. ## Action-evolution policy diff --git a/include/morph/core/connection_writer.hpp b/include/morph/core/connection_writer.hpp index cceb5673f..e64ed9157 100644 --- a/include/morph/core/connection_writer.hpp +++ b/include/morph/core/connection_writer.hpp @@ -19,9 +19,9 @@ namespace morph::backend { /// @brief Opaque id a transport uses to scope model registrations to one /// connection so `RemoteServer::closeConnection` can reclaim them. /// -/// `0` is reserved and means *unscoped* — the meaning today's two-argument -/// `RemoteServer::handle()`/`handleInline()` calls always have. Non-zero -/// values are minted by `RemoteServer::openConnection()`. +/// `0` is reserved and means *unscoped*: it is what the two-argument +/// `RemoteServer::handle()`/`handleInline()` overloads pass. Non-zero values +/// are minted by `RemoteServer::openConnection()`. using ConnectionId = std::uint64_t; /// @brief A transport's outbound side of one connection, for what the server diff --git a/include/morph/core/detail/watch_table.hpp b/include/morph/core/detail/watch_table.hpp index 3c71321a7..d73d5f8d2 100644 --- a/include/morph/core/detail/watch_table.hpp +++ b/include/morph/core/detail/watch_table.hpp @@ -1,9 +1,12 @@ // SPDX-License-Identifier: Apache-2.0 #pragma once -#include +#include +#include +#include #include #include +#include #include #include #include @@ -35,41 +38,63 @@ struct Watch { std::string principal; }; +/// @brief What an admission answer in a `WatchTable` is about. +enum class AdmissionTopic : std::uint8_t { + /// @brief A record type, by wire name. + record, + /// @brief The instances of a model type, by type name. + modelType, +}; + /// @brief Each connection's watched record types and keyed instances. On the /// server strand: not synchronised. /// /// A record topic is stored by record name. An instance topic is stored by /// `(modelType, primary)` and never resolved to an instance id, because the /// watch is sent before the fetch that may create the instance. Watching a -/// topic the connection already watches keeps the newer `Watch`. +/// topic the connection already watches keeps the newer `Watch`. One +/// connection watches at most `wire::kMaxWatchTopics` topics, records and +/// instances together, which bounds what a peer can make the server hold. /// -/// The table also caches, per principal and record, whether the principal may -/// watch the record, so a connection's repeated watches do not re-run the -/// admission rule. A principal's answers are forgotten once none of its -/// watches is left, so a principal that watches again is judged afresh. +/// The table also caches admission answers per verified principal and token: +/// two tokens of one principal may carry different claims, so they are judged +/// apart. Each `(principal, token)` counts the topics watched with it, and its +/// answers are forgotten when that count returns to zero, so a session that +/// watches again is judged afresh. class WatchTable { public: /// @brief Adds @p body's topics to @p cid's watch set, or, when /// `body.replace` is set, makes the set exactly @p body's topics. + /// + /// A topic not watched yet is refused while the connection already + /// watches `wire::kMaxWatchTopics` topics. /// @param cid Connection the topics are watched on. - /// @param body The admitted topics; `instance` ids are ignored. - /// @param watch What the topics were admitted on. - void add(ConnectionId cid, ::morph::wire::WatchBody const& body, Watch watch) { + /// @param body The admitted topics, without duplicates; `instance` ids are ignored. + /// @param watch What the topics were admitted on, shared by all of them. + /// @return The topics refused because the connection was at its cap. + ::morph::wire::WatchReply add(ConnectionId cid, ::morph::wire::WatchBody const& body, Watch watch) { if (body.replace) { - _connections.erase(cid); - } - if (body.records.empty() && body.instances.empty()) { - forgetIdleAdmissions(); - return; + dropConnection(cid); } + auto const shared = std::make_shared(std::move(watch)); + SessionKey const key = keyOf(*shared); + ::morph::wire::WatchReply overCap; Connection& connection = _connections[cid]; for (auto const& record : body.records) { - connection.records.insert_or_assign(record, watch); + if (!place(connection, connection.records, record, shared, key)) { + overCap.refusedRecords.push_back(record); + } } for (auto const& topic : body.instances) { - connection.instances.insert_or_assign(InstanceKey{topic.modelType, topic.primary}, watch); + if (!place(connection, connection.instances, InstanceKey{topic.modelType, topic.primary}, shared, key)) { + overCap.refusedInstances.push_back(topic); + } + } + if (connection.size() == 0) { + _connections.erase(cid); } - forgetIdleAdmissions(); + forgetIfIdle(key); + return overCap; } /// @brief Removes @p body's topics from @p cid's watch set. A topic not @@ -82,23 +107,30 @@ class WatchTable { return; } for (auto const& record : body.records) { - found->second.records.erase(record); + erase(found->second.records, record); } for (auto const& topic : body.instances) { - found->second.instances.erase(InstanceKey{topic.modelType, topic.primary}); + erase(found->second.instances, InstanceKey{topic.modelType, topic.primary}); } - if (found->second.records.empty() && found->second.instances.empty()) { + if (found->second.size() == 0) { _connections.erase(found); } - forgetIdleAdmissions(); } /// @brief Drops every watch @p cid holds. /// @param cid Connection that closed, failed a write, or changed principal. void dropConnection(ConnectionId cid) { - if (_connections.erase(cid) != 0) { - forgetIdleAdmissions(); + auto found = _connections.find(cid); + if (found == _connections.end()) { + return; + } + for (auto const& [record, watch] : found->second.records) { + release(keyOf(*watch)); } + for (auto const& [topic, watch] : found->second.instances) { + release(keyOf(*watch)); + } + _connections.erase(found); } /// @brief The connections watching record type @p record. @@ -140,64 +172,124 @@ class WatchTable { return nullptr; } auto const found = connection->second.records.find(record); - return found == connection->second.records.end() ? nullptr : &found->second; + return found == connection->second.records.end() ? nullptr : found->second.get(); + } + + /// @brief How many topics @p cid watches, records and instances together. + /// @param cid Connection. + /// @return The count; 0 for a connection with no watches. + [[nodiscard]] std::size_t topicCount(ConnectionId cid) const { + auto const found = _connections.find(cid); + return found == _connections.end() ? 0 : found->second.size(); } - /// @brief The cached answer to whether @p principal may watch @p record. - /// @param principal A verified principal; empty for none. - /// @param record A record's wire name. + /// @brief The cached answer to whether @p session may watch @p name. + /// @param session A session whose principal is the verified one. + /// @param kind What @p name names. + /// @param name A record's wire name or a model type name. /// @return The answer, or `nullopt` when none is cached. - [[nodiscard]] std::optional admission(std::string_view principal, std::string_view record) const { - auto const byPrincipal = _admissions.find(principal); - if (byPrincipal == _admissions.end()) { + [[nodiscard]] std::optional admission(::morph::session::Context const& session, AdmissionTopic kind, + std::string_view name) const { + auto const bySession = _sessions.find(SessionKey{session.principal, session.token}); + if (bySession == _sessions.end()) { return std::nullopt; } - auto const found = byPrincipal->second.find(record); - if (found == byPrincipal->second.end()) { + auto const& answers = bySession->second.answers.at(static_cast(kind)); + auto const found = answers.find(name); + if (found == answers.end()) { return std::nullopt; } return found->second; } - /// @brief Caches whether @p principal may watch @p record. - /// @param principal A verified principal; empty for none. - /// @param record A registered record's wire name. - /// @param admitted The answer. - void noteAdmission(std::string_view principal, std::string_view record, bool admitted) { - auto byPrincipal = _admissions.find(principal); - if (byPrincipal == _admissions.end()) { - byPrincipal = _admissions.emplace(std::string{principal}, Answers{}).first; - } - byPrincipal->second.insert_or_assign(std::string{record}, admitted); + /// @brief Caches whether @p session may watch @p name. + /// + /// Kept while a topic watched with the same principal and token remains; + /// forgotten at the end of the next `add` otherwise. + /// @param session A session whose principal is the verified one. + /// @param kind What @p name names. + /// @param name A registered record's wire name or model type name. + /// @param admitted The answer. + void noteAdmission(::morph::session::Context const& session, AdmissionTopic kind, std::string_view name, + bool admitted) { + auto& answers = _sessions[SessionKey{session.principal, session.token}].answers; + answers.at(static_cast(kind)).insert_or_assign(std::string{name}, admitted); } private: using InstanceKey = std::pair; + // (verified principal, token): what an admission answer was decided on. + using SessionKey = std::pair; + using WatchPtr = std::shared_ptr; using Answers = std::map>; /// One connection's watches, by topic. struct Connection { - std::map> records; - std::map instances; + std::map> records; + std::map instances; + + [[nodiscard]] std::size_t size() const { return records.size() + instances.size(); } }; - /// Forgets the cached answers of every principal no watch carries. - void forgetIdleAdmissions() { - std::erase_if(_admissions, [this](auto const& entry) { - for (auto const& [cid, connection] : _connections) { - auto const carries = [&](auto const& topic) { return topic.second.principal == entry.first; }; - if (std::ranges::any_of(connection.records, carries) || - std::ranges::any_of(connection.instances, carries)) { - return false; - } - } + /// How many watched topics carry one session, and its cached answers by `AdmissionTopic`. + struct Session { + std::size_t topics = 0; + std::array answers; + }; + + [[nodiscard]] static SessionKey keyOf(Watch const& watch) { return {watch.principal, watch.context.token}; } + + /// Stores @p watch for @p topic, unless that would take @p connection past the cap. + template + bool place(Connection const& connection, Topics& topics, Topic const& topic, WatchPtr const& watch, + SessionKey const& key) { + auto const found = topics.find(topic); + if (found != topics.end()) { + // Counted before the old session is released, so a re-watch by the + // same session never drops that session's answers. + ++_sessions[key].topics; + release(keyOf(*found->second)); + found->second = watch; return true; - }); + } + if (connection.size() >= ::morph::wire::kMaxWatchTopics) { + return false; + } + topics.emplace(topic, watch); + ++_sessions[key].topics; + return true; + } + + /// Removes @p topic, if watched, and releases its session. + template + void erase(Topics& topics, Topic const& topic) { + auto const found = topics.find(topic); + if (found == topics.end()) { + return; + } + release(keyOf(*found->second)); + topics.erase(found); + } + + /// One topic watched with @p key is gone; the session goes with its last. + void release(SessionKey const& key) { + auto const found = _sessions.find(key); + if (found != _sessions.end() && --found->second.topics == 0) { + _sessions.erase(found); + } + } + + /// Forgets @p key's answers when no watched topic carries it. + void forgetIfIdle(SessionKey const& key) { + auto const found = _sessions.find(key); + if (found != _sessions.end() && found->second.topics == 0) { + _sessions.erase(found); + } } // Ordered by connection id, so `watchersOf` answers in ascending order. std::map _connections; - std::map> _admissions; + std::map _sessions; }; } // namespace morph::backend::detail diff --git a/include/morph/core/registry.hpp b/include/morph/core/registry.hpp index 302748b40..6eeca87e5 100644 --- a/include/morph/core/registry.hpp +++ b/include/morph/core/registry.hpp @@ -1270,6 +1270,24 @@ class ActionDispatcher { return models; } + /// @brief The action type-ids registered under @p modelId. + /// + /// What a server authorizes an instance watch against: a principal may + /// watch an instance of a model only when it may run one of its actions. + /// + /// @param modelId Model type-id. + /// @return The action type-ids, sorted; empty for a model with none. + [[nodiscard]] std::vector actionsOf(std::string_view modelId) const { + std::vector actions; + for (const auto& [key, entry] : _actions) { + if (key.first == modelId) { + actions.push_back(key.second); + } + } + std::ranges::sort(actions); + return actions; + } + /// @brief Returns the process-level singleton dispatcher. static ActionDispatcher& instance() { return defaultDispatcher(); } @@ -1423,6 +1441,14 @@ class ModelRegistryFactory { return holder; } + /// @brief Whether a model type is registered under @p modelId. + /// @param modelId Model type-id. + /// @return True when `create(modelId)` would find a factory. + [[nodiscard]] bool contains(std::string_view modelId) const { + detail::noteRegistryRead(this == &defaultRegistry()); + return _factories.contains(std::string{modelId}); + } + /// @brief Returns the process-level singleton registry. static ModelRegistryFactory& instance() { return defaultRegistry(); } diff --git a/include/morph/core/remote.hpp b/include/morph/core/remote.hpp index c93a3bee7..669bec3b3 100644 --- a/include/morph/core/remote.hpp +++ b/include/morph/core/remote.hpp @@ -10,11 +10,13 @@ #include #include #include +#include #include #include #include #include #include +#include #include #include #include @@ -1009,7 +1011,9 @@ class RemoteServer : public std::enable_shared_from_this { /// Refused with an `err`: `cid == 0`, since an unscoped message has no /// connection whose watch set it could change; a connection already /// closed; and a body `readWatchBody` refuses. Otherwise `ok` with a - /// `WatchReply` listing the topics not admitted, which `unwatch` never has. + /// `WatchReply` listing the topics not admitted, which `unwatch` never + /// has. A topic listed twice in one body is judged, and reported, once. + /// Topics past `wire::kMaxWatchTopics` on the connection are refused. /// @param env The envelope, its principal the verified one. /// @param reply Reply sink, invoked once. /// @param cid Connection the envelope arrived on. @@ -1034,62 +1038,111 @@ class RemoteServer : public std::enable_shared_from_this { _watches.remove(cid, *body); } else { ::morph::wire::WatchBody admitted{.records = {}, .instances = {}, .replace = body->replace}; - for (auto& record : body->records) { - (mayWatchRecord(env.session, record) ? admitted.records : refused.refusedRecords) - .push_back(std::move(record)); + std::set seenRecords; + for (auto const& record : body->records) { + if (seenRecords.insert(record).second) { + (mayWatchRecord(env.session, record) ? admitted.records : refused.refusedRecords) + .push_back(record); + } } - for (auto& topic : body->instances) { - // No instance id and no owner: the instance may not exist yet, - // and a shared one has no owner, so what admits the topic is - // the authorizer's rule for the model type. - bool const allowed = _authorizer->authorizeInstance(env.session, topic.modelType, {}, 0, {}); - (allowed ? admitted.instances : refused.refusedInstances).push_back(std::move(topic)); + std::set> seenInstances; + for (auto const& topic : body->instances) { + if (seenInstances.emplace(topic.modelType, topic.primary).second) { + (mayWatchInstance(env.session, topic.modelType) ? admitted.instances : refused.refusedInstances) + .push_back(topic); + } } - _watches.add(cid, admitted, detail::Watch{.context = env.session, .principal = env.session.principal}); + auto overCap = + _watches.add(cid, admitted, detail::Watch{.context = env.session, .principal = env.session.principal}); + std::ranges::move(overCap.refusedRecords, std::back_inserter(refused.refusedRecords)); + std::ranges::move(overCap.refusedInstances, std::back_inserter(refused.refusedInstances)); } std::string json; (void)glz::write_json(refused, json); reply(::morph::wire::encode(::morph::wire::makeOk(env.callId, std::move(json)))); } + /// @brief Answers an admission question through the watch table's cache. + /// On the server strand. + /// + /// Cached per verified principal and token, since two tokens of one + /// principal may carry different claims. A caller with no verified + /// principal is judged afresh each time: an authorizer that does not + /// authenticate may still tell such callers apart by their token. + /// @tparam Judge Callable `() -> bool`, the uncached answer. + /// @param session The `watch` envelope's session, its principal the verified one. + /// @param kind What @p name names. + /// @param name A registered record's wire name or model type name. + /// @param judge Decides when no answer is cached. + /// @return The answer. + template + [[nodiscard]] bool cachedAdmission(const ::morph::session::Context& session, detail::AdmissionTopic kind, + std::string_view name, Judge judge) { + bool const cacheable = !session.principal.empty(); + if (cacheable) { + if (auto const cached = _watches.admission(session, kind, name)) { + return *cached; + } + } + bool const admitted = judge(); + if (cacheable) { + _watches.noteAdmission(session, kind, name, admitted); + } + return admitted; + } + /// @brief Whether @p session's principal may watch record type @p record: /// whether it may run at least one action whose result holds it. /// On the server strand. /// - /// The answer for a registered record is cached in the watch table per - /// verified principal. A caller with no verified principal is judged - /// afresh each time, since the authorizer may tell such callers apart by - /// something other than a principal. A name no record has is refused - /// without caching, so a peer cannot grow the cache with names it makes up. + /// Each holding action is authorized as `authorize(session, modelType, + /// actionId)` against every model type it is registered under in this + /// server's dispatcher; which actions hold the record comes from the + /// process effects registry. A name no record has is refused without + /// caching, so a peer cannot grow the cache with names it makes up. /// @param session The `watch` envelope's session, its principal the verified one. /// @param record The record's wire name. /// @return True when the record may be watched. /// @throws morph::model::EffectsRegistrationError when the registered /// records and effects are invalid. [[nodiscard]] bool mayWatchRecord(const ::morph::session::Context& session, std::string_view record) { - bool const cacheable = !session.principal.empty(); - if (auto const cached = cacheable ? _watches.admission(session.principal, record) : std::nullopt) { - return *cached; - } if (::morph::effects::recordNamed(record) == nullptr) { return false; } - bool admitted = false; - for (auto const& actionId : ::morph::effects::actionsHolding(record)) { - for (auto const& modelType : _dispatcher.modelsOf(actionId)) { - if (_authorizer->authorize(session, modelType, actionId)) { - admitted = true; - break; + return cachedAdmission(session, detail::AdmissionTopic::record, record, [&] { + for (auto const& actionId : ::morph::effects::actionsHolding(record)) { + for (auto const& modelType : _dispatcher.modelsOf(actionId)) { + if (_authorizer->authorize(session, modelType, actionId)) { + return true; + } } } - if (admitted) { - break; - } - } - if (cacheable) { - _watches.noteAdmission(session.principal, record, admitted); + return false; + }); + } + + /// @brief Whether @p session's principal may watch keyed instances of + /// @p modelType. On the server strand. + /// + /// Three conditions: the model type is registered here (refused without + /// caching otherwise); the principal may run at least one of its actions, + /// `authorize(session, modelType, actionId)`, cached as for a record; and + /// `authorizeInstance(session, modelType, {}, 0, "")` admits it. The last + /// carries no instance id, because the instance may not exist yet, and no + /// owner, because a shared instance has none. + /// @param session The `watch` envelope's session, its principal the verified one. + /// @param modelType The topic's model type. + /// @return True when the instance topic may be watched. + [[nodiscard]] bool mayWatchInstance(const ::morph::session::Context& session, std::string_view modelType) { + if (!_registry.contains(modelType)) { + return false; } - return admitted; + bool const mayRun = cachedAdmission(session, detail::AdmissionTopic::modelType, modelType, [&] { + return std::ranges::any_of(_dispatcher.actionsOf(modelType), [&](std::string const& actionId) { + return _authorizer->authorize(session, modelType, actionId); + }); + }); + return mayRun && _authorizer->authorizeInstance(session, modelType, {}, 0, {}); } /// @brief Registers a drain waiter, or answers it at once. On the server strand. diff --git a/include/morph/core/wire.hpp b/include/morph/core/wire.hpp index f257cea73..d87766ae0 100644 --- a/include/morph/core/wire.hpp +++ b/include/morph/core/wire.hpp @@ -655,7 +655,8 @@ inline constexpr std::string_view kWatchKind = "watch"; /// @brief The envelope kind that removes topics from a connection's watch set. inline constexpr std::string_view kUnwatchKind = "unwatch"; -/// @brief The most topics (records or instances) one `watch`, `unwatch` or `changes` body may list. +/// @brief The most topics (records or instances) one `watch`, `unwatch` or `changes` body may list, and +/// the most topics, records and instances together, one connection may watch. /// /// A watch set is one entry per record type and open instance of a screen, so a list past this is a /// malformed or hostile peer rather than a large application, and the bound keeps its allocation finite. diff --git a/tests/test_push_watch.cpp b/tests/test_push_watch.cpp index 64f508bbd..06763bdaf 100644 --- a/tests/test_push_watch.cpp +++ b/tests/test_push_watch.cpp @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -77,25 +78,49 @@ using ::morph::backend::RemoteServer; constexpr std::string_view kSecretModel = "T_WatchSecretModel"; -/// Verifies a session by its token, which is the principal itself, and refuses -/// every action and instance of the secret model. +/// What one `authorizeInstance` call was asked. +struct InstanceQuestion { + std::string modelType; + std::string actionType; + std::uint64_t modelId{}; + std::string owner; +}; + +/// Verifies a session by its token: the principal is the token up to a `:`, +/// so `alice` and `alice:scoped` are two tokens of one principal. Refuses every +/// action and instance of the secret model, and every action to a `:scoped` +/// token. struct SecretGuard final : ::morph::session::IAuthorizer { mutable std::atomic authorizeCalls{0}; + mutable std::mutex mtx; + mutable std::vector instanceQuestions; - [[nodiscard]] bool authorize(const ::morph::session::Context& /*ctx*/, std::string_view modelType, + [[nodiscard]] bool authorize(const ::morph::session::Context& ctx, std::string_view modelType, std::string_view /*actionType*/) const override { authorizeCalls.fetch_add(1); - return modelType != kSecretModel; + return modelType != kSecretModel && !ctx.token.ends_with(":scoped"); } [[nodiscard]] std::optional authenticate(const ::morph::session::Context& ctx) const override { if (ctx.token.empty()) { return std::nullopt; } - return ctx.token; + return ctx.token.substr(0, ctx.token.find(':')); } [[nodiscard]] bool authorizeInstance(const ::morph::session::Context& /*ctx*/, std::string_view modelType, - std::string_view /*actionType*/, std::uint64_t /*modelId*/, - std::string_view /*ownerPrincipal*/) const override { + std::string_view actionType, std::uint64_t modelId, + std::string_view ownerPrincipal) const override { + std::scoped_lock const lock{mtx}; + instanceQuestions.push_back( + {std::string{modelType}, std::string{actionType}, modelId, std::string{ownerPrincipal}}); + return modelType != kSecretModel; + } +}; + +/// Overrides only `authorize`, refusing the secret model's actions: its +/// `authorizeInstance` is the allow-all default. +struct AuthorizeOnly final : ::morph::session::IAuthorizer { + [[nodiscard]] bool authorize(const ::morph::session::Context& /*ctx*/, std::string_view modelType, + std::string_view /*actionType*/) const override { return modelType != kSecretModel; } }; @@ -103,7 +128,10 @@ struct SecretGuard final : ::morph::session::IAuthorizer { struct Rig { ::morph::exec::ThreadPoolExecutor pool{2}; std::shared_ptr guard = std::make_shared(); - std::shared_ptr server = std::make_shared(pool, guard); + std::shared_ptr server; + + explicit Rig(std::shared_ptr<::morph::session::IAuthorizer> authorizer = nullptr) + : server{std::make_shared(pool, authorizer ? std::move(authorizer) : guard)} {} /// Sends @p kind with @p body on @p cid, signed as @p principal, and returns the reply. ::morph::wire::Envelope send(ConnectionId cid, std::string_view kind, ::morph::wire::WatchBody const& body, @@ -322,3 +350,131 @@ TEST_CASE("record_admission_cached_per_principal", "[push][watch]") { (void)rig.watch(again, records({"T_WatchRow"})); CHECK(rig.guard->authorizeCalls.load() > afterAnonymous + 1); } + +TEST_CASE("admission_cache_keys_on_the_token", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + auto const scoped = rig.server->openConnection(); + + // alice's full token is admitted, and the answer cached while she watches. + CHECK(rig.watch(cid, records({"T_WatchRow"}), "alice").refusedRecords.empty()); + ::morph::wire::WatchBody body = records({"T_WatchRow"}); + body.instances = {{.modelType = "T_WatchRowModel", .primary = "a"}}; + CHECK(rig.watch(cid, body, "alice").refusedInstances.empty()); + + // Her scoped token verifies to the same principal but carries other claims: judged on its own. + auto const reply = rig.watch(scoped, body, "alice:scoped"); + CHECK(reply.refusedRecords == std::vector{"T_WatchRow"}); + REQUIRE(reply.refusedInstances.size() == 1U); + CHECK(rig.recordWatchers("T_WatchRow") == std::vector{cid}); +} + +TEST_CASE("instance_topic_needs_a_runnable_action", "[push][watch]") { + Rig rig{std::make_shared()}; + auto const cid = rig.server->openConnection(); + + ::morph::wire::WatchBody body = instance(std::string{kSecretModel}, "k"); + body.instances.push_back({.modelType = "T_WatchRowModel", .primary = "k"}); + auto const reply = rig.watch(cid, body); + + REQUIRE(reply.refusedInstances.size() == 1U); + CHECK(reply.refusedInstances.front().modelType == kSecretModel); + CHECK(rig.instanceWatchers(kSecretModel, "k").empty()); + CHECK(rig.instanceWatchers("T_WatchRowModel", "k") == std::vector{cid}); +} + +TEST_CASE("instance_topic_of_unregistered_type_refused", "[push][watch]") { + Rig rig{std::make_shared()}; + auto const cid = rig.server->openConnection(); + + auto const reply = rig.watch(cid, instance("T_WatchNoSuchModel", "k")); + + REQUIRE(reply.refusedInstances.size() == 1U); + CHECK(reply.refusedInstances.front().modelType == "T_WatchNoSuchModel"); + CHECK(rig.instanceWatchers("T_WatchNoSuchModel", "k").empty()); +} + +TEST_CASE("instance_admission_asks_without_instance_or_owner", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + + (void)rig.watch(cid, instance("T_WatchRowModel", "k")); + + std::scoped_lock const lock{rig.guard->mtx}; + REQUIRE(rig.guard->instanceQuestions.size() == 1U); + auto const& asked = rig.guard->instanceQuestions.front(); + CHECK(asked.modelType == "T_WatchRowModel"); + CHECK(asked.actionType.empty()); + CHECK(asked.modelId == 0U); + CHECK(asked.owner.empty()); +} + +TEST_CASE("connection_watch_cap", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + + ::morph::wire::WatchBody full; + for (std::size_t index = 0; index < ::morph::wire::kMaxWatchTopics; ++index) { + full.instances.push_back({.modelType = "T_WatchRowModel", .primary = std::to_string(index)}); + } + CHECK(rig.watch(cid, full).refusedInstances.empty()); + CHECK(rig.onStrand([cid](auto const& table) { return table.topicCount(cid); }) == ::morph::wire::kMaxWatchTopics); + + // At the cap, a new topic is refused; one already watched is not. + ::morph::wire::WatchBody more = records({"T_WatchRow"}); + more.instances = {{.modelType = "T_WatchRowModel", .primary = "0"}}; + auto const refused = rig.watch(cid, more); + CHECK(refused.refusedRecords == std::vector{"T_WatchRow"}); + CHECK(refused.refusedInstances.empty()); + CHECK(rig.recordWatchers("T_WatchRow").empty()); + + // A topic freed by unwatch makes room again. + CHECK(rig.send(cid, ::morph::wire::kUnwatchKind, instance("T_WatchRowModel", "0")).kind == "ok"); + CHECK(rig.watch(cid, records({"T_WatchRow"})).refusedRecords.empty()); + CHECK(rig.recordWatchers("T_WatchRow") == std::vector{cid}); + CHECK(rig.onStrand([cid](auto const& table) { return table.topicCount(cid); }) == ::morph::wire::kMaxWatchTopics); +} + +TEST_CASE("duplicates_in_one_body_judged_once", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + + ::morph::wire::WatchBody body = records({"T_WatchRow", "T_WatchSecret", "T_WatchRow", "T_WatchSecret"}); + body.instances = {{.modelType = std::string{kSecretModel}, .primary = "k"}, + {.modelType = "T_WatchRowModel", .primary = "k"}, + {.modelType = std::string{kSecretModel}, .primary = "k"}, + {.modelType = "T_WatchRowModel", .primary = "k"}}; + auto const reply = rig.watch(cid, body); + + CHECK(reply.refusedRecords == std::vector{"T_WatchSecret"}); + REQUIRE(reply.refusedInstances.size() == 1U); + CHECK(reply.refusedInstances.front().modelType == kSecretModel); + CHECK(rig.onStrand([cid](auto const& table) { return table.topicCount(cid); }) == 2U); +} + +TEST_CASE("replace_with_refused_topics", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + (void)rig.watch(cid, records({"T_WatchRow"})); + + ::morph::wire::WatchBody full = records({"T_WatchSecret"}, true); + full.instances = {{.modelType = "T_WatchRowModel", .primary = "a"}}; + auto const reply = rig.watch(cid, full); + + CHECK(reply.refusedRecords == std::vector{"T_WatchSecret"}); + CHECK(rig.recordWatchers("T_WatchRow").empty()); + CHECK(rig.recordWatchers("T_WatchSecret").empty()); + CHECK(rig.instanceWatchers("T_WatchRowModel", "a") == std::vector{cid}); + CHECK(rig.onStrand([cid](auto const& table) { return table.topicCount(cid); }) == 1U); +} + +TEST_CASE("unwatch_of_unwatched_topic_is_ok", "[push][watch]") { + Rig rig; + auto const bare = rig.server->openConnection(); + auto const cid = rig.server->openConnection(); + (void)rig.watch(cid, records({"T_WatchRow"})); + + CHECK(rig.send(bare, ::morph::wire::kUnwatchKind, records({"T_WatchRow"})).kind == "ok"); + CHECK(rig.send(cid, ::morph::wire::kUnwatchKind, instance("T_WatchRowModel", "never")).kind == "ok"); + CHECK(rig.recordWatchers("T_WatchRow") == std::vector{cid}); +} From 94d8dbba6fabd430948ed3f3753434cb39d2aaf7 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 18:44:11 +0200 Subject: [PATCH 32/68] core: a successful execute tells every watching connection what changed Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/backend.md | 45 ++++- include/morph/core/remote.hpp | 162 ++++++++++++++- tests/test_push_watch.cpp | 366 ++++++++++++++++++++++++++++++++++ 3 files changed, 567 insertions(+), 6 deletions(-) diff --git a/docs/spec/core/backend.md b/docs/spec/core/backend.md index 709e444b4..f3817b629 100644 --- a/docs/spec/core/backend.md +++ b/docs/spec/core/backend.md @@ -48,6 +48,7 @@ and react to backend changes. - [Connection scopes](#connection-scopes) - [Writing to a connection](#writing-to-a-connection) - [Watches](#watches) +- [Fan-out](#fan-out) - [`SimulatedRemoteBackend` — adapter for testing](#simulatedremotebackend--adapter-for-testing) - [`QtWebSocketBackend` — client-side WebSocket transport](#qtwebsocketbackend--client-side-websocket-transport) - [`QtWebSocketServer` — server-side WebSocket transport](#qtwebsocketserver--server-side-websocket-transport) @@ -956,7 +957,7 @@ limit: | Field | Default | Enforcement | |---|---|---| -| `executeTimeout` | `0` (disabled) | A timer arms when an `execute` is admitted. If it fires first, the server replies `err "timeout"` and the eventual strand result (if the model finishes later) is discarded via a shared once-flag — `handle()`'s reply-exactly-once contract holds regardless of which path resolves first. An ordinary handler keeps running to completion on its strand; morph never interrupts it. A handler returning `core::async::Task` is also asked to stop, through its stop token, and unwinds at its next stop-aware `co_await` (see `docs/spec/core/coroutines.md`, "Execute deadlines"). | +| `executeTimeout` | `0` (disabled) | A timer arms when an `execute` is admitted. If it fires first, the server replies `err "timeout"` and the eventual strand result (if the model finishes later) is not sent as a reply, via a shared once-flag; if it succeeds, its changes still reach the watching connections, the caller included ([Fan-out](#fan-out)) — `handle()`'s reply-exactly-once contract holds regardless of which path resolves first. An ordinary handler keeps running to completion on its strand; morph never interrupts it. A handler returning `core::async::Task` is also asked to stop, through its stop token, and unwinds at its next stop-aware `co_await` (see `docs/spec/core/coroutines.md`, "Execute deadlines"). | | `maxLiveModels` | `0` (unbounded) | Checked on the server strand before `register` constructs a new instance, and again after the construction and before the insert (the construction is host code that may itself register through `handleInline`); over the cap → `err "too many models"`. Exact: nothing else inserts between the check and the insert. A shared `register`/`attach` that finds its key already live takes a reference and is not counted against the cap. | | `maxInFlightExecutes` | `0` (unbounded) | The in-flight count, server-strand state, incremented when an `execute` is admitted and decremented (posted back to the strand) when its reply is sent — success, exception, or timeout, whichever resolves the call first; at the cap → `err "server busy"`, no dispatch. Exact, for the same reason. | @@ -1167,6 +1168,45 @@ execute path reads it. The UI layer sends them; nothing on the server outside cannot be written to is sent no events, and its client sends the whole set again when it reconnects. +### Fan-out + +After an `execute` succeeds, the server tells every connection that watches +what it changed. Every executing message takes this path: the `execute` kind, +offline replays (which arrive as executes), and anything else that dispatches +through `dispatchExecute`. + +- **The model strand derives.** `finishRemote`, on success only and before the + reply, calls `effects::deriveChanges(actionType, result, ranOn)`. `ranOn` is + the instance the action ran on: its model type, its id, and, for a keyed + shared instance, the primary of the directory key it held when the execute + was admitted (`RemoteRun::sharedKey`; empty for a private instance). Changes + that are not empty are posted to the server strand as + `fanOut(changes, caller)`, where `caller` is the execute's connection + (`RemoteRun::cid`). The reply is written exactly as without fan-out. A failed + execute derives nothing. A derivation that throws is logged; the reply and + the action gate's release still follow. +- **The server strand finds the recipients.** A record topic goes to every + connection `watchersOf(record)` names, and a keyed shared instance's topic to + every connection `watchersOf(modelType, primary)` names, **the caller + included**. The caller may never see its own reply: an execute the + `executeTimeout` answered first but that succeeded, a reply the client + discarded, a replay whose reply went to the offline worker. A private + instance's topic goes only to the connection whose scope holds the instance, + whether or not it is the caller, and needs no watch. A connection that has + not negotiated events (`eventsNegotiated(cid)`) is sent nothing. +- **One event per recipient.** Each recipient gets one `changes` event whose + `ChangesBody` holds only the topics it is sent: its watched records, with the + keys derived, and its instance topics as `InstanceTopic{modelType, primary, + instance}`. Before it is encoded, the body passes through + `RemoteServer::reauthorise(cid, body)`, the one place a recipient's topics + are narrowed; it keeps every topic, since each watch was authorised when it + was admitted. The event is then written as `post` writes: dropped above + `wire::kMaxEnvelopeBytes`, held back behind a full backlog. +- **The registries are read at construction.** Every `RemoteServer` + constructor reads the record and effects registries once, so a process whose + declarations are invalid fails with `EffectsRegistrationError` when the server + is built, not at its first successful execute. + ### Graceful shutdown (`beginShutdown()` / `drainedWithin()`) `beginShutdown()` enters shutdown: every subsequent `register`, `attach` and @@ -2270,7 +2310,7 @@ server: each call is a loop task, so the loop serialises them. | `RemoteServer(workerPool, dispatcher, registry)` | Allow-all authorizer, default `ServerConfig`. | | `RemoteServer(workerPool, authorizer, dispatcher, registry)` | Custom authorizer; null → allow-all. Default `ServerConfig`. | | `RemoteServer(workerPool, config, dispatcher, registry)` | Allow-all authorizer, `config` fixed for the server's life. Throws `std::invalid_argument` if `config.minProtocolVersion > config.maxProtocolVersion`. Calls `config.healthHandler` once before returning. | -| `RemoteServer(workerPool, authorizer, config, dispatcher, registry)` | Both. | +| `RemoteServer(workerPool, authorizer, config, dispatcher, registry)` | Both. Every constructor reads the record and effects registries and throws `EffectsRegistrationError` if they are invalid. | | `handle(msg, reply)` | Async: decodes on the calling thread, posts to the server strand, calls `reply` once from a pool thread. Unscoped (`cid == 0`). Callable from any thread; calls from one thread are handled in call order — see [Per-model execute ordering](#per-model-execute-ordering). | | `handle(msg, reply, cid)` | Like `handle(msg, reply)`, additionally attributing any `register` in `msg` to connection `cid`'s scope. `cid == 0` behaves exactly like the two-argument overload. | | `handleInline(msg)` | Sync: runs the control envelope on the server strand — posted and waited for, or inline when already on it — and returns the reply JSON. **Rejects `execute`** — returns an `err` reply without dispatching, because an `execute` reply is produced asynchronously after this call returns. Needs a pool thread other than the caller's free to run the strand. Unscoped. | @@ -2280,6 +2320,7 @@ server: each call is a loop task, so the loop serialises them. | `post(cid, envelope)` | Any thread. Above `wire::kMaxEnvelopeBytes`: dropped and counted (`Metric::eventOversize`). On the strand: written through `cid`'s writer, or held back — dropped, with one `resync` owed — while `backlog()` exceeds `kEventBacklogHigh` or a `resync` is still owed. No writer: dropped. | | `connectionDrained(cid)` | Any thread; posts. Sends the owed `resync` event, if any, and lets events flow again. What a transport calls when the backlog falls below `kEventBacklogLow`. | | `watchTable() const` | Server strand only (debug-asserted). The `detail::WatchTable` of every open connection's watches, read-only. See [Watches](#watches). | +| `fanOut(changes, caller)` | Server strand only (debug-asserted). Sends each connection that watches a topic of `changes`, or whose scope holds a private instance in it, one `changes` event with the topics it is sent; `caller` included, connections without negotiated events skipped. A successful `execute` posts it. See [Fan-out](#fan-out). | | `eventsNegotiated(cid) const` | Server strand only (debug-asserted). Whether `cid` negotiated events: the `ConnectionFacts::events` a service set for it. `false` for `0`, an unknown `cid` or a closed one. | | `payloadCompleteness()` | The `ServerConfig::payloadCompleteness` given at construction. | | `strand()` | The server strand, as an `exec::IExecutor`: `runningOn(server.strand())` is true inside every task on the server's state. | diff --git a/include/morph/core/remote.hpp b/include/morph/core/remote.hpp index 669bec3b3..06e9635eb 100644 --- a/include/morph/core/remote.hpp +++ b/include/morph/core/remote.hpp @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -29,6 +30,7 @@ #include "../core/backend.hpp" #include "../journal/action_log.hpp" #include "../session/session.hpp" +#include "changes.hpp" #include "completion.hpp" #include "connection_writer.hpp" #include "detail/owner_probe.hpp" @@ -54,6 +56,9 @@ struct LimitPolicy { /// sends an `err "timeout"` reply and discards the eventual strand /// result. `0` = no timeout (today's behavior). /// + /// A handler that succeeds after the timeout answered still tells the + /// connections watching what it changed, the caller included, by event. + /// /// An ordinary handler is never interrupted — it keeps running to /// completion on its strand, so this bounds the *caller's wait*, not the /// model. A handler returning `core::async::Task` is also asked to stop: its @@ -365,6 +370,9 @@ class RemoteServer : public std::enable_shared_from_this { /// singleton. Borrowed: it must outlive this server. /// @param registry Model factory registry; defaults to the process-level /// singleton. Borrowed: it must outlive this server. + /// @throws morph::model::EffectsRegistrationError if the process's record + /// and effects declarations are invalid: the server reads them + /// here, so such a server is never constructed. explicit RemoteServer(::morph::exec::IExecutor& workerPool MORPH_LIFETIMEBOUND, ::morph::model::detail::ActionDispatcher& dispatcher MORPH_LIFETIMEBOUND = ::morph::model::detail::defaultDispatcher(), @@ -383,6 +391,9 @@ class RemoteServer : public std::enable_shared_from_this { /// singleton. Borrowed: it must outlive this server. /// @param registry Model factory registry; defaults to the process-level /// singleton. Borrowed: it must outlive this server. + /// @throws morph::model::EffectsRegistrationError if the process's record + /// and effects declarations are invalid: the server reads them + /// here, so such a server is never constructed. RemoteServer(::morph::exec::IExecutor& workerPool MORPH_LIFETIMEBOUND, std::shared_ptr<::morph::session::IAuthorizer> authorizer, ::morph::model::detail::ActionDispatcher& dispatcher MORPH_LIFETIMEBOUND = @@ -404,6 +415,9 @@ class RemoteServer : public std::enable_shared_from_this { /// @throws std::invalid_argument if `config.minProtocolVersion` exceeds /// `config.maxProtocolVersion`, or `config.services` holds a null /// entry. + /// @throws morph::model::EffectsRegistrationError if the process's record + /// and effects declarations are invalid: the server reads them + /// here, so such a server is never constructed. RemoteServer(::morph::exec::IExecutor& workerPool MORPH_LIFETIMEBOUND, ServerConfig config, ::morph::model::detail::ActionDispatcher& dispatcher MORPH_LIFETIMEBOUND = ::morph::model::detail::defaultDispatcher(), @@ -429,6 +443,9 @@ class RemoteServer : public std::enable_shared_from_this { /// @throws std::invalid_argument if `config.minProtocolVersion` exceeds /// `config.maxProtocolVersion`, or `config.services` holds a null /// entry. + /// @throws morph::model::EffectsRegistrationError if the process's record + /// and effects declarations are invalid: the server reads them + /// here, so such a server is never constructed. RemoteServer(::morph::exec::IExecutor& workerPool MORPH_LIFETIMEBOUND, std::shared_ptr<::morph::session::IAuthorizer> authorizer, ServerConfig config, ::morph::model::detail::ActionDispatcher& dispatcher MORPH_LIFETIMEBOUND = @@ -444,6 +461,10 @@ class RemoteServer : public std::enable_shared_from_this { _executeTimeouts{_config.limits.executeTimeout.count() > 0 ? std::make_unique<::morph::async::detail::TimeoutScheduler>() : nullptr} { + // The first read of the record and effects registries: an invalid + // declaration fails the server's construction rather than its first + // successful execute, whose changes are derived from them. + static_cast(::morph::effects::detail::resolved()); if (_config.healthHandler) { _config.healthHandler(HealthStatus{.ready = true, .liveModels = 0, .inFlight = 0}); } @@ -661,10 +682,7 @@ class RemoteServer : public std::enable_shared_from_this { /// @param cid Connection to write to. /// @param envelope The encoded envelope (via `wire::encode`). void post(ConnectionId cid, std::string envelope) { - if (envelope.size() > ::morph::wire::kMaxEnvelopeBytes) { - ::morph::observe::detail::emitMetric(::morph::observe::Metric::eventOversize, 1.0); - ::morph::log::logWarn("[post] envelope for connection {} dropped: {} bytes exceeds {}", cid, - envelope.size(), ::morph::wire::kMaxEnvelopeBytes); + if (oversized(cid, envelope)) { return; } if (cid == 0) { @@ -721,6 +739,61 @@ class RemoteServer : public std::enable_shared_from_this { return _watches; } + /// @brief Tells every connection watching what @p changes names that it + /// changed, as one `changes` event per connection. + /// + /// Server strand only: call it from a task on `strand()`; a debug build + /// asserts that. A successful `execute` posts its changes here. + /// + /// A record topic, and the instance topic of a keyed shared instance, go + /// to every connection watching them, @p caller included: the caller + /// may never see its own reply (it timed out, the client discarded it, or + /// it went to an offline replay). A private instance's topic goes to the + /// connection whose scope holds that instance, which needs no watch, and + /// to no other. A connection that has not negotiated events is sent + /// nothing. Each connection's event holds only the topics it is sent, and + /// passes through `post`'s backlog hold-back and size cap. + /// @param changes What one execute changed, from `effects::deriveChanges`. + /// @param caller Connection the execute arrived on; `0` for unscoped. + void fanOut(::morph::effects::Changes changes, ConnectionId caller) { + assert(_strand.runningHere() && "RemoteServer::fanOut runs on the server strand"); + noteOwner("RemoteServer::fanOut"); + std::map bodies; + for (auto const& change : changes.records) { + for (ConnectionId const cid : _watches.watchersOf(change.record)) { + if (eventsNegotiated(cid)) { + bodies[cid].records.push_back(change); + } + } + } + for (auto& change : changes.instances) { + auto const recipients = change.primary.empty() + ? scopesHolding(::morph::exec::detail::ModelId{change.instance}, caller) + : _watches.watchersOf(change.modelType, change.primary); + ::morph::wire::InstanceTopic const topic{ + .modelType = std::move(change.modelType), .primary = std::move(change.primary), + .instance = change.instance}; + for (ConnectionId const cid : recipients) { + if (eventsNegotiated(cid)) { + bodies[cid].instances.push_back(topic); + } + } + } + for (auto& [cid, body] : bodies) { + reauthorise(cid, body); + if (body.records.empty() && body.instances.empty()) { + continue; + } + std::string payload; + (void)glz::write_json(body, payload); + auto envelope = + ::morph::wire::encode(::morph::wire::makeEvent(::morph::wire::kChangesTopic, std::move(payload))); + if (!oversized(cid, envelope)) { + postHere(cid, std::move(envelope)); + } + } + } + /// @brief The server strand, as an executor. /// /// What `runningOn(server.strand())` asks about: true inside every task the @@ -932,6 +1005,52 @@ class RemoteServer : public std::enable_shared_from_this { return _facts[cid]; } + /// @brief Whether @p envelope exceeds `wire::kMaxEnvelopeBytes`; such an + /// envelope is counted as `Metric::eventOversize`, logged, and not sent. + /// @param cid Connection it was meant for. + /// @param envelope The encoded envelope. + /// @return `true` when it must be dropped. + static bool oversized(ConnectionId cid, std::string const& envelope) { + if (envelope.size() <= ::morph::wire::kMaxEnvelopeBytes) { + return false; + } + ::morph::observe::detail::emitMetric(::morph::observe::Metric::eventOversize, 1.0); + ::morph::log::logWarn("[post] envelope for connection {} dropped: {} bytes exceeds {}", cid, envelope.size(), + ::morph::wire::kMaxEnvelopeBytes); + return true; + } + + /// @brief The connections whose scope holds @p mid. On the server strand. + /// + /// A private instance is held by the one connection that registered it; + /// @p caller is asked first because it usually is that connection. + /// @param mid Instance. + /// @param caller Connection to ask first; `0` for none. + /// @return The connections, at most one for a private instance. + [[nodiscard]] std::vector scopesHolding(::morph::exec::detail::ModelId mid, + ConnectionId caller) const { + if (auto const own = _connectionScopes.find(caller); + own != _connectionScopes.end() && own->second.contains(mid)) { + return {caller}; + } + std::vector holders; + for (auto const& [cid, scope] : _connectionScopes) { + if (scope.contains(mid)) { + holders.push_back(cid); + } + } + return holders; + } + + /// @brief Narrows @p body to the topics connection @p cid may still be + /// sent. On the server strand. + /// + /// Every topic is kept: each was authorised when its watch was admitted, + /// and a private instance's topic goes only to the connection holding it. + /// @param cid Recipient. + /// @param body The topics `fanOut` chose for it; narrowed in place. + static void reauthorise([[maybe_unused]] ConnectionId cid, [[maybe_unused]] ::morph::wire::ChangesBody& body) {} + /// @brief Body of `post`. On the server strand. /// @param cid Connection to write to. /// @param envelope The encoded envelope, within the size cap. @@ -2001,6 +2120,8 @@ class RemoteServer : public std::enable_shared_from_this { run.timeoutHandle = timeoutHandle; run.stopSource = std::move(stopSource); run.mid = mid; + run.cid = cid; + run.sharedKey = inst->sharedKey; // Posted in this strand's order, which is the order `handle()` was // called in; the model's strand runs its tasks in the order they were // posted. That is the whole of per-model execute ordering. @@ -2067,6 +2188,10 @@ class RemoteServer : public std::enable_shared_from_this { /// Null without an `executeTimeout`. std::shared_ptr<::core::async::StopSource> stopSource; ::morph::exec::detail::ModelId mid{}; + /// Connection the execute arrived on; `0` for unscoped. + ConnectionId cid{0}; + /// The instance's directory key at admission; empty for a private instance. + std::optional sharedKey; std::chrono::steady_clock::time_point start; ::morph::observe::SpanId spanId{}; }; @@ -2142,6 +2267,34 @@ class RemoteServer : public std::enable_shared_from_this { } } + /// Derives what a successful execute changed and posts it to `fanOut` on + /// the server strand. On the model's strand, before the reply, so it runs + /// whether or not the caller is still waiting for that reply. + /// + /// A failure here is logged and goes no further: the reply and the action + /// gate's release still follow. + /// @param run The finished execute. + /// @param result Its result JSON. + static void postChanges(RemoteRun const& run, std::string_view result) { + try { + auto changes = ::morph::effects::deriveChanges( + run.env.actionType, result, + ::morph::effects::InstanceChange{.modelType = run.env.modelType, + .instance = run.mid.v, + .primary = run.sharedKey ? run.sharedKey->second : std::string{}}); + if (changes.empty()) { + return; + } + run.self->_strand.postTask([self = run.self, changes = std::move(changes), caller = run.cid]() mutable { + self->fanOut(std::move(changes), caller); + }); + } catch (const std::exception& exc) { + ::morph::log::logError("[fanOut] changes of {} not derived: {}", run.env.actionType, exc.what()); + } catch (...) { + ::morph::log::logError("[fanOut] changes of {} not derived: unknown exception", run.env.actionType); + } + } + /// Records a finished execute, replies, and leaves the action gate so the /// next execute on the instance can start. On the strand. static void finishRemote(RemoteRun& run, std::string result, const std::exception_ptr& error) { @@ -2167,6 +2320,7 @@ class RemoteServer : public std::enable_shared_from_this { {{"modelType", run.env.modelType}, {"actionType", run.env.actionType}}}; ::morph::observe::detail::emitMetric(::morph::observe::Metric::executeLatencyMs, elapsedMs, tags); if (succeeded) { + postChanges(run, result); run.complete(::morph::wire::encode(::morph::wire::makeOk(run.env.callId, std::move(result)))); } else { ::morph::observe::detail::emitMetric(::morph::observe::Metric::executeErrors, 1.0, tags); diff --git a/tests/test_push_watch.cpp b/tests/test_push_watch.cpp index 06763bdaf..9c973a846 100644 --- a/tests/test_push_watch.cpp +++ b/tests/test_push_watch.cpp @@ -5,11 +5,14 @@ #include #include +#include #include #include #include +#include #include #include +#include #include #include #include @@ -20,9 +23,12 @@ #include #include #include +#include #include #include +#include "test_support.hpp" + // Glaze reflects member names through an external-linkage object, so these types cannot sit in an // anonymous namespace. namespace push_watch_test { @@ -60,6 +66,61 @@ struct SecretModel { SecretList execute(ListSecrets const& /*action*/) { return {}; } }; +struct FanRow { + std::int64_t id{}; + std::string name; +}; + +struct FanRowList { + std::vector rows; +}; + +struct FanOther { + std::int64_t id{}; +}; + +struct FanOtherList { + std::vector rows; +}; + +/// Edits one row and returns it. +struct EditFanRow { + std::int64_t id{}; +}; + +/// Edits one row after `delayMs`, so an execute timeout can answer first. +struct SlowEditFanRow { + std::int64_t id{}; + int delayMs{}; +}; + +/// Declares it edits rows, then fails. +struct FailFanRow { + std::int64_t id{}; +}; + +/// Reads the other record type; only there so a connection can watch it. +struct ListFanOthers { + int limit{}; +}; + +/// Changes the instance's own state. +struct TouchFan { + int by{}; +}; + +struct FanModel { + int touched = 0; + FanRowList execute(EditFanRow const& action) { return {.rows = {{.id = action.id, .name = "edited"}}}; } + FanRowList execute(SlowEditFanRow const& action) { + std::this_thread::sleep_for(std::chrono::milliseconds{action.delayMs}); + return {.rows = {{.id = action.id, .name = "slow"}}}; + } + FanRowList execute(FailFanRow const& /*action*/) { throw std::runtime_error("edit failed"); } + FanOtherList execute(ListFanOthers const& /*action*/) { return {}; } + int execute(TouchFan const& action) { return touched += action.by; } +}; + } // namespace push_watch_test using namespace push_watch_test; @@ -71,6 +132,19 @@ BRIDGE_REGISTER_ACTION(RowModel, ListRows, "T_WatchListRows") BRIDGE_REGISTER_MODEL(SecretModel, "T_WatchSecretModel") BRIDGE_REGISTER_ACTION(SecretModel, ListSecrets, "T_WatchListSecrets") +BRIDGE_REGISTER_RECORD(FanRow, "T_FanRow") +BRIDGE_REGISTER_RECORD(FanOther, "T_FanOther") +BRIDGE_REGISTER_MODEL(FanModel, "T_FanModel") +BRIDGE_REGISTER_ACTION(FanModel, EditFanRow, "T_FanEdit") +BRIDGE_REGISTER_EFFECTS(EditFanRow, morph::effects::updates) +BRIDGE_REGISTER_ACTION(FanModel, SlowEditFanRow, "T_FanSlowEdit") +BRIDGE_REGISTER_EFFECTS(SlowEditFanRow, morph::effects::updates) +BRIDGE_REGISTER_ACTION(FanModel, FailFanRow, "T_FanFail") +BRIDGE_REGISTER_EFFECTS(FailFanRow, morph::effects::updates) +BRIDGE_REGISTER_ACTION(FanModel, ListFanOthers, "T_FanListOthers") +BRIDGE_REGISTER_ACTION(FanModel, TouchFan, "T_FanTouch") +BRIDGE_REGISTER_EFFECTS(TouchFan, morph::effects::changesInstance) + namespace { using ::morph::backend::ConnectionId; @@ -478,3 +552,295 @@ TEST_CASE("unwatch_of_unwatched_topic_is_ok", "[push][watch]") { CHECK(rig.send(cid, ::morph::wire::kUnwatchKind, instance("T_WatchRowModel", "never")).kind == "ok"); CHECK(rig.recordWatchers("T_WatchRow") == std::vector{cid}); } + +// ── Fan-out: what a successful execute tells the watching connections ─────── + +namespace { + +constexpr std::string_view kFanModel = "T_FanModel"; +constexpr std::string_view kFanHello = "t-fan-hello"; + +/// Negotiates events for the connection a `t-fan-hello` arrives on, as the UI +/// service's `ui-hello` does. +class FanHello final : public ::morph::backend::IEnvelopeService { +public: + using IEnvelopeService::handle; + + [[nodiscard]] bool serves(std::string_view kind) const override { return kind == kFanHello; } + + [[nodiscard]] std::string handle(::morph::wire::Envelope const& request, + ::morph::session::IAuthorizer const& /*authorizer*/) override { + return ::morph::wire::encode(::morph::wire::makeOk(request.callId)); + } + + [[nodiscard]] std::string handle(::morph::wire::Envelope const& request, + ::morph::session::IAuthorizer const& authorizer, + ::morph::backend::ConnectionFacts& facts) override { + facts.events = true; + return handle(request, authorizer); + } +}; + +/// What the server wrote to one connection's writer: events only, since +/// replies go to the reply callback. +struct EventWire { + std::mutex mtx; + std::vector sent; + + ::morph::backend::ConnectionWriter writer() { + return {.write = + [this](std::string frame) { + std::scoped_lock const lock{mtx}; + sent.push_back(std::move(frame)); + }, + .backlog = [] { return std::size_t{0}; }}; + } + + std::vector snapshot() { + std::scoped_lock const lock{mtx}; + return sent; + } + + /// The `changes` payloads received, decoded. + std::vector<::morph::wire::ChangesBody> changes() { + std::vector<::morph::wire::ChangesBody> bodies; + for (auto const& frame : snapshot()) { + auto const env = ::morph::wire::decode(frame); + REQUIRE(env.kind == ::morph::wire::kEventKind); + ::morph::wire::EventBody event; + REQUIRE_FALSE(glz::read_json(event, env.body)); + REQUIRE(event.topic == ::morph::wire::kChangesTopic); + auto body = ::morph::wire::readChangesBody(event.payload.str); + REQUIRE(body); + bodies.push_back(std::move(*body)); + } + return bodies; + } +}; + +struct FanRig { + ::morph::exec::ThreadPoolExecutor pool{2}; + std::shared_ptr server; + std::map> wires; + + explicit FanRig(std::chrono::milliseconds executeTimeout = std::chrono::milliseconds{0}) { + ::morph::backend::ServerConfig config; + config.limits.executeTimeout = executeTimeout; + config.services.push_back(std::make_shared()); + server = std::make_shared(pool, std::move(config)); + } + + /// Opens a connection with a writer, negotiating events when @p events. + ConnectionId connect(bool events = true) { + auto const cid = server->openConnection(); + auto& wire = wires[cid]; + wire = std::make_unique(); + server->setConnectionWriter(cid, wire->writer()); + if (events) { + ::morph::wire::Envelope hello; + hello.kind = std::string{kFanHello}; + hello.callId = 1; + REQUIRE(::morph::wire::decode(server->handleInline(::morph::wire::encode(hello), cid)).kind == "ok"); + } + return cid; + } + + EventWire& wire(ConnectionId cid) { return *wires.at(cid); } + + void watch(ConnectionId cid, ::morph::wire::WatchBody const& body) { + ::morph::wire::Envelope env; + env.kind = std::string{::morph::wire::kWatchKind}; + env.callId = 2; + env.body = glz::write_json(body).value_or(std::string{}); + auto const reply = ::morph::wire::decode(server->handleInline(::morph::wire::encode(env), cid)); + REQUIRE(reply.kind == "ok"); + auto const parsed = ::morph::wire::readWatchReply(reply.body); + REQUIRE(parsed); + REQUIRE(parsed->refusedRecords.empty()); + REQUIRE(parsed->refusedInstances.empty()); + } + + /// Registers a private instance on @p cid, or a keyed shared one when @p primary is set. + std::uint64_t create(ConnectionId cid, std::string primary = {}) { + auto const env = primary.empty() ? ::morph::wire::makeRegister(std::string{kFanModel}) + : ::morph::wire::makeRegisterShared(std::string{kFanModel}, primary); + auto const reply = ::morph::wire::decode(server->handleInline(::morph::wire::encode(env), cid)); + REQUIRE(reply.kind == "ok"); + REQUIRE(reply.modelId != 0U); + return reply.modelId; + } + + /// Executes @p actionType with @p body on @p modelId through @p cid and + /// waits for the reply. + ::morph::wire::Envelope execute(ConnectionId cid, std::uint64_t modelId, std::string_view actionType, + std::string body) { + ::morph::wire::Envelope env; + env.kind = "execute"; + env.callId = 9; + env.modelId = modelId; + env.modelType = std::string{kFanModel}; + env.actionType = std::string{actionType}; + env.body = std::move(body); + auto waiter = std::make_shared<::morph::testing::WaitReply>(); + server->handle(::morph::wire::encode(env), [waiter](std::string const& msg) { (*waiter)(msg); }, cid); + REQUIRE(waiter->await()); + return waiter->env; + } + + /// Runs every task already posted to the server strand. + void flush() { + std::latch done{1}; + server->strand().post([&done] { done.count_down(); }); + done.wait(); + } +}; + +/// The keys of @p change, as raw JSON text. +std::vector keysOf(::morph::effects::RecordChange const& change) { + std::vector keys; + if (change.keys) { + for (auto const& key : *change.keys) { + keys.push_back(key.str); + } + } + return keys; +} + +} // namespace + +TEST_CASE("two_watchers_both_receive_caller_included", "[push][watch]") { + FanRig rig; + auto const caller = rig.connect(); + auto const other = rig.connect(); + rig.watch(caller, records({"T_FanRow"})); + rig.watch(other, records({"T_FanRow"})); + auto const mid = rig.create(caller); + + CHECK(rig.execute(caller, mid, "T_FanEdit", R"({"id":5})").kind == "ok"); + rig.flush(); + + for (auto const cid : {caller, other}) { + auto const bodies = rig.wire(cid).changes(); + REQUIRE(bodies.size() == 1U); + REQUIRE(bodies.front().records.size() == 1U); + CHECK(bodies.front().records.front().record == "T_FanRow"); + CHECK(keysOf(bodies.front().records.front()) == std::vector{"5"}); + CHECK(bodies.front().instances.empty()); + } +} + +TEST_CASE("unwatched_type_posts_nothing", "[push][watch]") { + FanRig rig; + auto const caller = rig.connect(); + auto const other = rig.connect(); + rig.watch(other, records({"T_FanOther"})); + auto const mid = rig.create(caller); + + CHECK(rig.execute(caller, mid, "T_FanEdit", R"({"id":5})").kind == "ok"); + // A reader changes nothing, so it posts nothing either. + CHECK(rig.execute(caller, mid, "T_FanListOthers", R"({"limit":1})").kind == "ok"); + rig.flush(); + + CHECK(rig.wire(caller).snapshot().empty()); + CHECK(rig.wire(other).snapshot().empty()); +} + +TEST_CASE("unnegotiated_connection_receives_nothing", "[push][watch]") { + FanRig rig; + auto const negotiated = rig.connect(); + auto const silent = rig.connect(false); + rig.watch(negotiated, records({"T_FanRow"})); + rig.watch(silent, records({"T_FanRow"})); + auto const mid = rig.create(silent); + + CHECK(rig.execute(silent, mid, "T_FanEdit", R"({"id":3})").kind == "ok"); + rig.flush(); + + CHECK(rig.wire(silent).snapshot().empty()); + CHECK(rig.wire(negotiated).changes().size() == 1U); +} + +TEST_CASE("private_instance_change_only_to_owner", "[push][watch]") { + FanRig rig; + auto const owner = rig.connect(); + auto const stranger = rig.connect(); + auto const mid = rig.create(owner); + // The stranger watches a keyed topic of the same type; a private instance + // never matches it. + rig.watch(stranger, instance(std::string{kFanModel}, "k")); + + // Run by a connection that does not own the instance: the owner is told, + // the caller is not. + CHECK(rig.execute(stranger, mid, "T_FanTouch", R"({"by":1})").kind == "ok"); + CHECK(rig.execute(owner, mid, "T_FanTouch", R"({"by":1})").kind == "ok"); + rig.flush(); + + CHECK(rig.wire(stranger).snapshot().empty()); + auto const bodies = rig.wire(owner).changes(); + REQUIRE(bodies.size() == 2U); + for (auto const& body : bodies) { + CHECK(body.records.empty()); + REQUIRE(body.instances.size() == 1U); + CHECK(body.instances.front().modelType == kFanModel); + CHECK(body.instances.front().primary.empty()); + CHECK(body.instances.front().instance == mid); + } +} + +TEST_CASE("keyed_instance_created_later_reaches_watcher", "[push][watch]") { + FanRig rig; + auto const watcher = rig.connect(); + auto const caller = rig.connect(); + auto const bystander = rig.connect(); + // Watched before any connection has created the instance. + rig.watch(watcher, instance(std::string{kFanModel}, "k1")); + rig.watch(bystander, instance(std::string{kFanModel}, "k2")); + + auto const mid = rig.create(caller, "k1"); + CHECK(rig.execute(caller, mid, "T_FanTouch", R"({"by":2})").kind == "ok"); + rig.flush(); + + auto const bodies = rig.wire(watcher).changes(); + REQUIRE(bodies.size() == 1U); + REQUIRE(bodies.front().instances.size() == 1U); + CHECK(bodies.front().instances.front().modelType == kFanModel); + CHECK(bodies.front().instances.front().primary == "k1"); + CHECK(bodies.front().instances.front().instance == mid); + // The caller does not watch the key, and a keyed instance has no owner to tell. + CHECK(rig.wire(caller).snapshot().empty()); + CHECK(rig.wire(bystander).snapshot().empty()); +} + +TEST_CASE("timed_out_call_still_reaches_caller_as_event", "[push][watch]") { + FanRig rig{std::chrono::milliseconds{10}}; + auto const caller = rig.connect(); + rig.watch(caller, records({"T_FanRow"})); + auto const mid = rig.create(caller); + + auto const reply = rig.execute(caller, mid, "T_FanSlowEdit", R"({"id":8,"delayMs":50})"); + REQUIRE(reply.kind == "err"); + CHECK(reply.message == "timeout"); + CHECK(rig.wire(caller).snapshot().empty()); + + // The handler finishes after the timeout answered; its change still arrives. + REQUIRE(::morph::testing::waitUntil([&] { return !rig.wire(caller).snapshot().empty(); })); + auto const bodies = rig.wire(caller).changes(); + REQUIRE(bodies.size() == 1U); + REQUIRE(bodies.front().records.size() == 1U); + CHECK(keysOf(bodies.front().records.front()) == std::vector{"8"}); +} + +TEST_CASE("failed_execute_posts_nothing", "[push][watch]") { + FanRig rig; + auto const caller = rig.connect(); + auto const other = rig.connect(); + rig.watch(caller, records({"T_FanRow"})); + rig.watch(other, records({"T_FanRow"})); + auto const mid = rig.create(caller); + + CHECK(rig.execute(caller, mid, "T_FanFail", R"({"id":5})").kind == "err"); + rig.flush(); + + CHECK(rig.wire(caller).snapshot().empty()); + CHECK(rig.wire(other).snapshot().empty()); +} From 6d5211e4964494836f625379bcbb7f581de57315 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 19:11:17 +0200 Subject: [PATCH 33/68] core: a connection's watches drop when its principal changes Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/backend.md | 4 ++ docs/spec/security.md | 34 +++++++++++ include/morph/core/remote.hpp | 56 +++++++++++++----- tests/test_push_watch.cpp | 107 ++++++++++++++++++++++++++++++++++ 4 files changed, 186 insertions(+), 15 deletions(-) diff --git a/docs/spec/core/backend.md b/docs/spec/core/backend.md index f3817b629..4a42062c5 100644 --- a/docs/spec/core/backend.md +++ b/docs/spec/core/backend.md @@ -1167,6 +1167,10 @@ execute path reads it. The UI layer sends them; nothing on the server outside as does a failed `write` or `backlog()` read on its writer: a connection that cannot be written to is sent no events, and its client sends the whole set again when it reconnects. + So does an envelope under another principal: `stampVerifiedPrincipal(env, + cid)` compares each stamped principal with the connection's baseline and + drops the watches on a difference + ([security.md](../security.md#a-connections-principal-baseline)). ### Fan-out diff --git a/docs/spec/security.md b/docs/spec/security.md index a6eab8d6e..0786c5797 100644 --- a/docs/spec/security.md +++ b/docs/spec/security.md @@ -370,6 +370,40 @@ Both signature and payload segments are decoded through this path, so a mutated trailing character in either segment fails verification (`Malformed`, or `BadSignature` if it survives decoding but changes the MAC). +### A connection's principal baseline + +A connection has no principal of its own: the server verifies one on every +envelope. A watch, though, outlives the envelope that created it, and keeps +delivering events on the connection. So the server holds, per open connection, +a **baseline**: the verified principal of the first envelope stamped on it +after `openConnection`. + +`stampVerifiedPrincipal(env, cid)` is where every kind that authorises stamps +its principal — `register`, `attach`, `assign`, `instances`, `schemas`, +`deregister`, `execute`, `cancel`, `watch`, `unwatch`, and every service kind. +After stamping, it compares the stamped principal with `cid`'s baseline: + +- **The first stamp sets the baseline**, whatever it is — an empty principal + included, from a token `authenticate` refuses or an authorizer that never + authenticates. +- **A different principal drops every watch the connection holds**, and becomes + the baseline. An envelope whose token fails `authenticate` is stamped with no + principal, so after an authenticated baseline it counts as different. This + covers signing in as someone else on the same connection, and a client that + keeps its watches across a sign-out: neither goes on receiving events admitted + for the previous principal. The client's cache is cleared on a principal + change, and it watches again with its next envelope, which is admitted afresh. +- **A new token for the same principal keeps the watches.** The comparison is + on the verified principal, not on the token, so a refreshed token does not + drop and re-admit the connection's whole watch set. +- **`hello` is not stamped**, so it neither sets nor changes the baseline. +- **`cid == 0` has no baseline**, and neither does a connection not open; + `closeConnection` forgets the baseline with the rest of the connection's + state. + +The baseline lives on the server strand beside the connection's other state, +and is read and written only there. + ### Role-based policy `SigningAuthorizer`'s optional `Policy` runs over the verified claims: diff --git a/include/morph/core/remote.hpp b/include/morph/core/remote.hpp index 06e9635eb..ec7ad4fd4 100644 --- a/include/morph/core/remote.hpp +++ b/include/morph/core/remote.hpp @@ -974,6 +974,7 @@ class RemoteServer : public std::enable_shared_from_this { _writers.erase(cid); _facts.erase(cid); _watches.dropConnection(cid); + _principalBaselines.erase(cid); auto scopeIter = _connectionScopes.find(cid); if (scopeIter == _connectionScopes.end()) { return; @@ -1414,16 +1415,35 @@ class RemoteServer : public std::enable_shared_from_this { /// `authenticate()` returns `nullopt` the authorizer cannot vouch for the /// caller, so the client-asserted principal is cleared rather than passed /// through unverified — every envelope kind this is called for (register, - /// attach, assign, instances, schemas, deregister, execute) makes an authorization or - /// ownership decision that must key on the verified identity, never the - /// client's raw claim. See docs/spec/security.md. + /// attach, assign, instances, schemas, deregister, execute, cancel, watch, + /// unwatch, a service's kind) makes an authorization or ownership decision + /// that must key on the verified identity, never the client's raw claim. + /// `hello`, which a client sends with no session, is not stamped. See + /// docs/spec/security.md. + /// + /// It then compares the stamped principal with @p cid's baseline, the + /// principal of the first envelope stamped on that connection. A different + /// one, including the empty principal of a token `authenticate` refuses, + /// drops every watch @p cid holds and becomes the baseline: a watch was + /// admitted for one principal, and must not keep delivering to another who + /// now speaks on the same connection. A new token for the same principal + /// keeps them. `cid == 0` and a connection not open have no baseline. /// @param env Envelope whose `session.principal` is stamped or cleared in place. - void stampVerifiedPrincipal(::morph::wire::Envelope& env) { + /// @param cid Connection @p env arrived on; `0` for unscoped. + void stampVerifiedPrincipal(::morph::wire::Envelope& env, ConnectionId cid) { if (auto verified = _authorizer->authenticate(env.session)) { env.session.principal = std::move(*verified); } else { env.session.principal.clear(); } + if (cid == 0 || !_connectionScopes.contains(cid)) { + return; + } + auto const [baseline, first] = _principalBaselines.try_emplace(cid, env.session.principal); + if (!first && baseline->second != env.session.principal) { + _watches.dropConnection(cid); + baseline->second = env.session.principal; + } } /// @brief Releases one reference to @p mid, destroying it at zero. On the server strand. @@ -1768,7 +1788,7 @@ class RemoteServer : public std::enable_shared_from_this { // (including allow-all) clears it, so the register decision // below — and the owner recorded from it — never key on the // client's unverified claim. - stampVerifiedPrincipal(env); + stampVerifiedPrincipal(env, cid); // Bound *who may create* an instance. The default hook allows // all, so an unconfigured server registers any known type; a // deployer opts into gating registration by overriding @@ -1821,7 +1841,7 @@ class RemoteServer : public std::enable_shared_from_this { if (env.typeId.empty()) { throw std::runtime_error("attach requires a typeId"); } - stampVerifiedPrincipal(env); + stampVerifiedPrincipal(env, cid); if (!_authorizer->authorizeRegister(env.session, env.typeId)) { reply(::morph::wire::encode(::morph::wire::makeErr("unauthorized", env.callId))); return; @@ -1838,7 +1858,7 @@ class RemoteServer : public std::enable_shared_from_this { // constructs a model, but it still changes what a future // attacher of `primary` reaches, so it must not be reachable // by an unauthenticated or unauthorized caller either. - stampVerifiedPrincipal(env); + stampVerifiedPrincipal(env, cid); if (!_authorizer->authorizeRegister(env.session, env.typeId)) { reply(::morph::wire::encode(::morph::wire::makeErr("unauthorized", env.callId))); return; @@ -1849,7 +1869,7 @@ class RemoteServer : public std::enable_shared_from_this { if (env.typeId.empty()) { throw std::runtime_error("instances requires a typeId"); } - stampVerifiedPrincipal(env); + stampVerifiedPrincipal(env, cid); // Enumeration is a read channel over the directory: gate it with // `authorize` for the model type (empty action id) so a deployer // can refuse listing without refusing use. It discloses the live @@ -1863,7 +1883,7 @@ class RemoteServer : public std::enable_shared_from_this { if (env.typeId.empty()) { throw std::runtime_error("schemas requires a typeId"); } - stampVerifiedPrincipal(env); + stampVerifiedPrincipal(env, cid); // Gated exactly like `instances`, and for the same reason: a // description is a read channel over the model type, so // `authorize` with an empty action id lets a deployer refuse @@ -1883,7 +1903,7 @@ class RemoteServer : public std::enable_shared_from_this { // env.session.principal, so that principal must be the verified // one: without the stamp a caller holding any token, or none, // could claim the owner's name and release the owner's instance. - stampVerifiedPrincipal(env); + stampVerifiedPrincipal(env, cid); // Type-level gate, as for `instances`/`schemas`. A deregister // names no type, so both ids are empty; it lets a verifying // authorizer refuse a caller with no valid token outright. @@ -1918,7 +1938,7 @@ class RemoteServer : public std::enable_shared_from_this { // Stamped before the ownership comparison in cancelCall, for // the reason deregister is: the comparison keys on the // verified principal, never on the client's claim. - stampVerifiedPrincipal(env); + stampVerifiedPrincipal(env, cid); cancelCall(env, cid); // One answer whatever became of the call -- stopped, unknown, // finished, or another caller's -- so a cancel cannot be used @@ -1940,13 +1960,13 @@ class RemoteServer : public std::enable_shared_from_this { } else if (env.kind == ::morph::wire::kWatchKind || env.kind == ::morph::wire::kUnwatchKind) { // Stamped first: the watch keeps the verified principal, and // admission decides on it. - stampVerifiedPrincipal(env); + stampVerifiedPrincipal(env, cid); handleWatch(env, reply, cid); } else if (IEnvelopeService* const service = serviceFor(env.kind); service != nullptr) { // Stamped first, as for every built-in kind that authorises: // the service decides on the verified principal, never on the // client's claim. - stampVerifiedPrincipal(env); + stampVerifiedPrincipal(env, cid); reply(service->handle(env, *_authorizer, factsOf(cid))); } else { reply(::morph::wire::encode(::morph::wire::makeErr("unknown envelope kind: " + env.kind, env.callId))); @@ -1998,7 +2018,7 @@ class RemoteServer : public std::enable_shared_from_this { // principal, never the attacker's claim), and (2) an authorize-only or // allow-all authorizer that never authenticates (the model never sees an // untrusted principal as authoritative). See docs/spec/security.md. - stampVerifiedPrincipal(env); + stampVerifiedPrincipal(env, cid); ::morph::exec::detail::ModelId const mid{env.modelId}; // One lookup for the holder, its recorded owner and its hydration state // together: they are fields of one record. @@ -2386,6 +2406,11 @@ class RemoteServer : public std::enable_shared_from_this { // What each open connection watches, filled by `watch`/`unwatch` and // dropped with the connection's scope or on a failed write. detail::WatchTable _watches; + // The verified principal of the first stamped envelope on each open + // connection, which a later one under another principal replaces after + // dropping the connection's watches (`stampVerifiedPrincipal`). Dropped + // with the connection's scope. + std::unordered_map _principalBaselines; /// Names one cancellable call: the connection it arrived on and the /// `callId` its client chose. struct CancelKey { @@ -2820,7 +2845,8 @@ class SimulatedRemoteBackend : public detail::IBackend { /// @param target Executor it is posted to. void set(std::function next, ::morph::exec::IExecutor* target) { std::lock_guard lock{mutex}; - handler = next ? std::make_shared>(std::move(next)) : nullptr; + handler = + next ? std::make_shared>(std::move(next)) : nullptr; exec = target; } diff --git a/tests/test_push_watch.cpp b/tests/test_push_watch.cpp index 9c973a846..8cec870a1 100644 --- a/tests/test_push_watch.cpp +++ b/tests/test_push_watch.cpp @@ -553,6 +553,113 @@ TEST_CASE("unwatch_of_unwatched_topic_is_ok", "[push][watch]") { CHECK(rig.recordWatchers("T_WatchRow") == std::vector{cid}); } +// ── The principal baseline: a connection's watches belong to one principal ─── + +namespace { + +/// Sends a `schemas` for the row model on @p cid, signed with @p token: a kind +/// that authorises and is not a watch, so it changes no watch of its own. +::morph::wire::Envelope describe(Rig& rig, ConnectionId cid, std::string const& token) { + ::morph::wire::Envelope env; + env.kind = "schemas"; + env.callId = 9; + env.typeId = "T_WatchRowModel"; + env.session.token = token; + return ::morph::wire::decode(rig.server->handleInline(::morph::wire::encode(env), cid)); +} + +/// Sends a core `hello` on @p cid, signed with @p token, and returns its reply's kind. +std::string hello(Rig& rig, ConnectionId cid, std::string const& token) { + ::morph::wire::Envelope env = ::morph::wire::makeHello(); + env.session.token = token; + return ::morph::wire::decode(rig.server->handleInline(::morph::wire::encode(env), cid)).kind; +} + +} // namespace + +TEST_CASE("envelope_under_other_principal_drops_watches", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + auto const other = rig.server->openConnection(); + ::morph::wire::WatchBody body = records({"T_WatchRow"}); + body.instances = {{.modelType = "T_WatchRowModel", .primary = "a"}}; + (void)rig.watch(cid, body, "alice"); + (void)rig.watch(other, records({"T_WatchRow"}), "alice"); + + // Any kind that authorises compares, not only `watch`. + CHECK(describe(rig, cid, "bob").kind == "ok"); + CHECK(rig.recordWatchers("T_WatchRow") == std::vector{other}); + CHECK(rig.instanceWatchers("T_WatchRowModel", "a").empty()); + + // bob is the baseline now: his watch stays while he sends, and goes when alice sends again. + (void)rig.watch(cid, instance("T_WatchRowModel", "b"), "bob"); + CHECK(describe(rig, cid, "bob").kind == "ok"); + CHECK(rig.instanceWatchers("T_WatchRowModel", "b") == std::vector{cid}); + (void)rig.watch(cid, records({"T_WatchRow"}), "alice"); + CHECK(rig.instanceWatchers("T_WatchRowModel", "b").empty()); + // A `watch` under the new principal drops the old set before it adds its own. + CHECK(rig.recordWatchers("T_WatchRow") == std::vector{cid, other}); +} + +TEST_CASE("unauthenticated_envelope_drops_watches", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + (void)rig.watch(cid, records({"T_WatchRow"}), "alice"); + REQUIRE(rig.recordWatchers("T_WatchRow") == std::vector{cid}); + + // A token `authenticate` refuses is stamped with no principal, which is not alice. + (void)describe(rig, cid, ""); + + CHECK(rig.recordWatchers("T_WatchRow").empty()); +} + +TEST_CASE("hello_does_not_reset_baseline", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + + // A `hello` first, under another principal, sets no baseline... + REQUIRE(hello(rig, cid, "bob") == "ok"); + (void)rig.watch(cid, records({"T_WatchRow"}), "alice"); + CHECK(describe(rig, cid, "alice").kind == "ok"); + REQUIRE(rig.recordWatchers("T_WatchRow") == std::vector{cid}); + + // ...and one after the watches, under another principal or none, drops none. + REQUIRE(hello(rig, cid, "bob") == "ok"); + REQUIRE(hello(rig, cid, "") == "ok"); + CHECK(rig.recordWatchers("T_WatchRow") == std::vector{cid}); + CHECK(describe(rig, cid, "alice").kind == "ok"); + CHECK(rig.recordWatchers("T_WatchRow") == std::vector{cid}); +} + +TEST_CASE("same_principal_new_token_keeps_watches", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + (void)rig.watch(cid, records({"T_WatchRow"}), "alice"); + + // A refreshed token verifies to the same principal: the baseline holds. + CHECK(describe(rig, cid, "alice:refreshed").kind == "ok"); + + CHECK(rig.recordWatchers("T_WatchRow") == std::vector{cid}); + auto const token = rig.onStrand([cid](auto const& table) -> std::optional { + auto const* watch = table.watchOf(cid, "T_WatchRow"); + return watch == nullptr ? std::nullopt : std::optional{watch->context.token}; + }); + CHECK(token == std::optional{"alice"}); +} + +TEST_CASE("first_stamp_sets_baseline_even_when_unauthenticated", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + + // The first stamped envelope carries no valid token: its empty principal is the baseline. + (void)rig.watch(cid, records({"T_WatchRow"}), ""); + REQUIRE(rig.recordWatchers("T_WatchRow") == std::vector{cid}); + + CHECK(describe(rig, cid, "alice").kind == "ok"); + + CHECK(rig.recordWatchers("T_WatchRow").empty()); +} + // ── Fan-out: what a successful execute tells the watching connections ─────── namespace { From af295294f1234c11f33a9f1e2470e734bef24b6e Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 19:33:19 +0200 Subject: [PATCH 34/68] core: every execute compares its principal with the connection's, even one that is refused Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/backend.md | 18 +++++-- docs/spec/security.md | 27 ++++++---- include/morph/core/remote.hpp | 88 +++++++++++++++++++------------ tests/test_push_watch.cpp | 97 ++++++++++++++++++++++++++++++++--- 4 files changed, 175 insertions(+), 55 deletions(-) diff --git a/docs/spec/core/backend.md b/docs/spec/core/backend.md index 4a42062c5..fcb5107be 100644 --- a/docs/spec/core/backend.md +++ b/docs/spec/core/backend.md @@ -635,22 +635,30 @@ external reference is dropped. **Execute flow.** Admission runs on the server strand (`dispatchExecute`), the action on the model's strand. Admission, in order: +0. **Verify and compare, before anything can refuse.** The server calls + `_authorizer->authenticate(env.session)` once, on entry, and compares the + result (empty when it returns `nullopt`) with the connection's principal + baseline ([security.md](../security.md#a-connections-principal-baseline)). + An execute under another principal, or under none, therefore drops the + connection's watches even when a later step refuses it, and before its own + fan-out when none does. The verified value is kept for step 3; `authorize` + still sees the session as the client sent it. 1. **In-flight cap.** With `LimitPolicy::maxInFlightExecutes` set and the in-flight count at it → `err "server busy"`. The count is server-strand state, so the check and the increment in step 7 cannot be separated by another admission: the cap is exact. 2. **Authorize.** `_authorizer->authorize(env.session, env.modelType, env.actionType)`. Denied → `err "unauthorized"` (with the request's `callId`), no dispatch. -3. **Authenticate / make the principal authoritative.** After `authorize` - succeeds, the server calls `_authorizer->authenticate(env.session)`. If it - returns a value, the server **overwrites** `env.session.principal` with that +3. **Make the principal authoritative.** After `authorize` succeeds, the + server uses the `authenticate` result from step 0. If it returned a value, + the server **overwrites** `env.session.principal` with that verified principal *before* building the `ScopedContext`. So model code that reads `session::current()->principal` on the remote path sees the identity the authorizer extracted from a valid token, not the client's asserted claim (`Context::principal` is untrusted wire input on its own). If `authenticate` returns `nullopt` — a non-verifying authorizer, including the default - `AllowAllAuthorizer`, or a token that passed `authorize` but expired in the - window before `authenticate` — the server **clears** `env.session.principal` + `AllowAllAuthorizer`, or a token that had already expired when it was + verified — the server **clears** `env.session.principal` to the empty string. The client's unverified claim is therefore never presented to the model as authoritative: the worst case is an empty principal, never an attacker-chosen one (see security.md). This is the only diff --git a/docs/spec/security.md b/docs/spec/security.md index 0786c5797..f2f9c4b48 100644 --- a/docs/spec/security.md +++ b/docs/spec/security.md @@ -304,8 +304,9 @@ auto server = std::make_shared(pool, authz); policy admits. An invalid/absent/expired token → `false` → the server replies `err "unauthorized"`. - `authenticate(ctx)` returns the verified `principal`. `RemoteServer` - (`dispatchExecute`) calls it after `authorize` succeeds and **overwrites** - `env.session.principal` with the verified value before building the + (`dispatchExecute`) calls it once, on entry, before any admission check, and + once `authorize` has succeeded **overwrites** `env.session.principal` with + the verified value before building the `ScopedContext`. So a model reading `session::current()->principal` sees the authenticated identity, not the client's claim. @@ -315,13 +316,13 @@ returns `nullopt` — the authorizer cannot vouch for the caller — dispatch rather than leaving the client-supplied claim in place. Two cases depend on this: -- **TOCTOU (time-of-check/time-of-use).** `authorize` and `authenticate` each - verify the token independently against a fresh clock reading. A token can pass - `authorize` and then expire in the window before `authenticate` runs, so - `authenticate` returns `nullopt`. Without the clear, the request would have - been dispatched carrying the *client's* asserted principal as if it were - authoritative. With the clear, the worst case is an empty principal — never - the attacker's chosen value. +- **TOCTOU (time-of-check/time-of-use).** `authenticate` and `authorize` each + verify the token independently against a fresh clock reading. A token + `authenticate` refuses can still pass a lenient `authorize`, so + `authenticate` returns `nullopt` for a call that is admitted. Without the + clear, the request would be dispatched carrying the *client's* asserted + principal as if it were authoritative. With the clear, the worst case is an + empty principal — never the attacker's chosen value. - **Authorize-only / allow-all authorizers.** An authorizer that admits calls but never authenticates (a custom authorize-only policy, or the default `AllowAllAuthorizer` whose `authenticate` inherits the `nullopt` default) now @@ -380,8 +381,12 @@ after `openConnection`. `stampVerifiedPrincipal(env, cid)` is where every kind that authorises stamps its principal — `register`, `attach`, `assign`, `instances`, `schemas`, -`deregister`, `execute`, `cancel`, `watch`, `unwatch`, and every service kind. -After stamping, it compares the stamped principal with `cid`'s baseline: +`deregister`, `cancel`, `watch`, `unwatch`, and every service kind. After +stamping, it compares the stamped principal with `cid`'s baseline. An +`execute` makes the same comparison on entry to `dispatchExecute`, before the +in-flight cap or `authorize` can refuse it, so a client that keeps sending +executes after a sign-out loses its watches even though every one is refused. +It reuses that one `authenticate` result for its stamp. - **The first stamp sets the baseline**, whatever it is — an empty principal included, from a token `authenticate` refuses or an authorizer that never diff --git a/include/morph/core/remote.hpp b/include/morph/core/remote.hpp index ec7ad4fd4..6e5d402be 100644 --- a/include/morph/core/remote.hpp +++ b/include/morph/core/remote.hpp @@ -1415,34 +1415,51 @@ class RemoteServer : public std::enable_shared_from_this { /// `authenticate()` returns `nullopt` the authorizer cannot vouch for the /// caller, so the client-asserted principal is cleared rather than passed /// through unverified — every envelope kind this is called for (register, - /// attach, assign, instances, schemas, deregister, execute, cancel, watch, - /// unwatch, a service's kind) makes an authorization or ownership decision - /// that must key on the verified identity, never the client's raw claim. - /// `hello`, which a client sends with no session, is not stamped. See - /// docs/spec/security.md. - /// - /// It then compares the stamped principal with @p cid's baseline, the - /// principal of the first envelope stamped on that connection. A different - /// one, including the empty principal of a token `authenticate` refuses, - /// drops every watch @p cid holds and becomes the baseline: a watch was - /// admitted for one principal, and must not keep delivering to another who - /// now speaks on the same connection. A new token for the same principal - /// keeps them. `cid == 0` and a connection not open have no baseline. + /// attach, assign, instances, schemas, deregister, cancel, watch, unwatch, + /// a service's kind) makes an authorization or ownership decision that + /// must key on the verified identity, never the client's raw claim. + /// `execute` does the same in `dispatchExecute`, split in two around its + /// `authorize`. `hello`, which a client sends with no session, is not + /// stamped. See docs/spec/security.md. + /// + /// It then compares the stamped principal with @p cid's baseline + /// (`compareWithBaseline`). /// @param env Envelope whose `session.principal` is stamped or cleared in place. /// @param cid Connection @p env arrived on; `0` for unscoped. void stampVerifiedPrincipal(::morph::wire::Envelope& env, ConnectionId cid) { - if (auto verified = _authorizer->authenticate(env.session)) { - env.session.principal = std::move(*verified); - } else { - env.session.principal.clear(); - } + env.session.principal = verifiedPrincipal(env.session); + compareWithBaseline(env.session.principal, cid); + } + + /// @brief The principal `_authorizer` verifies from @p session, or empty + /// when it cannot vouch for one. + /// @param session An envelope's session, its principal the client's claim. + /// @return The verified principal, or an empty string. + [[nodiscard]] std::string verifiedPrincipal(const ::morph::session::Context& session) const { + auto verified = _authorizer->authenticate(session); + return verified ? std::move(*verified) : std::string{}; + } + + /// @brief Compares @p principal with @p cid's baseline, dropping the + /// connection's watches when they differ. On the server strand. + /// + /// The baseline is the principal of the first envelope verified on the + /// connection. A different one, including the empty principal of a token + /// `authenticate` refuses, drops every watch @p cid holds and becomes the + /// baseline: a watch was admitted for one principal, and must not keep + /// delivering to another who now speaks on the same connection. A new + /// token for the same principal keeps them. `cid == 0` and a connection + /// not open have no baseline. + /// @param principal The envelope's verified principal, or empty. + /// @param cid Connection the envelope arrived on; `0` for unscoped. + void compareWithBaseline(const std::string& principal, ConnectionId cid) { if (cid == 0 || !_connectionScopes.contains(cid)) { return; } - auto const [baseline, first] = _principalBaselines.try_emplace(cid, env.session.principal); - if (!first && baseline->second != env.session.principal) { + auto const [baseline, first] = _principalBaselines.try_emplace(cid, principal); + if (!first && baseline->second != principal) { _watches.dropConnection(cid); - baseline->second = env.session.principal; + baseline->second = principal; } } @@ -1997,6 +2014,12 @@ class RemoteServer : public std::enable_shared_from_this { auto reject = [&env, &reply](const char* message) { reply(::morph::wire::encode(::morph::wire::makeErr(message, env.callId))); }; + // Verified once, on entry, and compared with the connection's baseline + // before any check below can refuse the call: an execute under another + // principal, or under none, drops the connection's watches even when it + // is refused, and before its own fan-out when it is not. + std::string verified = verifiedPrincipal(env.session); + compareWithBaseline(verified, cid); LimitPolicy const& limits = _config.limits; // Exact, not advisory: the count is this strand's own state, so nothing // can be admitted between this check and the increment below. @@ -2008,17 +2031,16 @@ class RemoteServer : public std::enable_shared_from_this { reject("unauthorized"); return; } - // Make the identity authoritative. A verifying authorizer returns the - // principal it extracted from a valid token; we stamp it so model code - // reading session::current()->principal can trust it. If authenticate() - // returns nullopt the authorizer cannot vouch for the caller, so we CLEAR - // the client-asserted principal rather than passing it through unverified. - // This closes two holes: (1) the TOCTOU window where a token that passed - // authorize() expires before authenticate() (worst case is an empty - // principal, never the attacker's claim), and (2) an authorize-only or - // allow-all authorizer that never authenticates (the model never sees an - // untrusted principal as authoritative). See docs/spec/security.md. - stampVerifiedPrincipal(env, cid); + // Make the identity authoritative: the principal verified on entry + // replaces the client's claim, so model code reading + // session::current()->principal can trust it. When authenticate() + // could not vouch for the caller it is empty, never the client's claim: + // a token that expires between the two checks yields at worst an empty + // principal, and an authorize-only or allow-all authorizer, which never + // authenticates, never presents an unverified principal to the model. + // `authorize` above saw the session as the client sent it; the verified + // principal lands only here. See docs/spec/security.md. + env.session.principal = std::move(verified); ::morph::exec::detail::ModelId const mid{env.modelId}; // One lookup for the holder, its recorded owner and its hydration state // together: they are fields of one record. @@ -2408,7 +2430,7 @@ class RemoteServer : public std::enable_shared_from_this { detail::WatchTable _watches; // The verified principal of the first stamped envelope on each open // connection, which a later one under another principal replaces after - // dropping the connection's watches (`stampVerifiedPrincipal`). Dropped + // dropping the connection's watches (`compareWithBaseline`). Dropped // with the connection's scope. std::unordered_map _principalBaselines; /// Names one cancellable call: the connection it arrived on and the diff --git a/tests/test_push_watch.cpp b/tests/test_push_watch.cpp index 8cec870a1..c2bba735d 100644 --- a/tests/test_push_watch.cpp +++ b/tests/test_push_watch.cpp @@ -166,6 +166,7 @@ struct InstanceQuestion { /// token. struct SecretGuard final : ::morph::session::IAuthorizer { mutable std::atomic authorizeCalls{0}; + mutable std::atomic authenticateCalls{0}; mutable std::mutex mtx; mutable std::vector instanceQuestions; @@ -175,6 +176,7 @@ struct SecretGuard final : ::morph::session::IAuthorizer { return modelType != kSecretModel && !ctx.token.ends_with(":scoped"); } [[nodiscard]] std::optional authenticate(const ::morph::session::Context& ctx) const override { + authenticateCalls.fetch_add(1); if (ctx.token.empty()) { return std::nullopt; } @@ -568,6 +570,24 @@ ::morph::wire::Envelope describe(Rig& rig, ConnectionId cid, std::string const& return ::morph::wire::decode(rig.server->handleInline(::morph::wire::encode(env), cid)); } +/// Sends an `execute` of @p actionType on @p modelId of @p modelType through +/// @p cid, signed with @p token, and waits for the reply. +::morph::wire::Envelope executeOn(Rig& rig, ConnectionId cid, std::string_view modelType, std::string_view actionType, + std::uint64_t modelId, std::string const& token) { + ::morph::wire::Envelope env; + env.kind = "execute"; + env.callId = 11; + env.modelId = modelId; + env.modelType = std::string{modelType}; + env.actionType = std::string{actionType}; + env.body = R"({"limit":1})"; + env.session.token = token; + auto waiter = std::make_shared<::morph::testing::WaitReply>(); + rig.server->handle(::morph::wire::encode(env), [waiter](std::string const& msg) { (*waiter)(msg); }, cid); + REQUIRE(waiter->await()); + return waiter->env; +} + /// Sends a core `hello` on @p cid, signed with @p token, and returns its reply's kind. std::string hello(Rig& rig, ConnectionId cid, std::string const& token) { ::morph::wire::Envelope env = ::morph::wire::makeHello(); @@ -660,6 +680,36 @@ TEST_CASE("first_stamp_sets_baseline_even_when_unauthenticated", "[push][watch]" CHECK(rig.recordWatchers("T_WatchRow").empty()); } +TEST_CASE("refused_execute_still_drops_watches", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + (void)rig.watch(cid, records({"T_WatchRow"}), "alice"); + REQUIRE(rig.recordWatchers("T_WatchRow") == std::vector{cid}); + + // Signed out, the client sends only executes, each refused at `authorize`. + auto const reply = executeOn(rig, cid, kSecretModel, "T_WatchListSecrets", 1, ""); + CHECK(reply.kind == "err"); + CHECK(reply.message == "unauthorized"); + + CHECK(rig.recordWatchers("T_WatchRow").empty()); +} + +TEST_CASE("execute_authenticates_once", "[push][watch]") { + Rig rig; + auto const cid = rig.server->openConnection(); + auto registerRow = ::morph::wire::makeRegister("T_WatchRowModel"); + registerRow.session.token = "alice"; + auto const registered = ::morph::wire::decode(rig.server->handleInline(::morph::wire::encode(registerRow), cid)); + REQUIRE(registered.kind == "ok"); + + int const before = rig.guard->authenticateCalls.load(); + CHECK(executeOn(rig, cid, "T_WatchRowModel", "T_WatchListRows", registered.modelId, "alice").kind == "ok"); + CHECK(rig.guard->authenticateCalls.load() == before + 1); + + CHECK(executeOn(rig, cid, kSecretModel, "T_WatchListSecrets", 1, "alice").kind == "err"); + CHECK(rig.guard->authenticateCalls.load() == before + 2); +} + // ── Fan-out: what a successful execute tells the watching connections ─────── namespace { @@ -730,11 +780,13 @@ struct FanRig { std::shared_ptr server; std::map> wires; - explicit FanRig(std::chrono::milliseconds executeTimeout = std::chrono::milliseconds{0}) { + explicit FanRig(std::chrono::milliseconds executeTimeout = std::chrono::milliseconds{0}, + std::shared_ptr<::morph::session::IAuthorizer> authorizer = nullptr) { ::morph::backend::ServerConfig config; config.limits.executeTimeout = executeTimeout; config.services.push_back(std::make_shared()); - server = std::make_shared(pool, std::move(config)); + server = authorizer ? std::make_shared(pool, std::move(authorizer), std::move(config)) + : std::make_shared(pool, std::move(config)); } /// Opens a connection with a writer, negotiating events when @p events. @@ -754,10 +806,11 @@ struct FanRig { EventWire& wire(ConnectionId cid) { return *wires.at(cid); } - void watch(ConnectionId cid, ::morph::wire::WatchBody const& body) { + void watch(ConnectionId cid, ::morph::wire::WatchBody const& body, std::string const& token = {}) { ::morph::wire::Envelope env; env.kind = std::string{::morph::wire::kWatchKind}; env.callId = 2; + env.session.token = token; env.body = glz::write_json(body).value_or(std::string{}); auto const reply = ::morph::wire::decode(server->handleInline(::morph::wire::encode(env), cid)); REQUIRE(reply.kind == "ok"); @@ -768,9 +821,10 @@ struct FanRig { } /// Registers a private instance on @p cid, or a keyed shared one when @p primary is set. - std::uint64_t create(ConnectionId cid, std::string primary = {}) { - auto const env = primary.empty() ? ::morph::wire::makeRegister(std::string{kFanModel}) + std::uint64_t create(ConnectionId cid, std::string primary = {}, std::string const& token = {}) { + auto env = primary.empty() ? ::morph::wire::makeRegister(std::string{kFanModel}) : ::morph::wire::makeRegisterShared(std::string{kFanModel}, primary); + env.session.token = token; auto const reply = ::morph::wire::decode(server->handleInline(::morph::wire::encode(env), cid)); REQUIRE(reply.kind == "ok"); REQUIRE(reply.modelId != 0U); @@ -780,9 +834,10 @@ struct FanRig { /// Executes @p actionType with @p body on @p modelId through @p cid and /// waits for the reply. ::morph::wire::Envelope execute(ConnectionId cid, std::uint64_t modelId, std::string_view actionType, - std::string body) { + std::string body, std::string const& token = {}) { ::morph::wire::Envelope env; env.kind = "execute"; + env.session.token = token; env.callId = 9; env.modelId = modelId; env.modelType = std::string{kFanModel}; @@ -800,6 +855,18 @@ struct FanRig { server->strand().post([&done] { done.count_down(); }); done.wait(); } + + /// The connections watching record type @p record, read on the server strand. + std::vector recordWatchers(std::string_view record) { + std::vector watchers; + std::latch done{1}; + server->strand().post([&] { + watchers = server->watchTable().watchersOf(record); + done.count_down(); + }); + done.wait(); + return watchers; + } }; /// The keys of @p change, as raw JSON text. @@ -951,3 +1018,21 @@ TEST_CASE("failed_execute_posts_nothing", "[push][watch]") { CHECK(rig.wire(caller).snapshot().empty()); CHECK(rig.wire(other).snapshot().empty()); } + +TEST_CASE("execute_under_other_principal_drops_watches_before_its_fanout", "[push][watch]") { + FanRig rig{std::chrono::milliseconds{0}, std::make_shared()}; + auto const shared = rig.connect(); + auto const other = rig.connect(); + rig.watch(shared, records({"T_FanRow"}), "alice"); + rig.watch(other, records({"T_FanRow"}), "alice"); + auto const mid = rig.create(shared, {}, "alice"); + + // bob executes on alice's connection: admitted, and its change fans out... + CHECK(rig.execute(shared, mid, "T_FanEdit", R"({"id":5})", "bob").kind == "ok"); + rig.flush(); + + // ...to the connection still watching, but not to the watch bob's execute dropped. + CHECK(rig.wire(other).changes().size() == 1U); + CHECK(rig.wire(shared).changes().empty()); + CHECK(rig.recordWatchers("T_FanRow") == std::vector{other}); +} From 92eed1ced2f6cfe1ab697f45ad6b7b0ea9e8d632 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 19:52:33 +0200 Subject: [PATCH 35/68] core: fan-out re-authorises each watch by the rule that admitted it At send time each recipient watch's stored session must still authenticate as the principal it was admitted for (an expired token drops every watch the connection holds under that session), and the topic's admission rule runs again on it (mayWatchRecord / mayWatchInstance, through the per-principal, per-token cache). A failure drops the watch and removes the topic; an event left empty is not sent. A privateKeys record reaches principals other than the caller's without keys. An instance whose change carries no primary is looked up in the directory at fan-out, so one an assign keyed while its execute ran reaches the key's watchers. scopesHolding names every holding scope. Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/backend.md | 56 ++++- docs/spec/security.md | 41 ++++ include/morph/core/detail/watch_table.hpp | 50 +++- include/morph/core/remote.hpp | 167 ++++++++++--- tests/test_push_watch.cpp | 282 +++++++++++++++++++++- 5 files changed, 553 insertions(+), 43 deletions(-) diff --git a/docs/spec/core/backend.md b/docs/spec/core/backend.md index fcb5107be..d29a9918e 100644 --- a/docs/spec/core/backend.md +++ b/docs/spec/core/backend.md @@ -1122,8 +1122,10 @@ execute path reads it. The UI layer sends them; nothing on the server outside created it and the principal verified from it, because a connection has no principal of its own. Watching a topic again keeps the newer one. `watchersOf(record)` and `watchersOf(modelType, primary)` answer the - connections watching a topic, ascending; `watchOf(cid, record)` the watch - itself; `topicCount(cid)` how many topics `cid` watches. `RemoteServer::watchTable()` exposes it read-only, on the strand. + connections watching a topic, ascending; `watchOf(cid, record)` and + `watchOf(cid, modelType, primary)` the watch itself; `dropSession(cid, + principal, token)` drops every watch `cid` holds under one session; + `topicCount(cid)` how many topics `cid` watches. `RemoteServer::watchTable()` exposes it read-only, on the strand. - **Admission.** Each topic of a `watch` is admitted or refused on its own, on the principal `stampVerifiedPrincipal` verified for the envelope; the `ok` reply's `WatchReply` lists the refused ones, and the admitted ones are @@ -1178,7 +1180,9 @@ execute path reads it. The UI layer sends them; nothing on the server outside So does an envelope under another principal: `stampVerifiedPrincipal(env, cid)` compares each stamped principal with the connection's baseline and drops the watches on a difference - ([security.md](../security.md#a-connections-principal-baseline)). + ([security.md](../security.md#a-connections-principal-baseline)). And so + does fan-out, for a watch that no longer passes the rule that admitted it + ([Fan-out](#fan-out)). ### Fan-out @@ -1193,27 +1197,55 @@ through `dispatchExecute`. shared instance, the primary of the directory key it held when the execute was admitted (`RemoteRun::sharedKey`; empty for a private instance). Changes that are not empty are posted to the server strand as - `fanOut(changes, caller)`, where `caller` is the execute's connection - (`RemoteRun::cid`). The reply is written exactly as without fan-out. A failed + `fanOut(changes, caller, callerPrincipal)`, where `caller` is the execute's + connection (`RemoteRun::cid`) and `callerPrincipal` its verified principal. The reply is written exactly as without fan-out. A failed execute derives nothing. A derivation that throws is logged; the reply and the action gate's release still follow. - **The server strand finds the recipients.** A record topic goes to every connection `watchersOf(record)` names, and a keyed shared instance's topic to every connection `watchersOf(modelType, primary)` names, **the caller - included**. The caller may never see its own reply: an execute the + included**. An instance whose change carries no primary is looked up in the + directory on the strand: one an `assign` keyed while its execute ran is + matched by the key it holds now, since matching is against + `Instance::sharedKey` at fan-out. The caller may never see its own reply: an execute the `executeTimeout` answered first but that succeeded, a reply the client discarded, a replay whose reply went to the offline worker. A private - instance's topic goes only to the connection whose scope holds the instance, - whether or not it is the caller, and needs no watch. A connection that has + instance's topic goes to every connection whose scope holds the instance + (`scopesHolding`), whether or not it is the caller, and needs no watch. A connection that has not negotiated events (`eventsNegotiated(cid)`) is sent nothing. - **One event per recipient.** Each recipient gets one `changes` event whose `ChangesBody` holds only the topics it is sent: its watched records, with the keys derived, and its instance topics as `InstanceTopic{modelType, primary, instance}`. Before it is encoded, the body passes through - `RemoteServer::reauthorise(cid, body)`, the one place a recipient's topics - are narrowed; it keeps every topic, since each watch was authorised when it - was admitted. The event is then written as `post` writes: dropped above + `RemoteServer::reauthorise`, the one place a recipient's topics are + narrowed. An event left with no topic is not sent. The event is then + written as `post` writes: dropped above `wire::kMaxEnvelopeBytes`, held back behind a full backlog. +- **Re-authorised at send time, by the rule that admitted it.** For each + watched topic in a recipient's body: + 1. the watch's stored session must still `authenticate` as the principal it + was admitted for. An expired token verifies as no principal, so every + watch the connection holds under that session is dropped. A watch + admitted with no verified principal, under an authorizer that never + authenticates, passes. `authenticate` is asked once per session per + fan-out; + 2. the topic's admission rule runs again on that session: `mayWatchRecord` + for a record, `mayWatchInstance` for an instance, through the same cache + per principal and token ([Watches](#watches)). A refusal drops that + watch. The action that wrote the change is not consulted, so a principal + that may list a record but not write it still receives its changes. + + A dropped watch's topic leaves the body; the client watches again with its + next envelope. A private instance's topic has no watch and is kept. A + verified principal's cached `authorize` answer stands until that principal + watches with a new token, so a policy change reaches its existing watches + then; the `authenticate` and `authorizeInstance` checks are asked afresh. +- **Private keys.** A record registered with `morph::record::privateKeys` + reaches a recipient without its `keys` (a whole-type change) unless the + recipient is the calling connection or its watch's principal is the + caller's verified principal. Two connections with no verified principal are + not the same principal for this purpose. See + [security.md](../security.md#what-an-event-reveals). - **The registries are read at construction.** Every `RemoteServer` constructor reads the record and effects registries once, so a process whose declarations are invalid fails with `EffectsRegistrationError` when the server @@ -2332,7 +2364,7 @@ server: each call is a loop task, so the loop serialises them. | `post(cid, envelope)` | Any thread. Above `wire::kMaxEnvelopeBytes`: dropped and counted (`Metric::eventOversize`). On the strand: written through `cid`'s writer, or held back — dropped, with one `resync` owed — while `backlog()` exceeds `kEventBacklogHigh` or a `resync` is still owed. No writer: dropped. | | `connectionDrained(cid)` | Any thread; posts. Sends the owed `resync` event, if any, and lets events flow again. What a transport calls when the backlog falls below `kEventBacklogLow`. | | `watchTable() const` | Server strand only (debug-asserted). The `detail::WatchTable` of every open connection's watches, read-only. See [Watches](#watches). | -| `fanOut(changes, caller)` | Server strand only (debug-asserted). Sends each connection that watches a topic of `changes`, or whose scope holds a private instance in it, one `changes` event with the topics it is sent; `caller` included, connections without negotiated events skipped. A successful `execute` posts it. See [Fan-out](#fan-out). | +| `fanOut(changes, caller, callerPrincipal)` | Server strand only (debug-asserted). Sends each connection that watches a topic of `changes`, or whose scope holds a private instance in it, one `changes` event with the topics it is sent; `caller` included, connections without negotiated events skipped. Each watch is re-authorised first, and a `privateKeys` record loses its keys for principals other than `callerPrincipal`. A successful `execute` posts it. See [Fan-out](#fan-out). | | `eventsNegotiated(cid) const` | Server strand only (debug-asserted). Whether `cid` negotiated events: the `ConnectionFacts::events` a service set for it. `false` for `0`, an unknown `cid` or a closed one. | | `payloadCompleteness()` | The `ServerConfig::payloadCompleteness` given at construction. | | `strand()` | The server strand, as an `exec::IExecutor`: `runningOn(server.strand())` is true inside every task on the server's state. | diff --git a/docs/spec/security.md b/docs/spec/security.md index f2f9c4b48..20c40f6db 100644 --- a/docs/spec/security.md +++ b/docs/spec/security.md @@ -409,6 +409,47 @@ It reuses that one `authenticate` result for its stamp. The baseline lives on the server strand beside the connection's other state, and is read and written only there. +### Fan-out re-authorises each watch + +A watch is admitted once, but delivers for as long as it lasts, and a token +expires or a policy changes in the meantime. So `RemoteServer::fanOut` +re-authorises every recipient watch at send time, by the rule that admitted it, +on the session stored with it: + +- **Its token must still authenticate** as the principal the watch was admitted + for. An expired token verifies as no principal, so the check fails, and every + watch the connection holds under that session is dropped. A watch admitted + with no verified principal, under an authorizer that never authenticates, was + not admitted on authentication and passes. +- **Then the admission rule runs again.** For a record type: the principal may + run some action whose result holds it. For an instance topic: the model type + is registered, the principal may run one of its actions, and + `authorizeInstance(ctx, modelType, {}, 0, "")` admits it. A failure drops + that watch. The action that wrote the change is not checked: a role that may + list results but not capture them still receives the change. + +A dropped watch's topic leaves the event, and an event with no topic left is +not sent. The client watches again with its next envelope, whose fresh token is +admitted afresh. The "may run some action" answers come from the watch table's +cache per verified principal and token, so for an authenticated principal a +policy change reaches its existing watches when it next watches with a new +token; `authenticate` and `authorizeInstance` are asked each time. A private +instance's topic has no watch: it goes only to the connections whose scope +holds the instance. + +### What an event reveals + +An event carries no values, only which topics changed and, for a record, the +keys of the rows that did. Those keys are ids of rows the recipient could list +through some action returning that record, but not necessarily the same rows: +a model may filter rows per principal. A record registered with +`morph::record::privateKeys` therefore reaches every recipient other than the +caller's own principal as a whole-type change, without keys, and every entry +holding it refetches through the read path, where row filtering and +`secretFields` are enforced. The keys are kept for the calling connection and +for watches whose verified principal is the caller's; two connections with no +verified principal do not count as the same principal. + ### Role-based policy `SigningAuthorizer`'s optional `Policy` runs over the verified claims: diff --git a/include/morph/core/detail/watch_table.hpp b/include/morph/core/detail/watch_table.hpp index d73d5f8d2..ccd7dd984 100644 --- a/include/morph/core/detail/watch_table.hpp +++ b/include/morph/core/detail/watch_table.hpp @@ -30,7 +30,8 @@ namespace morph::backend::detail { /// /// A connection has no principal of its own, so each watch keeps the session /// of the `watch` envelope that created it and the principal verified from -/// that session. Fan-out re-authorises the watch on them at send time. +/// that session. Fan-out re-authorises the watch on them at send time, by the +/// rule that admitted it. struct Watch { /// @brief The `watch` envelope's session, its principal already the verified one. ::morph::session::Context context; @@ -175,6 +176,53 @@ class WatchTable { return found == connection->second.records.end() ? nullptr : found->second.get(); } + /// @brief What @p cid's watch on the keyed instance `(modelType, primary)` + /// was admitted on. + /// @param cid Connection. + /// @param modelType The instance's registered model type. + /// @param primary The instance's directory key text. + /// @return The watch, valid until the table next changes, or null when + /// @p cid does not watch that instance. + [[nodiscard]] Watch const* watchOf(ConnectionId cid, std::string_view modelType, std::string_view primary) const { + auto const connection = _connections.find(cid); + if (connection == _connections.end()) { + return nullptr; + } + auto const found = connection->second.instances.find(InstanceKey{std::string{modelType}, std::string{primary}}); + return found == connection->second.instances.end() ? nullptr : found->second.get(); + } + + /// @brief Drops every watch @p cid holds that was admitted with + /// @p principal and @p token, and keeps the rest. + /// + /// What a watch whose token no longer authenticates takes with it: every + /// topic the connection watched under that same session. + /// @param cid Connection. + /// @param principal The verified principal the watches were admitted for. + /// @param token The token they were admitted with. + void dropSession(ConnectionId cid, std::string const& principal, std::string const& token) { + auto found = _connections.find(cid); + if (found == _connections.end()) { + return; + } + SessionKey const key{principal, token}; + auto const dropMatching = [this, &key](auto& topics) { + for (auto it = topics.begin(); it != topics.end();) { + if (keyOf(*it->second) == key) { + release(key); + it = topics.erase(it); + } else { + ++it; + } + } + }; + dropMatching(found->second.records); + dropMatching(found->second.instances); + if (found->second.size() == 0) { + _connections.erase(found); + } + } + /// @brief How many topics @p cid watches, records and instances together. /// @param cid Connection. /// @return The count; 0 for a connection with no watches. diff --git a/include/morph/core/remote.hpp b/include/morph/core/remote.hpp index 6e5d402be..5a688617a 100644 --- a/include/morph/core/remote.hpp +++ b/include/morph/core/remote.hpp @@ -748,14 +748,24 @@ class RemoteServer : public std::enable_shared_from_this { /// A record topic, and the instance topic of a keyed shared instance, go /// to every connection watching them, @p caller included: the caller /// may never see its own reply (it timed out, the client discarded it, or - /// it went to an offline replay). A private instance's topic goes to the + /// it went to an offline replay). An instance is matched by the directory + /// key it holds now, so one an `assign` keyed while its execute ran + /// reaches that key's watchers. A private instance's topic goes to every /// connection whose scope holds that instance, which needs no watch, and /// to no other. A connection that has not negotiated events is sent - /// nothing. Each connection's event holds only the topics it is sent, and - /// passes through `post`'s backlog hold-back and size cap. - /// @param changes What one execute changed, from `effects::deriveChanges`. - /// @param caller Connection the execute arrived on; `0` for unscoped. - void fanOut(::morph::effects::Changes changes, ConnectionId caller) { + /// nothing. + /// + /// Each recipient's watches are then re-authorised by the rule that + /// admitted them (`reauthorise`): a topic whose watch fails is dropped + /// from the table and from the event. A record registered with + /// `morph::record::privateKeys` reaches a principal other than + /// @p callerPrincipal without its keys. Each connection's event holds only + /// the topics left, is not sent when none are, and passes through + /// `post`'s backlog hold-back and size cap. + /// @param changes What one execute changed, from `effects::deriveChanges`. + /// @param caller Connection the execute arrived on; `0` for unscoped. + /// @param callerPrincipal The execute's verified principal; empty when none was. + void fanOut(::morph::effects::Changes changes, ConnectionId caller, std::string_view callerPrincipal) { assert(_strand.runningHere() && "RemoteServer::fanOut runs on the server strand"); noteOwner("RemoteServer::fanOut"); std::map bodies; @@ -767,9 +777,15 @@ class RemoteServer : public std::enable_shared_from_this { } } for (auto& change : changes.instances) { - auto const recipients = change.primary.empty() - ? scopesHolding(::morph::exec::detail::ModelId{change.instance}, caller) - : _watches.watchersOf(change.modelType, change.primary); + ::morph::exec::detail::ModelId const mid{change.instance}; + if (change.primary.empty()) { + // Keyed after admission: the topic is the key it holds now. + if (auto const* inst = _instances.find(mid); inst != nullptr && inst->sharedKey) { + change.primary = inst->sharedKey->second; + } + } + auto const recipients = change.primary.empty() ? scopesHolding(mid) + : _watches.watchersOf(change.modelType, change.primary); ::morph::wire::InstanceTopic const topic{ .modelType = std::move(change.modelType), .primary = std::move(change.primary), .instance = change.instance}; @@ -779,8 +795,9 @@ class RemoteServer : public std::enable_shared_from_this { } } } + Authentications authentications; for (auto& [cid, body] : bodies) { - reauthorise(cid, body); + reauthorise(cid, body, caller, callerPrincipal, authentications); if (body.records.empty() && body.instances.empty()) { continue; } @@ -1023,17 +1040,12 @@ class RemoteServer : public std::enable_shared_from_this { /// @brief The connections whose scope holds @p mid. On the server strand. /// - /// A private instance is held by the one connection that registered it; - /// @p caller is asked first because it usually is that connection. - /// @param mid Instance. - /// @param caller Connection to ask first; `0` for none. - /// @return The connections, at most one for a private instance. - [[nodiscard]] std::vector scopesHolding(::morph::exec::detail::ModelId mid, - ConnectionId caller) const { - if (auto const own = _connectionScopes.find(caller); - own != _connectionScopes.end() && own->second.contains(mid)) { - return {caller}; - } + /// A private instance is held by the connection that registered it; every + /// holder is named, so routing does not depend on which connection ran + /// the action. + /// @param mid Instance. + /// @return The connections, ascending. + [[nodiscard]] std::vector scopesHolding(::morph::exec::detail::ModelId mid) const { std::vector holders; for (auto const& [cid, scope] : _connectionScopes) { if (scope.contains(mid)) { @@ -1043,14 +1055,109 @@ class RemoteServer : public std::enable_shared_from_this { return holders; } + /// One fan-out's `authenticate` answers, by (stored principal, token): + /// whether the token still verifies as the principal it was admitted for. + using Authentications = std::map, bool>; + + /// @brief Whether @p watch's session still authenticates as the principal + /// it was admitted for. On the server strand. + /// + /// An expired or revoked token verifies as no principal, which differs + /// from a non-empty stored one. A watch admitted with no verified + /// principal, as under an authorizer that never authenticates, was not + /// admitted on authentication and still passes. Asked once per session + /// per fan-out. + /// @param watch What the watch was admitted on. + /// @param asked This fan-out's answers so far; extended in place. + /// @return True when the session still verifies as `watch.principal`. + [[nodiscard]] bool stillAuthenticated(detail::Watch const& watch, Authentications& asked) const { + auto [found, fresh] = asked.try_emplace({watch.principal, watch.context.token}, false); + if (fresh) { + found->second = verifiedPrincipal(watch.context) == watch.principal; + } + return found->second; + } + /// @brief Narrows @p body to the topics connection @p cid may still be - /// sent. On the server strand. + /// sent, re-authorising each watch by the rule that admitted it. + /// On the server strand. /// - /// Every topic is kept: each was authorised when its watch was admitted, - /// and a private instance's topic goes only to the connection holding it. - /// @param cid Recipient. - /// @param body The topics `fanOut` chose for it; narrowed in place. - static void reauthorise([[maybe_unused]] ConnectionId cid, [[maybe_unused]] ::morph::wire::ChangesBody& body) {} + /// For each watched topic, in order: the watch's stored session must + /// still authenticate (`stillAuthenticated`), or every watch @p cid holds + /// under that session is dropped; then the topic's admission rule is run + /// again on that session, `mayWatchRecord` or `mayWatchInstance`, with + /// the answers they cache per principal and token, or that one watch is + /// dropped. A dropped watch's topic leaves @p body. Which action wrote + /// the change is not checked: a principal that may list a record but not + /// write it still receives its changes. The client watches again with its + /// next envelope, whose fresh token is admitted afresh. + /// + /// A private instance's topic has no watch: it is sent to the scope that + /// holds the instance, and kept. + /// + /// A record registered with `morph::record::privateKeys` loses its keys + /// unless @p cid is @p caller or its watch's principal is + /// @p callerPrincipal, which must be a verified one: keys name rows the + /// writer's principal may see, and a principal that filters rows + /// differently must not learn them. Without keys, the recipient refetches + /// every entry holding the record. + /// @param cid Recipient. + /// @param body The topics `fanOut` chose for it; narrowed in place. + /// @param caller Connection the execute arrived on; `0` for unscoped. + /// @param callerPrincipal The execute's verified principal; empty when none was. + /// @param asked This fan-out's `authenticate` answers; extended in place. + void reauthorise(ConnectionId cid, ::morph::wire::ChangesBody& body, ConnectionId caller, + std::string_view callerPrincipal, Authentications& asked) { + // Copied out before a drop: dropping frees the `Watch` it came from. + auto const authenticated = [&](detail::Watch const& watch) { + if (stillAuthenticated(watch, asked)) { + return true; + } + std::string const principal = watch.principal; + std::string const token = watch.context.token; + _watches.dropSession(cid, principal, token); + return false; + }; + std::vector<::morph::effects::RecordChange> records; + records.reserve(body.records.size()); + for (auto& change : body.records) { + auto const* watch = _watches.watchOf(cid, change.record); + if (watch == nullptr || !authenticated(*watch)) { + continue; + } + if (!mayWatchRecord(watch->context, change.record)) { + _watches.remove(cid, ::morph::wire::WatchBody{.records = {change.record}, .instances = {}}); + continue; + } + bool const keepsKeys = cid == caller || (!callerPrincipal.empty() && watch->principal == callerPrincipal); + if (!keepsKeys) { + auto const* info = ::morph::effects::recordNamed(change.record); + if (info != nullptr && info->privateKeys) { + change.keys.reset(); + } + } + records.push_back(std::move(change)); + } + body.records = std::move(records); + std::vector<::morph::wire::InstanceTopic> instances; + instances.reserve(body.instances.size()); + for (auto& topic : body.instances) { + if (!topic.primary.empty()) { + auto const* watch = _watches.watchOf(cid, topic.modelType, topic.primary); + if (watch == nullptr || !authenticated(*watch)) { + continue; + } + if (!mayWatchInstance(watch->context, topic.modelType)) { + _watches.remove(cid, ::morph::wire::WatchBody{ + .records = {}, + .instances = {{.modelType = topic.modelType, .primary = topic.primary}}}); + continue; + } + } + instances.push_back(std::move(topic)); + } + body.instances = std::move(instances); + } /// @brief Body of `post`. On the server strand. /// @param cid Connection to write to. @@ -2327,8 +2434,10 @@ class RemoteServer : public std::enable_shared_from_this { if (changes.empty()) { return; } - run.self->_strand.postTask([self = run.self, changes = std::move(changes), caller = run.cid]() mutable { - self->fanOut(std::move(changes), caller); + // The execute's principal is the verified one by now (`dispatchExecute`). + run.self->_strand.postTask([self = run.self, changes = std::move(changes), caller = run.cid, + principal = run.env.session.principal]() mutable { + self->fanOut(std::move(changes), caller, principal); }); } catch (const std::exception& exc) { ::morph::log::logError("[fanOut] changes of {} not derived: {}", run.env.actionType, exc.what()); diff --git a/tests/test_push_watch.cpp b/tests/test_push_watch.cpp index c2bba735d..6659a2b47 100644 --- a/tests/test_push_watch.cpp +++ b/tests/test_push_watch.cpp @@ -3,6 +3,7 @@ // The server's per-connection watch table and the core `watch`/`unwatch` // kinds: what a watch may name, how it is stored, and what drops it. +#include #include #include #include @@ -20,6 +21,8 @@ #include #include #include +#include +#include #include #include #include @@ -109,6 +112,37 @@ struct TouchFan { int by{}; }; +/// Lists rows; holds the row record without changing it. +struct ListFanRows { + int limit{}; +}; + +/// A record whose keys must not reach other principals. +struct FanPrivate { + std::int64_t id{}; +}; + +struct FanPrivateList { + std::vector rows; +}; + +/// Edits one private-keyed row and returns it. +struct EditFanPrivate { + std::int64_t id{}; +}; + +/// Changes the instance's own state once the test releases `fanGate()`, so +/// the test can act between its admission and its fan-out. +struct GatedTouchFan { + int by{}; +}; + +/// Released by a test to let one `GatedTouchFan` finish. +inline std::binary_semaphore& fanGate() { + static std::binary_semaphore gate{0}; + return gate; +} + struct FanModel { int touched = 0; FanRowList execute(EditFanRow const& action) { return {.rows = {{.id = action.id, .name = "edited"}}}; } @@ -119,6 +153,12 @@ struct FanModel { FanRowList execute(FailFanRow const& /*action*/) { throw std::runtime_error("edit failed"); } FanOtherList execute(ListFanOthers const& /*action*/) { return {}; } int execute(TouchFan const& action) { return touched += action.by; } + FanRowList execute(ListFanRows const& /*action*/) { return {}; } + FanPrivateList execute(EditFanPrivate const& action) { return {.rows = {{.id = action.id}}}; } + int execute(GatedTouchFan const& action) { + (void)fanGate().try_acquire_for(std::chrono::seconds{5}); + return touched += action.by; + } }; } // namespace push_watch_test @@ -144,6 +184,12 @@ BRIDGE_REGISTER_EFFECTS(FailFanRow, morph::effects::updates) BRIDGE_REGISTER_ACTION(FanModel, ListFanOthers, "T_FanListOthers") BRIDGE_REGISTER_ACTION(FanModel, TouchFan, "T_FanTouch") BRIDGE_REGISTER_EFFECTS(TouchFan, morph::effects::changesInstance) +BRIDGE_REGISTER_ACTION(FanModel, ListFanRows, "T_FanListRows") +BRIDGE_REGISTER_RECORD(FanPrivate, "T_FanPrivate", morph::record::privateKeys) +BRIDGE_REGISTER_ACTION(FanModel, EditFanPrivate, "T_FanEditPrivate") +BRIDGE_REGISTER_EFFECTS(EditFanPrivate, morph::effects::updates) +BRIDGE_REGISTER_ACTION(FanModel, GatedTouchFan, "T_FanGatedTouch") +BRIDGE_REGISTER_EFFECTS(GatedTouchFan, morph::effects::changesInstance) namespace { @@ -867,6 +913,18 @@ struct FanRig { done.wait(); return watchers; } + + /// The connections watching `(modelType, primary)`, read on the server strand. + std::vector instanceWatchers(std::string_view modelType, std::string_view primary) { + std::vector watchers; + std::latch done{1}; + server->strand().post([&] { + watchers = server->watchTable().watchersOf(modelType, primary); + done.count_down(); + }); + done.wait(); + return watchers; + } }; /// The keys of @p change, as raw JSON text. @@ -991,7 +1049,7 @@ TEST_CASE("timed_out_call_still_reaches_caller_as_event", "[push][watch]") { rig.watch(caller, records({"T_FanRow"})); auto const mid = rig.create(caller); - auto const reply = rig.execute(caller, mid, "T_FanSlowEdit", R"({"id":8,"delayMs":50})"); + auto const reply = rig.execute(caller, mid, "T_FanSlowEdit", R"({"id":8,"delayMs":300})"); REQUIRE(reply.kind == "err"); CHECK(reply.message == "timeout"); CHECK(rig.wire(caller).snapshot().empty()); @@ -1036,3 +1094,225 @@ TEST_CASE("execute_under_other_principal_drops_watches_before_its_fanout", "[pus CHECK(rig.wire(shared).changes().empty()); CHECK(rig.recordWatchers("T_FanRow") == std::vector{other}); } + +namespace { + +/// The principal a token names: the token up to a `:`. +std::string principalOf(std::string_view token) { return std::string{token.substr(0, token.find(':'))}; } + +/// An authorizer whose answers a test changes between a watch and a send. +/// +/// A token names its principal as `SecretGuard`'s does, and `authorize` reads +/// the principal from the token, so it tells callers apart whether or not it +/// authenticates. +struct FlipGuard final : ::morph::session::IAuthorizer { + mutable std::mutex mtx; + /// Whether `authenticate` vouches for any token at all. + bool authenticates = true; + /// Tokens `authenticate` refuses, as an expired one is. + std::set> expired; + /// `principal/actionType` pairs `authorize` refuses. + std::set> deniedActions; + /// Principals `authorizeInstance` refuses. + std::set> deniedInstances; + + void expire(std::string token) { + std::scoped_lock const lock{mtx}; + expired.insert(std::move(token)); + } + void deny(std::string const& principal, std::string_view actionType) { + std::scoped_lock const lock{mtx}; + deniedActions.insert(principal + "/" + std::string{actionType}); + } + void denyInstances(std::string principal) { + std::scoped_lock const lock{mtx}; + deniedInstances.insert(std::move(principal)); + } + + [[nodiscard]] bool authorize(const ::morph::session::Context& ctx, std::string_view /*modelType*/, + std::string_view actionType) const override { + std::scoped_lock const lock{mtx}; + return !deniedActions.contains(principalOf(ctx.token) + "/" + std::string{actionType}); + } + [[nodiscard]] std::optional authenticate(const ::morph::session::Context& ctx) const override { + std::scoped_lock const lock{mtx}; + if (!authenticates || ctx.token.empty() || expired.contains(ctx.token)) { + return std::nullopt; + } + return principalOf(ctx.token); + } + [[nodiscard]] bool authorizeInstance(const ::morph::session::Context& ctx, std::string_view /*modelType*/, + std::string_view /*actionType*/, std::uint64_t /*modelId*/, + std::string_view /*ownerPrincipal*/) const override { + std::scoped_lock const lock{mtx}; + return !deniedInstances.contains(principalOf(ctx.token)); + } +}; + +/// The actions that hold `T_FanRow` and write it. +constexpr std::array kFanRowWriters{"T_FanEdit", "T_FanSlowEdit", "T_FanFail"}; + +} // namespace + +TEST_CASE("expired_watch_dropped_at_send", "[push][watch]") { + auto guard = std::make_shared(); + FanRig rig{std::chrono::milliseconds{0}, guard}; + auto const caller = rig.connect(); + auto const watcher = rig.connect(); + rig.watch(caller, records({"T_FanRow"}), "alice"); + rig.watch(watcher, records({"T_FanRow"}), "carol"); + auto const mid = rig.create(caller, {}, "alice"); + + CHECK(rig.execute(caller, mid, "T_FanEdit", R"({"id":1})", "alice").kind == "ok"); + rig.flush(); + REQUIRE(rig.wire(watcher).changes().size() == 1U); + + // carol's token expires; nothing she sends tells the server, so only the + // send-time check can notice. + guard->expire("carol"); + CHECK(rig.execute(caller, mid, "T_FanEdit", R"({"id":2})", "alice").kind == "ok"); + rig.flush(); + + CHECK(rig.wire(watcher).changes().size() == 1U); + CHECK(rig.wire(caller).changes().size() == 2U); + CHECK(rig.recordWatchers("T_FanRow") == std::vector{caller}); +} + +TEST_CASE("lister_not_writer_still_receives", "[push][watch]") { + auto guard = std::make_shared(); + for (auto const writer : kFanRowWriters) { + guard->deny("rita", writer); + } + FanRig rig{std::chrono::milliseconds{0}, guard}; + auto const caller = rig.connect(); + auto const reader = rig.connect(); + // rita may list rows but run none of the actions that write them. + rig.watch(reader, records({"T_FanRow"}), "rita"); + auto const mid = rig.create(caller, {}, "alice"); + + CHECK(rig.execute(caller, mid, "T_FanEdit", R"({"id":5})", "alice").kind == "ok"); + rig.flush(); + + auto const bodies = rig.wire(reader).changes(); + REQUIRE(bodies.size() == 1U); + REQUIRE(bodies.front().records.size() == 1U); + CHECK(keysOf(bodies.front().records.front()) == std::vector{"5"}); + CHECK(rig.recordWatchers("T_FanRow") == std::vector{reader}); +} + +TEST_CASE("private_keys_stripped_for_other_principals_kept_for_caller", "[push][watch]") { + auto guard = std::make_shared(); + FanRig rig{std::chrono::milliseconds{0}, guard}; + auto const caller = rig.connect(); + auto const sameAlice = rig.connect(); + auto const bob = rig.connect(); + rig.watch(caller, records({"T_FanPrivate", "T_FanRow"}), "alice"); + rig.watch(sameAlice, records({"T_FanPrivate"}), "alice:other"); + rig.watch(bob, records({"T_FanPrivate", "T_FanRow"}), "bob"); + auto const mid = rig.create(caller, {}, "alice"); + + CHECK(rig.execute(caller, mid, "T_FanEditPrivate", R"({"id":4})", "alice").kind == "ok"); + CHECK(rig.execute(caller, mid, "T_FanEdit", R"({"id":6})", "alice").kind == "ok"); + rig.flush(); + + for (auto const cid : {caller, sameAlice}) { + auto const bodies = rig.wire(cid).changes(); + REQUIRE_FALSE(bodies.empty()); + REQUIRE(bodies.front().records.size() == 1U); + CHECK(bodies.front().records.front().record == "T_FanPrivate"); + CHECK(keysOf(bodies.front().records.front()) == std::vector{"4"}); + } + auto const bobBodies = rig.wire(bob).changes(); + REQUIRE(bobBodies.size() == 2U); + REQUIRE(bobBodies.front().records.size() == 1U); + // bob learns that the type changed, not which rows. + CHECK(bobBodies.front().records.front().record == "T_FanPrivate"); + CHECK_FALSE(bobBodies.front().records.front().keys.has_value()); + // A record without privateKeys keeps its keys for every principal. + REQUIRE(bobBodies.back().records.size() == 1U); + CHECK(keysOf(bobBodies.back().records.front()) == std::vector{"6"}); +} + +TEST_CASE("refused_after_policy_change_drops_watch", "[push][watch]") { + auto guard = std::make_shared(); + // An authorizer that does not authenticate: no principal is verified, so + // no admission answer is cached and the send-time check asks it afresh. + // A verified principal's cached answer would stand until its token is + // refreshed. + guard->authenticates = false; + FanRig rig{std::chrono::milliseconds{0}, guard}; + auto const caller = rig.connect(); + auto const reader = rig.connect(); + rig.watch(reader, records({"T_FanRow"}), "rita"); + auto const mid = rig.create(caller, {}, "alice"); + + CHECK(rig.execute(caller, mid, "T_FanEdit", R"({"id":1})", "alice").kind == "ok"); + rig.flush(); + REQUIRE(rig.wire(reader).changes().size() == 1U); + + // rita loses every action that holds the row record. + guard->deny("rita", "T_FanListRows"); + for (auto const writer : kFanRowWriters) { + guard->deny("rita", writer); + } + CHECK(rig.execute(caller, mid, "T_FanEdit", R"({"id":2})", "alice").kind == "ok"); + rig.flush(); + + CHECK(rig.wire(reader).changes().size() == 1U); + CHECK(rig.recordWatchers("T_FanRow").empty()); +} + +TEST_CASE("instance_watch_reauthorised_at_send", "[push][watch]") { + auto guard = std::make_shared(); + FanRig rig{std::chrono::milliseconds{0}, guard}; + auto const caller = rig.connect(); + auto const watcher = rig.connect(); + rig.watch(watcher, instance(std::string{kFanModel}, "k1"), "carol"); + auto const mid = rig.create(caller, "k1", "alice"); + + CHECK(rig.execute(caller, mid, "T_FanTouch", R"({"by":1})", "alice").kind == "ok"); + rig.flush(); + REQUIRE(rig.wire(watcher).changes().size() == 1U); + + guard->denyInstances("carol"); + CHECK(rig.execute(caller, mid, "T_FanTouch", R"({"by":1})", "alice").kind == "ok"); + rig.flush(); + + CHECK(rig.wire(watcher).changes().size() == 1U); + CHECK(rig.instanceWatchers(kFanModel, "k1").empty()); +} + +TEST_CASE("assign_promoted_instance_reaches_watcher", "[push][watch]") { + FanRig rig; + auto const caller = rig.connect(); + auto const watcher = rig.connect(); + rig.watch(watcher, instance(std::string{kFanModel}, "k9")); + auto const mid = rig.create(caller); + + // Admitted while the instance is private... + ::morph::wire::Envelope env; + env.kind = "execute"; + env.callId = 9; + env.modelId = mid; + env.modelType = std::string{kFanModel}; + env.actionType = "T_FanGatedTouch"; + env.body = R"({"by":1})"; + auto waiter = std::make_shared<::morph::testing::WaitReply>(); + rig.server->handle(::morph::wire::encode(env), [waiter](std::string const& msg) { (*waiter)(msg); }, caller); + // ...promoted to a key before it finishes... + auto const assigned = ::morph::wire::decode(rig.server->handleInline( + ::morph::wire::encode(::morph::wire::makeAssign(std::string{kFanModel}, "k9", mid)), caller)); + REQUIRE(assigned.kind == "ok"); + fanGate().release(); + REQUIRE(waiter->await()); + REQUIRE(waiter->env.kind == "ok"); + rig.flush(); + + // ...so its change is matched against the key it holds now. + auto const bodies = rig.wire(watcher).changes(); + REQUIRE(bodies.size() == 1U); + REQUIRE(bodies.front().instances.size() == 1U); + CHECK(bodies.front().instances.front().primary == "k9"); + CHECK(bodies.front().instances.front().instance == mid); + CHECK(rig.wire(caller).snapshot().empty()); +} From 134c215ee5e2f3e31303dab14a6a59ad817ac3ca Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 20:15:07 +0200 Subject: [PATCH 36/68] core: a connection whose watches the server drops is owed a resync A principal change that drops any watch, and a fan-out whose re-authorisation drops any, send the connection one resync event, through post's backlog hold-back and only when events are negotiated. The client answers every resync by re-sending its whole watch set with replace: true. Unwatch, replace, close and write failure send none. Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/backend.md | 32 +++-- docs/spec/security.md | 20 ++- .../specs/2026-10-16-server-push-design.md | 21 ++- include/morph/core/remote.hpp | 64 ++++++--- tests/test_push_watch.cpp | 125 +++++++++++++++++- 5 files changed, 223 insertions(+), 39 deletions(-) diff --git a/docs/spec/core/backend.md b/docs/spec/core/backend.md index d29a9918e..7783f7739 100644 --- a/docs/spec/core/backend.md +++ b/docs/spec/core/backend.md @@ -1183,6 +1183,19 @@ execute path reads it. The UI layer sends them; nothing on the server outside ([security.md](../security.md#a-connections-principal-baseline)). And so does fan-out, for a watch that no longer passes the rule that admitted it ([Fan-out](#fan-out)). +- **Resync for dropped watches.** When + the server drops watches on its own, the client is not otherwise told, and + would stop receiving those topics. So a principal change that drops any + watch, and a fan-out whose re-authorisation drops any, send the connection + one `resync` event (`RemoteServer::sendResync`): one per envelope or per + fan-out, however many watches went. It is sent only to a connection that + negotiated events, and through `post`'s hold-back, so a connection already + held back behind its backlog gets the one `resync` its drain owes. The + client answers every `resync` by marking its watched entries stale and + sending its whole watch set again with `replace: true`; a topic still + refused comes back in the `WatchReply`. An `unwatch` and a `replace` are the + client's own and owe nothing; a close and a write failure leave no + connection to tell. ### Fan-out @@ -1205,9 +1218,9 @@ through `dispatchExecute`. connection `watchersOf(record)` names, and a keyed shared instance's topic to every connection `watchersOf(modelType, primary)` names, **the caller included**. An instance whose change carries no primary is looked up in the - directory on the strand: one an `assign` keyed while its execute ran is - matched by the key it holds now, since matching is against - `Instance::sharedKey` at fan-out. The caller may never see its own reply: an execute the + directory on the strand: the directory is consulted at fan-out when a change + carries no primary, so an instance an `assign` keyed while its execute ran + is matched by the key it holds now. The caller may never see its own reply: an execute the `executeTimeout` answered first but that succeeded, a reply the client discarded, a replay whose reply went to the offline worker. A private instance's topic goes to every connection whose scope holds the instance @@ -1235,11 +1248,14 @@ through `dispatchExecute`. watch. The action that wrote the change is not consulted, so a principal that may list a record but not write it still receives its changes. - A dropped watch's topic leaves the body; the client watches again with its - next envelope. A private instance's topic has no watch and is kept. A - verified principal's cached `authorize` answer stands until that principal - watches with a new token, so a policy change reaches its existing watches - then; the `authenticate` and `authorizeInstance` checks are asked afresh. + A dropped watch's topic leaves the body, and the connection is sent one + `resync` after its event, however many watches the fan-out dropped + (`sendResync`; see [Resync for dropped watches](#resync-for-dropped-watches)). + A private instance's topic has no watch and is kept. A verified principal's + cached `authorize` answer stands while the session it was cached for keeps + watching, so a policy change reaches an existing watch when its stored token + stops verifying or the topic is watched again; the `authenticate` and + `authorizeInstance` checks are asked afresh. - **Private keys.** A record registered with `morph::record::privateKeys` reaches a recipient without its `keys` (a whole-type change) unless the recipient is the calling connection or its watch's principal is the diff --git a/docs/spec/security.md b/docs/spec/security.md index 20c40f6db..e53d6c676 100644 --- a/docs/spec/security.md +++ b/docs/spec/security.md @@ -396,8 +396,11 @@ It reuses that one `authenticate` result for its stamp. principal, so after an authenticated baseline it counts as different. This covers signing in as someone else on the same connection, and a client that keeps its watches across a sign-out: neither goes on receiving events admitted - for the previous principal. The client's cache is cleared on a principal - change, and it watches again with its next envelope, which is admitted afresh. + for the previous principal. A connection that held any watch is sent one + `resync` event; the client answers every `resync` by marking its watched + entries stale and sending its whole watch set again with `replace: true`, + which is admitted afresh for the new principal, and a refused topic comes + back in the `WatchReply`. - **A new token for the same principal keeps the watches.** The comparison is on the verified principal, not on the token, so a refreshed token does not drop and re-admit the connection's whole watch set. @@ -429,11 +432,14 @@ on the session stored with it: list results but not capture them still receives the change. A dropped watch's topic leaves the event, and an event with no topic left is -not sent. The client watches again with its next envelope, whose fresh token is -admitted afresh. The "may run some action" answers come from the watch table's -cache per verified principal and token, so for an authenticated principal a -policy change reaches its existing watches when it next watches with a new -token; `authenticate` and `authorizeInstance` are asked each time. A private +not sent. The connection is then sent one `resync`, however many watches the +fan-out dropped, and the client answers it by sending its whole watch set again +with `replace: true`, under its current token; a topic still refused comes back +in the `WatchReply`. The "may run some action" answers come from the watch +table's cache per verified principal and token, so for an authenticated +principal a policy change reaches an existing watch when its stored token stops +verifying or the topic is watched again; `authenticate` and +`authorizeInstance` are asked each time. A private instance's topic has no watch: it goes only to the connections whose scope holds the instance. diff --git a/docs/superpowers/specs/2026-10-16-server-push-design.md b/docs/superpowers/specs/2026-10-16-server-push-design.md index c3d890991..a5be909bc 100644 --- a/docs/superpowers/specs/2026-10-16-server-push-design.md +++ b/docs/superpowers/specs/2026-10-16-server-push-design.md @@ -340,6 +340,8 @@ A client watches the topics its cache holds, so the server sends each connection - **Lifetime.** A topic is watched while at least one cache entry holds it, mounted or kept (spec 5 §5: for 5 minutes). The client counts references per topic. It sends `watch` when a count leaves zero and `unwatch` when it returns to zero. + The server may also drop a watch on its own (§12): it then sends a `resync`, and the client answers every `resync` + by sending its whole watch set again with `replace: true`, as it does after a reconnect. - **Watch before fetch.** Watch changes made in one reactive batch are sent as one message, **before** the fetches of that batch, on the same connection. The server handles `watch` on its strand when the message arrives, and an `execute` is admitted on that same strand before it is posted to the model strand. The watch is therefore in the @@ -445,7 +447,7 @@ The interpreter's query cache applies a change the same way whether it came from | `updates R` with keys | Patch: each entry holding `R` merges the new records into the rows with those keys, at the paths and by the key field the catalog gives (§5). It uses the existing `patch` merge, and the catalog's `listFields` for a list inside a record. No refetch. | Each entry holding `R` whose value contains one of the keys is marked stale, and refetched if mounted. | | `changes R` | Each entry holding `R` is marked stale, and refetched if mounted. | The same. | | `changesInstance` | Each entry read through the same instance is marked stale, and refetched if mounted. | The same, for the instance named: the private one this connection owns, or a keyed shared one it watches. | -| `resync` | none | Every watched entry is marked stale, and refetched if mounted. | +| `resync` | none | Every watched entry is marked stale, and refetched if mounted. The whole watch set is sent again with `replace: true` (§12). | - **A change during a fetch is kept.** A stale mark on an entry whose fetch is pending sets `refetchAfter`. When the fetch settles, the entry stays stale and fetches once more, even if it succeeded. Otherwise the success would clear @@ -555,8 +557,8 @@ changes visible. authorises passes through it; `hello`, which carries no session, does not. - **The baseline** is the verified principal of the first stamped envelope after the connection opens. An envelope whose token fails `authenticate` is stamped with no principal, so it counts as different: the watches drop, and - the client sends them again with its next, freshly signed, envelope. That covers signing in as someone else, and a client that keeps old watches across a sign-out. - The client's cache is already cleared on a principal change (spec 5 §5), and it watches again. + the connection is sent one `resync` (below). That covers signing in as someone else, and a client that keeps old + watches across a sign-out. The client's cache is already cleared on a principal change (spec 5 §5). - **Fan-out re-authorises each watch at send time, by the rule that admitted it.** - It runs `authenticate` on the stored context, which fails for an expired token. - For a record topic, it then requires again that the principal may run some action whose `records` hold the @@ -564,9 +566,16 @@ changes visible. results but not capture them still receives the change. - For an instance topic, it re-runs the admission check for instance topics below. - A failure drops the watch. The client watches again with its next envelope, which carries a fresh token. The - "may run some action holding `R`" answer is cached per principal and record type, and refreshed when that - principal's token is. + A failure drops the watch, and the connection is sent one `resync` (below). The "may run some action holding + `R`" answer is cached per principal and token, so a policy change reaches an existing watch when its stored + token stops verifying or the topic is watched again. + - **A watch the server drops is answered with a `resync`.** Whenever the server drops any of a connection's + watches other than by that connection's own `unwatch` or `replace` (a principal change, or a re-authorisation + failure at fan-out), it sends that connection one `resync` event: one per envelope or fan-out, however many + watches went, only when `events` is negotiated, and through the backlog hold-back (§6), so a connection held + back gets the one `resync` its drain owes. The client answers **every** `resync` by marking its watched entries + stale and sending its whole watch set again with `replace: true`; a topic still refused comes back in the + `WatchReply`. A close and a write failure send nothing: there is no connection left to tell. - **What a watch may name.** - A record type: only when the principal may run at least one action whose `records` contain it. Otherwise the `ok` reply lists it as refused, and the client logs it. diff --git a/include/morph/core/remote.hpp b/include/morph/core/remote.hpp index 5a688617a..78026cfd4 100644 --- a/include/morph/core/remote.hpp +++ b/include/morph/core/remote.hpp @@ -757,7 +757,8 @@ class RemoteServer : public std::enable_shared_from_this { /// /// Each recipient's watches are then re-authorised by the rule that /// admitted them (`reauthorise`): a topic whose watch fails is dropped - /// from the table and from the event. A record registered with + /// from the table and from the event, and a connection that lost any + /// watch is sent one `resync` after its event (`sendResync`). A record registered with /// `morph::record::privateKeys` reaches a principal other than /// @p callerPrincipal without its keys. Each connection's event holds only /// the topics left, is not sent when none are, and passes through @@ -797,16 +798,18 @@ class RemoteServer : public std::enable_shared_from_this { } Authentications authentications; for (auto& [cid, body] : bodies) { - reauthorise(cid, body, caller, callerPrincipal, authentications); - if (body.records.empty() && body.instances.empty()) { - continue; + bool const dropped = reauthorise(cid, body, caller, callerPrincipal, authentications); + if (!body.records.empty() || !body.instances.empty()) { + std::string payload; + (void)glz::write_json(body, payload); + auto envelope = + ::morph::wire::encode(::morph::wire::makeEvent(::morph::wire::kChangesTopic, std::move(payload))); + if (!oversized(cid, envelope)) { + postHere(cid, std::move(envelope)); + } } - std::string payload; - (void)glz::write_json(body, payload); - auto envelope = - ::morph::wire::encode(::morph::wire::makeEvent(::morph::wire::kChangesTopic, std::move(payload))); - if (!oversized(cid, envelope)) { - postHere(cid, std::move(envelope)); + if (dropped) { + sendResync(cid); } } } @@ -1089,8 +1092,9 @@ class RemoteServer : public std::enable_shared_from_this { /// the answers they cache per principal and token, or that one watch is /// dropped. A dropped watch's topic leaves @p body. Which action wrote /// the change is not checked: a principal that may list a record but not - /// write it still receives its changes. The client watches again with its - /// next envelope, whose fresh token is admitted afresh. + /// write it still receives its changes. The caller owes the connection a + /// `resync` when this drops anything; the client answers it by sending + /// its whole watch set again, which is admitted afresh. /// /// A private instance's topic has no watch: it is sent to the scope that /// holds the instance, and kept. @@ -1106,8 +1110,10 @@ class RemoteServer : public std::enable_shared_from_this { /// @param caller Connection the execute arrived on; `0` for unscoped. /// @param callerPrincipal The execute's verified principal; empty when none was. /// @param asked This fan-out's `authenticate` answers; extended in place. - void reauthorise(ConnectionId cid, ::morph::wire::ChangesBody& body, ConnectionId caller, - std::string_view callerPrincipal, Authentications& asked) { + /// @return True when any of @p cid's watches was dropped. + [[nodiscard]] bool reauthorise(ConnectionId cid, ::morph::wire::ChangesBody& body, ConnectionId caller, + std::string_view callerPrincipal, Authentications& asked) { + bool dropped = false; // Copied out before a drop: dropping frees the `Watch` it came from. auto const authenticated = [&](detail::Watch const& watch) { if (stillAuthenticated(watch, asked)) { @@ -1116,6 +1122,7 @@ class RemoteServer : public std::enable_shared_from_this { std::string const principal = watch.principal; std::string const token = watch.context.token; _watches.dropSession(cid, principal, token); + dropped = true; return false; }; std::vector<::morph::effects::RecordChange> records; @@ -1127,6 +1134,7 @@ class RemoteServer : public std::enable_shared_from_this { } if (!mayWatchRecord(watch->context, change.record)) { _watches.remove(cid, ::morph::wire::WatchBody{.records = {change.record}, .instances = {}}); + dropped = true; continue; } bool const keepsKeys = cid == caller || (!callerPrincipal.empty() && watch->principal == callerPrincipal); @@ -1151,12 +1159,33 @@ class RemoteServer : public std::enable_shared_from_this { _watches.remove(cid, ::morph::wire::WatchBody{ .records = {}, .instances = {{.modelType = topic.modelType, .primary = topic.primary}}}); + dropped = true; continue; } } instances.push_back(std::move(topic)); } body.instances = std::move(instances); + return dropped; + } + + /// @brief Sends @p cid one `resync` event, owed because the server dropped + /// some of its watches on its own. On the server strand. + /// + /// The client answers a `resync` by marking what it watches stale and + /// sending its whole watch set again with `replace: true`; a topic still + /// refused comes back in the `WatchReply`. Without it, a client whose + /// watch was dropped would stop receiving that topic's changes and never + /// know. Sent only to a connection that negotiated events, and through + /// `postHere`, so a connection held back behind its backlog gets the one + /// `resync` its drain already owes it. An `unwatch`, a `replace`, a close + /// and a write failure drop watches without one: the first two are the + /// client's own, and the last two leave no connection to tell. + /// @param cid Connection whose watches were dropped. + void sendResync(ConnectionId cid) { + if (eventsNegotiated(cid)) { + postHere(cid, ::morph::wire::encode(::morph::wire::makeEvent(::morph::wire::kResyncTopic, "{}"))); + } } /// @brief Body of `post`. On the server strand. @@ -1555,7 +1584,8 @@ class RemoteServer : public std::enable_shared_from_this { /// `authenticate` refuses, drops every watch @p cid holds and becomes the /// baseline: a watch was admitted for one principal, and must not keep /// delivering to another who now speaks on the same connection. A new - /// token for the same principal keeps them. `cid == 0` and a connection + /// token for the same principal keeps them. A connection that held any + /// watch is sent one `resync` (`sendResync`). `cid == 0` and a connection /// not open have no baseline. /// @param principal The envelope's verified principal, or empty. /// @param cid Connection the envelope arrived on; `0` for unscoped. @@ -1565,8 +1595,12 @@ class RemoteServer : public std::enable_shared_from_this { } auto const [baseline, first] = _principalBaselines.try_emplace(cid, principal); if (!first && baseline->second != principal) { + bool const watched = _watches.topicCount(cid) != 0; _watches.dropConnection(cid); baseline->second = principal; + if (watched) { + sendResync(cid); + } } } diff --git a/tests/test_push_watch.cpp b/tests/test_push_watch.cpp index 6659a2b47..e1db562af 100644 --- a/tests/test_push_watch.cpp +++ b/tests/test_push_watch.cpp @@ -3,6 +3,7 @@ // The server's per-connection watch table and the core `watch`/`unwatch` // kinds: what a watch may name, how it is stored, and what drops it. +#include #include #include #include @@ -804,7 +805,23 @@ struct EventWire { return sent; } - /// The `changes` payloads received, decoded. + /// The topic of every event received, in order. + std::vector topics() { + std::vector names; + for (auto const& frame : snapshot()) { + auto const env = ::morph::wire::decode(frame); + REQUIRE(env.kind == ::morph::wire::kEventKind); + ::morph::wire::EventBody event; + REQUIRE_FALSE(glz::read_json(event, env.body)); + names.push_back(event.topic); + } + return names; + } + + /// How many `resync` events were received. + std::size_t resyncs() { return static_cast(std::ranges::count(topics(), ::morph::wire::kResyncTopic)); } + + /// The `changes` payloads received, decoded; `resync` events are skipped. std::vector<::morph::wire::ChangesBody> changes() { std::vector<::morph::wire::ChangesBody> bodies; for (auto const& frame : snapshot()) { @@ -812,6 +829,9 @@ struct EventWire { REQUIRE(env.kind == ::morph::wire::kEventKind); ::morph::wire::EventBody event; REQUIRE_FALSE(glz::read_json(event, env.body)); + if (event.topic == ::morph::wire::kResyncTopic) { + continue; + } REQUIRE(event.topic == ::morph::wire::kChangesTopic); auto body = ::morph::wire::readChangesBody(event.payload.str); REQUIRE(body); @@ -852,9 +872,10 @@ struct FanRig { EventWire& wire(ConnectionId cid) { return *wires.at(cid); } - void watch(ConnectionId cid, ::morph::wire::WatchBody const& body, std::string const& token = {}) { + void watch(ConnectionId cid, ::morph::wire::WatchBody const& body, std::string const& token = {}, + std::string_view kind = ::morph::wire::kWatchKind) { ::morph::wire::Envelope env; - env.kind = std::string{::morph::wire::kWatchKind}; + env.kind = std::string{kind}; env.callId = 2; env.session.token = token; env.body = glz::write_json(body).value_or(std::string{}); @@ -1092,6 +1113,8 @@ TEST_CASE("execute_under_other_principal_drops_watches_before_its_fanout", "[pus // ...to the connection still watching, but not to the watch bob's execute dropped. CHECK(rig.wire(other).changes().size() == 1U); CHECK(rig.wire(shared).changes().empty()); + CHECK(rig.wire(shared).resyncs() == 1U); + CHECK(rig.wire(other).resyncs() == 0U); CHECK(rig.recordWatchers("T_FanRow") == std::vector{other}); } @@ -1176,6 +1199,9 @@ TEST_CASE("expired_watch_dropped_at_send", "[push][watch]") { CHECK(rig.wire(watcher).changes().size() == 1U); CHECK(rig.wire(caller).changes().size() == 2U); CHECK(rig.recordWatchers("T_FanRow") == std::vector{caller}); + // carol is told once, after the event she did receive, and the caller not at all. + CHECK(rig.wire(watcher).topics() == std::vector{"changes", "resync"}); + CHECK(rig.wire(caller).resyncs() == 0U); } TEST_CASE("lister_not_writer_still_receives", "[push][watch]") { @@ -1259,6 +1285,7 @@ TEST_CASE("refused_after_policy_change_drops_watch", "[push][watch]") { rig.flush(); CHECK(rig.wire(reader).changes().size() == 1U); + CHECK(rig.wire(reader).resyncs() == 1U); CHECK(rig.recordWatchers("T_FanRow").empty()); } @@ -1279,6 +1306,7 @@ TEST_CASE("instance_watch_reauthorised_at_send", "[push][watch]") { rig.flush(); CHECK(rig.wire(watcher).changes().size() == 1U); + CHECK(rig.wire(watcher).resyncs() == 1U); CHECK(rig.instanceWatchers(kFanModel, "k1").empty()); } @@ -1316,3 +1344,94 @@ TEST_CASE("assign_promoted_instance_reaches_watcher", "[push][watch]") { CHECK(bodies.front().instances.front().instance == mid); CHECK(rig.wire(caller).snapshot().empty()); } + +TEST_CASE("expired_session_drop_is_one_resync", "[push][watch]") { + auto guard = std::make_shared(); + FanRig rig{std::chrono::milliseconds{0}, guard}; + auto const caller = rig.connect(); + auto const watcher = rig.connect(); + rig.watch(watcher, {.records = {"T_FanRow"}, .instances = {{.modelType = std::string{kFanModel}, .primary = "k1"}}}, + "carol"); + auto const mid = rig.create(caller, "k1", "alice"); + + guard->expire("carol"); + // A write that reaches both of carol's topics drops both, and owes one resync. + CHECK(rig.execute(caller, mid, "T_FanEdit", R"({"id":1})", "alice").kind == "ok"); + CHECK(rig.execute(caller, mid, "T_FanTouch", R"({"by":1})", "alice").kind == "ok"); + rig.flush(); + + CHECK(rig.wire(watcher).topics() == std::vector{"resync"}); +} + +TEST_CASE("principal_change_owes_one_resync", "[push][watch]") { + FanRig rig{std::chrono::milliseconds{0}, std::make_shared()}; + auto const cid = rig.connect(); + auto const idle = rig.connect(); + rig.watch(cid, records({"T_FanRow", "T_FanOther"}), "alice"); + rig.watch(idle, records({}), "alice"); + + // A register under bob drops both of alice's watches at once. + (void)rig.create(cid, {}, "bob"); + (void)rig.create(idle, {}, "bob"); + rig.flush(); + + CHECK(rig.wire(cid).topics() == std::vector{"resync"}); + // A connection that watched nothing has nothing to re-send. + CHECK(rig.wire(idle).snapshot().empty()); +} + +TEST_CASE("client_drops_owe_no_resync", "[push][watch]") { + FanRig rig; + auto const cid = rig.connect(); + rig.watch(cid, records({"T_FanRow", "T_FanOther"})); + rig.watch(cid, records({"T_FanRow"}), {}, ::morph::wire::kUnwatchKind); + rig.watch(cid, records({"T_FanRow"}, true)); + rig.flush(); + + CHECK(rig.recordWatchers("T_FanOther").empty()); + CHECK(rig.wire(cid).snapshot().empty()); +} + +TEST_CASE("private_keys_stripped_between_unauthenticated_connections", "[push][watch]") { + // No principal is verified on either connection: they are not the same + // principal, so the other one is not given the keys, and the caller keeps + // them only because it is the calling connection. + FanRig rig; + auto const caller = rig.connect(); + auto const other = rig.connect(); + rig.watch(caller, records({"T_FanPrivate"})); + rig.watch(other, records({"T_FanPrivate"})); + auto const mid = rig.create(caller); + + CHECK(rig.execute(caller, mid, "T_FanEditPrivate", R"({"id":4})").kind == "ok"); + rig.flush(); + + auto const mine = rig.wire(caller).changes(); + REQUIRE(mine.size() == 1U); + CHECK(keysOf(mine.front().records.front()) == std::vector{"4"}); + auto const theirs = rig.wire(other).changes(); + REQUIRE(theirs.size() == 1U); + REQUIRE(theirs.front().records.size() == 1U); + CHECK_FALSE(theirs.front().records.front().keys.has_value()); +} + +TEST_CASE("private_keys_kept_for_the_calling_connection", "[push][watch]") { + auto guard = std::make_shared(); + FanRig rig{std::chrono::milliseconds{0}, guard}; + auto const caller = rig.connect(); + auto const bob = rig.connect(); + // No other alice connection: the caller keeps its keys, bob does not. + rig.watch(caller, records({"T_FanPrivate"}), "alice"); + rig.watch(bob, records({"T_FanPrivate"}), "bob"); + auto const mid = rig.create(caller, {}, "alice"); + + CHECK(rig.execute(caller, mid, "T_FanEditPrivate", R"({"id":7})", "alice").kind == "ok"); + rig.flush(); + + auto const mine = rig.wire(caller).changes(); + REQUIRE(mine.size() == 1U); + CHECK(keysOf(mine.front().records.front()) == std::vector{"7"}); + auto const theirs = rig.wire(bob).changes(); + REQUIRE(theirs.size() == 1U); + CHECK_FALSE(theirs.front().records.front().keys.has_value()); +} From 31a533f737b3f0c0d855b102126c1259aa48d9da Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 20:37:33 +0200 Subject: [PATCH 37/68] core: coalesce a connection's changes for 50 ms or 256 keys Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/backend.md | 46 +++- .../morph/core/detail/change_coalescer.hpp | 162 +++++++++++ include/morph/core/remote.hpp | 158 +++++++++-- tests/test_push_watch.cpp | 256 +++++++++++++++++- 4 files changed, 592 insertions(+), 30 deletions(-) create mode 100644 include/morph/core/detail/change_coalescer.hpp diff --git a/docs/spec/core/backend.md b/docs/spec/core/backend.md index 7783f7739..ed867321d 100644 --- a/docs/spec/core/backend.md +++ b/docs/spec/core/backend.md @@ -1069,8 +1069,8 @@ server one per connection. keeping a `resync` the connection is owed — the held-back events were the connection's, not the old writer's; an empty `write` removes it. `closeConnection(cid)` drops it. The writer lives - next to the connection scopes, in a `cid → {writer, owesResync}` map that - only the strand touches. + next to the connection scopes, in a `cid → {writer, owesResync, queued + changes}` map that only the strand touches ([Coalescing](#coalescing)). - `post(cid, envelope)` is callable from any thread. On the calling thread, an envelope larger than `wire::kMaxEnvelopeBytes` is dropped, logged at `warn` and counted as `Metric::eventOversize` — the peer would refuse to decode it. @@ -1190,7 +1190,9 @@ execute path reads it. The UI layer sends them; nothing on the server outside one `resync` event (`RemoteServer::sendResync`): one per envelope or per fan-out, however many watches went. It is sent only to a connection that negotiated events, and through `post`'s hold-back, so a connection already - held back behind its backlog gets the one `resync` its drain owes. The + held back behind its backlog gets the one `resync` its drain owes. A + fan-out first sends what the connection had queued + ([Coalescing](#coalescing)), so its `resync` comes after those changes. The client answers every `resync` by marking its watched entries stale and sending its whole watch set again with `replace: true`; a topic still refused comes back in the `WatchReply`. An `unwatch` and a `replace` are the @@ -1231,8 +1233,9 @@ through `dispatchExecute`. keys derived, and its instance topics as `InstanceTopic{modelType, primary, instance}`. Before it is encoded, the body passes through `RemoteServer::reauthorise`, the one place a recipient's topics are - narrowed. An event left with no topic is not sent. The event is then - written as `post` writes: dropped above + narrowed. A body left with no topic is not sent. What is left is queued + on the connection and sent merged with the connection's other changes + ([Coalescing](#coalescing)), written as `post` writes: dropped above `wire::kMaxEnvelopeBytes`, held back behind a full backlog. - **Re-authorised at send time, by the rule that admitted it.** For each watched topic in a recipient's body: @@ -1262,6 +1265,38 @@ through `dispatchExecute`. caller's verified principal. Two connections with no verified principal are not the same principal for this purpose. See [security.md](../security.md#what-an-event-reveals). +- **Coalescing.** Each connection's writer entry + holds a `detail::ChangeCoalescer`, the changes queued for its next + `changes` event. Re-authorisation happens at fan-out, before a body is + queued; the queue only merges what each fan-out already allowed. + - *Merging.* Records are merged per type, each key text once, in arrival + order. A whole-type record (keys absent) absorbs that type's keys, queued + before or after it, so a type a recipient was sent without keys + (`privateKeys`) is never given keys by a merge. A record whose merged keys + would exceed `effects::kMaxChangeKeys` (4096) becomes whole-type: the + client's reader refuses a longer list, and a whole-type change + over-notifies but never misses. Instance topics are deduplicated on + `{modelType, primary, instance}`. + - *When the queue is sent.* The first change queued opens a window of + `ServerConfig::coalesceWindow` (50 ms by default) on the server's strand + timer — the `TimeoutScheduler` drain deadlines use, created by the first + that needs it. Its callback holds the server weakly and posts the flush + to the strand, where the task holds the server; each window is numbered, + so a late close cannot send a later window's queue. The queue is sent at + once, without waiting for the window, when `keyCount()` reaches + `detail::kCoalesceKeyLimit` (256: one per key, one per whole-type record, + one per instance topic), and before a `resync` a fan-out owes the + connection. A window of zero sends each fan-out at once. + - *What is not queued.* Nothing is queued for a connection with no writer, + or one held back behind its backlog: its drain sends a `resync`, which + covers every change since. A closed connection drops its queue with its + writer, and its window's close then finds nothing to send. + - *Volume.* A window opens only when the previous one has closed, so window + closes send a connection at most 20 events a second at 50 ms. A queue + sent at the key limit can come sooner, so the full bound is 20 events a + second plus one per 256 keys (or topics) the connection is sent. + A burst of one hundred writes to one type reaches each watching + connection as one event, so one refetch per showing query. - **The registries are read at construction.** Every `RemoteServer` constructor reads the record and effects registries once, so a process whose declarations are invalid fails with `EffectsRegistrationError` when the server @@ -2398,6 +2433,7 @@ server: each call is a loop task, so the loop serialises them. | `minProtocolVersion`, `maxProtocolVersion` | Inclusive `hello` range; default `{kProtocolVersion, kProtocolVersion}`. | | `payloadCompleteness` | `PayloadCompleteness`; default `Lenient`. | | `services` | `std::vector>`: extensions consulted, in order, for a kind the server does not answer itself; a null entry throws `std::invalid_argument`. See [Envelope services](#envelope-services). | +| `coalesceWindow` | `std::chrono::milliseconds`, default `50 ms`: how long a connection's changes are merged before one `changes` event is sent; zero sends each fan-out at once; negative throws `std::invalid_argument`. See [Coalescing](#coalescing). | ### `SimulatedRemoteBackend` diff --git a/include/morph/core/detail/change_coalescer.hpp b/include/morph/core/detail/change_coalescer.hpp new file mode 100644 index 000000000..21636168a --- /dev/null +++ b/include/morph/core/detail/change_coalescer.hpp @@ -0,0 +1,162 @@ +// SPDX-License-Identifier: Apache-2.0 + +#pragma once +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../change_types.hpp" +#include "../wire.hpp" + +/// @file +/// @brief What `RemoteServer` has queued for one connection's next `changes` +/// event. + +namespace morph::backend::detail { + +/// @brief The queue size at which `RemoteServer` sends a connection's queued +/// changes without waiting for its coalescing window to close. +/// +/// Counted as `ChangeCoalescer::keyCount()` counts. It bounds how much one event +/// carries, so a burst of keyed writes is sent in pieces a client can refetch +/// for, rather than held until one body nears `effects::kMaxChangeKeys`. +inline constexpr std::size_t kCoalesceKeyLimit = 256; + +/// @brief The changes queued for one connection, merged into the body of its +/// next `changes` event. +/// +/// - **Records are merged per type.** Keys are appended in arrival order, each +/// key text once. +/// - **A whole-type change absorbs keys.** A record queued without keys (a +/// `changes` effect, or a type whose keys were stripped for this recipient) +/// stays whole-type: the keys queued before it are discarded and the keys +/// queued after it are ignored. A merge never adds keys to a whole-type entry, +/// so a recipient that was sent no keys for a record is sent none. +/// - **Never more than `effects::kMaxChangeKeys` keys in one record.** A record +/// whose merged keys would exceed it becomes whole-type, which over-notifies +/// but never misses, and which the client's reader would otherwise refuse. +/// - **Instance topics are deduplicated** on `{modelType, primary, instance}`. +/// +/// Not synchronised: `RemoteServer` keeps one per connection on its strand. +class ChangeCoalescer { +public: + /// @brief Queues what one execute changed. + /// @param changes The changes to merge into the queue. + void add(::morph::effects::Changes const& changes) { + for (auto const& change : changes.records) { + addRecord(change); + } + for (auto const& change : changes.instances) { + addInstance(change.modelType, change.primary, change.instance); + } + } + + /// @brief Queues one recipient's share of a fan-out, already narrowed for it. + /// @param body The topics this connection is to be sent. + void add(::morph::wire::ChangesBody const& body) { + for (auto const& change : body.records) { + addRecord(change); + } + for (auto const& topic : body.instances) { + addInstance(topic.modelType, topic.primary, topic.instance); + } + } + + /// @brief How much is queued: one per key, plus one per whole-type record + /// and one per instance topic. + /// + /// A topic without keys is counted, so the count bounds the topics of the + /// next event as well as its keys. + /// @return The count `kCoalesceKeyLimit` is compared with. + [[nodiscard]] std::size_t keyCount() const { return _keyCount; } + + /// @brief Whether nothing is queued. + /// @return `true` when `take()` would return an empty body. + [[nodiscard]] bool empty() const { return _records.empty() && _instances.empty(); } + + /// @brief Hands over the merged body and empties the queue. + /// @return Every queued record, in the order its type was first queued, and + /// every distinct instance topic, in the order first queued. + [[nodiscard]] ::morph::wire::ChangesBody take() { + ::morph::wire::ChangesBody body; + body.records.reserve(_records.size()); + for (auto& entry : _records) { + ::morph::effects::RecordChange change{.record = std::move(entry.record), .keys = std::nullopt}; + if (!entry.whole) { + change.keys = std::move(entry.keys); + } + body.records.push_back(std::move(change)); + } + body.instances = std::move(_instances); + _records.clear(); + _recordIndex.clear(); + _instances.clear(); + _seenInstances.clear(); + _keyCount = 0; + return body; + } + +private: + /// One record type's merged change. + struct Entry { + std::string record; + bool whole{false}; + std::vector keys; + std::unordered_set seen; + }; + + void addRecord(::morph::effects::RecordChange const& change) { + auto [found, inserted] = _recordIndex.try_emplace(change.record, _records.size()); + if (inserted) { + _records.push_back(Entry{.record = change.record, .whole = false, .keys = {}, .seen = {}}); + } + Entry& entry = _records[found->second]; + if (entry.whole) { + return; + } + if (!change.keys) { + makeWhole(entry); + return; + } + for (auto const& key : *change.keys) { + if (entry.seen.insert(key.str).second) { + entry.keys.push_back(key); + ++_keyCount; + } + } + if (entry.keys.size() > ::morph::effects::kMaxChangeKeys) { + makeWhole(entry); + } + } + + void makeWhole(Entry& entry) { + _keyCount -= entry.keys.size(); + ++_keyCount; + entry.whole = true; + entry.keys = {}; + entry.seen = {}; + } + + void addInstance(std::string_view modelType, std::string_view primary, std::uint64_t instance) { + if (_seenInstances.emplace(std::string{modelType}, std::string{primary}, instance).second) { + _instances.push_back(::morph::wire::InstanceTopic{ + .modelType = std::string{modelType}, .primary = std::string{primary}, .instance = instance}); + ++_keyCount; + } + } + + std::vector _records; + std::unordered_map _recordIndex; + std::vector<::morph::wire::InstanceTopic> _instances; + std::set> _seenInstances; + std::size_t _keyCount{0}; +}; + +} // namespace morph::backend::detail diff --git a/include/morph/core/remote.hpp b/include/morph/core/remote.hpp index 78026cfd4..85357276b 100644 --- a/include/morph/core/remote.hpp +++ b/include/morph/core/remote.hpp @@ -33,6 +33,7 @@ #include "changes.hpp" #include "completion.hpp" #include "connection_writer.hpp" +#include "detail/change_coalescer.hpp" #include "detail/owner_probe.hpp" #include "detail/reply_router.hpp" #include "detail/watch_table.hpp" @@ -326,6 +327,15 @@ struct ServerConfig { /// answered `err "unknown envelope kind: "`. A null entry is /// refused by the constructor. std::vector> services; + + /// @brief How long a connection's changes are merged before they are sent + /// as one `changes` event. + /// + /// The window opens at the first change queued for the connection, and its + /// queue is sent when the window closes, or at once when + /// `detail::kCoalesceKeyLimit` keys are queued. Zero sends each fan-out's + /// event at once, unmerged. Negative is refused by the constructor. + std::chrono::milliseconds coalesceWindow{50}; }; /// @brief Server-side message handler that owns model instances and dispatches actions. @@ -758,11 +768,16 @@ class RemoteServer : public std::enable_shared_from_this { /// Each recipient's watches are then re-authorised by the rule that /// admitted them (`reauthorise`): a topic whose watch fails is dropped /// from the table and from the event, and a connection that lost any - /// watch is sent one `resync` after its event (`sendResync`). A record registered with - /// `morph::record::privateKeys` reaches a principal other than - /// @p callerPrincipal without its keys. Each connection's event holds only - /// the topics left, is not sent when none are, and passes through - /// `post`'s backlog hold-back and size cap. + /// watch is sent one `resync` (`sendResync`), after whatever it had queued. A + /// record registered with `morph::record::privateKeys` reaches a principal + /// other than @p callerPrincipal without its keys. + /// + /// What is left is queued on the connection (`detail::ChangeCoalescer`), + /// not sent: the queue is merged for `ServerConfig::coalesceWindow` from + /// its first change, or until `detail::kCoalesceKeyLimit` keys are queued, + /// and then sent as one `changes` event through `post`'s backlog hold-back + /// and size cap. Nothing is queued for a connection with no writer, or one + /// held back behind its backlog, which its drain's `resync` covers. /// @param changes What one execute changed, from `effects::deriveChanges`. /// @param caller Connection the execute arrived on; `0` for unscoped. /// @param callerPrincipal The execute's verified principal; empty when none was. @@ -785,11 +800,11 @@ class RemoteServer : public std::enable_shared_from_this { change.primary = inst->sharedKey->second; } } - auto const recipients = change.primary.empty() ? scopesHolding(mid) - : _watches.watchersOf(change.modelType, change.primary); - ::morph::wire::InstanceTopic const topic{ - .modelType = std::move(change.modelType), .primary = std::move(change.primary), - .instance = change.instance}; + auto const recipients = + change.primary.empty() ? scopesHolding(mid) : _watches.watchersOf(change.modelType, change.primary); + ::morph::wire::InstanceTopic const topic{.modelType = std::move(change.modelType), + .primary = std::move(change.primary), + .instance = change.instance}; for (ConnectionId const cid : recipients) { if (eventsNegotiated(cid)) { bodies[cid].instances.push_back(topic); @@ -800,15 +815,12 @@ class RemoteServer : public std::enable_shared_from_this { for (auto& [cid, body] : bodies) { bool const dropped = reauthorise(cid, body, caller, callerPrincipal, authentications); if (!body.records.empty() || !body.instances.empty()) { - std::string payload; - (void)glz::write_json(body, payload); - auto envelope = - ::morph::wire::encode(::morph::wire::makeEvent(::morph::wire::kChangesTopic, std::move(payload))); - if (!oversized(cid, envelope)) { - postHere(cid, std::move(envelope)); - } + queueChanges(cid, body); } if (dropped) { + // What was queued before the drop goes first, so the resync is + // the last word on everything the connection watched. + flushChanges(cid); sendResync(cid); } } @@ -921,6 +933,12 @@ class RemoteServer : public std::enable_shared_from_this { struct ConnectionOutbox { ConnectionWriter writer; bool owesResync{false}; + // The changes merged for the connection's next `changes` event. + detail::ChangeCoalescer queued; + // The armed coalescing window, `0` when none is: the timer's handle, + // and the number the flush it posts must match to send the queue. + ::morph::async::detail::TimeoutScheduler::Handle windowTimer{0}; + std::uint64_t window{0}; }; /// @brief Rejects a configuration whose protocol range is empty. @@ -931,6 +949,9 @@ class RemoteServer : public std::enable_shared_from_this { if (config.minProtocolVersion > config.maxProtocolVersion) { throw std::invalid_argument("ServerConfig: minProtocolVersion must not exceed maxProtocolVersion"); } + if (config.coalesceWindow.count() < 0) { + throw std::invalid_argument("ServerConfig: coalesceWindow must not be negative"); + } if (std::ranges::find(config.services, nullptr) != config.services.end()) { throw std::invalid_argument("ServerConfig: services must not hold a null entry"); } @@ -991,6 +1012,10 @@ class RemoteServer : public std::enable_shared_from_this { /// @param cid Connection scope to close. void closeConnectionHere(ConnectionId cid) { noteOwner("RemoteServer::closeConnection"); + // The connection's queued changes go with its writer. + if (auto found = _writers.find(cid); found != _writers.end() && found->second.windowTimer != 0) { + _strandTimer->cancel(found->second.windowTimer); + } _writers.erase(cid); _facts.erase(cid); _watches.dropConnection(cid); @@ -1188,6 +1213,89 @@ class RemoteServer : public std::enable_shared_from_this { } } + /// @brief Queues @p body for connection @p cid's next `changes` event. On + /// the server strand. + /// + /// Sent at once when the queue reaches `detail::kCoalesceKeyLimit` or the + /// window is zero; otherwise the first change queued opens the window, + /// whose close posts `closeWindow` back to the strand. Dropped for a + /// connection with no writer, and for one owed a `resync`, which tells it + /// more than the changes would. + /// @param cid Recipient connection. + /// @param body Its re-authorised share of one fan-out. + void queueChanges(ConnectionId cid, ::morph::wire::ChangesBody const& body) { + auto found = _writers.find(cid); + if (found == _writers.end() || found->second.owesResync) { + return; + } + ConnectionOutbox& outbox = found->second; + outbox.queued.add(body); + if (_config.coalesceWindow.count() == 0 || outbox.queued.keyCount() >= detail::kCoalesceKeyLimit) { + flushChanges(cid); + return; + } + if (outbox.window != 0) { + return; + } + if (!_strandTimer) { + _strandTimer = std::make_unique<::morph::async::detail::TimeoutScheduler>(); + } + std::uint64_t const window = ++_nextWindow; + outbox.window = window; + // Weak, as a drain deadline is: the timer does not keep alive a server + // nothing else refers to, whose connections are gone with it. The + // strand task it posts holds the server, as every strand task does. + outbox.windowTimer = _strandTimer->schedule(_config.coalesceWindow, [weak = weak_from_this(), cid, window] { + if (auto self = weak.lock()) { + self->_strand.postTask([self, cid, window] { self->closeWindow(cid, window); }); + } + }); + } + + /// @brief Sends connection @p cid's queue when its window @p window + /// closes. On the server strand. + /// + /// Nothing is sent when the queue was sent since (by the key limit or a + /// `resync`) and a later window is open, or when the connection closed. + /// @param cid Connection whose window closed. + /// @param window The number its `queueChanges` gave the window. + void closeWindow(ConnectionId cid, std::uint64_t window) { + noteOwner("RemoteServer::closeWindow"); + auto found = _writers.find(cid); + if (found == _writers.end() || found->second.window != window) { + return; + } + found->second.window = 0; + found->second.windowTimer = 0; + flushChanges(cid); + } + + /// @brief Sends what connection @p cid has queued as one `changes` event + /// through `postHere`, and closes its window. On the server strand. + /// @param cid Connection whose queue is sent. + void flushChanges(ConnectionId cid) { + auto found = _writers.find(cid); + if (found == _writers.end()) { + return; + } + ConnectionOutbox& outbox = found->second; + if (outbox.windowTimer != 0) { + _strandTimer->cancel(outbox.windowTimer); + } + outbox.window = 0; + outbox.windowTimer = 0; + if (outbox.queued.empty()) { + return; + } + std::string payload; + (void)glz::write_json(outbox.queued.take(), payload); + auto envelope = + ::morph::wire::encode(::morph::wire::makeEvent(::morph::wire::kChangesTopic, std::move(payload))); + if (!oversized(cid, envelope)) { + postHere(cid, std::move(envelope)); + } + } + /// @brief Body of `post`. On the server strand. /// @param cid Connection to write to. /// @param envelope The encoded envelope, within the size cap. @@ -1414,13 +1522,13 @@ class RemoteServer : public std::enable_shared_from_this { return; } std::uint64_t const waiterId = ++_nextDrainWaiter; - if (!_drainTimer) { - _drainTimer = std::make_unique<::morph::async::detail::TimeoutScheduler>(); + if (!_strandTimer) { + _strandTimer = std::make_unique<::morph::async::detail::TimeoutScheduler>(); } // Weak: a pending deadline does not keep a server alive that nothing // else refers to. The expiry is posted back, since the waiters are // strand state and the timer fires on its I/O loop. - auto const handle = _drainTimer->schedule(deadline, [weak = weak_from_this(), waiterId] { + auto const handle = _strandTimer->schedule(deadline, [weak = weak_from_this(), waiterId] { if (auto self = weak.lock()) { self->_strand.postTask([self, waiterId] { self->expireDrainWaiter(waiterId); }); } @@ -1455,7 +1563,7 @@ class RemoteServer : public std::enable_shared_from_this { auto waiters = std::move(_drainWaiters); _drainWaiters.clear(); for (auto& waiter : waiters) { - _drainTimer->cancel(waiter.timer); + _strandTimer->cancel(waiter.timer); waiter.promise->resolve(true); } } @@ -2623,8 +2731,12 @@ class RemoteServer : public std::enable_shared_from_this { }; std::vector _drainWaiters; std::uint64_t _nextDrainWaiter{0}; - // Arms drain deadlines; created by the first waiter that needs one. - std::unique_ptr<::morph::async::detail::TimeoutScheduler> _drainTimer; + // Arms drain deadlines and coalescing windows; created by the first that + // needs one. Its callbacks only post back to the strand. + std::unique_ptr<::morph::async::detail::TimeoutScheduler> _strandTimer; + // Numbers each coalescing window, so a window's late close cannot send a + // queue a later window holds. + std::uint64_t _nextWindow{0}; // Drawn off the strand by openConnection(), which answers at once. std::atomic _nextConnectionId{0}; diff --git a/tests/test_push_watch.cpp b/tests/test_push_watch.cpp index e1db562af..dc94668c2 100644 --- a/tests/test_push_watch.cpp +++ b/tests/test_push_watch.cpp @@ -14,6 +14,8 @@ #include #include #include +#include +#include #include #include #include @@ -790,6 +792,8 @@ class FanHello final : public ::morph::backend::IEnvelopeService { struct EventWire { std::mutex mtx; std::vector sent; + /// What `backlog()` reports; the test raises it to hold events back. + std::atomic backlog{0}; ::morph::backend::ConnectionWriter writer() { return {.write = @@ -797,7 +801,7 @@ struct EventWire { std::scoped_lock const lock{mtx}; sent.push_back(std::move(frame)); }, - .backlog = [] { return std::size_t{0}; }}; + .backlog = [this] { return backlog.load(); }}; } std::vector snapshot() { @@ -846,10 +850,14 @@ struct FanRig { std::shared_ptr server; std::map> wires; + /// @p coalesceWindow defaults to zero, so each fan-out's event is written + /// before its strand task ends; the coalescing cases pass their own. explicit FanRig(std::chrono::milliseconds executeTimeout = std::chrono::milliseconds{0}, - std::shared_ptr<::morph::session::IAuthorizer> authorizer = nullptr) { + std::shared_ptr<::morph::session::IAuthorizer> authorizer = nullptr, + std::chrono::milliseconds coalesceWindow = std::chrono::milliseconds{0}) { ::morph::backend::ServerConfig config; config.limits.executeTimeout = executeTimeout; + config.coalesceWindow = coalesceWindow; config.services.push_back(std::make_shared()); server = authorizer ? std::make_shared(pool, std::move(authorizer), std::move(config)) : std::make_shared(pool, std::move(config)); @@ -916,6 +924,19 @@ struct FanRig { return waiter->env; } + /// Runs @p batch through `fanOut` in one server-strand task, as executes + /// on @p caller, and waits for it. + void fanOut(std::vector<::morph::effects::Changes> batch, ConnectionId caller, std::string const& principal = {}) { + std::latch done{1}; + server->strand().post([&] { + for (auto& changes : batch) { + server->fanOut(std::move(changes), caller, principal); + } + done.count_down(); + }); + done.wait(); + } + /// Runs every task already posted to the server strand. void flush() { std::latch done{1}; @@ -1435,3 +1456,234 @@ TEST_CASE("private_keys_kept_for_the_calling_connection", "[push][watch]") { REQUIRE(theirs.size() == 1U); CHECK_FALSE(theirs.front().records.front().keys.has_value()); } + +// ── Coalescing: one connection's changes merged into one event ─────────────── + +namespace { + +using ::morph::backend::detail::ChangeCoalescer; + +/// A change to `T_FanRow` naming @p keys, or the whole type when @p keys is empty. +::morph::effects::Changes rowChange(std::vector const& keys) { + ::morph::effects::RecordChange change{.record = "T_FanRow", .keys = std::nullopt}; + if (!keys.empty()) { + change.keys.emplace(); + for (auto const& key : keys) { + change.keys->push_back(glz::raw_json{key}); + } + } + return {.records = {std::move(change)}, .instances = {}}; +} + +/// @p count changes to `T_FanRow`, one key each, numbered from @p first. +std::vector<::morph::effects::Changes> rowBurst(std::size_t count, std::size_t first = 0) { + std::vector<::morph::effects::Changes> burst; + for (std::size_t idx = 0; idx < count; ++idx) { + burst.push_back(rowChange({std::to_string(first + idx)})); + } + return burst; +} + +/// The keys "0" up to @p count - 1, as text. +std::vector numbered(std::size_t count) { + std::vector keys; + for (std::size_t idx = 0; idx < count; ++idx) { + keys.push_back(std::to_string(idx)); + } + return keys; +} + +constexpr auto kForever = std::chrono::hours{1}; + +} // namespace + +TEST_CASE("burst_of_100_is_one_event", "[push][watch]") { + // The default window, as a deployment runs it. + FanRig rig{std::chrono::milliseconds{0}, nullptr, ::morph::backend::ServerConfig{}.coalesceWindow}; + auto const cid = rig.connect(); + rig.watch(cid, records({"T_FanRow"})); + + rig.fanOut(rowBurst(100), cid); + REQUIRE(::morph::testing::waitUntil([&] { return !rig.wire(cid).snapshot().empty(); })); + rig.flush(); + + auto const bodies = rig.wire(cid).changes(); + REQUIRE(bodies.size() == 1U); + REQUIRE(bodies.front().records.size() == 1U); + CHECK(keysOf(bodies.front().records.front()) == numbered(100)); +} + +TEST_CASE("changes_absorbs_updates_keys", "[push][watch]") { + ChangeCoalescer queue; + queue.add(rowChange({"1", "2"})); + queue.add(rowChange({})); + queue.add(rowChange({"3"})); + queue.add(::morph::effects::Changes{.records = {{.record = "T_FanOther", .keys = std::vector{glz::raw_json{"9"}}}}, + .instances = {}}); + CHECK(queue.keyCount() == 2U); + + auto const body = queue.take(); + REQUIRE(body.records.size() == 2U); + CHECK(body.records[0].record == "T_FanRow"); + CHECK_FALSE(body.records[0].keys.has_value()); + CHECK(body.records[1].record == "T_FanOther"); + CHECK(keysOf(body.records[1]) == std::vector{"9"}); + CHECK(queue.empty()); + CHECK(queue.keyCount() == 0U); +} + +TEST_CASE("updates_keys_merge_once_each", "[push][watch]") { + ChangeCoalescer queue; + queue.add(rowChange({"1", "2"})); + queue.add(rowChange({"2", "3"})); + CHECK(queue.keyCount() == 3U); + auto const body = queue.take(); + REQUIRE(body.records.size() == 1U); + CHECK(keysOf(body.records.front()) == std::vector{"1", "2", "3"}); +} + +TEST_CASE("flush_at_256_keys_without_waiting", "[push][watch]") { + FanRig rig{std::chrono::milliseconds{0}, nullptr, kForever}; + auto const cid = rig.connect(); + rig.watch(cid, records({"T_FanRow"})); + + rig.fanOut(rowBurst(::morph::backend::detail::kCoalesceKeyLimit - 1), cid); + CHECK(rig.wire(cid).snapshot().empty()); + + rig.fanOut(rowBurst(1, ::morph::backend::detail::kCoalesceKeyLimit - 1), cid); + auto const bodies = rig.wire(cid).changes(); + REQUIRE(bodies.size() == 1U); + CHECK(keysOf(bodies.front().records.front()) == numbered(::morph::backend::detail::kCoalesceKeyLimit)); +} + +TEST_CASE("burst_over_backlog_yields_one_resync", "[push][watch]") { + FanRig rig{std::chrono::milliseconds{0}, nullptr, kForever}; + auto const cid = rig.connect(); + rig.watch(cid, records({"T_FanRow"})); + rig.wire(cid).backlog = ::morph::backend::kEventBacklogHigh + 1; + + // Past the key limit, so the queue is sent, and held back, at once. + rig.fanOut(rowBurst(300), cid); + CHECK(rig.wire(cid).snapshot().empty()); + + rig.wire(cid).backlog = 0; + rig.server->connectionDrained(cid); + rig.server->connectionDrained(cid); + rig.flush(); + CHECK(rig.wire(cid).topics() == std::vector{"resync"}); +} + +TEST_CASE("merged_keys_never_exceed_client_bound", "[push][watch]") { + std::vector many; + for (std::size_t idx = 0; idx < ::morph::effects::kMaxChangeKeys; ++idx) { + many.push_back(std::to_string(idx)); + } + + SECTION("one body at the bound stays keyed") { + ChangeCoalescer queue; + queue.add(rowChange(many)); + CHECK(queue.keyCount() == ::morph::effects::kMaxChangeKeys); + auto const body = queue.take(); + REQUIRE(body.records.front().keys.has_value()); + CHECK(body.records.front().keys->size() == ::morph::effects::kMaxChangeKeys); + } + SECTION("a merge past the bound becomes whole-type") { + ChangeCoalescer queue; + queue.add(rowChange({"first"})); + queue.add(rowChange(many)); + CHECK(queue.keyCount() == 1U); + auto const body = queue.take(); + REQUIRE(body.records.size() == 1U); + CHECK_FALSE(body.records.front().keys.has_value()); + + // What the client's reader accepts. + std::string payload; + REQUIRE_FALSE(glz::write_json(body, payload)); + CHECK(::morph::wire::readChangesBody(payload).has_value()); + } +} + +TEST_CASE("instance_topics_deduplicated", "[push][watch]") { + ChangeCoalescer queue; + ::morph::effects::InstanceChange const keyed{.modelType = std::string{kFanModel}, .instance = 4, .primary = "k1"}; + ::morph::effects::InstanceChange const other{.modelType = std::string{kFanModel}, .instance = 5, .primary = {}}; + queue.add(::morph::effects::Changes{.records = {}, .instances = {keyed, other}}); + queue.add(::morph::effects::Changes{.records = {}, .instances = {keyed}}); + queue.add(::morph::wire::ChangesBody{ + .records = {}, .instances = {{.modelType = std::string{kFanModel}, .primary = "k1", .instance = 4}}}); + CHECK(queue.keyCount() == 2U); + + auto const body = queue.take(); + REQUIRE(body.instances.size() == 2U); + CHECK(body.instances[0].primary == "k1"); + CHECK(body.instances[0].instance == 4U); + CHECK(body.instances[1].instance == 5U); +} + +TEST_CASE("closed_connection_drops_its_queue", "[push][watch]") { + FanRig rig{std::chrono::milliseconds{0}, nullptr, ::morph::backend::ServerConfig{}.coalesceWindow}; + auto const closing = rig.connect(); + auto const staying = rig.connect(); + rig.watch(closing, records({"T_FanRow"})); + rig.watch(staying, records({"T_FanRow"})); + + std::latch done{1}; + rig.server->strand().post([&] { + rig.server->fanOut(rowChange({"1"}), staying, {}); + rig.server->closeConnection(closing); + done.count_down(); + }); + done.wait(); + + // Both windows opened in the same task, the closed connection's first. + REQUIRE(::morph::testing::waitUntil([&] { return !rig.wire(staying).snapshot().empty(); })); + rig.flush(); + CHECK(rig.wire(closing).snapshot().empty()); + CHECK(rig.wire(staying).changes().size() == 1U); +} + +TEST_CASE("resync_follows_queued_changes", "[push][watch]") { + auto guard = std::make_shared(); + FanRig rig{std::chrono::milliseconds{0}, guard, kForever}; + auto const caller = rig.connect(); + auto const watcher = rig.connect(); + rig.watch(watcher, records({"T_FanRow"}), "carol"); + + rig.fanOut({rowChange({"1"})}, caller, "alice"); + CHECK(rig.wire(watcher).snapshot().empty()); + + // This fan-out drops carol's watch: what was queued goes first, then the resync. + guard->expire("carol"); + rig.fanOut({rowChange({"2"})}, caller, "alice"); + CHECK(rig.wire(watcher).topics() == std::vector{"changes", "resync"}); + auto const bodies = rig.wire(watcher).changes(); + REQUIRE(bodies.size() == 1U); + CHECK(keysOf(bodies.front().records.front()) == std::vector{"1"}); +} + +TEST_CASE("resync_held_back_is_sent_once_on_drain", "[push][watch]") { + auto guard = std::make_shared(); + FanRig rig{std::chrono::milliseconds{0}, guard, ::morph::backend::ServerConfig{}.coalesceWindow}; + auto const caller = rig.connect(); + auto const watcher = rig.connect(); + rig.watch(watcher, records({"T_FanRow"}), "carol"); + rig.wire(watcher).backlog = ::morph::backend::kEventBacklogHigh + 1; + + guard->expire("carol"); + rig.fanOut({rowChange({"1"})}, caller, "alice"); + CHECK(rig.recordWatchers("T_FanRow").empty()); + CHECK(rig.wire(watcher).snapshot().empty()); + + rig.wire(watcher).backlog = 0; + rig.server->connectionDrained(watcher); + rig.server->connectionDrained(watcher); + rig.flush(); + CHECK(rig.wire(watcher).topics() == std::vector{"resync"}); +} + +TEST_CASE("negative_coalesce_window_refused", "[push][watch]") { + ::morph::exec::ThreadPoolExecutor pool{1}; + ::morph::backend::ServerConfig config; + config.coalesceWindow = std::chrono::milliseconds{-1}; + CHECK_THROWS_AS(std::make_shared(pool, std::move(config)), std::invalid_argument); +} From d22c3dc22ac39d9419c0cd80f81c271aec9af875 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 20:55:47 +0200 Subject: [PATCH 38/68] core: a principal change discards changes queued for the previous principal Co-Authored-By: Claude Opus 5.5 --- docs/spec/core/backend.md | 8 +++- .../specs/2026-10-16-server-push-design.md | 3 +- include/morph/core/remote.hpp | 43 ++++++++++++++----- tests/test_push_watch.cpp | 28 +++++++++++- 4 files changed, 67 insertions(+), 15 deletions(-) diff --git a/docs/spec/core/backend.md b/docs/spec/core/backend.md index ed867321d..0c1975288 100644 --- a/docs/spec/core/backend.md +++ b/docs/spec/core/backend.md @@ -1289,8 +1289,12 @@ through `dispatchExecute`. connection. A window of zero sends each fan-out at once. - *What is not queued.* Nothing is queued for a connection with no writer, or one held back behind its backlog: its drain sends a `resync`, which - covers every change since. A closed connection drops its queue with its - writer, and its window's close then finds nothing to send. + covers every change since. A closed connection, and one whose writer is + removed, drops its queue with its writer, and its window's close then + finds nothing to send. A principal change (the baseline in + [Watches](#watches)) discards the queue without sending it, since it was + authorised for the previous principal, and the connection is sent one + `resync` if it held any watch or had anything queued. - *Volume.* A window opens only when the previous one has closed, so window closes send a connection at most 20 events a second at 50 ms. A queue sent at the key limit can come sooner, so the full bound is 20 events a diff --git a/docs/superpowers/specs/2026-10-16-server-push-design.md b/docs/superpowers/specs/2026-10-16-server-push-design.md index a5be909bc..b56c6bcd3 100644 --- a/docs/superpowers/specs/2026-10-16-server-push-design.md +++ b/docs/superpowers/specs/2026-10-16-server-push-design.md @@ -592,7 +592,8 @@ changes visible. - **Values never travel to other connections** (§9), so `secretFields` and row-level filtering stay on the read path, where they are enforced today. - **Volume.** - - Coalescing bounds events to 20 a second per connection. + - Coalescing bounds a connection to at most 20 timer flushes a second, plus one event per 256 keys or topics it + is sent. - The backlog bound (§6) holds events back from a slow reader and replaces them with one `resync`, and the transport's `sendTimeout` closes a reader that stops. - The query-must-read rule (§10) removes the way a fetch could feed itself. diff --git a/include/morph/core/remote.hpp b/include/morph/core/remote.hpp index 85357276b..5ddc5be51 100644 --- a/include/morph/core/remote.hpp +++ b/include/morph/core/remote.hpp @@ -332,9 +332,9 @@ struct ServerConfig { /// as one `changes` event. /// /// The window opens at the first change queued for the connection, and its - /// queue is sent when the window closes, or at once when - /// `detail::kCoalesceKeyLimit` keys are queued. Zero sends each fan-out's - /// event at once, unmerged. Negative is refused by the constructor. + /// queue is sent when the window closes, or at once when 256 keys are + /// queued. Zero sends each fan-out's event at once, unmerged. Negative is + /// refused by the constructor. std::chrono::milliseconds coalesceWindow{50}; }; @@ -663,6 +663,7 @@ class RemoteServer : public std::enable_shared_from_this { return; } if (!writer.write) { + self->discardChanges(cid); self->_writers.erase(cid); return; } @@ -1012,10 +1013,7 @@ class RemoteServer : public std::enable_shared_from_this { /// @param cid Connection scope to close. void closeConnectionHere(ConnectionId cid) { noteOwner("RemoteServer::closeConnection"); - // The connection's queued changes go with its writer. - if (auto found = _writers.find(cid); found != _writers.end() && found->second.windowTimer != 0) { - _strandTimer->cancel(found->second.windowTimer); - } + discardChanges(cid); _writers.erase(cid); _facts.erase(cid); _watches.dropConnection(cid); @@ -1270,6 +1268,26 @@ class RemoteServer : public std::enable_shared_from_this { flushChanges(cid); } + /// @brief Drops what connection @p cid has queued and closes its window, + /// sending nothing. On the server strand. + /// @param cid Connection whose queue is discarded. + /// @return `true` when something was queued. + bool discardChanges(ConnectionId cid) { + auto found = _writers.find(cid); + if (found == _writers.end()) { + return false; + } + ConnectionOutbox& outbox = found->second; + if (outbox.windowTimer != 0) { + _strandTimer->cancel(outbox.windowTimer); + } + outbox.window = 0; + outbox.windowTimer = 0; + bool const queued = !outbox.queued.empty(); + (void)outbox.queued.take(); + return queued; + } + /// @brief Sends what connection @p cid has queued as one `changes` event /// through `postHere`, and closes its window. On the server strand. /// @param cid Connection whose queue is sent. @@ -1692,9 +1710,11 @@ class RemoteServer : public std::enable_shared_from_this { /// `authenticate` refuses, drops every watch @p cid holds and becomes the /// baseline: a watch was admitted for one principal, and must not keep /// delivering to another who now speaks on the same connection. A new - /// token for the same principal keeps them. A connection that held any - /// watch is sent one `resync` (`sendResync`). `cid == 0` and a connection - /// not open have no baseline. + /// token for the same principal keeps them. The changes queued for the + /// connection are discarded, never sent: they were authorised for the + /// previous principal. A connection that held any watch, or had changes + /// queued, is sent one `resync` (`sendResync`), which covers them. + /// `cid == 0` and a connection not open have no baseline. /// @param principal The envelope's verified principal, or empty. /// @param cid Connection the envelope arrived on; `0` for unscoped. void compareWithBaseline(const std::string& principal, ConnectionId cid) { @@ -1704,9 +1724,10 @@ class RemoteServer : public std::enable_shared_from_this { auto const [baseline, first] = _principalBaselines.try_emplace(cid, principal); if (!first && baseline->second != principal) { bool const watched = _watches.topicCount(cid) != 0; + bool const queued = discardChanges(cid); _watches.dropConnection(cid); baseline->second = principal; - if (watched) { + if (watched || queued) { sendResync(cid); } } diff --git a/tests/test_push_watch.cpp b/tests/test_push_watch.cpp index dc94668c2..ef86430db 100644 --- a/tests/test_push_watch.cpp +++ b/tests/test_push_watch.cpp @@ -1557,7 +1557,8 @@ TEST_CASE("flush_at_256_keys_without_waiting", "[push][watch]") { } TEST_CASE("burst_over_backlog_yields_one_resync", "[push][watch]") { - FanRig rig{std::chrono::milliseconds{0}, nullptr, kForever}; + constexpr auto kWindow = std::chrono::milliseconds{20}; + FanRig rig{std::chrono::milliseconds{0}, nullptr, kWindow}; auto const cid = rig.connect(); rig.watch(cid, records({"T_FanRow"})); rig.wire(cid).backlog = ::morph::backend::kEventBacklogHigh + 1; @@ -1569,6 +1570,8 @@ TEST_CASE("burst_over_backlog_yields_one_resync", "[push][watch]") { rig.wire(cid).backlog = 0; rig.server->connectionDrained(cid); rig.server->connectionDrained(cid); + // Past the window, so a queue left behind the hold-back would have been sent. + std::this_thread::sleep_for(kWindow * 5); rig.flush(); CHECK(rig.wire(cid).topics() == std::vector{"resync"}); } @@ -1687,3 +1690,26 @@ TEST_CASE("negative_coalesce_window_refused", "[push][watch]") { config.coalesceWindow = std::chrono::milliseconds{-1}; CHECK_THROWS_AS(std::make_shared(pool, std::move(config)), std::invalid_argument); } + +TEST_CASE("principal_change_discards_queued_changes", "[push][watch]") { + auto guard = std::make_shared(); + FanRig rig{std::chrono::milliseconds{0}, guard, kForever}; + auto const caller = rig.connect(); + auto const cid = rig.connect(); + rig.watch(cid, records({"T_FanRow"}), "carol"); + rig.fanOut({rowChange({"7777"})}, caller, "alice"); + + // An envelope under dave: carol's queue is discarded, not sent, and one resync covers it. + (void)rig.create(cid, {}, "dave"); + rig.flush(); + CHECK(rig.wire(cid).topics() == std::vector{"resync"}); + + // The next event dave is sent holds none of what was queued for carol. + rig.watch(cid, records({"T_FanRow"}), "dave"); + rig.fanOut(rowBurst(::morph::backend::detail::kCoalesceKeyLimit, 1000), caller, "alice"); + auto const bodies = rig.wire(cid).changes(); + REQUIRE(bodies.size() == 1U); + auto const keys = keysOf(bodies.front().records.front()); + CHECK(keys.size() == ::morph::backend::detail::kCoalesceKeyLimit); + CHECK(std::ranges::find(keys, "7777") == keys.end()); +} From 69dfd0124f2b404df577f3cc23cca774647b0712 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 22:29:38 +0200 Subject: [PATCH 39/68] ui: a change marked during a fetch is kept until the next fetch Session::markStale marks an entry stale, and while its fetch is in flight sets refetchAfter: that fetch may have read before the change, so when it settles the entry stays stale and fetches once more through a mounted query. A backend switch marks through it too. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- docs/spec/ui/interpreter.md | 7 ++++- include/morph/ui/interpreter.hpp | 36 +++++++++++++++++++-- tests/CMakeLists.txt | 1 + tests/test_push_cache.cpp | 54 ++++++++++++++++++++++++++++++++ 4 files changed, 94 insertions(+), 4 deletions(-) create mode 100644 tests/test_push_cache.cpp diff --git a/docs/spec/ui/interpreter.md b/docs/spec/ui/interpreter.md index fe0cd6705..8a003af0b 100644 --- a/docs/spec/ui/interpreter.md +++ b/docs/spec/ui/interpreter.md @@ -106,7 +106,7 @@ unbound or the body is `null`; otherwise `(model type, instance token, action, c canonical body is the body's JSON with record fields in name order. - **Cache entries** (`Session`) hold the shown value, `hasValue`, the error text and kind, `pending`, a - generation, `fetchedAt` and `stale`. A fetch takes a new generation; a reply of an older one is + generation, `fetchedAt`, `stale` and `refetchAfter`. A fetch takes a new generation; a reply of an older one is dropped. One entry serves every query with its key, so two screens on one key send one request. - **Latest wins.** A key change moves the query to another entry. The previous key's value is shown until the new entry has a value; a failure for the new key clears it. A failed refetch of the same key @@ -123,6 +123,11 @@ canonical body is the body's JSON with record fields in name order. form's choice options are reads, not submits, and refetch nothing. - **Invalidation** (`invalidates`, `refreshOn`, `refetch`, a backend switch) marks the entry stale and refetches it. +- **A mark during a fetch is kept.** `Session::markStale` marks an entry stale. While the entry's fetch is + in flight, it also sets `refetchAfter`. That fetch may have read before the change, so when it settles + the entry keeps its new value, stays stale, and fetches once more through a mounted query that uses it. + A success never clears a mark that arrived during the fetch. An entry no query uses when its fetch + settles stays stale, and fetches when a query mounts on it. - **Eviction.** An entry no mounted query uses is dropped `keepUnused` after its last user left. Without a scheduler, entries are kept. - **Secrets.** An action's `secretFields` are removed from what its entry keeps. diff --git a/include/morph/ui/interpreter.hpp b/include/morph/ui/interpreter.hpp index 08a0f2cdc..89c3aaa6f 100644 --- a/include/morph/ui/interpreter.hpp +++ b/include/morph/ui/interpreter.hpp @@ -605,8 +605,11 @@ struct CacheEntry { std::uint64_t generation = 0; /// @brief When the last reply was applied. std::optional fetchedAt; - /// @brief Marked stale by an invalidation, `refreshOn` or a backend switch. + /// @brief Marked stale by an invalidation, `refreshOn`, a backend switch or a change. bool stale = false; + /// @brief Marked stale while a fetch was in flight: that fetch may have read before the change, so when it + /// settles the entry stays stale and fetches once more. + bool refetchAfter = false; /// @brief The mounted queries using it. int users = 0; /// @brief The eviction timer, while no query uses it. @@ -985,6 +988,16 @@ class Session { /// @param actionId The action. void actionSucceeded(std::string const& typeId, std::string const& actionId); + /// @brief Marks an entry stale. While its fetch is in flight, the entry also refetches once that fetch settles, + /// because the fetch may have read before the change. + /// @param entry The entry. + void markStale(detail::CacheEntry& entry) { + entry.stale = true; + if (entry.pending.peek()) { + entry.refetchAfter = true; + } + } + private: void settleReply(detail::CacheEntry& entry, std::string const& reply) { auto json = parseJson(reply); @@ -1000,10 +1013,18 @@ class Session { entry.errorKind.set({}); entry.pending.set(false); entry.fetchedAt = now(); - entry.stale = false; + entry.stale = entry.refetchAfter; + if (entry.refetchAfter) { + entry.refetchAfter = false; + refetchMounted(entry); + } }); } + /// @brief Refetches an entry through the mounted queries that use it; one fetch even when several do. + /// @param entry The entry. + void refetchMounted(detail::CacheEntry const& entry); + void settleFailure(detail::CacheEntry& entry, std::string text, std::string kind) { _runtime->batch([&] { noteWrite(); @@ -2514,7 +2535,7 @@ inline void Session::backendChanged(std::string const& backend) { _env.set(std::move(env)); for (auto const& [key, entry] : _entries) { static_cast(key); - entry->stale = true; + markStale(*entry); } std::vector const mounted = _mounted; for (detail::QueryRuntime* const query : mounted) { @@ -2523,6 +2544,15 @@ inline void Session::backendChanged(std::string const& backend) { }); } +inline void Session::refetchMounted(detail::CacheEntry const& entry) { + std::vector const mounted = _mounted; + for (detail::QueryRuntime* const query : mounted) { + if (_runtime->untracked([query] { return query->entry(); }).get() == &entry) { + query->refetchCurrent(); + } + } +} + inline void Session::actionSucceeded(std::string const& typeId, std::string const& actionId) { std::vector const mounted = _mounted; for (detail::QueryRuntime* const query : mounted) { diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index f630285c1..6863f85b1 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -27,6 +27,7 @@ add_executable(morph_tests test_push_catalog.cpp test_push_channel.cpp test_push_watch.cpp + test_push_cache.cpp push_records_late.cpp test_executor_extra.cpp test_strand.cpp diff --git a/tests/test_push_cache.cpp b/tests/test_push_cache.cpp new file mode 100644 index 000000000..b133d45b3 --- /dev/null +++ b/tests/test_push_cache.cpp @@ -0,0 +1,54 @@ +// SPDX-License-Identifier: Apache-2.0 + +// The client cache under server push (spec 8 §9): a change marked while an entry's fetch is in flight is kept until +// the next fetch. + +#include + +#include "ui_interpreter_support.hpp" + +namespace ui = morph::ui; +using ui::testing::Harness; + +namespace { + +constexpr char const* kList = R"({ + "vocab": "ui/1", "id": "list", + "models": { "sample": { "type": "Lims_SampleModel" } }, + "queries": { "rows": { "model": "sample", "action": "Lims_ListResults" } }, + "view": { "kind": "text", "testId": "count", "text": { "fn": "format", "args": [ "{0}", { "query": "rows" } ] } } })"; + +} // namespace + +TEST_CASE("stale_mark_during_fetch_fetches_once_more", "[push][cache]") { + Harness h; + auto screen = h.mount(kList); + REQUIRE(h.dispatcher.count("Lims_ListResults") == 1); + + // The mark lands while the first fetch is in flight, so that fetch may have read before the change. + h.session.backendChanged("primary"); + h.settle(); + REQUIRE(h.dispatcher.count("Lims_ListResults") == 1); + + h.reply(0, "[1]"); + REQUIRE(h.dispatcher.count("Lims_ListResults") == 2); + + h.reply(1, "[1,2]"); + CHECK(h.dispatcher.count("Lims_ListResults") == 2); +} + +TEST_CASE("stale_mark_on_unmounted_fetching_entry_waits_for_a_mount", "[push][cache]") { + Harness h; + auto screen = h.mount(kList); + REQUIRE(h.dispatcher.count("Lims_ListResults") == 1); + h.session.backendChanged("primary"); + screen.reset(); + h.settle(); + + // No query uses the entry when its fetch settles, so it stays stale and is not refetched. + h.reply(0, "[1]"); + CHECK(h.dispatcher.count("Lims_ListResults") == 1); + + auto again = h.mount(kList); + CHECK(h.dispatcher.count("Lims_ListResults") == 2); +} From 2d1e705b5efc87e98bd2eb1c0d7f03c8e7960942 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 22:53:05 +0200 Subject: [PATCH 40/68] ui: every reply applies its action's effects to the cache Session::actionSucceeded takes the reply and the alias's instance token. updates R merges the reply's rows into every entry holding R, matched by the record's key field at the catalog's paths, with no fetch; an entry whose fetch is in flight is also marked. changes R and changesInstance mark the entries that hold R or were read through that instance, and a marked entry refetches through a mounted query. A discarded latest reply only marks. Form, collection and wizard submits go through the same path. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- docs/spec/ui/interpreter.md | 14 ++ include/morph/ui/interpreter.hpp | 264 +++++++++++++++++++++++++++++-- tests/test_push_cache.cpp | 160 ++++++++++++++++++- tests/ui_interpreter_support.hpp | 18 +++ 4 files changed, 444 insertions(+), 12 deletions(-) diff --git a/docs/spec/ui/interpreter.md b/docs/spec/ui/interpreter.md index 8a003af0b..873704c0b 100644 --- a/docs/spec/ui/interpreter.md +++ b/docs/spec/ui/interpreter.md @@ -121,6 +121,20 @@ canonical body is the body's JSON with record fields in name order. collection or a wizard that succeeds, refetches every mounted query of the same model type whose `refreshOn` names its action, in every screen of the session. A collection's own list query and a form's choice options are reads, not submits, and refetch nothing. +- **Every reply applies its action's effects** (spec 8 §9). `Session::actionSucceeded`, which every + success goes through (mutations, and the submits of forms, collections and wizards), takes the reply + and the instance token of the alias the call went through. Each entry knows the records its action + returns, with each record's key field and paths, from the catalog's `ActionInfo::records`; an + `evaluate` entry holds none. + - `updates R` merges the reply's rows of `R` into every entry holding `R`. Rows are matched by + the record's key field, at each path the catalog gives (`header.rows` follows members; a list + on the way is looked through). There is no fetch. An entry whose fetch is in flight is also + marked, because that fetch may have read before the write and would settle over the patch. + - `changes R` marks every entry holding `R` stale, and `changesInstance` every entry read through + the same instance token. A marked entry refetches through a mounted query that uses it. + - A `latest` run whose reply is discarded applies its effects as marks only: its `updates` act as + `changes`, because its values are older than the reply already applied. It runs no `refreshOn`. + - One reply's effects are applied in one reactive batch. - **Invalidation** (`invalidates`, `refreshOn`, `refetch`, a backend switch) marks the entry stale and refetches it. - **A mark during a fetch is kept.** `Session::markStale` marks an entry stale. While the entry's fetch is diff --git a/include/morph/ui/interpreter.hpp b/include/morph/ui/interpreter.hpp index 89c3aaa6f..756580a1e 100644 --- a/include/morph/ui/interpreter.hpp +++ b/include/morph/ui/interpreter.hpp @@ -558,6 +558,116 @@ enum class PatchMiss : std::uint8_t { return value; } +/// @brief The members of a catalog record path: `"header.rows"` is `header`, then `rows`; `""` is none. +/// @param path The path. +/// @return Its members in order. +[[nodiscard]] inline std::vector pathMembers(std::string_view path) { + std::vector members; + for (auto const part : std::views::split(path, '.')) { + if (!part.empty()) { + members.emplace_back(part.begin(), part.end()); + } + } + return members; +} + +/// @brief The rows of a keyed record that a value holds at a path. +/// +/// A list on the way is looked through, as the result walker looks through a `std::vector`. At the path's end a +/// record with the key field is a row, and a record without it is a map whose values are the rows. +/// @param value The value. +/// @param path The members still to follow. +/// @param keyField The record's key field. +/// @param out Where the rows go. +// NOLINTNEXTLINE(misc-no-recursion): one level per path member or nested list. +inline void keyedRowsAt(Value const& value, std::span path, std::string_view keyField, + std::vector& out) { + if (ValueList const* const items = value.asList()) { + for (Value const& item : *items) { + keyedRowsAt(item, path, keyField, out); + } + return; + } + ValueRecord const* const record = value.asRecord(); + if (record == nullptr) { + return; + } + if (!path.empty()) { + if (auto const found = record->find(path.front()); found != record->end()) { + keyedRowsAt(found->second, path.subspan(1), keyField, out); + } + return; + } + if (record->contains(keyField)) { + out.push_back(&value); + return; + } + for (auto const& [name, item] : *record) { + static_cast(name); + if (ValueRecord const* const row = item.asRecord(); row != nullptr && row->contains(keyField)) { + out.push_back(&item); + } + } +} + +/// @brief New rows by the canonical text of their key. +using RowsByKey = std::map>; + +/// @brief Merges new rows over the rows with the same key that a value holds at a path. +/// @param value The value. +/// @param path The members still to follow. +/// @param keyField The record's key field. +/// @param rows The new rows. +/// @return The new value, or nothing when no row matched. +// NOLINTNEXTLINE(misc-no-recursion): one level per path member or nested list. +[[nodiscard]] inline std::optional patchKeyedRows(Value const& value, std::span path, + std::string_view keyField, RowsByKey const& rows) { + if (ValueList const* const items = value.asList()) { + ValueList out = *items; + bool changed = false; + for (Value& item : out) { + if (auto patched = patchKeyedRows(item, path, keyField, rows)) { + item = std::move(*patched); + changed = true; + } + } + return changed ? std::optional{Value{std::move(out)}} : std::nullopt; + } + ValueRecord const* const record = value.asRecord(); + if (record == nullptr) { + return std::nullopt; + } + if (!path.empty()) { + auto const found = record->find(path.front()); + if (found == record->end()) { + return std::nullopt; + } + auto patched = patchKeyedRows(found->second, path.subspan(1), keyField, rows); + if (!patched) { + return std::nullopt; + } + ValueRecord out = *record; + out.insert_or_assign(path.front(), std::move(*patched)); + return Value{std::move(out)}; + } + if (auto const key = record->find(keyField); key != record->end()) { + auto const row = rows.find(canonicalJson(key->second)); + return row == rows.end() ? std::nullopt : std::optional{mergeRow(value, *row->second)}; + } + ValueRecord out = *record; + bool changed = false; + for (auto& [name, item] : out) { + static_cast(name); + if (ValueRecord const* const row = item.asRecord(); row != nullptr && row->contains(keyField)) { + if (auto patched = patchKeyedRows(item, {}, keyField, rows)) { + item = std::move(*patched); + changed = true; + } + } + } + return changed ? std::optional{Value{std::move(out)}} : std::nullopt; +} + /// @brief One optimistic write still waiting for its reply. struct OptimisticLayer { /// @brief The run that made it. @@ -616,6 +726,19 @@ struct CacheEntry { reactive::TimerHandle evict; /// @brief The fields never cached. std::vector secretFields; + /// @brief The alias's instance token the entry was read through. + std::string instanceToken; + /// @brief The records the entry's action returns, with the key field and paths of each; empty for an + /// `evaluate` entry, which holds none. + std::vector records; + + /// @brief The catalog entry of a record the entry holds. + /// @param record The record's wire name. + /// @return It, or `nullptr` when the entry does not hold the record. + [[nodiscard]] ActionRecordInfo const* holding(std::string_view record) const { + auto const found = std::ranges::find(records, record, &ActionRecordInfo::record); + return found == records.end() ? nullptr : &*found; + } }; class QueryRuntime; @@ -624,6 +747,12 @@ class ScreenCore; } // namespace detail +/// @brief How a mutation's reply was used. +enum class ReplyUse : std::uint8_t { + Applied, ///< The reply is the latest: its values may patch the cache. + Discarded, ///< A newer run's reply was applied first, so its values are older and only mark. +}; + /// @brief What running a host command (`host/1`) did. struct HostCommandResult { /// @brief The outcome. @@ -877,9 +1006,13 @@ class Session { entry->action = action; entry->body = text; entry->evaluate = evaluate; + entry->instanceToken = token; if (_options.catalog.action) { if (auto const info = _options.catalog.action(action)) { entry->secretFields = info->secretFields; + if (!evaluate) { + entry->records = info->records; + } } } found = _entries.emplace(std::move(key), std::move(entry)).first; @@ -982,11 +1115,21 @@ class Session { /// @param query The query. void forgetQuery(detail::QueryRuntime& query) { std::erase(_mounted, &query); } - /// @brief A mutation of @p typeId running @p actionId succeeded: refetch every mounted query of that model type - /// whose `refreshOn` names it (spec 5 §5). - /// @param typeId The model type. - /// @param actionId The action. - void actionSucceeded(std::string const& typeId, std::string const& actionId); + /// @brief A call of @p actionId on a model of @p typeId succeeded: applies the action's effects to the cache, and + /// refetches every mounted query of that model type whose `refreshOn` names it (spec 5 §5). + /// + /// `updates R` merges the reply's rows of `R` into each entry holding `R`, matched by the record's key field at + /// the entry's paths, with no fetch; an entry whose fetch is in flight also fetches once more, since that fetch + /// may have read before the write. `changes R` marks each entry holding `R` stale. `changesInstance` marks each + /// entry read through the same instance. A marked entry refetches through a mounted query that uses it. A + /// discarded reply only marks: its `updates` act as `changes`, and `refreshOn` is not run. + /// @param typeId The model type. + /// @param actionId The action. + /// @param reply The reply's value. + /// @param use Whether the reply was applied or discarded. + /// @param instanceToken The instance token of the alias the call went through. + void actionSucceeded(std::string const& typeId, std::string const& actionId, Value const& reply, ReplyUse use, + std::string const& instanceToken); /// @brief Marks an entry stale. While its fetch is in flight, the entry also refetches once that fetch settles, /// because the fetch may have read before the change. @@ -1025,6 +1168,15 @@ class Session { /// @param entry The entry. void refetchMounted(detail::CacheEntry const& entry); + /// @brief Applies a reply's effects to the cache, for `actionSucceeded`. + /// @param typeId The model type. + /// @param actionId The action. + /// @param reply The reply's value. + /// @param use Whether the reply was applied or discarded. + /// @param instanceToken The instance token of the alias the call went through. + void applyReplyEffects(std::string const& typeId, std::string const& actionId, Value const& reply, ReplyUse use, + std::string const& instanceToken); + void settleFailure(detail::CacheEntry& entry, std::string text, std::string kind) { _runtime->batch([&] { noteWrite(); @@ -2553,7 +2705,90 @@ inline void Session::refetchMounted(detail::CacheEntry const& entry) { } } -inline void Session::actionSucceeded(std::string const& typeId, std::string const& actionId) { +inline void Session::applyReplyEffects(std::string const& typeId, std::string const& actionId, Value const& reply, + ReplyUse use, std::string const& instanceToken) { + if (!_options.catalog.action) { + return; + } + std::optional const info = _options.catalog.action(actionId); + if (!info || !info->effects) { + return; + } + std::vector marked; + auto const mark = [&](detail::CacheEntry& entry) { + markStale(entry); + if (std::ranges::find(marked, &entry) == marked.end()) { + marked.push_back(&entry); + } + }; + for (ActionEffect const& effect : *info->effects) { + if (effect.kind == "changesInstance") { + for (auto const& [key, entry] : _entries) { + static_cast(key); + if (!entry->evaluate && entry->typeId == typeId && entry->instanceToken == instanceToken) { + mark(*entry); + } + } + continue; + } + if (!effect.record || (effect.kind != "updates" && effect.kind != "changes")) { + continue; + } + std::string const& record = *effect.record; + // The reply's rows of the record, by key; none when the reply does not hold it keyed, or must only mark. + std::vector found; + auto const own = std::ranges::find(info->records, record, &ActionRecordInfo::record); + bool const patch = effect.kind == "updates" && use == ReplyUse::Applied && own != info->records.end() && + own->keyField.has_value(); + if (patch) { + for (std::string const& path : own->paths) { + std::vector const members = detail::pathMembers(path); + detail::keyedRowsAt(reply, members, *own->keyField, found); + } + } + detail::RowsByKey rows; + for (Value const* const row : found) { + rows.insert_or_assign(detail::canonicalJson(*row->field(*own->keyField)), row); + } + for (auto const& [key, entry] : _entries) { + static_cast(key); + ActionRecordInfo const* const held = entry->holding(record); + if (held == nullptr) { + continue; + } + if (!patch || !held->keyField) { + mark(*entry); + continue; + } + Value confirmed = entry->confirmed; + bool changed = false; + for (std::string const& path : held->paths) { + std::vector const members = detail::pathMembers(path); + if (auto patched = detail::patchKeyedRows(confirmed, members, *held->keyField, rows)) { + confirmed = std::move(*patched); + changed = true; + } + } + if (changed) { + confirm(*entry, confirmed); + } + // A fetch in flight may have read before the write, and would settle over the patch. + if (entry->pending.peek()) { + markStale(*entry); + } + } + } + for (detail::CacheEntry* const entry : marked) { + refetchMounted(*entry); + } +} + +inline void Session::actionSucceeded(std::string const& typeId, std::string const& actionId, Value const& reply, + ReplyUse use, std::string const& instanceToken) { + _runtime->batch([&] { applyReplyEffects(typeId, actionId, reply, use, instanceToken); }); + if (use == ReplyUse::Discarded) { + return; + } std::vector const mounted = _mounted; for (detail::QueryRuntime* const query : mounted) { auto const& names = query->decl().refreshOn; @@ -3136,7 +3371,10 @@ inline void MutationRuntime::settle(std::shared_ptr const& run, std::option } } delivered = withoutFields(*reply, secret); + std::string const token = + _alias == nullptr ? std::string{} : core.runtime().untracked([&] { return _alias->token->get(); }); if (!applied) { + session.actionSucceeded(_typeId, _decl->action, *reply, ReplyUse::Discarded, token); return; } _result.set(*reply); @@ -3169,7 +3407,7 @@ inline void MutationRuntime::settle(std::shared_ptr const& run, std::option target->query->invalidate(); } } - session.actionSucceeded(_typeId, _decl->action); + session.actionSucceeded(_typeId, _decl->action, *reply, ReplyUse::Applied, token); core.runList(_decl->onSuccess, commandCtx(*run->bindings), {}); if (!secret.empty()) { _result.set(delivered); @@ -5401,11 +5639,15 @@ inline forms::Submitter ScreenCore::aliasSubmits(ModelAlias* alias, std::string std::string_view actionId, std::string body) -> async::Completion { async::Completion completion = call(actionId, std::move(body)); if (actionId != read) { - completion.then([this, alive = std::weak_ptr{_alive}, typeId = alias->decl->type, - action = std::string{actionId}](std::string const&) { - if (!alive.expired()) { - session.actionSucceeded(typeId, action); + completion.then([this, alive = std::weak_ptr{_alive}, alias, typeId = alias->decl->type, + action = std::string{actionId}](std::string const& reply) { + if (alive.expired()) { + return; } + auto json = parseJson(reply); + auto value = json ? valueFromJson(*json) : std::unexpected(json.error()); + std::string const token = runtime().untracked([&] { return alias->token->get(); }); + session.actionSucceeded(typeId, action, value ? *value : Value{}, ReplyUse::Applied, token); }); } return completion; diff --git a/tests/test_push_cache.cpp b/tests/test_push_cache.cpp index b133d45b3..11de6662a 100644 --- a/tests/test_push_cache.cpp +++ b/tests/test_push_cache.cpp @@ -1,9 +1,13 @@ // SPDX-License-Identifier: Apache-2.0 // The client cache under server push (spec 8 §9): a change marked while an entry's fetch is in flight is kept until -// the next fetch. +// the next fetch, and every reply applies its action's effects. The test catalog's `T_Row` is keyed by `code`. +#include #include +#include +#include +#include #include "ui_interpreter_support.hpp" @@ -52,3 +56,157 @@ TEST_CASE("stale_mark_on_unmounted_fetching_entry_waits_for_a_mount", "[push][ca auto again = h.mount(kList); CHECK(h.dispatcher.count("Lims_ListResults") == 2); } + +namespace { + +// `rows` lists `T_Row`; `edit` updates one, `rename` changes them all. Each text is `code:name`. +constexpr char const* kRows = R"({ + "vocab": "ui/1", "id": "rows", + "models": { "sample": { "type": "Lims_SampleModel" } }, + "queries": { "rows": { "model": "sample", "action": "Push_ListRows" } }, + "mutations": { + "edit": { "model": "sample", "action": "Push_EditRow", "concurrency": "latest" }, + "rename": { "model": "sample", "action": "Push_RenameAll" } }, + "view": { "kind": "column", "children": [ + { "kind": "textInput", "testId": "edit", "onSubmit": { "run": "edit", "body": { "obj": { "code": "b", "name": { "ref": "event" } } } } }, + { "kind": "button", "testId": "rename", "label": "rename", "onClick": { "run": "rename" } }, + { "kind": "forEach", "rows": { "query": "rows" }, "key": { "row": "code" }, + "children": [ { "kind": "text", "text": { "fn": "format", "args": [ "{0}:{1}", { "row": "code" }, { "row": "name" } ] } } ] } ] } })"; + +constexpr char const* kTwoRows = R"([{"code":"a","name":"x"},{"code":"b","name":"y"}])"; + +std::vector rowTexts(Harness& h) { + std::vector texts; + for (int const id : h.backend.all("Text")) { + texts.push_back(h.backend.prop(id, "text")); + } + return texts; +} + +void edit(Harness& h, std::string const& name) { + h.backend.submit(h.byTestId("TextInput", "edit"), name); + h.settle(); +} + +} // namespace + +TEST_CASE("own_updates_reply_patches_by_key_field_without_fetch", "[push][cache]") { + Harness h; + auto screen = h.mount(kRows); + h.reply(0, kTwoRows); + edit(h, "z"); + h.reply(h.dispatcher.last("Push_EditRow"), R"({"code":"b","name":"z"})"); + CHECK(rowTexts(h) == std::vector{"a:x", "b:z"}); + CHECK(h.dispatcher.count("Push_ListRows") == 1); +} + +TEST_CASE("reply_patch_at_nested_path", "[push][cache]") { + Harness h; + auto screen = h.mount(R"({ + "vocab": "ui/1", "id": "header", + "models": { "sample": { "type": "Lims_SampleModel" } }, + "queries": { "head": { "model": "sample", "action": "Push_Header" } }, + "mutations": { "edit": { "model": "sample", "action": "Push_EditRow" } }, + "let": { "doc": { "query": "head" } }, + "view": { "kind": "column", "children": [ + { "kind": "textInput", "testId": "edit", "onSubmit": { "run": "edit", "body": { "obj": { "code": "b", "name": { "ref": "event" } } } } }, + { "kind": "forEach", "rows": { "ref": "doc.header.rows" }, "key": { "row": "code" }, + "children": [ { "kind": "text", "text": { "fn": "format", "args": [ "{0}:{1}", { "row": "code" }, { "row": "name" } ] } } ] } ] } })"); + h.reply(0, R"({"header":{"title":"t","rows":[{"code":"a","name":"x"},{"code":"b","name":"y"}]}})"); + REQUIRE(rowTexts(h) == std::vector{"a:x", "b:y"}); + edit(h, "z"); + h.reply(h.dispatcher.last("Push_EditRow"), R"({"code":"b","name":"z"})"); + CHECK(rowTexts(h) == std::vector{"a:x", "b:z"}); + CHECK(h.dispatcher.count("Push_Header") == 1); +} + +TEST_CASE("changes_reply_refetches_holding_entries", "[push][cache]") { + Harness h; + auto screen = h.mount(kRows); + h.reply(0, kTwoRows); + h.backend.click(h.byTestId("Button", "rename")); + h.settle(); + h.reply(h.dispatcher.last("Push_RenameAll"), "null"); + REQUIRE(h.dispatcher.count("Push_ListRows") == 2); + h.reply(h.dispatcher.last("Push_ListRows"), R"([{"code":"a","name":"X"},{"code":"b","name":"Y"}])"); + CHECK(rowTexts(h) == std::vector{"a:X", "b:Y"}); + CHECK(h.dispatcher.count("Push_ListRows") == 2); +} + +TEST_CASE("submit_applies_effects", "[push][cache]") { + Harness h; + auto screen = h.mount(R"({ + "vocab": "ui/1", "id": "submit", + "models": { "sample": { "type": "Lims_SampleModel" } }, + "queries": { "rows": { "model": "sample", "action": "Push_ListRows" } }, + "view": { "kind": "column", "children": [ + { "kind": "form", "model": "sample", "action": "Lims_CaptureConcentration", "submit": "explicit" }, + { "kind": "forEach", "rows": { "query": "rows" }, "key": { "row": "code" }, + "children": [ { "kind": "text", "text": { "fn": "format", "args": [ "{0}:{1}", { "row": "code" }, { "row": "name" } ] } } ] } ] } })"); + h.reply(h.dispatcher.last("Push_ListRows"), kTwoRows); + h.backend.edit(h.byTestId("TextInput", "field_sampleId"), "4"); + h.backend.edit(h.byTestId("TextInput", "field_concentration"), "1"); + h.settle(); + h.backend.click(h.byTestId("Button", "form_submit")); + h.settle(); + h.reply(h.dispatcher.last("Lims_CaptureConcentration"), R"({"code":"a","name":"captured"})"); + std::vector const texts = rowTexts(h); + CHECK(std::ranges::find(texts, "a:captured") != texts.end()); + CHECK(h.dispatcher.count("Push_ListRows") == 1); +} + +TEST_CASE("discarded_latest_reply_marks_only", "[push][cache]") { + Harness h; + auto screen = h.mount(kRows); + h.reply(0, kTwoRows); + edit(h, "old"); + edit(h, "new"); + std::size_t const newer = h.dispatcher.last("Push_EditRow"); + h.reply(newer, R"({"code":"b","name":"new"})"); + REQUIRE(rowTexts(h) == std::vector{"a:x", "b:new"}); + REQUIRE(h.dispatcher.count("Push_ListRows") == 1); + + // The older run's reply carries older values: it must not patch, only mark. + h.reply(newer - 1, R"({"code":"b","name":"old"})"); + CHECK(rowTexts(h) == std::vector{"a:x", "b:new"}); + CHECK(h.dispatcher.count("Push_ListRows") == 2); +} + +TEST_CASE("patch_on_fetching_entry_refetches_once", "[push][cache]") { + Harness h; + auto screen = h.mount(kRows); + REQUIRE(h.dispatcher.count("Push_ListRows") == 1); + // The edit commits while the list's first fetch is in flight, which may have read before it. + edit(h, "z"); + h.reply(h.dispatcher.last("Push_EditRow"), R"({"code":"b","name":"z"})"); + REQUIRE(h.dispatcher.count("Push_ListRows") == 1); + h.reply(0, kTwoRows); + REQUIRE(h.dispatcher.count("Push_ListRows") == 2); + h.reply(h.dispatcher.last("Push_ListRows"), R"([{"code":"a","name":"x"},{"code":"b","name":"z"}])"); + CHECK(rowTexts(h) == std::vector{"a:x", "b:z"}); + CHECK(h.dispatcher.count("Push_ListRows") == 2); +} + +TEST_CASE("changes_instance_refetches_only_that_instance", "[push][cache]") { + Harness h; + auto screen = h.mount(R"({ + "vocab": "ui/1", "id": "instances", + "models": { "first": { "type": "Lims_SampleModel", "instance": "private" }, + "second": { "type": "Lims_SampleModel", "instance": "private" } }, + "queries": { "a": { "model": "first", "action": "Push_Rows" }, + "b": { "model": "second", "action": "Push_Rows" } }, + "mutations": { "reset": { "model": "first", "action": "Push_Reset" } }, + "view": { "kind": "column", "children": [ + { "kind": "button", "testId": "reset", "label": "reset", "onClick": { "run": "reset" } }, + { "kind": "text", "text": { "fn": "format", "args": [ "{0} {1}", { "query": "a" }, { "query": "b" } ] } } ] } })"); + REQUIRE(h.dispatcher.count("Push_Rows") == 2); + h.reply(0, "[]"); + h.reply(1, "[]"); + h.backend.click(h.byTestId("Button", "reset")); + h.settle(); + std::size_t const reset = h.dispatcher.last("Push_Reset"); + std::size_t const resetChannel = h.dispatcher.calls.at(reset).channel; + h.reply(reset, "null"); + REQUIRE(h.dispatcher.count("Push_Rows") == 3); + CHECK(h.dispatcher.calls.at(h.dispatcher.last("Push_Rows")).channel == resetChannel); +} diff --git a/tests/ui_interpreter_support.hpp b/tests/ui_interpreter_support.hpp index 2bd68fad3..efade8eeb 100644 --- a/tests/ui_interpreter_support.hpp +++ b/tests/ui_interpreter_support.hpp @@ -312,6 +312,24 @@ inline Catalog corpusCatalog() { if (action == "Lims_ListResults") { info.calculatedFields = {"concentration"}; } + // Server push (spec 8): `T_Row` is keyed by `code`, not `id`, so a patch must use the catalog's key field. + ActionRecordInfo const rowsAtTop{.record = "T_Row", .keyField = "code", .paths = {""}}; + if (action == "Push_ListRows" || action == "Push_Rows") { + info.records = {rowsAtTop}; + } + if (action == "Push_Header") { + info.records = {ActionRecordInfo{.record = "T_Row", .keyField = "code", .paths = {"header.rows"}}}; + } + if (action == "Push_EditRow" || action == "Lims_CaptureConcentration") { + info.effects = std::vector{ActionEffect{.kind = "updates", .record = "T_Row"}}; + info.records = {rowsAtTop}; + } + if (action == "Push_RenameAll") { + info.effects = std::vector{ActionEffect{.kind = "changes", .record = "T_Row"}}; + } + if (action == "Push_Reset") { + info.effects = std::vector{ActionEffect{.kind = "changesInstance", .record = std::nullopt}}; + } return info; }; // The actions a `form` node, a collection's editor and a wizard's steps build from. From fdebb7f5b8f97ca5333be0dc3cc848eb28ff354f Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 23:14:04 +0200 Subject: [PATCH 41/68] ui: the cache applies change events and resync Session::deliverEvent takes a server event: a changes payload marks the entries holding a changed record (filtered by the event's keys, at the record's paths) or read through a changed instance, and a resync marks every entry but an evaluate one. A marked entry refetches through a mounted query; one no query uses waits for a mount. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- docs/spec/ui/interpreter.md | 13 ++++ include/morph/ui/interpreter.hpp | 111 +++++++++++++++++++++++++++++++ tests/test_push_cache.cpp | 89 +++++++++++++++++++++++++ 3 files changed, 213 insertions(+) diff --git a/docs/spec/ui/interpreter.md b/docs/spec/ui/interpreter.md index 873704c0b..4c542c6d9 100644 --- a/docs/spec/ui/interpreter.md +++ b/docs/spec/ui/interpreter.md @@ -135,6 +135,19 @@ canonical body is the body's JSON with record fields in name order. - A `latest` run whose reply is discarded applies its effects as marks only: its `updates` act as `changes`, because its values are older than the reply already applied. It runs no `refreshOn`. - One reply's effects are applied in one reactive batch. +- **Server events** reach the cache through `Session::deliverEvent`, which the source wires to its + backend's `setEventHandler`. Each event is applied in one reactive batch, and other clients' changes + only mark, so each receiver refetches through its own authorised read. + - A `changes` event's record change with keys marks each entry holding the record whose value has + a row with one of the keys (by the key field, at the record's paths). An entry still waiting + for its first value is marked too, because it may come to hold the key. A record change without + keys, or on a keyless record, marks every entry holding the record. + - An instance change marks every entry read through that instance: for a primary key, the keyed + shared alias with that key (`shared:`); without one, the connection's private instances of + that model type. + - A `resync` marks every entry but an `evaluate` one. Another topic, or a payload that cannot be + read, is ignored. + - A marked entry that no query uses is only marked, and fetches when a query mounts on it. - **Invalidation** (`invalidates`, `refreshOn`, `refetch`, a backend switch) marks the entry stale and refetches it. - **A mark during a fetch is kept.** `Session::markStale` marks an entry stale. While the entry's fetch is diff --git a/include/morph/ui/interpreter.hpp b/include/morph/ui/interpreter.hpp index 756580a1e..30dc3fc9a 100644 --- a/include/morph/ui/interpreter.hpp +++ b/include/morph/ui/interpreter.hpp @@ -27,6 +27,7 @@ #include "../core/backend.hpp" #include "../core/bridge.hpp" #include "../core/completion.hpp" +#include "../core/wire.hpp" #include "../forms/engine/collection.hpp" #include "../forms/engine/collection_view.hpp" #include "../forms/engine/field_model.hpp" @@ -1131,6 +1132,23 @@ class Session { void actionSucceeded(std::string const& typeId, std::string const& actionId, Value const& reply, ReplyUse use, std::string const& instanceToken); + /// @brief Applies a server event to the cache (spec 8 §9). + /// + /// A `changes` event marks the entries it names stale, and a `resync` marks every entry but an `evaluate` one; + /// each marked entry refetches through a mounted query that uses it, and one no query uses refetches when a + /// query mounts on it. An event of another topic, or a payload that cannot be read, is ignored. + /// @param event The event. + void deliverEvent(wire::EventBody const& event); + + /// @brief Applies a `changes` event's payload to the cache, in one reactive batch. + /// + /// A record change with keys marks each entry holding the record whose value has a row with one of the keys, + /// and an entry holding it keyless; a record change without keys marks every entry holding the record. An + /// instance change marks every entry read through that instance: the keyed shared one its primary key names, + /// or, with no primary key, the connection's own private ones of that model type. + /// @param changes The payload. + void applyChanges(wire::ChangesBody const& changes); + /// @brief Marks an entry stale. While its fetch is in flight, the entry also refetches once that fetch settles, /// because the fetch may have read before the change. /// @param entry The entry. @@ -2783,6 +2801,99 @@ inline void Session::applyReplyEffects(std::string const& typeId, std::string co } } +inline void Session::deliverEvent(wire::EventBody const& event) { + if (event.topic == wire::kResyncTopic) { + _runtime->batch([this] { + std::vector marked; + for (auto const& [key, entry] : _entries) { + static_cast(key); + if (!entry->evaluate) { + markStale(*entry); + marked.push_back(entry.get()); + } + } + for (detail::CacheEntry* const entry : marked) { + refetchMounted(*entry); + } + }); + return; + } + if (event.topic != wire::kChangesTopic) { + return; + } + if (auto changes = wire::readChangesBody(event.payload.str)) { + applyChanges(*changes); + } +} + +inline void Session::applyChanges(wire::ChangesBody const& changes) { + // A key's canonical text, so a key the server wrote is compared with the rows' own. + auto const keyText = [](glz::raw_json const& key) -> std::optional { + auto json = parseJson(key.str); + auto value = json ? valueFromJson(*json) : std::unexpected(json.error()); + return value ? std::optional{detail::canonicalJson(*value)} : std::nullopt; + }; + _runtime->batch([&] { + std::vector marked; + auto const mark = [&](detail::CacheEntry& entry) { + if (std::ranges::find(marked, &entry) == marked.end()) { + markStale(entry); + marked.push_back(&entry); + } + }; + for (effects::RecordChange const& change : changes.records) { + std::set> keys; + if (change.keys) { + for (glz::raw_json const& key : *change.keys) { + if (auto text = keyText(key)) { + keys.insert(std::move(*text)); + } + } + } + for (auto const& [name, entry] : _entries) { + static_cast(name); + ActionRecordInfo const* const held = entry->holding(change.record); + if (held == nullptr) { + continue; + } + if (!change.keys || !held->keyField) { + mark(*entry); + continue; + } + std::vector rows; + for (std::string const& path : held->paths) { + std::vector const members = detail::pathMembers(path); + detail::keyedRowsAt(entry->confirmed, members, *held->keyField, rows); + } + bool const holdsKey = std::ranges::any_of(rows, [&](Value const* row) { + return keys.contains(detail::canonicalJson(*row->field(*held->keyField))); + }); + // An entry still waiting for its first value may come to hold the key. + if (holdsKey || !entry->hasValue.peek()) { + mark(*entry); + } + } + } + for (wire::InstanceTopic const& instance : changes.instances) { + std::string const shared = "shared:" + instance.primary; + for (auto const& [name, entry] : _entries) { + static_cast(name); + if (entry->evaluate || entry->typeId != instance.modelType) { + continue; + } + bool const same = instance.primary.empty() ? !entry->instanceToken.starts_with("shared") + : entry->instanceToken == shared; + if (same) { + mark(*entry); + } + } + } + for (detail::CacheEntry* const entry : marked) { + refetchMounted(*entry); + } + }); +} + inline void Session::actionSucceeded(std::string const& typeId, std::string const& actionId, Value const& reply, ReplyUse use, std::string const& instanceToken) { _runtime->batch([&] { applyReplyEffects(typeId, actionId, reply, use, instanceToken); }); diff --git a/tests/test_push_cache.cpp b/tests/test_push_cache.cpp index 11de6662a..5bfebac7d 100644 --- a/tests/test_push_cache.cpp +++ b/tests/test_push_cache.cpp @@ -210,3 +210,92 @@ TEST_CASE("changes_instance_refetches_only_that_instance", "[push][cache]") { REQUIRE(h.dispatcher.count("Push_Rows") == 3); CHECK(h.dispatcher.calls.at(h.dispatcher.last("Push_Rows")).channel == resetChannel); } + +namespace { + +// A local backend's event seam, wired to the session as a remote source wires its transport. +struct EventSeam { + explicit EventSeam(Harness& h) : backend{h.owner} { + backend.setEventHandler([&h](morph::wire::EventBody const& event) { h.session.deliverEvent(event); }, + &h.owner); + } + void changes(Harness& h, std::string payload) { + backend.deliverEvent({.topic = std::string{morph::wire::kChangesTopic}, .payload = {std::move(payload)}}); + h.settle(); + } + morph::backend::LocalBackend backend; +}; + +} // namespace + +TEST_CASE("changes_event_via_local_seam_refetches", "[push][cache]") { + Harness h; + EventSeam seam{h}; + auto screen = h.mount(kRows); + h.reply(0, kTwoRows); + seam.changes(h, R"({"records":[{"record":"T_Row","keys":["b"]}]})"); + REQUIRE(h.dispatcher.count("Push_ListRows") == 2); + h.reply(h.dispatcher.last("Push_ListRows"), R"([{"code":"a","name":"x"},{"code":"b","name":"other"}])"); + CHECK(rowTexts(h) == std::vector{"a:x", "b:other"}); +} + +TEST_CASE("event_keys_filter_entries_without_them", "[push][cache]") { + Harness h; + EventSeam seam{h}; + auto screen = h.mount(kRows); + h.reply(0, kTwoRows); + seam.changes(h, R"({"records":[{"record":"T_Row","keys":["zzz", 7]}]})"); + CHECK(h.dispatcher.count("Push_ListRows") == 1); + seam.changes(h, R"({"records":[{"record":"T_Other"}]})"); + CHECK(h.dispatcher.count("Push_ListRows") == 1); + // Without keys the whole type changed, so every entry holding it refetches. + seam.changes(h, R"({"records":[{"record":"T_Row"}]})"); + CHECK(h.dispatcher.count("Push_ListRows") == 2); +} + +TEST_CASE("resync_marks_all_watched", "[push][cache]") { + Harness h; + EventSeam seam{h}; + auto screen = h.mount(R"({ + "vocab": "ui/1", "id": "two", + "models": { "sample": { "type": "Lims_SampleModel" } }, + "queries": { "rows": { "model": "sample", "action": "Push_ListRows" }, + "results": { "model": "sample", "action": "Lims_ListResults" } }, + "view": { "kind": "text", "text": { "fn": "format", "args": [ "{0} {1}", { "query": "rows" }, { "query": "results" } ] } } })"); + h.reply(h.dispatcher.last("Push_ListRows"), "[]"); + h.reply(h.dispatcher.last("Lims_ListResults"), "[]"); + seam.backend.deliverEvent({.topic = std::string{morph::wire::kResyncTopic}, .payload = {"null"}}); + h.settle(); + CHECK(h.dispatcher.count("Push_ListRows") == 2); + CHECK(h.dispatcher.count("Lims_ListResults") == 2); +} + +TEST_CASE("event_on_unmounted_entry_marks_only", "[push][cache]") { + Harness h; + EventSeam seam{h}; + auto screen = h.mount(kRows); + REQUIRE(h.dispatcher.count("Push_ListRows") == 1); + screen.reset(); + h.settle(); + // The entry is kept, unused, with its fetch in flight when the change arrives. + seam.changes(h, R"({"records":[{"record":"T_Row"}]})"); + h.reply(0, kTwoRows); + CHECK(h.dispatcher.count("Push_ListRows") == 1); + auto again = h.mount(kRows); + CHECK(h.dispatcher.count("Push_ListRows") == 2); +} + +TEST_CASE("instance_event_refetches_reads_of_that_instance", "[push][cache]") { + Harness h; + EventSeam seam{h}; + auto screen = h.mount(R"({ + "vocab": "ui/1", "id": "instance", + "models": { "sample": { "type": "Lims_SampleModel", "instance": "private" } }, + "queries": { "info": { "model": "sample", "action": "Push_Rows" } }, + "view": { "kind": "text", "text": { "fn": "format", "args": [ "{0}", { "query": "info" } ] } } })"); + h.reply(0, "[]"); + seam.changes(h, R"({"instances":[{"modelType":"Lims_ClientModel","primary":""}]})"); + CHECK(h.dispatcher.count("Push_Rows") == 1); + seam.changes(h, R"({"instances":[{"modelType":"Lims_SampleModel","primary":""}]})"); + CHECK(h.dispatcher.count("Push_Rows") == 2); +} From ffc6a0ef55e750644f54855029527619a8bc1bd2 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 23:15:59 +0200 Subject: [PATCH 42/68] core, tests: build on libstdc++ and MSVC, install every detail header, pass format and tidy The result walker's recursive visit(walk, std::string) found std::visit by argument-dependent lookup through the string argument; libstdc++ rejects that, so every Linux build of the examples failed. The call is qualified. The anonymous-namespace check also matches MSVC's spelling, and the effects test helper no longer has code after a throwing FAIL, which MSVC flags as unreachable. watch_table.hpp and change_coalescer.hpp are installed, since remote.hpp includes them. The branch is formatted with clang-format 22 and its clang-tidy-diff findings are fixed. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- CMakeLists.txt | 2 + include/morph/core/backend.hpp | 11 +- include/morph/core/changes.hpp | 119 +++++++++------ .../morph/core/detail/change_coalescer.hpp | 2 +- include/morph/core/detail/result_walker.hpp | 6 +- include/morph/core/detail/watch_table.hpp | 3 +- include/morph/core/effects.hpp | 14 +- include/morph/core/records.hpp | 18 ++- include/morph/core/registry.hpp | 5 +- include/morph/core/remote.hpp | 77 ++++++---- include/morph/core/wire.hpp | 12 +- include/morph/net/socket_server.hpp | 4 +- include/morph/qt/qt_websocket_backend.hpp | 2 +- include/morph/ui/author.hpp | 27 ++-- src/qt/qt_websocket_server.cpp | 10 +- tests/compile_checks/effects_unseen_a.cpp | 5 +- tests/net/test_push_net.cpp | 13 +- tests/push_conformance.hpp | 6 +- tests/qt/test_qt_websocket.cpp | 71 ++++----- tests/test_push_catalog.cpp | 28 ++-- tests/test_push_channel.cpp | 24 ++-- tests/test_push_effects.cpp | 135 +++++++++--------- tests/test_push_records.cpp | 10 +- tests/test_push_watch.cpp | 33 +++-- 24 files changed, 362 insertions(+), 275 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index 46f24ddf0..db53d3f96 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -607,6 +607,8 @@ target_sources(morph include/morph/core/detail/reply_router.hpp include/morph/core/detail/subscription_registry.hpp include/morph/core/detail/result_walker.hpp + include/morph/core/detail/watch_table.hpp + include/morph/core/detail/change_coalescer.hpp include/morph/forms/detail/session_common.hpp include/morph/forms/detail/schema_name.hpp include/morph/forms/engine/detail/text.hpp diff --git a/include/morph/core/backend.hpp b/include/morph/core/backend.hpp index 355ea0920..c82116889 100644 --- a/include/morph/core/backend.hpp +++ b/include/morph/core/backend.hpp @@ -1126,9 +1126,9 @@ class LocalBackend : public detail::IBackend { /// @param exec Executor the handler is posted to. Borrowed: it must outlive the installation. void setEventHandler(std::function handler, ::morph::exec::IExecutor* exec) override { - std::lock_guard lock{_eventMutex}; - _eventHandler = handler ? std::make_shared>(std::move(handler)) - : nullptr; + std::scoped_lock const lock{_eventMutex}; + _eventHandler = + handler ? std::make_shared>(std::move(handler)) : nullptr; _eventExec = exec; } @@ -1143,14 +1143,15 @@ class LocalBackend : public detail::IBackend { std::shared_ptr> handler; ::morph::exec::IExecutor* exec = nullptr; { - std::lock_guard lock{_eventMutex}; + std::scoped_lock const lock{_eventMutex}; handler = _eventHandler; exec = _eventExec; } if (!handler || exec == nullptr) { return; } - exec->post([handler = std::move(handler), event = std::move(event)]() mutable { (*handler)(std::move(event)); }); + exec->post( + [handler = std::move(handler), event = std::move(event)]() mutable { (*handler)(std::move(event)); }); } LocalBackend(const LocalBackend&) = delete; diff --git a/include/morph/core/changes.hpp b/include/morph/core/changes.hpp index 4d11f24c2..d5e514f05 100644 --- a/include/morph/core/changes.hpp +++ b/include/morph/core/changes.hpp @@ -32,7 +32,7 @@ inline void collectKey(ResultJson const& value, std::string_view keyField, std:: return; } auto const& members = value.get_object(); - auto const key = members.find(keyField); + auto const* const key = members.find(keyField); if (key == members.end()) { return; } @@ -47,6 +47,7 @@ inline void collectKey(ResultJson const& value, std::string_view keyField, std:: /// @param value The value at the end of a path. /// @param keyField The record's key member. /// @param out Where to append. +// NOLINTNEXTLINE(misc-no-recursion): an array nests records to any depth, and the walk follows it. inline void collectKeysAtLeaf(ResultJson const& value, std::string_view keyField, std::vector& out) { if (value.is_array()) { for (auto const& element : value.get_array()) { @@ -72,6 +73,7 @@ inline void collectKeysAtLeaf(ResultJson const& value, std::string_view keyField /// @param path The dot-joined member names; `""` is the value itself. /// @param keyField The record's key member. /// @param out Where to append. +// NOLINTNEXTLINE(misc-no-recursion): one call per path segment or nested array, bounded by the result's depth. inline void collectKeysAtPath(ResultJson const& value, std::string_view path, std::string_view keyField, std::vector& out) { if (path.empty()) { @@ -91,7 +93,7 @@ inline void collectKeysAtPath(ResultJson const& value, std::string_view path, st auto const segment = path.substr(0, dot); auto const rest = dot == std::string_view::npos ? std::string_view{} : path.substr(dot + 1); auto const& members = value.get_object(); - if (auto const next = members.find(segment); next != members.end()) { + if (auto const* const next = members.find(segment); next != members.end()) { collectKeysAtPath(next->second, rest, keyField, out); return; } @@ -140,8 +142,10 @@ inline void addDerivedChanges(Resolved const& resolved, std::string const& sourc std::vector reached{root->type}; // A record is visited once, which also ends a walk over records that share a source. for (std::size_t next = 0; next < reached.size(); ++next) { + // A copy, not a reference: the loop below appends to `reached`. + auto const current = reached.at(next); for (auto const& candidate : resolved.records) { - if (std::ranges::find(candidate.derivedFrom, reached[next]) == candidate.derivedFrom.end() || + if (std::ranges::find(candidate.derivedFrom, current) == candidate.derivedFrom.end() || std::ranges::find(reached, candidate.type) != reached.end()) { continue; } @@ -151,12 +155,53 @@ inline void addDerivedChanges(Resolved const& resolved, std::string const& sourc } } +/// @brief Adds the change an execute's instance makes, unless it is already there. +/// @param changes Where to add. +/// @param ranOn The instance the action ran on. +inline void addInstanceChange(Changes& changes, InstanceChange const& ranOn) { + auto const same = [&](InstanceChange const& have) { + return have.modelType == ranOn.modelType && have.instance == ranOn.instance && have.primary == ranOn.primary; + }; + if (std::ranges::none_of(changes.instances, same)) { + changes.instances.push_back(ranOn); + } +} + +/// @brief Adds the change an `updates` effect makes: the keys of the `R` rows found in the result. +/// +/// A result that does not parse, or a record without a key, cannot name rows: the whole type is reported, +/// which over-notifies but never misses. +/// @param resolved The resolved registries. +/// @param actionId The action's id. +/// @param record The updated record's wire name. +/// @param parsed The parsed result, or `nullptr` when it did not parse. +/// @param changes Where to add. +inline void addUpdatesChange(Resolved const& resolved, std::string_view actionId, std::string const& record, + ResultJson const* parsed, Changes& changes) { + auto const* const info = resolved.recordNamed(record); + if (parsed == nullptr || info == nullptr || !info->keyField) { + addRecordChange(changes, record, std::nullopt); + return; + } + std::vector keys; + for (auto const& held : resolved.recordsOf(actionId)) { + if (held.record != record) { + continue; + } + for (auto const& path : held.paths) { + collectKeysAtPath(*parsed, path, *info->keyField, keys); + } + } + addRecordChange(changes, record, std::move(keys)); +} + /// @brief See `morph::effects::deriveChanges`, over given registries. /// @param resolved The resolved registries. /// @param actionId The action's id. /// @param resultJson The action's result as JSON. /// @param ranOn The instance the action ran on. /// @return The changes. +// NOLINTNEXTLINE(bugprone-easily-swappable-parameters): an id, then a JSON document; swapped, no effects are found. [[nodiscard]] inline Changes deriveChangesFrom(Resolved const& resolved, std::string_view actionId, std::string_view resultJson, InstanceChange const& ranOn) { Changes changes; @@ -164,54 +209,36 @@ inline void addDerivedChanges(Resolved const& resolved, std::string const& sourc if (!declared) { return changes; } + // The result is parsed once, and only when an `updates` effect needs its rows. std::optional parsed; bool parseTried = false; + auto const parsedResult = [&]() -> ResultJson const* { + if (!parseTried) { + parseTried = true; + ResultJson tree; + if (!glz::read_json(tree, resultJson)) { + parsed = std::move(tree); + } + } + return parsed ? &*parsed : nullptr; + }; for (auto const& effect : *declared) { switch (effect.kind) { - case Kind::None: - break; - case Kind::ChangesInstance: - if (std::ranges::none_of(changes.instances, [&](InstanceChange const& have) { - return have.modelType == ranOn.modelType && have.instance == ranOn.instance && - have.primary == ranOn.primary; - })) { - changes.instances.push_back(ranOn); - } - break; - case Kind::Changes: - addRecordChange(changes, effect.record, std::nullopt); - addDerivedChanges(resolved, effect.record, changes); - break; - case Kind::Updates: { - if (!parseTried) { - parseTried = true; - ResultJson tree; - if (!glz::read_json(tree, resultJson)) { - parsed = std::move(tree); - } - } - auto const* const info = resolved.recordNamed(effect.record); - // A result that does not parse, or a record without a key, cannot name rows: the whole type - // is reported, which over-notifies but never misses. - if (!parsed || info == nullptr || !info->keyField) { + case Kind::None: + break; + case Kind::ChangesInstance: + addInstanceChange(changes, ranOn); + break; + case Kind::Changes: addRecordChange(changes, effect.record, std::nullopt); - } else { - std::vector keys; - for (auto const& held : resolved.recordsOf(actionId)) { - if (held.record != effect.record) { - continue; - } - for (auto const& path : held.paths) { - collectKeysAtPath(*parsed, path, *info->keyField, keys); - } - } - addRecordChange(changes, effect.record, std::move(keys)); - } - addDerivedChanges(resolved, effect.record, changes); - break; - } - default: - break; + addDerivedChanges(resolved, effect.record, changes); + break; + case Kind::Updates: + addUpdatesChange(resolved, actionId, effect.record, parsedResult(), changes); + addDerivedChanges(resolved, effect.record, changes); + break; + default: + break; } } // A record naming more rows than a reader accepts is reported whole: it over-notifies, never misses. diff --git a/include/morph/core/detail/change_coalescer.hpp b/include/morph/core/detail/change_coalescer.hpp index 21636168a..a40e976e5 100644 --- a/include/morph/core/detail/change_coalescer.hpp +++ b/include/morph/core/detail/change_coalescer.hpp @@ -117,7 +117,7 @@ class ChangeCoalescer { if (inserted) { _records.push_back(Entry{.record = change.record, .whole = false, .keys = {}, .seen = {}}); } - Entry& entry = _records[found->second]; + Entry& entry = _records.at(found->second); if (entry.whole) { return; } diff --git a/include/morph/core/detail/result_walker.hpp b/include/morph/core/detail/result_walker.hpp index c81062267..8a35b9fa2 100644 --- a/include/morph/core/detail/result_walker.hpp +++ b/include/morph/core/detail/result_walker.hpp @@ -89,7 +89,7 @@ template consteval bool hasAnonymousName() { constexpr std::string_view name = glz::type_name; return name.contains("(anonymous namespace)") || name.contains("{anonymous}") || - name.contains("`anonymous namespace'"); + name.contains("`anonymous namespace'") || name.contains("`anonymous-namespace'"); } /// @brief A type whose members the walker visits: one glaze writes as an object, member by member. @@ -135,7 +135,7 @@ void visitMembers(Walk& walk, std::string const& path, std::index_sequence using Member = std::remove_cvref_t(std::declval()))>; if constexpr (mayHoldSlots()) { constexpr std::string_view key = glz::reflect::keys[Index]; - visit(walk, path.empty() ? std::string{key} : path + "." + std::string{key}); + result_walker::visit(walk, path.empty() ? std::string{key} : path + "." + std::string{key}); } }; (member.template operator()(), ...); @@ -148,7 +148,7 @@ void visitMembers(Walk& walk, std::string const& path, std::index_sequence template void visit(Walk& walk, std::string const& path) { if constexpr (!std::is_void_v::type>) { - visit::type>>(walk, path); + result_walker::visit::type>>(walk, path); } else if constexpr (Object) { std::type_index const self{typeid(T)}; if (std::ranges::find(walk.onPath, self) != walk.onPath.end()) { diff --git a/include/morph/core/detail/watch_table.hpp b/include/morph/core/detail/watch_table.hpp index ccd7dd984..b57293345 100644 --- a/include/morph/core/detail/watch_table.hpp +++ b/include/morph/core/detail/watch_table.hpp @@ -188,7 +188,8 @@ class WatchTable { if (connection == _connections.end()) { return nullptr; } - auto const found = connection->second.instances.find(InstanceKey{std::string{modelType}, std::string{primary}}); + auto const found = + connection->second.instances.find(InstanceKey{std::string{modelType}, std::string{primary}}); return found == connection->second.instances.end() ? nullptr : found->second.get(); } diff --git a/include/morph/core/effects.hpp b/include/morph/core/effects.hpp index 2f4a82e10..36308bc82 100644 --- a/include/morph/core/effects.hpp +++ b/include/morph/core/effects.hpp @@ -348,8 +348,8 @@ inline void findDerivationCycles(std::vector const& records, /// @return The resolved registries. /// @throws morph::model::EffectsRegistrationError listing every problem, when there is one. [[nodiscard]] inline Resolved resolve(std::vector records, - model::detail::ActionResultSlotMap const& resultSlots, EffectsMap const& declared, - std::vector const& slotConflicts) { + model::detail::ActionResultSlotMap const& resultSlots, + EffectsMap const& declared, std::vector const& slotConflicts) { std::vector problems; // An identical registration, as a record registered in a header gives once per translation unit, is @@ -359,8 +359,9 @@ inline void findDerivationCycles(std::vector const& records, auto const sameType = std::ranges::find(distinct, info.type, &record::RecordInfo::type); if (sameType != distinct.end()) { if (sameType->name != info.name) { - problems.push_back(std::string{"type "} + info.type.name() + " is registered under two record names (`" + - sameType->name + "` and `" + info.name + "`)"); + problems.push_back(std::string{"type "} + info.type.name() + + " is registered under two record names (`" + sameType->name + "` and `" + + info.name + "`)"); } else if (!sameRegistration(*sameType, info)) { problems.push_back("record `" + info.name + "` is registered twice with different options"); } @@ -433,9 +434,8 @@ inline void findDerivationCycles(std::vector const& records, continue; } auto const& declaration = declarations.front(); - if (declaration.size() > 1 && std::ranges::any_of(declaration, [](DeclaredEffect const& effect) { - return effect.kind == Kind::None; - })) { + if (declaration.size() > 1 && + std::ranges::any_of(declaration, [](DeclaredEffect const& effect) { return effect.kind == Kind::None; })) { problems.push_back("action `" + actionId + "` declares none together with other effects"); continue; } diff --git a/include/morph/core/records.hpp b/include/morph/core/records.hpp index df98d7ac3..58087a978 100644 --- a/include/morph/core/records.hpp +++ b/include/morph/core/records.hpp @@ -12,6 +12,7 @@ #include #include #include + #include "../detail/reflected_member.hpp" #include "model_key.hpp" #include "registry.hpp" @@ -187,9 +188,7 @@ inline std::vector& mutableRecordRegistry() { /// @brief The records declared so far, raw and unvalidated. /// @return The entries in registration order. -inline std::vector const& recordRegistry() { - return mutableRecordRegistry(); -} +inline std::vector const& recordRegistry() { return mutableRecordRegistry(); } /// @brief Static-init helper for `BRIDGE_REGISTER_RECORD`. /// @tparam T The record type. @@ -209,9 +208,8 @@ inline bool registerRecordOnce(std::string_view name) noexcept { info.privateKeys = (std::is_same_v || ... || false); auto const addSources = [](RecordInfo& target) { if constexpr (IsDerivedFrom::value) { - [](RecordInfo& out, derivedFrom) { - (out.derivedFrom.emplace_back(typeid(S)), ...); - }(target, O{}); + [](RecordInfo& out, derivedFrom) { (out.derivedFrom.emplace_back(typeid(S)), ...); }( + target, O{}); } }; (addSources.template operator()(info), ...); @@ -234,9 +232,9 @@ inline bool registerRecordOnce(std::string_view name) noexcept { #ifdef MORPH_CLIENT_ONLY #define BRIDGE_REGISTER_RECORD(T, NAME, ...) #else -#define BRIDGE_REGISTER_RECORD(T, NAME, ...) \ - namespace { \ - [[maybe_unused]] const bool BRIDGE_DETAIL_CAT(bridge_record_reg_, __COUNTER__) = \ - morph::record::detail::registerRecordOnce(NAME); \ +#define BRIDGE_REGISTER_RECORD(T, NAME, ...) \ + namespace { \ + [[maybe_unused]] const bool BRIDGE_DETAIL_CAT(bridge_record_reg_, __COUNTER__) = \ + morph::record::detail::registerRecordOnce(NAME); \ } #endif diff --git a/include/morph/core/registry.hpp b/include/morph/core/registry.hpp index 6eeca87e5..0d20dbbac 100644 --- a/include/morph/core/registry.hpp +++ b/include/morph/core/registry.hpp @@ -1571,8 +1571,9 @@ inline bool registerActionResultOnce(std::string_view actionId) noexcept { } return true; } - entries.emplace(std::string{actionId}, - ActionResult{.action = action, .resultSlots = walkResult::Result>()}); + entries.emplace( + std::string{actionId}, + ActionResult{.action = action, .resultSlots = walkResult::Result>()}); return true; } diff --git a/include/morph/core/remote.hpp b/include/morph/core/remote.hpp index 5ddc5be51..0025ea6cb 100644 --- a/include/morph/core/remote.hpp +++ b/include/morph/core/remote.hpp @@ -794,23 +794,7 @@ class RemoteServer : public std::enable_shared_from_this { } } for (auto& change : changes.instances) { - ::morph::exec::detail::ModelId const mid{change.instance}; - if (change.primary.empty()) { - // Keyed after admission: the topic is the key it holds now. - if (auto const* inst = _instances.find(mid); inst != nullptr && inst->sharedKey) { - change.primary = inst->sharedKey->second; - } - } - auto const recipients = - change.primary.empty() ? scopesHolding(mid) : _watches.watchersOf(change.modelType, change.primary); - ::morph::wire::InstanceTopic const topic{.modelType = std::move(change.modelType), - .primary = std::move(change.primary), - .instance = change.instance}; - for (ConnectionId const cid : recipients) { - if (eventsNegotiated(cid)) { - bodies[cid].instances.push_back(topic); - } - } + addInstanceTopic(std::move(change), bodies); } Authentications authentications; for (auto& [cid, body] : bodies) { @@ -1104,6 +1088,34 @@ class RemoteServer : public std::enable_shared_from_this { return found->second; } + /// @brief Adds one changed instance's topic to the body of every connection + /// it reaches, for `fanOut`. On the server strand. + /// + /// An instance is matched by the directory key it holds now; a private one + /// reaches the scopes that hold it. + /// @param change The changed instance. + /// @param bodies Each recipient's share of the fan-out; extended in place. + void addInstanceTopic(::morph::effects::InstanceChange change, + std::map& bodies) { + ::morph::exec::detail::ModelId const mid{change.instance}; + if (change.primary.empty()) { + // Keyed after admission: the topic is the key it holds now. + if (auto const* inst = _instances.find(mid); inst != nullptr && inst->sharedKey) { + change.primary = inst->sharedKey->second; + } + } + auto const recipients = + change.primary.empty() ? scopesHolding(mid) : _watches.watchersOf(change.modelType, change.primary); + ::morph::wire::InstanceTopic const topic{.modelType = std::move(change.modelType), + .primary = std::move(change.primary), + .instance = change.instance}; + for (ConnectionId const cid : recipients) { + if (eventsNegotiated(cid)) { + bodies[cid].instances.push_back(topic); + } + } + } + /// @brief Narrows @p body to the topics connection @p cid may still be /// sent, re-authorising each watch by the rule that admitted it. /// On the server strand. @@ -1160,13 +1172,8 @@ class RemoteServer : public std::enable_shared_from_this { dropped = true; continue; } - bool const keepsKeys = cid == caller || (!callerPrincipal.empty() && watch->principal == callerPrincipal); - if (!keepsKeys) { - auto const* info = ::morph::effects::recordNamed(change.record); - if (info != nullptr && info->privateKeys) { - change.keys.reset(); - } - } + bool const writer = cid == caller || (!callerPrincipal.empty() && watch->principal == callerPrincipal); + hidePrivateKeys(change, writer); records.push_back(std::move(change)); } body.records = std::move(records); @@ -1192,6 +1199,21 @@ class RemoteServer : public std::enable_shared_from_this { return dropped; } + /// @brief Drops @p change's keys when its record is registered with + /// `morph::record::privateKeys` and the recipient is not the writer. + /// @param change The change for one recipient; narrowed in place. + /// @param writer True when the recipient is the execute's own connection, or + /// watches as the execute's verified principal. + static void hidePrivateKeys(::morph::effects::RecordChange& change, bool writer) { + if (writer) { + return; + } + auto const* info = ::morph::effects::recordNamed(change.record); + if (info != nullptr && info->privateKeys) { + change.keys.reset(); + } + } + /// @brief Sends @p cid one `resync` event, owed because the server dropped /// some of its watches on its own. On the server strand. /// @@ -1257,6 +1279,7 @@ class RemoteServer : public std::enable_shared_from_this { /// `resync`) and a later window is open, or when the connection closed. /// @param cid Connection whose window closed. /// @param window The number its `queueChanges` gave the window. + // NOLINTNEXTLINE(bugprone-easily-swappable-parameters): a connection, then its window's number; swapped, nothing matches and nothing is sent. void closeWindow(ConnectionId cid, std::uint64_t window) { noteOwner("RemoteServer::closeWindow"); auto found = _writers.find(cid); @@ -2832,7 +2855,7 @@ class SimulatedRemoteBackend : public detail::IBackend { return; } _server.setConnectionWriter( - _cid, ConnectionWriter{.write = [inbox = _inbox](std::string frame) { inbox->receive(frame); }, + _cid, ConnectionWriter{.write = [inbox = _inbox](std::string const& frame) { inbox->receive(frame); }, .backlog = [] { return std::size_t{0}; }}); } @@ -3142,7 +3165,7 @@ class SimulatedRemoteBackend : public detail::IBackend { /// @param next New handler; empty clears it. /// @param target Executor it is posted to. void set(std::function next, ::morph::exec::IExecutor* target) { - std::lock_guard lock{mutex}; + std::scoped_lock const lock{mutex}; handler = next ? std::make_shared>(std::move(next)) : nullptr; exec = target; @@ -3154,7 +3177,7 @@ class SimulatedRemoteBackend : public detail::IBackend { std::shared_ptr> current; ::morph::exec::IExecutor* target = nullptr; { - std::lock_guard lock{mutex}; + std::scoped_lock const lock{mutex}; current = handler; target = exec; } diff --git a/include/morph/core/wire.hpp b/include/morph/core/wire.hpp index d87766ae0..c0fc68467 100644 --- a/include/morph/core/wire.hpp +++ b/include/morph/core/wire.hpp @@ -5,8 +5,8 @@ #include #include #include -#include #include +#include #include #include #include @@ -638,7 +638,7 @@ struct EventBody { if (payloadJson.empty()) { payloadJson = "null"; } - EventBody body{std::string{topic}, glz::raw_json{std::move(payloadJson)}}; + EventBody body{.topic = std::string{topic}, .payload = glz::raw_json{std::move(payloadJson)}}; env.body = glz::write_json(body).value_or(std::string{}); return env; } @@ -712,12 +712,13 @@ template } T out{}; static constexpr glz::opts kLenient{.null_terminated = false, .error_on_unknown_keys = false}; - if (auto ec = glz::read(out, json)) { - return std::unexpected(std::string{what} + ": " + glz::format_error(ec, json)); + if (auto const error = glz::read(out, json)) { + return std::unexpected(std::string{what} + ": " + glz::format_error(error, json)); } return out; } +// NOLINTNEXTLINE(bugprone-easily-swappable-parameters): a name, then how a refusal describes it; swapped, only the message reads oddly. [[nodiscard]] inline std::optional checkTopicName(std::string_view name, std::string_view what) { if (name.size() > kMaxTopicNameBytes) { return std::string{what} + " is longer than " + std::to_string(kMaxTopicNameBytes) + " bytes"; @@ -816,7 +817,8 @@ template return std::unexpected(*reason); } if (change.keys && change.keys->size() > effects::kMaxChangeKeys) { - return std::unexpected("a record's keys list more than " + std::to_string(effects::kMaxChangeKeys) + " keys"); + return std::unexpected("a record's keys list more than " + std::to_string(effects::kMaxChangeKeys) + + " keys"); } } if (auto reason = detail::checkInstances(body->instances, "an instance topic")) { diff --git a/include/morph/net/socket_server.hpp b/include/morph/net/socket_server.hpp index 2c8ea2159..b9d97ecc9 100644 --- a/include/morph/net/socket_server.hpp +++ b/include/morph/net/socket_server.hpp @@ -304,10 +304,10 @@ class SocketServer { /// backlog read. Neither keeps the connection: the server may hold /// them until it learns the connection is gone. [[nodiscard]] ::morph::backend::ConnectionWriter writerFor(std::shared_ptr const& client) const { - std::weak_ptr weak = client; + std::weak_ptr const weak = client; auto const counter = client->conn->pending; return {.write = - [loopHandle = loop.weak(), weak, counter](std::string envelope) { + [loopHandle = loop.weak(), weak, counter](std::string const& envelope) { if (!sendEnvelope(loopHandle, weak, counter, envelope)) { throw std::runtime_error("the connection's I/O loop is gone"); } diff --git a/include/morph/qt/qt_websocket_backend.hpp b/include/morph/qt/qt_websocket_backend.hpp index bcfd7864d..4f17e3537 100644 --- a/include/morph/qt/qt_websocket_backend.hpp +++ b/include/morph/qt/qt_websocket_backend.hpp @@ -13,12 +13,12 @@ #include #include #include -#include #include #include #include #include #include +#include #include #include #include diff --git a/include/morph/ui/author.hpp b/include/morph/ui/author.hpp index b2c0755a4..4c2d16b89 100644 --- a/include/morph/ui/author.hpp +++ b/include/morph/ui/author.hpp @@ -134,9 +134,9 @@ template } else if (effect.kind == ::morph::effects::Kind::ChangesInstance) { kind = "changesInstance"; } - out.push_back(ActionEffect{.kind = std::string{kind}, - .record = effect.record.empty() ? std::nullopt - : std::optional{effect.record}}); + out.push_back( + ActionEffect{.kind = std::string{kind}, + .record = effect.record.empty() ? std::nullopt : std::optional{effect.record}}); } return out; } @@ -199,17 +199,16 @@ struct CatalogFacts { void action() { model(); auto const actionId = std::string{::morph::model::ActionTraits::typeId()}; - actions.emplace(actionId, - ActionInfo{ - .modelType = std::string{::morph::model::ModelTraits::typeId()}, - .calculatedFields = ActionFacts::calculatedFields(), - .secretFields = ActionFacts::secretFields(), - .credential = ActionFacts::credential, - .listFields = detail::listFields<::morph::model::HandlerResultT< - decltype(std::declval().execute(std::declval()))>>(), - .effects = detail::effectsInfo(actionId), - .records = detail::recordsInfo(actionId), - }); + actions.emplace(actionId, ActionInfo{ + .modelType = std::string{::morph::model::ModelTraits::typeId()}, + .calculatedFields = ActionFacts::calculatedFields(), + .secretFields = ActionFacts::secretFields(), + .credential = ActionFacts::credential, + .listFields = detail::listFields<::morph::model::HandlerResultT< + decltype(std::declval().execute(std::declval()))>>(), + .effects = detail::effectsInfo(actionId), + .records = detail::recordsInfo(actionId), + }); } /// @brief The facts as a `ui::Catalog`. diff --git a/src/qt/qt_websocket_server.cpp b/src/qt/qt_websocket_server.cpp index f9ee38346..59f8995b0 100644 --- a/src/qt/qt_websocket_server.cpp +++ b/src/qt/qt_websocket_server.cpp @@ -221,9 +221,10 @@ void QtWebSocketServer::onNewConnection() { if (_wsServer.secureMode() == QWebSocketServer::SecureMode) { state.tls = findTlsSocket(socket); if (state.tls.isNull()) { - ::morph::log::logWarn("[QtWebSocketServer] connection {}: TLS socket not found; its backlog counts " - "only plaintext not yet encrypted, so a slow reader is not held back", - state.cid); + ::morph::log::logWarn( + "[QtWebSocketServer] connection {}: TLS socket not found; its backlog counts " + "only plaintext not yet encrypted, so a slow reader is not held back", + state.cid); } } #endif @@ -255,8 +256,7 @@ void QtWebSocketServer::onNewConnection() { } bool QtWebSocketServer::queueEnvelope(QPointer const& server, QPointer const& socket, - std::shared_ptr> const& backlog, - std::string envelope) { + std::shared_ptr> const& backlog, std::string envelope) { auto* app = QCoreApplication::instance(); if (app == nullptr) { return false; diff --git a/tests/compile_checks/effects_unseen_a.cpp b/tests/compile_checks/effects_unseen_a.cpp index ebb57e28a..e37bf4365 100644 --- a/tests/compile_checks/effects_unseen_a.cpp +++ b/tests/compile_checks/effects_unseen_a.cpp @@ -5,13 +5,16 @@ // into one program and read the same effects. #include + #include "effects_unseen_shared.hpp" BRIDGE_REGISTER_RECORD(effects_unseen::Row, "Unseen_Row") BRIDGE_REGISTER_EFFECTS(effects_unseen::EditRows, morph::effects::updates) +// NOLINTNEXTLINE(bugprone-exception-escape): a probe; an uncaught failure ends the process, which the harness reads as a failure. int main() { - std::vector const expected{{morph::effects::Kind::Updates, "Unseen_Row"}}; + std::vector const expected{ + {.kind = morph::effects::Kind::Updates, .record = "Unseen_Row"}}; auto const seenHere = morph::effects::effectsOf("Unseen_EditRows"); auto const seenFromB = effects_unseen::effectsSeenFromB(); if (seenHere != expected || seenFromB != expected) { diff --git a/tests/net/test_push_net.cpp b/tests/net/test_push_net.cpp index 324ad663e..2a7650942 100644 --- a/tests/net/test_push_net.cpp +++ b/tests/net/test_push_net.cpp @@ -108,6 +108,7 @@ std::string wsUrl(std::uint16_t port) { return "ws://127.0.0.1:" + std::to_strin /// A raw WebSocket client against a real `SocketServer`. class RawClient { public: + // NOLINTNEXTLINE(bugprone-easily-swappable-parameters): a port, then an optional buffer size every caller names in a comment. explicit RawClient(std::uint16_t port, int receiveBuffer = 0) : _reader{/*expectMasked=*/false} { _socket = ::morph::net::detail::TcpSocket::connect("127.0.0.1", port, 2000ms); if (receiveBuffer > 0) { @@ -165,6 +166,7 @@ struct LoopConnectionGuard { } }; +// NOLINTNEXTLINE(bugprone-easily-swappable-parameters): the envelope's fields in wire order; swapped, the server refuses an unknown model. ::morph::wire::Envelope executeEnvelope(std::uint64_t callId, std::uint64_t modelId, std::string_view action, std::string body) { ::morph::wire::Envelope env; @@ -399,7 +401,7 @@ TEST_CASE("net_closing_a_connection_releases_what_it_still_held", "[push][channe conn->onDrained = [&drained] { drained.fetch_add(1); }; // Nothing reads the peer: most of 4 MiB stays queued. for (int i = 0; i < 64; ++i) { - std::string const frame(64 * 1024, 'q'); + std::string const frame(std::size_t{64} * 1024, 'q'); conn->pending->fetch_add(frame.size()); static_cast(::morph::net::detail::enqueueFrame(conn, frame, frame.size())); } @@ -489,7 +491,7 @@ TEST_CASE("net_event_posted_during_the_handshake_is_dropped", "[push][channel]") REQUIRE(socket.setRecvTimeout(5000ms)); std::string received; std::array buf{}; - while (received.find("\r\n\r\n") == std::string::npos) { + while (!received.contains("\r\n\r\n")) { std::size_t const got = socket.recvSome(buf.data(), buf.size()); REQUIRE(got != 0); received.append(buf.data(), got); @@ -560,10 +562,9 @@ TEST_CASE("net_channel_conformance", "[push][channel]") { ::morph::net::SocketBackend backend{wsUrl(wsServer.port())}; REQUIRE(backend.waitForConnected()); ::morph::testing::push::ChannelFixture fixture{ - .server = server.get(), - .cid = cid, - .backend = &backend, - .waitFor = [](const std::function& done) { return ::morph::testing::waitUntil(done); }}; + .server = server.get(), .cid = cid, .backend = &backend, .waitFor = [](const std::function& done) { + return ::morph::testing::waitUntil(done); + }}; body(fixture); }); } diff --git a/tests/push_conformance.hpp b/tests/push_conformance.hpp index e7d61e130..01d982211 100644 --- a/tests/push_conformance.hpp +++ b/tests/push_conformance.hpp @@ -38,6 +38,7 @@ struct PcfGateAction { }; /// Open while the model's action may finish; closed, the action holds, so a /// call stays pending for as long as a test needs it. +// NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables): a flag shared between the test thread and the running action. inline std::atomic pcfGateOpen{true}; struct PcfGateModel { int execute(PcfGateAction /*action*/) { @@ -107,6 +108,8 @@ struct GateClosed { ~GateClosed() { pcfGateOpen.store(true); } GateClosed(const GateClosed&) = delete; GateClosed& operator=(const GateClosed&) = delete; + GateClosed(GateClosed&&) = delete; + GateClosed& operator=(GateClosed&&) = delete; }; inline std::string eventJson(std::size_t n) { return "{\"n\":" + std::to_string(n) + "}"; } @@ -200,7 +203,8 @@ inline void channelConformance(const FixtureFactory& make) { constexpr std::size_t kEvents = 50; for (std::size_t n = 0; n < kEvents; ++n) { - fx.server->post(fx.cid, ::morph::wire::encode(::morph::wire::makeEvent("changes", detail::eventJson(n)))); + fx.server->post(fx.cid, + ::morph::wire::encode(::morph::wire::makeEvent("changes", detail::eventJson(n)))); } REQUIRE(fx.waitFor([&] { return log.size() == kEvents; })); auto const got = log.snapshot(); diff --git a/tests/qt/test_qt_websocket.cpp b/tests/qt/test_qt_websocket.cpp index 5eba8e7cb..ca3e16b49 100644 --- a/tests/qt/test_qt_websocket.cpp +++ b/tests/qt/test_qt_websocket.cpp @@ -3121,10 +3121,12 @@ TEST_CASE("qt_event_is_routed_before_reply_matching", "[push][channel]") { }, 200); REQUIRE(log.size() == 2); - pumpUntil([&] { - owner.drain(); - return false; - }, 5); + pumpUntil( + [&] { + owner.drain(); + return false; + }, + 5); CHECK(result.load() == -1); CHECK_FALSE(failed.load()); @@ -3185,16 +3187,16 @@ TEST_CASE("qt_stalled_peer_closed_after_send_timeout", "[push][channel]") { std::mutex logMtx; std::vector> logged; - morph::log::ScopedLoggerOverride guard{[&](morph::log::LogLevel /*level*/, std::string_view msg) { - std::scoped_lock const lock{logMtx}; - logged.emplace_back(std::chrono::steady_clock::now(), - std::string{msg}); - }, - morph::log::LogLevel::debug}; + morph::log::ScopedLoggerOverride const guard{[&](morph::log::LogLevel /*level*/, std::string_view msg) { + std::scoped_lock const lock{logMtx}; + logged.emplace_back(std::chrono::steady_clock::now(), + std::string{msg}); + }, + morph::log::LogLevel::debug}; auto findLogged = [&](std::string_view needle) -> std::optional { std::scoped_lock const lock{logMtx}; for (auto const& [when, msg] : logged) { - if (msg.find(needle) != std::string::npos) { + if (msg.contains(needle)) { return when; } } @@ -3313,6 +3315,7 @@ class RawWsPeer { void resume() { _socket->setReadBufferSize(0); } /// Pumps and reads until @p count whole text messages have arrived. + // NOLINTNEXTLINE(bugprone-easily-swappable-parameters): a message count, then an optional pump bound; swapped, the read ends early and the check after it fails. std::vector readMessages(std::size_t count, int maxIterations = 500) { pumpUntil( [&] { @@ -3331,16 +3334,18 @@ class RawWsPeer { if (_buffer.size() < 2) { return; } - auto const* data = reinterpret_cast(_buffer.constData()); - bool const fin = (data[0] & 0x80U) != 0; - unsigned const opcode = data[0] & 0x0FU; - std::uint64_t length = data[1] & 0x7FU; + auto const byteAt = [this](qsizetype index) { + return std::uint64_t{static_cast(_buffer.at(index))}; + }; + bool const fin = (byteAt(0) & 0x80U) != 0; + auto const opcode = byteAt(0) & 0x0FU; + std::uint64_t length = byteAt(1) & 0x7FU; qsizetype header = 2; if (length == 126) { if (_buffer.size() < 4) { return; } - length = (std::uint64_t{data[2]} << 8U) | data[3]; + length = (byteAt(2) << 8U) | byteAt(3); header = 4; } else if (length == 127) { if (_buffer.size() < 10) { @@ -3348,7 +3353,7 @@ class RawWsPeer { } length = 0; for (int i = 0; i < 8; ++i) { - length = (length << 8U) | data[2 + i]; + length = (length << 8U) | byteAt(2 + i); } header = 10; } @@ -3357,7 +3362,7 @@ class RawWsPeer { return; } if (opcode == 0x1 || opcode == 0x0) { - _partial.append(_buffer.constData() + header, static_cast(length)); + _partial += _buffer.sliced(header, static_cast(length)).toStdString(); if (fin) { _messages.push_back(std::move(_partial)); _partial.clear(); @@ -3414,8 +3419,8 @@ TEST_CASE("qt_stalled_reader_is_held_back_then_resynced", "[push][channel]") { CHECK(morph::wire::decode(got[0]).kind == "event"); auto const second = morph::wire::decode(got[1]); UNSCOPED_INFO("second message: " << got[1].substr(0, 120)); - CHECK(second.body.find(R"("topic":"resync")") != std::string::npos); - CHECK(second.body.find("held") == std::string::npos); + CHECK(second.body.contains(R"("topic":"resync")")); + CHECK_FALSE(second.body.contains("held")); } TEST_CASE("qt_stalled_peer_closed_after_send_timeout_wss", "[push][channel]") { @@ -3427,16 +3432,16 @@ TEST_CASE("qt_stalled_peer_closed_after_send_timeout_wss", "[push][channel]") { std::mutex logMtx; std::vector> logged; - morph::log::ScopedLoggerOverride guard{[&](morph::log::LogLevel /*level*/, std::string_view msg) { - std::scoped_lock const lock{logMtx}; - logged.emplace_back(std::chrono::steady_clock::now(), - std::string{msg}); - }, - morph::log::LogLevel::debug}; + morph::log::ScopedLoggerOverride const guard{[&](morph::log::LogLevel /*level*/, std::string_view msg) { + std::scoped_lock const lock{logMtx}; + logged.emplace_back(std::chrono::steady_clock::now(), + std::string{msg}); + }, + morph::log::LogLevel::debug}; auto findLogged = [&](std::string_view needle) -> std::optional { std::scoped_lock const lock{logMtx}; for (auto const& [when, msg] : logged) { - if (msg.find(needle) != std::string::npos) { + if (msg.contains(needle)) { return when; } } @@ -3496,13 +3501,11 @@ TEST_CASE("qt_channel_conformance", "[push][channel]") { auto const cid = nextQtConnectionId(*server); morph::qt::QtWebSocketBackend backend{QUrl{QString("ws://127.0.0.1:%1").arg(wsServer.port())}}; REQUIRE(backend.waitForConnected()); - morph::testing::push::ChannelFixture fixture{.server = server.get(), - .cid = cid, - .backend = &backend, - .waitFor = [](const std::function& done) { - pumpUntil(done, 300); - return done(); - }}; + morph::testing::push::ChannelFixture fixture{ + .server = server.get(), .cid = cid, .backend = &backend, .waitFor = [](const std::function& done) { + pumpUntil(done, 300); + return done(); + }}; body(fixture); }); } diff --git a/tests/test_push_catalog.cpp b/tests/test_push_catalog.cpp index 8db5674b8..acb406c4c 100644 --- a/tests/test_push_catalog.cpp +++ b/tests/test_push_catalog.cpp @@ -14,6 +14,7 @@ #include #include #include +#include #include #include "ui_service_support.hpp" @@ -126,8 +127,8 @@ TEST_CASE("catalog_facts_carry_effects_and_records", "[push][catalog]") { auto const& edit = facts.actions.at("T_CatalogEdit"); REQUIRE(edit.effects.has_value()); - REQUIRE(*edit.effects == std::vector{{"updates", "T_CatalogEntry"}, - {"changes", "T_CatalogEntry"}}); + REQUIRE(*edit.effects == + std::vector{{"updates", "T_CatalogEntry"}, {"changes", "T_CatalogEntry"}}); REQUIRE(edit.records.size() == 1U); CHECK(edit.records.front().record == "T_CatalogEntry"); CHECK(edit.records.front().keyField == std::optional{"id"}); @@ -163,8 +164,8 @@ TEST_CASE("ui_catalog_reply_round_trips_effects", "[push][catalog]") { CHECK(edit.effects == reply.actions->at("T_CatalogEdit").effects); CHECK(edit.records == reply.actions->at("T_CatalogEdit").records); CHECK_FALSE(parsed->actions->at("T_CatalogList").effects.has_value()); - CHECK(text.find(R"("effects":[{"kind":"updates","record":"T_CatalogEntry"})") != std::string::npos); - CHECK(text.find(R"({"kind":"changesInstance"})") != std::string::npos); + CHECK(text.contains(R"("effects":[{"kind":"updates","record":"T_CatalogEntry"})")); + CHECK(text.contains(R"({"kind":"changesInstance"})")); } TEST_CASE("old_reply_without_fields_reads_as_reads", "[push][catalog]") { @@ -234,7 +235,8 @@ TEST_CASE("hello_events_round_trip_on_the_wire", "[push][catalog]") { TEST_CASE("watch_body_round_trips_and_is_bounded", "[push][catalog]") { morph::wire::WatchBody body; body.records = {"Entry", "Tag"}; - body.instances = {{"Editor", "k1", 0}, {"Editor", "", 7}}; + body.instances = {{.modelType = "Editor", .primary = "k1", .instance = 0}, + {.modelType = "Editor", .primary = "", .instance = 7}}; body.replace = true; auto const text = glz::write_json(body).value_or(""); auto const back = morph::wire::readWatchBody(text); @@ -260,7 +262,7 @@ TEST_CASE("watch_body_round_trips_and_is_bounded", "[push][catalog]") { } SECTION("an over-long instance list or name is refused") { morph::wire::WatchBody big; - big.instances.assign(morph::wire::kMaxWatchTopics + 1, {"M", "", 0}); + big.instances.assign(morph::wire::kMaxWatchTopics + 1, {.modelType = "M", .primary = "", .instance = 0}); CHECK_FALSE(morph::wire::readWatchBody(glz::write_json(big).value_or(""))); morph::wire::WatchBody longName; longName.records = {std::string(morph::wire::kMaxTopicNameBytes + 1, 'x')}; @@ -275,7 +277,7 @@ TEST_CASE("watch_body_round_trips_and_is_bounded", "[push][catalog]") { TEST_CASE("watch_reply_round_trips", "[push][catalog]") { morph::wire::WatchReply reply; reply.refusedRecords = {"Secret"}; - reply.refusedInstances = {{"Editor", "p", 3}}; + reply.refusedInstances = {{.modelType = "Editor", .primary = "p", .instance = 3}}; auto const back = morph::wire::readWatchReply(glz::write_json(reply).value_or("")); REQUIRE(back); CHECK(back->refusedRecords == reply.refusedRecords); @@ -289,8 +291,8 @@ TEST_CASE("watch_reply_round_trips", "[push][catalog]") { } TEST_CASE("changes_body_round_trips_and_is_bounded", "[push][catalog]") { - auto const text = R"({"records":[{"record":"Entry","keys":[1,"b"]},{"record":"Tag"}],)" - R"("instances":[{"modelType":"Editor","primary":"p","instance":9}]})"; + std::string_view const text = R"({"records":[{"record":"Entry","keys":[1,"b"]},{"record":"Tag"}],)" + R"("instances":[{"modelType":"Editor","primary":"p","instance":9}]})"; auto const back = morph::wire::readChangesBody(text); REQUIRE(back); REQUIRE(back->records.size() == 2); @@ -301,7 +303,7 @@ TEST_CASE("changes_body_round_trips_and_is_bounded", "[push][catalog]") { CHECK(back->instances[0].instance == 9); SECTION("it is the payload makeEvent carries") { - auto const env = morph::wire::makeEvent(morph::wire::kChangesTopic, text); + auto const env = morph::wire::makeEvent(morph::wire::kChangesTopic, std::string{text}); morph::wire::EventBody event; REQUIRE_FALSE(glz::read_json(event, env.body)); CHECK(event.topic == "changes"); @@ -310,7 +312,7 @@ TEST_CASE("changes_body_round_trips_and_is_bounded", "[push][catalog]") { SECTION("exactly the key bound is accepted") { std::string keys = "["; for (std::size_t i = 0; i < morph::effects::kMaxChangeKeys; ++i) { - keys += (i ? ",1" : "1"); + keys += (i != 0 ? ",1" : "1"); } keys += "]"; auto const at = morph::wire::readChangesBody(R"({"records":[{"record":"E","keys":)" + keys + "}]}"); @@ -320,7 +322,7 @@ TEST_CASE("changes_body_round_trips_and_is_bounded", "[push][catalog]") { SECTION("too many keys in one record is refused") { std::string keys = "["; for (std::size_t i = 0; i <= morph::effects::kMaxChangeKeys; ++i) { - keys += (i ? ",1" : "1"); + keys += (i != 0 ? ",1" : "1"); } keys += "]"; CHECK_FALSE(morph::wire::readChangesBody(R"({"records":[{"record":"E","keys":)" + keys + "}]}")); @@ -328,7 +330,7 @@ TEST_CASE("changes_body_round_trips_and_is_bounded", "[push][catalog]") { SECTION("too many records is refused") { std::string records = "["; for (std::size_t i = 0; i <= morph::wire::kMaxWatchTopics; ++i) { - records += (i ? R"(,{"record":"R"})" : R"({"record":"R"})"); + records += (i != 0 ? R"(,{"record":"R"})" : R"({"record":"R"})"); } records += "]"; CHECK_FALSE(morph::wire::readChangesBody(R"({"records":)" + records + "}")); diff --git a/tests/test_push_channel.cpp b/tests/test_push_channel.cpp index 86db47d76..9aa2ca022 100644 --- a/tests/test_push_channel.cpp +++ b/tests/test_push_channel.cpp @@ -63,7 +63,7 @@ TEST_CASE("reply_router_routes_event_before_call_match", "[push][channel]") { table.insert(0, 7); int calls = 0; - auto const handler = [&](::morph::wire::EventBody) { ++calls; }; + auto const handler = [&](::morph::wire::EventBody const& /*event*/) { ++calls; }; CHECK(routeEvent(::morph::wire::makeEvent("resync", "null"), handler)); CHECK(calls == 1); CHECK(table.size() == 1); // the pending callId 0 is untouched @@ -78,7 +78,7 @@ TEST_CASE("reply_router_drops_malformed_event_and_event_without_handler", "[push bad.kind = "event"; bad.body = "not json"; int calls = 0; - CHECK(routeEvent(bad, [&](::morph::wire::EventBody) { ++calls; })); + CHECK(routeEvent(bad, [&](::morph::wire::EventBody const& /*event*/) { ++calls; })); CHECK(calls == 0); CHECK(routeEvent(::morph::wire::makeEvent("changes", "{}"), {})); } @@ -88,18 +88,18 @@ TEST_CASE("local_backend_deliver_event_reaches_handler_on_exec", "[push][channel InlineExec cb; ::morph::backend::LocalBackend backend{pool}; - backend.deliverEvent({"changes", {"{}"}}); // no handler: dropped + backend.deliverEvent({.topic = "changes", .payload = {"{}"}}); // no handler: dropped CHECK(cb.posted == 0); std::vector topics; - backend.setEventHandler([&](::morph::wire::EventBody e) { topics.push_back(e.topic); }, &cb); - backend.deliverEvent({"changes", {"{}"}}); - backend.deliverEvent({"resync", {"null"}}); + backend.setEventHandler([&](::morph::wire::EventBody const& event) { topics.push_back(event.topic); }, &cb); + backend.deliverEvent({.topic = "changes", .payload = {"{}"}}); + backend.deliverEvent({.topic = "resync", .payload = {"null"}}); CHECK(cb.posted == 2); CHECK(topics == std::vector{"changes", "resync"}); backend.setEventHandler(nullptr, nullptr); - backend.deliverEvent({"changes", {"{}"}}); + backend.deliverEvent({.topic = "changes", .payload = {"{}"}}); CHECK(topics.size() == 2); } @@ -355,8 +355,7 @@ TEST_CASE("throwing_backlog_read_is_caught_like_a_throwing_write", "[push][chann { std::scoped_lock const lock{logMtx}; auto const reported = std::ranges::count_if(lines, [cid](const std::string& line) { - return line.find("write to connection " + std::to_string(cid) + " failed: counter gone") != - std::string::npos; + return line.contains("write to connection " + std::to_string(cid) + " failed: counter gone"); }); CHECK(reported == 1); } @@ -375,10 +374,9 @@ TEST_CASE("simulated_backend_channel_conformance", "[push][channel]") { auto const cid = server->openConnection(); ::morph::backend::SimulatedRemoteBackend backend{*server, cid}; ::morph::testing::push::ChannelFixture fixture{ - .server = server.get(), - .cid = cid, - .backend = &backend, - .waitFor = [](const std::function& done) { return ::morph::testing::waitUntil(done); }}; + .server = server.get(), .cid = cid, .backend = &backend, .waitFor = [](const std::function& done) { + return ::morph::testing::waitUntil(done); + }}; body(fixture); }); } diff --git a/tests/test_push_effects.cpp b/tests/test_push_effects.cpp index 4e8d6dfba..9dda99105 100644 --- a/tests/test_push_effects.cpp +++ b/tests/test_push_effects.cpp @@ -8,6 +8,7 @@ #include #include #include +#include #include #include @@ -75,6 +76,7 @@ namespace { // Whether the late record was already registered when this file's registrars ran. The ordering test // proves something only when it was not. +// NOLINTNEXTLINE(bugprone-throwing-static-initialization,cert-err58-cpp): reading the registry during static initialisation is the measurement. bool const lateRecordSeenEarly = std::ranges::any_of(morph::record::detail::recordRegistry(), [](morph::record::RecordInfo const& info) { return info.name == "T_LateRow"; }); @@ -117,15 +119,15 @@ DeclaredEffect updatesOf(std::type_index type) { return DeclaredEffect{.kind = K DeclaredEffect changesOf(std::type_index type) { return DeclaredEffect{.kind = Kind::Changes, .record = type}; } // Fails the test when resolving does not throw; returns the message when it does. -std::string refusal(std::vector records, ActionResultSlotMap slots, EffectsMap declared, - std::vector slotConflicts = {}) { +std::string refusal(std::vector records, ActionResultSlotMap const& resultSlots, + EffectsMap const& declared, std::vector const& slotConflicts = {}) { try { - static_cast(morph::effects::detail::resolveForTesting(std::move(records), std::move(slots), - std::move(declared), std::move(slotConflicts))); + static_cast( + morph::effects::detail::resolveForTesting(std::move(records), resultSlots, declared, slotConflicts)); } catch (morph::model::EffectsRegistrationError const& error) { return error.what(); } - FAIL("resolving did not throw EffectsRegistrationError"); + FAIL_CHECK("resolving did not throw EffectsRegistrationError"); return {}; } @@ -189,15 +191,15 @@ TEST_CASE("reading_effects_closes_the_registration_phase", "[push][records]") { TEST_CASE("matching_stops_at_first_record_on_a_path", "[push][records]") { ActionResultSlotMap slots; - slots.emplace("A_Nested", result({{typeid(VList), ""}, - {typeid(VHeader), "header"}, - {typeid(VRow), "header.rows"}, - {typeid(VInner), "header.rows.inner"}, - {typeid(VRow), "loose"}})); - slots.emplace("A_Root", result({{typeid(VHeader), ""}, {typeid(VRow), "rows"}})); + slots.emplace("A_Nested", result({{.type = typeid(VList), .path = ""}, + {.type = typeid(VHeader), .path = "header"}, + {.type = typeid(VRow), .path = "header.rows"}, + {.type = typeid(VInner), .path = "header.rows.inner"}, + {.type = typeid(VRow), .path = "loose"}})); + slots.emplace("A_Root", result({{.type = typeid(VHeader), .path = ""}, {.type = typeid(VRow), .path = "rows"}})); auto const resolved = morph::effects::detail::resolveForTesting( {record("V_Row", typeid(VRow)), record("V_Inner", typeid(VInner)), record("V_Header", typeid(VHeader), {})}, - std::move(slots), {}); + slots, {}); REQUIRE(resolved.recordsOf("A_Nested") == std::vector{ {"V_Header", std::nullopt, {"header"}}, {"V_Row", "id", {"loose"}}}); @@ -215,9 +217,8 @@ TEST_CASE("effects_twice_same_is_noop", "[push][records]") { REQUIRE(declared.at("A_List").size() == 1); ActionResultSlotMap slots; - slots.emplace("A_List", result({{typeid(VList), ""}, {typeid(VRow), "rows"}})); - auto const resolved = morph::effects::detail::resolveForTesting({record("V_Row", typeid(VRow))}, std::move(slots), - std::move(declared)); + slots.emplace("A_List", result({{.type = typeid(VList), .path = ""}, {.type = typeid(VRow), .path = "rows"}})); + auto const resolved = morph::effects::detail::resolveForTesting({record("V_Row", typeid(VRow))}, slots, declared); REQUIRE(resolved.effectsOf("A_List") == std::vector{{Kind::Updates, "V_Row"}, {Kind::Changes, "V_Row"}}); } @@ -227,36 +228,35 @@ TEST_CASE("effects_conflict_throws", "[push][records]") { morph::effects::detail::addEffects(declared, "A_Conflicted", {changesOf(typeid(VRow))}); morph::effects::detail::addEffects(declared, "A_Conflicted", {DeclaredEffect{.kind = Kind::None, .record = {}}}); REQUIRE(declared.at("A_Conflicted").size() == 2); - auto const message = refusal({record("V_Row", typeid(VRow))}, {}, std::move(declared)); + auto const message = refusal({record("V_Row", typeid(VRow))}, {}, declared); REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("`A_Conflicted`")); REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("different effects")); } TEST_CASE("action_id_held_by_two_action_types_throws", "[push][records]") { ActionResultSlotMap slots; - slots.emplace("A_Shared", result({{typeid(VRow), ""}})); - auto const message = refusal({record("V_Row", typeid(VRow))}, std::move(slots), {}, {"A_Shared"}); + slots.emplace("A_Shared", result({{.type = typeid(VRow), .path = ""}})); + auto const message = refusal({record("V_Row", typeid(VRow))}, slots, {}, {"A_Shared"}); REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("`A_Shared`")); REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("two action types")); } TEST_CASE("updates_without_record_in_result_throws", "[push][records]") { ActionResultSlotMap slots; - slots.emplace("A_Counts", result({{typeid(VList), ""}})); + slots.emplace("A_Counts", result({{.type = typeid(VList), .path = ""}})); EffectsMap declared; morph::effects::detail::addEffects(declared, "A_Counts", {updatesOf(typeid(VRow))}); - auto const message = refusal({record("V_Row", typeid(VRow))}, std::move(slots), std::move(declared)); + auto const message = refusal({record("V_Row", typeid(VRow))}, slots, declared); REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("`A_Counts`")); REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("result holds no `V_Row`")); } TEST_CASE("updates_on_keyless_throws", "[push][records]") { ActionResultSlotMap slots; - slots.emplace("A_Report", result({{typeid(VKeyless), ""}})); + slots.emplace("A_Report", result({{.type = typeid(VKeyless), .path = ""}})); EffectsMap declared; morph::effects::detail::addEffects(declared, "A_Report", {updatesOf(typeid(VKeyless))}); - auto const message = - refusal({record("V_Keyless", typeid(VKeyless), std::nullopt)}, std::move(slots), std::move(declared)); + auto const message = refusal({record("V_Keyless", typeid(VKeyless), std::nullopt)}, slots, declared); REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("`A_Report`")); REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("`V_Keyless`, which has no key")); } @@ -264,7 +264,7 @@ TEST_CASE("updates_on_keyless_throws", "[push][records]") { TEST_CASE("effect_on_unregistered_type_throws", "[push][records]") { EffectsMap declared; morph::effects::detail::addEffects(declared, "A_Stray", {changesOf(typeid(VOther))}); - auto const message = refusal({record("V_Row", typeid(VRow))}, {}, std::move(declared)); + auto const message = refusal({record("V_Row", typeid(VRow))}, {}, declared); REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("`A_Stray`")); REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("not a registered record")); } @@ -281,14 +281,15 @@ TEST_CASE("duplicate_record_name_throws", "[push][records]") { auto const byName = refusal({record("V_Row", typeid(VRow)), record("V_Row", typeid(VOther))}, {}, {}); REQUIRE_THAT(byName, Catch::Matchers::ContainsSubstring("record name `V_Row` is registered for two types")); auto const byType = refusal({record("V_Row", typeid(VRow)), record("V_Row2", typeid(VRow))}, {}, {}); - REQUIRE_THAT(byType, Catch::Matchers::ContainsSubstring("registered under two record names (`V_Row` and `V_Row2`)")); + REQUIRE_THAT(byType, + Catch::Matchers::ContainsSubstring("registered under two record names (`V_Row` and `V_Row2`)")); } TEST_CASE("identical_record_registrations_are_one", "[push][records]") { ActionResultSlotMap slots; - slots.emplace("A_List", result({{typeid(VList), ""}, {typeid(VRow), "rows"}})); + slots.emplace("A_List", result({{.type = typeid(VList), .path = ""}, {.type = typeid(VRow), .path = "rows"}})); auto const resolved = morph::effects::detail::resolveForTesting( - {record("V_Row", typeid(VRow)), record("V_Row", typeid(VRow))}, std::move(slots), {}); + {record("V_Row", typeid(VRow)), record("V_Row", typeid(VRow))}, slots, {}); REQUIRE(resolved.records.size() == 1); REQUIRE(resolved.recordNamed("V_Row") != nullptr); REQUIRE(resolved.recordsOf("A_List") == std::vector{{"V_Row", "id", {"rows"}}}); @@ -296,21 +297,24 @@ TEST_CASE("identical_record_registrations_are_one", "[push][records]") { TEST_CASE("record_same_type_different_options_throws", "[push][records]") { auto const message = refusal({record("V_Row", typeid(VRow)), record("V_Row", typeid(VRow), "code")}, {}, {}); - REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("record `V_Row` is registered twice with different options")); + REQUIRE_THAT(message, + Catch::Matchers::ContainsSubstring("record `V_Row` is registered twice with different options")); } TEST_CASE("keyed_derived_record_throws", "[push][records]") { auto const message = refusal({record("V_Row", typeid(VRow)), record("V_Header", typeid(VHeader), "id", {typeid(VRow)})}, {}, {}); - REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("record `V_Header` is derived from other records but has a key")); + REQUIRE_THAT(message, + Catch::Matchers::ContainsSubstring("record `V_Header` is derived from other records but has a key")); } TEST_CASE("effect_none_with_others_throws", "[push][records]") { EffectsMap declared; morph::effects::detail::addEffects(declared, "A_Mixed", {changesOf(typeid(VRow)), DeclaredEffect{.kind = Kind::None, .record = {}}}); - auto const message = refusal({record("V_Row", typeid(VRow))}, {}, std::move(declared)); - REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("action `A_Mixed` declares none together with other effects")); + auto const message = refusal({record("V_Row", typeid(VRow))}, {}, declared); + REQUIRE_THAT(message, + Catch::Matchers::ContainsSubstring("action `A_Mixed` declares none together with other effects")); } TEST_CASE("repeated_effect_in_declaration_is_one", "[push][records]") { @@ -319,23 +323,22 @@ TEST_CASE("repeated_effect_in_declaration_is_one", "[push][records]") { morph::effects::detail::addEffects(declared, "A_List", {updatesOf(typeid(VRow))}); REQUIRE(declared.at("A_List").size() == 1); ActionResultSlotMap slots; - slots.emplace("A_List", result({{typeid(VRow), ""}})); - auto const resolved = morph::effects::detail::resolveForTesting({record("V_Row", typeid(VRow))}, std::move(slots), - std::move(declared)); + slots.emplace("A_List", result({{.type = typeid(VRow), .path = ""}})); + auto const resolved = morph::effects::detail::resolveForTesting({record("V_Row", typeid(VRow))}, slots, declared); REQUIRE(resolved.effectsOf("A_List") == std::vector{{Kind::Updates, "V_Row"}}); } TEST_CASE("every_problem_is_listed", "[push][records]") { ActionResultSlotMap slots; - slots.emplace("A_Counts", result({{typeid(VList), ""}})); - slots.emplace("A_Report", result({{typeid(VKeyless), ""}})); + slots.emplace("A_Counts", result({{.type = typeid(VList), .path = ""}})); + slots.emplace("A_Report", result({{.type = typeid(VKeyless), .path = ""}})); EffectsMap declared; morph::effects::detail::addEffects(declared, "A_Counts", {updatesOf(typeid(VRow))}); morph::effects::detail::addEffects(declared, "A_Report", {updatesOf(typeid(VKeyless))}); auto const message = refusal( {record("V_Row", typeid(VRow)), record("V_Keyless", typeid(VKeyless), std::nullopt), record("V_CycleA", typeid(VCycleA), std::nullopt, {typeid(VCycleA)}), record("V_Row", typeid(VOther))}, - std::move(slots), std::move(declared)); + slots, declared); REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("result holds no `V_Row`")); REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("`V_Keyless`, which has no key")); REQUIRE_THAT(message, Catch::Matchers::ContainsSubstring("V_CycleA -> V_CycleA")); @@ -364,34 +367,36 @@ std::vector keyTexts(RecordChange const& change) { } // Records: D_Row (keyed), D_Txn (keyed), D_Report derived from D_Txn, D_Summary derived from D_Report. -morph::effects::detail::Resolved derivation(EffectsMap declared) { +morph::effects::detail::Resolved derivation(EffectsMap const& declared) { ActionResultSlotMap slots; - slots.emplace("A_Rows", result({{typeid(VList), ""}, {typeid(VRow), "rows"}})); - slots.emplace("A_Root", result({{typeid(VRow), ""}})); - slots.emplace("A_Deep", result({{typeid(VList), ""}, {typeid(VRow), "header.rows"}})); - slots.emplace("A_Named", result({{typeid(VList), ""}, {typeid(DNamed), "byName"}})); + slots.emplace("A_Rows", result({{.type = typeid(VList), .path = ""}, {.type = typeid(VRow), .path = "rows"}})); + slots.emplace("A_Root", result({{.type = typeid(VRow), .path = ""}})); + slots.emplace("A_Deep", + result({{.type = typeid(VList), .path = ""}, {.type = typeid(VRow), .path = "header.rows"}})); + slots.emplace("A_Named", + result({{.type = typeid(VList), .path = ""}, {.type = typeid(DNamed), .path = "byName"}})); return morph::effects::detail::resolveForTesting( {record("D_Row", typeid(VRow)), record("D_Txn", typeid(DTxn)), record("D_Report", typeid(DReport), std::nullopt, {typeid(DTxn)}), - record("D_Summary", typeid(DSummary), std::nullopt, {typeid(DReport)}), record("D_Header", typeid(VHeader), {}), - record("D_Named", typeid(DNamed), "code")}, - std::move(slots), declared); + record("D_Summary", typeid(DSummary), std::nullopt, {typeid(DReport)}), + record("D_Header", typeid(VHeader), {}), record("D_Named", typeid(DNamed), "code")}, + slots, declared); } -EffectsMap declare(std::string id, std::vector effects) { +EffectsMap declare(std::string_view actionId, std::vector effects) { EffectsMap declared; - morph::effects::detail::addEffects(declared, id, std::move(effects)); + morph::effects::detail::addEffects(declared, actionId, std::move(effects)); return declared; } -InstanceChange const ranOnInstance{.modelType = "M", .instance = 7, .primary = "k"}; +InstanceChange ranOnInstance() { return {.modelType = "M", .instance = 7, .primary = "k"}; } } // namespace TEST_CASE("derive_updates_reads_keys_at_paths", "[push][records]") { auto const resolved = derivation(declare("A_Rows", {updatesOf(typeid(VRow))})); auto const changes = morph::effects::detail::deriveChangesFrom( - resolved, "A_Rows", R"({"rows":[{"id":1,"n":"a"},{"id":2},{"n":"no key"}]})", ranOnInstance); + resolved, "A_Rows", R"({"rows":[{"id":1,"n":"a"},{"id":2},{"n":"no key"}]})", ranOnInstance()); REQUIRE(changes.instances.empty()); REQUIRE(changes.records.size() == 1); REQUIRE(changes.records[0].record == "D_Row"); @@ -409,13 +414,13 @@ TEST_CASE("derive_updates_collapses_past_the_key_bound", "[push][records]") { return json + "]}"; }; auto const atBound = morph::effects::detail::deriveChangesFrom( - resolved, "A_Rows", rowsJson(morph::effects::kMaxChangeKeys), ranOnInstance); + resolved, "A_Rows", rowsJson(morph::effects::kMaxChangeKeys), ranOnInstance()); REQUIRE(atBound.records.size() == 1); REQUIRE(atBound.records[0].keys.has_value()); CHECK(atBound.records[0].keys->size() == morph::effects::kMaxChangeKeys); auto const pastBound = morph::effects::detail::deriveChangesFrom( - resolved, "A_Rows", rowsJson(morph::effects::kMaxChangeKeys + 1), ranOnInstance); + resolved, "A_Rows", rowsJson(morph::effects::kMaxChangeKeys + 1), ranOnInstance()); REQUIRE(pastBound.records.size() == 1); CHECK(pastBound.records[0].record == "D_Row"); CHECK_FALSE(pastBound.records[0].keys.has_value()); @@ -424,27 +429,27 @@ TEST_CASE("derive_updates_collapses_past_the_key_bound", "[push][records]") { TEST_CASE("derive_updates_reads_every_result_shape", "[push][records]") { auto const rootResolved = derivation(declare("A_Root", {updatesOf(typeid(VRow))})); auto const one = morph::effects::detail::deriveChangesFrom(rootResolved, "A_Root", R"({"id":9007199254740993})", - ranOnInstance); + ranOnInstance()); REQUIRE(keyTexts(one.records.at(0)) == std::vector{"9007199254740993"}); - auto const absent = morph::effects::detail::deriveChangesFrom(rootResolved, "A_Root", "null", ranOnInstance); + auto const absent = morph::effects::detail::deriveChangesFrom(rootResolved, "A_Root", "null", ranOnInstance()); REQUIRE(absent.records.at(0).keys.has_value()); REQUIRE(absent.records.at(0).keys->empty()); auto const deepResolved = derivation(declare("A_Deep", {updatesOf(typeid(VRow))})); auto const deep = morph::effects::detail::deriveChangesFrom( - deepResolved, "A_Deep", R"({"header":{"rows":[{"id":"x"},{"id":"y"}]}})", ranOnInstance); + deepResolved, "A_Deep", R"({"header":{"rows":[{"id":"x"},{"id":"y"}]}})", ranOnInstance()); REQUIRE(keyTexts(deep.records.at(0)) == std::vector{"\"x\"", "\"y\""}); auto const namedResolved = derivation(declare("A_Named", {updatesOf(typeid(DNamed))})); auto const map = morph::effects::detail::deriveChangesFrom( - namedResolved, "A_Named", R"({"byName":{"a":{"code":"c1"},"b":{"code":"c2"}}})", ranOnInstance); + namedResolved, "A_Named", R"({"byName":{"a":{"code":"c1"},"b":{"code":"c2"}}})", ranOnInstance()); REQUIRE(keyTexts(map.records.at(0)) == std::vector{"\"c1\"", "\"c2\""}); } TEST_CASE("derive_updates_with_unparseable_result_reports_the_type", "[push][records]") { auto const resolved = derivation(declare("A_Rows", {updatesOf(typeid(VRow))})); - auto const changes = morph::effects::detail::deriveChangesFrom(resolved, "A_Rows", "{not json", ranOnInstance); + auto const changes = morph::effects::detail::deriveChangesFrom(resolved, "A_Rows", "{not json", ranOnInstance()); REQUIRE(changes.records.size() == 1); REQUIRE_FALSE(changes.records[0].keys.has_value()); } @@ -452,7 +457,7 @@ TEST_CASE("derive_updates_with_unparseable_result_reports_the_type", "[push][rec TEST_CASE("derive_changes_has_no_keys", "[push][records]") { auto const resolved = derivation(declare("A_Rows", {changesOf(typeid(VRow))})); auto const changes = - morph::effects::detail::deriveChangesFrom(resolved, "A_Rows", R"({"rows":[{"id":1}]})", ranOnInstance); + morph::effects::detail::deriveChangesFrom(resolved, "A_Rows", R"({"rows":[{"id":1}]})", ranOnInstance()); REQUIRE(changes.records.size() == 1); REQUIRE(changes.records[0].record == "D_Row"); REQUIRE_FALSE(changes.records[0].keys.has_value()); @@ -461,14 +466,14 @@ TEST_CASE("derive_changes_has_no_keys", "[push][records]") { TEST_CASE("derive_changes_absorbs_updates_keys", "[push][records]") { auto const resolved = derivation(declare("A_Rows", {updatesOf(typeid(VRow)), changesOf(typeid(VRow))})); auto const changes = - morph::effects::detail::deriveChangesFrom(resolved, "A_Rows", R"({"rows":[{"id":1}]})", ranOnInstance); + morph::effects::detail::deriveChangesFrom(resolved, "A_Rows", R"({"rows":[{"id":1}]})", ranOnInstance()); REQUIRE(changes.records.size() == 1); REQUIRE_FALSE(changes.records[0].keys.has_value()); } TEST_CASE("derive_expands_derived_from", "[push][records]") { auto const resolved = derivation(declare("A_Rows", {changesOf(typeid(DTxn))})); - auto const changes = morph::effects::detail::deriveChangesFrom(resolved, "A_Rows", "{}", ranOnInstance); + auto const changes = morph::effects::detail::deriveChangesFrom(resolved, "A_Rows", "{}", ranOnInstance()); REQUIRE(changes.records.size() == 3); REQUIRE(changes.records[0].record == "D_Txn"); // A derived record of a derived record is reached too, and neither carries keys. @@ -480,7 +485,7 @@ TEST_CASE("derive_expands_derived_from", "[push][records]") { TEST_CASE("derive_changes_instance_uses_ran_on", "[push][records]") { auto const resolved = derivation(declare("A_Rows", {DeclaredEffect{.kind = Kind::ChangesInstance, .record = {}}})); - auto const changes = morph::effects::detail::deriveChangesFrom(resolved, "A_Rows", "{}", ranOnInstance); + auto const changes = morph::effects::detail::deriveChangesFrom(resolved, "A_Rows", "{}", ranOnInstance()); REQUIRE(changes.records.empty()); REQUIRE(changes.instances.size() == 1); REQUIRE(changes.instances[0].modelType == "M"); @@ -490,21 +495,21 @@ TEST_CASE("derive_changes_instance_uses_ran_on", "[push][records]") { TEST_CASE("derive_reads_nothing_for_reader", "[push][records]") { auto const resolved = derivation(declare("A_Other", {changesOf(typeid(VRow))})); - REQUIRE(morph::effects::detail::deriveChangesFrom(resolved, "A_Rows", "{}", ranOnInstance).empty()); + REQUIRE(morph::effects::detail::deriveChangesFrom(resolved, "A_Rows", "{}", ranOnInstance()).empty()); auto const none = derivation(declare("A_Rows", {DeclaredEffect{.kind = Kind::None, .record = {}}})); - REQUIRE(morph::effects::detail::deriveChangesFrom(none, "A_Rows", "{}", ranOnInstance).empty()); + REQUIRE(morph::effects::detail::deriveChangesFrom(none, "A_Rows", "{}", ranOnInstance()).empty()); // The process registry: a registered action that declared nothing reads. - REQUIRE(morph::effects::deriveChanges("T_WalkerListRows", "{}", ranOnInstance).empty()); + REQUIRE(morph::effects::deriveChanges("T_WalkerListRows", "{}", ranOnInstance()).empty()); } TEST_CASE("derive_from_process_registry", "[push][records]") { auto const changes = - morph::effects::deriveChanges("T_EffectsEdit", R"({"items":[{"id":5,"name":"a"}]})", ranOnInstance); + morph::effects::deriveChanges("T_EffectsEdit", R"({"items":[{"id":5,"name":"a"}]})", ranOnInstance()); // updates and changes together: the whole type. REQUIRE(changes.records.size() == 1); REQUIRE(changes.records[0].record == "T_EItem"); REQUIRE_FALSE(changes.records[0].keys.has_value()); - auto const instance = morph::effects::deriveChanges("T_EffectsLoad", "{}", ranOnInstance); + auto const instance = morph::effects::deriveChanges("T_EffectsLoad", "{}", ranOnInstance()); REQUIRE(instance.instances.size() == 1); REQUIRE(instance.records.empty()); } diff --git a/tests/test_push_records.cpp b/tests/test_push_records.cpp index 392728b5e..566b474a1 100644 --- a/tests/test_push_records.cpp +++ b/tests/test_push_records.cpp @@ -104,8 +104,10 @@ struct WalkerModel { template <> struct glz::meta { + // NOLINTBEGIN(readability-identifier-naming): glaze looks these names up. static constexpr bool custom_read = true; static constexpr bool custom_write = true; + // NOLINTEND(readability-identifier-naming) }; template <> @@ -133,7 +135,8 @@ BRIDGE_REGISTER_ACTION(WalkerModel, CountRows, "T_WalkerCountRows") namespace { [[maybe_unused]] bool const walkerSameFirst = morph::model::detail::registerActionResultOnce("T_WalkerSame"); -[[maybe_unused]] bool const walkerSameSecond = morph::model::detail::registerActionResultOnce("T_WalkerSame"); +[[maybe_unused]] bool const walkerSameSecond = + morph::model::detail::registerActionResultOnce("T_WalkerSame"); } // namespace namespace { @@ -280,6 +283,11 @@ TEST_CASE("action_result_second_type_under_one_id_is_a_conflict", "[push][record morph::model::closeRegistrationPhase(); } struct Restore { + Restore() = default; + Restore(Restore const&) = delete; + Restore& operator=(Restore const&) = delete; + Restore(Restore&&) = delete; + Restore& operator=(Restore&&) = delete; ~Restore() { morph::model::detail::mutableActionResultSlots().erase("T_WalkerConflict"); std::erase(morph::model::detail::mutableActionResultConflicts(), std::string{"T_WalkerConflict"}); diff --git a/tests/test_push_watch.cpp b/tests/test_push_watch.cpp index ef86430db..3febfb1e8 100644 --- a/tests/test_push_watch.cpp +++ b/tests/test_push_watch.cpp @@ -235,8 +235,10 @@ struct SecretGuard final : ::morph::session::IAuthorizer { std::string_view actionType, std::uint64_t modelId, std::string_view ownerPrincipal) const override { std::scoped_lock const lock{mtx}; - instanceQuestions.push_back( - {std::string{modelType}, std::string{actionType}, modelId, std::string{ownerPrincipal}}); + instanceQuestions.push_back({.modelType = std::string{modelType}, + .actionType = std::string{actionType}, + .modelId = modelId, + .owner = std::string{ownerPrincipal}}); return modelType != kSecretModel; } }; @@ -621,6 +623,7 @@ ::morph::wire::Envelope describe(Rig& rig, ConnectionId cid, std::string const& /// Sends an `execute` of @p actionType on @p modelId of @p modelType through /// @p cid, signed with @p token, and waits for the reply. +// NOLINTNEXTLINE(bugprone-easily-swappable-parameters): a model type, then an action type; swapped, the server refuses an unknown model. ::morph::wire::Envelope executeOn(Rig& rig, ConnectionId cid, std::string_view modelType, std::string_view actionType, std::uint64_t modelId, std::string const& token) { ::morph::wire::Envelope env; @@ -823,7 +826,9 @@ struct EventWire { } /// How many `resync` events were received. - std::size_t resyncs() { return static_cast(std::ranges::count(topics(), ::morph::wire::kResyncTopic)); } + std::size_t resyncs() { + return static_cast(std::ranges::count(topics(), ::morph::wire::kResyncTopic)); + } /// The `changes` payloads received, decoded; `resync` events are skipped. std::vector<::morph::wire::ChangesBody> changes() { @@ -896,9 +901,10 @@ struct FanRig { } /// Registers a private instance on @p cid, or a keyed shared one when @p primary is set. - std::uint64_t create(ConnectionId cid, std::string primary = {}, std::string const& token = {}) { + // NOLINTNEXTLINE(bugprone-easily-swappable-parameters): a key, then an optional token; swapped, the register is refused. + std::uint64_t create(ConnectionId cid, std::string const& primary = {}, std::string const& token = {}) { auto env = primary.empty() ? ::morph::wire::makeRegister(std::string{kFanModel}) - : ::morph::wire::makeRegisterShared(std::string{kFanModel}, primary); + : ::morph::wire::makeRegisterShared(std::string{kFanModel}, primary); env.session.token = token; auto const reply = ::morph::wire::decode(server->handleInline(::morph::wire::encode(env), cid)); REQUIRE(reply.kind == "ok"); @@ -906,20 +912,22 @@ struct FanRig { return reply.modelId; } - /// Executes @p actionType with @p body on @p modelId through @p cid and - /// waits for the reply. - ::morph::wire::Envelope execute(ConnectionId cid, std::uint64_t modelId, std::string_view actionType, + /// Executes @p actionType with @p body on @p instance through @p connection + /// and waits for the reply. + // NOLINTBEGIN(bugprone-easily-swappable-parameters): the envelope's fields in wire order; swapped, the server refuses it. + ::morph::wire::Envelope execute(ConnectionId connection, std::uint64_t instance, std::string_view actionType, std::string body, std::string const& token = {}) { + // NOLINTEND(bugprone-easily-swappable-parameters) ::morph::wire::Envelope env; env.kind = "execute"; env.session.token = token; env.callId = 9; - env.modelId = modelId; + env.modelId = instance; env.modelType = std::string{kFanModel}; env.actionType = std::string{actionType}; env.body = std::move(body); auto waiter = std::make_shared<::morph::testing::WaitReply>(); - server->handle(::morph::wire::encode(env), [waiter](std::string const& msg) { (*waiter)(msg); }, cid); + server->handle(::morph::wire::encode(env), [waiter](std::string const& msg) { (*waiter)(msg); }, connection); REQUIRE(waiter->await()); return waiter->env; } @@ -1371,7 +1379,8 @@ TEST_CASE("expired_session_drop_is_one_resync", "[push][watch]") { FanRig rig{std::chrono::milliseconds{0}, guard}; auto const caller = rig.connect(); auto const watcher = rig.connect(); - rig.watch(watcher, {.records = {"T_FanRow"}, .instances = {{.modelType = std::string{kFanModel}, .primary = "k1"}}}, + rig.watch(watcher, + {.records = {"T_FanRow"}, .instances = {{.modelType = std::string{kFanModel}, .primary = "k1"}}}, "carol"); auto const mid = rig.create(caller, "k1", "alice"); @@ -1469,7 +1478,7 @@ ::morph::effects::Changes rowChange(std::vector const& keys) { if (!keys.empty()) { change.keys.emplace(); for (auto const& key : keys) { - change.keys->push_back(glz::raw_json{key}); + change.keys->emplace_back(key); } } return {.records = {std::move(change)}, .instances = {}}; From c89a73194388281d7e4f38f7aaf632504440232b Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 23:23:19 +0200 Subject: [PATCH 43/68] tests: the net backlog test reads through a 256 KiB window, not 2 KiB A receive window below the loopback MSS (64 KiB on Linux) stalls every segment: measured on Linux, each 128 KiB reply took about 6.6 s, so the 16 MiB of replies would take minutes and the server's send deadline closed the connection after 19 of them. 256 KiB still absorbs far less than the 15 MiB the hold-back needs to be engaged, and with the hold-back disabled the test still fails. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- tests/net/test_push_net.cpp | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/tests/net/test_push_net.cpp b/tests/net/test_push_net.cpp index 2a7650942..fe32edbfa 100644 --- a/tests/net/test_push_net.cpp +++ b/tests/net/test_push_net.cpp @@ -519,17 +519,19 @@ TEST_CASE("net_backlog_holds_events_and_drain_sends_resync", "[push][channel]") // drain sends one `resync` in its place. // // Assumes the kernel's loopback buffers (the server's send buffer and the - // client's receive window, shrunk to 2 KiB) absorb far less than 15 MiB, + // client's receive buffer, fixed at 256 KiB) absorb far less than 15 MiB, // so more than `kEventBacklogHigh` is still in the transport's backlog // when the event is posted. Were the kernel to take nearly everything, - // the event would be delivered and this test would fail, not pass. + // the event would be delivered and this test would fail, not pass. The + // receive buffer is not smaller because a window below the loopback MSS + // (64 KiB on Linux) stalls every segment, and 16 MiB then takes minutes. ::morph::exec::ThreadPoolExecutor pool{2}; auto server = std::make_shared<::morph::backend::RemoteServer>(pool); ::morph::net::SocketServer wsServer{*server, 0}; REQUIRE(wsServer.listen()); auto const cid = nextConnectionId(*server); - RawClient client{wsServer.port(), /*receiveBuffer=*/2048}; + RawClient client{wsServer.port(), /*receiveBuffer=*/256 * 1024}; client.boundReceives(20s); client.send(::morph::wire::makeRegister("PnModel")); auto const reg = client.receive(); From a9cee7ee2291f31f0544bc1ec9bcf282fe60b2eb Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 23:36:17 +0200 Subject: [PATCH 44/68] ui: the session watches what its cache holds, before it fetches The session counts, per topic, the entries that hold it: each record an entry's action returns, and the keyed shared instance of an entry whose action returns none. A count leaving zero queues a watch, one returning to zero an unwatch, and fetch sends the queue first, so the server holds the watch before the read. SessionOptions::watcher sends them; watcherOver builds one over an exchange, stamping the session current at each send. Session::reconnected and every resync send the whole set with replace and mark every entry stale; one whose fetch is in flight fetches once more. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- docs/spec/offline/offline.md | 8 +- docs/spec/ui/interpreter.md | 19 +++- docs/spec/ui/service.md | 7 ++ include/morph/ui/interpreter.hpp | 171 +++++++++++++++++++++++++--- include/morph/ui/service_client.hpp | 35 ++++++ tests/test_push_cache.cpp | 135 ++++++++++++++++++++++ tests/ui_interpreter_support.hpp | 31 ++++- 7 files changed, 389 insertions(+), 17 deletions(-) diff --git a/docs/spec/offline/offline.md b/docs/spec/offline/offline.md index de0d6696a..34154366b 100644 --- a/docs/spec/offline/offline.md +++ b/docs/spec/offline/offline.md @@ -930,7 +930,13 @@ more. only to handlers on the same `Bridge` (one client process), so a result another client produced does not reach this client's subscribers — it is observed on this client's next read. See the README's "Instance - subscriptions are best-effort and in-process". + subscriptions are best-effort and in-process". A `ui::Session` with a + `watcher` is the exception at the UI layer: the server tells it which + records changed, and it re-reads them itself (`docs/spec/ui/interpreter.md`, + the query cache). A replayed write's effects reach that cache the same way, + as an event, because the server does not leave the caller out of its + fan-out. Events missed while disconnected are not replayed: on reconnect the + session re-sends its watch set and marks every entry stale, which covers them. None of these is a defect to fix; they are the shape of a queue that stores opaque payloads and replays them through an ordinary action path. A host needing diff --git a/docs/spec/ui/interpreter.md b/docs/spec/ui/interpreter.md index 4c542c6d9..708c5ed36 100644 --- a/docs/spec/ui/interpreter.md +++ b/docs/spec/ui/interpreter.md @@ -42,7 +42,9 @@ Read this before changing what a mounted document does. What a document may say `staleTime` is measured on, how long an unused entry is kept (five minutes), the `Catalog` (`stateless` decides an alias's default instance; an action's `secretFields` are never cached or kept in `result`) and the error classifier (the bridge's own failures map to `offline`, `timeout` and -`backendChanged`; everything else is `error`). +`backendChanged`; everything else is `error`) and the `watcher`, which sends the cache's `watch` and `unwatch` requests +(`ui::watcherOver` builds one over a remote source's exchange; empty for a local source). + ## The app root scope @@ -148,6 +150,21 @@ canonical body is the body's JSON with record fields in name order. - A `resync` marks every entry but an `evaluate` one. Another topic, or a payload that cannot be read, is ignored. - A marked entry that no query uses is only marked, and fetches when a query mounts on it. +- **Watching what the cache holds** (spec 8 §7). The session counts, per topic, the entries that hold + it: a record name for every record an entry's action returns, and `(model type, primary key)` for an + entry read through a keyed shared alias whose action returns no record. A private instance needs no + watch, because the server knows its owner. A count that leaves zero queues a `watch`, and one that + returns to zero (the entry evicted, or the cache cleared) queues an `unwatch`. + - **Watch before fetch.** `fetch` sends the queued changes first, on the same connection, so the + server holds the watch before the read. A change committed before the read is in its value, and + one committed after arrives as an event, which a mark during the fetch keeps. + - The changes queued before one fetch go out as at most one `watch` and one `unwatch`. A screen + whose queries hold different records sends one small `watch` per fetch. + - Without a `watcher` the session still counts, so `Session::watchSet()` answers, and sends nothing. + - **`Session::reconnected()`**, and every `resync`, sends the whole set with `replace: true`, which + supersedes anything queued, then marks every entry but an `evaluate` one stale. A mounted entry + refetches, and an entry whose fetch is in flight fetches once more when it settles: that fetch + may have been filed before the connection came back. - **Invalidation** (`invalidates`, `refreshOn`, `refetch`, a backend switch) marks the entry stale and refetches it. - **A mark during a fetch is kept.** `Session::markStale` marks an entry stale. While the entry's fetch is diff --git a/docs/spec/ui/service.md b/docs/spec/ui/service.md index d114ee635..28f95a4b4 100644 --- a/docs/spec/ui/service.md +++ b/docs/spec/ui/service.md @@ -268,6 +268,13 @@ the action's body with its computed fields filled and its field errors, without members, with the field errors under `errors` and the verdict under `ready`, so `errors.` attaches each error to its element. An action field named `errors` or `ready` is shadowed by them. +`watcherOver(exchange, currentSession, cbExec)` is the client's route for the cache's `watch` and +`unwatch` (spec 8 §7), which `RemoteServer` handles itself rather than `UiService`. Each request goes +through the exchange with the session `currentSession` returns when it is sent: the exchange does not +stamp one, and the server refuses an unsigned `watch`. An `ok` resolves with its body (the topics the +server refused); a transport failure or an `err` rejects with its reason. A remote source builds it +beside its evaluator and hands it to the shell as `SessionOptions::watcher`. + ## Files `ui-file` is the file side channel of `files/1` (spec 6 §18). A server serves it when its `UiService` is built with diff --git a/include/morph/ui/interpreter.hpp b/include/morph/ui/interpreter.hpp index 30dc3fc9a..cdd1d862f 100644 --- a/include/morph/ui/interpreter.hpp +++ b/include/morph/ui/interpreter.hpp @@ -368,6 +368,10 @@ class InterpreterHost { // Session // --------------------------------------------------------------------------------------------------------------- +/// @brief Sends a `watch` or `unwatch` request for a session's cache (spec 8 §7): the kind, and the `WatchBody` as +/// JSON. `ui::watcherOver` builds one over a remote source's exchange. +using Watcher = std::function(std::string const& kind, std::string const& bodyJson)>; + /// @brief What a `Session` is built with. struct SessionOptions { /// @brief Timers for `debounce`, `refreshEvery` and the cache's eviction. Without one, `debounce` fetches at once, @@ -385,6 +389,9 @@ struct SessionOptions { std::function classify; /// @brief Development mode: whether this session's screens record what the inspector reads. Off by default. DevelopmentTools developmentTools = DevelopmentTools::Off; + /// @brief Sends the cache's watch changes to the server. Empty: the session still counts what its cache holds, + /// and sends nothing, which is right for a local source, whose replies carry every change it sees. + Watcher watcher; }; namespace detail { @@ -910,6 +917,7 @@ class Session { static_cast(key); ++entry->generation; entry->evict.cancel(); + countTopics(*entry, -1); } _entries.clear(); for (detail::QueryRuntime* const query : _mounted) { @@ -1016,6 +1024,7 @@ class Session { } } } + countTopics(*entry, +1); found = _entries.emplace(std::move(key), std::move(entry)).first; } ++found->second->users; @@ -1033,7 +1042,9 @@ class Session { entry->evict = _options.scheduler->after(_options.keepUnused, [this, weak] { auto const held = weak.lock(); if (held && held->users == 0) { + countTopics(*held, -1); _entries.erase(held->key); + flushWatches(); } }); } @@ -1042,6 +1053,9 @@ class Session { /// @param entry The entry. /// @param channel The channel to send it through; `nullptr` fails the request at once. void fetch(std::shared_ptr const& entry, std::shared_ptr const& channel) { + // The watch goes out before the read, on the same connection, so a change committed after the read is + // seen as an event. + flushWatches(); std::uint64_t const generation = ++entry->generation; if (channel == nullptr) { settleFailure(*entry, "the model alias has no instance", "error"); @@ -1132,9 +1146,21 @@ class Session { void actionSucceeded(std::string const& typeId, std::string const& actionId, Value const& reply, ReplyUse use, std::string const& instanceToken); + /// @brief The connection to the server was made again (spec 8 §7, "Reconnect"): sends the whole watch set with + /// `replace: true`, then marks every entry but an `evaluate` one stale. A mounted entry refetches, and one + /// whose fetch is in flight fetches once more when that fetch settles, so a change missed while the + /// connection was down is read again. + void reconnected(); + + /// @brief The topics the cache holds now: every record an entry holds, and the keyed shared instance of every + /// entry whose action holds no record. What `reconnected` sends. + /// @return The topics, records and instances each in name order; `replace` is false. + [[nodiscard]] wire::WatchBody watchSet() const; + /// @brief Applies a server event to the cache (spec 8 §9). /// - /// A `changes` event marks the entries it names stale, and a `resync` marks every entry but an `evaluate` one; + /// A `changes` event marks the entries it names stale, and a `resync` sends the whole watch set again, as + /// `reconnected` does, and marks every entry but an `evaluate` one; /// each marked entry refetches through a mounted query that uses it, and one no query uses refetches when a /// query mounts on it. An event of another topic, or a payload that cannot be read, is ignored. /// @param event The event. @@ -1195,6 +1221,24 @@ class Session { void applyReplyEffects(std::string const& typeId, std::string const& actionId, Value const& reply, ReplyUse use, std::string const& instanceToken); + /// @brief Counts an entry's topics in or out, queueing a `watch` for a count that leaves zero and an `unwatch` + /// for one that returns to it. + /// @param entry The entry. + /// @param delta `+1` when the entry is made, `-1` when it is dropped. + void countTopics(detail::CacheEntry const& entry, int delta); + + /// @brief Sends the queued watch changes, at most one `watch` and one `unwatch`; drops them when there is no + /// watcher. + void flushWatches(); + + /// @brief Sends one watch request, ignoring its reply: a refused topic simply sends no events. + /// @param kind The kind, `watch` or `unwatch`. + /// @param body The body. + void sendWatch(std::string_view kind, wire::WatchBody const& body); + + /// @brief Marks every entry but an `evaluate` one stale and refetches the mounted ones, in one batch. + void markAllStale(); + void settleFailure(detail::CacheEntry& entry, std::string text, std::string kind) { _runtime->batch([&] { noteWrite(); @@ -1209,6 +1253,12 @@ class Session { reactive::Runtime* _runtime; SessionOptions _options; reactive::Signal>> _permitted; + std::map> _recordRefs; + std::map, int> _instanceRefs; + std::set> _watchRecords; + std::set> _unwatchRecords; + std::set> _watchInstances; + std::set> _unwatchInstances; reactive::Signal> _format; reactive::Signal _env; reactive::Signal _sessionValues; @@ -2801,21 +2851,114 @@ inline void Session::applyReplyEffects(std::string const& typeId, std::string co } } -inline void Session::deliverEvent(wire::EventBody const& event) { - if (event.topic == wire::kResyncTopic) { - _runtime->batch([this] { - std::vector marked; - for (auto const& [key, entry] : _entries) { - static_cast(key); - if (!entry->evaluate) { - markStale(*entry); - marked.push_back(entry.get()); - } +inline void Session::countTopics(detail::CacheEntry const& entry, int delta) { + if (entry.evaluate) { + return; + } + // One topic: its count, and the queues a change of the count from or to zero goes to. + auto const count = [delta](auto& refs, auto const& topic, auto& watches, auto& unwatches) { + int& refsOf = refs[topic]; + refsOf += delta; + if (delta > 0 && refsOf == 1 && unwatches.erase(topic) == 0) { + watches.insert(topic); + } else if (refsOf <= 0) { + refs.erase(topic); + if (watches.erase(topic) == 0) { + unwatches.insert(topic); } - for (detail::CacheEntry* const entry : marked) { - refetchMounted(*entry); + } + }; + if (!entry.records.empty()) { + for (ActionRecordInfo const& record : entry.records) { + count(_recordRefs, record.record, _watchRecords, _unwatchRecords); + } + return; + } + constexpr std::string_view kShared = "shared:"; + if (entry.instanceToken.starts_with(kShared)) { + count(_instanceRefs, std::pair{entry.typeId, entry.instanceToken.substr(kShared.size())}, _watchInstances, + _unwatchInstances); + } +} + +inline void Session::sendWatch(std::string_view kind, wire::WatchBody const& body) { + auto completion = _options.watcher(std::string{kind}, glz::write_json(body).value_or(std::string{})); + completion.then([](std::string const&) {}).onError([](std::exception_ptr const&) {}); +} + +inline void Session::flushWatches() { + if (!_options.watcher) { + _watchRecords.clear(); + _unwatchRecords.clear(); + _watchInstances.clear(); + _unwatchInstances.clear(); + return; + } + auto const take = [](auto& records, auto& instances) { + wire::WatchBody body; + body.records.assign(records.begin(), records.end()); + for (auto const& [modelType, primary] : instances) { + body.instances.push_back(wire::InstanceTopic{.modelType = modelType, .primary = primary, .instance = 0}); + } + records.clear(); + instances.clear(); + return body; + }; + if (!_watchRecords.empty() || !_watchInstances.empty()) { + sendWatch(wire::kWatchKind, take(_watchRecords, _watchInstances)); + } + if (!_unwatchRecords.empty() || !_unwatchInstances.empty()) { + sendWatch(wire::kUnwatchKind, take(_unwatchRecords, _unwatchInstances)); + } +} + +inline wire::WatchBody Session::watchSet() const { + wire::WatchBody body; + for (auto const& [record, refs] : _recordRefs) { + static_cast(refs); + body.records.push_back(record); + } + for (auto const& [topic, refs] : _instanceRefs) { + static_cast(refs); + body.instances.push_back( + wire::InstanceTopic{.modelType = topic.first, .primary = topic.second, .instance = 0}); + } + return body; +} + +inline void Session::markAllStale() { + _runtime->batch([this] { + std::vector marked; + for (auto const& [key, entry] : _entries) { + static_cast(key); + if (!entry->evaluate) { + markStale(*entry); + marked.push_back(entry.get()); } - }); + } + for (detail::CacheEntry* const entry : marked) { + refetchMounted(*entry); + } + }); +} + +inline void Session::reconnected() { + // The whole set replaces what the server holds, so the queued changes are superseded. + _watchRecords.clear(); + _unwatchRecords.clear(); + _watchInstances.clear(); + _unwatchInstances.clear(); + if (_options.watcher) { + wire::WatchBody body = watchSet(); + body.replace = true; + sendWatch(wire::kWatchKind, body); + } + markAllStale(); +} + +inline void Session::deliverEvent(wire::EventBody const& event) { + if (event.topic == wire::kResyncTopic) { + reconnected(); return; } if (event.topic != wire::kChangesTopic) { diff --git a/include/morph/ui/service_client.hpp b/include/morph/ui/service_client.hpp index 04424c23f..e9876c6b3 100644 --- a/include/morph/ui/service_client.hpp +++ b/include/morph/ui/service_client.hpp @@ -665,6 +665,41 @@ evaluatorOver(Exchange exchange, ::morph::session::Context session, ::morph::exe }; } +/// @brief A watcher over the server (spec 8 §7): each `watch` or `unwatch` is one request of that kind through +/// @p exchange, carrying the session @p currentSession returns when it is sent. What `SessionOptions::watcher` +/// takes. +/// +/// The session is read at each send, not fixed when the watcher is built, because a `watch` must carry whoever +/// is signed in now: the exchange does not stamp it, and the server refuses an unsigned one. +/// @param exchange The transport. +/// @param currentSession The session to stamp on each request, read when it is sent. +/// @param cbExec The executor the completion is delivered on: the interpreter runtime's owner. Borrowed. +/// @return The watcher: the kind and the `WatchBody` JSON in; the reply body out, or the transport's or the +/// server's reason as the failure. +[[nodiscard]] inline std::function<::morph::async::Completion(std::string const&, std::string const&)> +watcherOver(Exchange exchange, std::function<::morph::session::Context()> currentSession, + ::morph::exec::IExecutor* cbExec) { + return [exchange = std::move(exchange), currentSession = std::move(currentSession), cbExec]( + std::string const& kind, std::string const& bodyJson) { + auto [completion, promise] = ::morph::async::Completion::makeSettleable(cbExec); + auto settle = std::make_shared(std::move(promise)); + ::morph::wire::Envelope request; + request.kind = kind; + request.session = currentSession ? currentSession() : ::morph::session::Context{}; + request.body = bodyJson; + exchange(std::move(request), [settle](ExchangeReply reply) { + if (!reply) { + settle->reject(std::make_exception_ptr(std::runtime_error{reply.error()})); + } else if (reply->kind != "ok") { + settle->reject(std::make_exception_ptr(std::runtime_error{reply->message})); + } else { + settle->resolve(std::move(reply->body)); + } + }); + return std::move(completion); + }; +} + /// @brief A file a download brought back. struct DownloadedFile { /// @brief Its bytes. diff --git a/tests/test_push_cache.cpp b/tests/test_push_cache.cpp index 5bfebac7d..c3354c1fc 100644 --- a/tests/test_push_cache.cpp +++ b/tests/test_push_cache.cpp @@ -6,6 +6,7 @@ #include #include #include +#include #include #include @@ -299,3 +300,137 @@ TEST_CASE("instance_event_refetches_reads_of_that_instance", "[push][cache]") { seam.changes(h, R"({"instances":[{"modelType":"Lims_SampleModel","primary":""}]})"); CHECK(h.dispatcher.count("Push_Rows") == 2); } + +namespace { + +// Two queries over different records: `rows` holds `T_Row`, `head` holds `T_Row` at `header.rows`. +constexpr char const* kSameRecordTwice = R"({ + "vocab": "ui/1", "id": "same", + "models": { "sample": { "type": "Lims_SampleModel" } }, + "queries": { "rows": { "model": "sample", "action": "Push_ListRows" }, + "head": { "model": "sample", "action": "Push_Header" } }, + "view": { "kind": "text", "text": { "fn": "format", "args": [ "{0} {1}", { "query": "rows" }, { "query": "head" } ] } } })"; + +} // namespace + +TEST_CASE("watch_sent_before_first_fetch", "[push][cache]") { + Harness h{true}; + auto screen = h.mount(kRows); + REQUIRE(h.dispatcher.count("Push_ListRows") == 1); + REQUIRE(h.watches.size() == 1); + CHECK(h.watches.front().kind == "watch"); + CHECK(h.watches.front().body == R"({"records":["T_Row"],"instances":[],"replace":false})"); + CHECK(h.watches.front().callsBefore == 0); +} + +TEST_CASE("one_watch_message_per_batch", "[push][cache]") { + Harness h{true}; + // Two entries holding one record are one topic, watched once. + auto screen = h.mount(kSameRecordTwice); + CHECK(h.dispatcher.calls.size() == 2); + REQUIRE(h.watches.size() == 1); + CHECK(h.watches.front().body == R"({"records":["T_Row"],"instances":[],"replace":false})"); + // A read whose action holds no record, through a private instance, needs no watch. + auto other = h.mount(kList); + CHECK(h.watches.size() == 1); +} + +TEST_CASE("unwatch_after_eviction", "[push][cache]") { + Harness h{true}; + auto screen = h.mount(kRows); + h.reply(0, kTwoRows); + screen.reset(); + h.settle(); + CHECK(h.watches.size() == 1); // still kept, so still watched + h.advance(std::chrono::minutes{5}); + REQUIRE(h.watches.size() == 2); + CHECK(h.watches.back().kind == "unwatch"); + CHECK(h.watches.back().body == R"({"records":["T_Row"],"instances":[],"replace":false})"); + CHECK(h.session.watchSet().records.empty()); +} + +TEST_CASE("no_watcher_counts_and_sends_nothing", "[push][cache]") { + Harness h; + auto screen = h.mount(kSameRecordTwice); + CHECK(h.session.watchSet().records == std::vector{"T_Row"}); + CHECK(h.watches.empty()); +} + +TEST_CASE("reconnected_sends_replace_and_marks_stale", "[push][cache]") { + Harness h{true}; + auto screen = h.mount(kRows); + h.reply(0, kTwoRows); + std::size_t const calls = h.dispatcher.calls.size(); + h.session.reconnected(); + h.settle(); + REQUIRE(h.watches.size() == 2); + CHECK(h.watches.back().body == R"({"records":["T_Row"],"instances":[],"replace":true})"); + CHECK(h.watches.back().callsBefore == calls); + CHECK(h.dispatcher.count("Push_ListRows") == 2); +} + +TEST_CASE("fetch_filed_before_reconnected_refetches_once", "[push][cache]") { + Harness h{true}; + auto screen = h.mount(kRows); + REQUIRE(h.dispatcher.count("Push_ListRows") == 1); + // The first fetch was filed before the connection came back, so it may have read before a missed change. + h.session.reconnected(); + h.settle(); + CHECK(h.dispatcher.count("Push_ListRows") == 1); + h.reply(0, kTwoRows); + REQUIRE(h.dispatcher.count("Push_ListRows") == 2); + h.reply(1, kTwoRows); + CHECK(h.dispatcher.count("Push_ListRows") == 2); +} + +TEST_CASE("resync_sends_the_whole_watch_set", "[push][cache]") { + Harness h{true}; + EventSeam seam{h}; + auto screen = h.mount(kRows); + h.reply(0, kTwoRows); + seam.backend.deliverEvent({.topic = std::string{morph::wire::kResyncTopic}, .payload = {"null"}}); + h.settle(); + REQUIRE(h.watches.size() == 2); + CHECK(h.watches.back().body == R"({"records":["T_Row"],"instances":[],"replace":true})"); +} + +TEST_CASE("watcher_over_stamps_the_current_session_and_reports_refusals", "[push][cache]") { + morph::testing::StepExecutor owner; + std::vector sent; + bool refuse = false; + morph::ui::Exchange const exchange = [&](morph::wire::Envelope request, + std::function const& done) { + sent.push_back(request); + morph::wire::Envelope reply; + reply.kind = refuse ? "err" : "ok"; + reply.message = refuse ? "unauthorized" : ""; + reply.body = refuse ? "" : R"({"refusedRecords":[],"refusedInstances":[]})"; + done(std::move(reply)); + }; + std::string principal = "ann"; + auto const watcher = + morph::ui::watcherOver(exchange, [&] { return morph::session::Context{.principal = principal}; }, &owner); + + std::optional body; + watcher("watch", R"({"records":["T_Row"]})").then([&](std::string const& reply) { body = reply; }); + principal = "bob"; + std::optional refusal; + refuse = true; + watcher("unwatch", R"({"records":["T_Row"]})").onError([&](std::exception_ptr const& error) { + try { + std::rethrow_exception(error); + } catch (std::exception const& thrown) { + refusal = thrown.what(); + } + }); + owner.runAll(); + + REQUIRE(sent.size() == 2); + CHECK(sent.at(0).kind == "watch"); + CHECK(sent.at(0).session.principal == "ann"); + CHECK(sent.at(0).body == R"({"records":["T_Row"]})"); + CHECK(sent.at(1).kind == "unwatch"); + CHECK(sent.at(1).session.principal == "bob"); + CHECK(body == R"({"refusedRecords":[],"refusedInstances":[]})"); + CHECK(refusal == "unauthorized"); +} diff --git a/tests/ui_interpreter_support.hpp b/tests/ui_interpreter_support.hpp index efade8eeb..4ca3cf912 100644 --- a/tests/ui_interpreter_support.hpp +++ b/tests/ui_interpreter_support.hpp @@ -381,7 +381,21 @@ struct Harness { /// @param mode Whether the session's screens record what the inspector reads. explicit Harness(ui::DevelopmentTools mode = ui::DevelopmentTools::On) : tools{mode} {} + /// @brief One watch request the session sent: its kind, its body, and how many calls the dispatcher had then. + struct WatchSent { + std::string kind; + std::string body; + std::size_t callsBefore = 0; + }; + + /// @brief A harness whose session sends its watch changes to `watches`, each answered at once. + /// @param recordWatches Always true; selects this constructor. + explicit Harness(bool recordWatches) : tools{ui::DevelopmentTools::On}, recording{recordWatches} {} + ui::DevelopmentTools tools; + bool recording = false; + /// @brief The watch requests sent, oldest first; only with `recordWatches`. + std::vector watches; morph::testing::StepExecutor owner; reactive::Runtime runtime{owner}; reactive::testing::ManualScheduler scheduler; @@ -398,7 +412,8 @@ struct Harness { return "error"; } }, - .developmentTools = tools}}; + .developmentTools = tools, + .watcher = recordingWatcher()}}; ScriptedDispatcher dispatcher{owner}; RecordingHost host; ui::testing::RecordingBackend backend; @@ -426,6 +441,20 @@ struct Harness { /// @brief Runs everything posted to the owner. void settle() { owner.runAll(); } + /// @brief The watcher of a recording harness; empty otherwise. + /// @return It. + ui::Watcher recordingWatcher() { + if (!recording) { + return {}; + } + return [this](std::string const& kind, std::string const& body) { + watches.push_back(WatchSent{.kind = kind, .body = body, .callsBefore = dispatcher.calls.size()}); + auto [completion, promise] = async::Completion::makeSettleable(&owner); + promise.resolve(std::string{"{}"}); + return std::move(completion); + }; + } + /// @brief Advances the clock, then settles. /// @param delta How far. void advance(std::chrono::milliseconds delta) { From 72f8294881c0ba4ee38b847f5d5bc95d6f707d6f Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 23:45:01 +0200 Subject: [PATCH 45/68] ui, examples: the generic client re-negotiates and re-watches on every reconnect AppSource::onConnected adds a listener called on the owner after each connect; its default does nothing. The generic client's source owns the backend's connect hook and event handler: on a connect after the first bundle loaded it sends ui-hello again, so the new connection negotiates events, then calls its listeners. It hands out watcher(), watcherOver its exchange for a server and empty in process, and forwards events. The shell builds its session with that watcher, registers Session::reconnected and passes each event to Session::deliverEvent. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- docs/spec/ui/frontend.md | 10 ++ examples/common/app/app_source.cpp | 52 ++++++++++- examples/common/app/app_source.hpp | 26 ++++++ examples/common/app/client_shell.cpp | 18 ++++ examples/common/app/tests/test_sources.cpp | 102 +++++++++++++++++++++ include/morph/ui/frontend.hpp | 9 ++ 6 files changed, 216 insertions(+), 1 deletion(-) diff --git a/docs/spec/ui/frontend.md b/docs/spec/ui/frontend.md index a857684a1..a6de8a3de 100644 --- a/docs/spec/ui/frontend.md +++ b/docs/spec/ui/frontend.md @@ -58,9 +58,19 @@ Where the screens and the models are. `main` builds one from its arguments; the | `open(ctx, ready)` | Connects (or sets up the local models), signs the user in natively when the application needs it — on the owner, before the catalog is requested — and calls `ready` once, on the owner, now or later, with a `Bundle` or a `ConnectError` | | `switchBackend(backend, done)` | Replaces the dispatch target under the mounted screens: `Bridge::switchBackend` posted to the owner. `done` is called once, on the owner, when the switch committed, or with why not; a failed switch leaves the old backend | | `close()` | Releases what `open` made — connections, bridges, handlers — and drops a `ready` not yet delivered. `runApp` calls it once the screens are gone and before the runtime is; the source may be opened again | +| `onConnected(listener)` | Adds a listener called on the owner after each successful connect, the first included. The default does nothing: a local source has no connection to lose. A remote source owns the backend's single connect hook, so anything that wants connects (a shell's `Session::reconnected`, a connection indicator) subscribes here | Every call is made on the context's owner, and every callback is called there. +**Reconnect** (server push, spec 8 §7). The generic client's remote source installs the backend's connect hook and +its event handler, and posts both to the owner. On a connect after the first bundle loaded, a new connection, it +sends `ui-hello` again (the handshake only, not a reload: the documents did not change because the connection +dropped), so the connection negotiates `events`, then calls its `onConnected` listeners. The first connect's +`ui-hello` is the one `open` filed. The shell builds its `Session` with `SessionOptions::watcher = source.watcher()` +(`watcherOver` the source's exchange; empty in process), registers `Session::reconnected` with `onConnected`, and +hands each event to `Session::deliverEvent`. The watch set and the refetches it causes are filed after the +`ui-hello`, on the same ordered connection. + - `ui::Bundle` carries the application's id, the manifest's SHA-256 digest, and every document — the screens, the app shell, and the app document (`app`) when there is one — by id, as verified JSON text, and in `themes` the theme index and token files by id ([`theme.md`](theme.md#surfaces)). The document model and diff --git a/examples/common/app/app_source.cpp b/examples/common/app/app_source.cpp index 37c3100e7..693ab4de9 100644 --- a/examples/common/app/app_source.cpp +++ b/examples/common/app/app_source.cpp @@ -291,6 +291,25 @@ void ClientSource::open(ui::AppContext& context, return; } exec::IExecutor& owner = context.executor(); + _serverEvents = transport->serverEvents; + // The source owns the backend's connect hook and its event handler; both reach the owner before anything else + // sees them, and nothing after `close`. + transport->backend->setEventHandler( + [this, alive](wire::EventBody const& event) { + if (*alive) { + for (auto const& listener : _eventListeners) { + listener(event); + } + } + }, + &owner); + transport->backend->setConnectHandler([this, alive, &owner] { + owner.post([this, alive] { + if (*alive) { + connected(); + } + }); + }); _bridge = std::make_unique(std::move(transport->backend), owner); _bridge->setDefaultSession(_session); // A reply already queued when the source closes still reaches the exchange's continuation; from then on it sends @@ -413,6 +432,37 @@ void ClientSource::signIn(std::string principal, std::string token, std::functio void ClientSource::signOut(std::function done) { signIn(signedOutPrincipal(), {}, std::move(done)); } +void ClientSource::connected() { + // The first connect's `ui-hello` is the one `open` filed, and its answer loads the first bundle; every later + // connect is a new connection, which negotiates events again. Only the handshake: the documents did not change + // because the connection dropped. + if (_loaded && _exchange) { + wire::Envelope hello; + hello.kind = std::string{ui::kHelloKind}; + hello.session = _session; + hello.body = ui::detail::writeMessage(clientHello()); + _exchange(std::move(hello), [](ui::ExchangeReply const&) {}); + } + std::vector> const listeners = _connectListeners; + for (auto const& listener : listeners) { + listener(); + } +} + +void ClientSource::onConnected(std::function listener) { _connectListeners.push_back(std::move(listener)); } + +void ClientSource::onEvent(std::function listener) { + _eventListeners.push_back(std::move(listener)); +} + +ui::Watcher ClientSource::watcher() { + if (!_serverEvents || !_exchange || _bridge == nullptr) { + return {}; + } + // The session is read at each send, so a watch after a sign-in carries the new principal. + return ui::watcherOver(_exchange, [this] { return _session; }, &_bridge->owner()); +} + void ClientSource::switchBackend(ui::Backend backend, std::function)> done) { done(std::unexpected("this client does not switch backends; the switch to `" + backend.name + "` is the hosting layer's")); @@ -451,7 +501,7 @@ std::expected RemoteAppSource::connec .message = "cannot connect to " + _url + ": " + made.error()}); } ui::Exchange exchange = ui::exchangeOver(*made->requests, context.executor()); - return Transport{.backend = std::move(made->backend), .exchange = std::move(exchange)}; + return Transport{.backend = std::move(made->backend), .exchange = std::move(exchange), .serverEvents = true}; } // ── LocalAppSource ────────────────────────────────────────────────────────── diff --git a/examples/common/app/app_source.hpp b/examples/common/app/app_source.hpp index df033ae20..6acd1f512 100644 --- a/examples/common/app/app_source.hpp +++ b/examples/common/app/app_source.hpp @@ -167,6 +167,22 @@ class ClientSource : public ui::AppSource { /// changed; not called for an unchanged digest, a failed poll, or after `close`. void poll(std::function changed)> changed); + /// @brief Adds a listener called on the owner after each successful connect, the first included. On a reconnect + /// the source first sends `ui-hello` again, so the new connection negotiates events, then calls the + /// listeners. + /// @param listener Called on the owner; it must stay callable while the source is open. + void onConnected(std::function listener) final; + + /// @brief Adds a listener for the events the server sends: what a shell hands to `Session::deliverEvent`. + /// @param listener Called on the owner, once per event, in the order they arrived. + void onEvent(std::function listener); + + /// @brief What the session's `SessionOptions::watcher` sends the cache's watch changes through: `watcherOver` the + /// source's exchange, stamping the session current at each send. Empty for a source whose transport + /// carries no server events, such as an in-process one. + /// @return The watcher, or empty. + [[nodiscard]] ui::Watcher watcher(); + /// @brief The principal before any sign-in, which `signOut` returns to. /// @return It; empty for none. [[nodiscard]] virtual std::string signedOutPrincipal() const { return {}; } @@ -178,6 +194,9 @@ class ClientSource : public ui::AppSource { std::unique_ptr backend; /// @brief The exchange; it may borrow the backend, which the source guards. ui::Exchange exchange; + /// @brief Whether the server behind the transport sends events and handles `watch`: true for a server, false + /// in process. + bool serverEvents = false; }; /// @brief Makes the transport on @p context. @@ -213,6 +232,13 @@ class ClientSource : public ui::AppSource { // Retries the queue while it holds anything: the transports reconnect on their own and tell no one. std::optional _replay; ui::Exchange _exchange; + // Whether the open transport's server sends events and handles `watch`. + bool _serverEvents = false; + // Called after each connect, and with each event. + std::vector> _connectListeners; + std::vector> _eventListeners; + // A connect, on the owner: `ui-hello` again once the first bundle loaded, then the listeners. + void connected(); // Fetches the schemas of every model type the catalog names, then calls @p done; nothing after `close`. void loadSchemas(std::shared_ptr const& alive, std::function done); // The model types the loaded catalog names: its models, and the model of each action it has facts for. diff --git a/examples/common/app/client_shell.cpp b/examples/common/app/client_shell.cpp index 4350ef75b..902e96e86 100644 --- a/examples/common/app/client_shell.cpp +++ b/examples/common/app/client_shell.cpp @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -208,7 +209,22 @@ class Shell final : public ui::MountedShell { sessionOptions.scheduler = &_context->scheduler(); sessionOptions.catalog = source.catalog(); sessionOptions.developmentTools = _options.dev ? ui::DevelopmentTools::On : ui::DevelopmentTools::Off; + sessionOptions.watcher = source.watcher(); _session = std::make_unique(_context->runtime(), std::move(sessionOptions)); + // Registered once: they reach whichever session is current, and nothing once the shell is gone. + if (!_listening) { + _listening = true; + source.onConnected([this, alive = std::weak_ptr{_alive}] { + if (auto const live = alive.lock(); live != nullptr && *live && _session) { + _session->reconnected(); + } + }); + source.onEvent([this, alive = std::weak_ptr{_alive}](wire::EventBody const& event) { + if (auto const live = alive.lock(); live != nullptr && *live && _session) { + _session->deliverEvent(event); + } + }); + } showPermitted(); ui::DocumentScreenHostOptions hostOptions; hostOptions.documents = [this](std::string_view screenId) -> std::shared_ptr { @@ -632,6 +648,8 @@ class Shell final : public ui::MountedShell { std::map, std::less<>> _generated; ui::ScreenHostRegistry _hosts; std::unique_ptr _session; + // Whether the source's connect and event listeners are registered. + bool _listening = false; std::unique_ptr _application; std::unique_ptr _host; std::unique_ptr _frame; diff --git a/examples/common/app/tests/test_sources.cpp b/examples/common/app/tests/test_sources.cpp index 5017b067e..9c2640423 100644 --- a/examples/common/app/tests/test_sources.cpp +++ b/examples/common/app/tests/test_sources.cpp @@ -471,3 +471,105 @@ TEST_CASE("Client shell: the frame's navigator opens a screen as a tab", "[examp } source.close(); } + +namespace { + +// A local backend whose connect hook the test fires, as a transport does after each connect. +class ConnectingBackend final : public morph::backend::LocalBackend { +public: + using LocalBackend::LocalBackend; + void setConnectHandler(std::function const& handler) override { *_handler = handler; } + [[nodiscard]] std::shared_ptr> handler() const { return _handler; } + +private: + std::shared_ptr> _handler = std::make_shared>(); +}; + +// The probe application behind a server-like transport: the UI service answers the `ui-*` kinds in process, the +// exchange answers `watch` itself, as `RemoteServer` does, and every request's kind is logged in order. +class ReconnectingSource final : public morph::examples::ClientSource { +public: + // NOLINTBEGIN(cppcoreguidelines-non-private-member-variables-in-classes): the test fires and inspects these. + std::vector kinds; + std::vector watchBodies; + std::shared_ptr> connectHook; + ConnectingBackend* backend = nullptr; + // NOLINTEND(cppcoreguidelines-non-private-member-variables-in-classes) + + ReconnectingSource() = default; + ~ReconnectingSource() override { close(); } + ReconnectingSource(ReconnectingSource const&) = delete; + ReconnectingSource& operator=(ReconnectingSource const&) = delete; + ReconnectingSource(ReconnectingSource&&) = delete; + ReconnectingSource& operator=(ReconnectingSource&&) = delete; + +protected: + [[nodiscard]] std::expected connect( + morph::ui::AppContext& /*context*/) override { + auto served = probe::bundle(); + REQUIRE(served.has_value()); + _service = + std::make_shared(std::move(*served), morph::model::detail::defaultDispatcher()); + _workers = std::make_unique(1); + auto made = std::make_unique(*_workers); + backend = made.get(); + connectHook = made->handler(); + return Transport{ + .backend = std::move(made), + .exchange = + [this](morph::wire::Envelope const& request, + std::function const& done) { + kinds.push_back(request.kind); + if (request.kind == "watch" || request.kind == "unwatch") { + watchBodies.push_back(request.body); + morph::wire::Envelope ok; + ok.kind = "ok"; + ok.body = "{}"; + done(ok); + return; + } + done(morph::wire::decode(_service->handle(request, *morph::session::allowAllAuthorizer()))); + }, + .serverEvents = true}; + } + void disconnect() override { + _workers.reset(); + _service.reset(); + } + +private: + std::shared_ptr _service; + std::unique_ptr _workers; +}; + +} // namespace + +// Mutation: call the listeners before the ui-hello, or forward no events — the order or the last watch changes. +TEST_CASE("remote_source_rehellos_then_rewatches_on_reconnect", "[examples-common][source][push]") { + FakeAppContext context; + morph::ui::testing::RecordingBackend screens; + ReconnectingSource source; + ShellLog log; + { + auto shell = + morph::examples::mountClientShell(context, screens, openAndWait(context, source), source, log.options()); + REQUIRE(log.mounted == std::vector{"probe.home"}); + REQUIRE(source.connectHook != nullptr); + + // A connect once the bundle loaded is a new connection: it negotiates events again, then the session + // re-sends its whole watch set. + source.kinds.clear(); + (*source.connectHook)(); + REQUIRE(pumpUntil(context.owner(), [&] { return source.kinds.size() >= 2; })); + CHECK(source.kinds == std::vector{"ui-hello", "watch"}); + CHECK(source.watchBodies.back() == R"({"records":[],"instances":[],"replace":true})"); + + // A server event reaches the session: a resync answers with the whole watch set. + source.kinds.clear(); + source.backend->deliverEvent({.topic = "resync", .payload = {"null"}}); + REQUIRE(pumpUntil(context.owner(), [&] { return !source.kinds.empty(); })); + CHECK(source.kinds == std::vector{"watch"}); + CHECK(log.problems.empty()); + } + source.close(); +} diff --git a/include/morph/ui/frontend.hpp b/include/morph/ui/frontend.hpp index d201e2832..45984aa79 100644 --- a/include/morph/ui/frontend.hpp +++ b/include/morph/ui/frontend.hpp @@ -158,6 +158,15 @@ class AppSource { /// delivered. Called by `runApp` once the screens are gone and before the frontend's runtime is; the /// source may be opened again afterwards. virtual void close() = 0; + + /// @brief Adds a listener called on the owner after each successful connect to the server, the first included + /// (spec 8 §7, "Reconnect"). A shell registers `Session::reconnected` here, so the session re-sends its + /// watch set and re-reads what it shows after an outage. The source owns the backend's single connect + /// hook, so anything else that wants connects subscribes here too. + /// + /// The default does nothing: a local source has no connection to lose. + /// @param listener Called on the owner after each connect; it must stay callable while the source is open. + virtual void onConnected(std::function listener) { static_cast(listener); } }; /// @brief A frontend: owns an event loop and a renderer, and runs one source's screens. From 2814880f72fb27b224c9119c2a7da145b8dd86ee Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Sun, 18 Oct 2026 23:54:13 +0200 Subject: [PATCH 46/68] core: a record registered without options compiles under GCC The options fold is empty then, so GCC reports the helper lambda as set but not used, which -Werror makes an error. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- include/morph/core/records.hpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/morph/core/records.hpp b/include/morph/core/records.hpp index 58087a978..f75f6423d 100644 --- a/include/morph/core/records.hpp +++ b/include/morph/core/records.hpp @@ -206,7 +206,8 @@ inline bool registerRecordOnce(std::string_view name) noexcept { RecordInfo info{.name = std::string{name}, .type = std::type_index{typeid(T)}, .keyField = {}, .derivedFrom = {}}; applyKey(info); info.privateKeys = (std::is_same_v || ... || false); - auto const addSources = [](RecordInfo& target) { + // Unused when the record takes no options: the fold below is then empty. + [[maybe_unused]] auto const addSources = [](RecordInfo& target) { if constexpr (IsDerivedFrom::value) { [](RecordInfo& out, derivedFrom) { (out.derivedFrom.emplace_back(typeid(S)), ...); }( target, O{}); From e43964c706ee3f2a05d48a9f5540d5ddf4600976 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 00:03:38 +0200 Subject: [PATCH 47/68] ui, qt, tests: pass clang-tidy on the cache's effect and event paths The reply-effect and change-event functions are split into one method per kind of change, around a Marks collector and helpers for a record's rows at its paths, which brings each under the complexity limit. markStale is static. Qt's queued callable is owned by Qt, which the analyzer cannot see. The test leftovers reserve their vectors and drop an unused using. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- include/morph/ui/interpreter.hpp | 277 ++++++++++++++++++------------- src/qt/qt_websocket_server.cpp | 3 + tests/test_push_effects.cpp | 1 - tests/test_push_watch.cpp | 3 + 4 files changed, 166 insertions(+), 118 deletions(-) diff --git a/include/morph/ui/interpreter.hpp b/include/morph/ui/interpreter.hpp index cdd1d862f..db9fccdca 100644 --- a/include/morph/ui/interpreter.hpp +++ b/include/morph/ui/interpreter.hpp @@ -749,6 +749,50 @@ struct CacheEntry { } }; +/// @brief The rows of a keyed record an entry's value holds, at each of the record's paths. +/// @param value The entry's confirmed value. +/// @param held The record, as the entry's action returns it; it must have a key field. +/// @return The rows. +[[nodiscard]] inline std::vector keyedRowsOf(Value const& value, ActionRecordInfo const& held) { + std::vector rows; + for (std::string const& path : held.paths) { + std::vector const members = pathMembers(path); + keyedRowsAt(value, members, *held.keyField, rows); + } + return rows; +} + +/// @brief Merges new rows into an entry's value at each of the record's paths. +/// @param value The entry's confirmed value. +/// @param held The record, as the entry's action returns it; it must have a key field. +/// @param rows The new rows by key. +/// @return The new value, or nothing when no row matched. +[[nodiscard]] inline std::optional patchHeldRows(Value const& value, ActionRecordInfo const& held, + RowsByKey const& rows) { + std::optional patched; + for (std::string const& path : held.paths) { + std::vector const members = pathMembers(path); + if (auto next = patchKeyedRows(patched ? *patched : value, members, *held.keyField, rows)) { + patched = std::move(next); + } + } + return patched; +} + +/// @brief The entries one change marks stale, each once, in the order they were found. +struct Marks { + /// @brief The entries. + std::vector entries; + + /// @brief Adds an entry, unless it is already there. + /// @param entry The entry. + void add(CacheEntry& entry) { + if (std::ranges::find(entries, &entry) == entries.end()) { + entries.push_back(&entry); + } + } +}; + class QueryRuntime; class RtScope; class ScreenCore; @@ -1178,7 +1222,7 @@ class Session { /// @brief Marks an entry stale. While its fetch is in flight, the entry also refetches once that fetch settles, /// because the fetch may have read before the change. /// @param entry The entry. - void markStale(detail::CacheEntry& entry) { + static void markStale(detail::CacheEntry& entry) { entry.stale = true; if (entry.pending.peek()) { entry.refetchAfter = true; @@ -1239,6 +1283,30 @@ class Session { /// @brief Marks every entry but an `evaluate` one stale and refetches the mounted ones, in one batch. void markAllStale(); + /// @brief Marks each entry of @p marks stale and refetches it through a mounted query that uses it. + /// @param marks The entries. + void markAndRefetch(detail::Marks const& marks); + + /// @brief Applies one `updates` or `changes` effect of a reply: patches the entries holding the record, or marks + /// them. + /// @param effect The effect; its record is set. + /// @param info The action's catalog entry, for the records its reply holds. + /// @param reply The reply's value. + /// @param use Whether the reply was applied or discarded. + /// @param marks Where entries to mark go. + void applyRecordEffect(ActionEffect const& effect, ActionInfo const& info, Value const& reply, ReplyUse use, + detail::Marks& marks); + + /// @brief Finds the entries a `changes` event's record change names. + /// @param change The change. + /// @param marks Where they go. + void markRecordChange(effects::RecordChange const& change, detail::Marks& marks); + + /// @brief Finds the entries a `changes` event's instance change names. + /// @param instance The instance. + /// @param marks Where they go. + void markInstanceChange(wire::InstanceTopic const& instance, detail::Marks& marks); + void settleFailure(detail::CacheEntry& entry, std::string text, std::string kind) { _runtime->batch([&] { noteWrite(); @@ -2773,6 +2841,7 @@ inline void Session::refetchMounted(detail::CacheEntry const& entry) { } } +// NOLINTNEXTLINE(bugprone-easily-swappable-parameters): a model type and an action id, as every caller names them. inline void Session::applyReplyEffects(std::string const& typeId, std::string const& actionId, Value const& reply, ReplyUse use, std::string const& instanceToken) { if (!_options.catalog.action) { @@ -2782,71 +2851,60 @@ inline void Session::applyReplyEffects(std::string const& typeId, std::string co if (!info || !info->effects) { return; } - std::vector marked; - auto const mark = [&](detail::CacheEntry& entry) { - markStale(entry); - if (std::ranges::find(marked, &entry) == marked.end()) { - marked.push_back(&entry); - } - }; + detail::Marks marks; for (ActionEffect const& effect : *info->effects) { if (effect.kind == "changesInstance") { for (auto const& [key, entry] : _entries) { static_cast(key); if (!entry->evaluate && entry->typeId == typeId && entry->instanceToken == instanceToken) { - mark(*entry); + marks.add(*entry); } } - continue; + } else if (effect.record && (effect.kind == "updates" || effect.kind == "changes")) { + applyRecordEffect(effect, *info, reply, use, marks); + } + } + markAndRefetch(marks); +} + +inline void Session::applyRecordEffect(ActionEffect const& effect, ActionInfo const& info, Value const& reply, + ReplyUse use, detail::Marks& marks) { + std::string const& record = *effect.record; + // The reply's rows of the record, by key; none when the reply does not hold it keyed, or must only mark. + auto const own = std::ranges::find(info.records, record, &ActionRecordInfo::record); + bool const patch = + effect.kind == "updates" && use == ReplyUse::Applied && own != info.records.end() && own->keyField.has_value(); + detail::RowsByKey rows; + if (patch) { + for (Value const* const row : detail::keyedRowsOf(reply, *own)) { + rows.insert_or_assign(detail::canonicalJson(*row->field(*own->keyField)), row); } - if (!effect.record || (effect.kind != "updates" && effect.kind != "changes")) { + } + for (auto const& [key, entry] : _entries) { + static_cast(key); + ActionRecordInfo const* const held = entry->holding(record); + if (held == nullptr) { continue; } - std::string const& record = *effect.record; - // The reply's rows of the record, by key; none when the reply does not hold it keyed, or must only mark. - std::vector found; - auto const own = std::ranges::find(info->records, record, &ActionRecordInfo::record); - bool const patch = effect.kind == "updates" && use == ReplyUse::Applied && own != info->records.end() && - own->keyField.has_value(); - if (patch) { - for (std::string const& path : own->paths) { - std::vector const members = detail::pathMembers(path); - detail::keyedRowsAt(reply, members, *own->keyField, found); - } + if (!patch || !held->keyField) { + marks.add(*entry); + continue; } - detail::RowsByKey rows; - for (Value const* const row : found) { - rows.insert_or_assign(detail::canonicalJson(*row->field(*own->keyField)), row); + if (auto patched = detail::patchHeldRows(entry->confirmed, *held, rows)) { + confirm(*entry, *patched); } - for (auto const& [key, entry] : _entries) { - static_cast(key); - ActionRecordInfo const* const held = entry->holding(record); - if (held == nullptr) { - continue; - } - if (!patch || !held->keyField) { - mark(*entry); - continue; - } - Value confirmed = entry->confirmed; - bool changed = false; - for (std::string const& path : held->paths) { - std::vector const members = detail::pathMembers(path); - if (auto patched = detail::patchKeyedRows(confirmed, members, *held->keyField, rows)) { - confirmed = std::move(*patched); - changed = true; - } - } - if (changed) { - confirm(*entry, confirmed); - } - // A fetch in flight may have read before the write, and would settle over the patch. - if (entry->pending.peek()) { - markStale(*entry); - } + // A fetch in flight may have read before the write, and would settle over the patch. + if (entry->pending.peek()) { + markStale(*entry); } } - for (detail::CacheEntry* const entry : marked) { +} + +inline void Session::markAndRefetch(detail::Marks const& marks) { + for (detail::CacheEntry* const entry : marks.entries) { + markStale(*entry); + } + for (detail::CacheEntry const* const entry : marks.entries) { refetchMounted(*entry); } } @@ -2928,17 +2986,14 @@ inline wire::WatchBody Session::watchSet() const { inline void Session::markAllStale() { _runtime->batch([this] { - std::vector marked; + detail::Marks marks; for (auto const& [key, entry] : _entries) { static_cast(key); if (!entry->evaluate) { - markStale(*entry); - marked.push_back(entry.get()); + marks.add(*entry); } } - for (detail::CacheEntry* const entry : marked) { - refetchMounted(*entry); - } + markAndRefetch(marks); }); } @@ -2970,73 +3025,61 @@ inline void Session::deliverEvent(wire::EventBody const& event) { } inline void Session::applyChanges(wire::ChangesBody const& changes) { - // A key's canonical text, so a key the server wrote is compared with the rows' own. - auto const keyText = [](glz::raw_json const& key) -> std::optional { - auto json = parseJson(key.str); - auto value = json ? valueFromJson(*json) : std::unexpected(json.error()); - return value ? std::optional{detail::canonicalJson(*value)} : std::nullopt; - }; _runtime->batch([&] { - std::vector marked; - auto const mark = [&](detail::CacheEntry& entry) { - if (std::ranges::find(marked, &entry) == marked.end()) { - markStale(entry); - marked.push_back(&entry); - } - }; + detail::Marks marks; for (effects::RecordChange const& change : changes.records) { - std::set> keys; - if (change.keys) { - for (glz::raw_json const& key : *change.keys) { - if (auto text = keyText(key)) { - keys.insert(std::move(*text)); - } - } - } - for (auto const& [name, entry] : _entries) { - static_cast(name); - ActionRecordInfo const* const held = entry->holding(change.record); - if (held == nullptr) { - continue; - } - if (!change.keys || !held->keyField) { - mark(*entry); - continue; - } - std::vector rows; - for (std::string const& path : held->paths) { - std::vector const members = detail::pathMembers(path); - detail::keyedRowsAt(entry->confirmed, members, *held->keyField, rows); - } - bool const holdsKey = std::ranges::any_of(rows, [&](Value const* row) { - return keys.contains(detail::canonicalJson(*row->field(*held->keyField))); - }); - // An entry still waiting for its first value may come to hold the key. - if (holdsKey || !entry->hasValue.peek()) { - mark(*entry); - } - } + markRecordChange(change, marks); } for (wire::InstanceTopic const& instance : changes.instances) { - std::string const shared = "shared:" + instance.primary; - for (auto const& [name, entry] : _entries) { - static_cast(name); - if (entry->evaluate || entry->typeId != instance.modelType) { - continue; - } - bool const same = instance.primary.empty() ? !entry->instanceToken.starts_with("shared") - : entry->instanceToken == shared; - if (same) { - mark(*entry); - } - } - } - for (detail::CacheEntry* const entry : marked) { - refetchMounted(*entry); + markInstanceChange(instance, marks); } + markAndRefetch(marks); }); } +inline void Session::markRecordChange(effects::RecordChange const& change, detail::Marks& marks) { + // The keys' canonical text, so a key the server wrote is compared with the rows' own. + std::set> keys; + for (glz::raw_json const& key : change.keys.value_or(std::vector{})) { + auto json = parseJson(key.str); + auto value = json ? valueFromJson(*json) : std::unexpected(json.error()); + if (value) { + keys.insert(detail::canonicalJson(*value)); + } + } + for (auto const& [name, entry] : _entries) { + static_cast(name); + ActionRecordInfo const* const held = entry->holding(change.record); + if (held == nullptr) { + continue; + } + // An entry still waiting for its first value may come to hold a key; keyless rows cannot be told apart. + bool const marked = !change.keys || !held->keyField || !entry->hasValue.peek() || + std::ranges::any_of(detail::keyedRowsOf(entry->confirmed, *held), [&](Value const* row) { + return keys.contains(detail::canonicalJson(*row->field(*held->keyField))); + }); + if (marked) { + marks.add(*entry); + } + } +} + +inline void Session::markInstanceChange(wire::InstanceTopic const& instance, detail::Marks& marks) { + std::string const shared = "shared:" + instance.primary; + for (auto const& [name, entry] : _entries) { + static_cast(name); + if (entry->evaluate || entry->typeId != instance.modelType) { + continue; + } + // Without a primary key, the instance is one of the connection's own private ones. + bool const same = + instance.primary.empty() ? !entry->instanceToken.starts_with("shared") : entry->instanceToken == shared; + if (same) { + marks.add(*entry); + } + } +} + inline void Session::actionSucceeded(std::string const& typeId, std::string const& actionId, Value const& reply, ReplyUse use, std::string const& instanceToken) { _runtime->batch([&] { applyReplyEffects(typeId, actionId, reply, use, instanceToken); }); diff --git a/src/qt/qt_websocket_server.cpp b/src/qt/qt_websocket_server.cpp index 59f8995b0..51dc831bd 100644 --- a/src/qt/qt_websocket_server.cpp +++ b/src/qt/qt_websocket_server.cpp @@ -265,6 +265,9 @@ bool QtWebSocketServer::queueEnvelope(QPointer const& server, // Before the queueing, so a backlog read on the server strand already // covers a frame still on its way to the Qt thread. backlog->fetch_add(counted); + // Qt owns the queued callable and deletes it once it ran or was dropped; the analyzer cannot see into + // QMetaObject's slot-object bookkeeping and reports it as leaked. + // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) bool const queued = QMetaObject::invokeMethod( app, [server, socket, backlog, counted, envelope = std::move(envelope)]() { diff --git a/tests/test_push_effects.cpp b/tests/test_push_effects.cpp index 9dda99105..08dd49e1a 100644 --- a/tests/test_push_effects.cpp +++ b/tests/test_push_effects.cpp @@ -349,7 +349,6 @@ TEST_CASE("every_problem_is_listed", "[push][records]") { namespace { -using morph::effects::Changes; using morph::effects::InstanceChange; using morph::effects::RecordChange; diff --git a/tests/test_push_watch.cpp b/tests/test_push_watch.cpp index 3febfb1e8..b22dc48b5 100644 --- a/tests/test_push_watch.cpp +++ b/tests/test_push_watch.cpp @@ -1487,6 +1487,7 @@ ::morph::effects::Changes rowChange(std::vector const& keys) { /// @p count changes to `T_FanRow`, one key each, numbered from @p first. std::vector<::morph::effects::Changes> rowBurst(std::size_t count, std::size_t first = 0) { std::vector<::morph::effects::Changes> burst; + burst.reserve(count); for (std::size_t idx = 0; idx < count; ++idx) { burst.push_back(rowChange({std::to_string(first + idx)})); } @@ -1496,6 +1497,7 @@ std::vector<::morph::effects::Changes> rowBurst(std::size_t count, std::size_t f /// The keys "0" up to @p count - 1, as text. std::vector numbered(std::size_t count) { std::vector keys; + keys.reserve(count); for (std::size_t idx = 0; idx < count; ++idx) { keys.push_back(std::to_string(idx)); } @@ -1587,6 +1589,7 @@ TEST_CASE("burst_over_backlog_yields_one_resync", "[push][watch]") { TEST_CASE("merged_keys_never_exceed_client_bound", "[push][watch]") { std::vector many; + many.reserve(::morph::effects::kMaxChangeKeys); for (std::size_t idx = 0; idx < ::morph::effects::kMaxChangeKeys; ++idx) { many.push_back(std::to_string(idx)); } From 46545538730f2ac19be86b5ef786503fdacf5431 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 00:33:26 +0200 Subject: [PATCH 48/68] core: the result walker compiles under GCC for an aggregate with no members Its member fold is empty then, so GCC reports the helper lambda as set but not used, which -Werror makes an error. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- include/morph/core/detail/result_walker.hpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/morph/core/detail/result_walker.hpp b/include/morph/core/detail/result_walker.hpp index 8a35b9fa2..46f33f3b0 100644 --- a/include/morph/core/detail/result_walker.hpp +++ b/include/morph/core/detail/result_walker.hpp @@ -131,7 +131,8 @@ void visit(Walk& walk, std::string const& path); /// @param path The aggregate's own path. template void visitMembers(Walk& walk, std::string const& path, std::index_sequence /*indices*/) { - auto const member = [&]() { + // Unused for an aggregate with no members: the fold below is then empty. + [[maybe_unused]] auto const member = [&]() { using Member = std::remove_cvref_t(std::declval()))>; if constexpr (mayHoldSlots()) { constexpr std::string_view key = glz::reflect::keys[Index]; From 9cbd29a5fe32bbdd12b064856645c87e47eb7625 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 00:33:26 +0200 Subject: [PATCH 49/68] ui: documents are checked against action effects A query whose action declares effects is refused at load: a query must read. An evaluate query is exempt, since an evaluation runs no handler. Three warnings, collected in Document::warnings and logged by the bundle registry and the screen registry, never refuse a document: a refresh declaration (invalidates, patch, refreshOn) the mutation's effects already cover; a mutation whose action declares nothing and that refreshes nothing; and a watch of a query, from any source, that runs a mutation with effects, which can make two clients refetch each other. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- docs/spec/ui/document.md | 41 +++++- include/morph/ui/author.hpp | 4 + include/morph/ui/document.hpp | 264 +++++++++++++++++++++++++++++++++- include/morph/ui/service.hpp | 11 ++ tests/CMakeLists.txt | 1 + tests/test_push_documents.cpp | 246 +++++++++++++++++++++++++++++++ 6 files changed, 562 insertions(+), 5 deletions(-) create mode 100644 tests/test_push_documents.cpp diff --git a/docs/spec/ui/document.md b/docs/spec/ui/document.md index 99711d616..f8035453e 100644 --- a/docs/spec/ui/document.md +++ b/docs/spec/ui/document.md @@ -23,6 +23,7 @@ Read this before changing what a document may say. The expression language insid - [Load-time rules](#load-time-rules) - [The catalog](#the-catalog) - [Refusals](#refusals) +- [Warnings](#warnings) - [Design decisions](#design-decisions) - [Out of scope](#out-of-scope) @@ -68,6 +69,7 @@ declare the app-only customization kinds. | `components` | `components` | Name to `Component` | | `customize` | `customize` | Name to `CustomizePoint` | | `view` | `view` | The root `DocNode`; required on a screen | +| `warnings` | — | What validation found that does not refuse the document ([Warnings](#warnings)) | A key a client does not know is ignored, so additions within `ui/1` are compatible; a key whose name begins with `!` is must-understand, and an unknown one refuses the document wherever it @@ -243,8 +245,16 @@ know the result's shape, and a patch that finds no list is reported by the inter record's wire name for the first two; it is absent when the action declared nothing, which makes the action a read. `records` lists the records the action's result holds as `{record, keyField?, paths}`, empty when it holds none. They are read from the registries where `CatalogFacts::action` records the action, so an invalid effects and records -configuration (`EffectsRegistrationError`) fails at that call, at startup. No load-time rule reads them yet; they -are what a client's watch is derived from. +configuration (`EffectsRegistrationError`) fails at that call, at startup. They are what a client's watch is derived +from, and the loader reads them for one refusal and three warnings: + +- **A query must read.** A query whose action declares effects, `none` included, is refused: a fetch then never + produces a change, so two clients cannot refetch each other through their queries. An `evaluate` query is exempt, + because an evaluation runs no handler. +- **The warnings** are in [Warnings](#warnings). + +An action the catalog does not answer is refused before either is asked, so effects registered for an action id that +no action registers are never seen by a document: the catalog describes registered actions only. A lookup left empty skips its rules; the document is otherwise checked in full. `stateless` decides an alias's default instance, which is the interpreter's. A third lookup, `schema(actionId)`, answers an action's schema JSON; it is not a load-time rule but what the interpreter builds a `form` node's @@ -259,6 +269,33 @@ view document whose `v-query` is the action (a `collection` node's), and `wizard at the expression node. Text that is not JSON (`loadDocumentText`) is refused at the root, and so is text nested deeper than 64 objects and arrays (`kMaxJsonDepth`), which is checked before the JSON parser runs. +## Warnings + +`Document::warnings` holds the `LoadWarning{kind, path, text}` findings that do not refuse the document, in the order +found; `message(documentId)` prints them as a refusal is printed. They need the catalog's `action` lookup, and a +document loaded without it has none. A warning is given only where the declared effects certainly cover, or +certainly miss, what the document declares. + +| Kind | When | +|---|---| +| `Redundant` | An `invalidates` entry, a `patch` or a `refreshOn` entry that the effects already cover (below). The path names the entry. | +| `Uncovered` | A mutation whose action declares nothing, not even `none`, that has no `invalidates` or `patch`, no `refetch` anywhere in its `onSuccess`, and that no query of the document names in `refreshOn`. After it succeeds nothing on any screen updates. `effects::none` silences it for an action that changes nothing, such as one that opens or attaches. | +| `WatchFeedback` | A `watch` declaration whose `source` is `any` (the default), whose `on` reads a query's value (`{"query": q}` or a `ref` to it, directly or through a `let`), and whose `do` runs (`run`, `upload`, `signIn`, at any depth) a mutation whose action declares an effect other than `none`. Another client's change refetches the query, the watch runs the mutation, and its change reaches the other client. `"source": "user"` silences it. | + +**What covers a query.** A query holds exactly the records of its action (`records`); an `evaluate` query holds none. + +- `updates R` or `changes R` covers a query that holds `R`. +- `changesInstance` covers a query that reads through the same model alias declaration as the mutation, since that + is the instance the action ran on. Two aliases of one model type are two instances, so it covers nothing through + another alias. +- A `patch` is covered only by `updates R` of a record the patched query holds: the reply then patches the rows + itself. `changes R` refetches instead, which a patch exists to avoid. +- A `refreshOn` entry names an action run through any instance, so it is covered by the action's `updates` and + `changes` alone, never by `changesInstance`. + +`UiRegistry::addScreen`, `UiRegistry::addFile` for the app document, and `author::ScreenRegistry::add` log each warning at `warn` +(`morph::log`) when they register a document; the document registers as it would without them. + ## Design decisions - **One generic node, one kind table.** A typed struct per kind would double the description of diff --git a/include/morph/ui/author.hpp b/include/morph/ui/author.hpp index 4c2d16b89..654449be1 100644 --- a/include/morph/ui/author.hpp +++ b/include/morph/ui/author.hpp @@ -23,6 +23,7 @@ #include #include "../core/effects.hpp" +#include "../core/logger.hpp" #include "../core/model_key.hpp" #include "../core/registry.hpp" #include "../detail/fixed_string.hpp" @@ -4232,6 +4233,9 @@ class ScreenRegistry { if (auto refusal = checkAccessibility(*loaded)) { return std::unexpected(std::move(*refusal)); } + for (LoadWarning const& warning : loaded->warnings) { + morph::log::logWarn(warning.message(loaded->id)); + } _facts = std::move(facts); _screens.push_back(std::make_unique( RegisteredScreen{.id = screen.id(), .text = std::move(emitted->text), .document = std::move(*loaded)})); diff --git a/include/morph/ui/document.hpp b/include/morph/ui/document.hpp index 6c8ab91ee..a3ea090c7 100644 --- a/include/morph/ui/document.hpp +++ b/include/morph/ui/document.hpp @@ -490,6 +490,35 @@ struct CustomizePoint { Value defaultValue; }; +/// @brief What a load-time warning is about. A warning never refuses a document. +enum class LoadWarningKind : std::uint8_t { + WatchFeedback, ///< A watch of a query runs a mutation that changes data, on any change of it. + Redundant, ///< An `invalidates`, `patch` or `refreshOn` that the action's effects already cover. + Uncovered, ///< A mutation whose action declares nothing, and after which nothing refreshes. +}; + +/// @brief A finding of load-time validation that does not refuse the document. +struct LoadWarning { + /// @brief What it is about. + LoadWarningKind kind = LoadWarningKind::Redundant; + /// @brief The JSON Pointer of the declaration it names. + std::string path; + /// @brief The finding, in one sentence. + std::string text; + + /// @brief The warning as one line of text. + /// @param document The id of the document it was found in; empty leaves it out. + /// @return `document at : `, leaving out the parts that are empty. + [[nodiscard]] std::string message(std::string_view document) const { + std::string out; + if (!document.empty()) { + out += "document " + std::string{document} + " "; + } + out += "at " + (path.empty() ? std::string{"/"} : path) + ": " + text; + return out; + } +}; + /// @brief A loaded and validated UI document: a screen, or the app document (id `app`, spec 6 §5). struct Document { /// @brief The vocabulary, `ui/1`. @@ -520,6 +549,9 @@ struct Document { std::map> customize; /// @brief The view; `nullptr` for the app document. DocNodePtr view; + /// @brief What load-time validation found that does not refuse the document, in the order found. Empty when + /// the catalog answers no actions. + std::vector warnings; }; namespace detail { @@ -1927,6 +1959,8 @@ struct NameEntry { ExprPtr derivedFrom; /// @brief The query of a `Query` entry. QueryDecl const* query = nullptr; + /// @brief The mutation of a `Mutation` entry. + MutationDecl const* mutation = nullptr; /// @brief The element of an `Element` entry. DocNode const* element = nullptr; /// @brief 0 before its type is derived, 1 while it is, 2 after. @@ -2115,8 +2149,13 @@ class DocumentValidator { } checkCustomizePoints(); checkWatchGraph(); + checkEffects(); } + /// @brief What validation found that does not refuse the document. + /// @return The warnings, in the order found; the validator keeps none. + [[nodiscard]] std::vector takeWarnings() noexcept { return std::move(_warnings); } + /// @brief Finds a name in a scope chain. /// @param scope The innermost scope. /// @param name The name. @@ -2243,12 +2282,15 @@ class DocumentValidator { entry.query = &query; add(scope, query.name, std::move(entry)); _queries.emplace(query.name, &query); + _queryScopes.emplace_back(&query, &scope); } for (MutationDecl const& mutation : decl.mutations) { NameEntry entry; entry.category = NameCategory::Mutation; entry.path = mutation.path; + entry.mutation = &mutation; add(scope, mutation.name, std::move(entry)); + _mutations.emplace_back(&mutation, &scope); collectTransfers(mutation.onSuccess, scope); collectTransfers(mutation.onError, scope); } @@ -2430,7 +2472,13 @@ class DocumentValidator { static_cast(type); } for (QueryDecl const& query : decl.queries) { - checkModelAction(context, query.model, query.action, query.path); + auto const action = checkModelAction(context, query.model, query.action, query.path); + // An evaluation runs no handler, so only a call can change anything. + if (action && action->effects && !query.evaluate) { + refuse(query.path, std::format("a query reads, so its action declares no effects; `{}` declares {}, " + "so it runs as a mutation", + query.action, describeEffects(*action->effects))); + } if (query.body) { checkExpr(*query.body, context); } @@ -2981,6 +3029,210 @@ class DocumentValidator { } } + // The effect checks below read only what the catalog says about each action. They warn only where the + // effects certainly cover, or certainly miss, what the document declares, because a warning a document cannot + // act on would teach its author to ignore the rest. + + /// The effects as a phrase, such as "`changes T_Row`, `none`". + [[nodiscard]] static std::string describeEffects(std::vector const& effects) { + std::string out; + for (ActionEffect const& effect : effects) { + out += (out.empty() ? "`" : ", `") + effect.kind + (effect.record ? " " + *effect.record : "") + "`"; + } + return out; + } + + /// Whether an action's effects change anything; `none` alone does not. + [[nodiscard]] static bool changesData(ActionInfo const& info) { + return info.effects && + std::ranges::any_of(*info.effects, [](ActionEffect const& effect) { return effect.kind != "none"; }); + } + + [[nodiscard]] ModelDecl const* aliasOf(QueryDecl const& query) const { + auto const found = std::ranges::find(_queryScopes, &query, &std::pair::first); + return lookup(found->second, query.model)->model; + } + + /// Why the effects of @p actionId cover @p query, or nothing when they do not certainly cover it. A query holds + /// exactly the records of its action, so `updates` or `changes` of one of them reaches it. `changesInstance` + /// reaches the entries read through the instance the action ran on; @p alias is the alias the action runs + /// through, and covers only a query of that same alias, since two aliases of one model type are two instances. + /// Without an alias, as for `refreshOn`, which names an action run through any instance, it covers nothing. + /// @p updatesOnly limits the cover to `updates`, the effect whose reply patches rows as a `patch` does. + // NOLINTNEXTLINE(bugprone-easily-swappable-parameters): the action, then the query it may cover. + [[nodiscard]] std::optional covers(std::string const& actionId, ActionInfo const& action, + QueryDecl const& query, ModelDecl const* alias, + bool updatesOnly) const { + if (!action.effects || query.evaluate) { + return std::nullopt; + } + auto const held = _options->catalog.action(query.action); + for (ActionEffect const& effect : *action.effects) { + bool const records = effect.kind == "updates" || (!updatesOnly && effect.kind == "changes"); + if (records && held && effect.record && + std::ranges::find(held->records, *effect.record, &ActionRecordInfo::record) != held->records.end()) { + return std::format("`{}` {} `{}`, which `{}` holds", actionId, effect.kind, *effect.record, + query.name); + } + if (!updatesOnly && effect.kind == "changesInstance" && alias != nullptr && aliasOf(query) == alias) { + return std::format("`{}` changes the instance of `{}`, through which `{}` reads", actionId, + query.model, query.name); + } + } + return std::nullopt; + } + + void warn(LoadWarningKind kind, std::string path, std::string text) { + _warnings.push_back(LoadWarning{.kind = kind, .path = std::move(path), .text = std::move(text)}); + } + + void checkEffects() { + if (!_options->catalog.action) { + return; + } + for (auto const& [mutation, scope] : _mutations) { + checkMutationEffects(*mutation, *scope); + } + for (auto const& [query, scope] : _queryScopes) { + static_cast(scope); + for (std::size_t index = 0; index < query->refreshOn.size(); ++index) { + std::string const& actionId = query->refreshOn.at(index); + auto const action = _options->catalog.action(actionId); + if (auto const why = action ? covers(actionId, *action, *query, nullptr, false) : std::nullopt) { + warn(LoadWarningKind::Redundant, pointerAppend(pointerAppend(query->path, "refreshOn"), index), + "`refreshOn` is redundant: " + *why); + } + } + } + for (auto const& [watch, scope] : _watches) { + checkWatchFeedback(*watch, scope); + } + } + + void checkMutationEffects(MutationDecl const& mutation, ScopeInfo& scope) { + auto const action = _options->catalog.action(mutation.action); + if (!action) { + return; + } + ModelDecl const* const alias = lookup(&scope, mutation.model)->model; + for (std::size_t index = 0; index < mutation.invalidates.size(); ++index) { + QueryDecl const& query = *_queries.at(mutation.invalidates.at(index)); + if (auto const why = covers(mutation.action, *action, query, alias, false)) { + warn(LoadWarningKind::Redundant, pointerAppend(pointerAppend(mutation.path, "invalidates"), index), + "`invalidates` is redundant: " + *why); + } + } + if (mutation.patch) { + QueryDecl const& query = *_queries.at(mutation.patch->query); + if (auto const why = covers(mutation.action, *action, query, alias, true)) { + warn(LoadWarningKind::Redundant, pointerAppend(mutation.path, "patch"), + "`patch` is redundant: " + *why + ", and its reply patches the rows"); + } + } + bool const refreshed = + !mutation.invalidates.empty() || mutation.patch || refetches(mutation.onSuccess) || + std::ranges::any_of(_queryScopes, [&](auto const& entry) { + return std::ranges::find(entry.first->refreshOn, mutation.action) != entry.first->refreshOn.end(); + }); + if (!action->effects && !refreshed) { + warn(LoadWarningKind::Uncovered, mutation.path, + std::format("nothing refreshes after `{}` succeeds: `{}` declares no effects, and no `invalidates`, " + "`patch`, `refetch` or `refreshOn` names it; declare what it changes, or " + "`effects::none` when it changes nothing", + mutation.name, mutation.action)); + } + } + + /// Whether a command list refetches a query, which is a refresh the document declares itself. + [[nodiscard]] static bool refetches(Commands const& commands) { + return std::ranges::any_of(commands, [](Command const& cmd) { + return cmd.kind == CommandKind::Refetch || refetches(cmd.commands) || refetches(cmd.alternative); + }); + } + + // Another client's change refetches the watched query, the watch runs the mutation, and its change reaches the + // other client, which may do the same. Only `"source": "user"` keeps a refetch from running the watch. + void checkWatchFeedback(WatchDecl const& watch, ScopeInfo* scope) { + if (watch.source == WatchSource::User) { + return; + } + std::vector locals; + std::set seenLets; + auto const query = readQuery(*watch.on, scope, locals, seenLets); + if (!query) { + return; + } + if (auto const mutation = runsWriter(watch.body, scope)) { + warn(LoadWarningKind::WatchFeedback, watch.path, + std::format("the watch reads the query `{}` and runs `{}`, which changes data: another client's " + "change refetches the query and runs it again; give the watch \"source\": \"user\"", + *query, *mutation)); + } + } + + /// The first query whose value an expression reads, directly or through a `let`. + [[nodiscard]] static std::optional readQuery(Expr const& expr, ScopeInfo* scope, + std::vector& locals, + std::set& seenLets) { + if (expr.kind == ExprKind::Query) { + return expr.name; + } + if (expr.kind == ExprKind::Ref && std::ranges::find(locals, expr.name) == locals.end()) { + NameEntry const* const entry = lookup(scope, expr.name); + if (entry != nullptr && entry->category == NameCategory::Query) { + return expr.name; + } + if (entry != nullptr && entry->category == NameCategory::Let && seenLets.insert(entry).second) { + std::vector inner; + if (auto found = readQuery(*entry->derivedFrom, entry->scope, inner, seenLets)) { + return found; + } + } + } + bool const binds = expr.kind == ExprKind::Map || expr.kind == ExprKind::Filter; + for (std::size_t index = 0; index < expr.args.size(); ++index) { + if (binds && index == 1) { + locals.push_back(expr.name); + } + auto found = readQuery(*expr.args.at(index), scope, locals, seenLets); + if (binds && index == 1) { + locals.pop_back(); + } + if (found) { + return found; + } + } + for (auto const& [name, field] : expr.fields) { + static_cast(name); + if (auto found = readQuery(*field, scope, locals, seenLets)) { + return found; + } + } + return std::nullopt; + } + + /// The first mutation a command list runs whose action changes data. + [[nodiscard]] std::optional runsWriter(Commands const& commands, ScopeInfo* scope) const { + for (Command const& cmd : commands) { + bool const runs = + cmd.kind == CommandKind::Run || cmd.kind == CommandKind::Upload || cmd.kind == CommandKind::SignIn; + NameEntry const* const entry = runs ? lookup(scope, cmd.target) : nullptr; + if (entry != nullptr && entry->mutation != nullptr) { + auto const action = _options->catalog.action(entry->mutation->action); + if (action && changesData(*action)) { + return cmd.target; + } + } + if (auto found = runsWriter(cmd.commands, scope)) { + return found; + } + if (auto found = runsWriter(cmd.alternative, scope)) { + return found; + } + } + return std::nullopt; + } + Document const* _doc; LoadOptions const* _options; std::deque _scopes; @@ -2989,6 +3241,9 @@ class DocumentValidator { std::map> _queries; std::map> _elements; std::vector> _watches; + std::vector> _mutations; + std::vector> _queryScopes; + std::vector _warnings; std::vector _deriving; // NOLINTEND(misc-no-recursion) }; @@ -3123,7 +3378,8 @@ inline std::expected ScopeResolver::pref(std::string_view poi /// list is in `docs/spec/ui/document.md`. The rules that need catalog facts run when @p options supplies them. /// @param json The document's JSON. /// @param options The catalog facts. -/// @return The document, or the first refusal, naming the document id, the JSON Pointer and the reason. +/// @return The document, with what validation found that does not refuse it in `Document::warnings`, or the first +/// refusal, naming the document id, the JSON Pointer and the reason. [[nodiscard]] inline std::expected loadDocument(Json const& json, LoadOptions const& options = {}) { std::string documentId; if (auto const* const object = std::get_if(&json.data)) { @@ -3136,7 +3392,9 @@ inline std::expected ScopeResolver::pref(std::string_view poi } try { Document doc = detail::DocumentParser{}.parse(json); - detail::DocumentValidator{doc, options}.run(); + detail::DocumentValidator validator{doc, options}; + validator.run(); + doc.warnings = validator.takeWarnings(); return doc; } catch (detail::DocumentRefused const& refused) { Refusal refusal = refused.refusal(); diff --git a/include/morph/ui/service.hpp b/include/morph/ui/service.hpp index 83fe68496..ac45f52e1 100644 --- a/include/morph/ui/service.hpp +++ b/include/morph/ui/service.hpp @@ -16,6 +16,7 @@ #include #include +#include "../core/logger.hpp" #include "../core/remote.hpp" #include "../core/wire.hpp" #include "../session/session.hpp" @@ -324,6 +325,7 @@ class UiRegistry { if (!loaded->view) { return refuse(entry.id, "the app document is added with addFile"); } + logWarnings(*loaded); detail::DocumentActions const named{*loaded}; Screen screen{.served = ServedScreen{.entry = std::move(entry), .hash = contentHash(text), @@ -503,12 +505,21 @@ class UiRegistry { if (loaded->id != kAppFile) { return refuse(file.id, "the app document's id is `app`"); } + logWarnings(*loaded); detail::DocumentActions const named{*loaded}; _appActions = named.actions(); _appModelTypes = named.modelTypes(); return {}; } + /// @brief Logs what load-time validation found in @p document that does not refuse it. + /// @param document The loaded document. + static void logWarnings(Document const& document) { + for (LoadWarning const& warning : document.warnings) { + morph::log::logWarn(warning.message(document.id)); + } + } + /// @brief A refusal of the registration of @p document. /// @param document The screen or file id. /// @param reason Why. diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 6863f85b1..1839075b2 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -28,6 +28,7 @@ add_executable(morph_tests test_push_channel.cpp test_push_watch.cpp test_push_cache.cpp + test_push_documents.cpp push_records_late.cpp test_executor_extra.cpp test_strand.cpp diff --git a/tests/test_push_documents.cpp b/tests/test_push_documents.cpp new file mode 100644 index 000000000..70dfcdbe0 --- /dev/null +++ b/tests/test_push_documents.cpp @@ -0,0 +1,246 @@ +// SPDX-License-Identifier: Apache-2.0 + +// Documents checked against action effects (spec 8 §10): a query must read, and three warnings point at refresh +// declarations that effects make redundant, at mutations nothing refreshes after, and at watches that may feed +// another client's changes back. + +#include +#include +#include +#include +#include +#include +#include + +namespace ui = morph::ui; +using ui::ActionEffect; +using ui::ActionInfo; +using ui::ActionRecordInfo; +using ui::LoadWarning; +using ui::LoadWarningKind; + +namespace { + +/// One model, `T_Model`. Its reads hold `T_Row` (`T_List`) or `T_Other` (`T_ListOther`); its writes declare +/// `updates T_Row` (`T_Edit`), `changes T_Row` (`T_Rename`), `changesInstance` (`T_Reset`), `none` (`T_Open`), or +/// nothing (`T_Write`). +ui::Catalog testCatalog() { + ui::Catalog catalog; + catalog.model = [](std::string_view type) -> std::optional { + if (type != "T_Model") { + return std::nullopt; + } + return ui::ModelInfo{}; + }; + catalog.action = [](std::string_view action) -> std::optional { + ActionInfo info; + info.modelType = "T_Model"; + auto const effect = [](std::string kind, std::optional record) { + return std::vector{ActionEffect{.kind = std::move(kind), .record = std::move(record)}}; + }; + if (action == "T_List") { + info.records = {ActionRecordInfo{.record = "T_Row", .keyField = "id", .paths = {""}}}; + } else if (action == "T_ListOther") { + info.records = {ActionRecordInfo{.record = "T_Other", .keyField = "id", .paths = {""}}}; + } else if (action == "T_Edit") { + info.effects = effect("updates", "T_Row"); + info.records = {ActionRecordInfo{.record = "T_Row", .keyField = "id", .paths = {""}}}; + } else if (action == "T_Rename") { + info.effects = effect("changes", "T_Row"); + } else if (action == "T_Reset") { + info.effects = effect("changesInstance", std::nullopt); + } else if (action == "T_Open") { + info.effects = effect("none", std::nullopt); + } else if (action != "T_Write") { + return std::nullopt; + } + return info; + }; + return catalog; +} + +/// A document with one model alias `m`, the given queries, mutations and watches, and a text view. +std::string document(std::string_view queries, std::string_view mutations, std::string_view watches = "[]") { + return R"({"vocab":"ui/1","id":"t","models":{"m":{"type":"T_Model"}},"queries":)" + std::string{queries} + + R"(,"mutations":)" + std::string{mutations} + R"(,"watch":)" + std::string{watches} + + R"(,"view":{"kind":"text","text":"x"}})"; +} + +ui::Document load(std::string const& text) { + auto loaded = ui::loadDocumentText(text, ui::LoadOptions{.catalog = testCatalog()}); + if (!loaded) { + FAIL(loaded.error().message()); + } + return std::move(*loaded); +} + +std::vector warningsOf(std::string const& text, LoadWarningKind kind) { + std::vector out; + for (LoadWarning const& warning : load(text).warnings) { + if (warning.kind == kind) { + out.push_back(warning); + } + } + return out; +} + +} // namespace + +TEST_CASE("read_only_document_loads_without_warnings", "[push][documents]") { + auto const doc = + load(document(R"({"rows":{"model":"m","action":"T_List"}})", + R"({"edit":{"model":"m","action":"T_Edit"},"open":{"model":"m","action":"T_Open"}})")); + CHECK(doc.warnings.empty()); +} + +TEST_CASE("query_of_action_with_effects_refused", "[push][documents]") { + for (std::string_view const action : {"T_Edit", "T_Rename", "T_Reset", "T_Open"}) { + CAPTURE(action); + auto const loaded = + ui::loadDocumentText(document(R"({"rows":{"model":"m","action":")" + std::string{action} + R"("}})", "{}"), + ui::LoadOptions{.catalog = testCatalog()}); + REQUIRE_FALSE(loaded.has_value()); + CHECK(loaded.error().path == "/queries/rows"); + CHECK(loaded.error().reason.contains("a query reads")); + } + // An evaluation runs no handler, so it changes nothing whatever the action declares. + auto const evaluated = ui::loadDocumentText(document(R"({"preview":{"model":"m","evaluate":"T_Edit"}})", "{}"), + ui::LoadOptions{.catalog = testCatalog()}); + CHECK(evaluated.has_value()); + // Without the catalog's actions nothing is known about effects. + CHECK(ui::loadDocumentText(document(R"({"rows":{"model":"m","action":"T_Edit"}})", "{}")).has_value()); +} + +TEST_CASE("redundant_invalidates_warns", "[push][documents]") { + auto const covered = warningsOf(document(R"({"rows":{"model":"m","action":"T_List"}, + "other":{"model":"m","action":"T_ListOther"}})", + R"({"rename":{"model":"m","action":"T_Rename", + "invalidates":["other","rows"]}})"), + LoadWarningKind::Redundant); + REQUIRE(covered.size() == 1); + CHECK(covered.front().path == "/mutations/rename/invalidates/1"); + CHECK(covered.front().text.contains("`T_Rename` changes `T_Row`")); + + // `updates` covers the query too: a reply patches the rows it holds. + CHECK(warningsOf(document(R"({"rows":{"model":"m","action":"T_List"}})", + R"({"edit":{"model":"m","action":"T_Edit","invalidates":["rows"]}})"), + LoadWarningKind::Redundant) + .size() == 1); + + // `changesInstance` covers a query through the same alias, and only that. + auto const instance = warningsOf(document(R"({"rows":{"model":"m","action":"T_List"}})", + R"({"reset":{"model":"m","action":"T_Reset","invalidates":["rows"]}})"), + LoadWarningKind::Redundant); + REQUIRE(instance.size() == 1); + CHECK(instance.front().path == "/mutations/reset/invalidates/0"); + auto const otherAlias = warningsOf( + R"({"vocab":"ui/1","id":"t","models":{"m":{"type":"T_Model"},"n":{"type":"T_Model"}}, + "queries":{"rows":{"model":"n","action":"T_List"}}, + "mutations":{"reset":{"model":"m","action":"T_Reset","invalidates":["rows"]}}, + "view":{"kind":"text","text":"x"}})", + LoadWarningKind::Redundant); + CHECK(otherAlias.empty()); +} + +TEST_CASE("redundant_patch_warns", "[push][documents]") { + auto const patched = warningsOf(document(R"({"rows":{"model":"m","action":"T_List"}})", + R"({"edit":{"model":"m","action":"T_Edit", + "patch":{"query":"rows","key":{"ref":"result.id"},"with":{"ref":"result"}}}})"), + LoadWarningKind::Redundant); + REQUIRE(patched.size() == 1); + CHECK(patched.front().path == "/mutations/edit/patch"); + + // A patch of a query that holds no record the reply updates is the only thing that updates it. + CHECK(warningsOf(document(R"({"other":{"model":"m","action":"T_ListOther"}})", + R"({"edit":{"model":"m","action":"T_Edit", + "patch":{"query":"other","key":{"ref":"result.id"},"with":{"ref":"result"}}}})"), + LoadWarningKind::Redundant) + .empty()); +} + +TEST_CASE("redundant_refresh_on_warns", "[push][documents]") { + auto const refreshed = + warningsOf(document(R"({"rows":{"model":"m","action":"T_List","refreshOn":["T_Write","T_Rename"]}, + "other":{"model":"m","action":"T_ListOther","refreshOn":["T_Rename"]}})", + "{}"), + LoadWarningKind::Redundant); + REQUIRE(refreshed.size() == 1); + CHECK(refreshed.front().path == "/queries/rows/refreshOn/1"); + CHECK(refreshed.front().text.contains("`T_Rename` changes `T_Row`")); + + // `refreshOn` names an action run anywhere, through any instance, so `changesInstance` does not cover it. + CHECK(warningsOf(document(R"({"rows":{"model":"m","action":"T_List","refreshOn":["T_Reset"]}})", "{}"), + LoadWarningKind::Redundant) + .empty()); +} + +TEST_CASE("uncovered_mutation_warns", "[push][documents]") { + auto const uncovered = warningsOf( + document(R"({"rows":{"model":"m","action":"T_List"}})", R"({"write":{"model":"m","action":"T_Write"}})"), + LoadWarningKind::Uncovered); + REQUIRE(uncovered.size() == 1); + CHECK(uncovered.front().path == "/mutations/write"); + CHECK(uncovered.front().text.contains("`T_Write`")); + + // Each of these refreshes something after a success. + for (std::string const& covered : + {document(R"({"rows":{"model":"m","action":"T_List"}})", + R"({"write":{"model":"m","action":"T_Write","invalidates":["rows"]}})"), + document(R"({"rows":{"model":"m","action":"T_List"}})", + R"({"write":{"model":"m","action":"T_Write", + "patch":{"query":"rows","key":{"ref":"result.id"},"with":{"ref":"result"}}}})"), + document(R"({"rows":{"model":"m","action":"T_List","refreshOn":["T_Write"]}})", + R"({"write":{"model":"m","action":"T_Write"}})"), + document(R"({"rows":{"model":"m","action":"T_List"}})", + R"({"write":{"model":"m","action":"T_Write","onSuccess":[{"refetch":"rows"}]}})")}) { + CAPTURE(covered); + CHECK(warningsOf(covered, LoadWarningKind::Uncovered).empty()); + } + // Without the catalog's actions nothing is known about effects. + auto const blind = ui::loadDocumentText(document("{}", R"({"write":{"model":"m","action":"T_Write"}})")); + REQUIRE(blind.has_value()); + CHECK(blind->warnings.empty()); +} + +TEST_CASE("effects_none_silences_uncovered", "[push][documents]") { + auto const doc = load(document("{}", R"({"open":{"model":"m","action":"T_Open"}})")); + CHECK(doc.warnings.empty()); +} + +TEST_CASE("watch_feedback_warns", "[push][documents]") { + auto const fed = warningsOf( + document(R"({"rows":{"model":"m","action":"T_List"}})", R"({"rename":{"model":"m","action":"T_Rename"}})", + R"([{"on":{"query":"rows"},"do":[{"run":"rename"}]}])"), + LoadWarningKind::WatchFeedback); + REQUIRE(fed.size() == 1); + CHECK(fed.front().path == "/watch/0"); + CHECK(fed.front().text.contains("\"source\": \"user\"")); + + // Read through a `let`, and run from a nested command. + auto const nested = warningsOf( + R"({"vocab":"ui/1","id":"t","models":{"m":{"type":"T_Model"}}, + "queries":{"rows":{"model":"m","action":"T_List"}}, + "let":{"count":{"fn":"length","args":[{"query":"rows"}]}}, + "mutations":{"reset":{"model":"m","action":"T_Reset"}}, + "watch":[{"on":{"ref":"count"},"do":[{"if":[true,[{"run":"reset"}],[]]}]}], + "view":{"kind":"text","text":"x"}})", + LoadWarningKind::WatchFeedback); + CHECK(nested.size() == 1); + + // A watch of state, or one running a mutation that changes nothing, cannot feed a change back. + CHECK(warningsOf(R"({"vocab":"ui/1","id":"t","models":{"m":{"type":"T_Model"}},"state":{"s":{"init":0}}, + "queries":{"rows":{"model":"m","action":"T_List"}}, + "mutations":{"rename":{"model":"m","action":"T_Rename"},"open":{"model":"m","action":"T_Open"}}, + "watch":[{"on":{"ref":"s"},"do":[{"run":"rename"}]}, + {"on":{"query":"rows"},"do":[{"run":"open"}]}], + "view":{"kind":"text","text":"x"}})", + LoadWarningKind::WatchFeedback) + .empty()); +} + +TEST_CASE("watch_source_user_does_not_warn", "[push][documents]") { + auto const doc = + load(document(R"({"rows":{"model":"m","action":"T_List"}})", R"({"rename":{"model":"m","action":"T_Rename"}})", + R"([{"on":{"query":"rows"},"source":"user","do":[{"run":"rename"}]}])")); + CHECK(doc.warnings.empty()); +} From 37fb712c50df487a93ee3718eab0b55cdb5adfae Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 00:36:10 +0200 Subject: [PATCH 50/68] tests: the net backlog test waits for every execute before the drain The drain finds nothing in flight as soon as the server has answered what it has read, which under UBSan was before it had read all 128 requests: the event then went out ahead of the replies. The test now waits until the model has run every execute, then for the drain. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- tests/net/test_push_net.cpp | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/tests/net/test_push_net.cpp b/tests/net/test_push_net.cpp index fe32edbfa..0e2790b22 100644 --- a/tests/net/test_push_net.cpp +++ b/tests/net/test_push_net.cpp @@ -50,10 +50,15 @@ struct PnEcho { struct PnBig { int size = 0; }; +// NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables): the model's strand counts, the test thread reads. +inline std::atomic pnBigRuns{0}; struct PnModel { int execute(PnEcho action) { return action.value; } - // NOLINTNEXTLINE(modernize-return-braced-init-list): braces would pick the initializer_list constructor. - std::string execute(PnBig action) { return std::string(static_cast(action.size), 'x'); } + std::string execute(PnBig action) { + pnBigRuns.fetch_add(1); + // NOLINTNEXTLINE(modernize-return-braced-init-list): braces would pick the initializer_list constructor. + return std::string(static_cast(action.size), 'x'); + } }; // NOLINTEND(misc-use-internal-linkage) @@ -538,10 +543,13 @@ TEST_CASE("net_backlog_holds_events_and_drain_sends_resync", "[push][channel]") REQUIRE(reg.kind == "ok"); constexpr int kReplies = 128; + pnBigRuns.store(0); for (int i = 0; i < kReplies; ++i) { client.send(executeEnvelope(static_cast(i + 1), reg.modelId, "PnBig", R"({"size":131072})")); } - // Every reply has been handed to the transport once none is in flight. + // Every execute has run, and then every reply has been handed to the transport once none is in flight. The + // drain alone is not enough: it can find nothing in flight before the server has read every request. + REQUIRE(::morph::testing::waitUntil([] { return pnBigRuns.load() == kReplies; })); REQUIRE(::morph::testing::awaitAnswer([&](auto& owner) { return server->drainedWithin(20s, owner); })); server->post(cid, ::morph::wire::encode(::morph::wire::makeEvent("changes", R"({"held":true})"))); From bd79aa626880874028fa3b88f6da2f9c3d7b4758 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 00:40:17 +0200 Subject: [PATCH 51/68] ui: the explorer shows what an action changes and which records it returns Its Facts gain an effects row (each effect, or reads when the action declares nothing) and, when the result holds records, a records row with each record's key field and paths, so why a screen did or did not update can be read without the source. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- docs/spec/ui/explorer.md | 2 +- include/morph/ui/explorer.hpp | 36 +++++++++++++++++++++++++++++++++++ tests/test_ui_explorer.cpp | 27 ++++++++++++++++++++++++++ 3 files changed, 64 insertions(+), 1 deletion(-) diff --git a/docs/spec/ui/explorer.md b/docs/spec/ui/explorer.md index 0066ee2fd..9a00c2c24 100644 --- a/docs/spec/ui/explorer.md +++ b/docs/spec/ui/explorer.md @@ -77,7 +77,7 @@ Choosing an action (or `select(actionId)`) replaces the right side with, in orde | Section | Test id | What it shows | |---|---|---| | The heading | `explorer_title` | ` / ` | -| Facts | `explorer_facts` | the model; with catalog facts, whether it sends a credential, the server-calculated, secret and list fields; the schema's `x-payloadFingerprint` and `x-payloadShape`; and whether the principal may run it | +| Facts | `explorer_facts` | the model; with catalog facts, whether it sends a credential, the server-calculated, secret and list fields, its effects (`changes Bank_Account`, `updates Lims_Result`, `changes its instance`, `none`, or `reads` when it declares nothing) and the records its result holds, each with its key field and paths (`Lims_Result by id at results`; `keyless`, and `the result` for the result itself); the schema's `x-payloadFingerprint` and `x-payloadShape`; and whether the principal may run it | | Fields | `explorer_fields` | a grid of name, type, required and rules; an unreadable schema as "unreadable schema: " with its first 200 bytes | | Used by | `explorer_uses` | per use, the screen id, the pointer and the kind, with an "Open" button (`explorer_open_`) unless it is the app document | | Try it | `explorer_tryit` | the generated document, mounted into a slot; or why it is not offered (`explorer_tryit_status`) | diff --git a/include/morph/ui/explorer.hpp b/include/morph/ui/explorer.hpp index f9da7706d..41216fc38 100644 --- a/include/morph/ui/explorer.hpp +++ b/include/morph/ui/explorer.hpp @@ -770,9 +770,45 @@ class Explorer final : public ScreenHandle { if (info.listFields) { cells.insert(cells.end(), {"list fields", listed(*info.listFields)}); } + cells.insert(cells.end(), {"effects", effectsText(info)}); + if (!info.records.empty()) { + cells.insert(cells.end(), {"records", recordsText(info)}); + } return cells; } + // What an action declares it changes (spec 8 §5): why a screen did or did not update after it ran. + [[nodiscard]] static std::string effectsText(ActionInfo const& info) { + if (!info.effects) { + return "reads"; + } + std::vector effects; + for (ActionEffect const& effect : *info.effects) { + if (effect.kind == "changesInstance") { + effects.emplace_back("changes its instance"); + } else if (effect.record) { + effects.push_back(effect.kind + " " + *effect.record); + } else { + effects.push_back(effect.kind); + } + } + return detail::explorer::joined(effects, ", "); + } + + // The records an action's result holds, each with its key field and where the result holds it. + [[nodiscard]] static std::string recordsText(ActionInfo const& info) { + std::vector records; + for (ActionRecordInfo const& record : info.records) { + std::vector paths; + for (std::string const& path : record.paths) { + paths.push_back(path.empty() ? std::string{"the result"} : path); + } + std::string const key = record.keyField ? " by " + *record.keyField : std::string{" keyless"}; + records.push_back(record.record + key + " at " + detail::explorer::joined(paths, ", ")); + } + return detail::explorer::joined(records, "; "); + } + // The payload fingerprint and shape a schema carries, as label and value cells. [[nodiscard]] static std::vector payloadFacts(std::string const& schema) { std::vector cells; diff --git a/tests/test_ui_explorer.cpp b/tests/test_ui_explorer.cpp index ced919416..28d50de4b 100644 --- a/tests/test_ui_explorer.cpp +++ b/tests/test_ui_explorer.cpp @@ -97,6 +97,11 @@ CatalogReply twoModels() { ActionInfo capture; capture.modelType = "Lims_SampleModel"; capture.calculatedFields = {"concentration"}; + capture.effects = std::vector{ + morph::ui::ActionEffect{.kind = "changes", .record = "T_Row"}, + morph::ui::ActionEffect{.kind = "changesInstance", .record = std::nullopt}}; + capture.records = {morph::ui::ActionRecordInfo{.record = "T_Row", .keyField = "id", .paths = {"rows"}}, + morph::ui::ActionRecordInfo{.record = "T_Note", .keyField = std::nullopt, .paths = {""}}}; ActionInfo confirm; confirm.modelType = "Lims_SampleModel"; reply.actions = std::map>{{"Lims_CaptureConcentration", capture}, @@ -332,3 +337,25 @@ TEST_CASE("Explorer: an action the catalog has no facts for is decided when it r explorer->requestClose([&closed](bool done) { closed = done; }); CHECK(closed); } + +// Mutation: show `reads` for every action (the declared effects go missing), or drop the key field from a record. +TEST_CASE("facts_show_changes_record_and_records_with_key_and_path", "[ui][explorer][push]") { + Fixture f; + auto const explorer = f.explorer(); + REQUIRE(explorer->select("Lims_CaptureConcentration")); + f.harness.settle(); + CHECK(f.shows("Text", "effects")); + CHECK(f.shows("Text", "changes T_Row, changes its instance")); + CHECK(f.shows("Text", "records")); + CHECK(f.shows("Text", "T_Row by id at rows; T_Note keyless at the result")); +} + +TEST_CASE("facts_show_reads_when_undeclared", "[ui][explorer][push]") { + Fixture f; + auto const explorer = f.explorer(); + REQUIRE(explorer->select("Lims_ConfirmSample")); + f.harness.settle(); + CHECK(f.shows("Text", "effects")); + CHECK(f.shows("Text", "reads")); + CHECK_FALSE(f.shows("Text", "records")); +} From 5ad80d15f3c05c7ffc68eebd63c83e946281e1ca Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 00:52:17 +0200 Subject: [PATCH 52/68] tests: one client's mutation updates another client's screen over net One RemoteServer and SocketServer, two SocketBackend clients, each with a UI session wired to the server's events and watches, mounting one screen whose query declares no refresh. Client A renames a row; A updates from its reply's effects and B from the server's event. With B's events dropped, B's check fails. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- tests/net/CMakeLists.txt | 1 + tests/net/test_push_two_clients.cpp | 230 ++++++++++++++++++++++++++++ 2 files changed, 231 insertions(+) create mode 100644 tests/net/test_push_two_clients.cpp diff --git a/tests/net/CMakeLists.txt b/tests/net/CMakeLists.txt index 589a41876..741709959 100644 --- a/tests/net/CMakeLists.txt +++ b/tests/net/CMakeLists.txt @@ -8,6 +8,7 @@ add_executable(morph_net_tests test_socket_server.cpp test_socket_backend.cpp test_push_net.cpp + test_push_two_clients.cpp ) target_link_libraries(morph_net_tests diff --git a/tests/net/test_push_two_clients.cpp b/tests/net/test_push_two_clients.cpp new file mode 100644 index 000000000..822e43b13 --- /dev/null +++ b/tests/net/test_push_two_clients.cpp @@ -0,0 +1,230 @@ +// SPDX-License-Identifier: Apache-2.0 + +// Server push end to end (spec 8): one server, two clients over the `net` transport, each with a UI session whose +// screen lists rows through a query and declares no refresh. Client A's mutation reaches client B's screen as an +// event, and B refetches, with no `refreshEvery`. + +// clang-format off +#include +// clang-format on + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +// NOLINTBEGIN(misc-use-internal-linkage): registered types must have external linkage. +namespace two_clients { + +struct Row { + std::int64_t id = 0; + std::string name; +}; +struct ListRows {}; +struct EditRow { + std::int64_t id = 0; + std::string name; +}; + +// The rows every instance reads and writes: each client's connection has its own model instance. +struct Store { + std::mutex mutex; + std::vector rows; +}; +inline Store& store() { + static Store shared; + return shared; +} + +struct RowsModel { + std::vector execute(ListRows const& /*action*/) { + std::scoped_lock const lock{store().mutex}; + return store().rows; + } + Row execute(EditRow const& action) { + std::scoped_lock const lock{store().mutex}; + for (Row& row : store().rows) { + if (row.id == action.id) { + row.name = action.name; + return row; + } + } + return Row{}; + } +}; + +} // namespace two_clients +// NOLINTEND(misc-use-internal-linkage) + +BRIDGE_REGISTER_MODEL(two_clients::RowsModel, "Tc_RowsModel") +BRIDGE_REGISTER_ACTION(two_clients::RowsModel, two_clients::ListRows, "Tc_ListRows") +BRIDGE_REGISTER_ACTION(two_clients::RowsModel, two_clients::EditRow, "Tc_EditRow") +BRIDGE_REGISTER_RECORD(two_clients::Row, "Tc_Row") +BRIDGE_REGISTER_EFFECTS(two_clients::EditRow, morph::effects::changes) + +namespace { + +using namespace std::chrono_literals; +namespace ui = morph::ui; + +constexpr std::string_view kHello = "t-two-hello"; + +// Negotiates events for the connection its hello arrives on, as the UI service's `ui-hello` does. +class Hello final : public morph::backend::IEnvelopeService { +public: + using IEnvelopeService::handle; + + [[nodiscard]] bool serves(std::string_view kind) const override { return kind == kHello; } + + [[nodiscard]] std::string handle(morph::wire::Envelope const& request, + morph::session::IAuthorizer const& /*authorizer*/) override { + return morph::wire::encode(morph::wire::makeOk(request.callId)); + } + + [[nodiscard]] std::string handle(morph::wire::Envelope const& request, + morph::session::IAuthorizer const& authorizer, + morph::backend::ConnectionFacts& facts) override { + facts.events = true; + return handle(request, authorizer); + } +}; + +// The screen: the rows, by a query, as `id:name`; an input that renames row 1; no refresh declarations. +constexpr std::string_view kRows = R"({ + "vocab": "ui/1", "id": "rows", + "models": { "table": { "type": "Tc_RowsModel" } }, + "queries": { "rows": { "model": "table", "action": "Tc_ListRows" } }, + "mutations": { "rename": { "model": "table", "action": "Tc_EditRow" } }, + "view": { "kind": "column", "children": [ + { "kind": "textInput", "testId": "rename", "onSubmit": { "run": "rename", "body": { "obj": { "id": 1, "name": { "ref": "event" } } } } }, + { "kind": "forEach", "rows": { "query": "rows" }, "key": { "row": "id" }, + "children": [ { "kind": "text", "text": { "fn": "format", "args": [ "{0}:{1}", { "row": "id" }, { "row": "name" } ] } } ] } ] } })"; + +ui::Catalog catalog() { + ui::author::CatalogFacts facts; + facts.action(); + facts.action(); + return facts.catalog(); +} + +// One client: a socket backend under a bridge, a UI session wired to the server's events and watches, and the +// screen mounted on a recording view. +struct Client { + explicit Client(std::string const& url) { + auto made = std::make_unique(url); + socket = made.get(); + REQUIRE(socket->waitForConnected()); + exchange = ui::exchangeOver(*socket, owner); + bridge = std::make_unique(std::move(made), owner); + dispatcher = std::make_unique(*bridge, &owner); + ui::SessionOptions options; + options.catalog = catalog(); + options.watcher = ui::watcherOver(exchange, [] { return morph::session::Context{}; }, &owner); + session = std::make_unique(runtime, std::move(options)); + socket->setEventHandler([this](morph::wire::EventBody const& event) { session->deliverEvent(event); }, &owner); + bool negotiated = false; + morph::wire::Envelope hello; + hello.kind = std::string{kHello}; + exchange(std::move(hello), [&negotiated](ui::ExchangeReply const& reply) { + negotiated = reply.has_value() && reply->kind == "ok"; + }); + REQUIRE(pumpUntil([&] { return negotiated; })); + interpreter = std::make_unique(*session, *dispatcher, ui::InterpreterOptions{}); + auto loaded = ui::loadDocumentText(kRows, ui::LoadOptions{.catalog = catalog()}); + if (!loaded) { + FAIL(loaded.error().message()); + } + REQUIRE(loaded->warnings.empty()); + auto mounted = interpreter->mount(std::make_shared(std::move(*loaded)), {}, view, {}); + REQUIRE(mounted.has_value()); + screen = std::move(*mounted); + } + + // Runs the owner's tasks until @p done holds, or 5 s pass. + template + bool pumpUntil(Done done) { + auto const deadline = std::chrono::steady_clock::now() + 5s; + while (!done()) { + if (std::chrono::steady_clock::now() > deadline) { + return false; + } + if (!owner.runOnce()) { + std::this_thread::sleep_for(1ms); + } + } + return true; + } + + [[nodiscard]] std::vector rows() const { + std::vector texts; + for (int const id : view.all("Text")) { + texts.push_back(view.prop(id, "text")); + } + return texts; + } + + morph::exec::MainThreadExecutor owner; + morph::reactive::Runtime runtime{owner}; + morph::net::SocketBackend* socket = nullptr; + ui::Exchange exchange; + std::unique_ptr bridge; + std::unique_ptr dispatcher; + std::unique_ptr session; + std::unique_ptr interpreter; + ui::testing::RecordingBackend view; + std::unique_ptr screen; +}; + +} // namespace + +// Control (run by hand): with B's event handler dropping the events, A still updates from its reply's effects and +// B never sees the rename, so the last check fails. +TEST_CASE("one_clients_mutation_updates_another_clients_screen_over_net", "[push][e2e]") { + { + std::scoped_lock const lock{two_clients::store().mutex}; + two_clients::store().rows = {two_clients::Row{.id = 1, .name = "one"}, + two_clients::Row{.id = 2, .name = "two"}}; + } + morph::exec::ThreadPoolExecutor pool{2}; + morph::backend::ServerConfig config; + config.services.push_back(std::make_shared()); + auto server = std::make_shared(pool, std::move(config)); + morph::net::SocketServer wsServer{*server, 0}; + REQUIRE(wsServer.listen()); + std::string const url = "ws://127.0.0.1:" + std::to_string(static_cast(wsServer.port())); + + Client a{url}; + Client b{url}; + std::vector const before{"1:one", "2:two"}; + REQUIRE(a.pumpUntil([&] { return a.rows() == before; })); + REQUIRE(b.pumpUntil([&] { return b.rows() == before; })); + + a.view.submit(*a.view.find("TextInput", "testId", "rename"), "renamed"); + std::vector const after{"1:renamed", "2:two"}; + // A refetches from its own reply's effects; B from the server's event. + REQUIRE(a.pumpUntil([&] { return a.rows() == after; })); + CHECK(b.pumpUntil([&] { return b.rows() == after; })); +} From cf7123c352eaa9139fe6fa2ac6e0bdef8b982c1e Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 01:48:07 +0200 Subject: [PATCH 53/68] examples(bookmarks): actions declare their effects; screens drop refresh declarations Bookmarks_Bookmark, Bookmarks_BookmarkDetail and Bookmarks_Tag are records with private keys, and every action declares what it changes. The screens lose their invalidates and their refetch-only onSuccess commands. A shared testkit helper loads every document of a served bundle against its catalog facts, and each rung asserts it loads without refusals or warnings. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- docs/spec/testing_strategy.md | 12 ++++ examples/bookmarks/README.md | 5 ++ .../include/bookmarks/dto/bookmark_dto.hpp | 6 ++ .../include/bookmarks/dto/tag_dto.hpp | 4 ++ .../include/bookmarks/models/auth_model.hpp | 3 + .../bookmarks/models/bookmark_model.hpp | 28 +++++++++ .../include/bookmarks/models/tag_model.hpp | 9 +++ .../bookmarks/screens/bookmarks_screens.cpp | 23 +++---- .../screens/tests/test_bookmarks_screens.cpp | 14 +++++ examples/common/testkit/bundle_warnings.hpp | 62 +++++++++++++++++++ 10 files changed, 150 insertions(+), 16 deletions(-) create mode 100644 examples/common/testkit/bundle_warnings.hpp diff --git a/docs/spec/testing_strategy.md b/docs/spec/testing_strategy.md index 88b0bb4b8..df9828bbb 100644 --- a/docs/spec/testing_strategy.md +++ b/docs/spec/testing_strategy.md @@ -16,6 +16,7 @@ behind its own CMake option, default `OFF`, so an ordinary `cmake --build` - [Load / latency benchmark](#load--latency-benchmark-testsbench) - [Adversarial cross-socket run](#adversarial-cross-socket-run-testsqttest_qt_websocket_adversarialcpp) - [Install / export consumability](#install--export-consumability-scriptscheck_install_exportsh) +- [Rung bundles load without warnings](#rung-bundles-load-without-warnings) - [Declined techniques, and why](#declined-techniques-and-why) - [Cross-references](#cross-references) @@ -596,6 +597,17 @@ defect into a scratch copy of the tree one at a time and requires the checker to catch each one *for the stated reason* — the same "test the checker first" pattern as the deprecation-marker and test-type-name lints. +## Rung bundles load without warnings + +Each rung's screen tests hold one case, `_bundle_loads_without_warnings` (`bookmarks`, `bank`, `lims`), in +`examples//screens/tests/`. It builds the bundle the rung's server serves and loads every screen and app +document in it against the catalog facts the bundle carries, which include each action's effects and records +(`examples/common/testkit/bundle_warnings.hpp`). The case fails on any refusal and on any load-time warning +(`Document::warnings`): a mutation whose action declares no effects and after which nothing refreshes, a refresh +declaration the effects already cover, or a watch of a query that runs a mutation with effects. It is what keeps the +rungs' actions declaring their effects and their screens free of refresh declarations. It runs in the ordinary +`ctest -L ladder` sweep. + ## Declined techniques, and why Kept from `testing_charter.md`, which was removed on 2026-09-23 along with the diff --git a/examples/bookmarks/README.md b/examples/bookmarks/README.md index c92a289a7..f9322edd1 100644 --- a/examples/bookmarks/README.md +++ b/examples/bookmarks/README.md @@ -55,6 +55,11 @@ only meaningful against the server. - **CreateBookmark, EditBookmark, ImportBookmarks, RenameTag and MergeTags** are `form` nodes. - **The list** is a table with multiple selection. Archiving the selection is one `BulkEdit`. +Each action declares what it changes beside its registration (`BRIDGE_REGISTER_EFFECTS` in +`include/bookmarks/models/`), and the rows it changes are records registered beside their types +(`Bookmarks_Bookmark`, `Bookmarks_BookmarkDetail`, `Bookmarks_Tag`). A reply refreshes every query holding a changed +record, so the screens declare no refreshes. `bookmarks_bundle_loads_without_warnings` keeps it so. + In process, `screens::prepareLocal()` installs a development token issuer, which a server installs at start-up. `morph_examples_client --app bookmarks` runs it (`--db`, default `bookmarks.db`). ## Reference implementations diff --git a/examples/bookmarks/include/bookmarks/dto/bookmark_dto.hpp b/examples/bookmarks/include/bookmarks/dto/bookmark_dto.hpp index 33196f5a2..975f81790 100644 --- a/examples/bookmarks/include/bookmarks/dto/bookmark_dto.hpp +++ b/examples/bookmarks/include/bookmarks/dto/bookmark_dto.hpp @@ -3,6 +3,7 @@ #include #include +#include #include #include #include @@ -249,3 +250,8 @@ struct glz::meta { using enum bookmarks::ArchiveFilter; static constexpr auto value = glz::enumerate(Any, ActiveOnly, ArchivedOnly); }; + +// A bookmark is one row shown two ways: a list's summary and the edit drawer's full view. Every list is filtered to +// its owner's rows, so neither row's key reaches another user. +BRIDGE_REGISTER_RECORD(bookmarks::BookmarkSummary, "Bookmarks_Bookmark", ::morph::record::privateKeys) +BRIDGE_REGISTER_RECORD(bookmarks::BookmarkView, "Bookmarks_BookmarkDetail", ::morph::record::privateKeys) diff --git a/examples/bookmarks/include/bookmarks/dto/tag_dto.hpp b/examples/bookmarks/include/bookmarks/dto/tag_dto.hpp index 6d238dd04..767185b71 100644 --- a/examples/bookmarks/include/bookmarks/dto/tag_dto.hpp +++ b/examples/bookmarks/include/bookmarks/dto/tag_dto.hpp @@ -2,6 +2,7 @@ #pragma once #include +#include #include #include @@ -66,3 +67,6 @@ struct ListTagsResult { }; } // namespace bookmarks + +// Tags are per user, like the bookmarks that carry them. +BRIDGE_REGISTER_RECORD(bookmarks::TagSummary, "Bookmarks_Tag", ::morph::record::privateKeys) diff --git a/examples/bookmarks/include/bookmarks/models/auth_model.hpp b/examples/bookmarks/include/bookmarks/models/auth_model.hpp index e17d9a19d..253270777 100644 --- a/examples/bookmarks/include/bookmarks/models/auth_model.hpp +++ b/examples/bookmarks/include/bookmarks/models/auth_model.hpp @@ -2,6 +2,7 @@ #pragma once #include +#include #include #include "bookmarks/core/errors.hpp" @@ -39,3 +40,5 @@ BRIDGE_REGISTER_MODEL(bookmarks::AuthModel, "Bookmarks_AuthModel") // its result carries a live bearer token — neither belongs in a durable, // replayable action log. BRIDGE_REGISTER_ACTION(bookmarks::AuthModel, bookmarks::Login, "Login", ::morph::model::Loggable::No) +// Signing in changes no row a screen shows: the screen that runs it signs the session in, which reloads the catalog. +BRIDGE_REGISTER_EFFECTS(bookmarks::Login, ::morph::effects::none) diff --git a/examples/bookmarks/include/bookmarks/models/bookmark_model.hpp b/examples/bookmarks/include/bookmarks/models/bookmark_model.hpp index 417bd233d..d0483e569 100644 --- a/examples/bookmarks/include/bookmarks/models/bookmark_model.hpp +++ b/examples/bookmarks/include/bookmarks/models/bookmark_model.hpp @@ -2,12 +2,14 @@ #pragma once #include +#include #include #include "bookmarks/core/errors.hpp" #include "bookmarks/dto/bookmark_dto.hpp" #include "bookmarks/dto/bulk_dto.hpp" #include "bookmarks/dto/import_export_dto.hpp" +#include "bookmarks/dto/tag_dto.hpp" /// @file /// `BookmarkModel` — every action this rung's one entity-owning model @@ -69,11 +71,27 @@ class BookmarkModel { } // namespace bookmarks BRIDGE_REGISTER_MODEL(bookmarks::BookmarkModel, "BookmarkModel") +// A bookmark's tags are counted by the tag list, so whatever adds, removes or retags a bookmark changes the tags too. BRIDGE_REGISTER_ACTION(bookmarks::BookmarkModel, bookmarks::CreateBookmark, "CreateBookmark") +BRIDGE_REGISTER_EFFECTS(bookmarks::CreateBookmark, ::morph::effects::changes, + ::morph::effects::changes) +// The reply is the edited bookmark's full view; its summary in a list differs in shape, so the lists refetch. BRIDGE_REGISTER_ACTION(bookmarks::BookmarkModel, bookmarks::EditBookmark, "EditBookmark") +BRIDGE_REGISTER_EFFECTS(bookmarks::EditBookmark, ::morph::effects::updates, + ::morph::effects::changes, + ::morph::effects::changes) BRIDGE_REGISTER_ACTION(bookmarks::BookmarkModel, bookmarks::ArchiveBookmark, "ArchiveBookmark") +BRIDGE_REGISTER_EFFECTS(bookmarks::ArchiveBookmark, ::morph::effects::changes, + ::morph::effects::changes, + ::morph::effects::changes) BRIDGE_REGISTER_ACTION(bookmarks::BookmarkModel, bookmarks::UnarchiveBookmark, "UnarchiveBookmark") +BRIDGE_REGISTER_EFFECTS(bookmarks::UnarchiveBookmark, ::morph::effects::changes, + ::morph::effects::changes, + ::morph::effects::changes) BRIDGE_REGISTER_ACTION(bookmarks::BookmarkModel, bookmarks::DeleteBookmark, "DeleteBookmark") +BRIDGE_REGISTER_EFFECTS(bookmarks::DeleteBookmark, ::morph::effects::changes, + ::morph::effects::changes, + ::morph::effects::changes) BRIDGE_REGISTER_ACTION(bookmarks::BookmarkModel, bookmarks::GetBookmark, "GetBookmark") BRIDGE_REGISTER_ACTION(bookmarks::BookmarkModel, bookmarks::ListBookmarks, "ListBookmarks", ::morph::model::Loggable::No) @@ -83,7 +101,17 @@ BRIDGE_REGISTER_ACTION(bookmarks::BookmarkModel, bookmarks::GetChangesSince, "Ge // framework's own auto-append never double-logs alongside the model's own // outbox write. BRIDGE_REGISTER_ACTION(bookmarks::BookmarkModel, bookmarks::BulkEdit, "BulkEdit", ::morph::model::Loggable::No) +BRIDGE_REGISTER_EFFECTS(bookmarks::BulkEdit, ::morph::effects::changes, + ::morph::effects::changes, + ::morph::effects::changes) +// The metadata worker fills in a fetched title, which every list showing the bookmark then shows. BRIDGE_REGISTER_ACTION(bookmarks::BookmarkModel, bookmarks::RecordMetadata, "RecordMetadata") +BRIDGE_REGISTER_EFFECTS(bookmarks::RecordMetadata, ::morph::effects::changes, + ::morph::effects::changes) BRIDGE_REGISTER_ACTION(bookmarks::BookmarkModel, bookmarks::ImportBookmarks, "ImportBookmarks") +BRIDGE_REGISTER_EFFECTS(bookmarks::ImportBookmarks, ::morph::effects::changes, + ::morph::effects::changes) +// Export reads, but a screen runs it as a mutation, on a button, rather than fetching the file on every visit. BRIDGE_REGISTER_ACTION(bookmarks::BookmarkModel, bookmarks::ExportBookmarks, "ExportBookmarks", ::morph::model::Loggable::No) +BRIDGE_REGISTER_EFFECTS(bookmarks::ExportBookmarks, ::morph::effects::none) diff --git a/examples/bookmarks/include/bookmarks/models/tag_model.hpp b/examples/bookmarks/include/bookmarks/models/tag_model.hpp index d4ef0a96a..102af9245 100644 --- a/examples/bookmarks/include/bookmarks/models/tag_model.hpp +++ b/examples/bookmarks/include/bookmarks/models/tag_model.hpp @@ -2,9 +2,11 @@ #pragma once #include +#include #include #include "bookmarks/core/errors.hpp" +#include "bookmarks/dto/bookmark_dto.hpp" #include "bookmarks/dto/tag_dto.hpp" namespace bookmarks { @@ -26,8 +28,15 @@ class TagModel { } // namespace bookmarks BRIDGE_REGISTER_MODEL(bookmarks::TagModel, "TagModel") +// Renaming or merging a tag cascades to every bookmark carrying it, so the bookmarks change too. BRIDGE_REGISTER_ACTION(bookmarks::TagModel, bookmarks::RenameTag, "RenameTag") +BRIDGE_REGISTER_EFFECTS(bookmarks::RenameTag, ::morph::effects::changes, + ::morph::effects::changes, + ::morph::effects::changes) // MergeTags is outbox-managed (this task) -- Loggable::No so the framework // auto-append never double-logs alongside the model's own outbox write. BRIDGE_REGISTER_ACTION(bookmarks::TagModel, bookmarks::MergeTags, "MergeTags", ::morph::model::Loggable::No) +BRIDGE_REGISTER_EFFECTS(bookmarks::MergeTags, ::morph::effects::changes, + ::morph::effects::changes, + ::morph::effects::changes) BRIDGE_REGISTER_ACTION(bookmarks::TagModel, bookmarks::ListTags, "ListTags", ::morph::model::Loggable::No) diff --git a/examples/bookmarks/screens/bookmarks_screens.cpp b/examples/bookmarks/screens/bookmarks_screens.cpp index 16fff3540..f74c3f957 100644 --- a/examples/bookmarks/screens/bookmarks_screens.cpp +++ b/examples/bookmarks/screens/bookmarks_screens.cpp @@ -103,12 +103,9 @@ ui::Screen bookmarksScreen() { {}); auto const detail = screen.query("detail", marks, {{&GetBookmark::id, editing}}, {.when = ui::engaged(editing)}); - auto const archive = screen.mutation("archive", marks, {.invalidates = {bookmarks, tags}}); - auto const remove = screen.mutation("remove", marks, - {.body = {{&DeleteBookmark::id, editing}}, - .invalidates = {bookmarks, tags}, - .onSuccess = {ui::close(edit)}}); - auto const refreshed = ui::Commands{ui::refetch(bookmarks), ui::refetch(tags)}; + auto const archive = screen.mutation("archive", marks); + auto const remove = screen.mutation( + "remove", marks, {.body = {{&DeleteBookmark::id, editing}}, .onSuccess = {ui::close(edit)}}); auto const rows = bookmarks.value().at(&ListBookmarksResult::bookmarks); auto const knownTags = ui::map(tags.value().at(&ListTagsResult::tags), "t", [](ui::Handle const& entry) { @@ -171,8 +168,7 @@ ui::Screen bookmarksScreen() { .role = std::string{"error"}}) .testId("error"), ui::drawer(create, - {ui::form( - marks, {.onSuccess = {ui::close(create), ui::refetch(bookmarks), ui::refetch(tags)}}) + {ui::form(marks, {.onSuccess = {ui::close(create)}}) .id(ui::element("createForm")) .testId("createForm")}, {.title = "New bookmark", .side = std::string{"end"}}) @@ -180,9 +176,7 @@ ui::Screen bookmarksScreen() { ui::drawer( edit, {ui::form(marks, - {.prefill = detail.value(), - .key = editing, - .onSuccess = {ui::close(edit), ui::refetch(bookmarks), ui::refetch(tags)}}) + {.prefill = detail.value(), .key = editing, .onSuccess = {ui::close(edit)}}) .id(ui::element("editForm")) .testId("editForm"), ui::text({.text = "Every field is saved as shown: a field left empty is cleared.", @@ -205,7 +199,6 @@ ui::Screen tagsScreen() { auto const tags = screen.query("tags", tagModel, {}); // A single-selection table's selection is a list of at most one key. auto const picked = ui::element("tagList"); - auto const refreshed = ui::Commands{ui::refetch(tags)}; return ui::row( {ui::panel( {ui::table( @@ -226,8 +219,7 @@ ui::Screen tagsScreen() { .role = std::string{"muted"}}), ui::form( tagModel, {.prefill = ui::obj({{"id", ui::call("first", {picked.selection()})}}), - .fields = {{&RenameTag::id, {.hidden = true}}}, - .onSuccess = refreshed}) + .fields = {{&RenameTag::id, {.hidden = true}}}}) .id(ui::element("renameForm")) .testId("renameForm")}, {.title = "Rename the selected tag", .padding = 5}), @@ -237,8 +229,7 @@ ui::Screen tagsScreen() { .role = std::string{"muted"}}), ui::form( tagModel, {.prefill = ui::obj({{"sourceId", ui::call("first", {picked.selection()})}}), - .fields = {{&MergeTags::sourceId, {.hidden = true}}}, - .onSuccess = refreshed}) + .fields = {{&MergeTags::sourceId, {.hidden = true}}}}) .id(ui::element("mergeForm")) .testId("mergeForm")}, {.title = "Merge the selected tag", .padding = 5})}, diff --git a/examples/bookmarks/screens/tests/test_bookmarks_screens.cpp b/examples/bookmarks/screens/tests/test_bookmarks_screens.cpp index e23a5ab89..68959e0e7 100644 --- a/examples/bookmarks/screens/tests/test_bookmarks_screens.cpp +++ b/examples/bookmarks/screens/tests/test_bookmarks_screens.cpp @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -66,6 +67,19 @@ TEST_CASE("bookmarks screens: the screen validates, and the bundle builds", "[bo REQUIRE(bundle.has_value()); } +// Every document loads against the catalog the server serves with no warning: each mutation's action declares its +// effects (or `none`), and no refresh declaration repeats what the effects already cover. +TEST_CASE("bookmarks_bundle_loads_without_warnings", "[bookmarks-screens]") { + auto const bundle = bookmarks::screens::bundle(); + INFO((bundle ? std::string{} : bundle.error().message())); + REQUIRE(bundle.has_value()); + std::vector const findings = morph::examples::testing::bundleLoadFindings(**bundle); + for (std::string const& finding : findings) { + UNSCOPED_INFO(finding); + } + CHECK(findings.empty()); +} + // Mutation: start the queries before signing in (they fail without a principal). TEST_CASE("bookmarks screens: signing in starts the queries, and a created bookmark is listed", "[bookmarks-screens]") { diff --git a/examples/common/testkit/bundle_warnings.hpp b/examples/common/testkit/bundle_warnings.hpp new file mode 100644 index 000000000..a52255568 --- /dev/null +++ b/examples/common/testkit/bundle_warnings.hpp @@ -0,0 +1,62 @@ +// SPDX-License-Identifier: Apache-2.0 +#pragma once + +#include +#include +#include +#include +#include +#include +#include + +/// @file +/// Loads every screen and app document of a served bundle against the catalog facts the bundle serves, as a client +/// does, and lists what the loader reports: each refusal and each load-time warning (`Document::warnings`). A rung's +/// test asserts the list is empty, so a mutation that refreshes nothing, a refresh declaration its action's effects +/// already cover, or a query whose action declares effects fails the rung. + +namespace morph::examples::testing { + +/// @brief The refusals and warnings of loading every screen and app document of @p bundle. +/// @param bundle The bundle, as the rung's server serves it. +/// @return One line per refusal or warning, naming its document; empty when every document loads cleanly. +[[nodiscard]] inline std::vector bundleLoadFindings(morph::ui::ServedBundle const& bundle) { + morph::ui::Catalog catalog; + if (bundle.models) { + catalog.model = [models = *bundle.models](std::string_view type) -> std::optional { + auto const found = models.find(type); + return found == models.end() ? std::nullopt : std::optional{found->second}; + }; + } + if (bundle.actions) { + catalog.action = [actions = *bundle.actions](std::string_view id) -> std::optional { + auto const found = actions.find(id); + return found == actions.end() ? std::nullopt : std::optional{found->second}; + }; + } + std::vector findings; + if (!catalog.action) { + findings.emplace_back("the bundle carries no action facts, so nothing could be checked"); + return findings; + } + for (auto const& [hash, file] : bundle.files) { + bool document = false; + for (morph::ui::ServedFile::Holder const& holder : file.holders) { + document = document || holder.kind == morph::ui::kScreenFile || holder.kind == morph::ui::kAppFile; + } + if (!document) { + continue; + } + auto const loaded = morph::ui::loadDocumentText(file.bytes, morph::ui::LoadOptions{.catalog = catalog}); + if (!loaded) { + findings.push_back(loaded.error().message()); + continue; + } + for (morph::ui::LoadWarning const& warning : loaded->warnings) { + findings.push_back(warning.message(loaded->id)); + } + } + return findings; +} + +} // namespace morph::examples::testing From 6e5cc10cfd6e0ef7279fea17afca89352c4dc267 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 01:48:07 +0200 Subject: [PATCH 54/68] examples(bank): actions declare their effects; screens drop refresh declarations Accounts, transactions, cards, payees, loans, payments, budgets and notifications are records with private keys, declared identically for both model sets; Bank_SpendingReport is derived from transactions. The screens lose their invalidates and refetch-only onSuccess commands. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- examples/bank/README.md | 6 ++++ .../bank/include/bank/dto/account_dto.hpp | 4 +++ examples/bank/include/bank/dto/budget_dto.hpp | 9 +++++ examples/bank/include/bank/dto/card_dto.hpp | 4 +++ examples/bank/include/bank/dto/loan_dto.hpp | 4 +++ .../include/bank/dto/notification_dto.hpp | 4 +++ examples/bank/include/bank/dto/payee_dto.hpp | 4 +++ .../bank/include/bank/dto/payment_dto.hpp | 4 +++ .../bank/include/bank/dto/transaction_dto.hpp | 4 +++ .../include/bank/models/account_model.hpp | 2 ++ .../bank/include/bank/models/auth_model.hpp | 5 +++ .../bank/include/bank/models/budget_model.hpp | 3 ++ .../bank/include/bank/models/card_model.hpp | 8 +++++ .../include/bank/models/customer_model.hpp | 2 ++ .../bank/include/bank/models/loan_model.hpp | 11 ++++++ .../bank/models/notification_model.hpp | 4 +++ .../bank/include/bank/models/payee_model.hpp | 3 ++ .../include/bank/models/payment_model.hpp | 10 ++++++ .../include/bank/models/transaction_model.hpp | 10 ++++++ .../include/bank/models/account_model.hpp | 2 ++ .../memory/include/bank/models/auth_model.hpp | 5 +++ .../memory/include/bank/models/card_model.hpp | 8 +++++ .../include/bank/models/customer_model.hpp | 2 ++ .../memory/include/bank/models/loan_model.hpp | 11 ++++++ .../include/bank/models/payee_model.hpp | 3 ++ .../include/bank/models/payment_model.hpp | 10 ++++++ .../include/bank/models/transaction_model.hpp | 10 ++++++ examples/bank/screens/bank_screens.cpp | 36 +++++++------------ .../bank/screens/tests/test_bank_screens.cpp | 14 ++++++++ 29 files changed, 179 insertions(+), 23 deletions(-) diff --git a/examples/bank/README.md b/examples/bank/README.md index e577b73be..d1f1b4158 100644 --- a/examples/bank/README.md +++ b/examples/bank/README.md @@ -325,6 +325,12 @@ C++ builders: - **`bank.cards`**, **`bank.payees`** and **`bank.loans`**: lists with their actions and forms. A loan's schedule is the model's `LoanScheduleRequest`, shown through the `LoanSchedule` component. +Each action declares what it changes beside its registration (`BRIDGE_REGISTER_EFFECTS`), identically in both model +sets (`include/bank/models/` and `memory/include/bank/models/`); the records are registered beside their DTOs +(`Bank_Account`, `Bank_Transaction`, `Bank_SpendingReport` derived from transactions, and one per other list). A +reply refreshes every query holding a changed record, so the screens declare no refreshes. +`bank_bundle_loads_without_warnings` keeps it so. + Each signed-in screen has the sidebar, which also signs out. `AccountCards`, `TransferFlow` and `LoanSchedule` are host components; their built-in fallbacks are what a client without them shows. Amounts are entered in cents: bank's actions take minor units. diff --git a/examples/bank/include/bank/dto/account_dto.hpp b/examples/bank/include/bank/dto/account_dto.hpp index 1c119f93b..be1da12cf 100644 --- a/examples/bank/include/bank/dto/account_dto.hpp +++ b/examples/bank/include/bank/dto/account_dto.hpp @@ -2,6 +2,7 @@ #pragma once #include +#include #include #include @@ -65,3 +66,6 @@ struct CloseAccount { }; } // namespace bank::dto + +// Every list is filtered to the signed-in customer's own rows, so a row's key never reaches another customer. +BRIDGE_REGISTER_RECORD(bank::dto::AccountInfo, "Bank_Account", ::morph::record::privateKeys) diff --git a/examples/bank/include/bank/dto/budget_dto.hpp b/examples/bank/include/bank/dto/budget_dto.hpp index 87e80c62b..a00ebcb2d 100644 --- a/examples/bank/include/bank/dto/budget_dto.hpp +++ b/examples/bank/include/bank/dto/budget_dto.hpp @@ -2,9 +2,12 @@ #pragma once #include +#include #include #include +#include "bank/dto/transaction_dto.hpp" + /// @file /// Wire DTOs for the Budget model: per-category monthly limits plus simple /// spending analytics derived from the ledger. @@ -65,3 +68,9 @@ struct SpendingReport { }; } // namespace bank::dto + +// Every list is filtered to the signed-in customer's own rows, so a row's key never reaches another customer. +BRIDGE_REGISTER_RECORD(bank::dto::BudgetInfo, "Bank_Budget", ::morph::record::privateKeys) +// A spending report sums the ledger, so whatever adds a transaction changes every report. +BRIDGE_REGISTER_RECORD(bank::dto::SpendingReport, "Bank_SpendingReport", ::morph::record::keyless, + ::morph::record::derivedFrom) diff --git a/examples/bank/include/bank/dto/card_dto.hpp b/examples/bank/include/bank/dto/card_dto.hpp index 1dba2a931..25bebbc91 100644 --- a/examples/bank/include/bank/dto/card_dto.hpp +++ b/examples/bank/include/bank/dto/card_dto.hpp @@ -2,6 +2,7 @@ #pragma once #include +#include #include #include @@ -70,3 +71,6 @@ struct CardList { }; } // namespace bank::dto + +// Every list is filtered to the signed-in customer's own rows, so a row's key never reaches another customer. +BRIDGE_REGISTER_RECORD(bank::dto::CardInfo, "Bank_Card", ::morph::record::privateKeys) diff --git a/examples/bank/include/bank/dto/loan_dto.hpp b/examples/bank/include/bank/dto/loan_dto.hpp index cafb79c8b..a58076ca7 100644 --- a/examples/bank/include/bank/dto/loan_dto.hpp +++ b/examples/bank/include/bank/dto/loan_dto.hpp @@ -2,6 +2,7 @@ #pragma once #include +#include #include #include @@ -83,3 +84,6 @@ struct LoanScheduleResult { }; } // namespace bank::dto + +// Every list is filtered to the signed-in customer's own rows, so a row's key never reaches another customer. +BRIDGE_REGISTER_RECORD(bank::dto::LoanInfo, "Bank_Loan", ::morph::record::privateKeys) diff --git a/examples/bank/include/bank/dto/notification_dto.hpp b/examples/bank/include/bank/dto/notification_dto.hpp index ee7f56e58..e7086586e 100644 --- a/examples/bank/include/bank/dto/notification_dto.hpp +++ b/examples/bank/include/bank/dto/notification_dto.hpp @@ -2,6 +2,7 @@ #pragma once #include +#include #include #include @@ -51,3 +52,6 @@ struct MarkAllRead { }; } // namespace bank::dto + +// Every list is filtered to the signed-in customer's own rows, so a row's key never reaches another customer. +BRIDGE_REGISTER_RECORD(bank::dto::NotificationInfo, "Bank_Notification", ::morph::record::privateKeys) diff --git a/examples/bank/include/bank/dto/payee_dto.hpp b/examples/bank/include/bank/dto/payee_dto.hpp index c6034e61f..f2847cc66 100644 --- a/examples/bank/include/bank/dto/payee_dto.hpp +++ b/examples/bank/include/bank/dto/payee_dto.hpp @@ -3,6 +3,7 @@ #include #include +#include #include #include @@ -66,3 +67,6 @@ struct PayeeList { }; } // namespace bank::dto + +// Every list is filtered to the signed-in customer's own rows, so a row's key never reaches another customer. +BRIDGE_REGISTER_RECORD(bank::dto::PayeeInfo, "Bank_Payee", ::morph::record::privateKeys) diff --git a/examples/bank/include/bank/dto/payment_dto.hpp b/examples/bank/include/bank/dto/payment_dto.hpp index 26ab4e816..f9054146d 100644 --- a/examples/bank/include/bank/dto/payment_dto.hpp +++ b/examples/bank/include/bank/dto/payment_dto.hpp @@ -2,6 +2,7 @@ #pragma once #include +#include #include #include @@ -77,3 +78,6 @@ struct PaymentList { }; } // namespace bank::dto + +// Every list is filtered to the signed-in customer's own rows, so a row's key never reaches another customer. +BRIDGE_REGISTER_RECORD(bank::dto::PaymentInfo, "Bank_Payment", ::morph::record::privateKeys) diff --git a/examples/bank/include/bank/dto/transaction_dto.hpp b/examples/bank/include/bank/dto/transaction_dto.hpp index 88e3803a7..b3c73a036 100644 --- a/examples/bank/include/bank/dto/transaction_dto.hpp +++ b/examples/bank/include/bank/dto/transaction_dto.hpp @@ -3,6 +3,7 @@ #include #include +#include #include #include #include @@ -83,3 +84,6 @@ struct HistoryPage { }; } // namespace bank::dto + +// Every list is filtered to the signed-in customer's own rows, so a row's key never reaches another customer. +BRIDGE_REGISTER_RECORD(bank::dto::TxnInfo, "Bank_Transaction", ::morph::record::privateKeys) diff --git a/examples/bank/include/bank/models/account_model.hpp b/examples/bank/include/bank/models/account_model.hpp index 0cb5934e7..cd7965549 100644 --- a/examples/bank/include/bank/models/account_model.hpp +++ b/examples/bank/include/bank/models/account_model.hpp @@ -3,6 +3,7 @@ #include #include +#include #include #include #include @@ -62,6 +63,7 @@ using bank::dto::GetAccount; BRIDGE_REGISTER_MODEL(AccountModel, "AccountModel") BRIDGE_REGISTER_ACTION(AccountModel, GetAccount, "GetAccount", ::morph::model::Loggable::No) BRIDGE_REGISTER_ACTION(AccountModel, CloseAccount, "CloseAccount") +BRIDGE_REGISTER_EFFECTS(bank::dto::CloseAccount, ::morph::effects::changes) // Both actions name the account they act on, so a shared handler attaches (or // re-points) to that account's instance on the way through. diff --git a/examples/bank/include/bank/models/auth_model.hpp b/examples/bank/include/bank/models/auth_model.hpp index b58b5120f..f798fb226 100644 --- a/examples/bank/include/bank/models/auth_model.hpp +++ b/examples/bank/include/bank/models/auth_model.hpp @@ -2,6 +2,7 @@ #pragma once #include +#include #include #include "bank/db/db_model.hpp" @@ -41,7 +42,11 @@ using bank::dto::RegisterUser; using bank::dto::WhoAmI; BRIDGE_REGISTER_MODEL(AuthModel, "Bank_AuthModel") +// Registering and signing in change no row a screen shows: the screen signs the session in, which reloads the catalog. BRIDGE_REGISTER_ACTION(AuthModel, RegisterUser, "RegisterUser") +BRIDGE_REGISTER_EFFECTS(bank::dto::RegisterUser, ::morph::effects::none) BRIDGE_REGISTER_ACTION(AuthModel, LoginRequest, "LoginRequest") +BRIDGE_REGISTER_EFFECTS(bank::dto::LoginRequest, ::morph::effects::none) BRIDGE_REGISTER_ACTION(AuthModel, ChangePassword, "ChangePassword") +BRIDGE_REGISTER_EFFECTS(bank::dto::ChangePassword, ::morph::effects::none) BRIDGE_REGISTER_ACTION(AuthModel, WhoAmI, "WhoAmI", ::morph::model::Loggable::No) diff --git a/examples/bank/include/bank/models/budget_model.hpp b/examples/bank/include/bank/models/budget_model.hpp index 48cd22a49..978fb86fa 100644 --- a/examples/bank/include/bank/models/budget_model.hpp +++ b/examples/bank/include/bank/models/budget_model.hpp @@ -3,6 +3,7 @@ #include #include +#include #include #include "bank/db/db_model.hpp" @@ -37,6 +38,8 @@ using bank::dto::SpendingByKind; BRIDGE_REGISTER_MODEL(BudgetModel, "Bank_BudgetModel") BRIDGE_REGISTER_ACTION(BudgetModel, SetBudget, "SetBudget") +BRIDGE_REGISTER_EFFECTS(bank::dto::SetBudget, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(BudgetModel, DeleteBudget, "DeleteBudget") +BRIDGE_REGISTER_EFFECTS(bank::dto::DeleteBudget, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(BudgetModel, ListBudgets, "ListBudgets", ::morph::model::Loggable::No) BRIDGE_REGISTER_ACTION(BudgetModel, SpendingByKind, "SpendingByKind", ::morph::model::Loggable::No) diff --git a/examples/bank/include/bank/models/card_model.hpp b/examples/bank/include/bank/models/card_model.hpp index 79c00b227..13b7fc911 100644 --- a/examples/bank/include/bank/models/card_model.hpp +++ b/examples/bank/include/bank/models/card_model.hpp @@ -2,6 +2,7 @@ #pragma once #include +#include #include #include "bank/db/db_model.hpp" @@ -39,9 +40,16 @@ using bank::dto::UnfreezeCard; BRIDGE_REGISTER_MODEL(CardModel, "CardModel") BRIDGE_REGISTER_ACTION(CardModel, IssueCard, "IssueCard") +BRIDGE_REGISTER_EFFECTS(bank::dto::IssueCard, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(CardModel, FreezeCard, "FreezeCard") +BRIDGE_REGISTER_EFFECTS(bank::dto::FreezeCard, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(CardModel, UnfreezeCard, "UnfreezeCard") +BRIDGE_REGISTER_EFFECTS(bank::dto::UnfreezeCard, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(CardModel, CancelCard, "CancelCard") +BRIDGE_REGISTER_EFFECTS(bank::dto::CancelCard, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(CardModel, SetCardLimit, "SetCardLimit") +BRIDGE_REGISTER_EFFECTS(bank::dto::SetCardLimit, ::morph::effects::changes) +// A card's PIN is never shown, so changing it changes nothing a screen shows. BRIDGE_REGISTER_ACTION(CardModel, ChangePin, "ChangePin") +BRIDGE_REGISTER_EFFECTS(bank::dto::ChangePin, ::morph::effects::none) BRIDGE_REGISTER_ACTION(CardModel, ListCards, "ListCards", ::morph::model::Loggable::No) diff --git a/examples/bank/include/bank/models/customer_model.hpp b/examples/bank/include/bank/models/customer_model.hpp index 27fe3fb07..19f873f03 100644 --- a/examples/bank/include/bank/models/customer_model.hpp +++ b/examples/bank/include/bank/models/customer_model.hpp @@ -2,6 +2,7 @@ #pragma once #include +#include #include #include #include @@ -39,6 +40,7 @@ using bank::dto::OpenAccount; BRIDGE_REGISTER_MODEL(CustomerModel, "CustomerModel") BRIDGE_REGISTER_ACTION(CustomerModel, OpenAccount, "OpenAccount") +BRIDGE_REGISTER_EFFECTS(bank::dto::OpenAccount, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(CustomerModel, ListAccounts, "ListAccounts", ::morph::model::Loggable::No) // Keyed by owner. An empty `owner` means "the session principal", which is not diff --git a/examples/bank/include/bank/models/loan_model.hpp b/examples/bank/include/bank/models/loan_model.hpp index 675b0bb75..9d2d44da5 100644 --- a/examples/bank/include/bank/models/loan_model.hpp +++ b/examples/bank/include/bank/models/loan_model.hpp @@ -2,10 +2,13 @@ #pragma once #include +#include #include #include "bank/db/db_model.hpp" +#include "bank/dto/account_dto.hpp" #include "bank/dto/loan_dto.hpp" +#include "bank/dto/transaction_dto.hpp" /// @file /// The Loan model: apply (disbursing the principal into an account), repay @@ -35,8 +38,16 @@ using bank::dto::LoanScheduleRequest; using bank::dto::RepayLoan; BRIDGE_REGISTER_MODEL(LoanModel, "LoanModel") +// A loan is paid out into its account, as a ledger entry. BRIDGE_REGISTER_ACTION(LoanModel, ApplyLoan, "ApplyLoan") +BRIDGE_REGISTER_EFFECTS(bank::dto::ApplyLoan, ::morph::effects::changes, + ::morph::effects::changes, + ::morph::effects::changes) +// The reply is the repaid loan; the repayment is debited from an account. BRIDGE_REGISTER_ACTION(LoanModel, RepayLoan, "RepayLoan") +BRIDGE_REGISTER_EFFECTS(bank::dto::RepayLoan, ::morph::effects::updates, + ::morph::effects::changes, + ::morph::effects::changes) BRIDGE_REGISTER_ACTION(LoanModel, GetLoan, "GetLoan", ::morph::model::Loggable::No) BRIDGE_REGISTER_ACTION(LoanModel, ListLoans, "ListLoans", ::morph::model::Loggable::No) BRIDGE_REGISTER_ACTION(LoanModel, LoanScheduleRequest, "LoanScheduleRequest", ::morph::model::Loggable::No) diff --git a/examples/bank/include/bank/models/notification_model.hpp b/examples/bank/include/bank/models/notification_model.hpp index 2a8ed1503..ee8ca8218 100644 --- a/examples/bank/include/bank/models/notification_model.hpp +++ b/examples/bank/include/bank/models/notification_model.hpp @@ -2,6 +2,7 @@ #pragma once #include +#include #include #include "bank/db/db_model.hpp" @@ -32,6 +33,9 @@ using bank::dto::Notify; BRIDGE_REGISTER_MODEL(NotificationModel, "NotificationModel") BRIDGE_REGISTER_ACTION(NotificationModel, Notify, "Notify") +BRIDGE_REGISTER_EFFECTS(bank::dto::Notify, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(NotificationModel, ListNotifications, "ListNotifications", ::morph::model::Loggable::No) BRIDGE_REGISTER_ACTION(NotificationModel, MarkRead, "MarkRead") +BRIDGE_REGISTER_EFFECTS(bank::dto::MarkRead, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(NotificationModel, MarkAllRead, "MarkAllRead") +BRIDGE_REGISTER_EFFECTS(bank::dto::MarkAllRead, ::morph::effects::changes) diff --git a/examples/bank/include/bank/models/payee_model.hpp b/examples/bank/include/bank/models/payee_model.hpp index ffbd2dbeb..e8f418914 100644 --- a/examples/bank/include/bank/models/payee_model.hpp +++ b/examples/bank/include/bank/models/payee_model.hpp @@ -2,6 +2,7 @@ #pragma once #include +#include #include #include "bank/db/db_model.hpp" @@ -37,5 +38,7 @@ using bank::dto::RemovePayee; BRIDGE_REGISTER_MODEL(PayeeModel, "PayeeModel") BRIDGE_REGISTER_ACTION(PayeeModel, AddPayee, "AddPayee") +BRIDGE_REGISTER_EFFECTS(bank::dto::AddPayee, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(PayeeModel, RemovePayee, "RemovePayee") +BRIDGE_REGISTER_EFFECTS(bank::dto::RemovePayee, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(PayeeModel, ListPayees, "ListPayees", ::morph::model::Loggable::No) diff --git a/examples/bank/include/bank/models/payment_model.hpp b/examples/bank/include/bank/models/payment_model.hpp index 6554335f1..d46ce5497 100644 --- a/examples/bank/include/bank/models/payment_model.hpp +++ b/examples/bank/include/bank/models/payment_model.hpp @@ -2,11 +2,14 @@ #pragma once #include +#include #include #include "bank/db/db_model.hpp" +#include "bank/dto/account_dto.hpp" #include "bank/dto/common.hpp" #include "bank/dto/payment_dto.hpp" +#include "bank/dto/transaction_dto.hpp" /// @file /// The Payment model: immediate bill payments (which debit an account and post @@ -45,8 +48,15 @@ using bank::dto::PayBill; using bank::dto::SchedulePayment; BRIDGE_REGISTER_MODEL(PaymentModel, "PaymentModel") +// A bill is paid now, from an account. BRIDGE_REGISTER_ACTION(PaymentModel, PayBill, "PayBill") +BRIDGE_REGISTER_EFFECTS(bank::dto::PayBill, ::morph::effects::changes, + ::morph::effects::changes, + ::morph::effects::changes) BRIDGE_REGISTER_ACTION(PaymentModel, SchedulePayment, "SchedulePayment") +BRIDGE_REGISTER_EFFECTS(bank::dto::SchedulePayment, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(PaymentModel, CreateStandingOrder, "CreateStandingOrder") +BRIDGE_REGISTER_EFFECTS(bank::dto::CreateStandingOrder, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(PaymentModel, CancelPayment, "CancelPayment") +BRIDGE_REGISTER_EFFECTS(bank::dto::CancelPayment, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(PaymentModel, ListPayments, "ListPayments", ::morph::model::Loggable::No) diff --git a/examples/bank/include/bank/models/transaction_model.hpp b/examples/bank/include/bank/models/transaction_model.hpp index 3cc1e8b5a..967580c4e 100644 --- a/examples/bank/include/bank/models/transaction_model.hpp +++ b/examples/bank/include/bank/models/transaction_model.hpp @@ -2,9 +2,11 @@ #pragma once #include +#include #include #include "bank/db/db_model.hpp" +#include "bank/dto/account_dto.hpp" #include "bank/dto/transaction_dto.hpp" /// @file @@ -44,7 +46,15 @@ using bank::dto::Transfer; using bank::dto::Withdraw; BRIDGE_REGISTER_MODEL(TransactionModel, "TransactionModel") +// Moving money writes a ledger entry and changes the balances it moves between. BRIDGE_REGISTER_ACTION(TransactionModel, Deposit, "Deposit") +BRIDGE_REGISTER_EFFECTS(bank::dto::Deposit, ::morph::effects::changes, + ::morph::effects::changes) BRIDGE_REGISTER_ACTION(TransactionModel, Withdraw, "Withdraw") +BRIDGE_REGISTER_EFFECTS(bank::dto::Withdraw, ::morph::effects::changes, + ::morph::effects::changes) +// The reply is the two balances, not the accounts, so the accounts are changed, not updated. BRIDGE_REGISTER_ACTION(TransactionModel, Transfer, "Transfer") +BRIDGE_REGISTER_EFFECTS(bank::dto::Transfer, ::morph::effects::changes, + ::morph::effects::changes) BRIDGE_REGISTER_ACTION(TransactionModel, History, "History", ::morph::model::Loggable::No) diff --git a/examples/bank/memory/include/bank/models/account_model.hpp b/examples/bank/memory/include/bank/models/account_model.hpp index b599c2de6..515845ac4 100644 --- a/examples/bank/memory/include/bank/models/account_model.hpp +++ b/examples/bank/memory/include/bank/models/account_model.hpp @@ -7,6 +7,7 @@ #include #include +#include #include #include #include @@ -40,6 +41,7 @@ using bank::dto::GetAccount; BRIDGE_REGISTER_MODEL(AccountModel, "AccountModel") BRIDGE_REGISTER_ACTION(AccountModel, GetAccount, "GetAccount") BRIDGE_REGISTER_ACTION(AccountModel, CloseAccount, "CloseAccount") +BRIDGE_REGISTER_EFFECTS(bank::dto::CloseAccount, ::morph::effects::changes) BRIDGE_MODEL_KEY(AccountModel, GetAccount, &GetAccount::id); BRIDGE_KEY_FROM(CloseAccount, &CloseAccount::id); diff --git a/examples/bank/memory/include/bank/models/auth_model.hpp b/examples/bank/memory/include/bank/models/auth_model.hpp index ab37c40f1..f3d12a906 100644 --- a/examples/bank/memory/include/bank/models/auth_model.hpp +++ b/examples/bank/memory/include/bank/models/auth_model.hpp @@ -7,6 +7,7 @@ // is placed first on the WASM include path so it wins over the native one. #include +#include #include #include "bank/dto/auth_dto.hpp" @@ -32,7 +33,11 @@ using bank::dto::RegisterUser; using bank::dto::WhoAmI; BRIDGE_REGISTER_MODEL(AuthModel, "Bank_AuthModel") +// Registering and signing in change no row a screen shows: the screen signs the session in, which reloads the catalog. BRIDGE_REGISTER_ACTION(AuthModel, RegisterUser, "RegisterUser") +BRIDGE_REGISTER_EFFECTS(bank::dto::RegisterUser, ::morph::effects::none) BRIDGE_REGISTER_ACTION(AuthModel, LoginRequest, "LoginRequest") +BRIDGE_REGISTER_EFFECTS(bank::dto::LoginRequest, ::morph::effects::none) BRIDGE_REGISTER_ACTION(AuthModel, ChangePassword, "ChangePassword") +BRIDGE_REGISTER_EFFECTS(bank::dto::ChangePassword, ::morph::effects::none) BRIDGE_REGISTER_ACTION(AuthModel, WhoAmI, "WhoAmI") diff --git a/examples/bank/memory/include/bank/models/card_model.hpp b/examples/bank/memory/include/bank/models/card_model.hpp index aa4dd80de..7f68122ea 100644 --- a/examples/bank/memory/include/bank/models/card_model.hpp +++ b/examples/bank/memory/include/bank/models/card_model.hpp @@ -4,6 +4,7 @@ // WASM shadow of include/bank/models/card_model.hpp (in-memory backend). #include +#include #include #include "bank/dto/card_dto.hpp" @@ -36,9 +37,16 @@ using bank::dto::UnfreezeCard; BRIDGE_REGISTER_MODEL(CardModel, "CardModel") BRIDGE_REGISTER_ACTION(CardModel, IssueCard, "IssueCard") +BRIDGE_REGISTER_EFFECTS(bank::dto::IssueCard, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(CardModel, FreezeCard, "FreezeCard") +BRIDGE_REGISTER_EFFECTS(bank::dto::FreezeCard, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(CardModel, UnfreezeCard, "UnfreezeCard") +BRIDGE_REGISTER_EFFECTS(bank::dto::UnfreezeCard, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(CardModel, CancelCard, "CancelCard") +BRIDGE_REGISTER_EFFECTS(bank::dto::CancelCard, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(CardModel, SetCardLimit, "SetCardLimit") +BRIDGE_REGISTER_EFFECTS(bank::dto::SetCardLimit, ::morph::effects::changes) +// A card's PIN is never shown, so changing it changes nothing a screen shows. BRIDGE_REGISTER_ACTION(CardModel, ChangePin, "ChangePin") +BRIDGE_REGISTER_EFFECTS(bank::dto::ChangePin, ::morph::effects::none) BRIDGE_REGISTER_ACTION(CardModel, ListCards, "ListCards") diff --git a/examples/bank/memory/include/bank/models/customer_model.hpp b/examples/bank/memory/include/bank/models/customer_model.hpp index 9c3a404e8..e71e97c64 100644 --- a/examples/bank/memory/include/bank/models/customer_model.hpp +++ b/examples/bank/memory/include/bank/models/customer_model.hpp @@ -4,6 +4,7 @@ // WASM shadow of include/bank/models/customer_model.hpp (in-memory backend). #include +#include #include #include #include @@ -27,6 +28,7 @@ using bank::dto::OpenAccount; BRIDGE_REGISTER_MODEL(CustomerModel, "CustomerModel") BRIDGE_REGISTER_ACTION(CustomerModel, OpenAccount, "OpenAccount") +BRIDGE_REGISTER_EFFECTS(bank::dto::OpenAccount, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(CustomerModel, ListAccounts, "ListAccounts") BRIDGE_MODEL_KEY(CustomerModel, ListAccounts, &ListAccounts::owner); diff --git a/examples/bank/memory/include/bank/models/loan_model.hpp b/examples/bank/memory/include/bank/models/loan_model.hpp index 43e808f66..2566e59d6 100644 --- a/examples/bank/memory/include/bank/models/loan_model.hpp +++ b/examples/bank/memory/include/bank/models/loan_model.hpp @@ -4,9 +4,12 @@ // WASM shadow of include/bank/models/loan_model.hpp (in-memory backend). #include +#include #include +#include "bank/dto/account_dto.hpp" #include "bank/dto/loan_dto.hpp" +#include "bank/dto/transaction_dto.hpp" namespace bank { @@ -30,8 +33,16 @@ using bank::dto::LoanScheduleRequest; using bank::dto::RepayLoan; BRIDGE_REGISTER_MODEL(LoanModel, "LoanModel") +// A loan is paid out into its account, as a ledger entry. BRIDGE_REGISTER_ACTION(LoanModel, ApplyLoan, "ApplyLoan") +BRIDGE_REGISTER_EFFECTS(bank::dto::ApplyLoan, ::morph::effects::changes, + ::morph::effects::changes, + ::morph::effects::changes) +// The reply is the repaid loan; the repayment is debited from an account. BRIDGE_REGISTER_ACTION(LoanModel, RepayLoan, "RepayLoan") +BRIDGE_REGISTER_EFFECTS(bank::dto::RepayLoan, ::morph::effects::updates, + ::morph::effects::changes, + ::morph::effects::changes) BRIDGE_REGISTER_ACTION(LoanModel, GetLoan, "GetLoan") BRIDGE_REGISTER_ACTION(LoanModel, ListLoans, "ListLoans") BRIDGE_REGISTER_ACTION(LoanModel, LoanScheduleRequest, "LoanScheduleRequest") diff --git a/examples/bank/memory/include/bank/models/payee_model.hpp b/examples/bank/memory/include/bank/models/payee_model.hpp index 624fcddb4..cb5b48a5c 100644 --- a/examples/bank/memory/include/bank/models/payee_model.hpp +++ b/examples/bank/memory/include/bank/models/payee_model.hpp @@ -4,6 +4,7 @@ // WASM shadow of include/bank/models/payee_model.hpp (in-memory backend). #include +#include #include #include "bank/dto/common.hpp" @@ -28,5 +29,7 @@ using bank::dto::RemovePayee; BRIDGE_REGISTER_MODEL(PayeeModel, "PayeeModel") BRIDGE_REGISTER_ACTION(PayeeModel, AddPayee, "AddPayee") +BRIDGE_REGISTER_EFFECTS(bank::dto::AddPayee, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(PayeeModel, RemovePayee, "RemovePayee") +BRIDGE_REGISTER_EFFECTS(bank::dto::RemovePayee, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(PayeeModel, ListPayees, "ListPayees") diff --git a/examples/bank/memory/include/bank/models/payment_model.hpp b/examples/bank/memory/include/bank/models/payment_model.hpp index 8b54bfd53..95ba6977f 100644 --- a/examples/bank/memory/include/bank/models/payment_model.hpp +++ b/examples/bank/memory/include/bank/models/payment_model.hpp @@ -4,10 +4,13 @@ // WASM shadow of include/bank/models/payment_model.hpp (in-memory backend). #include +#include #include +#include "bank/dto/account_dto.hpp" #include "bank/dto/common.hpp" #include "bank/dto/payment_dto.hpp" +#include "bank/dto/transaction_dto.hpp" namespace bank { @@ -31,8 +34,15 @@ using bank::dto::PayBill; using bank::dto::SchedulePayment; BRIDGE_REGISTER_MODEL(PaymentModel, "PaymentModel") +// A bill is paid now, from an account. BRIDGE_REGISTER_ACTION(PaymentModel, PayBill, "PayBill") +BRIDGE_REGISTER_EFFECTS(bank::dto::PayBill, ::morph::effects::changes, + ::morph::effects::changes, + ::morph::effects::changes) BRIDGE_REGISTER_ACTION(PaymentModel, SchedulePayment, "SchedulePayment") +BRIDGE_REGISTER_EFFECTS(bank::dto::SchedulePayment, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(PaymentModel, CreateStandingOrder, "CreateStandingOrder") +BRIDGE_REGISTER_EFFECTS(bank::dto::CreateStandingOrder, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(PaymentModel, CancelPayment, "CancelPayment") +BRIDGE_REGISTER_EFFECTS(bank::dto::CancelPayment, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(PaymentModel, ListPayments, "ListPayments") diff --git a/examples/bank/memory/include/bank/models/transaction_model.hpp b/examples/bank/memory/include/bank/models/transaction_model.hpp index 1b9090712..f5721f90f 100644 --- a/examples/bank/memory/include/bank/models/transaction_model.hpp +++ b/examples/bank/memory/include/bank/models/transaction_model.hpp @@ -4,8 +4,10 @@ // WASM shadow of include/bank/models/transaction_model.hpp (in-memory backend). #include +#include #include +#include "bank/dto/account_dto.hpp" #include "bank/dto/transaction_dto.hpp" namespace bank { @@ -28,7 +30,15 @@ using bank::dto::Transfer; using bank::dto::Withdraw; BRIDGE_REGISTER_MODEL(TransactionModel, "TransactionModel") +// Moving money writes a ledger entry and changes the balances it moves between. BRIDGE_REGISTER_ACTION(TransactionModel, Deposit, "Deposit") +BRIDGE_REGISTER_EFFECTS(bank::dto::Deposit, ::morph::effects::changes, + ::morph::effects::changes) BRIDGE_REGISTER_ACTION(TransactionModel, Withdraw, "Withdraw") +BRIDGE_REGISTER_EFFECTS(bank::dto::Withdraw, ::morph::effects::changes, + ::morph::effects::changes) +// The reply is the two balances, not the accounts, so the accounts are changed, not updated. BRIDGE_REGISTER_ACTION(TransactionModel, Transfer, "Transfer") +BRIDGE_REGISTER_EFFECTS(bank::dto::Transfer, ::morph::effects::changes, + ::morph::effects::changes) BRIDGE_REGISTER_ACTION(TransactionModel, History, "History") diff --git a/examples/bank/screens/bank_screens.cpp b/examples/bank/screens/bank_screens.cpp index fd0fe9ba4..010747779 100644 --- a/examples/bank/screens/bank_screens.cpp +++ b/examples/bank/screens/bank_screens.cpp @@ -116,7 +116,7 @@ ui::Screen accountsScreen() { auto const history = screen.query("history", transactions, {{&dto::History::accountId, chosen.selected()}}, {.when = ui::engaged(chosen.selected())}); - auto const close = screen.mutation("close", account, {.invalidates = {accounts}}); + auto const close = screen.mutation("close", account); auto const cards = screen.component( "AccountCards", {.host = "AccountCards", @@ -150,8 +150,7 @@ ui::Screen accountsScreen() { ui::form(customer, {.submit = "explicit", .fields = {{&dto::OpenAccount::owner, {.omit = true}}, {&dto::OpenAccount::overdraftMinor, - {.label = std::string{"Overdraft (cents)"}}}}, - .onSuccess = {ui::refetch(accounts)}}) + {.label = std::string{"Overdraft (cents)"}}}}}) .id(ui::element("openForm")) .testId("openForm"), ui::select({.options = accountOptions(rows), .label = "History of"}).id(chosen).testId("historyAccount"), @@ -191,7 +190,6 @@ ui::Screen moveScreen() { .fallback = ui::text({.text = ui::format("From {0} to {1}", {ui::prop("from.label"), ui::prop("to.label")})}) .testId("transferFlowFallback")}); - auto const moved = ui::Commands{ui::refetch(accounts)}; auto const hidden = ui::FieldSettings{.hidden = true}; return page( kMove, @@ -204,8 +202,7 @@ ui::Screen moveScreen() { .submit = "explicit", .fields = {{&dto::Deposit::accountId, hidden}, {&dto::Deposit::amountMinor, {.label = std::string{"Amount (cents)"}}}, - {&dto::Deposit::description, {.blankAs = std::string{"empty"}}}}, - .onSuccess = moved}) + {&dto::Deposit::description, {.blankAs = std::string{"empty"}}}}}) .id(ui::element("depositForm")) .testId("depositForm"), ui::form( @@ -213,8 +210,7 @@ ui::Screen moveScreen() { .submit = "explicit", .fields = {{&dto::Withdraw::accountId, hidden}, {&dto::Withdraw::amountMinor, {.label = std::string{"Amount (cents)"}}}, - {&dto::Withdraw::description, {.blankAs = std::string{"empty"}}}}, - .onSuccess = moved}) + {&dto::Withdraw::description, {.blankAs = std::string{"empty"}}}}}) .id(ui::element("withdrawForm")) .testId("withdrawForm"), ui::select({.options = accountOptions(rows), .label = "Transfer to"}).id(target).testId("target"), @@ -229,8 +225,7 @@ ui::Screen moveScreen() { .fields = {{&dto::Transfer::fromAccountId, hidden}, {&dto::Transfer::toAccountId, hidden}, {&dto::Transfer::amountMinor, {.label = std::string{"Amount (cents)"}}}, - {&dto::Transfer::description, {.blankAs = std::string{"empty"}}}}, - .onSuccess = moved}) + {&dto::Transfer::description, {.blankAs = std::string{"empty"}}}}}) .id(ui::element("transferForm")) .testId("transferForm"), ui::text({.text = accounts.error()}).testId("error")}); @@ -243,9 +238,9 @@ ui::Screen cardsScreen() { screen.title("Cards"); auto const cardModel = screen.model("card", {.instance = ui::Instance::Private}); auto const cards = screen.query("cards", cardModel); - auto const freeze = screen.mutation("freeze", cardModel, {.invalidates = {cards}}); - auto const unfreeze = screen.mutation("unfreeze", cardModel, {.invalidates = {cards}}); - auto const cancel = screen.mutation("cancel", cardModel, {.invalidates = {cards}}); + auto const freeze = screen.mutation("freeze", cardModel); + auto const unfreeze = screen.mutation("unfreeze", cardModel); + auto const cancel = screen.mutation("cancel", cardModel); auto const action = [](auto const& mutation, auto member, std::string const& label) { return ui::button({.label = label, .onClick = ui::run(mutation, {{member, ui::row("id")}})}); }; @@ -266,8 +261,7 @@ ui::Screen cardsScreen() { .testId("cards"), ui::form(cardModel, {.submit = "explicit", .fields = {{&dto::IssueCard::dailyLimitMinor, - {.label = std::string{"Daily limit (cents)"}}}}, - .onSuccess = {ui::refetch(cards)}}) + {.label = std::string{"Daily limit (cents)"}}}}}) .id(ui::element("issueForm")) .testId("issueForm"), ui::text({.text = ui::coalesce({cards.error(), freeze.error(), unfreeze.error(), cancel.error()})}) @@ -281,7 +275,7 @@ ui::Screen payeesScreen() { screen.title("Payees"); auto const payeeModel = screen.model("payee", {.instance = ui::Instance::Private}); auto const payees = screen.query("payees", payeeModel); - auto const remove = screen.mutation("remove", payeeModel, {.invalidates = {payees}}); + auto const remove = screen.mutation("remove", payeeModel); return page( kPayees, {ui::table(payees.value().at(&dto::PayeeList::payees), @@ -296,9 +290,7 @@ ui::Screen payeesScreen() { .onClick = ui::run(remove, {{&dto::RemovePayee::id, ui::row("id")}})})}}}) .a11y("Payees") .testId("payees"), - ui::form(payeeModel, {.submit = "explicit", .onSuccess = {ui::refetch(payees)}}) - .id(ui::element("addForm")) - .testId("addForm"), + ui::form(payeeModel, {.submit = "explicit"}).id(ui::element("addForm")).testId("addForm"), ui::text({.text = ui::coalesce({payees.error(), remove.error()})}).testId("error")}); }); } @@ -349,14 +341,12 @@ ui::Screen loansScreen() { .testId("loans"), ui::form(loanModel, {.submit = "explicit", .fields = {{&dto::ApplyLoan::principalMinor, - {.label = std::string{"Principal (cents)"}}}}, - .onSuccess = {ui::refetch(loans)}}) + {.label = std::string{"Principal (cents)"}}}}}) .id(ui::element("applyForm")) .testId("applyForm"), ui::form( loanModel, {.submit = "explicit", - .fields = {{&dto::RepayLoan::amountMinor, {.label = std::string{"Amount (cents)"}}}}, - .onSuccess = {ui::refetch(loans)}}) + .fields = {{&dto::RepayLoan::amountMinor, {.label = std::string{"Amount (cents)"}}}}}) .id(ui::element("repayForm")) .testId("repayForm"), ui::select({.options = options, .label = "Schedule of"}).id(chosen).testId("scheduleLoan"), diff --git a/examples/bank/screens/tests/test_bank_screens.cpp b/examples/bank/screens/tests/test_bank_screens.cpp index 597ea6a0d..f132679e8 100644 --- a/examples/bank/screens/tests/test_bank_screens.cpp +++ b/examples/bank/screens/tests/test_bank_screens.cpp @@ -14,6 +14,7 @@ #include #include #include +#include #include #include @@ -104,6 +105,19 @@ TEST_CASE("bank screens: every screen validates, and the bundle builds", "[bank- REQUIRE(bundle.has_value()); } +// Every document loads against the catalog the server serves with no warning: each mutation's action declares its +// effects (or `none`), and no refresh declaration repeats what the effects already cover. +TEST_CASE("bank_bundle_loads_without_warnings", "[bank-screens]") { + auto const bundle = bank::screens::bundle(); + INFO((bundle ? std::string{} : bundle.error().message())); + REQUIRE(bundle.has_value()); + std::vector const findings = morph::examples::testing::bundleLoadFindings(**bundle); + for (std::string const& finding : findings) { + UNSCOPED_INFO(finding); + } + CHECK(findings.empty()); +} + // Mutation: navigate without signing in (the accounts screen's calls carry no principal and fail). TEST_CASE("bank screens: registering signs the customer in and opens their accounts", "[bank-screens]") { ScreenHarness harness = open(bank::screens::kLogin, {}); From 4d529cd2bd556237ec585e100e2f08a458a9c3b6 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 01:48:07 +0200 Subject: [PATCH 55/68] examples(lims): actions declare their effects; screens drop refresh declarations Results, samples, verifications, conflicts and analysis versions are lab-wide records. Lifecycle transitions update their sample, instance actions change their instance, and OpenSample changes nothing. The screens lose their invalidates, refreshOn, patch and refetch-only onSuccess commands; a re-capture, which replaces its row under a new key, now refetches. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- examples/lims/README.md | 7 +- .../lims/include/lims/dto/analysis_dto.hpp | 3 + .../lims/include/lims/dto/offline_dto.hpp | 3 + examples/lims/include/lims/dto/result_dto.hpp | 4 + examples/lims/include/lims/dto/sample_dto.hpp | 3 + .../include/lims/dto/verification_dto.hpp | 3 + .../lims/models/analysis_catalog_model.hpp | 3 + .../lims/include/lims/models/auth_model.hpp | 3 + .../lims/include/lims/models/sample_model.hpp | 38 ++++++++ examples/lims/screens/lims_screens.cpp | 86 +++++++------------ .../lims/screens/tests/test_lims_screens.cpp | 26 ++++-- 11 files changed, 116 insertions(+), 63 deletions(-) diff --git a/examples/lims/README.md b/examples/lims/README.md index cdb05352a..d31c49dd8 100644 --- a/examples/lims/README.md +++ b/examples/lims/README.md @@ -574,7 +574,7 @@ lims' screens are UI documents served by the framework (spec 4), authored with t - **`lims.register`** registers a client, then a sample for it, through mutations over a private instance of `SampleModel`, and opens the new sample's lifecycle. - **`lims.lifecycle`** (`sampleId`) attaches the keyed `SampleModel` instance with `OpenSample` and - shows the sample from a `GetSample` query that refetches after each transition. Each of the four + shows the sample from a `GetSample` query that each transition's reply updates. Each of the four transitions is a serial mutation enabled only in the state that allows it. - **`lims.results`** (`sampleId`) captures a concentration through a `form` node over `CaptureConcentration`, built from the schema the registry serves, lists the results with their @@ -589,6 +589,11 @@ lims' screens are UI documents served by the framework (spec 4), authored with t `ListProjects`, which the table engine sorts, filters and pages. The screen's identity is its sample, and it is dirty while the working copy holds unsaved edits. +Each action declares what it changes beside its registration (`BRIDGE_REGISTER_EFFECTS` in +`include/lims/models/`): a transition updates its `Lims_Sample`, a capture changes `Lims_Result`, the sample-entry +actions change their instance's working copy, and `OpenSample` changes nothing. A reply refreshes the queries it +changed, so the screens declare no refreshes; `lims_bundle_loads_without_warnings` keeps it so. + A model's refusal reaches the screen as the mutation's own error. Every typed field is schema-driven: the `x-rules` of `CaptureConcentration` run in the forms engine, not in hand-written UI code. diff --git a/examples/lims/include/lims/dto/analysis_dto.hpp b/examples/lims/include/lims/dto/analysis_dto.hpp index dd77d7102..e21390f14 100644 --- a/examples/lims/include/lims/dto/analysis_dto.hpp +++ b/examples/lims/include/lims/dto/analysis_dto.hpp @@ -3,6 +3,7 @@ #include #include +#include #include #include #include @@ -209,3 +210,5 @@ struct DefineAnalysisResult { }; } // namespace lims + +BRIDGE_REGISTER_RECORD(lims::AnalysisVersionView, "Lims_AnalysisVersion") diff --git a/examples/lims/include/lims/dto/offline_dto.hpp b/examples/lims/include/lims/dto/offline_dto.hpp index d25884eab..d06e63da1 100644 --- a/examples/lims/include/lims/dto/offline_dto.hpp +++ b/examples/lims/include/lims/dto/offline_dto.hpp @@ -3,6 +3,7 @@ #include #include +#include #include #include #include @@ -285,3 +286,5 @@ struct glz::meta { /// @brief The enumerator ⇄ name mapping glaze reads and writes. static constexpr auto value = glz::enumerate(Applied, Conflicted, Skipped); }; + +BRIDGE_REGISTER_RECORD(lims::ConflictView, "Lims_Conflict") diff --git a/examples/lims/include/lims/dto/result_dto.hpp b/examples/lims/include/lims/dto/result_dto.hpp index 28246874f..fd663d8cb 100644 --- a/examples/lims/include/lims/dto/result_dto.hpp +++ b/examples/lims/include/lims/dto/result_dto.hpp @@ -2,6 +2,7 @@ #pragma once #include +#include #include #include #include @@ -360,3 +361,6 @@ struct glz::meta { /// @brief The enumerator ⇄ name mapping glaze reads and writes. static constexpr auto value = glz::enumerate(Measured, NotMeasured, BelowLOD, AboveUDL); }; + +// Results are lab records, shown to every operator who opens the sample. +BRIDGE_REGISTER_RECORD(lims::ResultView, "Lims_Result") diff --git a/examples/lims/include/lims/dto/sample_dto.hpp b/examples/lims/include/lims/dto/sample_dto.hpp index 123f98c5f..14f650eca 100644 --- a/examples/lims/include/lims/dto/sample_dto.hpp +++ b/examples/lims/include/lims/dto/sample_dto.hpp @@ -3,6 +3,7 @@ #include #include +#include #include #include #include @@ -234,3 +235,5 @@ struct glz::meta { /// @brief The enumerator ⇄ name mapping glaze reads and writes. static constexpr auto value = glz::enumerate(Registered, Received, InProgress, ToBeVerified, Published, Rejected); }; + +BRIDGE_REGISTER_RECORD(lims::SampleView, "Lims_Sample") diff --git a/examples/lims/include/lims/dto/verification_dto.hpp b/examples/lims/include/lims/dto/verification_dto.hpp index 6195a5aac..83db1e8b2 100644 --- a/examples/lims/include/lims/dto/verification_dto.hpp +++ b/examples/lims/include/lims/dto/verification_dto.hpp @@ -3,6 +3,7 @@ #include #include +#include #include #include #include @@ -235,3 +236,5 @@ struct glz::meta { /// @brief The enumerator ⇄ name mapping glaze reads and writes. static constexpr auto value = glz::enumerate(Succeeded, Refused); }; + +BRIDGE_REGISTER_RECORD(lims::VerificationView, "Lims_Verification") diff --git a/examples/lims/include/lims/models/analysis_catalog_model.hpp b/examples/lims/include/lims/models/analysis_catalog_model.hpp index 864f3f36b..11074fa98 100644 --- a/examples/lims/include/lims/models/analysis_catalog_model.hpp +++ b/examples/lims/include/lims/models/analysis_catalog_model.hpp @@ -2,6 +2,7 @@ #pragma once #include #include +#include #include #include #include @@ -65,7 +66,9 @@ class AnalysisCatalogModel { BRIDGE_REGISTER_MODEL(lims::AnalysisCatalogModel, "AnalysisCatalogModel") BRIDGE_REGISTER_ACTION(lims::AnalysisCatalogModel, lims::DefineAnalysis, "DefineAnalysis") +BRIDGE_REGISTER_EFFECTS(lims::DefineAnalysis, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(lims::AnalysisCatalogModel, lims::ReviseAnalysis, "ReviseAnalysis") +BRIDGE_REGISTER_EFFECTS(lims::ReviseAnalysis, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(lims::AnalysisCatalogModel, lims::ListAnalyses, "ListAnalyses") BRIDGE_REGISTER_ACTION(lims::AnalysisCatalogModel, lims::GetAnalysisVersion, "GetAnalysisVersion") BRIDGE_REGISTER_ACTION(lims::AnalysisCatalogModel, lims::GetAnalysisSchema, "GetAnalysisSchema", diff --git a/examples/lims/include/lims/models/auth_model.hpp b/examples/lims/include/lims/models/auth_model.hpp index 04a6ca762..f33acb9a3 100644 --- a/examples/lims/include/lims/models/auth_model.hpp +++ b/examples/lims/include/lims/models/auth_model.hpp @@ -3,6 +3,7 @@ #include #include +#include #include #include "lims/dto/auth_dto.hpp" @@ -39,5 +40,7 @@ void setTokenIssuer(std::shared_ptr<::morph::session::TokenIssuer> issuer); // macros declare a non-const status object each. // NOLINTBEGIN(misc-const-correctness) BRIDGE_REGISTER_MODEL(lims::LimsAuthModel, "LimsAuth") +// Signing in changes no row a screen shows: the screen that runs it signs the session in, which reloads the catalog. BRIDGE_REGISTER_ACTION(lims::LimsAuthModel, lims::LimsLogin, "LimsLogin", ::morph::model::Loggable::No) +BRIDGE_REGISTER_EFFECTS(lims::LimsLogin, ::morph::effects::none) // NOLINTEND(misc-const-correctness) diff --git a/examples/lims/include/lims/models/sample_model.hpp b/examples/lims/include/lims/models/sample_model.hpp index da18a484f..359fffe33 100644 --- a/examples/lims/include/lims/models/sample_model.hpp +++ b/examples/lims/include/lims/models/sample_model.hpp @@ -4,6 +4,7 @@ #include #include #include +#include #include #include #include @@ -473,46 +474,83 @@ class SampleModel { } // namespace lims BRIDGE_REGISTER_MODEL(lims::SampleModel, "SampleModel") +// No screen lists clients: a registered client is only named by the sample registered for it next. BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::RegisterClient, "RegisterClient") +BRIDGE_REGISTER_EFFECTS(lims::RegisterClient, ::morph::effects::none) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::RegisterSample, "RegisterSample") +BRIDGE_REGISTER_EFFECTS(lims::RegisterSample, ::morph::effects::changes) +// Opening attaches this instance to a sample; it changes nothing stored. BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::OpenSample, "OpenSample", ::morph::model::Loggable::No) +BRIDGE_REGISTER_EFFECTS(lims::OpenSample, ::morph::effects::none) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::GetSample, "GetSample", ::morph::model::Loggable::No) +// A transition answers the sample as it now is. BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::ReceiveSample, "ReceiveSample") +BRIDGE_REGISTER_EFFECTS(lims::ReceiveSample, ::morph::effects::updates) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::StartWork, "StartWork") +BRIDGE_REGISTER_EFFECTS(lims::StartWork, ::morph::effects::updates) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::SubmitForVerification, "SubmitForVerification") +BRIDGE_REGISTER_EFFECTS(lims::SubmitForVerification, ::morph::effects::updates) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::ReturnForRework, "ReturnForRework") +BRIDGE_REGISTER_EFFECTS(lims::ReturnForRework, ::morph::effects::updates) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::PublishSample, "PublishSample") +BRIDGE_REGISTER_EFFECTS(lims::PublishSample, ::morph::effects::updates) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::RejectSample, "RejectSample") +BRIDGE_REGISTER_EFFECTS(lims::RejectSample, ::morph::effects::updates) +// A capture replaces the version's previous result with a new row under a new id, and moves the sample's +// version, so both are changed rather than updated. BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::CaptureConcentration, "CaptureConcentration") +BRIDGE_REGISTER_EFFECTS(lims::CaptureConcentration, ::morph::effects::changes, + ::morph::effects::changes) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::ListResults, "ListResults", ::morph::model::Loggable::No) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::ListResultQualifiers, "ListResultQualifiers", ::morph::model::Loggable::No) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::ListDilutionModes, "ListDilutionModes", ::morph::model::Loggable::No) +// An offline replay applies its capture, or records why it could not. BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::QueuedCapture, "QueuedCapture") +BRIDGE_REGISTER_EFFECTS(lims::QueuedCapture, ::morph::effects::changes, + ::morph::effects::changes, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::ListConflicts, "ListConflicts", ::morph::model::Loggable::No) +// Applying a conflict anyway applies its capture. BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::ResolveConflict, "ResolveConflict") +BRIDGE_REGISTER_EFFECTS(lims::ResolveConflict, ::morph::effects::changes, + ::morph::effects::changes, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::GrantRole, "GrantRole") BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::VerifyResult, "VerifyResult") +BRIDGE_REGISTER_EFFECTS(lims::VerifyResult, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::ListVerifications, "ListVerifications", ::morph::model::Loggable::No) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::GetAuditTrail, "GetAuditTrail", ::morph::model::Loggable::No) // The registration macros declare a non-const status object each. // NOLINTBEGIN(misc-const-correctness) +// The sample-entry actions edit this instance's working copy, which the entry screen's queries read. BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::NewSample, "NewSample", ::morph::model::Loggable::No) +BRIDGE_REGISTER_EFFECTS(lims::NewSample, ::morph::effects::changesInstance) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::LoadSample, "LoadSample", ::morph::model::Loggable::No) +BRIDGE_REGISTER_EFFECTS(lims::LoadSample, ::morph::effects::changesInstance) +// Saving stores the working copy as the sample. BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::SaveSample, "SaveSample") +BRIDGE_REGISTER_EFFECTS(lims::SaveSample, ::morph::effects::changesInstance, + ::morph::effects::changes) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::DeleteSample, "DeleteSample") +BRIDGE_REGISTER_EFFECTS(lims::DeleteSample, ::morph::effects::changesInstance, + ::morph::effects::changes, ::morph::effects::changes) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::HeaderSection, "HeaderSection", ::morph::model::Loggable::No) +BRIDGE_REGISTER_EFFECTS(lims::HeaderSection, ::morph::effects::changesInstance) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::SamplingSection, "SamplingSection", ::morph::model::Loggable::No) +BRIDGE_REGISTER_EFFECTS(lims::SamplingSection, ::morph::effects::changesInstance) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::TurbiditySection, "TurbiditySection", ::morph::model::Loggable::No) +BRIDGE_REGISTER_EFFECTS(lims::TurbiditySection, ::morph::effects::changesInstance) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::NitrateSection, "NitrateSection", ::morph::model::Loggable::No) +BRIDGE_REGISTER_EFFECTS(lims::NitrateSection, ::morph::effects::changesInstance) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::ClearOptionalMeasurement, "ClearOptionalMeasurement", ::morph::model::Loggable::No) +BRIDGE_REGISTER_EFFECTS(lims::ClearOptionalMeasurement, ::morph::effects::changesInstance) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::GetHeader, "GetHeader", ::morph::model::Loggable::No) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::GetMeasurements, "GetMeasurements", ::morph::model::Loggable::No) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::GetDilutionSeries, "GetDilutionSeries", ::morph::model::Loggable::No) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::ListProjects, "ListProjects", ::morph::model::Loggable::No) BRIDGE_REGISTER_ACTION(lims::SampleModel, lims::SelectProject, "SelectProject", ::morph::model::Loggable::No) +BRIDGE_REGISTER_EFFECTS(lims::SelectProject, ::morph::effects::changesInstance) // NOLINTEND(misc-const-correctness) // `OpenSample` is the action that names a sample, so it defines the model's diff --git a/examples/lims/screens/lims_screens.cpp b/examples/lims/screens/lims_screens.cpp index 07788b3bc..64313d436 100644 --- a/examples/lims/screens/lims_screens.cpp +++ b/examples/lims/screens/lims_screens.cpp @@ -91,11 +91,8 @@ ui::Screen lifecycleScreen() { auto const start = transition(StartWork{}, "start"); auto const submit = transition(SubmitForVerification{}, "submit"); auto const publish = transition(PublishSample{}, "publish"); - // The sample as it is now: fetched once the screen attached it, and again after every transition. - auto const current = screen.query( - "current", sample, {}, - {.when = opened, - .refreshOn = sample.actions()}); + // The sample as it is now: fetched once the screen attached it; every transition's reply updates it. + auto const current = screen.query("current", sample, {}, {.when = opened}); auto const view = current.value(); screen.title(ui::format("Sample {0}", {sampleId})); @@ -145,10 +142,7 @@ ui::Screen resultsScreen() { auto const open = screen.mutation( "open", sample, {.body = {{&OpenSample::sampleId, sampleId}}, .onSuccess = {ui::set(opened, true)}}); auto const verify = screen.mutation("verify", sample, {.concurrency = ui::Mode::Serial}); - // A capture is not in `refreshOn`: a row's capture patches its own row, and the form refetches the list - // itself, since it may add a row. - auto const results = screen.query("results", sample, {}, - {.when = opened, .refreshOn = sample.actions()}); + auto const results = screen.query("results", sample, {}, {.when = opened}); auto const conflicts = screen.query("conflicts", sample, {}, {.when = opened}); auto const analyses = screen.query("analyses", catalog); // The capture's reported concentration while it is typed: the server evaluates the form's fields as the @@ -176,7 +170,6 @@ ui::Screen resultsScreen() { ui::form(sample, { .fields = {{&CaptureConcentration::reportedValue, {.omit = true}}}, - .onSuccess = {ui::refetch(results)}, }) .id(capture) .testId("capture"), @@ -187,20 +180,8 @@ ui::Screen resultsScreen() { ui::table( results.value().at(&ListResultsResult::results), {.key = ui::row(&ResultView::id)}, [&](ui::DeclScope& row) { - // Write-through: a re-capture replaces the row's result with a new one under a new id, so the - // patch is keyed on the row it was typed into, not on the reply. auto const recapture = - row.mutation("recapture", sample, - { - .concurrency = ui::Mode::Serial, - .patch = - ui::Patch{ - .query = results, - .at = "results", - .key = ui::row(&ResultView::id), - .with = ui::ref("result"), - }, - }); + row.mutation("recapture", sample, {.concurrency = ui::Mode::Serial}); // At the places typed: the model refuses a reading finer than its version declares, and // padding the text would claim precision nobody measured. auto const recaptureBody = ui::run( @@ -275,26 +256,18 @@ ui::Screen sampleScreen() { auto const showNitrate = screen.state("showNitrate", false); auto const project = ui::element("project"); - auto const header = screen.query( - "header", sample, {}, - {.when = ready, .refreshOn = sample.actions()}); - auto const series = screen.query( - "series", sample, {}, {.when = ready, .refreshOn = sample.actions()}); - auto const measure = screen.query( - "measure", sample, {}, - {.when = ready, - .refreshOn = sample.actions()}); + auto const header = screen.query("header", sample, {}, {.when = ready}); + auto const series = screen.query("series", sample, {}, {.when = ready}); + auto const measure = screen.query("measure", sample, {}, {.when = ready}); auto const projects = screen.query("projects", sample); - // Loading or creating fills the working copy the queries read, so either one refetches them. A section form's - // submit is not a mutation, so `refreshOn` does not see it: each form refetches what it changed itself. - auto const load = screen.mutation("load", sample, - {.body = {{&LoadSample::id, sampleId}}, - .invalidates = {header, series, measure}, - .onSuccess = {ui::set(ready, true), ui::set(unsaved, false)}}); - auto const make = screen.mutation( - "make", sample, - {.invalidates = {header, series, measure}, .onSuccess = {ui::set(ready, true), ui::set(unsaved, false)}}); + // Every action here, the section forms' submits included, edits the working copy the queries read, and says + // so (`changesInstance`), so the queries refetch after each one. + auto const load = screen.mutation( + "load", sample, + {.body = {{&LoadSample::id, sampleId}}, .onSuccess = {ui::set(ready, true), ui::set(unsaved, false)}}); + auto const make = + screen.mutation("make", sample, {.onSuccess = {ui::set(ready, true), ui::set(unsaved, false)}}); auto const clear = screen.mutation("clear", sample, {.onSuccess = {ui::set(unsaved, true)}}); auto const pick = screen.mutation("pick", sample, @@ -384,25 +357,24 @@ ui::Screen sampleScreen() { {.label = ui::t("lims.receivedOn", "Received"), .widget = std::string{"epochDays"}}}, {&HeaderSection::remarks, {.blankAs = std::string{"empty"}}}}, - .onSuccess = {ui::set(unsaved, true), ui::refetch(header)}}) + .onSuccess = {ui::set(unsaved, true)}}) .id(ui::element("headerForm")) .testId("headerForm")}, {.title = ui::t("lims.header", "Header"), .padding = 2}), - ui::panel( - {ui::form(sample, {.onSuccess = {ui::set(unsaved, true), ui::refetch(series)}}) - .id(ui::element("samplingForm")) - .testId("samplingForm"), - ui::form(sample, {.onSuccess = {ui::set(unsaved, true), ui::refetch(measure)}}) - .id(ui::element("turbidityForm")) - .testId("turbidityForm") - .visible(turbidityShown), - ui::form(sample, {.onSuccess = {ui::set(unsaved, true), ui::refetch(measure)}}) - .id(ui::element("nitrateForm")) - .testId("nitrateForm") - .visible(nitrateShown), - optionalToggle(kTurbidity, "turbidity", showTurbidity), - optionalToggle(kNitrate, "nitrate", showNitrate)}, - {.title = ui::t("lims.measurements", "Measurements"), .padding = 2}), + ui::panel({ui::form(sample, {.onSuccess = {ui::set(unsaved, true)}}) + .id(ui::element("samplingForm")) + .testId("samplingForm"), + ui::form(sample, {.onSuccess = {ui::set(unsaved, true)}}) + .id(ui::element("turbidityForm")) + .testId("turbidityForm") + .visible(turbidityShown), + ui::form(sample, {.onSuccess = {ui::set(unsaved, true)}}) + .id(ui::element("nitrateForm")) + .testId("nitrateForm") + .visible(nitrateShown), + optionalToggle(kTurbidity, "turbidity", showTurbidity), + optionalToggle(kNitrate, "nitrate", showNitrate)}, + {.title = ui::t("lims.measurements", "Measurements"), .padding = 2}), ui::panel( {ui::custom(seriesGrid, {.props = {{"rows", rowsOf}}}).a11y("Dilution series").testId("seriesGrid"), ui::custom(recoveryCurve, {.props = {{"rows", rowsOf}}}) diff --git a/examples/lims/screens/tests/test_lims_screens.cpp b/examples/lims/screens/tests/test_lims_screens.cpp index d796cf8c3..768de9953 100644 --- a/examples/lims/screens/tests/test_lims_screens.cpp +++ b/examples/lims/screens/tests/test_lims_screens.cpp @@ -16,8 +16,10 @@ #include #include #include +#include #include #include +#include #include "lims/auth/lims_authorizer.hpp" #include "lims/dto/analysis_dto.hpp" @@ -60,6 +62,19 @@ TEST_CASE("lims screens: every screen validates, and the bundle builds", "[lims- REQUIRE(bundle.has_value()); } +// Every document loads against the catalog the server serves with no warning: each mutation's action declares its +// effects (or `none`), and no refresh declaration repeats what the effects already cover. +TEST_CASE("lims_bundle_loads_without_warnings", "[lims-screens]") { + auto const bundle = lims::screens::bundle(); + INFO((bundle ? std::string{} : bundle.error().message())); + REQUIRE(bundle.has_value()); + std::vector const findings = morph::examples::testing::bundleLoadFindings(**bundle); + for (std::string const& finding : findings) { + UNSCOPED_INFO(finding); + } + CHECK(findings.empty()); +} + // Mutations: show the sample before opening it (the header is empty); enable every transition whatever the state // (start is enabled while the sample is only registered). TEST_CASE("lims screens: the lifecycle screen opens its sample and allows the transition its state allows", @@ -249,8 +264,9 @@ TEST_CASE("lims screens: the results screen previews the reported concentration CHECK(harness.problems().empty()); } -// Mutation: drop the row capture's `patch` (the row keeps 12.500, since a row capture refetches nothing). -TEST_CASE("lims screens: a row's write-through capture patches its row in place", "[lims-screens]") { +// Mutation: drop `changes` from `CaptureConcentration`'s effects (the row keeps 12.500, since nothing else +// refetches the list after a row's capture). +TEST_CASE("lims screens: a row's re-capture shows its new value through the capture's effects", "[lims-screens]") { DbFixture fixture; std::int64_t sampleId = 0; { @@ -286,8 +302,8 @@ TEST_CASE("lims screens: a row's write-through capture patches its row in place" auto const entry = harness.find("TextInput", "recapture"); REQUIRE(entry); harness.backend().submit(*entry, "13.000"); - bool const patched = shows("13.000"); - if (!patched) { + bool const recaptured = shows("13.000"); + if (!recaptured) { for (int const widget : harness.backend().all("Text")) { UNSCOPED_INFO("text: " << harness.backend().prop(widget, "text")); } @@ -295,6 +311,6 @@ TEST_CASE("lims screens: a row's write-through capture patches its row in place" for (std::string const& problem : harness.problems()) { UNSCOPED_INFO(problem); } - CHECK(patched); + CHECK(recaptured); CHECK(harness.problems().empty()); } From cb34db977d64c263ed626171072331ff62902c2a Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 01:49:48 +0200 Subject: [PATCH 56/68] ui: an MCP adapter runs agent tool calls through an ordinary connection McpAdapter speaks MCP's JSON-RPC 2.0 over one server connection: one tool per action the session may run, with the action's schema, a keyed model's tool taking its instance, and an evaluate tool for a dry run. A call is an ordinary execute, so every check applies and every watching GUI is told; a refused call is a tool result with isError. serveStdio drives it over MCP's stdio transport, pumping the adapter's owner. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- CMakeLists.txt | 1 + docs/spec/ui/agent.md | 79 ++++++ include/morph/ui/agent.hpp | 522 +++++++++++++++++++++++++++++++++++++ tests/CMakeLists.txt | 1 + tests/test_push_agent.cpp | 348 +++++++++++++++++++++++++ 5 files changed, 951 insertions(+) create mode 100644 docs/spec/ui/agent.md create mode 100644 include/morph/ui/agent.hpp create mode 100644 tests/test_push_agent.cpp diff --git a/CMakeLists.txt b/CMakeLists.txt index db53d3f96..e31deab37 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -515,6 +515,7 @@ target_sources(morph include/morph/ui/service_client.hpp include/morph/ui/shell_frame.hpp include/morph/ui/explorer.hpp + include/morph/ui/agent.hpp include/morph/ui/preferences.hpp include/morph/ui/preference_sync.hpp include/morph/ui/host_vocabulary.hpp diff --git a/docs/spec/ui/agent.md b/docs/spec/ui/agent.md new file mode 100644 index 000000000..777a8928c --- /dev/null +++ b/docs/spec/ui/agent.md @@ -0,0 +1,79 @@ +# `morph::ui::McpAdapter` — agents through the server + +`include/morph/ui/agent.hpp`. An agent such as Claude Code acts on an application through the same dispatch as every +GUI: its tool calls are ordinary envelopes on a server connection of their own. Every check a GUI's calls get applies +to them, and every GUI watching what they change is told. The design is spec 8 §11 +(`docs/superpowers/specs/2026-10-16-server-push-design.md`). + +## Contents + +- [Shape](#shape) +- [Tools](#tools) +- [Identity and checks](#identity-and-checks) +- [Threads](#threads) +- [stdio](#stdio) +- [Design decisions](#design-decisions) + +## Shape + +| Type | Role | +|---|---| +| `McpOptions` | `session`, the identity every envelope carries; `serverName` and `serverVersion`, answered by `initialize`; `callbacks`, the serial owner `handle` completes on (required) | +| `McpAdapter(server, options)` | Opens a connection on `server` (`openConnection`). Its destructor closes it, which releases the adapter's private instances | +| `handle(message)` | One JSON-RPC 2.0 message in. Completes with the reply's text, or with nothing for a notification. It never fails: every fault is answered as a JSON-RPC error | +| `serveStdio(adapter, owner, in, out)` | MCP's stdio transport over newline-delimited streams | + +`handle` answers `initialize`, `ping`, `tools/list` and `tools/call`. Any other request is refused with error +`-32601` (`kMcpMethodNotFound`). A message that is not JSON is `-32700`; one with no `method` is `-32600`; a +`tools/call` the adapter cannot use (no tool name, arguments that are not an object, a tool the catalog does not +know, a keyed tool without its instance) is `-32602`. A notification, which carries no `id`, gets no reply, +whatever its method. + +## Tools + +The adapter reads the session's `ui-catalog` once, at the first call that needs it, and keeps it for the +adapter's life. + +- **One tool per permitted action**, named by the action id: the catalog's `permitted`, which covers the actions a + served document names and the principal may run. Its `inputSchema` is the action's schema from the model's + `schemas`, made an object. +- **A keyed model's tool** also takes `instance`, the instance key, which is required. The call runs on the shared + instance with that key, registered (`register` with `shared`) at its first use and kept by model type and key. +- **Any other model's tool** runs on one private instance per model type, registered on the adapter's connection at + its first call. +- **`evaluate`** takes `{action, body}` and is a dry run through `ui-evaluate`: nothing runs and nothing changes. +- **A call's result** is `{content: [{type: "text", text}], isError}`. The text is the action's result JSON, or the + server's `err` message with `isError: true`. A refused call is such a result, not a JSON-RPC error, so the agent + sees why. + +## Identity and checks + +The deployer builds the adapter with the `session::Context` its envelopes carry, for example from a token in the +environment. The server authenticates it on every envelope, as it does a GUI's, so authentication and the +principal, `authorize` and `authorizeInstance`, limits, the journal, cancellation and the in-flight count all +apply. `authorize` reads the session as sent, before the server writes the verified principal into it, so the +session carries the principal its token proves. + +Fan-out treats the adapter's connection like any other. A successful call's changes reach every watching +connection, and a refused call changes nothing and posts nothing. + +## Threads + +A `Completion` is delivered on a serial owner, so the adapter takes one, `McpOptions::callbacks`. It must outlive +the adapter and every call still in flight. The server answers on its own pool threads, and the adapter's caches +are guarded by a mutex that is never held while a continuation runs: a continuation may need the lock again. + +## stdio + +`serveStdio` reads one message per line of `in`, skipping empty lines, and writes each reply on a line of `out`, +flushed, in order, until `in` ends. Its calling thread is the adapter's owner: it pumps the `MainThreadExecutor` +it is given while it waits for each reply. An agent launches the server process with an option that turns it on +(bank's server: `--mcp-stdio`), and the process serves its WebSocket clients as before. + +## Design decisions + +- **No transport in the adapter.** `handle` is message in, message out, so the same adapter serves stdio today + and an HTTP transport later, and a test drives it directly. +- **A refused call is a tool result, not a protocol error.** An agent reads `isError` and the server's reason and + can act on it; a JSON-RPC error would tell it only that the call was malformed. +- **`ping` is answered** because MCP's base protocol requires it of every server. diff --git a/include/morph/ui/agent.hpp b/include/morph/ui/agent.hpp new file mode 100644 index 000000000..4a141a153 --- /dev/null +++ b/include/morph/ui/agent.hpp @@ -0,0 +1,522 @@ +// SPDX-License-Identifier: Apache-2.0 +#pragma once + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../core/completion.hpp" +#include "../core/executor.hpp" +#include "../core/remote.hpp" +#include "../core/wire.hpp" +#include "../session/session.hpp" +#include "bundle.hpp" +#include "value.hpp" + +/// @file +/// The MCP adapter (spec 8 §11): an agent's tool calls run through the server as ordinary envelopes on a +/// connection of their own, so every check a GUI gets applies to them, and every GUI watching what they change is +/// told. The adapter speaks MCP's JSON-RPC 2.0 and knows no transport; `serveStdio` drives it over MCP's stdio +/// transport. + +namespace morph::ui { + +/// @brief What an `McpAdapter` is built with. +struct McpOptions { + /// @brief The session every envelope carries: the agent's identity, which the server authenticates on each one, + /// as it does a GUI's. + session::Context session; + /// @brief The name `initialize` answers in `serverInfo`. + std::string serverName = "morph"; + /// @brief The version `initialize` answers in `serverInfo`. + std::string serverVersion = "1"; + /// @brief The owner `handle`'s completions are delivered on: a serial executor that outlives the adapter and + /// every call still in flight, such as the `exec::MainThreadExecutor` `serveStdio` pumps. Borrowed; + /// required. + exec::IExecutor* callbacks = nullptr; +}; + +/// @brief The JSON-RPC error code of a method the adapter does not answer. +inline constexpr std::int64_t kMcpMethodNotFound = -32601; +/// @brief The JSON-RPC error code of a message that is not JSON. +inline constexpr std::int64_t kMcpParseError = -32700; +/// @brief The JSON-RPC error code of a message that is not a request. +inline constexpr std::int64_t kMcpInvalidRequest = -32600; +/// @brief The JSON-RPC error code of a call whose arguments the adapter cannot use. +inline constexpr std::int64_t kMcpInvalidParams = -32602; + +/// @brief Serves MCP's tools over one server connection (spec 8 §11). +/// +/// It answers `initialize`, `tools/list`, `tools/call` and `ping`, and refuses every other request with JSON-RPC error +/// `-32601`; a notification, which carries no `id`, gets no reply. +/// +/// - **Tools.** One per action the catalog lists as permitted for the session, named by the action id, whose +/// `inputSchema` is the action's schema from `schemas`. A keyed model's tool also takes the instance key as the +/// argument `instance`, and runs on the shared instance with that key. Any other model's tool runs on one private +/// instance per model type, registered on the adapter's connection at its first call. A tool `evaluate`, taking +/// `{action, body}`, is a dry run through `ui-evaluate`. +/// - **A call's result** is the action's result JSON as text content, or the server's `err` message with `isError`. +/// - **The connection** is opened when the adapter is made and closed when it is destroyed, which releases its +/// private instances. The server treats it like any other connection, so its changes reach every watching GUI. +/// +/// `handle` is called on the owner, `McpOptions::callbacks`, and its completion is delivered there. +class McpAdapter { +public: + /// @brief An adapter on a new connection of @p server. + /// @param server The server; the adapter keeps it alive. + /// @param options Its options. + /// @throws std::invalid_argument without a server or a callbacks executor. + McpAdapter(std::shared_ptr server, McpOptions options) + : _shared{std::make_shared(std::move(server), std::move(options))} { + if (_shared->server == nullptr || _shared->options.callbacks == nullptr) { + throw std::invalid_argument{"McpAdapter: a server and a callbacks executor are required"}; + } + _shared->cid = _shared->server->openConnection(); + } + + /// @brief Closes the adapter's connection, releasing its private instances. + ~McpAdapter() { _shared->server->closeConnection(_shared->cid); } + + McpAdapter(McpAdapter const&) = delete; + McpAdapter& operator=(McpAdapter const&) = delete; + McpAdapter(McpAdapter&&) = delete; + McpAdapter& operator=(McpAdapter&&) = delete; + + /// @brief Handles one JSON-RPC message. + /// @param message The message's JSON text. + /// @return Completes with the reply's JSON text, or with nothing for a notification. It never fails: a fault is + /// answered as a JSON-RPC error. + [[nodiscard]] async::Completion> handle(std::string message) { + auto [completion, promise] = + async::Completion>::makeSettleable(_shared->options.callbacks); + auto settle = std::make_shared(std::move(promise)); + Shared::dispatch(_shared, std::move(message), + [settle](std::optional reply) { settle->resolve(std::move(reply)); }); + return std::move(completion); + } + +private: + using Reply = std::function)>; + using EnvelopeReply = std::function; + + // Everything a reply still in flight needs, kept alive by it after the adapter is gone. + struct Shared { + Shared(std::shared_ptr made, McpOptions given) + : server{std::move(made)}, options{std::move(given)} {} + + std::shared_ptr server; + McpOptions options; + backend::ConnectionId cid{}; + std::atomic nextCall{0}; + std::mutex mutex; + std::optional catalog; + // Model ids: a private instance by model type, a shared one by model type and key. + std::map> privateIds; + std::map, std::uint64_t> sharedIds; + + // Sends one envelope on the adapter's connection, carrying its session. + static void send(std::shared_ptr const& self, wire::Envelope env, EnvelopeReply done) { + env.callId = self->nextCall.fetch_add(1) + 1; + env.session = self->options.session; + self->server->handle( + wire::encode(env), + [done = std::move(done)](std::string const& reply) { + try { + done(wire::decode(reply)); + } catch (std::exception const& error) { + done(wire::makeErr(error.what())); + } + }, + self->cid); + } + + static std::string write(Json const& value) { return glz::write_json(value).value_or(std::string{"null"}); } + + static std::string result(Json const& id, Json value) { + Json reply; + reply["jsonrpc"] = "2.0"; + reply["id"] = id; + reply["result"] = std::move(value); + return write(reply); + } + + static std::string error(Json const& id, std::int64_t code, std::string const& message) { + Json reply; + reply["jsonrpc"] = "2.0"; + reply["id"] = id; + reply["error"]["code"] = code; + reply["error"]["message"] = message; + return write(reply); + } + + // A tool call's result: the text, flagged as an error or not. + static Json toolResult(std::string text, bool isError) { + Json content; + content["type"] = "text"; + content["text"] = std::move(text); + Json out; + out["content"] = Json::array_t{std::move(content)}; + out["isError"] = isError; + return out; + } + + static void dispatch(std::shared_ptr const& self, std::string const& message, Reply done) { + auto parsed = parseJson(message); + if (!parsed) { + done(error(Json{nullptr}, kMcpParseError, "not JSON: " + parsed.error())); + return; + } + Json::object_t const* const request = std::get_if(&parsed->data); + std::string const* const method = request != nullptr && request->contains("method") + ? std::get_if(&request->find("method")->second.data) + : nullptr; + bool const notification = request != nullptr && !request->contains("id"); + Json const id = + request != nullptr && request->contains("id") ? request->find("id")->second : Json{nullptr}; + if (method == nullptr) { + if (notification && request != nullptr) { + done(std::nullopt); + } else { + done(error(id, kMcpInvalidRequest, "no method")); + } + return; + } + if (notification) { + done(std::nullopt); + return; + } + Json const params = request->contains("params") ? request->find("params")->second : Json{}; + if (*method == "initialize") { + done(result(id, initialize(*self, params))); + } else if (*method == "ping") { + done(result(id, Json::object_t{})); + } else if (*method == "tools/list") { + listTools(self, [id, done](Json tools) { done(result(id, std::move(tools))); }); + } else if (*method == "tools/call") { + callTool(self, params, [id, done](std::expected called) { + done(called ? result(id, std::move(*called)) : error(id, kMcpInvalidParams, called.error())); + }); + } else { + done(error(id, kMcpMethodNotFound, "method not found: " + *method)); + } + } + + static Json initialize(Shared const& self, Json const& params) { + Json out; + std::string const* const asked = params.contains("protocolVersion") + ? std::get_if(¶ms["protocolVersion"].data) + : nullptr; + out["protocolVersion"] = asked != nullptr ? *asked : std::string{"2025-06-18"}; + out["capabilities"]["tools"]["listChanged"] = false; + out["serverInfo"]["name"] = self.options.serverName; + out["serverInfo"]["version"] = self.options.serverVersion; + return out; + } + + // The catalog for the session, fetched once; an unreadable or refused one is empty. + static void withCatalog(std::shared_ptr const& self, std::function then) { + // Copied under the lock and used after it: the continuation takes the lock again. + std::optional cached; + { + std::scoped_lock const lock{self->mutex}; + cached = self->catalog; + } + if (cached) { + then(*cached); + return; + } + wire::Envelope request; + request.kind = std::string{kCatalogKind}; + send(self, std::move(request), [self, then = std::move(then)](wire::Envelope const& reply) { + CatalogReply catalog; + if (reply.kind == "ok") { + if (auto read = parseCatalogReply(reply.body)) { + catalog = std::move(*read); + } + } + { + std::scoped_lock const lock{self->mutex}; + self->catalog = catalog; + } + then(catalog); + }); + } + + [[nodiscard]] static bool keyed(CatalogReply const& catalog, std::string const& modelType) { + if (!catalog.models) { + return false; + } + auto const found = catalog.models->find(modelType); + return found != catalog.models->end() && found->second.keyed; + } + + [[nodiscard]] static std::string modelOf(CatalogReply const& catalog, std::string_view actionId) { + if (!catalog.actions) { + return {}; + } + auto const found = catalog.actions->find(actionId); + return found == catalog.actions->end() ? std::string{} : found->second.modelType; + } + + // One tool: the action's schema, with `instance` added for a keyed model. + static Json tool(std::string const& actionId, Json schema, bool keyedModel) { + if (!std::holds_alternative(schema.data)) { + schema = Json::object_t{}; + } + schema["type"] = "object"; + if (keyedModel) { + schema["properties"]["instance"]["type"] = "string"; + schema["properties"]["instance"]["description"] = "the key of the instance to run on"; + if (!schema.contains("required") || !std::holds_alternative(schema["required"].data)) { + schema["required"] = Json::array_t{}; + } + std::get(schema["required"].data).emplace_back("instance"); + } + Json out; + out["name"] = actionId; + out["description"] = "Runs " + actionId + " on the server."; + out["inputSchema"] = std::move(schema); + return out; + } + + static Json evaluateTool() { + Json out; + out["name"] = "evaluate"; + out["description"] = + "A dry run: the action's body reconciled, its field errors, and whether it would run."; + out["inputSchema"]["type"] = "object"; + out["inputSchema"]["properties"]["action"]["type"] = "string"; + out["inputSchema"]["properties"]["body"]["type"] = "object"; + out["inputSchema"]["required"] = Json::array_t{Json{"action"}, Json{"body"}}; + return out; + } + + static void listTools(std::shared_ptr const& self, std::function done) { + withCatalog(self, [self, done = std::move(done)](CatalogReply const& catalog) { + std::set> types; + for (std::string const& action : catalog.permitted) { + if (std::string type = modelOf(catalog, action); !type.empty()) { + types.insert(std::move(type)); + } + } + // The schemas of every model a permitted action belongs to, then the tools. + struct Gather { + std::mutex mutex; + std::size_t waiting = 0; + std::map> schemas; + }; + auto const gather = std::make_shared(); + gather->waiting = types.size(); + auto const finish = [self, catalog, done, gather] { + Json::array_t tools; + for (std::string const& action : catalog.permitted) { + std::string const type = modelOf(catalog, action); + if (type.empty()) { + continue; + } + auto const found = gather->schemas.find(action); + tools.push_back(tool(action, found == gather->schemas.end() ? Json{} : found->second, + keyed(catalog, type))); + } + tools.push_back(evaluateTool()); + Json out; + out["tools"] = std::move(tools); + done(std::move(out)); + }; + if (types.empty()) { + finish(); + return; + } + for (std::string const& type : types) { + send(self, wire::makeSchemas(type), [gather, finish](wire::Envelope const& reply) { + bool last = false; + { + std::scoped_lock const lock{gather->mutex}; + if (reply.kind == "ok") { + if (auto parsed = parseJson(reply.body)) { + if (auto* const actions = std::get_if(&parsed->data)) { + for (auto& [action, schema] : *actions) { + gather->schemas.insert_or_assign(action, std::move(schema)); + } + } + } + } + last = --gather->waiting == 0; + } + if (last) { + finish(); + } + }); + } + }); + } + + // The model id a tool call runs on, registering the instance at its first use. + static void withInstance(std::shared_ptr const& self, std::string const& type, + std::optional const& key, + std::function)> then) { + std::optional cached; + { + std::scoped_lock const lock{self->mutex}; + if (key) { + if (auto const found = self->sharedIds.find(std::pair{type, *key}); + found != self->sharedIds.end()) { + cached = found->second; + } + } else if (auto const found = self->privateIds.find(type); found != self->privateIds.end()) { + cached = found->second; + } + } + if (cached) { + then(*cached); + return; + } + wire::Envelope request = key ? wire::makeRegisterShared(type, *key) : wire::makeRegister(type); + send(self, std::move(request), [self, type, key, then = std::move(then)](wire::Envelope const& reply) { + if (reply.kind != "ok" || reply.modelId == 0) { + then(std::unexpected(reply.message.empty() ? std::string{"the instance was not registered"} + : reply.message)); + return; + } + { + std::scoped_lock const lock{self->mutex}; + if (key) { + self->sharedIds.insert_or_assign(std::pair{type, *key}, reply.modelId); + } else { + self->privateIds.insert_or_assign(type, reply.modelId); + } + } + then(reply.modelId); + }); + } + + [[nodiscard]] static std::optional instanceKey(Json const& value) { + if (auto const* const text = std::get_if(&value.data)) { + return *text; + } + if (std::holds_alternative(value.data)) { + return std::nullopt; + } + return write(value); + } + + static void callTool(std::shared_ptr const& self, Json const& params, + std::function)> done) { + std::string const* const name = + params.contains("name") ? std::get_if(¶ms["name"].data) : nullptr; + if (name == nullptr) { + done(std::unexpected(std::string{"tools/call names no tool"})); + return; + } + Json arguments = params.contains("arguments") ? params["arguments"] : Json::object_t{}; + if (!std::holds_alternative(arguments.data)) { + done(std::unexpected(std::string{"the arguments are not an object"})); + return; + } + withCatalog(self, [self, tool = *name, arguments = std::move(arguments), + done = std::move(done)](CatalogReply const& catalog) mutable { + if (tool == "evaluate") { + evaluate(self, catalog, arguments, std::move(done)); + return; + } + std::string const type = modelOf(catalog, tool); + if (type.empty()) { + done(std::unexpected("no tool named " + tool)); + return; + } + auto& fields = std::get(arguments.data); + std::optional key; + if (keyed(catalog, type)) { + auto const found = fields.find("instance"); + key = found == fields.end() ? std::nullopt : instanceKey(found->second); + if (!key) { + done(std::unexpected(tool + " runs on a keyed model: give the instance key as `instance`")); + return; + } + fields.erase("instance"); + } + std::string body = write(arguments); + withInstance( + self, type, key, + [self, type, tool, body = std::move(body), + done = std::move(done)](std::expected const& id) mutable { + if (!id) { + done(toolResult(id.error(), true)); + return; + } + wire::Envelope request; + request.kind = "execute"; + request.modelId = *id; + request.modelType = type; + request.actionType = tool; + request.body = std::move(body); + send(self, std::move(request), [done = std::move(done)](wire::Envelope const& reply) { + done(reply.kind == "ok" ? toolResult(reply.body, false) : toolResult(reply.message, true)); + }); + }); + }); + } + + static void evaluate(std::shared_ptr const& self, CatalogReply const& catalog, Json const& arguments, + std::function)> done) { + std::string const* const action = + arguments.contains("action") ? std::get_if(&arguments["action"].data) : nullptr; + std::string const type = action == nullptr ? std::string{} : modelOf(catalog, *action); + if (type.empty()) { + done(std::unexpected(std::string{"evaluate names no action the catalog knows"})); + return; + } + wire::Envelope request; + request.kind = std::string{kEvaluateKind}; + request.modelType = type; + request.actionType = *action; + request.body = arguments.contains("body") ? write(arguments["body"]) : std::string{"{}"}; + send(self, std::move(request), [done = std::move(done)](wire::Envelope const& reply) { + done(reply.kind == "ok" ? toolResult(reply.body, false) : toolResult(reply.message, true)); + }); + } + }; + + std::shared_ptr _shared; +}; + +/// @brief Serves an adapter over MCP's stdio transport: one JSON-RPC message per line of @p in, each reply on a +/// line of @p out, in order, until @p in ends. An empty line is skipped. +/// +/// The calling thread is the adapter's owner: it pumps @p owner while it waits for each reply. +/// @param adapter The adapter; its `McpOptions::callbacks` is @p owner. +/// @param owner The adapter's owner, run on the calling thread. +/// @param in Where messages arrive. +/// @param out Where replies go; flushed after each. +inline void serveStdio(McpAdapter& adapter, exec::MainThreadExecutor& owner, std::istream& in, std::ostream& out) { + std::string line; + while (std::getline(in, line)) { + if (line.empty()) { + continue; + } + std::optional> replied; + adapter.handle(line).then([&replied](std::optional const& answer) { replied = answer; }); + while (!replied) { + owner.runFor(std::chrono::milliseconds{10}); + } + if (*replied) { + out << **replied << '\n' << std::flush; + } + } +} + +} // namespace morph::ui diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 1839075b2..b2e83fa6e 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -28,6 +28,7 @@ add_executable(morph_tests test_push_channel.cpp test_push_watch.cpp test_push_cache.cpp + test_push_agent.cpp test_push_documents.cpp push_records_late.cpp test_executor_extra.cpp diff --git a/tests/test_push_agent.cpp b/tests/test_push_agent.cpp new file mode 100644 index 000000000..c20ea86d9 --- /dev/null +++ b/tests/test_push_agent.cpp @@ -0,0 +1,348 @@ +// SPDX-License-Identifier: Apache-2.0 + +// The MCP adapter (spec 8 §11): an agent's tool calls run through the server on a connection of their own, with every +// check a GUI's calls get, and a GUI watching what they change is told. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +// NOLINTBEGIN(misc-use-internal-linkage): registered types must have external linkage. +namespace agent_test { + +struct Note { + std::int64_t id = 0; + std::string text; +}; +struct ListNotes {}; +struct AddNote { + std::string text; +}; +struct WipeNotes {}; + +struct Store { + std::mutex mutex; + std::vector notes; +}; +inline Store& store() { + static Store shared; + return shared; +} + +struct NotesModel { + std::vector execute(ListNotes const& /*action*/) { + std::scoped_lock const lock{store().mutex}; + return store().notes; + } + Note execute(AddNote const& action) { + std::scoped_lock const lock{store().mutex}; + store().notes.push_back(Note{.id = static_cast(store().notes.size()) + 1, .text = action.text}); + return store().notes.back(); + } + bool execute(WipeNotes const& /*action*/) { + std::scoped_lock const lock{store().mutex}; + store().notes.clear(); + return true; + } +}; + +} // namespace agent_test +// NOLINTEND(misc-use-internal-linkage) + +BRIDGE_REGISTER_MODEL(agent_test::NotesModel, "Ag_Notes") +BRIDGE_REGISTER_ACTION(agent_test::NotesModel, agent_test::ListNotes, "Ag_List") +BRIDGE_REGISTER_ACTION(agent_test::NotesModel, agent_test::AddNote, "Ag_Add") +BRIDGE_REGISTER_ACTION(agent_test::NotesModel, agent_test::WipeNotes, "Ag_Wipe") +BRIDGE_REGISTER_RECORD(agent_test::Note, "Ag_Note") +BRIDGE_REGISTER_EFFECTS(agent_test::AddNote, morph::effects::changes) +BRIDGE_REGISTER_EFFECTS(agent_test::WipeNotes, morph::effects::changes) + +namespace { + +using namespace std::chrono_literals; +namespace ui = morph::ui; + +// A screen naming every action, so the catalog lists each one the principal may run. +constexpr std::string_view kNotes = R"({ + "vocab": "ui/1", "id": "notes", + "models": { "notes": { "type": "Ag_Notes" } }, + "queries": { "list": { "model": "notes", "action": "Ag_List" } }, + "mutations": { "add": { "model": "notes", "action": "Ag_Add" }, "wipe": { "model": "notes", "action": "Ag_Wipe" } }, + "view": { "kind": "text", "text": "Notes" } })"; + +// The token is the principal; `agent` may not wipe. +struct Authorizer final : morph::session::IAuthorizer { + // NOLINTNEXTLINE(bugprone-easily-swappable-parameters): IAuthorizer's signature. + [[nodiscard]] bool authorize(morph::session::Context const& ctx, std::string_view /*modelType*/, + std::string_view actionType) const override { + return !(ctx.principal == "agent" && actionType == "Ag_Wipe"); + } + [[nodiscard]] std::optional authenticate(morph::session::Context const& ctx) const override { + return ctx.token.empty() ? std::nullopt : std::optional{ctx.token}; + } +}; + +// What the server wrote to a GUI connection: its events. +struct EventWire { + std::mutex mutex; + std::vector sent; + + morph::backend::ConnectionWriter writer() { + return {.write = + [this](std::string frame) { + std::scoped_lock const lock{mutex}; + sent.push_back(std::move(frame)); + }, + .backlog = [] { return std::size_t{0}; }}; + } + + std::size_t count() { + std::scoped_lock const lock{mutex}; + return sent.size(); + } +}; + +struct Rig { + Rig() { + { + std::scoped_lock const lock{agent_test::store().mutex}; + agent_test::store().notes.clear(); + } + ui::author::CatalogFacts facts; + facts.action(); + facts.action(); + facts.action(); + ui::UiRegistry registry{ui::ApplicationInfo{.id = "notes", .version = "1"}, facts.catalog()}; + ui::CatalogEntry entry; + entry.id = "notes"; + REQUIRE(registry.addScreen(entry, std::string{kNotes}).has_value()); + auto built = registry.build(); + REQUIRE(built.has_value()); + morph::backend::ServerConfig config; + config.services.push_back(std::make_shared(*built)); + server = + std::make_shared(pool, std::make_shared(), std::move(config)); + } + + // A GUI connection that negotiated events and watches `Ag_Note`. + morph::backend::ConnectionId gui() { + auto const cid = server->openConnection(); + server->setConnectionWriter(cid, wire.writer()); + morph::wire::Envelope hello; + hello.kind = std::string{ui::kHelloKind}; + hello.callId = 1; + hello.session.token = "gui"; + hello.body = R"({"vocabularies":["ui/1","expr/1"],"events":true})"; + REQUIRE(morph::wire::decode(server->handleInline(morph::wire::encode(hello), cid)).kind == "ok"); + morph::wire::Envelope watch; + watch.kind = std::string{morph::wire::kWatchKind}; + watch.callId = 2; + watch.session.token = "gui"; + watch.body = R"({"records":["Ag_Note"],"instances":[]})"; + REQUIRE(morph::wire::decode(server->handleInline(morph::wire::encode(watch), cid)).kind == "ok"); + return cid; + } + + std::unique_ptr adapter(std::string token = "agent") { + // `authorize` reads the session as sent, so it carries the principal its token proves. + morph::session::Context session; + session.principal = token; + session.token = std::move(token); + return std::make_unique(server, ui::McpOptions{.session = session, .callbacks = &callbacks}); + } + + morph::exec::ThreadPoolExecutor pool{2}; + // The adapter's owner, which this thread pumps: declared before the server, so it outlives every reply. + morph::exec::MainThreadExecutor callbacks; + std::shared_ptr server; + EventWire wire; +}; + +// The adapter's reply to @p message, waited for on the adapter's owner. +std::optional ask(Rig& rig, ui::McpAdapter& adapter, std::string message) { + std::optional> replied; + adapter.handle(std::move(message)).then([&replied](std::optional const& answer) { + replied = answer; + }); + auto const deadline = std::chrono::steady_clock::now() + 10s; + while (!replied && std::chrono::steady_clock::now() < deadline) { + rig.callbacks.runFor(10ms); + } + REQUIRE(replied.has_value()); + return *replied; +} + +ui::Json json(std::optional const& text) { + REQUIRE(text.has_value()); + auto parsed = ui::parseJson(*text); + REQUIRE(parsed.has_value()); + return *parsed; +} + +std::string dump(ui::Json const& value) { return glz::write_json(value).value_or(std::string{}); } + +std::vector toolNames(ui::Json const& reply) { + std::vector names; + for (auto const& tool : std::get(reply["result"]["tools"].data)) { + names.push_back(std::get(tool["name"].data)); + } + return names; +} + +bool waitFor(auto done) { + auto const deadline = std::chrono::steady_clock::now() + 5s; + while (!done()) { + if (std::chrono::steady_clock::now() > deadline) { + return false; + } + std::this_thread::sleep_for(5ms); + } + return true; +} + +constexpr std::string_view kAdd = + R"({"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"Ag_Add","arguments":{"text":"hello"}}})"; + +} // namespace + +TEST_CASE("initialize_answers_capabilities", "[push][agent]") { + Rig rig; + auto adapter = rig.adapter(); + auto const reply = json(ask(rig, *adapter, + R"({"jsonrpc":"2.0","id":1,"method":"initialize",)" + R"("params":{"protocolVersion":"2025-06-18","capabilities":{}}})")); + CHECK(std::get(reply["jsonrpc"].data) == "2.0"); + CHECK(dump(reply["id"]) == "1"); + CHECK(std::get(reply["result"]["protocolVersion"].data) == "2025-06-18"); + CHECK(reply["result"]["capabilities"].contains("tools")); + CHECK(std::get(reply["result"]["serverInfo"]["name"].data) == "morph"); +} + +// Mutation: list every action of the catalog, not only the permitted ones (Ag_Wipe appears). +TEST_CASE("tools_list_is_permitted_plus_evaluate", "[push][agent]") { + Rig rig; + auto adapter = rig.adapter(); + auto const reply = json(ask(rig, *adapter, R"({"jsonrpc":"2.0","id":2,"method":"tools/list"})")); + std::vector names = toolNames(reply); + std::ranges::sort(names); + CHECK(names == std::vector{"Ag_Add", "Ag_List", "evaluate"}); + for (auto const& tool : std::get(reply["result"]["tools"].data)) { + if (std::get(tool["name"].data) == "Ag_Add") { + CHECK(tool["inputSchema"]["properties"].contains("text")); + } + } +} + +TEST_CASE("tools_call_runs_and_returns_result_text", "[push][agent]") { + Rig rig; + auto adapter = rig.adapter(); + auto const reply = json(ask(rig, *adapter, std::string{kAdd})); + CHECK_FALSE(std::get(reply["result"]["isError"].data)); + auto const text = std::get(reply["result"]["content"][0]["text"].data); + CHECK(text == R"({"id":1,"text":"hello"})"); + auto const listed = json(ask(rig, *adapter, + R"({"jsonrpc":"2.0","id":4,"method":"tools/call",)" + R"("params":{"name":"Ag_List","arguments":{}}})")); + CHECK(std::get(listed["result"]["content"][0]["text"].data) == R"([{"id":1,"text":"hello"}])"); +} + +// Mutation: run the call without the session (the server refuses an unsigned execute) — or with it, through a path +// that skips `authorize` (the wipe runs and posts an event). +TEST_CASE("refused_call_is_error_and_posts_nothing", "[push][agent]") { + Rig rig; + auto const gui = rig.gui(); + static_cast(gui); + auto adapter = rig.adapter(); + auto const reply = json(ask(rig, *adapter, + R"({"jsonrpc":"2.0","id":5,"method":"tools/call",)" + R"("params":{"name":"Ag_Wipe","arguments":{}}})")); + CHECK(std::get(reply["result"]["isError"].data)); + // A refused call changes nothing, so no event follows; give the coalescer its window to prove it. + std::this_thread::sleep_for(200ms); + CHECK(rig.wire.count() == 0); +} + +// Mutation: open no connection of the adapter's own (call unscoped) — the GUI's watch still sees it, so instead drop +// the effects declaration of Ag_Add: no event reaches the GUI. +TEST_CASE("call_reaches_watching_gui_connection_as_event", "[push][agent]") { + Rig rig; + auto const gui = rig.gui(); + static_cast(gui); + auto adapter = rig.adapter(); + auto const reply = json(ask(rig, *adapter, std::string{kAdd})); + REQUIRE_FALSE(std::get(reply["result"]["isError"].data)); + REQUIRE(waitFor([&] { return rig.wire.count() >= 1; })); + std::scoped_lock const lock{rig.wire.mutex}; + auto const event = morph::wire::decode(rig.wire.sent.front()); + CHECK(event.kind == morph::wire::kEventKind); + CHECK(event.body.contains("Ag_Note")); +} + +TEST_CASE("unknown_method_is_32601", "[push][agent]") { + Rig rig; + auto adapter = rig.adapter(); + auto const reply = json(ask(rig, *adapter, R"({"jsonrpc":"2.0","id":"x","method":"resources/list"})")); + CHECK(dump(reply["id"]) == R"("x")"); + CHECK(dump(reply["error"]["code"]) == "-32601"); +} + +TEST_CASE("notification_gets_no_reply", "[push][agent]") { + Rig rig; + auto adapter = rig.adapter(); + CHECK_FALSE(ask(rig, *adapter, R"({"jsonrpc":"2.0","method":"notifications/initialized"})").has_value()); + CHECK_FALSE(ask(rig, *adapter, R"({"jsonrpc":"2.0","method":"tools/list"})").has_value()); +} + +TEST_CASE("evaluate_is_a_dry_run", "[push][agent]") { + Rig rig; + auto adapter = rig.adapter(); + auto const reply = json(ask(rig, *adapter, + R"({"jsonrpc":"2.0","id":6,"method":"tools/call","params":{"name":"evaluate",)" + R"("arguments":{"action":"Ag_Add","body":{"text":"draft"}}}})")); + CHECK_FALSE(std::get(reply["result"]["isError"].data)); + std::scoped_lock const lock{agent_test::store().mutex}; + CHECK(agent_test::store().notes.empty()); +} + +TEST_CASE("stdio_round_trip", "[push][agent]") { + Rig rig; + auto adapter = rig.adapter(); + std::istringstream in{R"({"jsonrpc":"2.0","id":1,"method":"initialize","params":{}})" + "\n\n" + R"({"jsonrpc":"2.0","method":"notifications/initialized"})" + "\n" + R"({"jsonrpc":"2.0","id":2,"method":"ping"})" + "\n"}; + std::ostringstream out; + ui::serveStdio(*adapter, rig.callbacks, in, out); + std::istringstream lines{out.str()}; + std::vector replies; + for (std::string line; std::getline(lines, line);) { + replies.push_back(line); + } + REQUIRE(replies.size() == 2); + CHECK(dump(json(replies.at(0))["id"]) == "1"); + CHECK(dump(json(replies.at(1))["id"]) == "2"); +} From 6e50258c8c9d7f6aca31b1e1fead02ab50716cf6 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 01:53:12 +0200 Subject: [PATCH 57/68] ui: the catalog states the records derived from what an action changes A record derived from one an action updates or changes changes as a whole too. The server expands derivations before an event leaves, but the caller's own reply applied only the declared effects, so in process, where no event comes back, a reply could not mark entries holding a derived record (read from the code; no rung screen shows one today). CatalogFacts::action now lists each derived record as changes. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- docs/spec/ui/document.md | 3 +++ include/morph/ui/author.hpp | 17 +++++++++++++++++ tests/test_push_catalog.cpp | 11 +++++++++-- 3 files changed, 29 insertions(+), 2 deletions(-) diff --git a/docs/spec/ui/document.md b/docs/spec/ui/document.md index f8035453e..5c563be81 100644 --- a/docs/spec/ui/document.md +++ b/docs/spec/ui/document.md @@ -243,6 +243,9 @@ know the result's shape, and a patch that finds no list is reported by the inter `effects` and `records` are what the core registries hold for the action: `effects` is the list of `{kind, record?}` the action declared, `kind` one of `updates`, `changes`, `changesInstance` and `none`, `record` the record's wire name for the first two; it is absent when the action declared nothing, which makes the action a read. +A record derived (`derivedFrom`) from one the action `updates` or `changes`, directly or through others, is listed as +`changes` too, once. The derivation itself is never sent: a client knows only that the derived record changes, which +lets the caller's own reply mark entries holding it, in process as well as remotely. `records` lists the records the action's result holds as `{record, keyField?, paths}`, empty when it holds none. They are read from the registries where `CatalogFacts::action` records the action, so an invalid effects and records configuration (`EffectsRegistrationError`) fails at that call, at startup. They are what a client's watch is derived diff --git a/include/morph/ui/author.hpp b/include/morph/ui/author.hpp index 654449be1..20e090085 100644 --- a/include/morph/ui/author.hpp +++ b/include/morph/ui/author.hpp @@ -22,6 +22,7 @@ #include #include +#include "../core/changes.hpp" #include "../core/effects.hpp" #include "../core/logger.hpp" #include "../core/model_key.hpp" @@ -139,6 +140,22 @@ template ActionEffect{.kind = std::string{kind}, .record = effect.record.empty() ? std::nullopt : std::optional{effect.record}}); } + // A record derived from a changed one changes as a whole too. The server expands derivations before an event + // leaves; stating them here as well lets the caller's own reply mark entries holding the derived record, which + // matters in process, where no event comes back. + ::morph::effects::Changes derived; + for (auto const& effect : *declared) { + if (!effect.record.empty() && (effect.kind == ::morph::effects::Kind::Updates || + effect.kind == ::morph::effects::Kind::Changes)) { + ::morph::effects::detail::addDerivedChanges(::morph::effects::detail::resolved(), effect.record, derived); + } + } + for (auto const& change : derived.records) { + ActionEffect added{.kind = "changes", .record = change.record}; + if (std::ranges::find(out, added) == out.end()) { + out.push_back(std::move(added)); + } + } return out; } diff --git a/tests/test_push_catalog.cpp b/tests/test_push_catalog.cpp index acb406c4c..186587f6d 100644 --- a/tests/test_push_catalog.cpp +++ b/tests/test_push_catalog.cpp @@ -32,6 +32,10 @@ struct EntryList { std::vector entries; }; +struct EntryReport { + int count{}; +}; + struct EditEntry { std::int64_t id{}; }; @@ -61,6 +65,7 @@ using namespace push_catalog_test; namespace effects = morph::effects; BRIDGE_REGISTER_RECORD(Entry, "T_CatalogEntry") +BRIDGE_REGISTER_RECORD(EntryReport, "T_CatalogReport", morph::record::keyless, morph::record::derivedFrom) BRIDGE_REGISTER_MODEL(CatalogModel, "T_CatalogModel") BRIDGE_REGISTER_ACTION(CatalogModel, EditEntry, "T_CatalogEdit") BRIDGE_REGISTER_EFFECTS(EditEntry, effects::updates, effects::changes) @@ -127,8 +132,10 @@ TEST_CASE("catalog_facts_carry_effects_and_records", "[push][catalog]") { auto const& edit = facts.actions.at("T_CatalogEdit"); REQUIRE(edit.effects.has_value()); - REQUIRE(*edit.effects == - std::vector{{"updates", "T_CatalogEntry"}, {"changes", "T_CatalogEntry"}}); + // A record derived from a changed one is stated as changed too, once. + REQUIRE(*edit.effects == std::vector{{"updates", "T_CatalogEntry"}, + {"changes", "T_CatalogEntry"}, + {"changes", "T_CatalogReport"}}); REQUIRE(edit.records.size() == 1U); CHECK(edit.records.front().record == "T_CatalogEntry"); CHECK(edit.records.front().keyField == std::optional{"id"}); From 9dd36d03d5843593dea450cee11f53a71dfdd2b8 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 02:05:28 +0200 Subject: [PATCH 58/68] ui, examples, ci: pass clang-tidy and the coverage gate on the cache, documents and client The cache's row helpers take a record's paths and key field, so every optional is read where it is checked; sendWatch is const. The document checks move a query's into checkQuery, under the complexity limit. The shell registers the source's listeners in listen(). onConnected takes its listener by reference. The analyzer's leak report on Qt's queued callable is suppressed where its path starts, in our code. The effects link probe is listed as deliberately not profiled, beside the other link canaries. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- examples/common/app/app_source.cpp | 2 +- examples/common/app/app_source.hpp | 2 +- examples/common/app/client_shell.cpp | 34 +++++---- examples/common/app/tests/test_sources.cpp | 8 +-- include/morph/ui/document.hpp | 41 ++++++----- include/morph/ui/frontend.hpp | 2 +- include/morph/ui/interpreter.hpp | 80 ++++++++++++---------- scripts/check_coverage_objects.sh | 3 + src/qt/qt_websocket_server.cpp | 7 +- tests/test_push_cache.cpp | 2 +- 10 files changed, 103 insertions(+), 78 deletions(-) diff --git a/examples/common/app/app_source.cpp b/examples/common/app/app_source.cpp index 693ab4de9..1f843f9da 100644 --- a/examples/common/app/app_source.cpp +++ b/examples/common/app/app_source.cpp @@ -449,7 +449,7 @@ void ClientSource::connected() { } } -void ClientSource::onConnected(std::function listener) { _connectListeners.push_back(std::move(listener)); } +void ClientSource::onConnected(std::function const& listener) { _connectListeners.push_back(listener); } void ClientSource::onEvent(std::function listener) { _eventListeners.push_back(std::move(listener)); diff --git a/examples/common/app/app_source.hpp b/examples/common/app/app_source.hpp index 6acd1f512..f8c9e5715 100644 --- a/examples/common/app/app_source.hpp +++ b/examples/common/app/app_source.hpp @@ -171,7 +171,7 @@ class ClientSource : public ui::AppSource { /// the source first sends `ui-hello` again, so the new connection negotiates events, then calls the /// listeners. /// @param listener Called on the owner; it must stay callable while the source is open. - void onConnected(std::function listener) final; + void onConnected(std::function const& listener) final; /// @brief Adds a listener for the events the server sends: what a shell hands to `Session::deliverEvent`. /// @param listener Called on the owner, once per event, in the order they arrived. diff --git a/examples/common/app/client_shell.cpp b/examples/common/app/client_shell.cpp index 902e96e86..0922f0f64 100644 --- a/examples/common/app/client_shell.cpp +++ b/examples/common/app/client_shell.cpp @@ -196,6 +196,25 @@ class Shell final : public ui::MountedShell { Shell(Shell&&) = delete; Shell& operator=(Shell&&) = delete; + /// Hands the session the source's connects and events. Registered once: they reach whichever session is + /// current, and nothing once the shell is gone. + void listen(ClientSource& source) { + if (_listening) { + return; + } + _listening = true; + source.onConnected([this, alive = std::weak_ptr{_alive}] { + if (auto const live = alive.lock(); live != nullptr && *live && _session) { + _session->reconnected(); + } + }); + source.onEvent([this, alive = std::weak_ptr{_alive}](wire::EventBody const& event) { + if (auto const live = alive.lock(); live != nullptr && *live && _session) { + _session->deliverEvent(event); + } + }); + } + /// Loads the bundle's screens and mounts the start screen. void start(ui::Bundle const& bundle, ClientSource& source) { _source = &source; @@ -211,20 +230,7 @@ class Shell final : public ui::MountedShell { sessionOptions.developmentTools = _options.dev ? ui::DevelopmentTools::On : ui::DevelopmentTools::Off; sessionOptions.watcher = source.watcher(); _session = std::make_unique(_context->runtime(), std::move(sessionOptions)); - // Registered once: they reach whichever session is current, and nothing once the shell is gone. - if (!_listening) { - _listening = true; - source.onConnected([this, alive = std::weak_ptr{_alive}] { - if (auto const live = alive.lock(); live != nullptr && *live && _session) { - _session->reconnected(); - } - }); - source.onEvent([this, alive = std::weak_ptr{_alive}](wire::EventBody const& event) { - if (auto const live = alive.lock(); live != nullptr && *live && _session) { - _session->deliverEvent(event); - } - }); - } + listen(source); showPermitted(); ui::DocumentScreenHostOptions hostOptions; hostOptions.documents = [this](std::string_view screenId) -> std::shared_ptr { diff --git a/examples/common/app/tests/test_sources.cpp b/examples/common/app/tests/test_sources.cpp index 9c2640423..f94cb8a4d 100644 --- a/examples/common/app/tests/test_sources.cpp +++ b/examples/common/app/tests/test_sources.cpp @@ -522,10 +522,10 @@ class ReconnectingSource final : public morph::examples::ClientSource { kinds.push_back(request.kind); if (request.kind == "watch" || request.kind == "unwatch") { watchBodies.push_back(request.body); - morph::wire::Envelope ok; - ok.kind = "ok"; - ok.body = "{}"; - done(ok); + morph::wire::Envelope answer; + answer.kind = "ok"; + answer.body = "{}"; + done(answer); return; } done(morph::wire::decode(_service->handle(request, *morph::session::allowAllAuthorizer()))); diff --git a/include/morph/ui/document.hpp b/include/morph/ui/document.hpp index a3ea090c7..fa8c43b8f 100644 --- a/include/morph/ui/document.hpp +++ b/include/morph/ui/document.hpp @@ -2472,20 +2472,7 @@ class DocumentValidator { static_cast(type); } for (QueryDecl const& query : decl.queries) { - auto const action = checkModelAction(context, query.model, query.action, query.path); - // An evaluation runs no handler, so only a call can change anything. - if (action && action->effects && !query.evaluate) { - refuse(query.path, std::format("a query reads, so its action declares no effects; `{}` declares {}, " - "so it runs as a mutation", - query.action, describeEffects(*action->effects))); - } - if (query.body) { - checkExpr(*query.body, context); - } - if (query.when) { - checkCondition(*query.when, context, "`when`"); - } - checkRefreshOn(query, *lookup(context.scope, query.model)->model); + checkQuery(query, context); } for (MutationDecl const& mutation : decl.mutations) { checkMutation(mutation, context); @@ -2507,6 +2494,23 @@ class DocumentValidator { } } + void checkQuery(QueryDecl const& query, CheckContext const& context) { + auto const action = checkModelAction(context, query.model, query.action, query.path); + // An evaluation runs no handler, so only a call can change anything. + if (action && action->effects && !query.evaluate) { + refuse(query.path, std::format("a query reads, so its action declares no effects; `{}` declares {}, " + "so it runs as a mutation", + query.action, describeEffects(*action->effects))); + } + if (query.body) { + checkExpr(*query.body, context); + } + if (query.when) { + checkCondition(*query.when, context, "`when`"); + } + checkRefreshOn(query, *lookup(context.scope, query.model)->model); + } + void checkRefreshOn(QueryDecl const& query, ModelDecl const& model) { if (!_options->catalog.action) { return; @@ -3069,10 +3073,11 @@ class DocumentValidator { auto const held = _options->catalog.action(query.action); for (ActionEffect const& effect : *action.effects) { bool const records = effect.kind == "updates" || (!updatesOnly && effect.kind == "changes"); - if (records && held && effect.record && - std::ranges::find(held->records, *effect.record, &ActionRecordInfo::record) != held->records.end()) { - return std::format("`{}` {} `{}`, which `{}` holds", actionId, effect.kind, *effect.record, - query.name); + if (records && held && effect.record) { + std::string const& record = *effect.record; + if (std::ranges::find(held->records, record, &ActionRecordInfo::record) != held->records.end()) { + return std::format("`{}` {} `{}`, which `{}` holds", actionId, effect.kind, record, query.name); + } } if (!updatesOnly && effect.kind == "changesInstance" && alias != nullptr && aliasOf(query) == alias) { return std::format("`{}` changes the instance of `{}`, through which `{}` reads", actionId, diff --git a/include/morph/ui/frontend.hpp b/include/morph/ui/frontend.hpp index 45984aa79..99cf98459 100644 --- a/include/morph/ui/frontend.hpp +++ b/include/morph/ui/frontend.hpp @@ -166,7 +166,7 @@ class AppSource { /// /// The default does nothing: a local source has no connection to lose. /// @param listener Called on the owner after each connect; it must stay callable while the source is open. - virtual void onConnected(std::function listener) { static_cast(listener); } + virtual void onConnected(std::function const& listener) { static_cast(listener); } }; /// @brief A frontend: owns an event loop and a renderer, and runs one source's screens. diff --git a/include/morph/ui/interpreter.hpp b/include/morph/ui/interpreter.hpp index db9fccdca..45bcb3168 100644 --- a/include/morph/ui/interpreter.hpp +++ b/include/morph/ui/interpreter.hpp @@ -749,30 +749,35 @@ struct CacheEntry { } }; -/// @brief The rows of a keyed record an entry's value holds, at each of the record's paths. -/// @param value The entry's confirmed value. -/// @param held The record, as the entry's action returns it; it must have a key field. +/// @brief The rows of a keyed record a value holds, at each of the record's paths. +/// @param value The value. +/// @param paths The record's paths. +/// @param keyField The record's key field. /// @return The rows. -[[nodiscard]] inline std::vector keyedRowsOf(Value const& value, ActionRecordInfo const& held) { +[[nodiscard]] inline std::vector keyedRowsOf(Value const& value, std::vector const& paths, + std::string_view keyField) { std::vector rows; - for (std::string const& path : held.paths) { + for (std::string const& path : paths) { std::vector const members = pathMembers(path); - keyedRowsAt(value, members, *held.keyField, rows); + keyedRowsAt(value, members, keyField, rows); } return rows; } -/// @brief Merges new rows into an entry's value at each of the record's paths. -/// @param value The entry's confirmed value. -/// @param held The record, as the entry's action returns it; it must have a key field. -/// @param rows The new rows by key. +/// @brief Merges new rows into a value at each of the record's paths. +/// @param value The value. +/// @param paths The record's paths. +/// @param keyField The record's key field. +/// @param rows The new rows by key. /// @return The new value, or nothing when no row matched. -[[nodiscard]] inline std::optional patchHeldRows(Value const& value, ActionRecordInfo const& held, - RowsByKey const& rows) { +[[nodiscard]] inline std::optional patchHeldRows(Value const& value, std::vector const& paths, + std::string_view keyField, RowsByKey const& rows) { std::optional patched; - for (std::string const& path : held.paths) { + for (std::string const& path : paths) { std::vector const members = pathMembers(path); - if (auto next = patchKeyedRows(patched ? *patched : value, members, *held.keyField, rows)) { + auto next = patched ? patchKeyedRows(*patched, members, keyField, rows) + : patchKeyedRows(value, members, keyField, rows); + if (next) { patched = std::move(next); } } @@ -1278,7 +1283,7 @@ class Session { /// @brief Sends one watch request, ignoring its reply: a refused topic simply sends no events. /// @param kind The kind, `watch` or `unwatch`. /// @param body The body. - void sendWatch(std::string_view kind, wire::WatchBody const& body); + void sendWatch(std::string_view kind, wire::WatchBody const& body) const; /// @brief Marks every entry but an `evaluate` one stale and refetches the mounted ones, in one batch. void markAllStale(); @@ -1289,13 +1294,14 @@ class Session { /// @brief Applies one `updates` or `changes` effect of a reply: patches the entries holding the record, or marks /// them. - /// @param effect The effect; its record is set. + /// @param effect The effect. + /// @param record The record it names. /// @param info The action's catalog entry, for the records its reply holds. /// @param reply The reply's value. /// @param use Whether the reply was applied or discarded. /// @param marks Where entries to mark go. - void applyRecordEffect(ActionEffect const& effect, ActionInfo const& info, Value const& reply, ReplyUse use, - detail::Marks& marks); + void applyRecordEffect(ActionEffect const& effect, std::string const& record, ActionInfo const& info, + Value const& reply, ReplyUse use, detail::Marks& marks); /// @brief Finds the entries a `changes` event's record change names. /// @param change The change. @@ -2861,23 +2867,23 @@ inline void Session::applyReplyEffects(std::string const& typeId, std::string co } } } else if (effect.record && (effect.kind == "updates" || effect.kind == "changes")) { - applyRecordEffect(effect, *info, reply, use, marks); + applyRecordEffect(effect, *effect.record, *info, reply, use, marks); } } markAndRefetch(marks); } -inline void Session::applyRecordEffect(ActionEffect const& effect, ActionInfo const& info, Value const& reply, - ReplyUse use, detail::Marks& marks) { - std::string const& record = *effect.record; +inline void Session::applyRecordEffect(ActionEffect const& effect, std::string const& record, ActionInfo const& info, + Value const& reply, ReplyUse use, detail::Marks& marks) { // The reply's rows of the record, by key; none when the reply does not hold it keyed, or must only mark. auto const own = std::ranges::find(info.records, record, &ActionRecordInfo::record); - bool const patch = - effect.kind == "updates" && use == ReplyUse::Applied && own != info.records.end() && own->keyField.has_value(); + std::optional const ownKey = + effect.kind == "updates" && use == ReplyUse::Applied && own != info.records.end() ? own->keyField + : std::nullopt; detail::RowsByKey rows; - if (patch) { - for (Value const* const row : detail::keyedRowsOf(reply, *own)) { - rows.insert_or_assign(detail::canonicalJson(*row->field(*own->keyField)), row); + if (ownKey) { + for (Value const* const row : detail::keyedRowsOf(reply, own->paths, *ownKey)) { + rows.insert_or_assign(detail::canonicalJson(*row->field(*ownKey)), row); } } for (auto const& [key, entry] : _entries) { @@ -2886,11 +2892,11 @@ inline void Session::applyRecordEffect(ActionEffect const& effect, ActionInfo co if (held == nullptr) { continue; } - if (!patch || !held->keyField) { + if (!ownKey || !held->keyField) { marks.add(*entry); continue; } - if (auto patched = detail::patchHeldRows(entry->confirmed, *held, rows)) { + if (auto patched = detail::patchHeldRows(entry->confirmed, held->paths, *held->keyField, rows)) { confirm(*entry, *patched); } // A fetch in flight may have read before the write, and would settle over the patch. @@ -2939,7 +2945,7 @@ inline void Session::countTopics(detail::CacheEntry const& entry, int delta) { } } -inline void Session::sendWatch(std::string_view kind, wire::WatchBody const& body) { +inline void Session::sendWatch(std::string_view kind, wire::WatchBody const& body) const { auto completion = _options.watcher(std::string{kind}, glz::write_json(body).value_or(std::string{})); completion.then([](std::string const&) {}).onError([](std::exception_ptr const&) {}); } @@ -3054,11 +3060,15 @@ inline void Session::markRecordChange(effects::RecordChange const& change, detai continue; } // An entry still waiting for its first value may come to hold a key; keyless rows cannot be told apart. - bool const marked = !change.keys || !held->keyField || !entry->hasValue.peek() || - std::ranges::any_of(detail::keyedRowsOf(entry->confirmed, *held), [&](Value const* row) { - return keys.contains(detail::canonicalJson(*row->field(*held->keyField))); - }); - if (marked) { + if (!change.keys || !held->keyField || !entry->hasValue.peek()) { + marks.add(*entry); + continue; + } + std::string const& keyField = *held->keyField; + bool const holdsKey = std::ranges::any_of( + detail::keyedRowsOf(entry->confirmed, held->paths, keyField), + [&](Value const* row) { return keys.contains(detail::canonicalJson(*row->field(keyField))); }); + if (holdsKey) { marks.add(*entry); } } diff --git a/scripts/check_coverage_objects.sh b/scripts/check_coverage_objects.sh index 352386238..b8c89acb4 100755 --- a/scripts/check_coverage_objects.sh +++ b/scripts/check_coverage_objects.sh @@ -79,6 +79,9 @@ coverage_exclusion_reason() { morph_declare_source_canary) echo "the BRIDGE_DECLARE_MODEL/BRIDGE_DECLARE_ACTION link canary's positive probe (tests/CMakeLists.txt): what it asserts is that the program links at all, and the fixture model it registers exists nowhere else in the tree, so instrumenting it would only score probe-unique template instantiations against the library" ;; + morph_effects_unseen_links) + echo "the BRIDGE_REGISTER_EFFECTS link probe (tests/CMakeLists.txt): two translation units, only one of which sees an action's effects, must link into one program and read the same effects; the fixture record and action exist nowhere else in the tree, so instrumenting it would only score probe-unique template instantiations against the library" + ;; fuzz_wire_decode|fuzz_dispatch_execute) echo "libFuzzer harnesses (MORPH_BUILD_FUZZERS=ON only, which the coverage leg does not set); apply_fuzzer() builds them at -O1 under -fsanitize=fuzzer,address, a different instrumentation from apply_coverage()'s" ;; diff --git a/src/qt/qt_websocket_server.cpp b/src/qt/qt_websocket_server.cpp index 51dc831bd..46a166ba1 100644 --- a/src/qt/qt_websocket_server.cpp +++ b/src/qt/qt_websocket_server.cpp @@ -248,6 +248,7 @@ void QtWebSocketServer::onNewConnection() { cid, ::morph::backend::ConnectionWriter{ .write = [self, weakSocket, backlog](std::string envelope) { + // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks): Qt owns the queued callable. if (!queueEnvelope(self, weakSocket, backlog, std::move(envelope))) { throw std::runtime_error("no Qt application to write the connection from"); } @@ -265,9 +266,7 @@ bool QtWebSocketServer::queueEnvelope(QPointer const& server, // Before the queueing, so a backlog read on the server strand already // covers a frame still on its way to the Qt thread. backlog->fetch_add(counted); - // Qt owns the queued callable and deletes it once it ran or was dropped; the analyzer cannot see into - // QMetaObject's slot-object bookkeeping and reports it as leaked. - // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) + // Qt owns the queued callable and deletes it once it ran or was dropped. bool const queued = QMetaObject::invokeMethod( app, [server, socket, backlog, counted, envelope = std::move(envelope)]() { @@ -483,6 +482,8 @@ void QtWebSocketServer::onTextMessage(const QString& message) { _server.handle( message.toStdString(), [self, weakSocket, backlog = state.backlog](const std::string& reply) { + // queueEnvelope's queued callable is owned by Qt, which the analyzer cannot see; it reports the leak here. + // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) static_cast(queueEnvelope(self, weakSocket, backlog, reply)); }, state.cid); diff --git a/tests/test_push_cache.cpp b/tests/test_push_cache.cpp index c3354c1fc..8801208be 100644 --- a/tests/test_push_cache.cpp +++ b/tests/test_push_cache.cpp @@ -398,7 +398,7 @@ TEST_CASE("watcher_over_stamps_the_current_session_and_reports_refusals", "[push morph::testing::StepExecutor owner; std::vector sent; bool refuse = false; - morph::ui::Exchange const exchange = [&](morph::wire::Envelope request, + morph::ui::Exchange const exchange = [&](morph::wire::Envelope const& request, std::function const& done) { sent.push_back(request); morph::wire::Envelope reply; From 0b05f18565a82d9bf3e72a6642aa2299a764c7cb Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 04:19:06 +0200 Subject: [PATCH 59/68] examples(bank): --mcp-stdio serves the MCP adapter beside the WebSocket server The server installs a UiService over bank's screens, as the other rung servers do, and with --mcp-stdio runs an McpAdapter on stdin and stdout on a thread of its own: each permitted bank action is a tool, and a call reaches every watching GUI. Stdout then carries only JSON-RPC, so the status lines go to stderr; the server stops when stdin ends. The agent's identity is MORPH_MCP_PRINCIPAL, with MORPH_MCP_TOKEN as its token. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- examples/bank/CMakeLists.txt | 9 +++- examples/bank/README.md | 17 ++++++ examples/bank/src/server/main.cpp | 71 ++++++++++++++++++++++++-- examples/bank/tests/test_mcp_stdio.cpp | 59 +++++++++++++++++++++ 4 files changed, 151 insertions(+), 5 deletions(-) create mode 100644 examples/bank/tests/test_mcp_stdio.cpp diff --git a/examples/bank/CMakeLists.txt b/examples/bank/CMakeLists.txt index ad5772d9d..ac6dd814b 100644 --- a/examples/bank/CMakeLists.txt +++ b/examples/bank/CMakeLists.txt @@ -173,7 +173,7 @@ if(MORPH_BUILD_QT) # requirements name it. find_package(Qt6 REQUIRED COMPONENTS Core WebSockets) add_executable(ladder_bank_server src/server/main.cpp) - target_link_libraries(ladder_bank_server PRIVATE bank_lib morph::qt morph_qt_impl Qt6::Core) + target_link_libraries(ladder_bank_server PRIVATE bank_lib bank_screens morph::qt morph_qt_impl Qt6::Core) target_compile_features(ladder_bank_server PRIVATE cxx_std_23) apply_bigobj(ladder_bank_server) # No apply_warnings(), for the same reason bank_lib skips it: this target @@ -221,6 +221,13 @@ if(MORPH_BUILD_TESTS) ${CMAKE_CURRENT_SOURCE_DIR}/tests ${PROJECT_SOURCE_DIR}/tests) target_link_libraries(ladder_bank_tests PRIVATE bank_lib morph_test_main) + # The server's --mcp-stdio, driven over a shell pipe: a POSIX shell is what the case launches it with. + if(TARGET ladder_bank_server AND NOT WIN32) + target_sources(ladder_bank_tests PRIVATE tests/test_mcp_stdio.cpp) + target_compile_definitions(ladder_bank_tests PRIVATE + BANK_SERVER_PATH="$") + add_dependencies(ladder_bank_tests ladder_bank_server) + endif() target_compile_features(ladder_bank_tests PRIVATE cxx_std_23) apply_bigobj(ladder_bank_tests) # A ctest command, so this block is not optional: without it the diff --git a/examples/bank/README.md b/examples/bank/README.md index d1f1b4158..1afa97d65 100644 --- a/examples/bank/README.md +++ b/examples/bank/README.md @@ -214,6 +214,23 @@ an audit trail to `bank_actions.jsonl` in its working directory, the same `morph::journal::FileActionLog` the CLI installs — the models' read-only actions carry `Loggable::No`, so what lands there is the mutating half of the surface. +It also serves bank's screens through a `UiService`, so the generic client can open it with `--server`. + +#### An agent over MCP (`--mcp-stdio`) + +With `--mcp-stdio` the server also serves an agent, such as Claude Code, over MCP's stdio transport, beside its +WebSocket clients. Each bank action the agent's principal may run is a tool, with the action's schema, and +`evaluate` is a dry run. A tool call is an ordinary execute on the agent's own connection, so every open GUI that +shows what it changes updates. Stdout then carries only JSON-RPC, so the status lines go to stderr. The agent's +identity is `MORPH_MCP_PRINCIPAL`, which bank's demo authorizer vouches for (`MORPH_MCP_TOKEN` is carried as its +token, for an authorizer that verifies one). The server stops when its stdin ends. + +```sh +claude mcp add bank -e MORPH_MCP_PRINCIPAL=alice \ + -e BANK_DB="DRIVER=SQLite3;Database=$PWD/bank.db" -e BANK_PORT=0 \ + -- ./build/examples/bank/ladder_bank_server --mcp-stdio +``` + #### Scenarios `scripts/scenario/scenarios/bank/` holds 22 scenario files that drive this diff --git a/examples/bank/src/server/main.cpp b/examples/bank/src/server/main.cpp index f838b7735..98918a846 100644 --- a/examples/bank/src/server/main.cpp +++ b/examples/bank/src/server/main.cpp @@ -11,9 +11,21 @@ /// Usage: /// @code /// BANK_DB="DRIVER=SQLite3;Database=bank.db;Timeout=5000" BANK_PORT=0 \ -/// ladder_bank_server +/// ladder_bank_server [--mcp-stdio] /// @endcode /// +/// The server also serves bank's screens through a `morph::ui::UiService`, as the +/// other rung servers do, so the generic client can open it with `--server`. +/// +/// @par `--mcp-stdio` +/// Serves an agent over MCP's stdio transport beside the WebSocket clients: a +/// `morph::ui::McpAdapter` on a connection of its own, its tool calls running as +/// ordinary executes, so every GUI watching what they change is told. Stdout then +/// carries only JSON-RPC, so the status lines go to stderr. The agent's identity +/// is `MORPH_MCP_PRINCIPAL`, which this demo authorizer vouches for, and +/// `MORPH_MCP_TOKEN`, carried as its token for an authorizer that verifies one. +/// When stdin ends the server stops. +/// /// @par Why this `main()` owns the server rather than `bank::app::App` /// Every ladder rung puts its `RemoteServer` in an `App` and lets `main()` name /// only that class. Bank cannot follow that shape without changing what @@ -49,6 +61,7 @@ /// the client, which is also the path a real user takes. #include +#include #include #include #include @@ -64,11 +77,16 @@ #include #include #include +#include +#include #include +#include #include #include +#include #include "bank/db/database.hpp" +#include "bank_screens.hpp" // Every model bank_lib hosts — see this file's "Why every model header is // included" note. Sorted, so a missing one is visible by reading. @@ -178,11 +196,18 @@ constexpr std::size_t kMaxLiveModels = 256; int main(int argc, char** argv) { QCoreApplication qtApp{argc, argv}; + bool mcpStdio = false; for (int i = 1; i < argc; ++i) { + if (std::string_view{argv[i]} == "--mcp-stdio") { + mcpStdio = true; + continue; + } std::cerr << "bank-server: unknown argument '" << argv[i] << "' (usage: BANK_DB=... BANK_PORT=... " - << "ladder_bank_server)\n"; + << "ladder_bank_server [--mcp-stdio])\n"; return 2; } + // With --mcp-stdio, stdout is the agent's JSON-RPC channel and nothing else may be written to it. + std::ostream& status = mcpStdio ? std::cerr : std::cout; const char* connectionString = std::getenv("BANK_DB"); try { @@ -208,9 +233,16 @@ int main(int argc, char** argv) { int exitCode = 0; { + auto bundle = bank::screens::bundle(); + if (!bundle) { + std::cerr << "bank-server: the screens do not build: " << bundle.error().message() << '\n'; + ::morph::journal::setActionLog(nullptr); + return 1; + } ::morph::exec::ThreadPoolExecutor pool{4}; ::morph::backend::ServerConfig serverConfig; serverConfig.limits.maxLiveModels = kMaxLiveModels; + serverConfig.services.push_back(std::make_shared<::morph::ui::UiService>(*bundle)); auto server = std::make_shared<::morph::backend::RemoteServer>(pool, std::make_shared(), std::move(serverConfig)); @@ -223,7 +255,29 @@ int main(int argc, char** argv) { // The one line scripts/scenario/run_scenarios.py parses to learn the // port (its `PORT_LINE`); `std::endl` because that tool reads this // pipe line by line and a buffered announcement would hang it. - std::cout << "bank-server: listening on ws://127.0.0.1:" << wsServer.port() << std::endl; + status << "bank-server: listening on ws://127.0.0.1:" << wsServer.port() << std::endl; + + // The agent's own thread: it owns the adapter and reads stdin, and stops the server once stdin ends. + std::thread mcp; + if (mcpStdio) { + mcp = std::thread{[server] { + ::morph::session::Context session; + if (char const* const principal = std::getenv("MORPH_MCP_PRINCIPAL")) { + session.principal = principal; + } + if (char const* const token = std::getenv("MORPH_MCP_TOKEN")) { + session.token = token; + } + { + ::morph::exec::MainThreadExecutor owner; + ::morph::ui::McpAdapter adapter{ + server, ::morph::ui::McpOptions{.session = session, .serverName = "bank", .callbacks = &owner}}; + ::morph::ui::serveStdio(adapter, owner, std::cin, std::cout); + } + QMetaObject::invokeMethod(QCoreApplication::instance(), [] { QCoreApplication::quit(); }, + Qt::QueuedConnection); + }}; + } std::signal(SIGINT, onStopSignal); std::signal(SIGTERM, onStopSignal); @@ -243,12 +297,21 @@ int main(int argc, char** argv) { // does (bookmarks' metadata-fetch worker) needs a second step here and // bank does not. static_cast(wsServer.closeGracefully(std::chrono::seconds{2})); + if (mcp.joinable()) { + if (gStopRequested != 0) { + // Stopped by a signal while the agent's stdin is still open: its read cannot be interrupted, so the + // process ends here rather than wait for an input that may never come. + ::morph::journal::setActionLog(nullptr); + std::_Exit(exitCode); + } + mcp.join(); + } } // Matches the clear-on-shutdown discipline every rung App follows: nothing // should observe a live log belonging to a process that has stopped. ::morph::journal::setActionLog(nullptr); - std::cout << "bank-server: stopped\n"; + status << "bank-server: stopped\n"; return exitCode; } diff --git a/examples/bank/tests/test_mcp_stdio.cpp b/examples/bank/tests/test_mcp_stdio.cpp new file mode 100644 index 000000000..30396589e --- /dev/null +++ b/examples/bank/tests/test_mcp_stdio.cpp @@ -0,0 +1,59 @@ +// SPDX-License-Identifier: Apache-2.0 +// +// The bank server serves an agent over MCP's stdio transport: started with +// --mcp-stdio, it answers JSON-RPC on its stdout, one reply per request line, +// and stops when its stdin ends. + +#include +#include +#include +#include +#include + +#include "unique_test_database.hpp" + +namespace { + +// Runs @p command through the shell and returns its stdout, one entry per line. +std::vector outputLines(std::string const& command) { + std::vector lines; + FILE* const pipe = ::popen(command.c_str(), "r"); + REQUIRE(pipe != nullptr); + std::string line; + for (int c = std::fgetc(pipe); c != EOF; c = std::fgetc(pipe)) { + if (c == '\n') { + lines.push_back(line); + line.clear(); + } else { + line.push_back(static_cast(c)); + } + } + if (!line.empty()) { + lines.push_back(line); + } + CHECK(::pclose(pipe) == 0); + return lines; +} + +} // namespace + +TEST_CASE("the server with --mcp-stdio answers tools/list on a pipe", "[bank][mcp]") { + bank::testing::detail::PrivateDatabase const database; + auto const directory = std::filesystem::temp_directory_path() / "bank_mcp_stdio"; + std::filesystem::create_directories(directory); + std::string const requests = R"({"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}\n)" + R"({"jsonrpc":"2.0","method":"notifications/initialized"}\n)" + R"({"jsonrpc":"2.0","id":2,"method":"tools/list"}\n)"; + std::string const command = "cd '" + directory.string() + "' && printf '%b' '" + requests + "' | BANK_DB='" + + database.connection() + "' BANK_PORT=0 MORPH_MCP_PRINCIPAL=alice '" + + std::string{BANK_SERVER_PATH} + "' --mcp-stdio 2>/dev/null"; + auto const lines = outputLines(command); + REQUIRE(lines.size() == 2); + CHECK(lines.at(0).contains(R"("id":1)")); + CHECK(lines.at(0).contains(R"("serverInfo")")); + CHECK(lines.at(1).contains(R"("id":2)")); + CHECK(lines.at(1).contains(R"("name":"evaluate")")); + // The bank's own actions are tools too, with their schemas. + CHECK(lines.at(1).contains(R"("inputSchema")")); + CHECK(lines.at(1).find(R"("name":")") != lines.at(1).rfind(R"("name":")")); +} From a1b6c3d7f3f1d6da3853e673685f7e65775fe65d Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 04:20:48 +0200 Subject: [PATCH 60/68] examples(bank): format the server's MCP thread Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- examples/bank/src/server/main.cpp | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/examples/bank/src/server/main.cpp b/examples/bank/src/server/main.cpp index 98918a846..af7587757 100644 --- a/examples/bank/src/server/main.cpp +++ b/examples/bank/src/server/main.cpp @@ -271,11 +271,12 @@ int main(int argc, char** argv) { { ::morph::exec::MainThreadExecutor owner; ::morph::ui::McpAdapter adapter{ - server, ::morph::ui::McpOptions{.session = session, .serverName = "bank", .callbacks = &owner}}; + server, + ::morph::ui::McpOptions{.session = session, .serverName = "bank", .callbacks = &owner}}; ::morph::ui::serveStdio(adapter, owner, std::cin, std::cout); } - QMetaObject::invokeMethod(QCoreApplication::instance(), [] { QCoreApplication::quit(); }, - Qt::QueuedConnection); + QMetaObject::invokeMethod( + QCoreApplication::instance(), [] { QCoreApplication::quit(); }, Qt::QueuedConnection); }}; } From 407401248ab5148650ad9c54a1e0d88570ffd87a Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 04:20:48 +0200 Subject: [PATCH 61/68] ui: ui/2 removes invalidates, patch and refreshOn; ui/1 documents still load Under ui/2 a document refuses the three refresh keys, since actions declare what they change; optimistic keeps its {query, at?, key, with} shape as a definition of its own, and refreshEvery and refetch stay. A ui/1 document loads exactly as before. The authoring layer emits ui/2 and no longer offers invalidates, patch or refreshOn; the generic client speaks both vocabularies. A bundle needs the vocabularies its documents declare. The goldens are regenerated. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- docs/spec/ui/author.md | 26 ++- docs/spec/ui/document.md | 30 ++- docs/spec/ui/service.md | 7 +- .../specs/2026-10-07-ui-document-design.md | 23 ++- examples/common/app/app_source.cpp | 2 +- examples/common/app/app_source.hpp | 2 +- include/morph/ui/author.hpp | 117 +++--------- include/morph/ui/document.hpp | 127 +++++++++---- include/morph/ui/service.hpp | 33 +++- tests/CMakeLists.txt | 1 + tests/test_push_ui2.cpp | 178 ++++++++++++++++++ tests/test_ui_author.cpp | 58 +++--- tests/ui/corpus/documents.json | 16 +- tests/ui/golden/commands.json | 2 +- tests/ui/golden/declarations.json | 20 +- tests/ui/golden/palette.json | 2 +- tests/ui/golden/results.json | 2 +- 17 files changed, 427 insertions(+), 219 deletions(-) create mode 100644 tests/test_push_ui2.cpp diff --git a/docs/spec/ui/author.md b/docs/spec/ui/author.md index be0e9153a..7029af244 100644 --- a/docs/spec/ui/author.md +++ b/docs/spec/ui/author.md @@ -7,6 +7,11 @@ Design spec for the server-side builders of the UI document and their registrati `ui::loadDocument` and then the accessibility rules, so a screen that breaks a rule is a failing server test and never reaches a client. +The builders write `ui/2` documents. `ui/2` has no `invalidates`, `patch` or `refreshOn`, because an action +declares what it changes beside the action (its effects) and every query holding a changed record refreshes, so the +builders offer no option for them and setting one does not compile. A screen that must refetch for a reason no action +declares runs `refetch(query)`; data that changes with time rather than through actions polls with `.refreshEvery`. + Read this before changing what a builder writes. What the JSON means is [`document.md`](document.md) and [`expressions.md`](expressions.md); the builders add no meaning of their own. @@ -88,9 +93,9 @@ case, the rows of a `forEach` or `table` — takes a builder function that recei | `model(alias, {.instance = ...})` | `models.alias`: `type` is `ModelTraits::typeId()`; `instance` is `Instance::Private`, `Instance::Shared`, a key handle, `Borrow{"alias"}`, or left out | | `state(name)`, `state(name, constant)`, `state(name, handle)` | `state.name` with its type and `init`; `state` leaves the type to the init | | `let(name, handle)` | `let.name` | -| `query(name, model, body, {.when, .debounce, .refreshEvery, .staleTime, .refreshOn})` | `queries.name` with `action` | +| `query(name, model, body, {.when, .debounce, .refreshEvery, .staleTime})` | `queries.name` with `action` | | `evaluate(name, model, body, options)` | `queries.name` with `evaluate` (spec 5 §7); its handle's value is typed as `A`, since the reply is the body with its computed fields filled | -| `mutation(name, model, {.body, .concurrency, .serialKey, .invalidates, .patch, .optimistic, .onSuccess, .onError, .offlineQueue})` | `mutations.name` | +| `mutation(name, model, {.body, .concurrency, .serialKey, .optimistic, .onSuccess, .onError, .offlineQueue})` | `mutations.name`; `.optimistic` is an `Optimistic{.query, .at, .key, .with}`, read against the run's body bound as `body` | | `watch(on, commands, Source::User)`, `check({.name, .assertion, .message, .targets, .blocking})` | `watch`, `checks` | | `title`, `identity`, `dirty`, `onMount`, `onUnmount`, `onClose`, `onSave`, `onBackendChange`, `key(chord, commands)`, `showErrors`, `prefetch(query)` | The screen's own keys | | `component(name, {.qml \| .host + .version, .props, .events, .fallback})` | `components.name`, returning the `ComponentRef` `custom` names | @@ -102,8 +107,6 @@ case, the rows of a `forEach` or `table` — takes a builder function that recei - **Bodies** are the action's own members: `{{&ListResults::sampleId, sampleId}, ...}`, a list of member pointer and value pairs (`Body`). The member pointer must be the action's, and the value's type must be able to stand for the member's. -- **`refreshOn`** is written `sample.actions()`: the actions of the query's model type, - named through its model handle, which requires the model to run each of them. - **Names are given.** A query or mutation is named by its first argument; the builders invent no names. @@ -194,15 +197,16 @@ which holds neither the UI facts nor which model an action belongs to: | A model type id | `ModelTraits::typeId()` of each `model` | | `keyed` | `model::KeyedModel` | | `stateless` | `ModelFacts::stateless`, `false` unless the application specialises it | -| An action id and its model type | `ActionTraits::typeId()`, and the model of the handle the action runs through (a query, mutation, `refreshOn`, form or table) | +| An action id and its model type | `ActionTraits::typeId()`, and the model of the handle the action runs through (a query, mutation, form or table) | | `calculatedFields`, `secretFields`, `credential` | `ActionFacts`, empty unless the application specialises it | | `listFields` | The members glaze writes as a JSON array, when glaze writes the result type of the model's `execute(A)` as a record; absent for any other result | Facts gathered by a screen take precedence over the registry's, and the first fact recorded for an id is kept. Because the builders check model and action types at compile time, the catalog's -unknown-model, unknown-action, wrong-model and `refreshOn` rules cannot fire on builder output; -the keyed-instance, optimistic, patch `at` and offline rules can, and registration refuses them with -their paths. A `Patch`'s `at` names the list member of a record result, and is written only when set. +unknown-model, unknown-action and wrong-model rules cannot fire on builder output; +the keyed-instance, optimistic, optimistic `at` and offline rules can, and registration refuses them with +their paths. An `Optimistic`'s `at` names the list member of a record result, and is written only when set; one +without a query is refused at registration. ## Accessibility rules @@ -244,8 +248,10 @@ is reviewed like code. members retyped as handles, which `{.sampleId = id}` would need. A member pointer keeps both guarantees of spec 5 §11 — a renamed or foreign field and a value of the wrong type are compile errors — at the cost of naming the action in each pair. -- **`refreshOn` through the model handle.** No trait says which model an action belongs to, so a - free `actions()` could not check that they are the query's model type's; `M::execute` can. +- **No refresh options.** The refresh keys restate, per document, what an action's effects already say once for + every client, and a restatement can drift from the action. The builders emit only `ui/2` rather than offering both + vocabularies: a server that must keep `ui/1` documents for older clients keeps them as hand-written JSON, which + the UI service still serves. - **Facts from the types.** `keyed` is a property of the C++ type, and the model of an action is the model it runs through. The facts a type cannot show are traits an application specialises beside the action, which keeps them next to the code they describe. diff --git a/docs/spec/ui/document.md b/docs/spec/ui/document.md index 5c563be81..0f0e105e2 100644 --- a/docs/spec/ui/document.md +++ b/docs/spec/ui/document.md @@ -5,7 +5,7 @@ Design spec for the UI document's C++ model and its load-time validation (`include/morph/ui/value.hpp`). A server defines a screen as a JSON document (spec 5, `docs/superpowers/specs/2026-10-07-ui-document-design.md`); `ui::loadDocument` parses it into a `ui::Document` and refuses it, with the JSON Pointer and the reason, when it breaks a rule of -`ui/1`. The interpreter mounts only a document that loaded; the server runs the same function when +its vocabulary, `ui/1` or `ui/2`. The interpreter mounts only a document that loaded; the server runs the same function when a screen is registered, so a refused document is a server defect found in the server's own tests. Read this before changing what a document may say. The expression language inside it is @@ -57,7 +57,7 @@ declare the app-only customization kinds. | Member | JSON key | Notes | |---|---|---| -| `vocab` | `vocab` | Must be `ui/1` | +| `vocab` | `vocab` | `ui/1` (`kVocabulary`) or `ui/2` (`kVocabulary2`) | | `id` | `id` | Non-empty | | `title`, `identity`, `dirty` | same | Expressions; `dirty` is a condition | | `params` | `params` | Name to type, in declaration order | @@ -71,10 +71,18 @@ declare the app-only customization kinds. | `view` | `view` | The root `DocNode`; required on a screen | | `warnings` | — | What validation found that does not refuse the document ([Warnings](#warnings)) | -A key a client does not know is ignored, so additions within `ui/1` are compatible; a key whose +A key a client does not know is ignored, so additions within a vocabulary are compatible; a key whose name begins with `!` is must-understand, and an unknown one refuses the document wherever it appears (spec 5 §12). +**The two vocabularies.** `ui/2` is `ui/1` without the refresh keys: a mutation's `invalidates` and `patch` and a +query's `refreshOn`. An action declares what it changes in its effects, and every query holding a changed record +refreshes, on every client, so a document that restates it per screen can only drift from the action. Removing keys +is not an addition, so it is a new version, and a `ui/2` document that declares one is refused rather than having it +ignored: the author meant a refresh the client would silently not do. Everything else is the same in both, including +`optimistic`, `refreshEvery` and the `refetch` command, which is how a `ui/2` document refetches for a reason no +action declares. A `ui/1` document loads with its refresh keys, their rules and their warnings as before. + ## Declarations A scope's declarations are a `ScopeDecl`, each kind in declaration order. @@ -84,9 +92,10 @@ A scope's declarations are a `ScopeDecl`, each kind in declaration order. | Model alias | `"alias": {"type": "T", "instance": "private" \| "shared" \| e \| {"from": alias}}` | `ModelDecl`, `InstanceMode` `Default`, `Private`, `Shared`, `Keyed` (a key expression) or `Borrowed`; `"shared"` on a model the catalog does not mark stateless is refused | | State | `"name": {"type": "T"?, "init": e?}` | `StateDecl`; without a type, the type is the init's | | Derived | `"name": e` under `let` | `LetDecl` | -| Query | `{"model": alias, "action" \| "evaluate": id, "body", "when", "debounce", "refreshOn", "refreshEvery", "staleTime"}` | `QueryDecl`; exactly one of `action` and `evaluate` | -| Mutation | `{"model", "action", "body", "concurrency", "serialKey", "invalidates", "patch", "optimistic", "onSuccess", "onError", "offline"}` | `MutationDecl`, `Concurrency`, `PatchDecl` | -| Patch | `{"query": q, "at": member, "key": e, "with": e}` under `patch` and `optimistic`; `at` is optional and not empty | `PatchDecl` | +| Query | `{"model": alias, "action" \| "evaluate": id, "body", "when", "debounce", "refreshOn", "refreshEvery", "staleTime"}`; `refreshOn` is `ui/1` only | `QueryDecl`; exactly one of `action` and `evaluate` | +| Mutation | `{"model", "action", "body", "concurrency", "serialKey", "invalidates", "patch", "optimistic", "onSuccess", "onError", "offline"}`; `invalidates` and `patch` are `ui/1` only | `MutationDecl`, `Concurrency` | +| Patch | `{"query": q, "at": member, "key": e, "with": e}` under `patch`, read against the reply bound as `result`; `at` is optional and not empty | `PatchDecl` | +| Optimistic write | The same shape under `optimistic`, read against the run's body bound as `body` | `OptimisticDecl` | | Watch | `{"on": e, "source": "any" \| "user", "do": commands}` under `watch` | `WatchDecl`, `WatchSource` | | Check | `{"id", "assert": e, "message": e, "targets": [ids], "blocking": bool}` under `checks` | `CheckDecl` | @@ -194,8 +203,9 @@ A node list is an array of nodes or one node. `nodeKinds()` lists the kinds. `loadDocument` returns the first refusal; each rule has a case in `tests/ui/corpus/documents.json`. -- **Vocabulary.** `vocab` is `ui/1`; node kinds, commands, expression forms and functions are those - of `ui/1`; no unknown `!` key. +- **Vocabulary.** `vocab` is `ui/1` or `ui/2`; node kinds, commands, expression forms and functions are + those of `ui/1`, which `ui/2` keeps; no unknown `!` key. A `ui/2` mutation declares no `invalidates` or + `patch` and a `ui/2` query no `refreshOn`; the refusal points at the key and says that effects replace it. - **Shape.** Each key has the shape its kind gives it; required keys are present; names match `[A-Za-z_][A-Za-z0-9_]*`; closed choices (`showErrors`, `concurrency`, `source`, `offline`, `commitOn`, `onStale`, `navigate`'s `as`, `notify`'s `role`) take one of their values; chords are @@ -281,8 +291,8 @@ certainly miss, what the document declares. | Kind | When | |---|---| -| `Redundant` | An `invalidates` entry, a `patch` or a `refreshOn` entry that the effects already cover (below). The path names the entry. | -| `Uncovered` | A mutation whose action declares nothing, not even `none`, that has no `invalidates` or `patch`, no `refetch` anywhere in its `onSuccess`, and that no query of the document names in `refreshOn`. After it succeeds nothing on any screen updates. `effects::none` silences it for an action that changes nothing, such as one that opens or attaches. | +| `Redundant` | An `invalidates` entry, a `patch` or a `refreshOn` entry that the effects already cover (below), in a `ui/1` document; a `ui/2` document has none of them. The path names the entry. | +| `Uncovered` | A mutation whose action declares nothing, not even `none`, that has no `invalidates` or `patch`, no `refetch` anywhere in its `onSuccess`, and that no query of the document names in `refreshOn`. After it succeeds nothing on any screen updates. `effects::none` silences it for an action that changes nothing, such as one that opens or attaches. In a `ui/2` document the text names only `refetch`, the one refresh that vocabulary has. | | `WatchFeedback` | A `watch` declaration whose `source` is `any` (the default), whose `on` reads a query's value (`{"query": q}` or a `ref` to it, directly or through a `let`), and whose `do` runs (`run`, `upload`, `signIn`, at any depth) a mutation whose action declares an effect other than `none`. Another client's change refetches the query, the watch runs the mutation, and its change reaches the other client. `"source": "user"` silences it. | **What covers a query.** A query holds exactly the records of its action (`records`); an `evaluate` query holds none. diff --git a/docs/spec/ui/service.md b/docs/spec/ui/service.md index 28f95a4b4..b0e5e546b 100644 --- a/docs/spec/ui/service.md +++ b/docs/spec/ui/service.md @@ -44,8 +44,11 @@ the kinds are in the wire kind table ([`../core/wire.md`](../core/wire.md#the-ui Every request carries the envelope's `session`; the server stamps the verified principal onto it before the service sees it. `ui-hello` refuses a client that does not speak every vocabulary the bundle needs with -`err "missing vocabulary: [, …]"`; the bundle always needs `expr/1` and `ui/1`, and whatever the -registry adds (`requireVocabulary`). `hello` and `kProtocolVersion` are unchanged. +`err "missing vocabulary: [, …]"`; the bundle needs `expr/1`, the vocabulary each of its documents declares +(`ui/1` when it has none), and whatever the registry adds (`requireVocabulary`). A document is served as it was +written, so a client must read each one: a bundle of `ui/1` documents keeps serving a client that speaks only +`ui/1`, and one with a `ui/2` document, which the C++ builders write, needs a client that speaks `ui/2`. The generic +client speaks both. `hello` and `kProtocolVersion` are unchanged. **Negotiating events.** `ui-hello` is where a connection negotiates server-sent `event` envelopes, since the generic client already opens with it, on the same connection as its executes. `HelloRequest::events: true` offers them; diff --git a/docs/superpowers/specs/2026-10-07-ui-document-design.md b/docs/superpowers/specs/2026-10-07-ui-document-design.md index fb225903f..b6dc0c4a8 100644 --- a/docs/superpowers/specs/2026-10-07-ui-document-design.md +++ b/docs/superpowers/specs/2026-10-07-ui-document-design.md @@ -276,6 +276,9 @@ A query is spec 1 §4b's `Query`, declared as data: trigger each other. A change made by another client is not pushed; `refreshEvery` is the polling answer, and it refetches on a timer and **skips a tick while a request is in flight**. - `staleTime` (milliseconds, default 0) is how long a cached value counts as fresh (below). +- `refreshOn` is `ui/1` only. In `ui/2` an action declares what it changes (its effects, spec 8 §10), and + every query holding a changed record refreshes, on every client; a `ui/2` query that declares `refreshOn` + is refused. `refreshEvery` is in both, for data that changes with time rather than through actions. ### The query cache @@ -324,8 +327,11 @@ fetcher that goes through the cache. - `patch` replaces one row of a query's cached list (the row whose `id` field equals `key`) with `with` — a record `with` is merged over a record row, its fields replacing the row's — without refetching. `key` and `with` are evaluated against the reply, bound as `result`. -- `optimistic` has `patch`'s shape — `{"query": "q", "key": e, "with": e}` — but is applied when the run - starts, evaluated against the run's body bound as `body`. A success confirms the written fields +- `invalidates` and `patch` are `ui/1` only: in `ui/2` the action's effects refresh what it changed, and a + `ui/2` mutation that declares either is refused. A `ui/2` document that must refetch for a reason no action + declares runs `{"refetch": "q"}` in `onSuccess`. +- `optimistic` is its own declaration, in both vocabularies, with the same shape as `patch` — + `{"query": "q", "at"?: m, "key": e, "with": e}` — but is applied when the run starts, evaluated against the run's body bound as `body`. A success confirms the written fields (and `patch`, when declared, then applies the reply); a failure rolls the row back to its value without that run's write, and `onError` runs after the rollback. Only client-owned fields may be written optimistically: a field @@ -689,8 +695,7 @@ MORPH_REGISTER_SCREEN(screen); compile error on the server. A query or mutation body is the action's own members with expression handles in place of values, written as `{&Action::member, handle}` pairs: C++ cannot synthesise the action's struct with its members retyped, which `{.sampleId = sampleId}` would - need. A query's `refreshOn` names actions through its model handle, `sample.actions()`, - which requires the model to run each one. + need. The builders write `ui/2`, so they offer no `invalidates`, `patch` or `refreshOn` (§5). - Registration emits the JSON once and runs the load-time validation of §3; a screen that fails is a failing server test, never a client surprise. - Registration also runs the **accessibility rules**: every text input, date-time input, slider, select, @@ -701,7 +706,7 @@ MORPH_REGISTER_SCREEN(screen); rule. The client does not repeat these rules: they guard authoring, and a client must not refuse a document an older server registered. - The builders cover every construct of the document: the lifecycle keys (`identity`, `title`, - `dirty`, `onMount`, `onUnmount`, `onClose`, `onBackendChange`), `refreshOn` over action types, + `dirty`, `onMount`, `onUnmount`, `onClose`, `onBackendChange`), `form` with its `fields` overlay, `dataTable`, `custom` and host components, and the command forms. Registration answers the catalog of §3 from the types the builders saw: `keyed` from the model type, an action's model from the handle it runs through, and `stateless`, the calculated @@ -746,9 +751,11 @@ MORPH_REGISTER_SCREEN(screen); - **Versions.** Within a vocabulary version, changes are additive: a client ignores an optional key it does not know, unless the key's name begins with `!` (`"!requires"`), which marks it must-understand: a client that does not know it refuses the node and renders its fallback, or - refuses the document. A new node kind, function or command is a new vocabulary version - (`ui/2`, `expr/2`). A server may keep documents for more than one version and sends the newest the - client speaks. + refuses the document. A new node kind, function or command, or a removed key, is a new vocabulary + version. `ui/2` is the first: `ui/1` without `invalidates`, `patch` and `refreshOn` (§5). A client + reads every version it speaks, and a document is served as written, so a bundle needs the + vocabulary of each of its documents: a server keeps `ui/1` documents to keep serving clients that + speak only `ui/1`, and the C++ builders (§11) write `ui/2`. - **Development reload.** A client started in development mode re-sends `ui-hello` every second. When the manifest digest changes, it fetches the changed entries and remounts the mounted screens whose documents changed, keeping the screen scope's state where the new document still declares it under diff --git a/examples/common/app/app_source.cpp b/examples/common/app/app_source.cpp index 1f843f9da..cf698fb33 100644 --- a/examples/common/app/app_source.cpp +++ b/examples/common/app/app_source.cpp @@ -123,7 +123,7 @@ std::expected readClientArgs(std::vector& } std::vector clientVocabularies() { - return {std::string{ui::kVocabulary}, std::string{ui::kExprVocabulary}}; + return {std::string{ui::kVocabulary}, std::string{ui::kVocabulary2}, std::string{ui::kExprVocabulary}}; } ui::HelloRequest clientHello() { diff --git a/examples/common/app/app_source.hpp b/examples/common/app/app_source.hpp index f8c9e5715..5d0d3d80b 100644 --- a/examples/common/app/app_source.hpp +++ b/examples/common/app/app_source.hpp @@ -79,7 +79,7 @@ struct LocalApp { std::function()> serverAuthorizer; }; -/// @brief The vocabularies the generic client speaks. +/// @brief The vocabularies the generic client speaks: both document vocabularies, `ui/1` and `ui/2`, and `expr/1`. [[nodiscard]] std::vector clientVocabularies(); /// @brief The `ui-hello` the generic client opens with, and sends again on each poll: its vocabularies, and the diff --git a/include/morph/ui/author.hpp b/include/morph/ui/author.hpp index 20e090085..1366c7e8c 100644 --- a/include/morph/ui/author.hpp +++ b/include/morph/ui/author.hpp @@ -145,8 +145,8 @@ template // matters in process, where no event comes back. ::morph::effects::Changes derived; for (auto const& effect : *declared) { - if (!effect.record.empty() && (effect.kind == ::morph::effects::Kind::Updates || - effect.kind == ::morph::effects::Kind::Changes)) { + if (!effect.record.empty() && + (effect.kind == ::morph::effects::Kind::Updates || effect.kind == ::morph::effects::Kind::Changes)) { ::morph::effects::detail::addDerivedChanges(::morph::effects::detail::resolved(), effect.record, derived); } } @@ -1221,7 +1221,7 @@ struct Self {}; /// @brief The dialog or drawer a command list runs in. inline constexpr Self self{}; -/// @brief A query of a scope, without its action type: what `invalidates`, `refetch`, `prefetch` and `patch` name. +/// @brief A query of a scope, without its action type: what `refetch`, `prefetch` and `optimistic` name. class QueryRef { public: /// @brief A query by name. @@ -1420,31 +1420,6 @@ class Body { std::vector> _fields; }; -/// @brief Actions of one model type, for `refreshOn`. Made by `ModelHandle::actions`, which checks that the model -/// runs each one. -class ActionList { -public: - /// @brief An empty list. - ActionList() = default; - - /// @brief A list. - /// @param ids The action ids. - /// @param facts The facts of the actions and their model. - ActionList(std::vector ids, CatalogFacts facts) : _ids{std::move(ids)}, _facts{std::move(facts)} {} - - /// @brief The action ids. - /// @return The ids, in order. - [[nodiscard]] std::vector const& ids() const noexcept { return _ids; } - - /// @brief The facts of the actions and their model. - /// @return The facts. - [[nodiscard]] CatalogFacts const& facts() const noexcept { return _facts; } - -private: - std::vector _ids; - CatalogFacts _facts; -}; - /// @brief A model alias of a scope. /// @tparam Model The model type. template @@ -1458,18 +1433,6 @@ class ModelHandle { /// @return The alias. [[nodiscard]] std::string const& alias() const noexcept { return _alias; } - /// @brief Actions of this model type, for a query's `refreshOn`: an action the model does not run is a compile - /// error. - /// @tparam Actions The actions. - /// @return The list. - template - requires(Runs && ...) - [[nodiscard]] ActionList actions() const { - CatalogFacts facts; - (facts.template action(), ...); - return ActionList{{std::string{::morph::model::ActionTraits::typeId()}...}, std::move(facts)}; - } - private: std::string _alias; }; @@ -1550,8 +1513,6 @@ struct QueryOptions { std::chrono::milliseconds refreshEvery{0}; /// @brief How long a cached value counts as fresh. std::chrono::milliseconds staleTime{0}; - /// @brief Actions of the query's model type whose success refetches it. - ActionList refreshOn; }; /// @brief What a mutation run does while another is in flight. @@ -1562,16 +1523,15 @@ enum class Mode : std::uint8_t { Parallel, ///< Every run is sent; replies apply as they arrive. }; -/// @brief A `patch` or `optimistic` row replacement. -struct Patch { - /// @brief The query whose cached list it patches. +/// @brief An `optimistic` write: the row a mutation run predicts before its reply. +struct Optimistic { + /// @brief The query whose cached list it writes. std::optional query; /// @brief The member of the query's record value that holds the list; empty when the value is the list itself. std::string at; - /// @brief The row key, read against the reply bound as `result` (patch) or the body bound as `body` - /// (optimistic). + /// @brief The row key, read against the run's body bound as `body`. AnyHandle key; - /// @brief The new row. + /// @brief The predicted row, read against the run's body bound as `body`. AnyHandle with; }; @@ -1603,7 +1563,7 @@ class Commands { std::vector _items; }; -/// @brief One command of `ui/1` (spec 5 §8, spec 6). +/// @brief One command of `ui/2` (spec 5 §8, spec 6). class Command { public: /// @brief A command over its JSON, for a form no builder spells. @@ -1639,12 +1599,8 @@ struct MutationOptions { Mode concurrency = Mode::Exclusive; /// @brief The queue key of `serial` runs. AnyHandle serialKey; - /// @brief Queries refetched after a success, in one batch with the result. - std::vector invalidates; - /// @brief The row a reply replaces without a refetch. - std::optional patch; /// @brief The row a run predicts before its reply. - std::optional optimistic; + std::optional optimistic; /// @brief Commands run after a success. Commands onSuccess; /// @brief Commands run after a failure. @@ -1996,7 +1952,7 @@ class DeclScope { /// @param name The query's name. /// @param model The alias it goes through. /// @param body The body, which is the query's key. - /// @param options The gate, timing and `refreshOn`. + /// @param options The gate and timing. /// @return The query's handle. template requires Runs @@ -2013,7 +1969,7 @@ class DeclScope { /// @param name The query's name. /// @param model The alias it goes through. /// @param body The body, which is the query's key. - /// @param options The gate, timing and `refreshOn`. + /// @param options The gate and timing. /// @return The query's handle. Its value is the action body as the server reconciled it, computed fields /// filled, with `errors` and `ready` beside the fields. template @@ -2029,7 +1985,7 @@ class DeclScope { /// @tparam Model The model type of the alias; it must run the action. /// @param name The mutation's name. /// @param model The alias it goes through. - /// @param options Its body, concurrency, invalidation, patches and command lists. + /// @param options Its body, concurrency, optimistic write and command lists. /// @return The mutation's handle. template requires Runs @@ -2041,8 +1997,8 @@ class DeclScope { if (!options.body.empty()) { detail::put(decl, "body", options.body.json()); } - writeMutation(decl, options.concurrency, options.serialKey, options.invalidates, options.patch, - options.optimistic, options.onSuccess, options.onError, options.offlineQueue); + writeMutation(decl, options.concurrency, options.serialKey, options.optimistic, options.onSuccess, + options.onError, options.offlineQueue); detail::put(_mutations, name, std::move(decl)); _facts.action(); return MutationHandle{name}; @@ -2153,17 +2109,11 @@ class DeclScope { duration("debounce", options.debounce); duration("refreshEvery", options.refreshEvery); duration("staleTime", options.staleTime); - if (!options.refreshOn.ids().empty()) { - detail::put(decl, "refreshOn", - Json::array_t(options.refreshOn.ids().begin(), options.refreshOn.ids().end())); - _facts.merge(options.refreshOn.facts()); - } detail::put(_queries, name, std::move(decl)); _facts.action(); } static void writeMutation(Json& decl, Mode concurrency, AnyHandle const& serialKey, - std::vector const& invalidates, std::optional const& patch, - std::optional const& optimistic, Commands const& onSuccess, + std::optional const& optimistic, Commands const& onSuccess, Commands const& onError, bool offlineQueue) { static constexpr std::array kModes = {"exclusive", "serial", "latest", "parallel"}; if (concurrency != Mode::Exclusive) { @@ -2172,32 +2122,19 @@ class DeclScope { if (serialKey.present()) { detail::put(decl, "serialKey", serialKey.json()); } - if (!invalidates.empty()) { - Json::array_t names; - names.reserve(invalidates.size()); - for (QueryRef const& query : invalidates) { - names.emplace_back(query.name()); - } - detail::put(decl, "invalidates", std::move(names)); - } - auto row = [&decl](char const* key, std::optional const& replacement) { - if (!replacement) { - return; - } - if (!replacement->query) { - throw AuthoringError{std::string{"a `"} + key + "` names the query whose row it replaces"}; + if (optimistic) { + if (!optimistic->query) { + throw AuthoringError{"an `optimistic` write names the query whose row it writes"}; } Json part = Json::object_t{}; - detail::put(part, "query", replacement->query->name()); - if (!replacement->at.empty()) { - detail::put(part, "at", replacement->at); + detail::put(part, "query", optimistic->query->name()); + if (!optimistic->at.empty()) { + detail::put(part, "at", optimistic->at); } - detail::put(part, "key", replacement->key.json()); - detail::put(part, "with", replacement->with.json()); - detail::put(decl, key, std::move(part)); - }; - row("patch", patch); - row("optimistic", optimistic); + detail::put(part, "key", optimistic->key.json()); + detail::put(part, "with", optimistic->with.json()); + detail::put(decl, "optimistic", std::move(part)); + } if (!onSuccess.empty()) { detail::put(decl, "onSuccess", onSuccess.json()); } @@ -2462,7 +2399,7 @@ class Scope : public DeclScope { /// @return The document. [[nodiscard]] Json document(std::string const& documentId, Node const& view) const { Json out = Json::object_t{}; - detail::put(out, "vocab", std::string{kVocabulary}); + detail::put(out, "vocab", std::string{kVocabulary2}); detail::put(out, "id", documentId); for (auto const& [key, value] : std::get(_head.data)) { detail::put(out, key, value); diff --git a/include/morph/ui/document.hpp b/include/morph/ui/document.hpp index fa8c43b8f..38949e51f 100644 --- a/include/morph/ui/document.hpp +++ b/include/morph/ui/document.hpp @@ -33,9 +33,14 @@ namespace morph::ui { -/// @brief The vocabulary a document declares; the only one this client reads. +/// @brief The first document vocabulary: queries and mutations declare their own refreshes (`invalidates`, `patch`, +/// `refreshOn`). inline constexpr std::string_view kVocabulary = "ui/1"; +/// @brief The document vocabulary without the refresh keys: an action declares what it changes, so a `ui/2` document +/// has no `invalidates`, `patch` or `refreshOn`. This client reads both. +inline constexpr std::string_view kVocabulary2 = "ui/2"; + /// @brief What the catalog says about a model type. struct ModelInfo { /// @brief Whether a shared singleton is equivalent to a private instance. @@ -211,7 +216,7 @@ struct QueryDecl { std::int64_t refreshEvery = 0; /// @brief Milliseconds a cached value counts as fresh. std::int64_t staleTime = 0; - /// @brief Action ids of the same model type whose success refetches the query. + /// @brief Action ids of the same model type whose success refetches the query; `ui/1` only. std::vector refreshOn; }; @@ -223,15 +228,27 @@ enum class Concurrency : std::uint8_t { Parallel, ///< Every run is sent; replies apply as they arrive. }; -/// @brief A `patch` or `optimistic` row replacement. +/// @brief A `patch`: the row a mutation's reply replaces, without refetching; `ui/1` only. struct PatchDecl { /// @brief The query whose cached list it patches. std::string query; /// @brief The member of the query's record value that holds the list; empty when the value is the list itself. std::string at; - /// @brief The row key to replace. + /// @brief The row key to replace, read against the reply bound as `result`. + ExprPtr key; + /// @brief The new row, read against the reply bound as `result`. + ExprPtr with; +}; + +/// @brief An `optimistic` write: the row a mutation run predicts before its reply. +struct OptimisticDecl { + /// @brief The query whose cached list it writes. + std::string query; + /// @brief The member of the query's record value that holds the list; empty when the value is the list itself. + std::string at; + /// @brief The row key to write, read against the run's body bound as `body`. ExprPtr key; - /// @brief The new row. + /// @brief The predicted row, read against the run's body bound as `body`. ExprPtr with; }; @@ -311,12 +328,12 @@ struct MutationDecl { Concurrency concurrency = Concurrency::Exclusive; /// @brief The queue key of `serial` runs. ExprPtr serialKey; - /// @brief Queries refetched after a success. + /// @brief Queries refetched after a success; `ui/1` only. std::vector invalidates; - /// @brief The row a reply replaces, without refetching. + /// @brief The row a reply replaces, without refetching; `ui/1` only. std::optional patch; /// @brief The row a run predicts before its reply. - std::optional optimistic; + std::optional optimistic; /// @brief Commands run after a success. Commands onSuccess; /// @brief Commands run after a failure. @@ -521,7 +538,7 @@ struct LoadWarning { /// @brief A loaded and validated UI document: a screen, or the app document (id `app`, spec 6 §5). struct Document { - /// @brief The vocabulary, `ui/1`. + /// @brief The vocabulary, `ui/1` or `ui/2`. std::string vocab; /// @brief The document id, such as `lims.results`. std::string id; @@ -1002,9 +1019,10 @@ class DocumentParser { refuseMustUnderstand(*object, ""); Document doc; doc.vocab = text(*object, "vocab", "", true); - if (doc.vocab != kVocabulary) { - refuse("/vocab", "the vocabulary `" + doc.vocab + "` is not supported; this client reads ui/1"); + if (doc.vocab != kVocabulary && doc.vocab != kVocabulary2) { + refuse("/vocab", "the vocabulary `" + doc.vocab + "` is not supported; this client reads ui/1 and ui/2"); } + _refreshKeys = doc.vocab == kVocabulary; doc.id = text(*object, "id", "", true); if (doc.id.empty()) { refuse("/id", "a document has a non-empty id"); @@ -1271,7 +1289,20 @@ class DocumentParser { return decl; } - [[nodiscard]] static QueryDecl query(std::string const& name, Json const& json, std::string const& path) { + /// @brief Refuses @p key of @p fields in a vocabulary without the refresh keys. + /// @param fields The declaration's fields. + /// @param path The declaration's JSON Pointer. + /// @param key The refresh key: `invalidates`, `patch` or `refreshOn`. + void refuseRefreshKey(Json::object_t const& fields, std::string const& path, std::string_view key) const { + if (!_refreshKeys && member(fields, key) != nullptr) { + refuse(pointerAppend(path, key), + std::format("`{}` is not in ui/2: an action's effects say what it changes, and every query holding " + "a changed record refreshes; a refresh no action declares is a `refetch` command", + key)); + } + } + + [[nodiscard]] QueryDecl query(std::string const& name, Json const& json, std::string const& path) const { checkName(name, path); auto const& fields = objectOf(json, path, "a query is an object"); refuseMustUnderstand(fields, path); @@ -1291,28 +1322,37 @@ class DocumentParser { decl.debounce = integer(fields, "debounce", path); decl.refreshEvery = integer(fields, "refreshEvery", path); decl.staleTime = integer(fields, "staleTime", path); + refuseRefreshKey(fields, path, "refreshOn"); if (Json const* const refreshOn = member(fields, "refreshOn")) { decl.refreshOn = nameList(*refreshOn, pointerAppend(path, "refreshOn")); } return decl; } - [[nodiscard]] static PatchDecl patch(Json const& json, std::string const& path) { - auto const& fields = objectOf(json, path, R"(a patch is {"query": q, "at"?: m, "key": e, "with": e})"); + /// @brief Parses a row replacement, a `patch` or an `optimistic` write: `{"query": q, "at"?: m, "key": e, + /// "with": e}`. + /// @tparam Decl `PatchDecl` or `OptimisticDecl`. + /// @param json The replacement's JSON. + /// @param path Its JSON Pointer. + /// @param shape The refusal of a replacement without that shape. + /// @return The replacement. + template + [[nodiscard]] static Decl rowReplacement(Json const& json, std::string const& path, char const* shape) { + auto const& fields = objectOf(json, path, shape); refuseMustUnderstand(fields, path); Json const* const key = member(fields, "key"); Json const* const with = member(fields, "with"); if (key == nullptr || with == nullptr) { - refuse(path, R"(a patch is {"query": q, "at"?: m, "key": e, "with": e})"); + refuse(path, shape); } std::string listMember = text(fields, "at", path, false); if (member(fields, "at") != nullptr && listMember.empty()) { refuse(pointerAppend(path, "at"), "`at` names the member of the query's record that holds the list"); } - return PatchDecl{.query = text(fields, "query", path, true), - .at = std::move(listMember), - .key = expr(*key, pointerAppend(path, "key")), - .with = expr(*with, pointerAppend(path, "with"))}; + return Decl{.query = text(fields, "query", path, true), + .at = std::move(listMember), + .key = expr(*key, pointerAppend(path, "key")), + .with = expr(*with, pointerAppend(path, "with"))}; } [[nodiscard]] MutationDecl mutation(std::string const& name, Json const& json, std::string const& path) { @@ -1338,14 +1378,19 @@ class DocumentParser { if (decl.serialKey && decl.concurrency != Concurrency::Serial) { refuse(pointerAppend(path, "serialKey"), "`serialKey` applies to `serial` concurrency only"); } + refuseRefreshKey(fields, path, "invalidates"); + refuseRefreshKey(fields, path, "patch"); if (Json const* const invalidates = member(fields, "invalidates")) { decl.invalidates = nameList(*invalidates, pointerAppend(path, "invalidates")); } if (Json const* const value = member(fields, "patch")) { - decl.patch = patch(*value, pointerAppend(path, "patch")); + decl.patch = rowReplacement(*value, pointerAppend(path, "patch"), + R"(a patch is {"query": q, "at"?: m, "key": e, "with": e})"); } if (Json const* const value = member(fields, "optimistic")) { - decl.optimistic = patch(*value, pointerAppend(path, "optimistic")); + decl.optimistic = rowReplacement( + *value, pointerAppend(path, "optimistic"), + R"(an optimistic write is {"query": q, "at"?: m, "key": e, "with": e})"); } if (Json const* const commands = member(fields, "onSuccess")) { decl.onSuccess = commandList(*commands, pointerAppend(path, "onSuccess")); @@ -1927,6 +1972,9 @@ class DocumentParser { } } // NOLINTEND(misc-no-recursion) + + /// @brief Whether the document's vocabulary has the refresh keys, as `ui/1` does. + bool _refreshKeys = true; }; /// @brief What a name in a scope declares. @@ -2542,7 +2590,7 @@ class DocumentValidator { } } // NOLINTNEXTLINE(bugprone-easily-swappable-parameters): the key of the patch, then the name its reply is bound to. - auto patch = [&](PatchDecl const& decl, std::string_view key, std::string_view bound) { + auto patch = [&](auto const& decl, std::string_view key, std::string_view bound) { std::string const path = pointerAppend(mutation.path, key); auto const target = _queries.find(decl.query); if (target == _queries.end()) { @@ -2552,7 +2600,7 @@ class DocumentValidator { bodyContext.bindings.emplace_back(std::string{bound}, Type::any()); checkExpr(*decl.key, bodyContext); checkExpr(*decl.with, bodyContext); - checkPatchAt(decl, *target->second, path); + checkPatchAt(decl.query, decl.at, *target->second, path); return target->second; }; if (mutation.patch) { @@ -2574,7 +2622,9 @@ class DocumentValidator { // Only a catalog that knows the result is a record can say a patch misses its list; otherwise the interpreter // reports the miss when the patch runs. - void checkPatchAt(PatchDecl const& decl, QueryDecl const& query, std::string const& path) { + // NOLINTNEXTLINE(bugprone-easily-swappable-parameters): the query's name, then the member holding the list. + void checkPatchAt(std::string const& queryName, std::string const& at, QueryDecl const& query, + std::string const& path) { if (!_options->catalog.action) { return; } @@ -2582,18 +2632,18 @@ class DocumentValidator { if (!info || !info->listFields) { return; } - if (decl.at.empty()) { + if (at.empty()) { refuse(path, std::format("`{}` answers a record, so a patch of `{}` names the member holding its list " "with `at`", - query.action, decl.query)); + query.action, queryName)); } - if (std::ranges::find(*info->listFields, decl.at) == info->listFields->end()) { + if (std::ranges::find(*info->listFields, at) == info->listFields->end()) { refuse(pointerAppend(path, "at"), - std::format("the result of `{}` has no list member `{}`", query.action, decl.at)); + std::format("the result of `{}` has no list member `{}`", query.action, at)); } } - void checkOptimistic(PatchDecl const& decl, QueryDecl const& query) { + void checkOptimistic(OptimisticDecl const& decl, QueryDecl const& query) { if (!_options->catalog.action || decl.with->kind != ExprKind::Object) { return; } @@ -3140,11 +3190,15 @@ class DocumentValidator { return std::ranges::find(entry.first->refreshOn, mutation.action) != entry.first->refreshOn.end(); }); if (!action->effects && !refreshed) { - warn(LoadWarningKind::Uncovered, mutation.path, - std::format("nothing refreshes after `{}` succeeds: `{}` declares no effects, and no `invalidates`, " - "`patch`, `refetch` or `refreshOn` names it; declare what it changes, or " - "`effects::none` when it changes nothing", - mutation.name, mutation.action)); + // A `ui/2` document has no refresh keys, so the only refresh it can name is a `refetch`. + std::string_view const names = _doc->vocab == kVocabulary + ? "no `invalidates`, `patch`, `refetch` or `refreshOn` names it" + : "no `refetch` follows it"; + warn( + LoadWarningKind::Uncovered, mutation.path, + std::format("nothing refreshes after `{}` succeeds: `{}` declares no effects, and {}; declare what it " + "changes, or `effects::none` when it changes nothing", + mutation.name, mutation.action, names)); } } @@ -3379,8 +3433,9 @@ inline std::expected ScopeResolver::pref(std::string_view poi /// /// A document is refused when it does not parse, when a reference does not resolve, when a type does not check, /// when the `let` or `watch` graph has a cycle, when an expression exceeds 256 nodes, when a node kind, command or -/// function is not in `ui/1`, when a must-understand key is unknown, or when a rule of spec 5 or spec 6 fails; the -/// list is in `docs/spec/ui/document.md`. The rules that need catalog facts run when @p options supplies them. +/// function is not in the vocabulary, when a `ui/2` document declares `invalidates`, `patch` or `refreshOn`, when a +/// must-understand key is unknown, or when a rule of spec 5 or spec 6 fails; the list is in +/// `docs/spec/ui/document.md`. The rules that need catalog facts run when @p options supplies them. /// @param json The document's JSON. /// @param options The catalog facts. /// @return The document, with what validation found that does not refuse it in `Document::warnings`, or the first diff --git a/include/morph/ui/service.hpp b/include/morph/ui/service.hpp index ac45f52e1..67ad15d23 100644 --- a/include/morph/ui/service.hpp +++ b/include/morph/ui/service.hpp @@ -95,7 +95,8 @@ struct ServedScreen { struct ServedBundle { /// @brief The application. ApplicationInfo application; - /// @brief The vocabularies the bundle needs, sorted; always `expr/1` and `ui/1`. + /// @brief The vocabularies the bundle needs, sorted: `expr/1`, the vocabulary each document declares (`ui/1` when + /// there is none), and those added with `requireVocabulary`. std::vector vocabularies; /// @brief The manifest's text. std::string manifest; @@ -326,6 +327,7 @@ class UiRegistry { return refuse(entry.id, "the app document is added with addFile"); } logWarnings(*loaded); + _documentVocabularies.insert(loaded->vocab); detail::DocumentActions const named{*loaded}; Screen screen{.served = ServedScreen{.entry = std::move(entry), .hash = contentHash(text), @@ -400,8 +402,8 @@ class UiRegistry { return {}; } - /// @brief Adds a vocabulary the bundle needs beyond `ui/1` and `expr/1`, such as `forms/1` or `files/1`. A - /// client that does not speak it is refused at `ui-hello`. + /// @brief Adds a vocabulary the bundle needs beyond `expr/1` and its documents' own, such as `forms/1` or + /// `files/1`. A client that does not speak it is refused at `ui-hello`. /// @param vocabulary The vocabulary. void requireVocabulary(std::string vocabulary) { _vocabularies.insert(std::move(vocabulary)); } @@ -417,7 +419,8 @@ class UiRegistry { } auto bundle = std::make_shared(); bundle->application = _application; - bundle->vocabularies.assign(_vocabularies.begin(), _vocabularies.end()); + std::set> const needed = vocabularies(); + bundle->vocabularies.assign(needed.begin(), needed.end()); bundle->appActions = _appActions; bundle->appModelTypes = _appModelTypes; serveFiles(*bundle); @@ -426,6 +429,19 @@ class UiRegistry { } private: + /// @brief The vocabularies the bundle needs. A client is served the documents as they are written, so it must + /// speak each document's vocabulary: a bundle whose documents are all `ui/1` serves a client that speaks + /// only `ui/1`. + /// @return `expr/1`, the documents' vocabularies (`ui/1` when there is no document), and the required ones. + [[nodiscard]] std::set> vocabularies() const { + std::set> needed = _vocabularies; + if (_documentVocabularies.empty()) { + needed.emplace(kVocabulary); + } + needed.insert(_documentVocabularies.begin(), _documentVocabularies.end()); + return needed; + } + /// @brief Checks every theme index: each combination of its variants, densities and surfaces over `base` /// resolves from the bundle's token files (`checkTheme`). /// @return The refusal, naming the index and the combination, or nothing. @@ -506,6 +522,7 @@ class UiRegistry { return refuse(file.id, "the app document's id is `app`"); } logWarnings(*loaded); + _documentVocabularies.insert(loaded->vocab); detail::DocumentActions const named{*loaded}; _appActions = named.actions(); _appModelTypes = named.modelTypes(); @@ -570,8 +587,9 @@ class UiRegistry { .path = {}, .reason = "the bundle holds more than " + std::to_string(kMaxBundleFiles) + " files or entries"}; } - std::vector const vocabularies{_vocabularies.begin(), _vocabularies.end()}; - if (auto reason = detail::checkNames(vocabularies, kMaxListItems, "the vocabularies", "a vocabulary")) { + std::set> const needed = vocabularies(); + std::vector const names{needed.begin(), needed.end()}; + if (auto reason = detail::checkNames(names, kMaxListItems, "the vocabularies", "a vocabulary")) { return Refusal{.document = {}, .path = {}, .reason = std::move(*reason)}; } return std::nullopt; @@ -664,7 +682,8 @@ class UiRegistry { std::map> _tokenFiles; std::vector _appActions; std::vector _appModelTypes; - std::set> _vocabularies{std::string{kExprVocabulary}, std::string{kVocabulary}}; + std::set> _vocabularies{std::string{kExprVocabulary}}; + std::set> _documentVocabularies; }; /// @brief Whether an authoriser is asked about one of the UI service's reads: `ui`'s `hello`, `catalog` or `fetch`. diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index b2e83fa6e..a233847d7 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -30,6 +30,7 @@ add_executable(morph_tests test_push_cache.cpp test_push_agent.cpp test_push_documents.cpp + test_push_ui2.cpp push_records_late.cpp test_executor_extra.cpp test_strand.cpp diff --git a/tests/test_push_ui2.cpp b/tests/test_push_ui2.cpp new file mode 100644 index 000000000..dd9f4779c --- /dev/null +++ b/tests/test_push_ui2.cpp @@ -0,0 +1,178 @@ +// SPDX-License-Identifier: Apache-2.0 + +// The `ui/2` vocabulary (spec 8 §10): actions declare their effects, so a `ui/2` document has no `invalidates`, +// `patch` or `refreshOn`. It keeps `optimistic`, `refreshEvery` and the `refetch` command. A `ui/1` document still +// loads with its refresh keys, the authoring layer emits `ui/2`, and a bundle needs the vocabularies its documents +// declare. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "ui_service_support.hpp" + +namespace ui = morph::ui; +namespace author = morph::ui::author; + +namespace { + +/// One model, `T_Model`: `T_List` reads `T_Row`, `T_Edit` declares no effects. +ui::Catalog testCatalog() { + ui::Catalog catalog; + catalog.model = [](std::string_view type) -> std::optional { + if (type != "T_Model") { + return std::nullopt; + } + return ui::ModelInfo{}; + }; + catalog.action = [](std::string_view action) -> std::optional { + if (action != "T_List" && action != "T_Edit") { + return std::nullopt; + } + ui::ActionInfo info; + info.modelType = "T_Model"; + if (action == "T_List") { + info.records = {ui::ActionRecordInfo{.record = "T_Row", .keyField = "id", .paths = {""}}}; + } + return info; + }; + return catalog; +} + +/// A document of @p vocab with one model alias `m`, the given queries and mutations, and a text view. +std::string document(std::string_view vocab, std::string_view queries, std::string_view mutations) { + return R"({"vocab":")" + std::string{vocab} + R"(","id":"t","models":{"m":{"type":"T_Model"}},"queries":)" + + std::string{queries} + R"(,"mutations":)" + std::string{mutations} + + R"(,"view":{"kind":"text","text":"x"}})"; +} + +std::expected load(std::string const& text) { + return ui::loadDocumentText(text, ui::LoadOptions{.catalog = testCatalog()}); +} + +constexpr std::string_view kRows = R"({"rows":{"model":"m","action":"T_List"}})"; + +} // namespace + +TEST_CASE("ui2_refuses_invalidates", "[push][ui2]") { + auto const loaded = + load(document(ui::kVocabulary2, kRows, R"({"edit":{"model":"m","action":"T_Edit","invalidates":["rows"]}})")); + REQUIRE_FALSE(loaded.has_value()); + CHECK(loaded.error().path == "/mutations/edit/invalidates"); + CHECK(loaded.error().reason.contains("ui/2")); + CHECK(loaded.error().reason.contains("effects")); +} + +TEST_CASE("ui2_refuses_refresh_on", "[push][ui2]") { + auto const loaded = + load(document(ui::kVocabulary2, R"({"rows":{"model":"m","action":"T_List","refreshOn":["T_Edit"]}})", "{}")); + REQUIRE_FALSE(loaded.has_value()); + CHECK(loaded.error().path == "/queries/rows/refreshOn"); + CHECK(loaded.error().reason.contains("ui/2")); + CHECK(loaded.error().reason.contains("effects")); +} + +TEST_CASE("ui2_refuses_patch", "[push][ui2]") { + auto const loaded = load(document( + ui::kVocabulary2, kRows, + R"({"edit":{"model":"m","action":"T_Edit","patch":{"query":"rows","key":{"ref":"result.id"},"with":{"ref":"result"}}}})")); + REQUIRE_FALSE(loaded.has_value()); + CHECK(loaded.error().path == "/mutations/edit/patch"); + CHECK(loaded.error().reason.contains("ui/2")); + CHECK(loaded.error().reason.contains("effects")); +} + +TEST_CASE("ui2_keeps_optimistic", "[push][ui2]") { + auto const loaded = + load(document(ui::kVocabulary2, R"({"rows":{"model":"m","action":"T_List","refreshEvery":1000}})", + R"({"edit":{"model":"m","action":"T_Edit", + "optimistic":{"query":"rows","key":{"ref":"body.id"},"with":{"ref":"body"}}, + "onSuccess":[{"refetch":"rows"}]}})")); + if (!loaded) { + FAIL(loaded.error().message()); + } else { + CHECK(loaded->vocab == "ui/2"); + REQUIRE(loaded->scope.queries.size() == 1); + CHECK(loaded->scope.queries.front().refreshEvery == 1000); + REQUIRE(loaded->scope.mutations.size() == 1); + ui::MutationDecl const& edit = loaded->scope.mutations.front(); + REQUIRE(edit.optimistic.has_value()); + CHECK(edit.optimistic->query == "rows"); + CHECK(edit.optimistic->key != nullptr); + CHECK(edit.optimistic->with != nullptr); + REQUIRE(edit.onSuccess.size() == 1); + CHECK(edit.onSuccess.front().kind == ui::CommandKind::Refetch); + } + // An optimistic write still names a query of the document. + auto const unknown = load(document( + ui::kVocabulary2, kRows, + R"({"edit":{"model":"m","action":"T_Edit","optimistic":{"query":"none","key":1,"with":{"obj":{}}}}})")); + REQUIRE_FALSE(unknown.has_value()); + CHECK(unknown.error().path == "/mutations/edit/optimistic/query"); +} + +TEST_CASE("ui1_document_still_loads_with_refresh_on", "[push][ui2]") { + auto const loaded = + load(document(ui::kVocabulary, R"({"rows":{"model":"m","action":"T_List","refreshOn":["T_Edit"]}})", + R"({"edit":{"model":"m","action":"T_Edit","invalidates":["rows"], + "patch":{"query":"rows","key":{"ref":"result.id"},"with":{"ref":"result"}}}})")); + if (!loaded) { + FAIL(loaded.error().message()); + } else { + CHECK(loaded->vocab == "ui/1"); + REQUIRE(loaded->scope.queries.size() == 1); + CHECK(loaded->scope.queries.front().refreshOn == std::vector{"T_Edit"}); + REQUIRE(loaded->scope.mutations.size() == 1); + CHECK(loaded->scope.mutations.front().invalidates == std::vector{"rows"}); + CHECK(loaded->scope.mutations.front().patch.has_value()); + // `T_Edit` declares no effects, so the document's own refreshes are what refreshes after it. + CHECK(loaded->warnings.empty()); + } + auto const other = load(document("ui/3", kRows, "{}")); + REQUIRE_FALSE(other.has_value()); + CHECK(other.error().path == "/vocab"); +} + +TEST_CASE("author_emits_ui2", "[push][ui2]") { + CHECK(ui::kVocabulary == "ui/1"); + CHECK(ui::kVocabulary2 == "ui/2"); + auto const emitted = author::screen("author.ui2", [](author::Scope&) { return author::spacer(); }).write(); + if (!emitted) { + FAIL(emitted.error().message()); + } else { + CHECK(emitted->text.starts_with(R"({"vocab":"ui/2","id":"author.ui2")")); + auto const reloaded = ui::loadDocumentText(emitted->text); + REQUIRE(reloaded.has_value()); + CHECK(reloaded->vocab == "ui/2"); + } +} + +TEST_CASE("bundle_needs_the_vocabularies_its_documents_declare", "[push][ui2]") { + auto const screenText = [](std::string_view vocab, std::string_view id) { + return R"({"vocab":")" + std::string{vocab} + R"(","id":")" + std::string{id} + + R"(","view":{"kind":"text","text":"x"}})"; + }; + auto const built = [&](std::vector const& vocabs) { + ui::UiRegistry registry{ui::ApplicationInfo{.id = "app", .version = "1"}}; + for (std::size_t index = 0; index < vocabs.size(); ++index) { + std::string const id = "s" + std::to_string(index); + REQUIRE(registry.addScreen(uiservice::entry(id), screenText(vocabs.at(index), id)).has_value()); + } + auto bundle = registry.build(); + REQUIRE(bundle.has_value()); + return (*bundle)->vocabularies; + }; + // A bundle of `ui/1` documents is served to a client that speaks only `ui/1`. + CHECK(built({"ui/1"}) == std::vector{"expr/1", "ui/1"}); + CHECK(built({"ui/2"}) == std::vector{"expr/1", "ui/2"}); + CHECK(built({"ui/1", "ui/2"}) == std::vector{"expr/1", "ui/1", "ui/2"}); + // With no document, the bundle needs the first vocabulary. + CHECK(built({}) == std::vector{"expr/1", "ui/1"}); +} diff --git a/tests/test_ui_author.cpp b/tests/test_ui_author.cpp index 971144fb9..8868c173f 100644 --- a/tests/test_ui_author.cpp +++ b/tests/test_ui_author.cpp @@ -444,12 +444,9 @@ ui::Screen declarationsScreen() { auto anything = screen.state("anything", ui::AnyHandle{1}); auto total = screen.let("total", picked + 1); - auto results = screen.query("results", sample, {{&ListResults::sampleId, sampleId}}, - {.when = ui::engaged(sampleId), - .debounce = 250ms, - .refreshEvery = 30s, - .staleTime = 5s, - .refreshOn = sample.actions()}); + auto results = screen.query( + "results", sample, {{&ListResults::sampleId, sampleId}}, + {.when = ui::engaged(sampleId), .debounce = 250ms, .refreshEvery = 30s, .staleTime = 5s}); auto header = screen.query("header", working); auto preview = screen.evaluate("preview", working, {{&EvaluateResult::resultId, picked}}); // An evaluation's value is the action body, its computed fields filled, so the handle reads the action. @@ -461,11 +458,9 @@ ui::Screen declarationsScreen() { {.body = {{&CaptureConcentration::resultId, picked}}, .concurrency = ui::Mode::Serial, .serialKey = picked, - .invalidates = {header}, - .patch = ui::Patch{.query = results, .key = ui::ref("result.id"), .with = ui::ref("result")}, - .optimistic = ui::Patch{.query = results, - .key = ui::ref("body.resultId"), - .with = ui::obj({{"inputs", ui::ref("body.inputs")}})}, + .optimistic = ui::Optimistic{.query = results, + .key = ui::ref("body.resultId"), + .with = ui::obj({{"inputs", ui::ref("body.inputs")}})}, .onSuccess = {ui::set(unsaved, true)}, .onError = ui::notify(ui::t("lims.captureFailed", "Capture failed"), ui::Role::Error), .offlineQueue = true})); @@ -832,13 +827,13 @@ TEST_CASE("a builder error refuses the screen at its root", "[ui][author][regist REQUIRE(setComputed.has_value()); CHECK(setComputed->reason == "`set` names a state, an element or a declared value"); - auto const patchWithoutQuery = refusalOf(ui::screen("author.patch", [](ui::Scope& screen) { + auto const optimisticWithoutQuery = refusalOf(ui::screen("author.optimistic", [](ui::Scope& screen) { auto sample = screen.model("sample", {.instance = ui::Instance::Private}); - static_cast(screen.mutation("load", sample, {.patch = ui::Patch{}})); + static_cast(screen.mutation("load", sample, {.optimistic = ui::Optimistic{}})); return ui::spacer(); })); - REQUIRE(patchWithoutQuery.has_value()); - CHECK(patchWithoutQuery->reason == "a `patch` names the query whose row it replaces"); + REQUIRE(optimisticWithoutQuery.has_value()); + CHECK(optimisticWithoutQuery->reason == "an `optimistic` write names the query whose row it writes"); auto const thrown = refusalOf(ui::screen("author.thrown", [](ui::Scope&) -> ui::Node { throw 42; })); REQUIRE(thrown.has_value()); @@ -868,26 +863,27 @@ TEST_CASE("registration runs the load-time validation with the builders' catalog auto results = screen.query("results", sample); static_cast(screen.mutation( "capture", sample, - {.optimistic = ui::Patch{ + {.optimistic = ui::Optimistic{ .query = results, .key = ui::ref("body.resultId"), .with = ui::obj({{"concentration", 1}})}})); return ui::spacer(); })); REQUIRE(refusal.has_value()); CHECK(refusal->path == "/mutations/capture/optimistic/with/obj/concentration"); } - SECTION("a patch of a query whose action answers a record, without `at`") { + SECTION("an optimistic write to a query whose action answers a record, without `at`") { auto const refusal = refusalOf(ui::screen("author.patchAt", [](ui::Scope& screen) { auto const sample = screen.model("sample", {.instance = ui::Instance::Private}); auto samples = screen.query("samples", sample); static_cast(screen.mutation( "capture", sample, { - .patch = ui::Patch{.query = samples, .key = ui::ref("result.id"), .with = ui::ref("result")}, + .optimistic = + ui::Optimistic{.query = samples, .key = ui::ref("body.resultId"), .with = ui::ref("body")}, })); return ui::spacer(); })); REQUIRE(refusal.has_value()); - CHECK(refusal->path == "/mutations/capture/patch"); + CHECK(refusal->path == "/mutations/capture/optimistic"); } SECTION("an `at` that is not one of the result's lists") { auto const refusal = refusalOf(ui::screen("author.patchAt", [](ui::Scope& screen) { @@ -895,18 +891,18 @@ TEST_CASE("registration runs the load-time validation with the builders' catalog auto samples = screen.query("samples", sample); static_cast(screen.mutation("capture", sample, { - .patch = - ui::Patch{ + .optimistic = + ui::Optimistic{ .query = samples, .at = "total", - .key = ui::ref("result.id"), - .with = ui::ref("result"), + .key = ui::ref("body.resultId"), + .with = ui::ref("body"), }, })); return ui::spacer(); })); REQUIRE(refusal.has_value()); - CHECK(refusal->path == "/mutations/capture/patch/at"); + CHECK(refusal->path == "/mutations/capture/optimistic/at"); } SECTION("a credential-bearing mutation queued offline") { auto const refusal = refusalOf(ui::screen("author.offline", [](ui::Scope& screen) { @@ -935,7 +931,7 @@ TEST_CASE("the registry keeps one screen per id and answers the catalog", "[ui][ CHECK(duplicate.error().path == "/id"); REQUIRE(registry.add(commandsScreen()).has_value()); CHECK(registry.size() == 2); - CHECK(registry.find("lims.commands")->text.starts_with(R"({"vocab":"ui/1","id":"lims.commands")")); + CHECK(registry.find("lims.commands")->text.starts_with(R"({"vocab":"ui/2","id":"lims.commands")")); morph::ui::Catalog const catalog = registry.catalog(); auto const sample = catalog.model("Lims_SampleModel"); @@ -957,7 +953,7 @@ TEST_CASE("the registry keeps one screen per id and answers the catalog", "[ui][ CHECK_FALSE(catalog.action("Unknown").has_value()); } -TEST_CASE("a patch names the list inside a record result with `at`, and the catalog knows the lists", +TEST_CASE("an optimistic write names the list inside a record result with `at`, and the catalog knows the lists", "[ui][author][registration]") { ui::Screen const screen{ "author.patchAt", [](ui::Scope& scope) { @@ -965,12 +961,12 @@ TEST_CASE("a patch names the list inside a record result with `at`, and the cata auto samples = scope.query("samples", sample); static_cast(scope.mutation("capture", sample, { - .patch = - ui::Patch{ + .optimistic = + ui::Optimistic{ .query = samples, .at = "samples", - .key = ui::ref("result.id"), - .with = ui::ref("result"), + .key = ui::ref("body.resultId"), + .with = ui::ref("body"), }, })); return ui::spacer(); @@ -981,7 +977,7 @@ TEST_CASE("a patch names the list inside a record result with `at`, and the cata FAIL(added.error().message()); } CHECK(registry.find("author.patchAt") - ->text.contains(R"("patch":{"query":"samples","at":"samples","key":{"ref":"result.id"})")); + ->text.contains(R"("optimistic":{"query":"samples","at":"samples","key":{"ref":"body.resultId"})")); morph::ui::Catalog const catalog = registry.catalog(); CHECK(catalog.action("Lims_ListSamples")->listFields == std::vector{"samples"}); diff --git a/tests/ui/corpus/documents.json b/tests/ui/corpus/documents.json index fe20ada8d..c41c29705 100644 --- a/tests/ui/corpus/documents.json +++ b/tests/ui/corpus/documents.json @@ -251,12 +251,24 @@ "patch": { "query": "q", "at": "samples", "key": { "ref": "result.id" }, "with": { "ref": "result" } }, "optimistic": { "query": "q", "at": "notes", "key": 1, "with": { "obj": { "a": 1 } } } } }, "view": { "kind": "text", "text": "x" } } + }, + { + "name": "a ui/2 document keeps optimistic, refreshEvery and refetch", + "doc": { "vocab": "ui/2", "id": "t", "models": { "m": { "type": "Lims_SampleModel", "instance": { "lit": 1 } } }, + "queries": { "q": { "model": "m", "action": "Lims_ListSamples", "refreshEvery": 5000 } }, + "mutations": { "c": { "model": "m", "action": "Lims_Save", + "optimistic": { "query": "q", "at": "notes", "key": 1, "with": { "obj": { "a": 1 } } }, + "onSuccess": [{ "refetch": "q" }] } }, + "view": { "kind": "text", "text": "x" } } } ], "invalid": [ { "name": "not an object", "raw": [1], "at": "", "reason": "a document is a JSON object" }, { "name": "no vocab", "omit": ["vocab"], "at": "", "reason": "`vocab` is required" }, - { "name": "another vocabulary", "doc": { "vocab": "ui/2" }, "at": "/vocab", "reason": "is not supported" }, + { "name": "another vocabulary", "doc": { "vocab": "ui/3" }, "at": "/vocab", "reason": "is not supported" }, + { "name": "invalidates in ui/2", "doc": { "vocab": "ui/2", "models": { "x": { "type": "M" } }, "queries": { "q": { "model": "x", "action": "A" } }, "mutations": { "m": { "model": "x", "action": "B", "invalidates": ["q"] } } }, "at": "/mutations/m/invalidates", "reason": "`invalidates` is not in ui/2" }, + { "name": "a patch in ui/2", "doc": { "vocab": "ui/2", "models": { "x": { "type": "M" } }, "queries": { "q": { "model": "x", "action": "A" } }, "mutations": { "m": { "model": "x", "action": "B", "patch": { "query": "q", "key": 1, "with": 1 } } } }, "at": "/mutations/m/patch", "reason": "`patch` is not in ui/2" }, + { "name": "refreshOn in ui/2", "doc": { "vocab": "ui/2", "models": { "x": { "type": "M" } }, "queries": { "q": { "model": "x", "action": "A", "refreshOn": ["B"] } } }, "at": "/queries/q/refreshOn", "reason": "`refreshOn` is not in ui/2" }, { "name": "a non-string vocab", "doc": { "vocab": 1 }, "at": "/vocab", "reason": "`vocab` is a string" }, { "name": "no id", "omit": ["id"], "at": "", "reason": "`id` is required" }, { "name": "an empty id", "doc": { "id": "" }, "at": "/id", "reason": "non-empty id" }, @@ -301,7 +313,7 @@ { "name": "an unknown concurrency", "doc": { "mutations": { "m": { "model": "x", "action": "A", "concurrency": "fast" } } }, "at": "/mutations/m/concurrency", "reason": "not one of the values" }, { "name": "a serialKey without serial", "doc": { "mutations": { "m": { "model": "x", "action": "A", "serialKey": 1 } } }, "at": "/mutations/m/serialKey", "reason": "applies to `serial`" }, { "name": "a patch without a key", "doc": { "mutations": { "m": { "model": "x", "action": "A", "patch": { "query": "q", "with": 1 } } } }, "at": "/mutations/m/patch", "reason": "a patch is" }, - { "name": "a patch that is not an object", "doc": { "mutations": { "m": { "model": "x", "action": "A", "optimistic": 1 } } }, "at": "/mutations/m/optimistic", "reason": "a patch is" }, + { "name": "a patch that is not an object", "doc": { "mutations": { "m": { "model": "x", "action": "A", "optimistic": 1 } } }, "at": "/mutations/m/optimistic", "reason": "an optimistic write is" }, { "name": "an unknown offline mode", "doc": { "mutations": { "m": { "model": "x", "action": "A", "offline": "later" } } }, "at": "/mutations/m/offline", "reason": "not one of the values" }, { "name": "invalidates naming no query", "doc": { "models": { "x": { "type": "T" } }, "mutations": { "m": { "model": "x", "action": "A", "invalidates": ["nope"] } } }, "at": "/mutations/m/invalidates/0", "reason": "is not a query of this document" }, { "name": "a patch of no query", "doc": { "models": { "x": { "type": "T" } }, "mutations": { "m": { "model": "x", "action": "A", "patch": { "query": "nope", "key": 1, "with": 1 } } } }, "at": "/mutations/m/patch/query", "reason": "is not a query of this document" }, diff --git a/tests/ui/golden/commands.json b/tests/ui/golden/commands.json index 9cabeaa2d..29f1a4449 100644 --- a/tests/ui/golden/commands.json +++ b/tests/ui/golden/commands.json @@ -1,5 +1,5 @@ { - "vocab": "ui/1", + "vocab": "ui/2", "id": "lims.commands", "models": { "sample": { diff --git a/tests/ui/golden/declarations.json b/tests/ui/golden/declarations.json index 8a09517d5..4c1f7759b 100644 --- a/tests/ui/golden/declarations.json +++ b/tests/ui/golden/declarations.json @@ -1,5 +1,5 @@ { - "vocab": "ui/1", + "vocab": "ui/2", "id": "lims.declarations", "title": { "t": "lims.declarations", @@ -105,11 +105,7 @@ }, "debounce": 250, "refreshEvery": 30000, - "staleTime": 5000, - "refreshOn": [ - "Lims_CaptureConcentration", - "Lims_LoadSample" - ] + "staleTime": 5000 }, "header": { "model": "working", @@ -142,18 +138,6 @@ "serialKey": { "ref": "picked" }, - "invalidates": [ - "header" - ], - "patch": { - "query": "results", - "key": { - "ref": "result.id" - }, - "with": { - "ref": "result" - } - }, "optimistic": { "query": "results", "key": { diff --git a/tests/ui/golden/palette.json b/tests/ui/golden/palette.json index 9a084fba1..4a5d60081 100644 --- a/tests/ui/golden/palette.json +++ b/tests/ui/golden/palette.json @@ -1,5 +1,5 @@ { - "vocab": "ui/1", + "vocab": "ui/2", "id": "lims.palette", "models": { "sample": { diff --git a/tests/ui/golden/results.json b/tests/ui/golden/results.json index edbe648ad..f1f3246f6 100644 --- a/tests/ui/golden/results.json +++ b/tests/ui/golden/results.json @@ -1,5 +1,5 @@ { - "vocab": "ui/1", + "vocab": "ui/2", "id": "lims.results", "params": { "sampleId": "int" From 3e66a4cbe1ce7a363c5d27454e53f0ad010cab8b Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 07:08:23 +0200 Subject: [PATCH 62/68] ui, core: clang-tidy fixes; the MCP adapter builds its JSON without operator[] - BRIDGE_REGISTER_RECORD and BRIDGE_REGISTER_EFFECTS suppress the unnamed namespace in a header once, at the macro, as BRIDGE_REGISTER_ACTION does. - McpAdapter reads members through findMember and builds replies from object initializers. That also makes a parse error's id null and evaluate's required list ["action","body"], which brace-initialising glaze's generic had turned into one-element arrays. - The adapter's destructor does not throw; handle() takes its message by const reference; serveStdio names its streams input and output. - Tests cover the keyed-model path (a tool's instance is the shared instance under that key) and each JSON-RPC fault. - Bank's server reads its arguments through a span and its environment before the agent thread starts; smaller fixes in the explorer, the document checks and the test kit. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- examples/bank/src/server/main.cpp | 29 +- examples/bank/tests/test_mcp_stdio.cpp | 7 +- examples/common/testkit/bundle_warnings.hpp | 5 +- include/morph/core/effects.hpp | 4 + include/morph/core/records.hpp | 4 + include/morph/ui/agent.hpp | 307 +++++++++++--------- include/morph/ui/document.hpp | 10 +- include/morph/ui/explorer.hpp | 6 +- tests/test_push_agent.cpp | 103 ++++++- 9 files changed, 303 insertions(+), 172 deletions(-) diff --git a/examples/bank/src/server/main.cpp b/examples/bank/src/server/main.cpp index af7587757..8e6dddfb2 100644 --- a/examples/bank/src/server/main.cpp +++ b/examples/bank/src/server/main.cpp @@ -65,6 +65,7 @@ #include #include #include +#include #include #include #include @@ -81,6 +82,7 @@ #include #include #include +#include #include #include #include @@ -197,12 +199,12 @@ int main(int argc, char** argv) { QCoreApplication qtApp{argc, argv}; bool mcpStdio = false; - for (int i = 1; i < argc; ++i) { - if (std::string_view{argv[i]} == "--mcp-stdio") { + for (std::string_view const argument : std::span{argv, static_cast(argc)}.subspan(1)) { + if (argument == "--mcp-stdio") { mcpStdio = true; continue; } - std::cerr << "bank-server: unknown argument '" << argv[i] << "' (usage: BANK_DB=... BANK_PORT=... " + std::cerr << "bank-server: unknown argument '" << argument << "' (usage: BANK_DB=... BANK_PORT=... " << "ladder_bank_server [--mcp-stdio])\n"; return 2; } @@ -255,19 +257,22 @@ int main(int argc, char** argv) { // The one line scripts/scenario/run_scenarios.py parses to learn the // port (its `PORT_LINE`); `std::endl` because that tool reads this // pipe line by line and a buffered announcement would hang it. - status << "bank-server: listening on ws://127.0.0.1:" << wsServer.port() << std::endl; + status << "bank-server: listening on ws://127.0.0.1:" << wsServer.port() + << std::endl; // NOLINT(performance-avoid-endl) // The agent's own thread: it owns the adapter and reads stdin, and stops the server once stdin ends. std::thread mcp; if (mcpStdio) { - mcp = std::thread{[server] { - ::morph::session::Context session; - if (char const* const principal = std::getenv("MORPH_MCP_PRINCIPAL")) { - session.principal = principal; - } - if (char const* const token = std::getenv("MORPH_MCP_TOKEN")) { - session.token = token; - } + ::morph::session::Context session; + // NOLINTNEXTLINE(concurrency-mt-unsafe): nothing in this process sets the environment. + if (char const* const principal = std::getenv("MORPH_MCP_PRINCIPAL")) { + session.principal = principal; + } + // NOLINTNEXTLINE(concurrency-mt-unsafe): nothing in this process sets the environment. + if (char const* const token = std::getenv("MORPH_MCP_TOKEN")) { + session.token = token; + } + mcp = std::thread{[server, session] { { ::morph::exec::MainThreadExecutor owner; ::morph::ui::McpAdapter adapter{ diff --git a/examples/bank/tests/test_mcp_stdio.cpp b/examples/bank/tests/test_mcp_stdio.cpp index 30396589e..6b6f5e0f0 100644 --- a/examples/bank/tests/test_mcp_stdio.cpp +++ b/examples/bank/tests/test_mcp_stdio.cpp @@ -17,15 +17,16 @@ namespace { // Runs @p command through the shell and returns its stdout, one entry per line. std::vector outputLines(std::string const& command) { std::vector lines; + // NOLINTNEXTLINE(bugprone-command-processor,cert-env33-c): the command is this test's own, from its own paths. FILE* const pipe = ::popen(command.c_str(), "r"); REQUIRE(pipe != nullptr); std::string line; - for (int c = std::fgetc(pipe); c != EOF; c = std::fgetc(pipe)) { - if (c == '\n') { + for (int next = std::fgetc(pipe); next != EOF; next = std::fgetc(pipe)) { + if (next == '\n') { lines.push_back(line); line.clear(); } else { - line.push_back(static_cast(c)); + line.push_back(static_cast(next)); } } if (!line.empty()) { diff --git a/examples/common/testkit/bundle_warnings.hpp b/examples/common/testkit/bundle_warnings.hpp index a52255568..431b5e280 100644 --- a/examples/common/testkit/bundle_warnings.hpp +++ b/examples/common/testkit/bundle_warnings.hpp @@ -29,8 +29,9 @@ namespace morph::examples::testing { }; } if (bundle.actions) { - catalog.action = [actions = *bundle.actions](std::string_view id) -> std::optional { - auto const found = actions.find(id); + catalog.action = [actions = + *bundle.actions](std::string_view actionId) -> std::optional { + auto const found = actions.find(actionId); return found == actions.end() ? std::nullopt : std::optional{found->second}; }; } diff --git a/include/morph/core/effects.hpp b/include/morph/core/effects.hpp index 36308bc82..1b058064a 100644 --- a/include/morph/core/effects.hpp +++ b/include/morph/core/effects.hpp @@ -570,6 +570,9 @@ inline void findDerivationCycles(std::vector const& records, /// nothing under `MORPH_CLIENT_ONLY`. /// @param A The action type. /// @param ... The effects. +// Each translation unit that includes a registration registers its own copy, which is the point, so the unnamed +// namespace in a header and its static initialiser are suppressed here, once, rather than at every use. +// NOLINTBEGIN(bugprone-throwing-static-initialization,cert-err58-cpp,misc-anonymous-namespace-in-header,cert-dcl59-cpp) #ifdef MORPH_CLIENT_ONLY #define BRIDGE_REGISTER_EFFECTS(A, ...) #else @@ -579,3 +582,4 @@ inline void findDerivationCycles(std::vector const& records, morph::effects::detail::registerEffectsOnce(); \ } #endif +// NOLINTEND(bugprone-throwing-static-initialization,cert-err58-cpp,misc-anonymous-namespace-in-header,cert-dcl59-cpp) diff --git a/include/morph/core/records.hpp b/include/morph/core/records.hpp index f75f6423d..6f919cde8 100644 --- a/include/morph/core/records.hpp +++ b/include/morph/core/records.hpp @@ -230,6 +230,9 @@ inline bool registerRecordOnce(std::string_view name) noexcept { /// @param T The record type. /// @param NAME The record's wire name. /// @param ... The options. +// Each translation unit that includes a registration registers its own copy, which is the point, so the unnamed +// namespace in a header and its static initialiser are suppressed here, once, rather than at every use. +// NOLINTBEGIN(bugprone-throwing-static-initialization,cert-err58-cpp,misc-anonymous-namespace-in-header,cert-dcl59-cpp) #ifdef MORPH_CLIENT_ONLY #define BRIDGE_REGISTER_RECORD(T, NAME, ...) #else @@ -239,3 +242,4 @@ inline bool registerRecordOnce(std::string_view name) noexcept { morph::record::detail::registerRecordOnce(NAME); \ } #endif +// NOLINTEND(bugprone-throwing-static-initialization,cert-err58-cpp,misc-anonymous-namespace-in-header,cert-dcl59-cpp) diff --git a/include/morph/ui/agent.hpp b/include/morph/ui/agent.hpp index 4a141a153..959a60e5f 100644 --- a/include/morph/ui/agent.hpp +++ b/include/morph/ui/agent.hpp @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -23,6 +24,7 @@ #include "../core/executor.hpp" #include "../core/remote.hpp" #include "../core/wire.hpp" +#include "../forms/forms.hpp" #include "../session/session.hpp" #include "bundle.hpp" #include "value.hpp" @@ -89,7 +91,13 @@ class McpAdapter { } /// @brief Closes the adapter's connection, releasing its private instances. - ~McpAdapter() { _shared->server->closeConnection(_shared->cid); } + ~McpAdapter() { + // A destructor must not throw, so a close that fails is dropped. + try { + _shared->server->closeConnection(_shared->cid); + } catch (...) { // NOLINT(bugprone-empty-catch) + } + } McpAdapter(McpAdapter const&) = delete; McpAdapter& operator=(McpAdapter const&) = delete; @@ -100,11 +108,11 @@ class McpAdapter { /// @param message The message's JSON text. /// @return Completes with the reply's JSON text, or with nothing for a notification. It never fails: a fault is /// answered as a JSON-RPC error. - [[nodiscard]] async::Completion> handle(std::string message) { + [[nodiscard]] async::Completion> handle(std::string const& message) { auto [completion, promise] = async::Completion>::makeSettleable(_shared->options.callbacks); - auto settle = std::make_shared(std::move(promise)); - Shared::dispatch(_shared, std::move(message), + auto const settle = std::make_shared(std::move(promise)); + Shared::dispatch(_shared, message, [settle](std::optional reply) { settle->resolve(std::move(reply)); }); return std::move(completion); } @@ -128,6 +136,13 @@ class McpAdapter { std::map> privateIds; std::map, std::uint64_t> sharedIds; + // The schemas of every model a permitted action belongs to, gathered for one tool list. + struct Gather { + std::mutex mutex; + std::size_t waiting = 0; + std::map> schemas; + }; + // Sends one envelope on the adapter's connection, carrying its session. static void send(std::shared_ptr const& self, wire::Envelope env, EnvelopeReply done) { env.callId = self->nextCall.fetch_add(1) + 1; @@ -146,52 +161,61 @@ class McpAdapter { static std::string write(Json const& value) { return glz::write_json(value).value_or(std::string{"null"}); } - static std::string result(Json const& id, Json value) { - Json reply; - reply["jsonrpc"] = "2.0"; - reply["id"] = id; - reply["result"] = std::move(value); - return write(reply); + // A member of an object, or nullptr when the value is not an object or has no such member. + [[nodiscard]] static Json const* member(Json const& value, std::string_view key) { + return forms::detail::findMember(value, key); } - static std::string error(Json const& id, std::int64_t code, std::string const& message) { - Json reply; - reply["jsonrpc"] = "2.0"; - reply["id"] = id; - reply["error"]["code"] = code; - reply["error"]["message"] = message; - return write(reply); + [[nodiscard]] static std::string const* textMember(Json const& value, std::string_view key) { + Json const* const found = member(value, key); + return found == nullptr ? nullptr : std::get_if(&found->data); + } + + // The object member @p key of @p object, made an object when it is missing or is not one. + static Json::object_t& objectMember(Json::object_t& object, std::string const& key) { + Json& slot = object.try_emplace(key, Json::object_t{}).first->second; + if (!std::holds_alternative(slot.data)) { + slot = Json::object_t{}; + } + return std::get(slot.data); + } + + // A JSON-RPC response: @p requestId's answer, under the member @p outcome (`result` or `error`). + static std::string answer(Json const& requestId, std::string const& outcome, Json content) { + return write(Json::object_t{{"jsonrpc", "2.0"}, {"id", requestId}, {outcome, std::move(content)}}); + } + + static std::string result(Json const& requestId, Json value) { + return answer(requestId, "result", std::move(value)); + } + + static std::string error(Json const& requestId, std::int64_t code, std::string const& message) { + return answer(requestId, "error", Json::object_t{{"code", code}, {"message", message}}); } // A tool call's result: the text, flagged as an error or not. - static Json toolResult(std::string text, bool isError) { - Json content; - content["type"] = "text"; - content["text"] = std::move(text); - Json out; - out["content"] = Json::array_t{std::move(content)}; - out["isError"] = isError; - return out; + static Json toolResult(std::string const& text, bool isError) { + Json::array_t content; + content.emplace_back(Json::object_t{{"type", "text"}, {"text", text}}); + return Json::object_t{{"content", std::move(content)}, {"isError", isError}}; } - static void dispatch(std::shared_ptr const& self, std::string const& message, Reply done) { + static void dispatch(std::shared_ptr const& self, std::string const& message, Reply const& done) { auto parsed = parseJson(message); if (!parsed) { - done(error(Json{nullptr}, kMcpParseError, "not JSON: " + parsed.error())); + done(error(Json{}, kMcpParseError, "not JSON: " + parsed.error())); return; } - Json::object_t const* const request = std::get_if(&parsed->data); - std::string const* const method = request != nullptr && request->contains("method") - ? std::get_if(&request->find("method")->second.data) - : nullptr; - bool const notification = request != nullptr && !request->contains("id"); - Json const id = - request != nullptr && request->contains("id") ? request->find("id")->second : Json{nullptr}; + bool const isObject = std::holds_alternative(parsed->data); + std::string const* const method = textMember(*parsed, "method"); + Json const* const givenId = member(*parsed, "id"); + Json const requestId = givenId == nullptr ? Json{} : *givenId; + bool const notification = isObject && givenId == nullptr; if (method == nullptr) { - if (notification && request != nullptr) { + if (notification) { done(std::nullopt); } else { - done(error(id, kMcpInvalidRequest, "no method")); + done(error(requestId, kMcpInvalidRequest, "no method")); } return; } @@ -199,32 +223,32 @@ class McpAdapter { done(std::nullopt); return; } - Json const params = request->contains("params") ? request->find("params")->second : Json{}; + Json const* const givenParams = member(*parsed, "params"); + Json const params = givenParams == nullptr ? Json{} : *givenParams; if (*method == "initialize") { - done(result(id, initialize(*self, params))); + done(result(requestId, initialize(*self, params))); } else if (*method == "ping") { - done(result(id, Json::object_t{})); + done(result(requestId, Json::object_t{})); } else if (*method == "tools/list") { - listTools(self, [id, done](Json tools) { done(result(id, std::move(tools))); }); + listTools(self, [requestId, done](Json tools) { done(result(requestId, std::move(tools))); }); } else if (*method == "tools/call") { - callTool(self, params, [id, done](std::expected called) { - done(called ? result(id, std::move(*called)) : error(id, kMcpInvalidParams, called.error())); + callTool(self, params, [requestId, done](std::expected called) { + done(called ? result(requestId, std::move(*called)) + : error(requestId, kMcpInvalidParams, called.error())); }); } else { - done(error(id, kMcpMethodNotFound, "method not found: " + *method)); + done(error(requestId, kMcpMethodNotFound, "method not found: " + *method)); } } static Json initialize(Shared const& self, Json const& params) { - Json out; - std::string const* const asked = params.contains("protocolVersion") - ? std::get_if(¶ms["protocolVersion"].data) - : nullptr; - out["protocolVersion"] = asked != nullptr ? *asked : std::string{"2025-06-18"}; - out["capabilities"]["tools"]["listChanged"] = false; - out["serverInfo"]["name"] = self.options.serverName; - out["serverInfo"]["version"] = self.options.serverVersion; - return out; + std::string const* const asked = textMember(params, "protocolVersion"); + return Json::object_t{ + {"protocolVersion", asked != nullptr ? *asked : std::string{"2025-06-18"}}, + {"capabilities", Json::object_t{{"tools", Json::object_t{{"listChanged", false}}}}}, + {"serverInfo", + Json::object_t{{"name", self.options.serverName}, {"version", self.options.serverVersion}}}, + }; } // The catalog for the session, fetched once; an unreadable or refused one is empty. @@ -277,32 +301,65 @@ class McpAdapter { if (!std::holds_alternative(schema.data)) { schema = Json::object_t{}; } - schema["type"] = "object"; + auto& object = std::get(schema.data); + object.insert_or_assign("type", "object"); if (keyedModel) { - schema["properties"]["instance"]["type"] = "string"; - schema["properties"]["instance"]["description"] = "the key of the instance to run on"; - if (!schema.contains("required") || !std::holds_alternative(schema["required"].data)) { - schema["required"] = Json::array_t{}; + objectMember(object, "properties") + .insert_or_assign( + "instance", + Json::object_t{{"type", "string"}, {"description", "the key of the instance to run on"}}); + Json& required = object.try_emplace("required", Json::array_t{}).first->second; + if (!std::holds_alternative(required.data)) { + required = Json::array_t{}; } - std::get(schema["required"].data).emplace_back("instance"); + std::get(required.data).emplace_back("instance"); } - Json out; - out["name"] = actionId; - out["description"] = "Runs " + actionId + " on the server."; - out["inputSchema"] = std::move(schema); - return out; + return Json::object_t{{"name", actionId}, + {"description", "Runs " + actionId + " on the server."}, + {"inputSchema", std::move(schema)}}; } static Json evaluateTool() { - Json out; - out["name"] = "evaluate"; - out["description"] = - "A dry run: the action's body reconciled, its field errors, and whether it would run."; - out["inputSchema"]["type"] = "object"; - out["inputSchema"]["properties"]["action"]["type"] = "string"; - out["inputSchema"]["properties"]["body"]["type"] = "object"; - out["inputSchema"]["required"] = Json::array_t{Json{"action"}, Json{"body"}}; - return out; + Json::object_t properties{{"action", Json::object_t{{"type", "string"}}}, + {"body", Json::object_t{{"type", "object"}}}}; + Json::array_t required{std::string{"action"}, std::string{"body"}}; + return Json::object_t{ + {"name", "evaluate"}, + {"description", + "A dry run: the action's body reconciled, its field errors, and whether it would run."}, + {"inputSchema", + Json::object_t{ + {"type", "object"}, {"properties", std::move(properties)}, {"required", std::move(required)}}}, + }; + } + + // The tool list: one tool per permitted action of a known model, then `evaluate`. + static Json toolList(CatalogReply const& catalog, std::map> const& schemas) { + Json::array_t tools; + for (std::string const& action : catalog.permitted) { + std::string const type = modelOf(catalog, action); + if (type.empty()) { + continue; + } + auto const found = schemas.find(action); + tools.push_back(tool(action, found == schemas.end() ? Json{} : found->second, keyed(catalog, type))); + } + tools.push_back(evaluateTool()); + return Json::object_t{{"tools", std::move(tools)}}; + } + + // Folds one model's `schemas` reply into @p gather; true when it was the last one awaited. + [[nodiscard]] static bool gathered(Gather& gather, wire::Envelope const& reply) { + std::scoped_lock const lock{gather.mutex}; + auto parsed = reply.kind == "ok" ? parseJson(reply.body) : std::unexpected(std::string{}); + if (parsed) { + if (auto* const actions = std::get_if(&parsed->data)) { + for (auto& [action, schema] : *actions) { + gather.schemas.insert_or_assign(action, std::move(schema)); + } + } + } + return --gather.waiting == 0; } static void listTools(std::shared_ptr const& self, std::function done) { @@ -313,52 +370,16 @@ class McpAdapter { types.insert(std::move(type)); } } - // The schemas of every model a permitted action belongs to, then the tools. - struct Gather { - std::mutex mutex; - std::size_t waiting = 0; - std::map> schemas; - }; - auto const gather = std::make_shared(); - gather->waiting = types.size(); - auto const finish = [self, catalog, done, gather] { - Json::array_t tools; - for (std::string const& action : catalog.permitted) { - std::string const type = modelOf(catalog, action); - if (type.empty()) { - continue; - } - auto const found = gather->schemas.find(action); - tools.push_back(tool(action, found == gather->schemas.end() ? Json{} : found->second, - keyed(catalog, type))); - } - tools.push_back(evaluateTool()); - Json out; - out["tools"] = std::move(tools); - done(std::move(out)); - }; if (types.empty()) { - finish(); + done(toolList(catalog, {})); return; } + auto const gather = std::make_shared(); + gather->waiting = types.size(); for (std::string const& type : types) { - send(self, wire::makeSchemas(type), [gather, finish](wire::Envelope const& reply) { - bool last = false; - { - std::scoped_lock const lock{gather->mutex}; - if (reply.kind == "ok") { - if (auto parsed = parseJson(reply.body)) { - if (auto* const actions = std::get_if(&parsed->data)) { - for (auto& [action, schema] : *actions) { - gather->schemas.insert_or_assign(action, std::move(schema)); - } - } - } - } - last = --gather->waiting == 0; - } - if (last) { - finish(); + send(self, wire::makeSchemas(type), [catalog, done, gather](wire::Envelope const& reply) { + if (gathered(*gather, reply)) { + done(toolList(catalog, gather->schemas)); } }); } @@ -404,25 +425,25 @@ class McpAdapter { }); } - [[nodiscard]] static std::optional instanceKey(Json const& value) { - if (auto const* const text = std::get_if(&value.data)) { - return *text; - } - if (std::holds_alternative(value.data)) { + [[nodiscard]] static std::optional instanceKey(Json const* value) { + if (value == nullptr || std::holds_alternative(value->data)) { return std::nullopt; } - return write(value); + if (auto const* const text = std::get_if(&value->data)) { + return *text; + } + return write(*value); } static void callTool(std::shared_ptr const& self, Json const& params, std::function)> done) { - std::string const* const name = - params.contains("name") ? std::get_if(¶ms["name"].data) : nullptr; + std::string const* const name = textMember(params, "name"); if (name == nullptr) { done(std::unexpected(std::string{"tools/call names no tool"})); return; } - Json arguments = params.contains("arguments") ? params["arguments"] : Json::object_t{}; + Json const* const given = member(params, "arguments"); + Json arguments = given == nullptr ? Json{Json::object_t{}} : *given; if (!std::holds_alternative(arguments.data)) { done(std::unexpected(std::string{"the arguments are not an object"})); return; @@ -438,29 +459,27 @@ class McpAdapter { done(std::unexpected("no tool named " + tool)); return; } - auto& fields = std::get(arguments.data); std::optional key; if (keyed(catalog, type)) { - auto const found = fields.find("instance"); - key = found == fields.end() ? std::nullopt : instanceKey(found->second); + key = instanceKey(member(arguments, "instance")); if (!key) { done(std::unexpected(tool + " runs on a keyed model: give the instance key as `instance`")); return; } - fields.erase("instance"); + std::get(arguments.data).erase("instance"); } std::string body = write(arguments); withInstance( self, type, key, [self, type, tool, body = std::move(body), - done = std::move(done)](std::expected const& id) mutable { - if (!id) { - done(toolResult(id.error(), true)); + done = std::move(done)](std::expected const& modelId) mutable { + if (!modelId) { + done(toolResult(modelId.error(), true)); return; } wire::Envelope request; request.kind = "execute"; - request.modelId = *id; + request.modelId = *modelId; request.modelType = type; request.actionType = tool; request.body = std::move(body); @@ -473,18 +492,17 @@ class McpAdapter { static void evaluate(std::shared_ptr const& self, CatalogReply const& catalog, Json const& arguments, std::function)> done) { - std::string const* const action = - arguments.contains("action") ? std::get_if(&arguments["action"].data) : nullptr; - std::string const type = action == nullptr ? std::string{} : modelOf(catalog, *action); - if (type.empty()) { + std::string const* const action = textMember(arguments, "action"); + if (action == nullptr || modelOf(catalog, *action).empty()) { done(std::unexpected(std::string{"evaluate names no action the catalog knows"})); return; } + Json const* const body = member(arguments, "body"); wire::Envelope request; request.kind = std::string{kEvaluateKind}; - request.modelType = type; + request.modelType = modelOf(catalog, *action); request.actionType = *action; - request.body = arguments.contains("body") ? write(arguments["body"]) : std::string{"{}"}; + request.body = body == nullptr ? std::string{"{}"} : write(*body); send(self, std::move(request), [done = std::move(done)](wire::Envelope const& reply) { done(reply.kind == "ok" ? toolResult(reply.body, false) : toolResult(reply.message, true)); }); @@ -494,17 +512,18 @@ class McpAdapter { std::shared_ptr _shared; }; -/// @brief Serves an adapter over MCP's stdio transport: one JSON-RPC message per line of @p in, each reply on a -/// line of @p out, in order, until @p in ends. An empty line is skipped. +/// @brief Serves an adapter over MCP's stdio transport: one JSON-RPC message per line of @p input, each reply on a +/// line of @p output, in order, until @p input ends. An empty line is skipped. /// /// The calling thread is the adapter's owner: it pumps @p owner while it waits for each reply. /// @param adapter The adapter; its `McpOptions::callbacks` is @p owner. /// @param owner The adapter's owner, run on the calling thread. -/// @param in Where messages arrive. -/// @param out Where replies go; flushed after each. -inline void serveStdio(McpAdapter& adapter, exec::MainThreadExecutor& owner, std::istream& in, std::ostream& out) { +/// @param input Where messages arrive. +/// @param output Where replies go; flushed after each. +inline void serveStdio(McpAdapter& adapter, exec::MainThreadExecutor& owner, std::istream& input, + std::ostream& output) { std::string line; - while (std::getline(in, line)) { + while (std::getline(input, line)) { if (line.empty()) { continue; } @@ -514,7 +533,7 @@ inline void serveStdio(McpAdapter& adapter, exec::MainThreadExecutor& owner, std owner.runFor(std::chrono::milliseconds{10}); } if (*replied) { - out << **replied << '\n' << std::flush; + output << **replied << '\n' << std::flush; } } } diff --git a/include/morph/ui/document.hpp b/include/morph/ui/document.hpp index 38949e51f..8318095db 100644 --- a/include/morph/ui/document.hpp +++ b/include/morph/ui/document.hpp @@ -2590,7 +2590,7 @@ class DocumentValidator { } } // NOLINTNEXTLINE(bugprone-easily-swappable-parameters): the key of the patch, then the name its reply is bound to. - auto patch = [&](auto const& decl, std::string_view key, std::string_view bound) { + auto const patch = [&](auto const& decl, std::string_view key, std::string_view bound) { std::string const path = pointerAppend(mutation.path, key); auto const target = _queries.find(decl.query); if (target == _queries.end()) { @@ -2623,7 +2623,7 @@ class DocumentValidator { // Only a catalog that knows the result is a record can say a patch misses its list; otherwise the interpreter // reports the miss when the patch runs. // NOLINTNEXTLINE(bugprone-easily-swappable-parameters): the query's name, then the member holding the list. - void checkPatchAt(std::string const& queryName, std::string const& at, QueryDecl const& query, + void checkPatchAt(std::string const& queryName, std::string const& listMember, QueryDecl const& query, std::string const& path) { if (!_options->catalog.action) { return; @@ -2632,14 +2632,14 @@ class DocumentValidator { if (!info || !info->listFields) { return; } - if (at.empty()) { + if (listMember.empty()) { refuse(path, std::format("`{}` answers a record, so a patch of `{}` names the member holding its list " "with `at`", query.action, queryName)); } - if (std::ranges::find(*info->listFields, at) == info->listFields->end()) { + if (std::ranges::find(*info->listFields, listMember) == info->listFields->end()) { refuse(pointerAppend(path, "at"), - std::format("the result of `{}` has no list member `{}`", query.action, at)); + std::format("the result of `{}` has no list member `{}`", query.action, listMember)); } } diff --git a/include/morph/ui/explorer.hpp b/include/morph/ui/explorer.hpp index 41216fc38..fa4ef359f 100644 --- a/include/morph/ui/explorer.hpp +++ b/include/morph/ui/explorer.hpp @@ -800,10 +800,14 @@ class Explorer final : public ScreenHandle { std::vector records; for (ActionRecordInfo const& record : info.records) { std::vector paths; + paths.reserve(record.paths.size()); for (std::string const& path : record.paths) { paths.push_back(path.empty() ? std::string{"the result"} : path); } - std::string const key = record.keyField ? " by " + *record.keyField : std::string{" keyless"}; + std::string key{" keyless"}; + if (auto const& field = record.keyField) { + key = " by " + *field; + } records.push_back(record.record + key + " at " + detail::explorer::joined(paths, ", ")); } return detail::explorer::joined(records, "; "); diff --git a/tests/test_push_agent.cpp b/tests/test_push_agent.cpp index c20ea86d9..5f499224b 100644 --- a/tests/test_push_agent.cpp +++ b/tests/test_push_agent.cpp @@ -41,6 +41,9 @@ struct AddNote { std::string text; }; struct WipeNotes {}; +struct Tally { + std::string label; +}; struct Store { std::mutex mutex; @@ -68,6 +71,13 @@ struct NotesModel { } }; +// Keyed: each key is an instance of its own, which counts the tallies made on it. +struct ShelfModel { + using PrimaryKey = std::string; + int count = 0; + int execute(Tally const& /*action*/) { return ++count; } +}; + } // namespace agent_test // NOLINTEND(misc-use-internal-linkage) @@ -75,6 +85,8 @@ BRIDGE_REGISTER_MODEL(agent_test::NotesModel, "Ag_Notes") BRIDGE_REGISTER_ACTION(agent_test::NotesModel, agent_test::ListNotes, "Ag_List") BRIDGE_REGISTER_ACTION(agent_test::NotesModel, agent_test::AddNote, "Ag_Add") BRIDGE_REGISTER_ACTION(agent_test::NotesModel, agent_test::WipeNotes, "Ag_Wipe") +BRIDGE_REGISTER_MODEL(agent_test::ShelfModel, "Ag_Shelf") +BRIDGE_REGISTER_ACTION(agent_test::ShelfModel, agent_test::Tally, "Ag_Tally") BRIDGE_REGISTER_RECORD(agent_test::Note, "Ag_Note") BRIDGE_REGISTER_EFFECTS(agent_test::AddNote, morph::effects::changes) BRIDGE_REGISTER_EFFECTS(agent_test::WipeNotes, morph::effects::changes) @@ -92,6 +104,13 @@ constexpr std::string_view kNotes = R"({ "mutations": { "add": { "model": "notes", "action": "Ag_Add" }, "wipe": { "model": "notes", "action": "Ag_Wipe" } }, "view": { "kind": "text", "text": "Notes" } })"; +// A screen over one shelf, a keyed instance. +constexpr std::string_view kShelf = R"({ + "vocab": "ui/1", "id": "shelf", "params": { "shelfKey": "string" }, + "models": { "shelf": { "type": "Ag_Shelf", "instance": { "ref": "shelfKey" } } }, + "mutations": { "tally": { "model": "shelf", "action": "Ag_Tally" } }, + "view": { "kind": "text", "text": "Shelf" } })"; + // The token is the principal; `agent` may not wipe. struct Authorizer final : morph::session::IAuthorizer { // NOLINTNEXTLINE(bugprone-easily-swappable-parameters): IAuthorizer's signature. @@ -134,10 +153,14 @@ struct Rig { facts.action(); facts.action(); facts.action(); + facts.action(); ui::UiRegistry registry{ui::ApplicationInfo{.id = "notes", .version = "1"}, facts.catalog()}; ui::CatalogEntry entry; entry.id = "notes"; REQUIRE(registry.addScreen(entry, std::string{kNotes}).has_value()); + ui::CatalogEntry shelf; + shelf.id = "shelf"; + REQUIRE(registry.addScreen(shelf, std::string{kShelf}).has_value()); auto built = registry.build(); REQUIRE(built.has_value()); morph::backend::ServerConfig config; @@ -181,11 +204,9 @@ struct Rig { }; // The adapter's reply to @p message, waited for on the adapter's owner. -std::optional ask(Rig& rig, ui::McpAdapter& adapter, std::string message) { +std::optional ask(Rig& rig, ui::McpAdapter& adapter, std::string const& message) { std::optional> replied; - adapter.handle(std::move(message)).then([&replied](std::optional const& answer) { - replied = answer; - }); + adapter.handle(message).then([&replied](std::optional const& answer) { replied = answer; }); auto const deadline = std::chrono::steady_clock::now() + 10s; while (!replied && std::chrono::steady_clock::now() < deadline) { rig.callbacks.runFor(10ms); @@ -247,7 +268,7 @@ TEST_CASE("tools_list_is_permitted_plus_evaluate", "[push][agent]") { auto const reply = json(ask(rig, *adapter, R"({"jsonrpc":"2.0","id":2,"method":"tools/list"})")); std::vector names = toolNames(reply); std::ranges::sort(names); - CHECK(names == std::vector{"Ag_Add", "Ag_List", "evaluate"}); + CHECK(names == std::vector{"Ag_Add", "Ag_List", "Ag_Tally", "evaluate"}); for (auto const& tool : std::get(reply["result"]["tools"].data)) { if (std::get(tool["name"].data) == "Ag_Add") { CHECK(tool["inputSchema"]["properties"].contains("text")); @@ -346,3 +367,75 @@ TEST_CASE("stdio_round_trip", "[push][agent]") { CHECK(dump(json(replies.at(0))["id"]) == "1"); CHECK(dump(json(replies.at(1))["id"]) == "2"); } + +// Mutation: register a private instance for each key — another connection tallies a shelf of its own. +TEST_CASE("keyed_tool_runs_on_the_instance_it_names", "[push][agent]") { + Rig rig; + auto adapter = rig.adapter(); + auto const listed = json(ask(rig, *adapter, R"({"jsonrpc":"2.0","id":1,"method":"tools/list"})")); + bool found = false; + for (auto const& tool : std::get(listed["result"]["tools"].data)) { + if (std::get(tool["name"].data) == "Ag_Tally") { + found = true; + CHECK(tool["inputSchema"]["properties"].contains("instance")); + CHECK(dump(tool["inputSchema"]["required"]) == R"(["label","instance"])"); + } + } + REQUIRE(found); + + auto const tally = [&](std::string_view instance, ui::McpAdapter& through) { + auto const reply = json(ask(rig, through, + R"({"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"Ag_Tally",)" + R"("arguments":{"label":"x","instance":)" + + std::string{instance} + "}}}")); + REQUIRE_FALSE(std::get(reply["result"]["isError"].data)); + return std::get(reply["result"]["content"][0]["text"].data); + }; + CHECK(tally(R"("a")", *adapter) == "1"); + CHECK(tally(R"("a")", *adapter) == "2"); + CHECK(tally(R"("b")", *adapter) == "1"); + // The instance is the shared one under that key, so another connection's call reaches it too. + auto other = rig.adapter(); + CHECK(tally(R"("a")", *other) == "3"); + + auto const keyless = json(ask(rig, *adapter, + R"({"jsonrpc":"2.0","id":3,"method":"tools/call",)" + R"("params":{"name":"Ag_Tally","arguments":{"label":"x"}}})")); + CHECK(dump(keyless["error"]["code"]) == "-32602"); +} + +TEST_CASE("faults_answer_as_json_rpc_errors", "[push][agent]") { + Rig rig; + auto adapter = rig.adapter(); + auto const unparsed = json(ask(rig, *adapter, "{not json")); + CHECK(dump(unparsed["id"]) == "null"); + CHECK(dump(unparsed["error"]["code"]) == "-32700"); + auto const noMethod = json(ask(rig, *adapter, R"({"jsonrpc":"2.0","id":7})")); + CHECK(dump(noMethod["id"]) == "7"); + CHECK(dump(noMethod["error"]["code"]) == "-32600"); + auto const noTool = json(ask(rig, *adapter, R"({"jsonrpc":"2.0","id":8,"method":"tools/call","params":{}})")); + CHECK(dump(noTool["error"]["code"]) == "-32602"); + auto const unknownTool = json(ask(rig, *adapter, + R"({"jsonrpc":"2.0","id":9,"method":"tools/call",)" + R"("params":{"name":"Ag_Missing","arguments":{}}})")); + CHECK(dump(unknownTool["error"]["code"]) == "-32602"); + auto const badArguments = json(ask(rig, *adapter, + R"({"jsonrpc":"2.0","id":10,"method":"tools/call",)" + R"("params":{"name":"Ag_Add","arguments":[1]}})")); + CHECK(dump(badArguments["error"]["code"]) == "-32602"); +} + +TEST_CASE("evaluate_tool_requires_action_and_body", "[push][agent]") { + Rig rig; + auto adapter = rig.adapter(); + auto const reply = json(ask(rig, *adapter, R"({"jsonrpc":"2.0","id":2,"method":"tools/list"})")); + for (auto const& tool : std::get(reply["result"]["tools"].data)) { + if (std::get(tool["name"].data) == "evaluate") { + CHECK(dump(tool["inputSchema"]["required"]) == R"(["action","body"])"); + } + } + auto const unknown = json(ask(rig, *adapter, + R"({"jsonrpc":"2.0","id":6,"method":"tools/call","params":{"name":"evaluate",)" + R"("arguments":{"action":"Ag_Missing","body":{}}}})")); + CHECK(dump(unknown["error"]["code"]) == "-32602"); +} From 0972e68f51507f27b8965d0b74c85b87cd2a11b8 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 07:08:23 +0200 Subject: [PATCH 63/68] effects: refuse a derivedFrom source that is not a registered record Such a source never changes, so the record derived from it would never be refreshed, and nothing said so. Resolution now refuses it with the record's name, beside the cycle check. The specs list it among the refusals. Tests drive each branch of deriving changes and of resolution: arrays and wrappers before a path ends, merged keys and whole types, instances kept once, a record derived along two routes, updates of an unknown or keyless record, pathWithin, sameRegistration and a record held at two paths. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- docs/spec/core/registry.md | 4 +- .../specs/2026-10-16-server-push-design.md | 5 +- include/morph/core/effects.hpp | 9 + tests/test_push_effects.cpp | 191 ++++++++++++++++++ 4 files changed, 206 insertions(+), 3 deletions(-) diff --git a/docs/spec/core/registry.md b/docs/spec/core/registry.md index 9f1cc9ff2..00f2d68b3 100644 --- a/docs/spec/core/registry.md +++ b/docs/spec/core/registry.md @@ -1791,9 +1791,11 @@ for every action, which records its result holds. translation unit for nothing; - a keyed record with `derivedFrom`: a key-level change to a source does not say which derived rows moved, so a derived record is keyless; + - a `derivedFrom` source that is not a registered record: it never changes, + so the derived record would never be refreshed; - a `derivedFrom` cycle among registered records, a record derived from itself included, found in registration order so the message is the same on - every platform. A source that is not a registered record is ignored. + every platform. - **Failure.** A refusal throws `morph::model::EffectsRegistrationError`, a `std::logic_error` whose `what()` lists every problem, one per line, each naming the action or record. The error is kept and the same error is thrown diff --git a/docs/superpowers/specs/2026-10-16-server-push-design.md b/docs/superpowers/specs/2026-10-16-server-push-design.md index b56c6bcd3..835065802 100644 --- a/docs/superpowers/specs/2026-10-16-server-push-design.md +++ b/docs/superpowers/specs/2026-10-16-server-push-design.md @@ -149,8 +149,9 @@ BRIDGE_REGISTER_RECORD(bank::dto::SpendingReport, "Bank_SpendingReport", derives a change (§8). A derived type is keyless, because a key-level change to a source does not say which derived rows moved. - **`morph::record::privateKeys`** marks a type whose keys must not reach other principals (§12). -- `derivedFrom` cycles and conflicting registrations (one type under two names, one name on two types, or one type - with different options) are refused when the registration phase closes. An identical registration repeated, as a +- `derivedFrom` cycles, a `derivedFrom` source that is not a registered record, and conflicting registrations (one + type under two names, one name on two types, or one type with different options) are refused when the registration + phase closes. An identical registration repeated, as a record registered in a header gives once per translation unit, is one registration. ### Effects diff --git a/include/morph/core/effects.hpp b/include/morph/core/effects.hpp index 1b058064a..6e1f0dd19 100644 --- a/include/morph/core/effects.hpp +++ b/include/morph/core/effects.hpp @@ -386,6 +386,15 @@ inline void findDerivationCycles(std::vector const& records, "` is derived from other records but has a key; a derived record is keyless"); } } + // A source that is no record never changes, so the derived record would never be refreshed: refused, not ignored. + for (auto const& info : records) { + for (auto const& source : info.derivedFrom) { + if (!indexOf.contains(source)) { + problems.push_back("record `" + info.name + "` is derived from " + source.name() + + ", which is not a registered record"); + } + } + } findDerivationCycles(records, indexOf, problems); for (auto const& id : slotConflicts) { diff --git a/tests/test_push_effects.cpp b/tests/test_push_effects.cpp index 08dd49e1a..b5f94d726 100644 --- a/tests/test_push_effects.cpp +++ b/tests/test_push_effects.cpp @@ -10,6 +10,7 @@ #include #include #include +#include #include #include "push_records_late.hpp" @@ -512,3 +513,193 @@ TEST_CASE("derive_from_process_registry", "[push][records]") { REQUIRE(instance.instances.size() == 1); REQUIRE(instance.records.empty()); } + +// ── Deriving changes, branch by branch ─────────────────────────────────────── + +namespace { + +using morph::effects::ActionRecord; +using morph::effects::Changes; +using morph::effects::Effect; +using morph::effects::detail::Resolved; + +std::vector rawKeys(std::vector const& texts) { + std::vector keys; + keys.reserve(texts.size()); + for (auto const& text : texts) { + keys.emplace_back(text); + } + return keys; +} + +// One action, `A`, whose result holds `D_Row` (keyed by `id`) at @p path, and which updates it. +Resolved rowsAt(std::string path) { + Resolved resolved; + resolved.records = {record("D_Row", typeid(VRow))}; + resolved.effects["A"] = {Effect{.kind = Kind::Updates, .record = "D_Row"}}; + resolved.actionRecords["A"] = {ActionRecord{.record = "D_Row", .keyField = "id", .paths = {std::move(path)}}}; + return resolved; +} + +std::vector updatedKeys(Resolved const& resolved, std::string_view result) { + auto const changes = morph::effects::detail::deriveChangesFrom(resolved, "A", result, ranOnInstance()); + REQUIRE(changes.records.size() == 1); + REQUIRE(changes.records[0].keys.has_value()); + return keyTexts(changes.records[0]); +} + +} // namespace + +// Mutation: stop at the first array, or at an object lacking the segment — the nested and wrapped rows are missed. +TEST_CASE("derive_walks_arrays_and_wrappers_before_the_path_ends", "[push][records]") { + auto const resolved = rowsAt("rows"); + // An array before the path is consumed is entered without consuming a segment; a scalar in it holds nothing. + CHECK(updatedKeys(resolved, R"([{"rows":[{"id":1}]},5,{"rows":[{"id":2}]}])") == + std::vector{"1", "2"}); + // An object without the segment is searched through its values, as a map would be. + CHECK(updatedKeys(resolved, R"({"wrap":{"rows":[{"id":3}]},"other":"text"})") == std::vector{"3"}); + // A map whose values lack the key, or are not records at all, names no row. + CHECK(updatedKeys(resolved, R"({"rows":{"a":{"n":1},"b":5,"c":{"id":4}}})") == std::vector{"4"}); +} + +// Mutation: append keys without the duplicate check, or let keys narrow a whole-type change. +TEST_CASE("record_changes_merge_keys_and_whole_types", "[push][records]") { + Changes changes; + morph::effects::detail::addRecordChange(changes, "R", rawKeys({"1", "2"})); + morph::effects::detail::addRecordChange(changes, "R", rawKeys({"2", "3"})); + REQUIRE(changes.records.size() == 1); + CHECK(keyTexts(changes.records[0]) == std::vector{"1", "2", "3"}); + morph::effects::detail::addRecordChange(changes, "R", std::nullopt); + CHECK_FALSE(changes.records[0].keys.has_value()); + morph::effects::detail::addRecordChange(changes, "R", rawKeys({"4"})); + CHECK_FALSE(changes.records[0].keys.has_value()); + morph::effects::detail::addRecordChange(changes, "S", rawKeys({"1"})); + CHECK(changes.records.size() == 2); +} + +// Mutation: compare instances by model type alone — a second instance of the same model is lost. +TEST_CASE("instance_changes_are_kept_once_each", "[push][records]") { + using morph::effects::InstanceChange; + Changes changes; + InstanceChange const first{.modelType = "M", .instance = 7, .primary = "k"}; + morph::effects::detail::addInstanceChange(changes, first); + morph::effects::detail::addInstanceChange(changes, first); + CHECK(changes.instances.size() == 1); + morph::effects::detail::addInstanceChange(changes, {.modelType = "N", .instance = 7, .primary = "k"}); + morph::effects::detail::addInstanceChange(changes, {.modelType = "M", .instance = 8, .primary = "k"}); + morph::effects::detail::addInstanceChange(changes, {.modelType = "M", .instance = 7, .primary = "j"}); + CHECK(changes.instances.size() == 4); +} + +// Mutation: drop the visited check — a record derived along two routes is reported twice. +TEST_CASE("derive_reports_a_record_derived_along_two_routes_once", "[push][records]") { + Resolved resolved; + resolved.records = {record("D_Txn", typeid(DTxn)), + record("D_Report", typeid(DReport), std::nullopt, {typeid(DTxn)}), + record("D_Summary", typeid(DSummary), std::nullopt, {typeid(DTxn), typeid(DReport)})}; + resolved.effects["A"] = {Effect{.kind = Kind::Changes, .record = "D_Txn"}}; + auto const changes = morph::effects::detail::deriveChangesFrom(resolved, "A", "{}", ranOnInstance()); + REQUIRE(changes.records.size() == 3); + CHECK(changes.records[1].record == "D_Report"); + CHECK(changes.records[2].record == "D_Summary"); + + // A record the registries do not know has no derived records to add. + Resolved unknown; + unknown.effects["A"] = {Effect{.kind = Kind::Changes, .record = "D_Ghost"}}; + auto const ghost = morph::effects::detail::deriveChangesFrom(unknown, "A", "{}", ranOnInstance()); + REQUIRE(ghost.records.size() == 1); + CHECK(ghost.records[0].record == "D_Ghost"); +} + +// Mutation: read every record the action holds as the updated one — D_Header's rows are reported as D_Row's. +TEST_CASE("derive_updates_names_rows_only_of_a_known_keyed_record", "[push][records]") { + Resolved resolved; + resolved.records = {record("D_Row", typeid(VRow)), record("D_Header", typeid(VHeader), std::nullopt)}; + resolved.effects["A"] = { + Effect{.kind = Kind::Updates, .record = "D_Row"}, Effect{.kind = Kind::Updates, .record = "D_Header"}, + Effect{.kind = Kind::Updates, .record = "D_Ghost"}, Effect{.kind = static_cast(99), .record = {}}}; + resolved.actionRecords["A"] = {ActionRecord{.record = "D_Header", .keyField = std::nullopt, .paths = {"header"}}, + ActionRecord{.record = "D_Row", .keyField = "id", .paths = {"rows"}}}; + auto const changes = morph::effects::detail::deriveChangesFrom( + resolved, "A", R"({"header":{"id":9},"rows":[{"id":1}]})", ranOnInstance()); + REQUIRE(changes.records.size() == 3); + CHECK(changes.records[0].record == "D_Row"); + CHECK(keyTexts(changes.records[0]) == std::vector{"1"}); + // A keyless record, and one the registries do not know, are reported whole; an unknown kind adds nothing. + CHECK(changes.records[1].record == "D_Header"); + CHECK_FALSE(changes.records[1].keys.has_value()); + CHECK(changes.records[2].record == "D_Ghost"); + CHECK_FALSE(changes.records[2].keys.has_value()); + CHECK(changes.instances.empty()); +} + +// Mutation: accept a prefix that only shares leading characters ("rows" within "rowsTotal"). +TEST_CASE("path_within_is_a_whole_segment_prefix", "[push][records]") { + using morph::effects::detail::pathWithin; + CHECK(pathWithin("", "rows")); + CHECK(pathWithin("rows", "rows")); + CHECK(pathWithin("rows", "rows.items")); + CHECK_FALSE(pathWithin("rows", "rowsTotal")); + CHECK_FALSE(pathWithin("rows.items", "rows")); + CHECK_FALSE(pathWithin("rows", "header.rows")); +} + +TEST_CASE("same_registration_compares_every_option", "[push][records]") { + using morph::effects::detail::sameRegistration; + RecordInfo const base = record("D_Row", typeid(VRow)); + CHECK(sameRegistration(base, base)); + auto const differing = [&](auto change) { + RecordInfo other = base; + change(other); + return !sameRegistration(base, other); + }; + CHECK(differing([](RecordInfo& info) { info.name = "D_Other"; })); + CHECK(differing([](RecordInfo& info) { info.type = typeid(VOther); })); + CHECK(differing([](RecordInfo& info) { info.keyField = "code"; })); + CHECK(differing([](RecordInfo& info) { info.derivedFrom = {typeid(DTxn)}; })); + CHECK(differing([](RecordInfo& info) { info.privateKeys = true; })); + CHECK(differing([](RecordInfo& info) { info.keyIsString = true; })); +} + +TEST_CASE("effects_and_action_records_compare_by_value", "[push][records]") { + CHECK(Effect{.kind = Kind::Updates, .record = "R"} == Effect{.kind = Kind::Updates, .record = "R"}); + CHECK_FALSE(Effect{.kind = Kind::Updates, .record = "R"} == Effect{.kind = Kind::Changes, .record = "R"}); + CHECK_FALSE(Effect{.kind = Kind::Updates, .record = "R"} == Effect{.kind = Kind::Updates, .record = "S"}); + ActionRecord const held{.record = "R", .keyField = "id", .paths = {"rows"}}; + CHECK(held == held); + CHECK_FALSE(held == ActionRecord{.record = "S", .keyField = "id", .paths = {"rows"}}); + CHECK_FALSE(held == ActionRecord{.record = "R", .keyField = std::nullopt, .paths = {"rows"}}); + CHECK_FALSE(held == ActionRecord{.record = "R", .keyField = "id", .paths = {"items"}}); + CHECK(updatesOf(typeid(VRow)) == updatesOf(typeid(VRow))); + CHECK_FALSE(updatesOf(typeid(VRow)) == changesOf(typeid(VRow))); + CHECK_FALSE(updatesOf(typeid(VRow)) == updatesOf(typeid(VOther))); +} + +// Mutation: keep one path per record — the second place the result holds D_Row is never read. +TEST_CASE("a_record_held_at_two_paths_keeps_both", "[push][records]") { + ActionResultSlotMap slots; + slots.emplace("A_Two", result({{.type = typeid(VList), .path = ""}, + {.type = typeid(VRow), .path = "mine"}, + {.type = typeid(VRow), .path = "theirs"}})); + auto const resolved = morph::effects::detail::resolveForTesting({record("D_Row", typeid(VRow))}, slots, {}); + auto const held = resolved.recordsOf("A_Two"); + REQUIRE(held.size() == 1); + CHECK(held[0].paths == std::vector{"mine", "theirs"}); +} + +TEST_CASE("updates_on_a_record_the_result_does_not_hold_throws", "[push][records]") { + ActionResultSlotMap slots; + slots.emplace("A_Header", result({{.type = typeid(VHeader), .path = ""}})); + EffectsMap declared; + morph::effects::detail::addEffects(declared, "A_Header", {updatesOf(typeid(VRow))}); + auto const message = + refusal({record("D_Row", typeid(VRow)), record("D_Header", typeid(VHeader))}, slots, declared); + CHECK_THAT(message, Catch::Matchers::ContainsSubstring("its result holds no `D_Row`")); +} + +// Mutation: skip the check — a record derived from a type that is no record is accepted and never refreshed. +TEST_CASE("derived_from_an_unregistered_type_throws", "[push][records]") { + auto const message = refusal({record("D_Loose", typeid(VOther), std::nullopt, {typeid(VInner)})}, {}, {}); + CHECK_THAT(message, Catch::Matchers::ContainsSubstring("record `D_Loose` is derived from")); + CHECK_THAT(message, Catch::Matchers::ContainsSubstring("which is not a registered record")); +} From 40eb522ded01977fd7eef77704d59f2bf2c5585b Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 07:08:23 +0200 Subject: [PATCH 64/68] tests: the watch table on its own and the channel's edge cases; moved allowlist hints Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- scripts/branch_partial_allowlist.json | 12 ++-- tests/CMakeLists.txt | 1 + tests/test_push_channel.cpp | 36 +++++++++++ tests/test_push_watch_table.cpp | 92 +++++++++++++++++++++++++++ 4 files changed, 135 insertions(+), 6 deletions(-) create mode 100644 tests/test_push_watch_table.cpp diff --git a/scripts/branch_partial_allowlist.json b/scripts/branch_partial_allowlist.json index 3d5f9dcec..355035fef 100644 --- a/scripts/branch_partial_allowlist.json +++ b/scripts/branch_partial_allowlist.json @@ -97,7 +97,7 @@ }, { "file": "include/morph/core/backend.hpp", - "line": 1214, + "line": 1281, "source": "if (const auto* inst = _instances.find(modelId)) {", "reason": "Unreachable by construction given the `_changeAware`/`_instances` invariant (core audit finding BK2). `_changeAware` is an index over the instance directory: an id enters it in `createHolder` (this file, when the holder answers `isBackendChangeAware()`) in the same `_regMtx`-held critical section that files the instance, and leaves it in `deregisterModel` only when `InstanceDirectory::release` reports the instance actually destroyed. `notifyBackendChanged()` (this function) holds the same `_regMtx` while walking `_changeAware` and looking each id up at this line, so every id it walks is still live -- the null arm cannot occur without a code change that breaks that subset invariant." }, @@ -115,34 +115,34 @@ }, { "file": "include/morph/net/socket_backend.hpp", - "line": 1043, + "line": 1103, "source": "default:", "reason": "Unreachable except via the adjacent `Error` case it deliberately shares a body with (net audit, `socket_backend.hpp` extra finding #7). `detail::ExecuteReplyKind` is a closed 3-value enum (`Value`/`Timeout`/`Error`), all three handled explicitly above this label; `default:` exists only to satisfy this project's `-Wswitch-default`, per the source's own inline comment directly below this line. Reaching it via any value other than through the `Error` case falling through would require an out-of-range `static_cast` producing a value outside the enum's domain -- undefined behavior, not a legitimate test target." }, { "file": "include/morph/ui/document.hpp", - "line": 1530, + "line": 1643, "source": "default:", "context": "download(cmd, fields, primary, path);", "reason": "The default arm is unreachable: the switch names every enumerator of CommandKind, so no value reaches `default`. The project's strict warnings require both at once -- -Wswitch-enum wants every enumerator listed and -Wswitch-default wants a default label -- so the default shares the last enumerator's body and is never the label taken. It is DocumentParser::command's switch over the command kinds." }, { "file": "include/morph/ui/document.hpp", - "line": 1856, + "line": 1969, "source": "default:", "context": "case Shape::Keys:", "reason": "The default arm is unreachable: the switch names every enumerator of detail::Shape, so no value reaches `default`. The project's strict warnings require both at once -- -Wswitch-enum wants every enumerator listed and -Wswitch-default wants a default label -- so the default shares the last enumerator's body and is never the label taken. It is DocumentParser::entry's switch over a key's shape." }, { "file": "include/morph/ui/document.hpp", - "line": 2819, + "line": 2957, "source": "default:", "context": "if (cmd.value) {", "reason": "The default arm is unreachable: the switch names every enumerator of CommandKind, so no value reaches `default`. The project's strict warnings require both at once -- -Wswitch-enum wants every enumerator listed and -Wswitch-default wants a default label -- so the default shares the last enumerator's body and is never the label taken. It is DocumentValidator::checkCommand's switch." }, { "file": "include/morph/ui/document.hpp", - "line": 3013, + "line": 3363, "source": "default: {", "reason": "The default arm is unreachable: the switch names every enumerator of NameCategory, so no value reaches `default`. The project's strict warnings require both at once -- -Wswitch-enum wants every enumerator listed and -Wswitch-default wants a default label -- so the default shares the last enumerator's body and is never the label taken. It is ScopeResolver::ref's switch over what a name declares." }, diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index a233847d7..d67f120ec 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -24,6 +24,7 @@ add_executable(morph_tests test_example.cpp test_push_records.cpp test_push_effects.cpp + test_push_watch_table.cpp test_push_catalog.cpp test_push_channel.cpp test_push_watch.cpp diff --git a/tests/test_push_channel.cpp b/tests/test_push_channel.cpp index 9aa2ca022..2ab30f98e 100644 --- a/tests/test_push_channel.cpp +++ b/tests/test_push_channel.cpp @@ -268,6 +268,42 @@ TEST_CASE("post_after_close_is_dropped", "[push][channel]") { CHECK(wire.snapshot().empty()); } +// Mutation: keep a writer whose `write` is empty — the post after it is handed to an empty function. +TEST_CASE("an_empty_writer_removes_the_connections_writer", "[push][channel]") { + ::morph::exec::ThreadPoolExecutor pool{2}; + auto const server = std::make_shared<::morph::backend::RemoteServer>(pool, pcEnv().dispatcher, pcEnv().registry); + FakeWire wire; + auto const cid = server->openConnection(); + // Connection 0 is no connection: neither a writer nor a drain for it does anything. + server->setConnectionWriter(0, wire.writer()); + server->connectionDrained(0); + // A writer without a backlog read never holds an event back. + server->setConnectionWriter(cid, + {.write = [&wire](std::string frame) { wire.push(std::move(frame)); }, .backlog = {}}); + auto const first = encodedEvent("changes", R"({"n":1})"); + server->post(cid, first); + flush(*server); + REQUIRE(wire.snapshot() == std::vector{first}); + + server->setConnectionWriter(cid, {}); + server->post(cid, encodedEvent("changes", R"({"n":2})")); + flush(*server); + CHECK(wire.snapshot() == std::vector{first}); +} + +TEST_CASE("a_watch_whose_body_does_not_parse_is_refused", "[push][channel]") { + ::morph::exec::ThreadPoolExecutor pool{2}; + auto const server = std::make_shared<::morph::backend::RemoteServer>(pool, pcEnv().dispatcher, pcEnv().registry); + auto const cid = server->openConnection(); + ::morph::wire::Envelope watch; + watch.kind = std::string{::morph::wire::kWatchKind}; + watch.callId = 3; + watch.body = "not json"; + auto const reply = ::morph::wire::decode(server->handleInline(::morph::wire::encode(watch), cid)); + CHECK(reply.kind == "err"); + CHECK(reply.callId == 3); +} + TEST_CASE("post_above_envelope_cap_is_dropped_and_counted", "[push][channel]") { ::morph::observe::ScopedObserveOverride const restore; std::atomic oversize{0}; diff --git a/tests/test_push_watch_table.cpp b/tests/test_push_watch_table.cpp new file mode 100644 index 000000000..55de15301 --- /dev/null +++ b/tests/test_push_watch_table.cpp @@ -0,0 +1,92 @@ +// SPDX-License-Identifier: Apache-2.0 + +// The server's watch table on its own: what each connection watches, under which session, and what is kept when a +// session's watches are dropped. + +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using morph::backend::detail::Watch; +using morph::backend::detail::WatchTable; +using morph::wire::InstanceTopic; +using morph::wire::WatchBody; + +Watch watchAs(std::string const& principal, std::string const& token) { + Watch watch; + watch.principal = principal; + watch.context.principal = principal; + watch.context.token = token; + return watch; +} + +WatchBody records(std::vector names) { return WatchBody{.records = std::move(names), .instances = {}}; } + +WatchBody instance(std::string modelType, std::string primary) { + return WatchBody{.records = {}, + .instances = {InstanceTopic{.modelType = std::move(modelType), .primary = std::move(primary)}}}; +} + +} // namespace + +TEST_CASE("watch_of_answers_null_for_an_unknown_connection_or_topic", "[push][watch]") { + WatchTable table; + CHECK(table.watchOf(1, "R") == nullptr); + CHECK(table.watchOf(1, "M", "k") == nullptr); + REQUIRE(table.add(1, records({"R"}), watchAs("alice", "t")).refusedRecords.empty()); + REQUIRE(table.add(1, instance("M", "k"), watchAs("alice", "t")).refusedInstances.empty()); + CHECK(table.watchOf(1, "Other") == nullptr); + CHECK(table.watchOf(1, "M", "other") == nullptr); + CHECK(table.watchOf(2, "R") == nullptr); + CHECK(table.watchOf(2, "M", "k") == nullptr); + REQUIRE(table.watchOf(1, "R") != nullptr); + CHECK(table.watchOf(1, "R")->principal == "alice"); + REQUIRE(table.watchOf(1, "M", "k") != nullptr); + CHECK(table.watchOf(1, "M", "k")->context.token == "t"); +} + +// Mutation: drop every watch of the connection, whatever its session — the other session's topics go too. +TEST_CASE("drop_session_keeps_the_other_sessions_watches", "[push][watch]") { + WatchTable table; + table.dropSession(1, "alice", "t"); + REQUIRE(table.topicCount(1) == 0); + static_cast(table.add(1, records({"Mine"}), watchAs("alice", "t"))); + static_cast(table.add(1, instance("M", "mine"), watchAs("alice", "t"))); + static_cast(table.add(1, records({"Theirs"}), watchAs("alice", "other"))); + static_cast(table.add(1, instance("M", "theirs"), watchAs("bob", "t"))); + REQUIRE(table.topicCount(1) == 4); + + table.dropSession(1, "alice", "t"); + CHECK(table.topicCount(1) == 2); + CHECK(table.watchOf(1, "Mine") == nullptr); + CHECK(table.watchOf(1, "M", "mine") == nullptr); + CHECK(table.watchOf(1, "Theirs") != nullptr); + CHECK(table.watchOf(1, "M", "theirs") != nullptr); + + table.dropSession(1, "alice", "other"); + table.dropSession(1, "bob", "t"); + CHECK(table.topicCount(1) == 0); + CHECK(table.watchersOf("Theirs").empty()); +} + +// Mutation: let an instance past the cap — the connection watches more than kMaxWatchTopics topics. +TEST_CASE("an_instance_past_the_cap_is_refused", "[push][watch]") { + WatchTable table; + std::vector names; + names.reserve(morph::wire::kMaxWatchTopics); + for (std::size_t index = 0; index < morph::wire::kMaxWatchTopics; ++index) { + names.push_back("R" + std::to_string(index)); + } + REQUIRE(table.add(1, records(names), watchAs("alice", "t")).refusedRecords.empty()); + auto const refused = table.add(1, instance("M", "k"), watchAs("alice", "t")); + REQUIRE(refused.refusedInstances.size() == 1); + CHECK(refused.refusedInstances.front().primary == "k"); + CHECK(table.topicCount(1) == morph::wire::kMaxWatchTopics); + CHECK(table.watchersOf("M", "k").empty()); +} From 81fdc2fb44a08bb846f6c04e9596ff9d8cecf096 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 07:54:37 +0200 Subject: [PATCH 65/68] ui: one vocabulary; ui/1 refuses invalidates, patch and refreshOn The refresh keys restate per document what an action's effects say once for every client, so a document that declares one is now refused, pointing at the key, rather than a second vocabulary keeping them. ui/2 and the per-document vocabulary needs of a bundle are gone; the generic client speaks ui/1 and expr/1 again, and the authoring layer emits ui/1. The redundant-refresh warning goes with the keys, and the uncovered warning asks only whether a refetch follows. The interpreter's patch, invalidation and refreshOn paths are removed; optimistic keeps the row replacement. Tests of the removed behaviour are dropped; the `at` and no-list cases move to optimistic, and the form, collection and wizard submits are checked to apply their actions' effects. Specs 5 and 8 and the ui specs describe the one vocabulary; the goldens change only their vocab. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- docs/spec/ui/author.md | 10 +- docs/spec/ui/document.md | 58 ++--- docs/spec/ui/interpreter.md | 31 ++- docs/spec/ui/service.md | 7 +- .../specs/2026-10-07-ui-document-design.md | 73 +++--- .../specs/2026-10-16-server-push-design.md | 41 ++-- examples/common/app/app_source.cpp | 2 +- examples/common/app/app_source.hpp | 2 +- include/morph/ui/author.hpp | 4 +- include/morph/ui/document.hpp | 217 +++--------------- include/morph/ui/interpreter.hpp | 53 +---- include/morph/ui/service.hpp | 38 +-- tests/CMakeLists.txt | 2 +- tests/test_push_cache.cpp | 16 +- tests/test_push_documents.cpp | 87 +------ ...ush_ui2.cpp => test_push_refresh_keys.cpp} | 91 ++------ tests/test_ui_author.cpp | 2 +- tests/test_ui_document.cpp | 2 - tests/test_ui_interpreter_calculation.cpp | 3 +- tests/test_ui_interpreter_control.cpp | 111 ++------- tests/test_ui_interpreter_forms.cpp | 38 +-- tests/test_ui_service.cpp | 18 -- tests/ui/corpus/documents.json | 34 +-- tests/ui/golden/commands.json | 2 +- tests/ui/golden/declarations.json | 2 +- tests/ui/golden/palette.json | 2 +- tests/ui/golden/results.json | 2 +- tests/ui_interpreter_support.hpp | 12 +- 28 files changed, 248 insertions(+), 712 deletions(-) rename tests/{test_push_ui2.cpp => test_push_refresh_keys.cpp} (51%) diff --git a/docs/spec/ui/author.md b/docs/spec/ui/author.md index 7029af244..2a6fc45e8 100644 --- a/docs/spec/ui/author.md +++ b/docs/spec/ui/author.md @@ -7,9 +7,9 @@ Design spec for the server-side builders of the UI document and their registrati `ui::loadDocument` and then the accessibility rules, so a screen that breaks a rule is a failing server test and never reaches a client. -The builders write `ui/2` documents. `ui/2` has no `invalidates`, `patch` or `refreshOn`, because an action -declares what it changes beside the action (its effects) and every query holding a changed record refreshes, so the -builders offer no option for them and setting one does not compile. A screen that must refetch for a reason no action +Documents have no `invalidates`, `patch` or `refreshOn`, because an action declares what it changes beside the +action (its effects) and every query holding a changed record refreshes, so the builders offer no option for them +and setting one does not compile. A screen that must refetch for a reason no action declares runs `refetch(query)`; data that changes with time rather than through actions polls with `.refreshEvery`. Read this before changing what a builder writes. What the JSON means is [`document.md`](document.md) @@ -249,9 +249,7 @@ is reviewed like code. guarantees of spec 5 §11 — a renamed or foreign field and a value of the wrong type are compile errors — at the cost of naming the action in each pair. - **No refresh options.** The refresh keys restate, per document, what an action's effects already say once for - every client, and a restatement can drift from the action. The builders emit only `ui/2` rather than offering both - vocabularies: a server that must keep `ui/1` documents for older clients keeps them as hand-written JSON, which - the UI service still serves. + every client, and a restatement can drift from the action. - **Facts from the types.** `keyed` is a property of the C++ type, and the model of an action is the model it runs through. The facts a type cannot show are traits an application specialises beside the action, which keeps them next to the code they describe. diff --git a/docs/spec/ui/document.md b/docs/spec/ui/document.md index 0f0e105e2..0b91edac7 100644 --- a/docs/spec/ui/document.md +++ b/docs/spec/ui/document.md @@ -5,7 +5,7 @@ Design spec for the UI document's C++ model and its load-time validation (`include/morph/ui/value.hpp`). A server defines a screen as a JSON document (spec 5, `docs/superpowers/specs/2026-10-07-ui-document-design.md`); `ui::loadDocument` parses it into a `ui::Document` and refuses it, with the JSON Pointer and the reason, when it breaks a rule of -its vocabulary, `ui/1` or `ui/2`. The interpreter mounts only a document that loaded; the server runs the same function when +its vocabulary, `ui/1`. The interpreter mounts only a document that loaded; the server runs the same function when a screen is registered, so a refused document is a server defect found in the server's own tests. Read this before changing what a document may say. The expression language inside it is @@ -57,7 +57,7 @@ declare the app-only customization kinds. | Member | JSON key | Notes | |---|---|---| -| `vocab` | `vocab` | `ui/1` (`kVocabulary`) or `ui/2` (`kVocabulary2`) | +| `vocab` | `vocab` | `ui/1` (`kVocabulary`) | | `id` | `id` | Non-empty | | `title`, `identity`, `dirty` | same | Expressions; `dirty` is a condition | | `params` | `params` | Name to type, in declaration order | @@ -75,13 +75,11 @@ A key a client does not know is ignored, so additions within a vocabulary are co name begins with `!` is must-understand, and an unknown one refuses the document wherever it appears (spec 5 §12). -**The two vocabularies.** `ui/2` is `ui/1` without the refresh keys: a mutation's `invalidates` and `patch` and a -query's `refreshOn`. An action declares what it changes in its effects, and every query holding a changed record -refreshes, on every client, so a document that restates it per screen can only drift from the action. Removing keys -is not an addition, so it is a new version, and a `ui/2` document that declares one is refused rather than having it -ignored: the author meant a refresh the client would silently not do. Everything else is the same in both, including -`optimistic`, `refreshEvery` and the `refetch` command, which is how a `ui/2` document refetches for a reason no -action declares. A `ui/1` document loads with its refresh keys, their rules and their warnings as before. +**No refresh keys.** A document declares no `invalidates` or `patch` on a mutation and no `refreshOn` on a query. An +action declares what it changes in its effects, and every query holding a changed record refreshes, on every client, +so a document that restated it per screen could only drift from the action. A document that declares one is refused +rather than having it ignored: the author meant a refresh the client would silently not do. `optimistic`, +`refreshEvery` and the `refetch` command stay; `refetch` is how a document refetches for a reason no action declares. ## Declarations @@ -92,10 +90,9 @@ A scope's declarations are a `ScopeDecl`, each kind in declaration order. | Model alias | `"alias": {"type": "T", "instance": "private" \| "shared" \| e \| {"from": alias}}` | `ModelDecl`, `InstanceMode` `Default`, `Private`, `Shared`, `Keyed` (a key expression) or `Borrowed`; `"shared"` on a model the catalog does not mark stateless is refused | | State | `"name": {"type": "T"?, "init": e?}` | `StateDecl`; without a type, the type is the init's | | Derived | `"name": e` under `let` | `LetDecl` | -| Query | `{"model": alias, "action" \| "evaluate": id, "body", "when", "debounce", "refreshOn", "refreshEvery", "staleTime"}`; `refreshOn` is `ui/1` only | `QueryDecl`; exactly one of `action` and `evaluate` | -| Mutation | `{"model", "action", "body", "concurrency", "serialKey", "invalidates", "patch", "optimistic", "onSuccess", "onError", "offline"}`; `invalidates` and `patch` are `ui/1` only | `MutationDecl`, `Concurrency` | -| Patch | `{"query": q, "at": member, "key": e, "with": e}` under `patch`, read against the reply bound as `result`; `at` is optional and not empty | `PatchDecl` | -| Optimistic write | The same shape under `optimistic`, read against the run's body bound as `body` | `OptimisticDecl` | +| Query | `{"model": alias, "action" \| "evaluate": id, "body", "when", "debounce", "refreshEvery", "staleTime"}` | `QueryDecl`; exactly one of `action` and `evaluate` | +| Mutation | `{"model", "action", "body", "concurrency", "serialKey", "optimistic", "onSuccess", "onError", "offline"}` | `MutationDecl`, `Concurrency` | +| Optimistic write | `{"query": q, "at": member, "key": e, "with": e}` under `optimistic`, read against the run's body bound as `body`; `at` is optional and not empty | `OptimisticDecl` | | Watch | `{"on": e, "source": "any" \| "user", "do": commands}` under `watch` | `WatchDecl`, `WatchSource` | | Check | `{"id", "assert": e, "message": e, "targets": [ids], "blocking": bool}` under `checks` | `CheckDecl` | @@ -188,8 +185,8 @@ A node list is an array of nodes or one node. `nodeKinds()` lists the kinds. of the `upload` commands in it. All share one namespace per scope; a name declared twice is refused, and `event`, `app`, `boundary`, `props`, `result` and `body` are reserved. - **Resolution.** A name resolves in the innermost scope first, then outward. Bound names come - before declarations: a comprehension's variable, a `seq` step's reply, an `each` element, `result` - in a `patch`, `body` in an `optimistic`, `props` in a fallback. `event` is valid inside a command + before declarations: a comprehension's variable, a `seq` step's reply, an `each` element, `body` + in an `optimistic`, `props` in a fallback. `event` is valid inside a command list, `boundary.error` and `retry` inside a boundary's `error`, `row` inside a row scope and any scope nested in one, `cancelClose` inside `onClose`. `app.name` reads the app document and is not checked against it. @@ -203,9 +200,9 @@ A node list is an array of nodes or one node. `nodeKinds()` lists the kinds. `loadDocument` returns the first refusal; each rule has a case in `tests/ui/corpus/documents.json`. -- **Vocabulary.** `vocab` is `ui/1` or `ui/2`; node kinds, commands, expression forms and functions are - those of `ui/1`, which `ui/2` keeps; no unknown `!` key. A `ui/2` mutation declares no `invalidates` or - `patch` and a `ui/2` query no `refreshOn`; the refusal points at the key and says that effects replace it. +- **Vocabulary.** `vocab` is `ui/1`; node kinds, commands, expression forms and functions are those of `ui/1`; + no unknown `!` key. A mutation declares no `invalidates` or `patch` and a query no `refreshOn`; the refusal + points at the key and says that effects replace it. - **Shape.** Each key has the shape its kind gives it; required keys are present; names match `[A-Za-z_][A-Za-z0-9_]*`; closed choices (`showErrors`, `concurrency`, `source`, `offline`, `commitOn`, `onStale`, `navigate`'s `as`, `notify`'s `role`) take one of their values; chords are @@ -223,7 +220,7 @@ A node list is an array of nodes or one node. `nodeKinds()` lists the kinds. `slider`) or `app.name`; `reset` state or a dialog; `run`, `signIn` and `upload` a mutation; `refetch` a query; `open` and `close` a dialog or drawer (`close` also `self`); `focus` an element; `resetPrefs` a point or `all`; a download to preview a state. A check's targets are elements; - `invalidates`, `patch` and `optimistic` name queries of the document. + `optimistic` names a query of the document. - **Models.** Queries, mutations and model-backed nodes name a model alias in scope. An alias in a row scope names its `instance`; only a screen's own aliases borrow (`{"from": alias}`). - **Components and points.** A custom node names a declared component and only its props and @@ -244,12 +241,12 @@ credential, listFields, effects, records}`. In local mode the caller answers the server, registration answers them from the same registry; a remote client answers them from the UI service's `ui-catalog` reply. With them, a document is also refused when a model type or action is unknown, an `instance` expression names a model that is not keyed, an action belongs to another -model type, `refreshOn` names an action of another model type, an `optimistic` `with` writes a field +model type, an `optimistic` `with` writes a field the server calculates, or a mutation that sends a credential or returns a secret is queued offline. `listFields`, when present, says the query's action answers a record and names the record's members -that hold a list; a `patch` or `optimistic` of such a query is then refused without an `at` (it could +that hold a list; an `optimistic` write to such a query is then refused without an `at` (it could never find a list) or with an `at` that is not one of them. When it is absent, the catalog does not -know the result's shape, and a patch that finds no list is reported by the interpreter when it runs. +know the result's shape, and a write that finds no list is reported by the interpreter when it runs. `effects` and `records` are what the core registries hold for the action: `effects` is the list of `{kind, record?}` the action declared, `kind` one of `updates`, `changes`, `changesInstance` and `none`, `record` the record's wire name for the first two; it is absent when the action declared nothing, which makes the action a read. @@ -286,26 +283,13 @@ objects and arrays (`kMaxJsonDepth`), which is checked before the JSON parser ru `Document::warnings` holds the `LoadWarning{kind, path, text}` findings that do not refuse the document, in the order found; `message(documentId)` prints them as a refusal is printed. They need the catalog's `action` lookup, and a -document loaded without it has none. A warning is given only where the declared effects certainly cover, or -certainly miss, what the document declares. +document loaded without it has none. | Kind | When | |---|---| -| `Redundant` | An `invalidates` entry, a `patch` or a `refreshOn` entry that the effects already cover (below), in a `ui/1` document; a `ui/2` document has none of them. The path names the entry. | -| `Uncovered` | A mutation whose action declares nothing, not even `none`, that has no `invalidates` or `patch`, no `refetch` anywhere in its `onSuccess`, and that no query of the document names in `refreshOn`. After it succeeds nothing on any screen updates. `effects::none` silences it for an action that changes nothing, such as one that opens or attaches. In a `ui/2` document the text names only `refetch`, the one refresh that vocabulary has. | +| `Uncovered` | A mutation whose action declares nothing, not even `none`, and that has no `refetch` anywhere in its `onSuccess`. After it succeeds nothing on any screen updates. `effects::none` silences it for an action that changes nothing, such as one that opens or attaches. | | `WatchFeedback` | A `watch` declaration whose `source` is `any` (the default), whose `on` reads a query's value (`{"query": q}` or a `ref` to it, directly or through a `let`), and whose `do` runs (`run`, `upload`, `signIn`, at any depth) a mutation whose action declares an effect other than `none`. Another client's change refetches the query, the watch runs the mutation, and its change reaches the other client. `"source": "user"` silences it. | -**What covers a query.** A query holds exactly the records of its action (`records`); an `evaluate` query holds none. - -- `updates R` or `changes R` covers a query that holds `R`. -- `changesInstance` covers a query that reads through the same model alias declaration as the mutation, since that - is the instance the action ran on. Two aliases of one model type are two instances, so it covers nothing through - another alias. -- A `patch` is covered only by `updates R` of a record the patched query holds: the reply then patches the rows - itself. `changes R` refetches instead, which a patch exists to avoid. -- A `refreshOn` entry names an action run through any instance, so it is covered by the action's `updates` and - `changes` alone, never by `changesInstance`. - `UiRegistry::addScreen`, `UiRegistry::addFile` for the app document, and `author::ScreenRegistry::add` log each warning at `warn` (`morph::log`) when they register a document; the document registers as it would without them. diff --git a/docs/spec/ui/interpreter.md b/docs/spec/ui/interpreter.md index 708c5ed36..fdc7687ae 100644 --- a/docs/spec/ui/interpreter.md +++ b/docs/spec/ui/interpreter.md @@ -34,7 +34,7 @@ Read this before changing what a mounted document does. What a document may say | `BridgeDispatcher` | The application's `Dispatcher`: `bind` makes a `bridge::RawHandler` (`bindByType`, and `attach` for a key), `execute` is `executeRaw`. An `evaluate` query goes through the `Evaluator` it was built with (`ui::evaluatorOver`, the UI service's `ui-evaluate`); without one it is refused with a `std::runtime_error`. It has no offline queue of its own: `queueOffline` is ignored unless a `QueueingDispatcher` wraps it. A shared binding without a key is left unattached, which the bridge refuses at the first call. | | `QueueingDispatcher` | The client's offline queue (`include/morph/ui/queueing_dispatcher.hpp`), around any `Dispatcher`. A call made with `queueOffline` that fails with `backend::DisconnectedError` is held, and the caller's completion stays pending, so the mutation stays `pending` and its elements `stale`. While anything is held, a later queued call joins the queue unsent, so the server sees calls in the order they were made. `replay()` re-sends the held calls oldest first, one at a time, and settles each caller with its replay: a refusal runs the mutation's `onError`, and a call still unreachable stops the replay and stays held with every call behind it. A call without `queueOffline`, and an evaluation, are never held. `held()` and `setOnHeld` report the count. When to replay is the host's: it calls `replay()` once its transport reconnects. | | `InterpreterHost` | What only the application can do: `navigate`, `closeSelf` (`return`, `close: self`), `hostCommand`, `switchBackend` (asynchronous: it answers whether the backend is listed and calls `done` once the switch committed or failed), `resetPrefs`, `setLocale`, `notify`, `signIn`, `signOut`, `setAppValue` (`set` of `app.name`), `uploadFor` (an `UploadRequest`: the file, and the model type and action of the mutation that will reference it; its default calls `upload(file, …)`), `download`, `preferenceChanged`, `permissionsStale`. Every default answers "not supported", and the interpreter reports it. A session with host commands (`Session::setHostCommands`, filled by `HostVocabulary`, [`host_vocabulary.md`](host_vocabulary.md)) runs `{"host": …}` itself and answers `{"can": "host:…"}` from them. | -| `Session` | What screens of one client session share: the query cache, `refreshOn` bookkeeping, the permitted action ids `can` reads (`setPermitted`), the formatting facts (`setFormatting`, `setLocale`), and the `env`, `session` and `host` values. `clear()` is `signOut` and a principal change; `backendChanged(backend)` carries out a committed backend switch. `appScope()` is the mounted app document's scope. | +| `Session` | What screens of one client session share: the query cache, the mounted queries, the permitted action ids `can` reads (`setPermitted`), the formatting facts (`setFormatting`, `setLocale`), and the `env`, `session` and `host` values. `clear()` is `signOut` and a principal change; `backendChanged(backend)` carries out a committed backend switch. `appScope()` is the mounted app document's scope. | | `Interpreter` | `mount(doc, params, backend, MountOptions)` → `Screen`. | | `Screen` | A mounted screen: `root()`, `title()`, `identity()`, `dirty()`, `requestClose()` (runs `onClose`; false when it ran `cancelClose`), `beginClose()` (runs `onClose` and returns the `cancelClose` flag it shares with what it starts; null without `onClose`), `pending()` (a tracked read: an open dialog or drawer, a mutation run, or an upload or download in flight), `declares(key)`, `runtime()`, `runLifecycle(key)`, `setPreference(point, value)`, `document()`, `lend(alias)`, `read(expression)`, `diagnostics()`. Destroying it runs `onUnmount` and unmounts. A host mounts and closes screens through [host.md](host.md). | @@ -119,12 +119,9 @@ canonical body is the body's JSON with record fields in name order. first key and a change to idle move at once. - **`refreshEvery`** refetches on the scheduler's period and skips a tick while a request is in flight or a debounce waits. -- **`refreshOn`** is the session's bookkeeping: a mutation that succeeds, or a submit of a form, a - collection or a wizard that succeeds, refetches every mounted query of the same model type whose - `refreshOn` names its action, in every screen of the session. A collection's own list query and a - form's choice options are reads, not submits, and refetch nothing. - **Every reply applies its action's effects** (spec 8 §9). `Session::actionSucceeded`, which every - success goes through (mutations, and the submits of forms, collections and wizards), takes the reply + success goes through (mutations, and the submits of forms, collections and wizards; a collection's own list + query and a form's choice options are reads, not submits), takes the reply and the instance token of the alias the call went through. Each entry knows the records its action returns, with each record's key field and paths, from the catalog's `ActionInfo::records`; an `evaluate` entry holds none. @@ -135,7 +132,7 @@ canonical body is the body's JSON with record fields in name order. - `changes R` marks every entry holding `R` stale, and `changesInstance` every entry read through the same instance token. A marked entry refetches through a mounted query that uses it. - A `latest` run whose reply is discarded applies its effects as marks only: its `updates` act as - `changes`, because its values are older than the reply already applied. It runs no `refreshOn`. + `changes`, because its values are older than the reply already applied. - One reply's effects are applied in one reactive batch. - **Server events** reach the cache through `Session::deliverEvent`, which the source wires to its backend's `setEventHandler`. Each event is applied in one reactive batch, and other clients' changes @@ -165,7 +162,7 @@ canonical body is the body's JSON with record fields in name order. supersedes anything queued, then marks every entry but an `evaluate` one stale. A mounted entry refetches, and an entry whose fetch is in flight fetches once more when it settles: that fetch may have been filed before the connection came back. -- **Invalidation** (`invalidates`, `refreshOn`, `refetch`, a backend switch) marks the entry stale and +- **Invalidation** (`refetch`, a backend switch) marks the entry stale and refetches it. - **A mark during a fetch is kept.** `Session::markStale` marks an entry stale. While the entry's fetch is in flight, it also sets `refetchAfter`. That fetch may have read before the change, so when it settles @@ -188,20 +185,18 @@ reply; `parallel` applies each. `pending` counts runs in flight and queued. value with every waiting layer over it, in run order. A success folds the run's layer into the confirmed value; a failure removes it before `onError` runs. Overlapping runs therefore end at the last confirmed value whatever order they settle in. -- **A success** (applied) sets `result`, clears the error, applies `patch` to the confirmed value, - refetches `invalidates`, tells the session for `refreshOn`, runs `onSuccess` — all in one batch — and - then removes the action's secret fields from `result`. +- **A success** (applied) sets `result`, clears the error, applies the action's effects through the + session, runs `onSuccess` — all in one batch — and then removes the action's secret fields from `result`. - **A failure** (applied) sets the error and its kind and runs `onError`; a `denied` kind calls `permissionsStale`. -- **`patch` and `optimistic`** replace the row of the query's list whose `id` field equals `key`; a +- **`optimistic`** replaces the row of the query's list whose `id` field equals `key`; a record `with` is merged over a record row (its fields replace the row's), anything else replaces it. The list is the query's value itself, or, with `at`, the member of that record value `at` names; the rest of the record is kept. A key no row has changes nothing and is not a fault: the row is not - loaded. A patch whose query has a value but finds no list there — the value is not a list and there + loaded. A write whose query has a value but finds no list there — the value is not a list and there is no `at`, or the value has no list at `at` — changes nothing and is reported once through - `diagnostics()` at the mutation's `/patch` or `/optimistic` path, since it can never apply. An - `optimistic` write is checked when the run starts, a `patch` when its reply arrives; before the query - has a value there is nothing to miss. + `diagnostics()` at the mutation's `/optimistic` path, since it can never apply. It is checked when the + run starts; before the query has a value there is nothing to miss. ## Relations @@ -356,8 +351,8 @@ stops when its scope is gone. | Decision | Why | |---|---| -| Queries and mutations built on `Signal`, `Computed`, `Effect` and `Scheduler`, not on `reactive::Query` and `Mutation` | `patch` and `optimistic` write into a query's value and two screens share one value; `reactive::Query` keeps its value privately. The rules are the same. | -| The cache entry is the single source of a query's value | One request in flight, one value and one error per key, across screens, is what spec 5 §5 asks for, and `patch` then reaches every screen showing the row. | +| Queries and mutations built on `Signal`, `Computed`, `Effect` and `Scheduler`, not on `reactive::Query` and `Mutation` | `optimistic` and an `updates` reply write into a query's value and two screens share one value; `reactive::Query` keeps its value privately. The rules are the same. | +| The cache entry is the single source of a query's value | One request in flight, one value and one error per key, across screens, is what spec 5 §5 asks for, and an `updates` reply then reaches every screen showing the row. | | Optimistic writes as layers | Rolling back by restoring a saved value is wrong when runs overlap; recomputing from the confirmed value and the remaining layers is right in every order. | | `source: "user"` from the newest write | A watch cannot see who wrote a value; the session can see whether the write that preceded the flush came from a user event. | | Scoped content behind a one-row `forEach` | The view tree builds a dialog's content once, as a fixed node; a row view is the one place it builds content each time it mounts. | diff --git a/docs/spec/ui/service.md b/docs/spec/ui/service.md index b0e5e546b..28f95a4b4 100644 --- a/docs/spec/ui/service.md +++ b/docs/spec/ui/service.md @@ -44,11 +44,8 @@ the kinds are in the wire kind table ([`../core/wire.md`](../core/wire.md#the-ui Every request carries the envelope's `session`; the server stamps the verified principal onto it before the service sees it. `ui-hello` refuses a client that does not speak every vocabulary the bundle needs with -`err "missing vocabulary: [, …]"`; the bundle needs `expr/1`, the vocabulary each of its documents declares -(`ui/1` when it has none), and whatever the registry adds (`requireVocabulary`). A document is served as it was -written, so a client must read each one: a bundle of `ui/1` documents keeps serving a client that speaks only -`ui/1`, and one with a `ui/2` document, which the C++ builders write, needs a client that speaks `ui/2`. The generic -client speaks both. `hello` and `kProtocolVersion` are unchanged. +`err "missing vocabulary: [, …]"`; the bundle always needs `expr/1` and `ui/1`, and whatever the +registry adds (`requireVocabulary`). `hello` and `kProtocolVersion` are unchanged. **Negotiating events.** `ui-hello` is where a connection negotiates server-sent `event` envelopes, since the generic client already opens with it, on the same connection as its executes. `HelloRequest::events: true` offers them; diff --git a/docs/superpowers/specs/2026-10-07-ui-document-design.md b/docs/superpowers/specs/2026-10-07-ui-document-design.md index b6dc0c4a8..40a9c2c5c 100644 --- a/docs/superpowers/specs/2026-10-07-ui-document-design.md +++ b/docs/superpowers/specs/2026-10-07-ui-document-design.md @@ -257,7 +257,7 @@ evaluator. ```json "results": { "model": "sample", "action": "Lims_ListResults", "body": { "obj": { "sampleId": { "ref": "sampleId" } } }, - "when": true, "debounce": 0, "refreshOn": ["Lims_CaptureConcentration"], "refreshEvery": 0 } + "when": true, "debounce": 0, "refreshEvery": 0 } ``` A query is spec 1 §4b's `Query`, declared as data: @@ -267,18 +267,11 @@ A query is spec 1 §4b's `Query`, declared as data: - `when` false, or a `body` that evaluates to `null`, makes the query idle. - `debounce` delays a fetch until the key has been stable for that many milliseconds. - The last value is kept while a refetch is in flight, so lists do not blank. -- `refreshOn` names action ids of the query's model type: the query refetches when a call to one of - them, made by this client through any alias of that model type in any mounted screen, completes - successfully. It matches by model type, not by instance, because two private instances of a - database-backed model read the same rows: a capture in one tab must refresh the list in another. It - is the interpreter's own bookkeeping over its dispatch (through the query cache, below), not a - bridge subscription, so it works without the result type and two actions with one reply type do not - trigger each other. A change made by another client is not pushed; `refreshEvery` is the polling - answer, and it refetches on a timer and **skips a tick while a request is in flight**. +- A query refreshes when an action changes a record it holds, on every client: the action declares what it + changes (its effects, spec 8), not the document. `refreshEvery` refetches on a timer, for data that changes with + time rather than through actions, and **skips a tick while a request is in flight**. - `staleTime` (milliseconds, default 0) is how long a cached value counts as fresh (below). -- `refreshOn` is `ui/1` only. In `ui/2` an action declares what it changes (its effects, spec 8 §10), and - every query holding a changed record refreshes, on every client; a `ui/2` query that declares `refreshOn` - is refused. `refreshEvery` is in both, for data that changes with time rather than through actions. +- A query declares no `refreshOn`: a document that names one is refused (spec 8 §10). ### The query cache @@ -292,9 +285,8 @@ share it. - **Freshness.** A query that mounts on an entry younger than its `staleTime` uses the cached value and sends nothing; an older entry is shown at once and refetched (the value is kept while it refetches, as for any query). An entry no mounted query uses is kept for 5 minutes, then dropped. -- **Invalidation.** `invalidates`, `refreshOn` and `{"refetch": "q"}` mark entries stale and refetch - those a mounted query uses, in one batch. `invalidates` names queries of the mutation's own document; - `refreshOn` reaches every mounted screen (above). +- **Invalidation.** An action's effects (spec 8 §9) and `{"refetch": "q"}` mark entries stale and refetch + those a mounted query uses, in one batch. - **Isolation.** The cache belongs to the session: `signOut` and a principal change clear it, and a backend switch (spec 6 §9) marks every entry stale. A `secret` field is never cached (§13). - **Prefetch.** A screen may declare `"prefetch": ["q", ...]`: queries whose bodies depend only on the @@ -310,8 +302,7 @@ fetcher that goes through the cache. ```json "capture": { "model": "sample", "action": "Lims_CaptureConcentration", "body": { ... }, - "concurrency": "exclusive", "invalidates": ["results"], - "patch": { "query": "results", "key": { "ref": "result.id" }, "with": { "ref": "result" } }, + "concurrency": "exclusive", "onSuccess": [ ... ], "onError": [ ... ] } ``` @@ -322,17 +313,14 @@ fetcher that goes through the cache. serializes per task whichever command started the run. - `latest`: every run is sent; only the newest reply is applied. - `parallel`: every run is sent; replies are applied as they arrive. -- `invalidates` refetches the named queries after a success, in one batch with the result, so the - screen updates in one frame. -- `patch` replaces one row of a query's cached list (the row whose `id` field equals `key`) with - `with` — a record `with` is merged over a record row, its fields replacing the row's — - without refetching. `key` and `with` are evaluated against the reply, bound as `result`. -- `invalidates` and `patch` are `ui/1` only: in `ui/2` the action's effects refresh what it changed, and a - `ui/2` mutation that declares either is refused. A `ui/2` document that must refetch for a reason no action - declares runs `{"refetch": "q"}` in `onSuccess`. -- `optimistic` is its own declaration, in both vocabularies, with the same shape as `patch` — - `{"query": "q", "at"?: m, "key": e, "with": e}` — but is applied when the run starts, evaluated against the run's body bound as `body`. A success confirms the written fields - (and `patch`, when declared, then applies the reply); a failure rolls the row back to its value +- A success applies the action's effects (spec 8 §9): an `updates` reply replaces the rows it holds in every + cached query, without refetching, and a `changes` refetches the queries holding the record. A mutation declares + no `invalidates` or `patch`; a document that names either is refused. A document that must refetch for a reason + no action declares runs `{"refetch": "q"}` in `onSuccess`. +- `optimistic` replaces one row of a query's cached list (the row whose `id` field equals `key`) with `with` — a + record `with` is merged over a record row — as `{"query": "q", "at"?: m, "key": e, "with": e}`, applied when the + run starts, evaluated against the run's body bound as `body`. A success confirms the written fields + (and the reply's effects then apply); a failure rolls the row back to its value without that run's write, and `onError` runs after the rollback. Only client-owned fields may be written optimistically: a field the server calculates (§7) is never predicted, and a document whose `with` names one is refused at @@ -473,8 +461,8 @@ Two declarations cover it. **Write-through: save each edit.** Committing a value runs a mutation that persists it and replies with the recalculated row. `concurrency: "serial"` with `serialKey` set to the row id sends one row's -edits in order. The mutation's `patch` replaces the row in the list, so the calculated cells update -without refetching the list. +edits in order. The action declares `updates` of the row's record, so its reply replaces the row in the list and +the calculated cells update without refetching the list. **Rules for both:** @@ -695,7 +683,7 @@ MORPH_REGISTER_SCREEN(screen); compile error on the server. A query or mutation body is the action's own members with expression handles in place of values, written as `{&Action::member, handle}` pairs: C++ cannot synthesise the action's struct with its members retyped, which `{.sampleId = sampleId}` would - need. The builders write `ui/2`, so they offer no `invalidates`, `patch` or `refreshOn` (§5). + need. Documents have no `invalidates`, `patch` or `refreshOn`, so the builders offer none (§5). - Registration emits the JSON once and runs the load-time validation of §3; a screen that fails is a failing server test, never a client surprise. - Registration also runs the **accessibility rules**: every text input, date-time input, slider, select, @@ -751,11 +739,11 @@ MORPH_REGISTER_SCREEN(screen); - **Versions.** Within a vocabulary version, changes are additive: a client ignores an optional key it does not know, unless the key's name begins with `!` (`"!requires"`), which marks it must-understand: a client that does not know it refuses the node and renders its fallback, or - refuses the document. A new node kind, function or command, or a removed key, is a new vocabulary - version. `ui/2` is the first: `ui/1` without `invalidates`, `patch` and `refreshOn` (§5). A client - reads every version it speaks, and a document is served as written, so a bundle needs the - vocabulary of each of its documents: a server keeps `ui/1` documents to keep serving clients that - speak only `ui/1`, and the C++ builders (§11) write `ui/2`. + refuses the document. A new node kind, function or command is a new vocabulary version + (`ui/2`, `expr/2`). A server may keep documents for more than one version and sends the newest the + client speaks. A key leaves a vocabulary by being refused, never ignored, so a document that still + declares it fails at load instead of silently losing what it did (`invalidates`, `patch` and + `refreshOn`, §5). - **Development reload.** A client started in development mode re-sends `ui-hello` every second. When the manifest digest changes, it fetches the changed entries and remounts the mounted screens whose documents changed, keeping the screen scope's state where the new document still declares it under @@ -834,7 +822,7 @@ places, none of which changes the document: - **Expressions:** a shared JSON corpus of expressions, inputs and results, covering exact arithmetic across units, null propagation, three-valued logic, every function, and the size bound. - **Control:** query latest-wins under forced reply orders; `debounce`; `refreshEvery` skipping while - pending; each mutation concurrency mode; `patch`; one-batch invalidation. + pending; each mutation concurrency mode; an `updates` reply replacing a row without a refetch. - **Relations:** cascading `watch` with `source: "user"` against a server patch; check targets and `ready`; dependent options with `onStale`; cycle refusal. - **Server-calculated values (§7):** replies out of order; typing while a calculation is in flight; @@ -864,10 +852,8 @@ places, none of which changes the document: handshake; a server without the UI service; a `!`-prefixed key an old client does not know. - **Bridge seams:** a binding by type id, a raw execute and a string-keyed attach behave as their typed counterparts for session, timeout, cancellation, accounting and `switchBackend`. -- **Dispatch refresh:** `refreshOn` fires on the named action through the alias and not on another - action with the same reply type. - **Query cache:** two screens with an equal key send one request; `staleTime` spares a remount's - fetch; a mutation in one tab refreshes, through `refreshOn`, a query of the same model type in + fetch; a mutation in one tab refreshes, through its effects, a query holding the record it changed in another; `signOut` clears the cache; a prefetch warms the entry the screen then mounts on. - **Optimistic:** the row shows `with` before the reply, the reply replaces it, a failure rolls it back before `onError` runs, two overlapping runs end at the last confirmed value, and an optimistic write @@ -889,9 +875,8 @@ places, none of which changes the document: - **Free layout editing.** A user adjusts a screen only through its declared customization points (§12b); positioning or sizing panels freely is not part of this design. Ordering and hiding the children a document declares is the `arrange` point. -- **Server push.** A document learns of a change by its own calls, by `refreshOn` over its own - dispatch and by polling with `refreshEvery`. A server-initiated message (pushed progress, another - client's change) needs a wire kind the protocol does not have; it is carried by the core - `event` kind, which `ui-hello` negotiates per connection. +- **Server push.** A document learns of a change through its actions' effects, from its own calls and, + over the core `event` kind that `ui-hello` negotiates per connection, from other clients' (spec 8), and + by polling with `refreshEvery`. - **Server-side composite actions.** Committing several models in one transaction is not part of this design; a dialog that must commit atomically runs one server action (§5). diff --git a/docs/superpowers/specs/2026-10-16-server-push-design.md b/docs/superpowers/specs/2026-10-16-server-push-design.md index 835065802..079505b7e 100644 --- a/docs/superpowers/specs/2026-10-16-server-push-design.md +++ b/docs/superpowers/specs/2026-10-16-server-push-design.md @@ -484,34 +484,31 @@ The interpreter's query cache applies a change the same way whether it came from With effects declared, a document needs no refresh declarations. -**At load, in `ui/1`, additively.** The loader already has the catalog, and it reports: +**At load.** The loader already has the catalog, and it reports: - **refused: a query whose action declares effects** (anything other than nothing declared). A query must read, so a - fetch never produces a change and two clients cannot refetch each other through their queries. It is new in what - it refuses, so it applies only to actions that declare effects, which no existing action does. + fetch never produces a change and two clients cannot refetch each other through their queries. +- **refused: `invalidates`, `patch` and `refreshOn`.** They restate, per document, what the action's effects say once + for every client, and a restatement can only drift from the action. A document that declares one is refused + rather than having it ignored, since its author meant a refresh the client would silently not do. + - `optimistic` stays, with its `{query, at?, key, with}` shape. + - A document that must refetch for a reason no action declares uses the existing `{"refetch": "q"}` command. + - `refreshEvery` stays, for data that changes with time rather than through actions, such as a clock or an + external feed. - **warning, watch feedback:** a `watch` declaration (spec 5 §6, not the `watch` wire kind of §7) whose `on` reads a query's value, whose `source` is `any` (the default), and whose `do` runs a mutation with effects. Another client's change refetches the query, the watch fires, and the mutation's change reaches the other client, which may do the same. Such a watch should use `"source": "user"`. The loader cannot prove a ring, so this is a warning, and the claim above is limited to queries. -- **warning, redundant:** an `invalidates`, `patch` or `refreshOn` that the mutation's action's effects already - cover, so a document can migrate one screen at a time. -- **warning, uncovered:** a mutation whose action declares nothing (not even `none`), with no `invalidates` or - `patch` of its own and no query naming it in `refreshOn`. After it succeeds, nothing on any screen updates. - `effects::none` silences it for actions that attach or open, such as lims's `OpenSample`. - -**In `ui/2`, the removal.** `invalidates`, `patch` and `refreshOn` are removed from mutations and queries. Removing -keys is not additive, so it is a new vocabulary version under spec 5 §12. A server that keeps `ui/1` documents for -older clients keeps them working. - -- `optimistic` stays, and keeps the `{query, key, with}` shape it has today, now as its own definition rather than - `patch`'s. -- A document that must refetch for a reason no action declares uses the existing `{"refetch": "q"}` command. -- `refreshEvery` stays, for data that changes with time rather than through actions, such as a clock or an external - feed. +- **warning, uncovered:** a mutation whose action declares nothing (not even `none`) and that runs no `refetch` in + its `onSuccess`. After it succeeds, nothing on any screen updates. `effects::none` silences it for actions that + attach or open, such as lims's `OpenSample`. + +The removal stays in `ui/1` rather than opening a `ui/2`: a refused key is loud, so no document loses a refresh +silently, and one vocabulary keeps the server from choosing documents per client. **The C++ authoring layer** (`include/morph/ui/author.hpp`) gains nothing for effects, because they are declared -beside the action, not in a document. It drops `.invalidates`, `.patch` and `.refreshOn` when it emits `ui/2`, and +beside the action, not in a document. It offers no `.invalidates`, `.patch` or `.refreshOn`, and its golden files (`tests/ui/golden/`) change with it. ## 11. Agents through the server @@ -649,7 +646,7 @@ Every test below must fail if the feature does nothing, so each names the change - *Mutation: restore the unconditional `stale = false`; the mid-flight case fails.* - **Documents (ui).** - A query whose action declares effects is refused at load. - - A redundant `invalidates` warns. + - An `invalidates` is refused. - An uncovered mutation warns, and `effects::none` silences it. - The watch-feedback shape warns. - **Explorer (ui).** An action with `changes` shows `changes R` in `explorer_facts`, and one with nothing @@ -680,9 +677,9 @@ row, and keeps every new public symbol's Doxygen complete. The implementation pl | 3 | **Catalog and wire shapes.** `ActionInfo::effects` and `records` (key field, paths), local lookup and `ui-catalog`; `events` in `ui-hello`, and `ConnectionFacts` on `IEnvelopeService::handle`; the `watch`/`unwatch` and `changes` body shapes, defined here so parts 4 and 5 share them | ui, core (`wire.hpp` shapes, `remote.hpp` `ConnectionFacts`) | 1, 2 | `ui/service.md`, `ui/document.md`, `core/wire.md`, spec 5 §15 (the push kind is core `event`, not `ui-event`) | | 4 | **Watching and fan-out.** Core `watch`/`unwatch` kinds in `RemoteServer` with the `cid`; the watch table with session contexts; the principal comparison in `stampVerifiedPrincipal`; fan-out in `dispatchExecute`, with private instances to their owner; `derivedFrom` expansion, coalescing, re-authorisation, `privateKeys` | core | 1, 2, 3 | `core/backend.md`, `core/shared_instances.md`, `security.md` | | 5 | **Client cache and reconnect seams.** `actionSucceeded` carries the reply and applies effects; patch by key field and path; `refetchAfter`, set by a stale mark or a patch on a fetching entry; discarded `latest` replies as marks; events and `resync`; topic reference counting; watch before fetch; `Session::reconnected()`; `AppSource::onConnected` with an empty default; `watcherOver` and `SessionOptions::watcher`; the generic client's remote source and shell wired to them (`examples/common/app`) | ui, examples/common | 2, 3 (4 for its remote tests) | `ui/interpreter.md`, `ui/frontend.md`, `ui/service.md`, `offline/offline.md`, spec 5 §5 | -| 6 | **Document checks.** Query-must-read refused; watch-feedback, redundant and uncovered warnings | ui | 3 | `ui/document.md` | +| 6 | **Document checks.** Query-must-read refused; watch-feedback and uncovered warnings | ui | 3 | `ui/document.md` | | 7 | **End-to-end harness, examples and the explorer.** The two-session harness in `tests/`. Records and effects in the three rungs: bookmarks, bank (both model sets) and lims, with `none` on attach-style actions; their 9 `invalidates`, 5 `refreshOn` and 1 `patch` removed from `ui/1` documents where the warnings say so. The explorer's effects and records rows | tests, examples, ui | 4, 5, 6 | the rungs' READMEs, `testing_strategy.md`, `ui/explorer.md` | -| 8 | **`ui/2`.** Remove `invalidates`, `patch` and `refreshOn`; `optimistic` keeps its own shape; the authoring layer and golden files emit `ui/2`; the server keeps serving `ui/1` | ui | 7 | `ui/document.md`, `ui/author.md`, spec 5 §5 and §12 | +| 8 | **No refresh keys.** `ui/1` refuses `invalidates`, `patch` and `refreshOn`; `optimistic` keeps its own shape; the authoring layer offers none of them | ui | 7 | `ui/document.md`, `ui/author.md`, spec 5 §5 and §12 | | 9 | **MCP adapter.** `McpAdapter` over a connection scope, tools from `permitted`, calls through `RemoteServer::handle`, `serveStdio`, bank's `--mcp-stdio` | ui, examples | 4 | a new `ui/agent.md` | **Order in the branch:** 1, 2, 3, 4, 5, 6, 7, 8, 9. Parts 1 and 2 are independent and 6 needs only 3, but one diff --git a/examples/common/app/app_source.cpp b/examples/common/app/app_source.cpp index cf698fb33..1f843f9da 100644 --- a/examples/common/app/app_source.cpp +++ b/examples/common/app/app_source.cpp @@ -123,7 +123,7 @@ std::expected readClientArgs(std::vector& } std::vector clientVocabularies() { - return {std::string{ui::kVocabulary}, std::string{ui::kVocabulary2}, std::string{ui::kExprVocabulary}}; + return {std::string{ui::kVocabulary}, std::string{ui::kExprVocabulary}}; } ui::HelloRequest clientHello() { diff --git a/examples/common/app/app_source.hpp b/examples/common/app/app_source.hpp index 5d0d3d80b..f8c9e5715 100644 --- a/examples/common/app/app_source.hpp +++ b/examples/common/app/app_source.hpp @@ -79,7 +79,7 @@ struct LocalApp { std::function()> serverAuthorizer; }; -/// @brief The vocabularies the generic client speaks: both document vocabularies, `ui/1` and `ui/2`, and `expr/1`. +/// @brief The vocabularies the generic client speaks. [[nodiscard]] std::vector clientVocabularies(); /// @brief The `ui-hello` the generic client opens with, and sends again on each poll: its vocabularies, and the diff --git a/include/morph/ui/author.hpp b/include/morph/ui/author.hpp index 1366c7e8c..616dcaa87 100644 --- a/include/morph/ui/author.hpp +++ b/include/morph/ui/author.hpp @@ -1563,7 +1563,7 @@ class Commands { std::vector _items; }; -/// @brief One command of `ui/2` (spec 5 §8, spec 6). +/// @brief One command of `ui/1` (spec 5 §8, spec 6). class Command { public: /// @brief A command over its JSON, for a form no builder spells. @@ -2399,7 +2399,7 @@ class Scope : public DeclScope { /// @return The document. [[nodiscard]] Json document(std::string const& documentId, Node const& view) const { Json out = Json::object_t{}; - detail::put(out, "vocab", std::string{kVocabulary2}); + detail::put(out, "vocab", std::string{kVocabulary}); detail::put(out, "id", documentId); for (auto const& [key, value] : std::get(_head.data)) { detail::put(out, key, value); diff --git a/include/morph/ui/document.hpp b/include/morph/ui/document.hpp index 8318095db..b259d5e90 100644 --- a/include/morph/ui/document.hpp +++ b/include/morph/ui/document.hpp @@ -33,14 +33,9 @@ namespace morph::ui { -/// @brief The first document vocabulary: queries and mutations declare their own refreshes (`invalidates`, `patch`, -/// `refreshOn`). +/// @brief The vocabulary a document declares; the only one this client reads. inline constexpr std::string_view kVocabulary = "ui/1"; -/// @brief The document vocabulary without the refresh keys: an action declares what it changes, so a `ui/2` document -/// has no `invalidates`, `patch` or `refreshOn`. This client reads both. -inline constexpr std::string_view kVocabulary2 = "ui/2"; - /// @brief What the catalog says about a model type. struct ModelInfo { /// @brief Whether a shared singleton is equivalent to a private instance. @@ -106,7 +101,7 @@ struct ActionInfo { /// model registry in-process, the server at registration, a remote client from the UI service's catalog. /// /// A lookup left empty skips the rules that need it: `model` the unknown-model and keyed-instance rules; `action` -/// the unknown-action, wrong-model, `refreshOn`, optimistic and offline rules. +/// the unknown-action, wrong-model, optimistic and offline rules. struct Catalog { /// @brief Looks up a model type by its registered type id. std::function(std::string_view typeId)> model; @@ -216,8 +211,6 @@ struct QueryDecl { std::int64_t refreshEvery = 0; /// @brief Milliseconds a cached value counts as fresh. std::int64_t staleTime = 0; - /// @brief Action ids of the same model type whose success refetches the query; `ui/1` only. - std::vector refreshOn; }; /// @brief What a mutation run does while another is in flight. @@ -228,18 +221,6 @@ enum class Concurrency : std::uint8_t { Parallel, ///< Every run is sent; replies apply as they arrive. }; -/// @brief A `patch`: the row a mutation's reply replaces, without refetching; `ui/1` only. -struct PatchDecl { - /// @brief The query whose cached list it patches. - std::string query; - /// @brief The member of the query's record value that holds the list; empty when the value is the list itself. - std::string at; - /// @brief The row key to replace, read against the reply bound as `result`. - ExprPtr key; - /// @brief The new row, read against the reply bound as `result`. - ExprPtr with; -}; - /// @brief An `optimistic` write: the row a mutation run predicts before its reply. struct OptimisticDecl { /// @brief The query whose cached list it writes. @@ -328,10 +309,6 @@ struct MutationDecl { Concurrency concurrency = Concurrency::Exclusive; /// @brief The queue key of `serial` runs. ExprPtr serialKey; - /// @brief Queries refetched after a success; `ui/1` only. - std::vector invalidates; - /// @brief The row a reply replaces, without refetching; `ui/1` only. - std::optional patch; /// @brief The row a run predicts before its reply. std::optional optimistic; /// @brief Commands run after a success. @@ -510,14 +487,13 @@ struct CustomizePoint { /// @brief What a load-time warning is about. A warning never refuses a document. enum class LoadWarningKind : std::uint8_t { WatchFeedback, ///< A watch of a query runs a mutation that changes data, on any change of it. - Redundant, ///< An `invalidates`, `patch` or `refreshOn` that the action's effects already cover. Uncovered, ///< A mutation whose action declares nothing, and after which nothing refreshes. }; /// @brief A finding of load-time validation that does not refuse the document. struct LoadWarning { /// @brief What it is about. - LoadWarningKind kind = LoadWarningKind::Redundant; + LoadWarningKind kind = LoadWarningKind::Uncovered; /// @brief The JSON Pointer of the declaration it names. std::string path; /// @brief The finding, in one sentence. @@ -538,7 +514,7 @@ struct LoadWarning { /// @brief A loaded and validated UI document: a screen, or the app document (id `app`, spec 6 §5). struct Document { - /// @brief The vocabulary, `ui/1` or `ui/2`. + /// @brief The vocabulary, `ui/1`. std::string vocab; /// @brief The document id, such as `lims.results`. std::string id; @@ -1019,10 +995,9 @@ class DocumentParser { refuseMustUnderstand(*object, ""); Document doc; doc.vocab = text(*object, "vocab", "", true); - if (doc.vocab != kVocabulary && doc.vocab != kVocabulary2) { - refuse("/vocab", "the vocabulary `" + doc.vocab + "` is not supported; this client reads ui/1 and ui/2"); + if (doc.vocab != kVocabulary) { + refuse("/vocab", "the vocabulary `" + doc.vocab + "` is not supported; this client reads ui/1"); } - _refreshKeys = doc.vocab == kVocabulary; doc.id = text(*object, "id", "", true); if (doc.id.empty()) { refuse("/id", "a document has a non-empty id"); @@ -1289,20 +1264,21 @@ class DocumentParser { return decl; } - /// @brief Refuses @p key of @p fields in a vocabulary without the refresh keys. + /// @brief Refuses @p key of @p fields: a refresh key, which a document does not declare. /// @param fields The declaration's fields. /// @param path The declaration's JSON Pointer. /// @param key The refresh key: `invalidates`, `patch` or `refreshOn`. - void refuseRefreshKey(Json::object_t const& fields, std::string const& path, std::string_view key) const { - if (!_refreshKeys && member(fields, key) != nullptr) { + static void refuseRefreshKey(Json::object_t const& fields, std::string const& path, std::string_view key) { + if (member(fields, key) != nullptr) { refuse(pointerAppend(path, key), - std::format("`{}` is not in ui/2: an action's effects say what it changes, and every query holding " - "a changed record refreshes; a refresh no action declares is a `refetch` command", + std::format("a document does not declare `{}`: an action's effects say what it changes, and every " + "query holding a changed record refreshes; a refresh no action declares is a `refetch` " + "command", key)); } } - [[nodiscard]] QueryDecl query(std::string const& name, Json const& json, std::string const& path) const { + [[nodiscard]] static QueryDecl query(std::string const& name, Json const& json, std::string const& path) { checkName(name, path); auto const& fields = objectOf(json, path, "a query is an object"); refuseMustUnderstand(fields, path); @@ -1323,21 +1299,15 @@ class DocumentParser { decl.refreshEvery = integer(fields, "refreshEvery", path); decl.staleTime = integer(fields, "staleTime", path); refuseRefreshKey(fields, path, "refreshOn"); - if (Json const* const refreshOn = member(fields, "refreshOn")) { - decl.refreshOn = nameList(*refreshOn, pointerAppend(path, "refreshOn")); - } return decl; } - /// @brief Parses a row replacement, a `patch` or an `optimistic` write: `{"query": q, "at"?: m, "key": e, - /// "with": e}`. - /// @tparam Decl `PatchDecl` or `OptimisticDecl`. - /// @param json The replacement's JSON. - /// @param path Its JSON Pointer. - /// @param shape The refusal of a replacement without that shape. - /// @return The replacement. - template - [[nodiscard]] static Decl rowReplacement(Json const& json, std::string const& path, char const* shape) { + /// @brief Parses an `optimistic` write: `{"query": q, "at"?: m, "key": e, "with": e}`. + /// @param json The write's JSON. + /// @param path Its JSON Pointer. + /// @return The write. + [[nodiscard]] static OptimisticDecl optimistic(Json const& json, std::string const& path) { + static constexpr char const* shape = R"(an optimistic write is {"query": q, "at"?: m, "key": e, "with": e})"; auto const& fields = objectOf(json, path, shape); refuseMustUnderstand(fields, path); Json const* const key = member(fields, "key"); @@ -1349,10 +1319,10 @@ class DocumentParser { if (member(fields, "at") != nullptr && listMember.empty()) { refuse(pointerAppend(path, "at"), "`at` names the member of the query's record that holds the list"); } - return Decl{.query = text(fields, "query", path, true), - .at = std::move(listMember), - .key = expr(*key, pointerAppend(path, "key")), - .with = expr(*with, pointerAppend(path, "with"))}; + return OptimisticDecl{.query = text(fields, "query", path, true), + .at = std::move(listMember), + .key = expr(*key, pointerAppend(path, "key")), + .with = expr(*with, pointerAppend(path, "with"))}; } [[nodiscard]] MutationDecl mutation(std::string const& name, Json const& json, std::string const& path) { @@ -1380,17 +1350,8 @@ class DocumentParser { } refuseRefreshKey(fields, path, "invalidates"); refuseRefreshKey(fields, path, "patch"); - if (Json const* const invalidates = member(fields, "invalidates")) { - decl.invalidates = nameList(*invalidates, pointerAppend(path, "invalidates")); - } - if (Json const* const value = member(fields, "patch")) { - decl.patch = rowReplacement(*value, pointerAppend(path, "patch"), - R"(a patch is {"query": q, "at"?: m, "key": e, "with": e})"); - } if (Json const* const value = member(fields, "optimistic")) { - decl.optimistic = rowReplacement( - *value, pointerAppend(path, "optimistic"), - R"(an optimistic write is {"query": q, "at"?: m, "key": e, "with": e})"); + decl.optimistic = optimistic(*value, pointerAppend(path, "optimistic")); } if (Json const* const commands = member(fields, "onSuccess")) { decl.onSuccess = commandList(*commands, pointerAppend(path, "onSuccess")); @@ -1972,9 +1933,6 @@ class DocumentParser { } } // NOLINTEND(misc-no-recursion) - - /// @brief Whether the document's vocabulary has the refresh keys, as `ui/1` does. - bool _refreshKeys = true; }; /// @brief What a name in a scope declares. @@ -2556,21 +2514,6 @@ class DocumentValidator { if (query.when) { checkCondition(*query.when, context, "`when`"); } - checkRefreshOn(query, *lookup(context.scope, query.model)->model); - } - - void checkRefreshOn(QueryDecl const& query, ModelDecl const& model) { - if (!_options->catalog.action) { - return; - } - for (std::size_t index = 0; index < query.refreshOn.size(); ++index) { - std::string const& action = query.refreshOn.at(index); - auto const info = _options->catalog.action(action); - if (!info || info->modelType != model.type) { - refuse(pointerAppend(pointerAppend(query.path, "refreshOn"), index), - "`refreshOn` names actions of the query's model type; `" + action + "` is not one"); - } - } } void checkMutation(MutationDecl const& mutation, CheckContext const& context) { @@ -2583,32 +2526,19 @@ class DocumentValidator { keyContext.bindings.emplace_back("body", Type::any()); checkExpr(*mutation.serialKey, keyContext); } - for (std::size_t index = 0; index < mutation.invalidates.size(); ++index) { - if (!_queries.contains(mutation.invalidates.at(index))) { - refuse(pointerAppend(pointerAppend(mutation.path, "invalidates"), index), - "`" + mutation.invalidates.at(index) + "` is not a query of this document"); - } - } - // NOLINTNEXTLINE(bugprone-easily-swappable-parameters): the key of the patch, then the name its reply is bound to. - auto const patch = [&](auto const& decl, std::string_view key, std::string_view bound) { - std::string const path = pointerAppend(mutation.path, key); + if (mutation.optimistic) { + OptimisticDecl const& decl = *mutation.optimistic; + std::string const path = pointerAppend(mutation.path, "optimistic"); auto const target = _queries.find(decl.query); if (target == _queries.end()) { refuse(pointerAppend(path, "query"), "`" + decl.query + "` is not a query of this document"); } CheckContext bodyContext = context; - bodyContext.bindings.emplace_back(std::string{bound}, Type::any()); + bodyContext.bindings.emplace_back("body", Type::any()); checkExpr(*decl.key, bodyContext); checkExpr(*decl.with, bodyContext); checkPatchAt(decl.query, decl.at, *target->second, path); - return target->second; - }; - if (mutation.patch) { - static_cast(patch(*mutation.patch, "patch", "result")); - } - if (mutation.optimistic) { - QueryDecl const* const query = patch(*mutation.optimistic, "optimistic", "body"); - checkOptimistic(*mutation.optimistic, *query); + checkOptimistic(decl, *target->second); } if (mutation.offlineQueue && action) { if (action->credential || !action->secretFields.empty()) { @@ -3102,41 +3032,6 @@ class DocumentValidator { std::ranges::any_of(*info.effects, [](ActionEffect const& effect) { return effect.kind != "none"; }); } - [[nodiscard]] ModelDecl const* aliasOf(QueryDecl const& query) const { - auto const found = std::ranges::find(_queryScopes, &query, &std::pair::first); - return lookup(found->second, query.model)->model; - } - - /// Why the effects of @p actionId cover @p query, or nothing when they do not certainly cover it. A query holds - /// exactly the records of its action, so `updates` or `changes` of one of them reaches it. `changesInstance` - /// reaches the entries read through the instance the action ran on; @p alias is the alias the action runs - /// through, and covers only a query of that same alias, since two aliases of one model type are two instances. - /// Without an alias, as for `refreshOn`, which names an action run through any instance, it covers nothing. - /// @p updatesOnly limits the cover to `updates`, the effect whose reply patches rows as a `patch` does. - // NOLINTNEXTLINE(bugprone-easily-swappable-parameters): the action, then the query it may cover. - [[nodiscard]] std::optional covers(std::string const& actionId, ActionInfo const& action, - QueryDecl const& query, ModelDecl const* alias, - bool updatesOnly) const { - if (!action.effects || query.evaluate) { - return std::nullopt; - } - auto const held = _options->catalog.action(query.action); - for (ActionEffect const& effect : *action.effects) { - bool const records = effect.kind == "updates" || (!updatesOnly && effect.kind == "changes"); - if (records && held && effect.record) { - std::string const& record = *effect.record; - if (std::ranges::find(held->records, record, &ActionRecordInfo::record) != held->records.end()) { - return std::format("`{}` {} `{}`, which `{}` holds", actionId, effect.kind, record, query.name); - } - } - if (!updatesOnly && effect.kind == "changesInstance" && alias != nullptr && aliasOf(query) == alias) { - return std::format("`{}` changes the instance of `{}`, through which `{}` reads", actionId, - query.model, query.name); - } - } - return std::nullopt; - } - void warn(LoadWarningKind kind, std::string path, std::string text) { _warnings.push_back(LoadWarning{.kind = kind, .path = std::move(path), .text = std::move(text)}); } @@ -3145,60 +3040,24 @@ class DocumentValidator { if (!_options->catalog.action) { return; } - for (auto const& [mutation, scope] : _mutations) { - checkMutationEffects(*mutation, *scope); - } - for (auto const& [query, scope] : _queryScopes) { - static_cast(scope); - for (std::size_t index = 0; index < query->refreshOn.size(); ++index) { - std::string const& actionId = query->refreshOn.at(index); - auto const action = _options->catalog.action(actionId); - if (auto const why = action ? covers(actionId, *action, *query, nullptr, false) : std::nullopt) { - warn(LoadWarningKind::Redundant, pointerAppend(pointerAppend(query->path, "refreshOn"), index), - "`refreshOn` is redundant: " + *why); - } - } + for (auto const& entry : _mutations) { + checkMutationEffects(*entry.first); } for (auto const& [watch, scope] : _watches) { checkWatchFeedback(*watch, scope); } } - void checkMutationEffects(MutationDecl const& mutation, ScopeInfo& scope) { + void checkMutationEffects(MutationDecl const& mutation) { auto const action = _options->catalog.action(mutation.action); if (!action) { return; } - ModelDecl const* const alias = lookup(&scope, mutation.model)->model; - for (std::size_t index = 0; index < mutation.invalidates.size(); ++index) { - QueryDecl const& query = *_queries.at(mutation.invalidates.at(index)); - if (auto const why = covers(mutation.action, *action, query, alias, false)) { - warn(LoadWarningKind::Redundant, pointerAppend(pointerAppend(mutation.path, "invalidates"), index), - "`invalidates` is redundant: " + *why); - } - } - if (mutation.patch) { - QueryDecl const& query = *_queries.at(mutation.patch->query); - if (auto const why = covers(mutation.action, *action, query, alias, true)) { - warn(LoadWarningKind::Redundant, pointerAppend(mutation.path, "patch"), - "`patch` is redundant: " + *why + ", and its reply patches the rows"); - } - } - bool const refreshed = - !mutation.invalidates.empty() || mutation.patch || refetches(mutation.onSuccess) || - std::ranges::any_of(_queryScopes, [&](auto const& entry) { - return std::ranges::find(entry.first->refreshOn, mutation.action) != entry.first->refreshOn.end(); - }); - if (!action->effects && !refreshed) { - // A `ui/2` document has no refresh keys, so the only refresh it can name is a `refetch`. - std::string_view const names = _doc->vocab == kVocabulary - ? "no `invalidates`, `patch`, `refetch` or `refreshOn` names it" - : "no `refetch` follows it"; - warn( - LoadWarningKind::Uncovered, mutation.path, - std::format("nothing refreshes after `{}` succeeds: `{}` declares no effects, and {}; declare what it " - "changes, or `effects::none` when it changes nothing", - mutation.name, mutation.action, names)); + if (!action->effects && !refetches(mutation.onSuccess)) { + warn(LoadWarningKind::Uncovered, mutation.path, + std::format("nothing refreshes after `{}` succeeds: `{}` declares no effects, and no `refetch` " + "follows it; declare what it changes, or `effects::none` when it changes nothing", + mutation.name, mutation.action)); } } @@ -3433,7 +3292,7 @@ inline std::expected ScopeResolver::pref(std::string_view poi /// /// A document is refused when it does not parse, when a reference does not resolve, when a type does not check, /// when the `let` or `watch` graph has a cycle, when an expression exceeds 256 nodes, when a node kind, command or -/// function is not in the vocabulary, when a `ui/2` document declares `invalidates`, `patch` or `refreshOn`, when a +/// function is not in the vocabulary, when a document declares `invalidates`, `patch` or `refreshOn`, when a /// must-understand key is unknown, or when a rule of spec 5 or spec 6 fails; the list is in /// `docs/spec/ui/document.md`. The rules that need catalog facts run when @p options supplies them. /// @param json The document's JSON. diff --git a/include/morph/ui/interpreter.hpp b/include/morph/ui/interpreter.hpp index 45bcb3168..569cab796 100644 --- a/include/morph/ui/interpreter.hpp +++ b/include/morph/ui/interpreter.hpp @@ -723,7 +723,7 @@ struct CacheEntry { std::uint64_t generation = 0; /// @brief When the last reply was applied. std::optional fetchedAt; - /// @brief Marked stale by an invalidation, `refreshOn`, a backend switch or a change. + /// @brief Marked stale by a `refetch`, a backend switch or a change. bool stale = false; /// @brief Marked stale while a fetch was in flight: that fetch may have read before the change, so when it /// settles the entry stays stale and fetches once more. @@ -832,7 +832,7 @@ struct HostCommands { std::function run; }; -/// @brief What screens of one client session share (spec 5 §5, §12b): the query cache, `refreshOn` bookkeeping, the +/// @brief What screens of one client session share (spec 5 §5, §12b): the query cache, the mounted queries, the /// principal's permitted actions, the formatting facts and the client's `env`, `session` and `host` values. /// /// It belongs to the runtime's owner, outlives every screen mounted on it, and is cleared by `signOut` and on a @@ -1171,7 +1171,7 @@ class Session { std::chrono::steady_clock::now().time_since_epoch()); } - /// @brief Registers a mounted query, for `refreshOn` and invalidation. + /// @brief Registers a mounted query, which a reconnect and a backend switch refetch. /// @param query The query. void attachQuery(detail::QueryRuntime& query) { _mounted.push_back(&query); } @@ -1179,14 +1179,14 @@ class Session { /// @param query The query. void forgetQuery(detail::QueryRuntime& query) { std::erase(_mounted, &query); } - /// @brief A call of @p actionId on a model of @p typeId succeeded: applies the action's effects to the cache, and - /// refetches every mounted query of that model type whose `refreshOn` names it (spec 5 §5). + /// @brief A call of @p actionId on a model of @p typeId succeeded: applies the action's effects to the cache + /// (spec 8 §9). /// /// `updates R` merges the reply's rows of `R` into each entry holding `R`, matched by the record's key field at /// the entry's paths, with no fetch; an entry whose fetch is in flight also fetches once more, since that fetch /// may have read before the write. `changes R` marks each entry holding `R` stale. `changesInstance` marks each /// entry read through the same instance. A marked entry refetches through a mounted query that uses it. A - /// discarded reply only marks: its `updates` act as `changes`, and `refreshOn` is not run. + /// discarded reply only marks: its `updates` act as `changes`. /// @param typeId The model type. /// @param actionId The action. /// @param reply The reply's value. @@ -1953,7 +1953,7 @@ class QueryRuntime { /// @brief One run of a mutation, as a command sees it: whether it succeeded, and the reply. using RunDone = std::function; -/// @brief A mutation at run time (spec 5 §5): concurrency, patch, optimistic writes, invalidation. +/// @brief A mutation at run time (spec 5 §5): concurrency, optimistic writes and the reply's effects. class MutationRuntime { public: /// @brief A mutation of @p scope. @@ -3093,16 +3093,6 @@ inline void Session::markInstanceChange(wire::InstanceTopic const& instance, det inline void Session::actionSucceeded(std::string const& typeId, std::string const& actionId, Value const& reply, ReplyUse use, std::string const& instanceToken) { _runtime->batch([&] { applyReplyEffects(typeId, actionId, reply, use, instanceToken); }); - if (use == ReplyUse::Discarded) { - return; - } - std::vector const mounted = _mounted; - for (detail::QueryRuntime* const query : mounted) { - auto const& names = query->decl().refreshOn; - if (query->typeId() == typeId && std::ranges::find(names, actionId) != names.end()) { - query->invalidate(); - } - } } inline void Session::detachQuery(detail::QueryRuntime& query) { query.detach(); } @@ -3687,33 +3677,6 @@ inline void MutationRuntime::settle(std::shared_ptr const& run, std::option _result.set(*reply); _error.set(Value{}); _errorKind.set(Value{}); - if (_decl->patch) { - Bindings bindings = *run->bindings; - bindings.push_back(Binding{.name = "result", .read = [value = *reply] { return value; }}); - NameRef const* const target = _scope->find(_decl->patch->query); - if (target != nullptr && target->category == NameCategory::Query) { - auto entry = core.runtime().untracked([&] { return target->query->entry(); }); - Value const key = - core.runtime().untracked([&] { return evalIn(*_scope, &bindings, _decl->patch->key); }); - Value const with = - core.runtime().untracked([&] { return evalIn(*_scope, &bindings, _decl->patch->with); }); - if (entry) { - auto patched = patchRows(entry->confirmed, _decl->patch->at, key, with); - if (patched) { - session.confirm(*entry, *patched); - } else if (patched.error() == PatchMiss::NoList && entry->hasValue.peek()) { - core.report(pointerAppend(_decl->path, "patch"), - patchMissReason(_decl->patch->query, _decl->patch->at)); - } - } - } - } - for (std::string const& name : _decl->invalidates) { - NameRef const* const target = _scope->find(name); - if (target != nullptr && target->category == NameCategory::Query) { - target->query->invalidate(); - } - } session.actionSucceeded(_typeId, _decl->action, *reply, ReplyUse::Applied, token); core.runList(_decl->onSuccess, commandCtx(*run->bindings), {}); if (!secret.empty()) { @@ -5940,7 +5903,7 @@ inline forms::Submitter ScreenCore::aliasCalls(ModelAlias* alias) { } // The calls of a form, collection or wizard: a success other than of @p read, the collection's own query, tells the -// session, which refetches the queries whose `refreshOn` names the action, as a mutation's success does. +// session, which applies the action's effects, as a mutation's success does. inline forms::Submitter ScreenCore::aliasSubmits(ModelAlias* alias, std::string read) { return [this, alias, call = aliasCalls(alias), read = std::move(read)]( std::string_view actionId, std::string body) -> async::Completion { diff --git a/include/morph/ui/service.hpp b/include/morph/ui/service.hpp index 67ad15d23..e3e21dbbe 100644 --- a/include/morph/ui/service.hpp +++ b/include/morph/ui/service.hpp @@ -95,8 +95,7 @@ struct ServedScreen { struct ServedBundle { /// @brief The application. ApplicationInfo application; - /// @brief The vocabularies the bundle needs, sorted: `expr/1`, the vocabulary each document declares (`ui/1` when - /// there is none), and those added with `requireVocabulary`. + /// @brief The vocabularies the bundle needs, sorted; always `expr/1` and `ui/1`. std::vector vocabularies; /// @brief The manifest's text. std::string manifest; @@ -199,11 +198,6 @@ class DocumentActions { if (top) { _top.push_back(ref); } - // The actions a query refreshes on are its model type's, named by the document like any other: the - // client's load-time validation needs their facts. - for (std::string const& action : query.refreshOn) { - _all.insert(ActionRef{.modelType = ref.modelType, .action = action}); - } _all.insert(std::move(ref)); } for (MutationDecl const& mutation : scope.mutations) { @@ -327,7 +321,6 @@ class UiRegistry { return refuse(entry.id, "the app document is added with addFile"); } logWarnings(*loaded); - _documentVocabularies.insert(loaded->vocab); detail::DocumentActions const named{*loaded}; Screen screen{.served = ServedScreen{.entry = std::move(entry), .hash = contentHash(text), @@ -402,8 +395,8 @@ class UiRegistry { return {}; } - /// @brief Adds a vocabulary the bundle needs beyond `expr/1` and its documents' own, such as `forms/1` or - /// `files/1`. A client that does not speak it is refused at `ui-hello`. + /// @brief Adds a vocabulary the bundle needs beyond `ui/1` and `expr/1`, such as `forms/1` or `files/1`. A + /// client that does not speak it is refused at `ui-hello`. /// @param vocabulary The vocabulary. void requireVocabulary(std::string vocabulary) { _vocabularies.insert(std::move(vocabulary)); } @@ -419,8 +412,7 @@ class UiRegistry { } auto bundle = std::make_shared(); bundle->application = _application; - std::set> const needed = vocabularies(); - bundle->vocabularies.assign(needed.begin(), needed.end()); + bundle->vocabularies.assign(_vocabularies.begin(), _vocabularies.end()); bundle->appActions = _appActions; bundle->appModelTypes = _appModelTypes; serveFiles(*bundle); @@ -429,19 +421,6 @@ class UiRegistry { } private: - /// @brief The vocabularies the bundle needs. A client is served the documents as they are written, so it must - /// speak each document's vocabulary: a bundle whose documents are all `ui/1` serves a client that speaks - /// only `ui/1`. - /// @return `expr/1`, the documents' vocabularies (`ui/1` when there is no document), and the required ones. - [[nodiscard]] std::set> vocabularies() const { - std::set> needed = _vocabularies; - if (_documentVocabularies.empty()) { - needed.emplace(kVocabulary); - } - needed.insert(_documentVocabularies.begin(), _documentVocabularies.end()); - return needed; - } - /// @brief Checks every theme index: each combination of its variants, densities and surfaces over `base` /// resolves from the bundle's token files (`checkTheme`). /// @return The refusal, naming the index and the combination, or nothing. @@ -522,7 +501,6 @@ class UiRegistry { return refuse(file.id, "the app document's id is `app`"); } logWarnings(*loaded); - _documentVocabularies.insert(loaded->vocab); detail::DocumentActions const named{*loaded}; _appActions = named.actions(); _appModelTypes = named.modelTypes(); @@ -587,9 +565,8 @@ class UiRegistry { .path = {}, .reason = "the bundle holds more than " + std::to_string(kMaxBundleFiles) + " files or entries"}; } - std::set> const needed = vocabularies(); - std::vector const names{needed.begin(), needed.end()}; - if (auto reason = detail::checkNames(names, kMaxListItems, "the vocabularies", "a vocabulary")) { + std::vector const vocabularies{_vocabularies.begin(), _vocabularies.end()}; + if (auto reason = detail::checkNames(vocabularies, kMaxListItems, "the vocabularies", "a vocabulary")) { return Refusal{.document = {}, .path = {}, .reason = std::move(*reason)}; } return std::nullopt; @@ -682,8 +659,7 @@ class UiRegistry { std::map> _tokenFiles; std::vector _appActions; std::vector _appModelTypes; - std::set> _vocabularies{std::string{kExprVocabulary}}; - std::set> _documentVocabularies; + std::set> _vocabularies{std::string{kExprVocabulary}, std::string{kVocabulary}}; }; /// @brief Whether an authoriser is asked about one of the UI service's reads: `ui`'s `hello`, `catalog` or `fetch`. diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index d67f120ec..71fd1d985 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -31,7 +31,7 @@ add_executable(morph_tests test_push_cache.cpp test_push_agent.cpp test_push_documents.cpp - test_push_ui2.cpp + test_push_refresh_keys.cpp push_records_late.cpp test_executor_extra.cpp test_strand.cpp diff --git a/tests/test_push_cache.cpp b/tests/test_push_cache.cpp index 8801208be..bde75f951 100644 --- a/tests/test_push_cache.cpp +++ b/tests/test_push_cache.cpp @@ -20,7 +20,7 @@ namespace { constexpr char const* kList = R"({ "vocab": "ui/1", "id": "list", "models": { "sample": { "type": "Lims_SampleModel" } }, - "queries": { "rows": { "model": "sample", "action": "Lims_ListResults" } }, + "queries": { "rows": { "model": "sample", "action": "Lims_GetLimits" } }, "view": { "kind": "text", "testId": "count", "text": { "fn": "format", "args": [ "{0}", { "query": "rows" } ] } } })"; } // namespace @@ -28,34 +28,34 @@ constexpr char const* kList = R"({ TEST_CASE("stale_mark_during_fetch_fetches_once_more", "[push][cache]") { Harness h; auto screen = h.mount(kList); - REQUIRE(h.dispatcher.count("Lims_ListResults") == 1); + REQUIRE(h.dispatcher.count("Lims_GetLimits") == 1); // The mark lands while the first fetch is in flight, so that fetch may have read before the change. h.session.backendChanged("primary"); h.settle(); - REQUIRE(h.dispatcher.count("Lims_ListResults") == 1); + REQUIRE(h.dispatcher.count("Lims_GetLimits") == 1); h.reply(0, "[1]"); - REQUIRE(h.dispatcher.count("Lims_ListResults") == 2); + REQUIRE(h.dispatcher.count("Lims_GetLimits") == 2); h.reply(1, "[1,2]"); - CHECK(h.dispatcher.count("Lims_ListResults") == 2); + CHECK(h.dispatcher.count("Lims_GetLimits") == 2); } TEST_CASE("stale_mark_on_unmounted_fetching_entry_waits_for_a_mount", "[push][cache]") { Harness h; auto screen = h.mount(kList); - REQUIRE(h.dispatcher.count("Lims_ListResults") == 1); + REQUIRE(h.dispatcher.count("Lims_GetLimits") == 1); h.session.backendChanged("primary"); screen.reset(); h.settle(); // No query uses the entry when its fetch settles, so it stays stale and is not refetched. h.reply(0, "[1]"); - CHECK(h.dispatcher.count("Lims_ListResults") == 1); + CHECK(h.dispatcher.count("Lims_GetLimits") == 1); auto again = h.mount(kList); - CHECK(h.dispatcher.count("Lims_ListResults") == 2); + CHECK(h.dispatcher.count("Lims_GetLimits") == 2); } namespace { diff --git a/tests/test_push_documents.cpp b/tests/test_push_documents.cpp index 70dfcdbe0..b49fd23a3 100644 --- a/tests/test_push_documents.cpp +++ b/tests/test_push_documents.cpp @@ -1,8 +1,7 @@ // SPDX-License-Identifier: Apache-2.0 -// Documents checked against action effects (spec 8 §10): a query must read, and three warnings point at refresh -// declarations that effects make redundant, at mutations nothing refreshes after, and at watches that may feed -// another client's changes back. +// Documents checked against action effects (spec 8 §10): a query must read, and two warnings point at mutations +// nothing refreshes after and at watches that may feed another client's changes back. #include #include @@ -111,69 +110,6 @@ TEST_CASE("query_of_action_with_effects_refused", "[push][documents]") { CHECK(ui::loadDocumentText(document(R"({"rows":{"model":"m","action":"T_Edit"}})", "{}")).has_value()); } -TEST_CASE("redundant_invalidates_warns", "[push][documents]") { - auto const covered = warningsOf(document(R"({"rows":{"model":"m","action":"T_List"}, - "other":{"model":"m","action":"T_ListOther"}})", - R"({"rename":{"model":"m","action":"T_Rename", - "invalidates":["other","rows"]}})"), - LoadWarningKind::Redundant); - REQUIRE(covered.size() == 1); - CHECK(covered.front().path == "/mutations/rename/invalidates/1"); - CHECK(covered.front().text.contains("`T_Rename` changes `T_Row`")); - - // `updates` covers the query too: a reply patches the rows it holds. - CHECK(warningsOf(document(R"({"rows":{"model":"m","action":"T_List"}})", - R"({"edit":{"model":"m","action":"T_Edit","invalidates":["rows"]}})"), - LoadWarningKind::Redundant) - .size() == 1); - - // `changesInstance` covers a query through the same alias, and only that. - auto const instance = warningsOf(document(R"({"rows":{"model":"m","action":"T_List"}})", - R"({"reset":{"model":"m","action":"T_Reset","invalidates":["rows"]}})"), - LoadWarningKind::Redundant); - REQUIRE(instance.size() == 1); - CHECK(instance.front().path == "/mutations/reset/invalidates/0"); - auto const otherAlias = warningsOf( - R"({"vocab":"ui/1","id":"t","models":{"m":{"type":"T_Model"},"n":{"type":"T_Model"}}, - "queries":{"rows":{"model":"n","action":"T_List"}}, - "mutations":{"reset":{"model":"m","action":"T_Reset","invalidates":["rows"]}}, - "view":{"kind":"text","text":"x"}})", - LoadWarningKind::Redundant); - CHECK(otherAlias.empty()); -} - -TEST_CASE("redundant_patch_warns", "[push][documents]") { - auto const patched = warningsOf(document(R"({"rows":{"model":"m","action":"T_List"}})", - R"({"edit":{"model":"m","action":"T_Edit", - "patch":{"query":"rows","key":{"ref":"result.id"},"with":{"ref":"result"}}}})"), - LoadWarningKind::Redundant); - REQUIRE(patched.size() == 1); - CHECK(patched.front().path == "/mutations/edit/patch"); - - // A patch of a query that holds no record the reply updates is the only thing that updates it. - CHECK(warningsOf(document(R"({"other":{"model":"m","action":"T_ListOther"}})", - R"({"edit":{"model":"m","action":"T_Edit", - "patch":{"query":"other","key":{"ref":"result.id"},"with":{"ref":"result"}}}})"), - LoadWarningKind::Redundant) - .empty()); -} - -TEST_CASE("redundant_refresh_on_warns", "[push][documents]") { - auto const refreshed = - warningsOf(document(R"({"rows":{"model":"m","action":"T_List","refreshOn":["T_Write","T_Rename"]}, - "other":{"model":"m","action":"T_ListOther","refreshOn":["T_Rename"]}})", - "{}"), - LoadWarningKind::Redundant); - REQUIRE(refreshed.size() == 1); - CHECK(refreshed.front().path == "/queries/rows/refreshOn/1"); - CHECK(refreshed.front().text.contains("`T_Rename` changes `T_Row`")); - - // `refreshOn` names an action run anywhere, through any instance, so `changesInstance` does not cover it. - CHECK(warningsOf(document(R"({"rows":{"model":"m","action":"T_List","refreshOn":["T_Reset"]}})", "{}"), - LoadWarningKind::Redundant) - .empty()); -} - TEST_CASE("uncovered_mutation_warns", "[push][documents]") { auto const uncovered = warningsOf( document(R"({"rows":{"model":"m","action":"T_List"}})", R"({"write":{"model":"m","action":"T_Write"}})"), @@ -182,20 +118,11 @@ TEST_CASE("uncovered_mutation_warns", "[push][documents]") { CHECK(uncovered.front().path == "/mutations/write"); CHECK(uncovered.front().text.contains("`T_Write`")); - // Each of these refreshes something after a success. - for (std::string const& covered : - {document(R"({"rows":{"model":"m","action":"T_List"}})", - R"({"write":{"model":"m","action":"T_Write","invalidates":["rows"]}})"), - document(R"({"rows":{"model":"m","action":"T_List"}})", - R"({"write":{"model":"m","action":"T_Write", - "patch":{"query":"rows","key":{"ref":"result.id"},"with":{"ref":"result"}}}})"), - document(R"({"rows":{"model":"m","action":"T_List","refreshOn":["T_Write"]}})", - R"({"write":{"model":"m","action":"T_Write"}})"), - document(R"({"rows":{"model":"m","action":"T_List"}})", - R"({"write":{"model":"m","action":"T_Write","onSuccess":[{"refetch":"rows"}]}})")}) { - CAPTURE(covered); - CHECK(warningsOf(covered, LoadWarningKind::Uncovered).empty()); - } + // A `refetch` after a success is a refresh the document declares itself. + CHECK(warningsOf(document(R"({"rows":{"model":"m","action":"T_List"}})", + R"({"write":{"model":"m","action":"T_Write","onSuccess":[{"refetch":"rows"}]}})"), + LoadWarningKind::Uncovered) + .empty()); // Without the catalog's actions nothing is known about effects. auto const blind = ui::loadDocumentText(document("{}", R"({"write":{"model":"m","action":"T_Write"}})")); REQUIRE(blind.has_value()); diff --git a/tests/test_push_ui2.cpp b/tests/test_push_refresh_keys.cpp similarity index 51% rename from tests/test_push_ui2.cpp rename to tests/test_push_refresh_keys.cpp index dd9f4779c..60f21d74d 100644 --- a/tests/test_push_ui2.cpp +++ b/tests/test_push_refresh_keys.cpp @@ -1,22 +1,15 @@ // SPDX-License-Identifier: Apache-2.0 -// The `ui/2` vocabulary (spec 8 §10): actions declare their effects, so a `ui/2` document has no `invalidates`, -// `patch` or `refreshOn`. It keeps `optimistic`, `refreshEvery` and the `refetch` command. A `ui/1` document still -// loads with its refresh keys, the authoring layer emits `ui/2`, and a bundle needs the vocabularies its documents -// declare. +// The refresh keys (spec 8 §10): actions declare their effects, so a document has no `invalidates`, `patch` or +// `refreshOn`. It keeps `optimistic`, `refreshEvery` and the `refetch` command. #include -#include #include #include #include -#include #include #include #include -#include - -#include "ui_service_support.hpp" namespace ui = morph::ui; namespace author = morph::ui::author; @@ -61,44 +54,44 @@ constexpr std::string_view kRows = R"({"rows":{"model":"m","action":"T_List"}})" } // namespace -TEST_CASE("ui2_refuses_invalidates", "[push][ui2]") { +TEST_CASE("refuses_invalidates", "[push][refresh]") { auto const loaded = - load(document(ui::kVocabulary2, kRows, R"({"edit":{"model":"m","action":"T_Edit","invalidates":["rows"]}})")); + load(document(ui::kVocabulary, kRows, R"({"edit":{"model":"m","action":"T_Edit","invalidates":["rows"]}})")); REQUIRE_FALSE(loaded.has_value()); CHECK(loaded.error().path == "/mutations/edit/invalidates"); - CHECK(loaded.error().reason.contains("ui/2")); + CHECK(loaded.error().reason.contains("does not declare")); CHECK(loaded.error().reason.contains("effects")); } -TEST_CASE("ui2_refuses_refresh_on", "[push][ui2]") { +TEST_CASE("refuses_refresh_on", "[push][refresh]") { auto const loaded = - load(document(ui::kVocabulary2, R"({"rows":{"model":"m","action":"T_List","refreshOn":["T_Edit"]}})", "{}")); + load(document(ui::kVocabulary, R"({"rows":{"model":"m","action":"T_List","refreshOn":["T_Edit"]}})", "{}")); REQUIRE_FALSE(loaded.has_value()); CHECK(loaded.error().path == "/queries/rows/refreshOn"); - CHECK(loaded.error().reason.contains("ui/2")); + CHECK(loaded.error().reason.contains("does not declare")); CHECK(loaded.error().reason.contains("effects")); } -TEST_CASE("ui2_refuses_patch", "[push][ui2]") { +TEST_CASE("refuses_patch", "[push][refresh]") { auto const loaded = load(document( - ui::kVocabulary2, kRows, + ui::kVocabulary, kRows, R"({"edit":{"model":"m","action":"T_Edit","patch":{"query":"rows","key":{"ref":"result.id"},"with":{"ref":"result"}}}})")); REQUIRE_FALSE(loaded.has_value()); CHECK(loaded.error().path == "/mutations/edit/patch"); - CHECK(loaded.error().reason.contains("ui/2")); + CHECK(loaded.error().reason.contains("does not declare")); CHECK(loaded.error().reason.contains("effects")); } -TEST_CASE("ui2_keeps_optimistic", "[push][ui2]") { +TEST_CASE("keeps_optimistic_refresh_every_and_refetch", "[push][refresh]") { auto const loaded = - load(document(ui::kVocabulary2, R"({"rows":{"model":"m","action":"T_List","refreshEvery":1000}})", + load(document(ui::kVocabulary, R"({"rows":{"model":"m","action":"T_List","refreshEvery":1000}})", R"({"edit":{"model":"m","action":"T_Edit", "optimistic":{"query":"rows","key":{"ref":"body.id"},"with":{"ref":"body"}}, "onSuccess":[{"refetch":"rows"}]}})")); if (!loaded) { FAIL(loaded.error().message()); } else { - CHECK(loaded->vocab == "ui/2"); + CHECK(loaded->vocab == "ui/1"); REQUIRE(loaded->scope.queries.size() == 1); CHECK(loaded->scope.queries.front().refreshEvery == 1000); REQUIRE(loaded->scope.mutations.size() == 1); @@ -112,67 +105,27 @@ TEST_CASE("ui2_keeps_optimistic", "[push][ui2]") { } // An optimistic write still names a query of the document. auto const unknown = load(document( - ui::kVocabulary2, kRows, + ui::kVocabulary, kRows, R"({"edit":{"model":"m","action":"T_Edit","optimistic":{"query":"none","key":1,"with":{"obj":{}}}}})")); REQUIRE_FALSE(unknown.has_value()); CHECK(unknown.error().path == "/mutations/edit/optimistic/query"); } -TEST_CASE("ui1_document_still_loads_with_refresh_on", "[push][ui2]") { - auto const loaded = - load(document(ui::kVocabulary, R"({"rows":{"model":"m","action":"T_List","refreshOn":["T_Edit"]}})", - R"({"edit":{"model":"m","action":"T_Edit","invalidates":["rows"], - "patch":{"query":"rows","key":{"ref":"result.id"},"with":{"ref":"result"}}}})")); - if (!loaded) { - FAIL(loaded.error().message()); - } else { - CHECK(loaded->vocab == "ui/1"); - REQUIRE(loaded->scope.queries.size() == 1); - CHECK(loaded->scope.queries.front().refreshOn == std::vector{"T_Edit"}); - REQUIRE(loaded->scope.mutations.size() == 1); - CHECK(loaded->scope.mutations.front().invalidates == std::vector{"rows"}); - CHECK(loaded->scope.mutations.front().patch.has_value()); - // `T_Edit` declares no effects, so the document's own refreshes are what refreshes after it. - CHECK(loaded->warnings.empty()); - } - auto const other = load(document("ui/3", kRows, "{}")); +// Mutation: accept another vocabulary alongside ui/1 — a `ui/2` document loads. +TEST_CASE("one_vocabulary_is_read", "[push][refresh]") { + auto const other = load(document("ui/2", kRows, "{}")); REQUIRE_FALSE(other.has_value()); CHECK(other.error().path == "/vocab"); } -TEST_CASE("author_emits_ui2", "[push][ui2]") { - CHECK(ui::kVocabulary == "ui/1"); - CHECK(ui::kVocabulary2 == "ui/2"); - auto const emitted = author::screen("author.ui2", [](author::Scope&) { return author::spacer(); }).write(); +TEST_CASE("author_emits_ui1", "[push][refresh]") { + auto const emitted = author::screen("author.refresh", [](author::Scope&) { return author::spacer(); }).write(); if (!emitted) { FAIL(emitted.error().message()); } else { - CHECK(emitted->text.starts_with(R"({"vocab":"ui/2","id":"author.ui2")")); + CHECK(emitted->text.starts_with(R"({"vocab":"ui/1","id":"author.refresh")")); auto const reloaded = ui::loadDocumentText(emitted->text); REQUIRE(reloaded.has_value()); - CHECK(reloaded->vocab == "ui/2"); + CHECK(reloaded->vocab == "ui/1"); } } - -TEST_CASE("bundle_needs_the_vocabularies_its_documents_declare", "[push][ui2]") { - auto const screenText = [](std::string_view vocab, std::string_view id) { - return R"({"vocab":")" + std::string{vocab} + R"(","id":")" + std::string{id} + - R"(","view":{"kind":"text","text":"x"}})"; - }; - auto const built = [&](std::vector const& vocabs) { - ui::UiRegistry registry{ui::ApplicationInfo{.id = "app", .version = "1"}}; - for (std::size_t index = 0; index < vocabs.size(); ++index) { - std::string const id = "s" + std::to_string(index); - REQUIRE(registry.addScreen(uiservice::entry(id), screenText(vocabs.at(index), id)).has_value()); - } - auto bundle = registry.build(); - REQUIRE(bundle.has_value()); - return (*bundle)->vocabularies; - }; - // A bundle of `ui/1` documents is served to a client that speaks only `ui/1`. - CHECK(built({"ui/1"}) == std::vector{"expr/1", "ui/1"}); - CHECK(built({"ui/2"}) == std::vector{"expr/1", "ui/2"}); - CHECK(built({"ui/1", "ui/2"}) == std::vector{"expr/1", "ui/1", "ui/2"}); - // With no document, the bundle needs the first vocabulary. - CHECK(built({}) == std::vector{"expr/1", "ui/1"}); -} diff --git a/tests/test_ui_author.cpp b/tests/test_ui_author.cpp index 8868c173f..9178ec49e 100644 --- a/tests/test_ui_author.cpp +++ b/tests/test_ui_author.cpp @@ -931,7 +931,7 @@ TEST_CASE("the registry keeps one screen per id and answers the catalog", "[ui][ CHECK(duplicate.error().path == "/id"); REQUIRE(registry.add(commandsScreen()).has_value()); CHECK(registry.size() == 2); - CHECK(registry.find("lims.commands")->text.starts_with(R"({"vocab":"ui/2","id":"lims.commands")")); + CHECK(registry.find("lims.commands")->text.starts_with(R"({"vocab":"ui/1","id":"lims.commands")")); morph::ui::Catalog const catalog = registry.catalog(); auto const sample = catalog.model("Lims_SampleModel"); diff --git a/tests/test_ui_document.cpp b/tests/test_ui_document.cpp index 9dfd9145c..808165509 100644 --- a/tests/test_ui_document.cpp +++ b/tests/test_ui_document.cpp @@ -170,7 +170,6 @@ TEST_CASE("ui document: the model holds what the document declares", "[ui][docum auto const& results = doc.scope.queries.at(0); CHECK(results.name == "results"); CHECK(results.staleTime == 1000); - CHECK(results.refreshOn == std::vector{"Lims_CaptureConcentration"}); CHECK_FALSE(results.evaluate); CHECK(doc.scope.queries.at(2).evaluate); @@ -179,7 +178,6 @@ TEST_CASE("ui document: the model holds what the document declares", "[ui][docum CHECK(capture.concurrency == Concurrency::Serial); CHECK(capture.serialKey != nullptr); CHECK(capture.offlineQueue); - CHECK(capture.patch.has_value()); CHECK(capture.optimistic.has_value()); CHECK(capture.onSuccess.size() == 1); CHECK(capture.onSuccess.front().kind == CommandKind::Set); diff --git a/tests/test_ui_interpreter_calculation.cpp b/tests/test_ui_interpreter_calculation.cpp index 0dcac334e..b08889055 100644 --- a/tests/test_ui_interpreter_calculation.cpp +++ b/tests/test_ui_interpreter_calculation.cpp @@ -28,8 +28,7 @@ constexpr char const* kCalc = R"({ "body": { "obj": { "resultId": { "row": "id" }, "inputs": { "ref": "draft" } } } } }, "mutations": { "save": { "model": "sample", "action": "Lims_CaptureConcentration", "concurrency": "serial", "serialKey": { "row": "id" }, - "body": { "obj": { "resultId": { "row": "id" }, "inputs": { "ref": "draft" } } }, - "patch": { "query": "results", "key": { "ref": "result.id" }, "with": { "ref": "result" } } } } }, + "body": { "obj": { "resultId": { "row": "id" }, "inputs": { "ref": "draft" } } } } } }, "columns": [ { "label": "Mass", "cell": { "kind": "textInput", "id": "mass", "testId": "mass", "commitOn": "change", "value": { "ref": "draft.mass" }, diff --git a/tests/test_ui_interpreter_control.cpp b/tests/test_ui_interpreter_control.cpp index 945bb4376..ccdce481c 100644 --- a/tests/test_ui_interpreter_control.cpp +++ b/tests/test_ui_interpreter_control.cpp @@ -1,8 +1,7 @@ // SPDX-License-Identifier: Apache-2.0 // The control corpus of spec 5 §14: queries (latest wins, debounce, timed refresh, the cache), mutations (the four -// concurrency modes, patch, optimistic writes, one-batch invalidation) and boundaries, mounted on RecordingBackend -// with a scripted dispatcher. +// concurrency modes and optimistic writes) and boundaries, mounted on RecordingBackend with a scripted dispatcher. #include #include @@ -150,7 +149,6 @@ constexpr char const* kMutations = R"({ "mutations": { "m": { "model": "sample", "action": "Lims_Save", "concurrency": "MODE", SERIAL "body": { "obj": { "n": { "ref": "n" } } }, - "invalidates": [ "rows", "other" ], "onSuccess": [ { "set": "n", "value": { "fn": "add", "args": [ { "ref": "n" }, 1 ] } } ], "onError": [ { "notify": { "error": "m" }, "role": "error" } ] } }, "view": { "kind": "column", "children": [ @@ -303,25 +301,10 @@ TEST_CASE("interpreter control: a failure sets the error and runs onError", "[ui CHECK(h.host.log.back() == "notify error it failed"); CHECK(readJson(*screen, R"({"errorKind": "m"})") == R"("denied")"); CHECK(h.host.stale == 1); - // A failure invalidates nothing. + // A failure fetches nothing. CHECK(h.dispatcher.calls.size() == before); } -TEST_CASE("interpreter control: invalidation refetches every named query in the success batch", "[ui][interpreter]") { - Harness h; - auto screen = h.mount(mutations("exclusive")); - loadQueries(h); - h.backend.click(h.byTestId("Button", "go")); - h.settle(); - h.dispatcher.reply(h.dispatcher.last("Lims_Save"), "1"); - // The delivery is one task of the owner: the result, the cleared pending flag and both refetches. - REQUIRE(h.owner.runOne()); - CHECK(h.dispatcher.count("Lims_ListResults") == 2); - CHECK(h.dispatcher.count("Lims_GetLimits") == 2); - h.settle(); - CHECK(h.text("result") == "1"); -} - namespace { constexpr char const* kRows = R"({ @@ -329,8 +312,6 @@ constexpr char const* kRows = R"({ "models": { "sample": { "type": "Lims_SampleModel" } }, "queries": { "results": { "model": "sample", "action": "Lims_ListResults" } }, "mutations": { - "capture": { "model": "sample", "action": "Lims_CaptureConcentration", "concurrency": "parallel", - "patch": { "query": "results", "key": { "ref": "result.id" }, "with": { "ref": "result" } } }, "guess": { "model": "sample", "action": "Lims_Save", "concurrency": "parallel", "optimistic": { "query": "results", "key": { "ref": "body.id" }, "with": { "obj": { "mass": { "ref": "body.mass" } } } }, @@ -338,7 +319,6 @@ constexpr char const* kRows = R"({ "let": { "row2": { "fn": "first", "args": [ { "filter": { "query": "results" }, "as": "r", "where": { "fn": "eq", "args": [ { "ref": "r.id" }, 2 ] } } ] } }, "view": { "kind": "column", "children": [ - { "kind": "textInput", "testId": "capture", "onSubmit": { "run": "capture", "body": { "obj": { "id": 2, "mass": { "ref": "event" } } } } }, { "kind": "textInput", "testId": "guess", "onSubmit": { "run": "guess", "body": { "obj": { "id": 2, "mass": { "ref": "event" } } } } }, { "kind": "forEach", "rows": { "query": "results" }, "key": { "row": "id" }, "children": [ { "kind": "text", "text": { "fn": "format", "args": [ "{0}:{1}", { "row": "id" }, { "row": "mass" } ] } } ] } ] } })"; @@ -353,18 +333,6 @@ std::vector rowTexts(Harness& h) { } // namespace -TEST_CASE("interpreter control: patch replaces the replied row without refetching", "[ui][interpreter]") { - Harness h; - auto screen = h.mount(kRows); - h.reply(0, R"([{"id":1,"mass":10},{"id":2,"mass":20}])"); - CHECK(rowTexts(h) == std::vector{"1:10", "2:20"}); - h.backend.submit(h.byTestId("TextInput", "capture"), "25"); - h.settle(); - h.reply(h.dispatcher.last("Lims_CaptureConcentration"), R"({"id":2,"mass":25,"concentration":5})"); - CHECK(rowTexts(h) == std::vector{"1:10", "2:25"}); - CHECK(h.dispatcher.count("Lims_ListResults") == 1); -} - TEST_CASE("interpreter control: an optimistic write shows before the reply and rolls back on failure", "[ui][interpreter]") { Harness h; @@ -414,20 +382,16 @@ TEST_CASE("interpreter control: an optimistic write shows before the reply and r namespace { // The rows of `results` live in the record member its `at` names, as a list action that answers -// `{"results": [...]}` gives them. AT is replaced with the `at` member of both patches, or with nothing. +// `{"results": [...]}` gives them. AT is replaced with the `at` member of the optimistic write, or with nothing. constexpr char const* kRecordRows = R"({ "vocab": "ui/1", "id": "recordrows", "models": { "sample": { "type": "Lims_SampleModel" } }, "queries": { "results": { "model": "sample", "action": "Lims_ListResults" } }, "mutations": { - "capture": { "model": "sample", "action": "Lims_CaptureConcentration", "concurrency": "parallel", - "patch": { "query": "results", AT "key": { "ref": "result.id" }, "with": { "ref": "result" } } }, "guess": { "model": "sample", "action": "Lims_Save", "concurrency": "parallel", "optimistic": { "query": "results", AT "key": { "ref": "body.id" }, "with": { "obj": { "mass": { "ref": "body.mass" } } } } } }, - "view": { "kind": "column", "children": [ - { "kind": "textInput", "testId": "capture", "onSubmit": { "run": "capture", "body": { "obj": { "id": 2, "mass": { "ref": "event" } } } } }, - { "kind": "textInput", "testId": "guess", "onSubmit": { "run": "guess", "body": { "obj": { "id": 2, "mass": { "ref": "event" } } } } } ] } })"; + "view": { "kind": "textInput", "testId": "guess", "onSubmit": { "run": "guess", "body": { "obj": { "id": 2, "mass": { "ref": "event" } } } } } })"; std::string recordRows(std::string_view at) { std::string text = kRecordRows; @@ -442,41 +406,30 @@ constexpr char const* kRecordReply = R"({"results":[{"id":1,"mass":10},{"id":2," } // namespace -TEST_CASE("interpreter control: a patch with `at` replaces a row of the list inside the query's record", +TEST_CASE("interpreter control: an optimistic write with `at` writes a row of the list inside the query's record", "[ui][interpreter]") { Harness h; auto const screen = h.mount(recordRows(R"("at": "results",)")); h.reply(0, kRecordReply); - - SECTION("patch") { - h.backend.submit(h.byTestId("TextInput", "capture"), "25"); - h.settle(); - h.reply(h.dispatcher.last("Lims_CaptureConcentration"), R"({"id":2,"mass":25})"); - CHECK(readJson(*screen, R"({"ref": "results"})") == R"({"results":[{"id":1,"mass":10},{"id":2,"mass":25}]})"); - CHECK(h.dispatcher.count("Lims_ListResults") == 1); - } - SECTION("optimistic, then rolled back on failure") { - h.backend.submit(h.byTestId("TextInput", "guess"), "30"); - h.settle(); - CHECK(readJson(*screen, R"({"ref": "results"})") == - R"({"results":[{"id":1,"mass":10},{"id":2,"mass":"30"}]})"); - h.fail(h.dispatcher.last("Lims_Save")); - CHECK(readJson(*screen, R"({"ref": "results"})") == kRecordReply); - } + h.backend.submit(h.byTestId("TextInput", "guess"), "30"); + h.settle(); + CHECK(readJson(*screen, R"({"ref": "results"})") == R"({"results":[{"id":1,"mass":10},{"id":2,"mass":"30"}]})"); + h.fail(h.dispatcher.last("Lims_Save")); + CHECK(readJson(*screen, R"({"ref": "results"})") == kRecordReply); CHECK(h.diagnostics.empty()); } -TEST_CASE("interpreter control: a patch that finds no list in the query's value is reported", "[ui][interpreter]") { +TEST_CASE("interpreter control: an optimistic write that finds no list in the query's value is reported", + "[ui][interpreter]") { Harness h; SECTION("a record value and no `at`") { auto const screen = h.mount(recordRows("")); h.reply(0, kRecordReply); - h.backend.submit(h.byTestId("TextInput", "capture"), "25"); + h.backend.submit(h.byTestId("TextInput", "guess"), "30"); h.settle(); - h.reply(h.dispatcher.last("Lims_CaptureConcentration"), R"({"id":2,"mass":25})"); CHECK(readJson(*screen, R"({"ref": "results"})") == kRecordReply); REQUIRE(h.diagnostics.size() == 1); - CHECK(h.diagnostics.front().path == "/mutations/capture/patch"); + CHECK(h.diagnostics.front().path == "/mutations/guess/optimistic"); CHECK(h.diagnostics.front().reason.contains("not a list")); } SECTION("an `at` the record does not have") { @@ -523,38 +476,6 @@ TEST_CASE("interpreter control: two queries with an equal key share one request" } } -TEST_CASE("interpreter control: refreshOn refetches a query of the same model type in another screen", - "[ui][interpreter]") { - Harness h; - auto lists = h.mount(R"({ - "vocab": "ui/1", "id": "list", - "models": { "sample": { "type": "Lims_SampleModel" } }, - "queries": { "results": { "model": "sample", "action": "Lims_ListResults", "refreshOn": [ "Lims_CaptureConcentration" ] } }, - "view": { "kind": "text", "text": { "query": "results" } } })"); - h.reply(0, "1"); - ui::testing::RecordingBackend other; - auto capture = h.interpreter.mount(ui::testing::loadOrFail(R"({ - "vocab": "ui/1", "id": "capture", - "models": { "work": { "type": "Lims_SampleModel", "instance": "private" } }, - "mutations": { "c": { "model": "work", "action": "Lims_CaptureConcentration" }, - "s": { "model": "work", "action": "Lims_Save" } }, - "view": { "kind": "column", "children": [ - { "kind": "button", "testId": "c", "label": "c", "onClick": { "run": "c" } }, - { "kind": "button", "testId": "s", "label": "s", "onClick": { "run": "s" } } ] } })", - ui::testing::corpusCatalog()), - {}, other); - REQUIRE(capture); - h.settle(); - other.click(*other.find("Button", "testId", "s")); - h.settle(); - h.reply(h.dispatcher.last("Lims_Save"), "{}"); - CHECK(h.dispatcher.count("Lims_ListResults") == 1); - other.click(*other.find("Button", "testId", "c")); - h.settle(); - h.reply(h.dispatcher.last("Lims_CaptureConcentration"), "{}"); - CHECK(h.dispatcher.count("Lims_ListResults") == 2); -} - TEST_CASE("interpreter control: a boundary shows loading, then its children, or its error and retries", "[ui][interpreter]") { Harness h; @@ -652,7 +573,8 @@ TEST_CASE("interpreter control: refetch ends a debounce wait; a reply that is no } TEST_CASE( - "interpreter control: a mutation whose reply is not a document value fails; patch of an idle query is a no-op", + "interpreter control: a mutation whose reply is not a document value fails; an optimistic write to an idle query " + "is a no-op", "[ui][interpreter]") { Harness h; auto screen = h.mount(R"({ @@ -661,7 +583,6 @@ TEST_CASE( "state": { "on": { "init": false } }, "queries": { "rows": { "model": "sample", "action": "Lims_ListResults", "when": { "ref": "on" } } }, "mutations": { "m": { "model": "sample", "action": "Lims_Save", - "patch": { "query": "rows", "key": 1, "with": { "ref": "result" } }, "optimistic": { "query": "rows", "key": 1, "with": { "ref": "body" } } } }, "view": { "kind": "button", "testId": "go", "label": "go", "onClick": { "run": "m" } } })"); h.backend.click(h.byTestId("Button", "go")); diff --git a/tests/test_ui_interpreter_forms.cpp b/tests/test_ui_interpreter_forms.cpp index 9db7b9349..e04b16f27 100644 --- a/tests/test_ui_interpreter_forms.cpp +++ b/tests/test_ui_interpreter_forms.cpp @@ -229,63 +229,69 @@ TEST_CASE("collection node: a view document listing another action is reported", namespace { -// A query of the sample model refetched by the submits of a form, a collection and a wizard on the same screen. +// A query holding `T_Row` rows beside a form, a collection or a wizard: their submits apply the action's effects, so +// `Lims_CaptureConcentration` (`updates T_Row`) patches the row it replies, and `Lims_DeleteSample` (`changes T_Row`) +// refetches the query. std::string refreshedBy(std::string_view node) { return std::string{R"({ "vocab": "ui/1", "id": "refreshed", "models": { "sample": { "type": "Lims_SampleModel" } }, - "queries": { "results": { "model": "sample", "action": "Lims_ListResults", - "refreshOn": [ "Lims_CaptureConcentration", "Lims_DeleteSample", "Lims_ListSamples", "Lims_ConfirmSample" ] } }, - "view": { "kind": "column", "children": [ { "kind": "text", "text": { "query": "results" } }, )"} + + "queries": { "rows": { "model": "sample", "action": "Push_ListRows" } }, + "view": { "kind": "column", "children": [ { "kind": "text", "text": { "fn": "format", "args": [ "{0}", { "query": "rows" } ] } }, )"} + std::string{node} + " ] } }"; } +constexpr char const* kOneRow = R"([{"code":"a","name":"x"}])"; +constexpr char const* kCaptured = R"({"code":"a","name":"captured"})"; + } // namespace -TEST_CASE("form, collection and wizard nodes: a submit that succeeds refetches the queries naming it in refreshOn", +// Mutation: leave the submits out of `actionSucceeded` — no row is patched and no delete refetches. +TEST_CASE("form, collection and wizard nodes: a submit that succeeds applies its action's effects", "[ui][interpreter][forms]") { Harness h; SECTION("a form") { auto const screen = h.mount(refreshedBy( R"({ "kind": "form", "model": "sample", "action": "Lims_CaptureConcentration", "submit": "explicit" })")); - h.reply(h.dispatcher.last("Lims_ListResults"), "[]"); + h.reply(h.dispatcher.last("Push_ListRows"), kOneRow); h.backend.edit(h.byTestId("TextInput", "field_sampleId"), "4"); h.backend.edit(h.byTestId("TextInput", "field_concentration"), "1"); h.settle(); h.backend.click(h.byTestId("Button", "form_submit")); h.settle(); h.fail(h.dispatcher.last("Lims_CaptureConcentration")); - CHECK(h.dispatcher.count("Lims_ListResults") == 1); // a failure is not a success + CHECK(readJson(*screen, R"({"query": "rows"})") == kOneRow); // a failure is not a success h.backend.click(h.byTestId("Button", "form_submit")); h.settle(); - h.reply(h.dispatcher.last("Lims_CaptureConcentration"), R"({"ok":true})"); - CHECK(h.dispatcher.count("Lims_ListResults") == 2); + h.reply(h.dispatcher.last("Lims_CaptureConcentration"), kCaptured); + CHECK(readJson(*screen, R"({"query": "rows"})") == R"([{"code":"a","name":"captured"}])"); + CHECK(h.dispatcher.count("Push_ListRows") == 1); } SECTION("a collection, whose own list query is a read, not a submit") { auto const screen = h.mount(refreshedBy(R"({ "kind": "collection", "model": "sample", "action": "Lims_ListSamples" })")); - h.reply(h.dispatcher.last("Lims_ListResults"), "[]"); + h.reply(h.dispatcher.last("Push_ListRows"), kOneRow); h.reply(h.dispatcher.last("Lims_ListSamples"), R"({"rows":[{"id":7,"name":"Water"}]})"); - CHECK(h.dispatcher.count("Lims_ListResults") == 1); + CHECK(h.dispatcher.count("Push_ListRows") == 1); h.backend.click(h.byTestId("Button", "rowAction_Lims_DeleteSample_7")); h.settle(); h.backend.click(h.byTestId("Button", "confirm_accept")); h.settle(); h.reply(h.dispatcher.last("Lims_DeleteSample"), "{}"); - CHECK(h.dispatcher.count("Lims_ListResults") == 2); + CHECK(h.dispatcher.count("Push_ListRows") == 2); // The collection reloads its list after the delete; that read refetches nothing. h.reply(h.dispatcher.last("Lims_ListSamples"), R"({"rows":[]})"); - CHECK(h.dispatcher.count("Lims_ListResults") == 2); + CHECK(h.dispatcher.count("Push_ListRows") == 2); } SECTION("a wizard's step") { auto const screen = h.mount(refreshedBy(R"({ "kind": "wizard", "model": "sample", "action": "Lims_CaptureConcentration" })")); - h.reply(h.dispatcher.last("Lims_ListResults"), "[]"); + h.reply(h.dispatcher.last("Push_ListRows"), kOneRow); h.backend.edit(h.byTestId("TextInput", "field_sampleId"), "4"); h.backend.edit(h.byTestId("TextInput", "field_concentration"), "1"); h.settle(); - h.reply(h.dispatcher.last("Lims_CaptureConcentration"), R"({"ok":true})"); - CHECK(h.dispatcher.count("Lims_ListResults") == 2); + h.reply(h.dispatcher.last("Lims_CaptureConcentration"), kCaptured); + CHECK(readJson(*screen, R"({"query": "rows"})") == R"([{"code":"a","name":"captured"}])"); } } diff --git a/tests/test_ui_service.cpp b/tests/test_ui_service.cpp index 4c062f7d7..f1e0350db 100644 --- a/tests/test_ui_service.cpp +++ b/tests/test_ui_service.cpp @@ -141,24 +141,6 @@ TEST_CASE("UiRegistry names every action and model type a document names, at any REQUIRE(bundle->actions->at("Auth_Login").credential); } -// Mutation: leave refreshOn out of the walk (Lims_Save has no fact, and the client refuses the document). -TEST_CASE("UiRegistry names the actions a query refreshes on, so the client has their facts", - "[ui-service][registry]") { - UiRegistry registry{morph::ui::ApplicationInfo{.id = "app", .version = "1"}, uiservice::facts()}; - std::string const document = - R"({"vocab":"ui/1","id":"lims.watch","models":{"sample":{"type":"Lims_Sample"}},)" - R"("queries":{"list":{"model":"sample","action":"Lims_List","refreshOn":["Lims_Save"]}},)" - R"("view":{"kind":"text","text":"Watch"}})"; - REQUIRE(registry.addScreen(entry("lims.watch"), document).has_value()); - auto const built = registry.build(); - REQUIRE(built.has_value()); - REQUIRE((*built)->actions.has_value()); - CHECK((*built)->actions->contains("Lims_Save")); - CHECK(std::ranges::any_of((*built)->allActions, [](morph::ui::ActionRef const& ref) { - return ref.action == "Lims_Save" && ref.modelType == "Lims_Sample"; - })); -} - TEST_CASE("UiRegistry without catalog lookups serves no facts", "[ui-service][registry]") { auto const built = uiservice::registry(morph::ui::Catalog{}).build(); REQUIRE(built); diff --git a/tests/ui/corpus/documents.json b/tests/ui/corpus/documents.json index c41c29705..bbce12a08 100644 --- a/tests/ui/corpus/documents.json +++ b/tests/ui/corpus/documents.json @@ -55,7 +55,7 @@ }, "queries": { "results": { "model": "sample", "action": "Lims_ListResults", "body": { "obj": { "sampleId": { "ref": "sampleId" } } }, - "when": true, "debounce": 0, "refreshOn": ["Lims_CaptureConcentration"], "refreshEvery": 0, "staleTime": 1000 }, + "when": true, "debounce": 0, "refreshEvery": 0, "staleTime": 1000 }, "limits": { "model": "catalog", "action": "Lims_GetLimits" }, "calcPreview": { "model": "sample", "evaluate": "Lims_CaptureConcentration", "body": { "obj": { "sampleId": { "ref": "sampleId" } } } }, "prefetchable": { "model": "sample", "action": "Lims_ListResults", @@ -63,8 +63,7 @@ }, "mutations": { "capture": { "model": "sample", "action": "Lims_CaptureConcentration", "body": { "obj": { "sampleId": { "ref": "sampleId" } } }, - "concurrency": "serial", "serialKey": { "ref": "sampleId" }, "invalidates": ["results"], - "patch": { "query": "results", "key": { "ref": "result.id" }, "with": { "ref": "result" } }, + "concurrency": "serial", "serialKey": { "ref": "sampleId" }, "optimistic": { "query": "results", "key": { "ref": "body.id" }, "with": { "obj": { "mass": { "ref": "body.mass" } } } }, "onSuccess": [{ "set": "unsaved", "value": true }], "onError": [{ "notify": { "error": "capture" }, "role": "error" }], @@ -240,21 +239,20 @@ { "name": "the catalog rules pass when the catalog has no lookups", "noCatalog": true, "doc": { "vocab": "ui/1", "id": "t", "models": { "m": { "type": "Unknown_Model", "instance": { "lit": 1 } } }, - "queries": { "q": { "model": "m", "action": "Unknown_Action", "refreshOn": ["Other_Action"] } }, + "queries": { "q": { "model": "m", "action": "Unknown_Action" } }, "view": { "kind": "text", "text": "x" } } }, { - "name": "a patch and an optimistic write name the list inside a record result with at", + "name": "an optimistic write names the list inside a record result with at", "doc": { "vocab": "ui/1", "id": "t", "models": { "m": { "type": "Lims_SampleModel", "instance": { "lit": 1 } } }, "queries": { "q": { "model": "m", "action": "Lims_ListSamples" } }, "mutations": { "c": { "model": "m", "action": "Lims_Save", - "patch": { "query": "q", "at": "samples", "key": { "ref": "result.id" }, "with": { "ref": "result" } }, "optimistic": { "query": "q", "at": "notes", "key": 1, "with": { "obj": { "a": 1 } } } } }, "view": { "kind": "text", "text": "x" } } }, { - "name": "a ui/2 document keeps optimistic, refreshEvery and refetch", - "doc": { "vocab": "ui/2", "id": "t", "models": { "m": { "type": "Lims_SampleModel", "instance": { "lit": 1 } } }, + "name": "a document keeps optimistic, refreshEvery and refetch", + "doc": { "vocab": "ui/1", "id": "t", "models": { "m": { "type": "Lims_SampleModel", "instance": { "lit": 1 } } }, "queries": { "q": { "model": "m", "action": "Lims_ListSamples", "refreshEvery": 5000 } }, "mutations": { "c": { "model": "m", "action": "Lims_Save", "optimistic": { "query": "q", "at": "notes", "key": 1, "with": { "obj": { "a": 1 } } }, @@ -266,9 +264,9 @@ { "name": "not an object", "raw": [1], "at": "", "reason": "a document is a JSON object" }, { "name": "no vocab", "omit": ["vocab"], "at": "", "reason": "`vocab` is required" }, { "name": "another vocabulary", "doc": { "vocab": "ui/3" }, "at": "/vocab", "reason": "is not supported" }, - { "name": "invalidates in ui/2", "doc": { "vocab": "ui/2", "models": { "x": { "type": "M" } }, "queries": { "q": { "model": "x", "action": "A" } }, "mutations": { "m": { "model": "x", "action": "B", "invalidates": ["q"] } } }, "at": "/mutations/m/invalidates", "reason": "`invalidates` is not in ui/2" }, - { "name": "a patch in ui/2", "doc": { "vocab": "ui/2", "models": { "x": { "type": "M" } }, "queries": { "q": { "model": "x", "action": "A" } }, "mutations": { "m": { "model": "x", "action": "B", "patch": { "query": "q", "key": 1, "with": 1 } } } }, "at": "/mutations/m/patch", "reason": "`patch` is not in ui/2" }, - { "name": "refreshOn in ui/2", "doc": { "vocab": "ui/2", "models": { "x": { "type": "M" } }, "queries": { "q": { "model": "x", "action": "A", "refreshOn": ["B"] } } }, "at": "/queries/q/refreshOn", "reason": "`refreshOn` is not in ui/2" }, + { "name": "invalidates is refused", "doc": { "vocab": "ui/1", "models": { "x": { "type": "M" } }, "queries": { "q": { "model": "x", "action": "A" } }, "mutations": { "m": { "model": "x", "action": "B", "invalidates": ["q"] } } }, "at": "/mutations/m/invalidates", "reason": "a document does not declare `invalidates`" }, + { "name": "a patch is refused", "doc": { "vocab": "ui/1", "models": { "x": { "type": "M" } }, "queries": { "q": { "model": "x", "action": "A" } }, "mutations": { "m": { "model": "x", "action": "B", "patch": { "query": "q", "key": 1, "with": 1 } } } }, "at": "/mutations/m/patch", "reason": "a document does not declare `patch`" }, + { "name": "refreshOn is refused", "doc": { "vocab": "ui/1", "models": { "x": { "type": "M" } }, "queries": { "q": { "model": "x", "action": "A", "refreshOn": ["B"] } } }, "at": "/queries/q/refreshOn", "reason": "a document does not declare `refreshOn`" }, { "name": "a non-string vocab", "doc": { "vocab": 1 }, "at": "/vocab", "reason": "`vocab` is a string" }, { "name": "no id", "omit": ["id"], "at": "", "reason": "`id` is required" }, { "name": "an empty id", "doc": { "id": "" }, "at": "/id", "reason": "non-empty id" }, @@ -302,8 +300,6 @@ { "name": "a query with action and evaluate", "doc": { "models": { "m": { "type": "T" } }, "queries": { "q": { "model": "m", "action": "A", "evaluate": "A" } } }, "at": "/queries/q", "reason": "exactly one of `action` and `evaluate`" }, { "name": "a query with neither", "doc": { "models": { "m": { "type": "T" } }, "queries": { "q": { "model": "m" } } }, "at": "/queries/q", "reason": "exactly one of `action` and `evaluate`" }, { "name": "a negative debounce", "doc": { "queries": { "q": { "model": "m", "action": "A", "debounce": -1 } } }, "at": "/queries/q/debounce", "reason": "non-negative integer" }, - { "name": "refreshOn that is not an array", "doc": { "queries": { "q": { "model": "m", "action": "A", "refreshOn": "B" } } }, "at": "/queries/q/refreshOn", "reason": "array of strings" }, - { "name": "refreshOn with a number", "doc": { "queries": { "q": { "model": "m", "action": "A", "refreshOn": [5] } } }, "at": "/queries/q/refreshOn/0", "reason": "array of strings" }, { "name": "a query whose model is not an alias", "doc": { "queries": { "q": { "model": "m", "action": "A" } } }, "at": "/queries/q/model", "reason": "is not a model alias of this scope" }, { "name": "a query through a state", "doc": { "state": { "m": { "init": 1 } }, "queries": { "q": { "model": "m", "action": "A" } } }, "at": "/queries/q/model", "reason": "is not a model alias of this scope" }, { "name": "a non-bool when", "doc": { "models": { "m": { "type": "T" } }, "queries": { "q": { "model": "m", "action": "A", "when": "yes" } } }, "at": "/queries/q/when", "reason": "`when` is string; expected bool" }, @@ -312,11 +308,8 @@ { "name": "a mutation without an action", "doc": { "mutations": { "m": { "model": "x" } } }, "at": "/mutations/m", "reason": "`action` is required" }, { "name": "an unknown concurrency", "doc": { "mutations": { "m": { "model": "x", "action": "A", "concurrency": "fast" } } }, "at": "/mutations/m/concurrency", "reason": "not one of the values" }, { "name": "a serialKey without serial", "doc": { "mutations": { "m": { "model": "x", "action": "A", "serialKey": 1 } } }, "at": "/mutations/m/serialKey", "reason": "applies to `serial`" }, - { "name": "a patch without a key", "doc": { "mutations": { "m": { "model": "x", "action": "A", "patch": { "query": "q", "with": 1 } } } }, "at": "/mutations/m/patch", "reason": "a patch is" }, - { "name": "a patch that is not an object", "doc": { "mutations": { "m": { "model": "x", "action": "A", "optimistic": 1 } } }, "at": "/mutations/m/optimistic", "reason": "an optimistic write is" }, + { "name": "an optimistic write that is not an object", "doc": { "mutations": { "m": { "model": "x", "action": "A", "optimistic": 1 } } }, "at": "/mutations/m/optimistic", "reason": "an optimistic write is" }, { "name": "an unknown offline mode", "doc": { "mutations": { "m": { "model": "x", "action": "A", "offline": "later" } } }, "at": "/mutations/m/offline", "reason": "not one of the values" }, - { "name": "invalidates naming no query", "doc": { "models": { "x": { "type": "T" } }, "mutations": { "m": { "model": "x", "action": "A", "invalidates": ["nope"] } } }, "at": "/mutations/m/invalidates/0", "reason": "is not a query of this document" }, - { "name": "a patch of no query", "doc": { "models": { "x": { "type": "T" } }, "mutations": { "m": { "model": "x", "action": "A", "patch": { "query": "nope", "key": 1, "with": 1 } } } }, "at": "/mutations/m/patch/query", "reason": "is not a query of this document" }, { "name": "result outside a patch", "doc": { "models": { "x": { "type": "T" } }, "mutations": { "m": { "model": "x", "action": "A", "body": { "ref": "result" } } } }, "at": "/mutations/m/body", "reason": "does not resolve" }, { "name": "a mutation command that does not resolve", "doc": { "models": { "x": { "type": "T" } }, "mutations": { "m": { "model": "x", "action": "A", "onSuccess": [{ "refetch": "nope" }] } } }, "at": "/mutations/m/onSuccess/0", "reason": "is not a query" }, { "name": "watch that is not an array", "doc": { "watch": {} }, "at": "/watch", "reason": "`watch` is an array" }, @@ -442,10 +435,7 @@ { "name": "an instance key on a model that is not keyed", "catalog": true, "doc": { "models": { "m": { "type": "Lims_CatalogModel", "instance": 1 } } }, "at": "/models/m/instance", "reason": "is not keyed" }, { "name": "an unknown action", "catalog": true, "doc": { "models": { "m": { "type": "Lims_SampleModel" } }, "queries": { "q": { "model": "m", "action": "Nope" } } }, "at": "/queries/q", "reason": "the action `Nope` is not in the catalog" }, { "name": "an action of another model", "catalog": true, "doc": { "models": { "m": { "type": "Lims_SampleModel" } }, "queries": { "q": { "model": "m", "action": "Lims_GetLimits" } } }, "at": "/queries/q", "reason": "belongs to `Lims_CatalogModel`, not `Lims_SampleModel`" }, - { "name": "refreshOn across model types", "catalog": true, "doc": { "models": { "m": { "type": "Lims_SampleModel" } }, "queries": { "q": { "model": "m", "action": "Lims_ListResults", "refreshOn": ["Lims_GetLimits"] } } }, "at": "/queries/q/refreshOn/0", "reason": "`refreshOn` names actions of the query's model type" }, - { "name": "refreshOn of an unknown action", "catalog": true, "doc": { "models": { "m": { "type": "Lims_SampleModel" } }, "queries": { "q": { "model": "m", "action": "Lims_ListResults", "refreshOn": ["Nope"] } } }, "at": "/queries/q/refreshOn/0", "reason": "`refreshOn` names actions" }, - { "name": "a patch with an empty at", "doc": { "models": { "x": { "type": "T" } }, "queries": { "q": { "model": "x", "action": "A" } }, "mutations": { "m": { "model": "x", "action": "A", "patch": { "query": "q", "at": "", "key": 1, "with": 1 } } } }, "at": "/mutations/m/patch/at", "reason": "`at` names the member of the query's record" }, - { "name": "a patch of a record result without at", "catalog": true, "doc": { "models": { "m": { "type": "Lims_SampleModel", "instance": { "lit": 1 } } }, "queries": { "q": { "model": "m", "action": "Lims_ListSamples" } }, "mutations": { "c": { "model": "m", "action": "Lims_Save", "patch": { "query": "q", "key": 1, "with": { "ref": "result" } } } } }, "at": "/mutations/c/patch", "reason": "`Lims_ListSamples` answers a record" }, + { "name": "an optimistic write to a record result without at", "catalog": true, "doc": { "models": { "m": { "type": "Lims_SampleModel", "instance": { "lit": 1 } } }, "queries": { "q": { "model": "m", "action": "Lims_ListSamples" } }, "mutations": { "c": { "model": "m", "action": "Lims_Save", "optimistic": { "query": "q", "key": 1, "with": { "obj": { "a": 1 } } } } } }, "at": "/mutations/c/optimistic", "reason": "`Lims_ListSamples` answers a record" }, { "name": "an optimistic write at a member the result does not hold as a list", "catalog": true, "doc": { "models": { "m": { "type": "Lims_SampleModel", "instance": { "lit": 1 } } }, "queries": { "q": { "model": "m", "action": "Lims_ListSamples" } }, "mutations": { "c": { "model": "m", "action": "Lims_Save", "optimistic": { "query": "q", "at": "rows", "key": 1, "with": { "obj": { "a": 1 } } } } } }, "at": "/mutations/c/optimistic/at", "reason": "has no list member `rows`" }, { "name": "an optimistic write to a calculated field", "catalog": true, "doc": { "models": { "m": { "type": "Lims_SampleModel" } }, "queries": { "q": { "model": "m", "action": "Lims_ListResults" } }, "mutations": { "c": { "model": "m", "action": "Lims_CaptureConcentration", "optimistic": { "query": "q", "key": 1, "with": { "obj": { "mass": 1, "concentration": 2 } } } } } }, "at": "/mutations/c/optimistic/with/obj/concentration", "reason": "`concentration` is calculated by the server" }, { "name": "a credential-bearing mutation queued offline", "catalog": true, "doc": { "models": { "a": { "type": "Auth_SessionModel" } }, "mutations": { "login": { "model": "a", "action": "Auth_Login", "offline": "queue" } } }, "at": "/mutations/login/offline", "reason": "never queued offline" }, @@ -527,9 +517,7 @@ { "name": "F0", "doc": { "keys": { "F0": [] } }, "at": "/keys/F0", "reason": "does not end in a key name" }, { "name": "Ctrl+w is reserved", "doc": { "keys": { "Ctrl+w": [] } }, "at": "/keys/Ctrl+w", "reason": "reserved" }, { "name": "a text debounce", "doc": { "queries": { "q": { "model": "m", "action": "A", "debounce": "x" } } }, "at": "/queries/q/debounce", "reason": "non-negative integer" }, - { "name": "refreshOn with an empty name", "doc": { "queries": { "q": { "model": "m", "action": "A", "refreshOn": [""] } } }, "at": "/queries/q/refreshOn/0", "reason": "array of strings" }, { "name": "a borrowed instance of a malformed alias", "doc": { "models": { "m": { "type": "T", "instance": { "from": "1x" } } } }, "at": "/models/m/instance", "reason": "a borrowed instance is" }, - { "name": "a patch without with", "doc": { "mutations": { "m": { "model": "x", "action": "A", "patch": { "query": "q", "key": 1 } } } }, "at": "/mutations/m/patch", "reason": "a patch is" }, { "name": "run of an empty name", "doc": { "view": { "kind": "button", "onClick": [{ "run": "" }] } }, "at": "/view/onClick/0/run", "reason": "`run` names a mutation" }, { "name": "host of a number", "doc": { "view": { "kind": "button", "onClick": [{ "host": 5 }] } }, "at": "/view/onClick/0/host", "reason": "names a registered command" }, { "name": "if that is not an array", "doc": { "view": { "kind": "button", "onClick": [{ "if": true }] } }, "at": "/view/onClick/0/if", "reason": "`if` takes" }, diff --git a/tests/ui/golden/commands.json b/tests/ui/golden/commands.json index 29f1a4449..9cabeaa2d 100644 --- a/tests/ui/golden/commands.json +++ b/tests/ui/golden/commands.json @@ -1,5 +1,5 @@ { - "vocab": "ui/2", + "vocab": "ui/1", "id": "lims.commands", "models": { "sample": { diff --git a/tests/ui/golden/declarations.json b/tests/ui/golden/declarations.json index 4c1f7759b..df0843de2 100644 --- a/tests/ui/golden/declarations.json +++ b/tests/ui/golden/declarations.json @@ -1,5 +1,5 @@ { - "vocab": "ui/2", + "vocab": "ui/1", "id": "lims.declarations", "title": { "t": "lims.declarations", diff --git a/tests/ui/golden/palette.json b/tests/ui/golden/palette.json index 4a5d60081..9a084fba1 100644 --- a/tests/ui/golden/palette.json +++ b/tests/ui/golden/palette.json @@ -1,5 +1,5 @@ { - "vocab": "ui/2", + "vocab": "ui/1", "id": "lims.palette", "models": { "sample": { diff --git a/tests/ui/golden/results.json b/tests/ui/golden/results.json index f1f3246f6..edbe648ad 100644 --- a/tests/ui/golden/results.json +++ b/tests/ui/golden/results.json @@ -1,5 +1,5 @@ { - "vocab": "ui/2", + "vocab": "ui/1", "id": "lims.results", "params": { "sampleId": "int" diff --git a/tests/ui_interpreter_support.hpp b/tests/ui_interpreter_support.hpp index 4ca3cf912..100064db3 100644 --- a/tests/ui_interpreter_support.hpp +++ b/tests/ui_interpreter_support.hpp @@ -309,8 +309,11 @@ inline Catalog corpusCatalog() { if (action == "Auth_Login") { info.credential = true; } + // A result row, keyed by `id`: the capture's reply updates it in every list holding it. + ActionRecordInfo const resultsAtTop{.record = "Lims_Result", .keyField = "id", .paths = {""}}; if (action == "Lims_ListResults") { info.calculatedFields = {"concentration"}; + info.records = {resultsAtTop}; } // Server push (spec 8): `T_Row` is keyed by `code`, not `id`, so a patch must use the catalog's key field. ActionRecordInfo const rowsAtTop{.record = "T_Row", .keyField = "code", .paths = {""}}; @@ -320,11 +323,16 @@ inline Catalog corpusCatalog() { if (action == "Push_Header") { info.records = {ActionRecordInfo{.record = "T_Row", .keyField = "code", .paths = {"header.rows"}}}; } - if (action == "Push_EditRow" || action == "Lims_CaptureConcentration") { + if (action == "Push_EditRow") { info.effects = std::vector{ActionEffect{.kind = "updates", .record = "T_Row"}}; info.records = {rowsAtTop}; } - if (action == "Push_RenameAll") { + if (action == "Lims_CaptureConcentration") { + info.effects = std::vector{ActionEffect{.kind = "updates", .record = "T_Row"}, + ActionEffect{.kind = "updates", .record = "Lims_Result"}}; + info.records = {rowsAtTop, resultsAtTop}; + } + if (action == "Push_RenameAll" || action == "Lims_DeleteSample") { info.effects = std::vector{ActionEffect{.kind = "changes", .record = "T_Row"}}; } if (action == "Push_Reset") { From 156c1b7c266bc42dcadb275f4fb0a4a80aaf2da3 Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 08:20:01 +0200 Subject: [PATCH 66/68] docs: a guide to keeping every screen current, and the docs that point to it docs/KEEPING-SCREENS-CURRENT.md is for application developers and coding agents: the three things to write (records, effects, screens), choosing an effect, what an action's result holds, what refreshes on the caller, the server and other clients, the screen rules, a checklist for every new or changed action, every startup error, load-time refusal and warning with its fix, testing, agents over MCP, the explorer and the limits. GETTING-STARTED's screen example no longer uses invalidates, and it and the README no longer say morph has no server push. ARCHITECTURE gains the section and the new namespaces and headers; IMPLEMENTATION.md and AGENTS.md make declaring effects a rule. The registry spec allows the effects macro in a header, as the rungs use it. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- AGENTS.md | 6 + README.md | 12 +- docs/ARCHITECTURE.md | 37 +- docs/GETTING-STARTED.md | 50 ++- docs/KEEPING-SCREENS-CURRENT.md | 326 ++++++++++++++++++ docs/spec/core/registry.md | 12 +- examples/IMPLEMENTATION.md | 10 + .../bank/screens/tests/test_bank_screens.cpp | 4 +- .../screens/tests/test_bookmarks_screens.cpp | 4 +- .../lims/screens/tests/test_lims_screens.cpp | 4 +- 10 files changed, 439 insertions(+), 26 deletions(-) create mode 100644 docs/KEEPING-SCREENS-CURRENT.md diff --git a/AGENTS.md b/AGENTS.md index eee1d8657..607cef6c7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -137,6 +137,12 @@ So, before claiming something works: State results as measured or as inferred, and never round the second up to the first. +## Adding or changing an action + +An action that changes data declares what it changes (`BRIDGE_REGISTER_EFFECTS`, beside its +`BRIDGE_REGISTER_ACTION`), and screens never say what to refresh. Work through the checklist in +[`docs/KEEPING-SCREENS-CURRENT.md`](docs/KEEPING-SCREENS-CURRENT.md) for every new or changed action. + ## Design specs (`docs/spec/`) One file per public type or subsystem. There is **no size limit** on spec diff --git a/README.md b/README.md index 97efbc5d9..567b9b2a2 100644 --- a/README.md +++ b/README.md @@ -530,9 +530,10 @@ in [`docs/spec/`](docs/spec) before relying on any of these in production: - **`Completion` is a leaf callback primitive**, not a composable future: one handler per outcome, no `T→U` chaining, no `co_await`, no cancellation. - **Instance subscriptions are best-effort and in-process.** `subscribe` - fans out to handlers on the same `Bridge`; there is no server-initiated push, - so two separate clients sharing an instance do not see each other's results - until they ask again. No replay, no durability, no coalescing. + fans out to handlers on the same `Bridge`, with no replay and no durability. + Across clients, screens stay current through declared effects, which tell + every watching connection what changed but not the new values + ([`docs/KEEPING-SCREENS-CURRENT.md`](docs/KEEPING-SCREENS-CURRENT.md)). - **Exact numbers are fixed-width.** `Rational` is an `int64` pair; `+`/`-`/`*` *saturate* at `±INT64_MAX/1` and log at `error` rather than returning an error value, so an out-of-envelope result is clamped and inexact rather than exact — @@ -561,7 +562,10 @@ in [`docs/spec/`](docs/spec) before relying on any of these in production: - [`docs/GETTING-STARTED.md`](docs/GETTING-STARTED.md) — the step-by-step tutorial: build one small app end to end and understand why each piece - exists. Read this before the two below. + exists. Read this before the others below. +- [`docs/KEEPING-SCREENS-CURRENT.md`](docs/KEEPING-SCREENS-CURRENT.md) — how + screens refresh after an action, on every client and for AI agents: records, + effects, the checklist for a new action, and every error with its fix. - [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) — namespace map, layer diagram, wire protocol, deployment topologies, and design rationale. - [`docs/spec/`](docs/spec) — the authoritative per-type/subsystem design specs. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 347e9b657..6b0caa00c 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -39,7 +39,9 @@ The public surface is split per topic so callers always know whether a name is p | `morph::log` | Configurable logging | `LogLevel`, `setLogger`, `setLogLevel`, `getLogLevel`, `logDebug`, `logInfo`, `logWarn`, `logError` | | `morph::exec` | Executor primitives | `IExecutor`, `ThreadPoolExecutor`, `MainThreadExecutor` | | `morph::async` | Async result handle | `Completion` | -| `morph::model` | Model & action traits | `ModelTraits<>`, `ActionTraits<>`, `ActionValidator<>`, `ActionLogPolicy<>`, `Loggable` | +| `morph::model` | Model & action traits | `ModelTraits<>`, `ActionTraits<>`, `ActionValidator<>`, `ActionLogPolicy<>`, `Loggable`, `EffectsRegistrationError` | +| `morph::record` | Result types that are shared data (see `docs/KEEPING-SCREENS-CURRENT.md`) | `BRIDGE_REGISTER_RECORD`, `key<&T::m>`, `keyless`, `derivedFrom`, `privateKeys`, `RecordInfo` | +| `morph::effects` | What an action changes, and the changes an execute made | `BRIDGE_REGISTER_EFFECTS`, `updates`, `changes`, `changesInstance`, `none`, `Effect`, `ActionRecord`, `effectsOf`, `recordsOf`, `recordNamed`, `actionsHolding`, `deriveChanges`, `Changes`, `kMaxChangeKeys` | | `morph::backend` | Pluggable backends | `LocalBackend`, `RemoteServer`, `SimulatedRemoteBackend` | | `morph::bridge` | Bridge between handler and backend | `Bridge`, `BridgeHandler` | | `morph::offline` | Connectivity + replay | `NetworkMonitor`, `NetworkMonitorConfig`, `IOfflineQueue`, `QueueItem`, `OfflineQueueFullError`, `InMemoryOfflineQueue`, `FileOfflineQueue`, `FileOfflineQueueError`, `SqliteOfflineQueue`, `SqliteOfflineQueueError`, `SyncWorker` (incl. `DeadLetterSink`), `SyncResult`, `ReconnectCoordinator`, `ReconnectOutcome`, `ReconnectCoordinatorConfig` | @@ -420,6 +422,14 @@ struct MyModel { BRIDGE_REGISTER_MODEL (MyModel, "MyModel") BRIDGE_REGISTER_ACTION(MyModel, MyAction, "MyAction") +``` + + An action that changes data also says what it changes, so every screen showing it refreshes + (`docs/KEEPING-SCREENS-CURRENT.md`): + +```cpp +BRIDGE_REGISTER_RECORD(MyRow, "MyRow") // a result type that is shared data +BRIDGE_REGISTER_EFFECTS(MyAction, ::morph::effects::changes) // beside the action's registration ``` 3. Use from the GUI (same code for local and remote): @@ -514,6 +524,26 @@ sidebar.execute(Deposit{.amountMinor = 5000}); | **Callback thread** | Always the executor passed at handler construction. | +## Keeping screens current across clients + +`subscribe` reaches handlers on one `Bridge`. What keeps every client's screens current, and an agent's changes +visible to them, is declared effects: + +- `BRIDGE_REGISTER_RECORD` names a result type that is shared data, with its key. +- `BRIDGE_REGISTER_EFFECTS`, beside an action's registration, says which records it changes (`updates` when the + reply carries the changed rows, `changes` when it does not, `changesInstance`, or `none`). +- After a successful execute, `RemoteServer` derives the changes from the effects and the result, and sends an + `event` to every connection watching a changed record, after re-authorising each watch. A UI client watches the + records its query cache holds, before it fetches them, and applies a change by patching or refetching the entries + that hold the record. The caller applies its own reply's effects at once. +- Screens declare no refreshes: a document naming `invalidates`, `patch` or `refreshOn` is refused, and so is a query + whose action declares effects. +- `morph::ui::McpAdapter` (`ui/agent.hpp`) lets an AI agent drive the same server over MCP, as one more connection. + +The user guide is [`KEEPING-SCREENS-CURRENT.md`](KEEPING-SCREENS-CURRENT.md); the rules are in +`docs/spec/core/registry.md` ("Effects"), `docs/spec/core/backend.md` ("Fan-out"), `docs/spec/ui/interpreter.md` and +`docs/spec/ui/document.md`. + ## Exact values, units, and schema-driven forms Three headers extend actions from "any aggregate" to *self-describing* @@ -657,6 +687,10 @@ folder. | `core/completion.hpp` | `CompletionState` (detail) + `Completion` (public) — result handle | | `core/model.hpp` | `IModelHolder`, `ModelHolder`, `ModelFactory`, `IBackendChangedSink`, `BackendChangedNotifiable` — type-erased model storage; `IModelHolder::attachActionLog`/`hasActionLog`/`recordIfAttached` (`morph::model::detail::`) | | `core/registry.hpp` | `ModelTraits<>`, `ActionTraits<>`, `ActionValidator<>`, `ActionLogPolicy<>`, `Loggable` (public) + `ActionDispatcher` (also tracking each action's `coalesce` policy), `ModelRegistryFactory`, `defaultDispatcher()`, `defaultRegistry()`, `ParseError`, `registerModelOnce`, `registerActionOnce`, `actionLoggable()` (detail). Registration macros `BRIDGE_REGISTER_MODEL`, `BRIDGE_REGISTER_ACTION` (optional 4th `Loggable` argument), `BRIDGE_REGISTER_VALIDATOR` are defined here at file scope. | +| `core/records.hpp` | `BRIDGE_REGISTER_RECORD` and its options `key<&T::m>`, `keyless`, `derivedFrom`, `privateKeys`; `RecordInfo` (`morph::record::`) — result types that are shared data. See `docs/spec/core/registry.md`, "Record registry". | +| `core/effects.hpp` | `BRIDGE_REGISTER_EFFECTS` and `updates`, `changes`, `changesInstance`, `none`; `Effect`, `ActionRecord`, `effectsOf`, `recordsOf`, `recordNamed`, `actionsHolding` (`morph::effects::`); `EffectsRegistrationError` (`morph::model::`) — what each action changes, matched with the records its result holds and validated on the first read. See `docs/spec/core/registry.md`, "Effects". | +| `core/changes.hpp`, `core/change_types.hpp` | `deriveChanges`, `Changes`, `RecordChange`, `InstanceChange`, `kMaxChangeKeys` (`morph::effects::`) — the changes a successful execute made. | +| `core/connection_writer.hpp` | `ConnectionWriter`, `kEventBacklogHigh`, `kEventBacklogLow` (`morph::backend::`) — how `RemoteServer` writes events to a connection, and the backlog above which it holds them back. | | `core/backend.hpp` | `LocalBackend` (public) + `ActionCall`, `IBackend` (detail), including the non-breaking `registerModelWithContext()` default method | | `core/remote.hpp` | `RemoteServer` (configured by `ServerConfig`, incl. `logProvider`), `SimulatedRemoteBackend` (`morph::backend::`) | | `core/bridge.hpp` | `Bridge`, `BridgeHandler` (public) + `HandlerBinding` (carrying `contextKey`), `MemberPointerTraits` (detail) | @@ -736,6 +770,7 @@ folder. | `ui/expression.hpp` | `Expr`, `ExprKind`, `FunctionId`, `parseExpression`, `NameResolver`, `PermissiveResolver`, `checkExpression`, `Environment`, `StaticEnvironment`, `FormatContext`, `UnitTable`, `UnitInfo`, `evaluate`, `walk`, `isName`, `functionNames` — the `expr/1` language. See `docs/spec/ui/expressions.md`. | | `ui/document.hpp` | `Document`, `ScopeDecl` and the declaration structs, `Command`, `CommandKind`, `DocNode`, `Component`, `CustomizePoint`, `Catalog`, `ModelInfo`, `ActionInfo`, `LoadOptions`, `loadDocument`, `loadDocumentText`, `nodeKinds` — the document model and its load-time validation. See `docs/spec/ui/document.md`. | | `ui/interpreter.hpp` | `Interpreter`, `Screen`, `Session`, `SessionOptions`, `Dispatcher`, `ModelChannel`, `CallOptions`, `BridgeDispatcher`, `InterpreterHost`, `NavigateRequest`, `InterpreterOptions`, `MountOptions`, `BorrowedModel` (`morph::ui::`) — mounts a loaded document: scopes, the session's query cache, mutations, relations and commands, lowered to the view tree. See `docs/spec/ui/interpreter.md`. | +| `ui/agent.hpp` | `McpAdapter`, `McpOptions`, `serveStdio` and the JSON-RPC error codes (`morph::ui::`) — an AI agent over MCP, as one more connection to the server. See `docs/spec/ui/agent.md`. | | `ui/author.hpp` | `Handle`, `AnyHandle`, `Body`, `ModelHandle`, `QueryHandle`, `MutationHandle`, `DeclScope`, `Scope`, `Command`, `Commands`, `Node`, `Nodes`, a builder per node kind and command, `Screen`, `screen`, `ScreenRegistry`, `screenRegistry`, `CatalogFacts`, `ModelFacts`, `ActionFacts`, `checkAccessibility`, `MORPH_REGISTER_SCREEN` (`morph::ui::author::`) — server-side authoring: typed builders that write a screen's document, and its registration with the load-time and accessibility rules. See `docs/spec/ui/author.md`. | | `ui/view.hpp` | `Node`, `NodeData`, `Prop`, `SlotId`, `SlotBinding`, `slot()`, `Key`, `Common`, `Accessibility`, `KeyBinding`, `FieldState`, `MenuItem`, `Sizing`, `LayoutHints`, every node kind and its builder, `switchOn`, `forEach`, `table` (`morph::ui::`) — the immutable view tree whose properties are constants or slots. See `docs/spec/ui/view_tree.md`. | | `ui/backend.hpp` | `Widget`, `ContainerWidget`, `FieldWidget`, one widget interface per kind, `MenuEntry`, `IViewBackend` — the retained widgets a renderer implements. See `docs/spec/ui/backend_contract.md`. | diff --git a/docs/GETTING-STARTED.md b/docs/GETTING-STARTED.md index c3ff8a5f3..9daf76bd2 100644 --- a/docs/GETTING-STARTED.md +++ b/docs/GETTING-STARTED.md @@ -638,10 +638,10 @@ abridged): auto const marks = screen.model("marks", {.instance = ui::Instance::Private}); auto const list = ui::element("list"); auto const bookmarks = screen.query("bookmarks", marks, {}); -auto const archive = screen.mutation("archive", marks, {.invalidates = {bookmarks, tags}}); +auto const archive = screen.mutation("archive", marks); auto const rows = bookmarks.value().at(&ListBookmarksResult::bookmarks); return ui::column({ - ui::form(marks, {.onSuccess = refreshed}).id(ui::element("createForm")).testId("createForm"), + ui::form(marks).id(ui::element("createForm")).testId("createForm"), ui::table(rows, {.key = ui::row("id"), /* title and URL columns */, .selection = ui::Selection::Multiple}) .id(list).testId("bookmarks"), ui::button({.label = "Archive selected", @@ -652,8 +652,9 @@ return ui::column({ ``` The screen decides nothing about the domain. It names the model, the query -that lists bookmarks, the mutation, and when the list refetches; every rule -about what a bookmark may be stays in `BookmarkModel`. Archiving a selection is +that lists bookmarks and the mutation; every rule about what a bookmark may be +stays in `BookmarkModel`, and nothing on the screen says when the list +refetches. Archiving a selection is one `BulkEdit` built from the table's own selection, so it is one transaction in the model however many rows are selected. @@ -676,6 +677,32 @@ REQUIRE(harness.submitForm("createForm")); REQUIRE(harness.waitForText("count", "1 bookmarks")); ``` +### What refreshes after an action + +The action says what it changes, beside its registration, and every screen +showing that data refreshes: on this client, on every other connected client, +and when an AI agent runs the action over MCP. Two declarations do it +([`models/bookmark_model.hpp`](../examples/bookmarks/include/bookmarks/models/bookmark_model.hpp), +[`dto/bookmark_dto.hpp`](../examples/bookmarks/include/bookmarks/dto/bookmark_dto.hpp)): + +```cpp +// A result type that is shared data, keyed by its `id` member +BRIDGE_REGISTER_RECORD(bookmarks::BookmarkSummary, "Bookmarks_Bookmark", ::morph::record::privateKeys) + +// What the action changes: archiving moves bookmarks between lists, and changes the tag counts +BRIDGE_REGISTER_ACTION(bookmarks::BookmarkModel, bookmarks::BulkEdit, "BulkEdit", ::morph::model::Loggable::No) +BRIDGE_REGISTER_EFFECTS(bookmarks::BulkEdit, ::morph::effects::changes, + ::morph::effects::changes, + ::morph::effects::changes) +``` + +`changes` refetches every query holding an `R`; `updates` is for an +action whose reply carries the changed rows, which the caller's screens show +at once, before any fetch; `none` is for an action that changes nothing. A query whose +action declares effects is refused, because a query must only read. How to +choose, the checklist for a new action and every error message with its fix +are in [`docs/KEEPING-SCREENS-CURRENT.md`](KEEPING-SCREENS-CURRENT.md). + ## 10. Forms from the compiled action type `morph::forms::schemaJson()` derives a JSON Schema from the action type @@ -856,9 +883,10 @@ client. A guide that only sells is worth less than one that tells you when to stop. Read [`docs/spec/`](spec/) before relying on any of these; the short version: -- **It is not a general-purpose RPC framework.** Request/response only. No - streaming, no bidirectional channels, no server-initiated push. bookmarks' - `GetChangesSince` is the polling answer a client uses instead. +- **It is not a general-purpose RPC framework.** Request/response, plus one + kind of server-initiated message: the change notifications that keep + screens current ([`docs/KEEPING-SCREENS-CURRENT.md`](KEEPING-SCREENS-CURRENT.md)). + No streaming and no general bidirectional channel. - **`Completion` is a leaf callback primitive, not a future.** No `T→U` chaining, no `co_await`, and no cancellation; lifetime binding is the opt-in `CallbackScope` of section 8. @@ -871,9 +899,9 @@ Read [`docs/spec/`](spec/) before relying on any of these; the short version: the default; a server that configures none of them assumes a trusted, authenticated transport and is not a hardened public-internet server. - **Instance subscriptions are best-effort and in-process.** `subscribe` - fans out to handlers on the same `Bridge`; two separate clients sharing an - instance do not see each other's results until they ask again. No replay, no - durability, no coalescing. + fans out to handlers on the same `Bridge`, with no replay and no durability. + Across clients, what keeps screens current is declared effects, which tell + every connection what changed but not the new values. - **Exact numbers are fixed-width.** `Rational` is an `int64` pair; `+`/`-`/`*` can overflow rather than returning an error, and high decimal precision shrinks the representable magnitude. @@ -905,6 +933,8 @@ You now have a question rather than a topic, which is what the map is for. | who is allowed to do what? | `docs/spec/security.md`, `docs/spec/session/session.md` | | what serialises access to my model? | `docs/spec/concurrency_and_lifetimes.md` | | how do I keep several screens on one model instance? | `docs/spec/core/shared_instances.md` | +| how do screens refresh after an action, and what do I declare? | [`docs/KEEPING-SCREENS-CURRENT.md`](KEEPING-SCREENS-CURRENT.md) | +| how does an AI agent drive my app? | `docs/spec/ui/agent.md`, and bank's `--mcp-stdio` ([`examples/bank/README.md`](../examples/bank/README.md)) | | how do I generate a form? | `docs/spec/forms/forms.md`, and [`examples/forms`](../examples/forms) | | how do I test all this? | [`examples/TESTING.md`](../examples/TESTING.md) | | what are the rules for an app built on morph? | [`examples/IMPLEMENTATION.md`](../examples/IMPLEMENTATION.md) | diff --git a/docs/KEEPING-SCREENS-CURRENT.md b/docs/KEEPING-SCREENS-CURRENT.md new file mode 100644 index 000000000..e5e3f796b --- /dev/null +++ b/docs/KEEPING-SCREENS-CURRENT.md @@ -0,0 +1,326 @@ +# Keeping every screen current: records and effects + +When an action changes data, every screen that shows that data refreshes: on the client that ran it, on every other +connected client, and when an AI agent runs it over MCP. Two things make that happen, and neither of them is in a +screen. + +- **A record** names a result type that is shared data, such as a bookmark, an account or a sample, and says how to + tell its rows apart. +- **An effect** declares, beside an action's registration, which records the action changes. + +The server tells every connection that is showing a changed record, and each client refreshes what it holds. A +screen only says what it shows and what an edit sends. + +This guide is for people and for coding agents. It covers what you write, how to choose an effect, what refreshes +when, every error you can hit and its fix, and how to test it. The authoritative reference is in +[`docs/spec/`](spec/): see [Where the rules live](#where-the-rules-live). + +## Contents + +- [The three things you write](#the-three-things-you-write) +- [Choosing an effect](#choosing-an-effect) +- [Records](#records) +- [What refreshes, and when](#what-refreshes-and-when) +- [Screens](#screens) +- [Checklist for a new or changed action](#checklist-for-a-new-or-changed-action) +- [Errors and warnings, and their fixes](#errors-and-warnings-and-their-fixes) +- [Testing](#testing) +- [Agents over MCP](#agents-over-mcp) +- [Seeing it in the explorer](#seeing-it-in-the-explorer) +- [Limits](#limits) +- [Where the rules live](#where-the-rules-live) + +## The three things you write + +The examples are from [`examples/bookmarks`](../examples/bookmarks). + +**1. Records, once per result type, beside the DTO.** The macro is in ``. + +```cpp +// bookmark_dto.hpp: a bookmark shown two ways, as a list row and as the full detail view +BRIDGE_REGISTER_RECORD(bookmarks::BookmarkSummary, "Bookmarks_Bookmark", ::morph::record::privateKeys) +BRIDGE_REGISTER_RECORD(bookmarks::BookmarkView, "Bookmarks_BookmarkDetail", ::morph::record::privateKeys) + +// tag_dto.hpp +BRIDGE_REGISTER_RECORD(bookmarks::TagSummary, "Bookmarks_Tag", ::morph::record::privateKeys) +``` + +The key is the member named `id` unless you say otherwise. `privateKeys` keeps one user's row keys away from other +users' clients. + +**2. Effects, beside each action that changes something.** The macro is in ``. + +```cpp +// tag_model.hpp: renaming a tag renames it on every bookmark that carries it +BRIDGE_REGISTER_ACTION(bookmarks::TagModel, bookmarks::RenameTag, "RenameTag") +BRIDGE_REGISTER_EFFECTS(bookmarks::RenameTag, ::morph::effects::changes, + ::morph::effects::changes, + ::morph::effects::changes) + +// bookmark_model.hpp: the reply is the edited bookmark's full view, so an open detail view is patched in place +BRIDGE_REGISTER_ACTION(bookmarks::BookmarkModel, bookmarks::EditBookmark, "EditBookmark") +BRIDGE_REGISTER_EFFECTS(bookmarks::EditBookmark, ::morph::effects::updates, + ::morph::effects::changes, + ::morph::effects::changes) + +// A read declares nothing. An action a screen runs as a mutation but that changes nothing says so: +BRIDGE_REGISTER_EFFECTS(bookmarks::ExportBookmarks, ::morph::effects::none) +``` + +**3. Screens that say only what they show and what they send.** + +```cpp +auto const tagModel = screen.model("tagModel", {.instance = ui::Instance::Private}); +auto const tags = screen.query("tags", tagModel, {}); +auto const picked = ui::element("tagList"); +return ui::row({ + ui::table(tags.value().at(&ListTagsResult::tags), {.key = ui::row("id"), /* columns */}).id(picked), + ui::form(tagModel, {.prefill = ui::obj({{"id", ui::call("first", {picked.selection()})}}), + .fields = {{&RenameTag::id, {.hidden = true}}}}), +}); +``` + +Nothing here says what to refresh. When the rename succeeds, this table refetches, and so does every other screen +holding tags, bookmark rows or a bookmark's detail, on every connected client. + +A screen written as JSON is the same: + +```json +{ "vocab": "ui/1", "id": "bookmarks.tags", + "models": { "tagModel": { "type": "TagModel", "instance": "private" } }, + "queries": { "tags": { "model": "tagModel", "action": "ListTags" } }, + "mutations": { "rename": { "model": "tagModel", "action": "RenameTag" } }, + "view": { "kind": "text", "text": "…" } } +``` + +## Choosing an effect + +Ask what the action does to the data, and what its reply carries. + +| The action… | Declare | What happens | +|---|---|---| +| changes rows of `R`, and its reply holds those rows with their keys | `updates` | The caller's screens show the reply's rows at once, before any fetch. Every watching client, the caller included, then refetches the entries holding one of those keys; for a `privateKeys` record, other principals refetch every entry holding `R`. | +| changes rows of `R`, but its reply does not hold them (a delete, a bulk edit, an import, a reply of `Ack` or `bool`) | `changes` | Every cache entry holding an `R`, on every watching client, is marked stale and refetched. | +| changes the state of the model instance it ran on, not a shared record | `changesInstance` | On the caller, entries read through that instance refresh. A keyed shared instance's change also reaches the other clients watching it; a private instance's change reaches only the connection that holds it. | +| changes nothing, but a screen runs it as a mutation (open, attach, export) | `none` | Nothing refreshes, and the document checks stay quiet. | +| only reads | nothing | It is a read. A screen runs it as a query. | + +Rules of thumb: + +- **List every record the action changes**, not only the one in its name. Renaming a tag changes the bookmarks that + carry it. Archiving a bookmark changes the tag counts. +- **Prefer `updates` when the reply really is the new row.** The caller's screens show it without waiting for a + fetch, and other clients refetch only the entries holding the changed keys. If the reply is a different shape of the + same data, such as a full view while the lists show a summary, declare `updates` for the + shape the reply holds and `changes` for the others, as `EditBookmark` does. +- **When unsure, declare `changes`.** It refetches more than it needs to, but it never misses a change. +- **`updates` and `changes` together mean the whole type**: the change is reported without keys. +- Declare each action's effects in **one** place. A header included by several translation units is still one + declaration; a second, different declaration for the same action is refused. + +## Records + +```cpp +BRIDGE_REGISTER_RECORD(Type, "WireName", options...) +``` + +| Option | Meaning | +|---|---| +| (none) | The key is the member glaze reflects as `id`. A type with no `id` member and no `key<>` has no key, like `keyless`. | +| `::morph::record::key<&T::member>` | The key is another member. A strong id (`WrappedModelKey`) works as well as a raw integer or string. | +| `::morph::record::keyless` | The record has no key: it can only be changed as a whole type (`changes`), never `updates`. | +| `::morph::record::derivedFrom` | A change to any `S` is a change to this record. For a report or a summary computed from other records. A derived record must be keyless. | +| `::morph::record::privateKeys` | Row keys reach only the principal who made the change. Every other watching principal is told the whole type changed, without keys. Use it when a key alone would reveal something, such as which accounts exist. | + +**What an action's result holds.** The registration of each action walks its result type and records the paths at +which it holds each record type. The result type itself counts, at the path `""`: `EditBookmark` returns a +`BookmarkView`, so it holds `Bookmarks_BookmarkDetail`. The walk looks through `std::vector`, `std::optional` and +`std::map`, so a `ListBookmarksResult` holding `std::vector bookmarks` holds `Bookmarks_Bookmark` at +`bookmarks`. It stops at the first record on each path, so a record's own members are not searched for further +records. `morph::effects::recordsOf("ListBookmarks")` answers what an action holds, with each record's key field and +paths, and the explorer shows the same. + +A derived record, from [`examples/bank`](../examples/bank): + +```cpp +// A spending report sums the ledger, so whatever adds a transaction changes every report. +BRIDGE_REGISTER_RECORD(bank::dto::SpendingReport, "Bank_SpendingReport", ::morph::record::keyless, + ::morph::record::derivedFrom) +``` + +Where to put the macros: `BRIDGE_REGISTER_RECORD` beside the DTO, and `BRIDGE_REGISTER_EFFECTS` directly under the +action's `BRIDGE_REGISTER_ACTION`. A header is fine for both: every translation unit that includes it registers the +same declaration again, which is a no-op, and one that does not include it still reads the same effects. Wire names +follow the application's prefix, as `Bookmarks_Bookmark` and `Bank_Account` do. Under +`MORPH_CLIENT_ONLY` both expand to nothing. + +## What refreshes, and when + +1. **On the caller,** as soon as the reply arrives, the reply's effects are applied to its own cache. + - `updates`: rows of `R` in every cached query holding them are replaced, by the record's key, at the paths the + catalog gives, so the screen shows the new values before any fetch. The server's event for the same change then + refetches the entries holding those keys, as on every other client. + - `changes` and `changesInstance`: the entries holding the record, or read through the instance, are marked + stale. A mounted query on a stale entry refetches. + - This covers mutations and the submits of forms, collections and wizards alike. +2. **On the server,** a successful execute derives its changes from the declared effects and the result. It sends + them to every connection watching a changed record, the caller's included, after re-checking that each watcher + may still see it. +3. **On every other client,** the change is applied the same way, filtered by its keys when it carries them. + +The client watches the records of what its cache holds, and sends the watch before the first fetch, so no change is +missed in between. You never write a watch. After a reconnect the client watches everything again and refetches what +it shows, since it may have missed changes while it was away. + +A fetch that is in flight when a change arrives fetches once more when it settles, because it may have read the data +before the change. Changes are coalesced per connection for 50 ms (`ServerConfig::coalesceWindow`), or +until 256 keys and topics are queued. A reader that falls behind, with more than 1 MiB unread, is sent no more events +until it is below 256 KiB again; its replies are never held back. Then it gets one `resync`, which refreshes everything +it watches. + +## Screens + +What a screen declares: + +- **Queries** read. A query whose action declares effects is refused: it would change data every time it fetched. + An `evaluate` query runs no handler, so it may name any action. +- **Mutations** run actions. They declare `body`, `concurrency`, `serialKey`, `optimistic`, `onSuccess`, `onError` + and `offline`, and nothing about refreshing. +- **`optimistic`** writes a predicted row into a query's cached list as soon as the run starts, and rolls it back on + failure. It is for instant feedback and is independent of effects. +- **`refetch`** (`ui::refetch(query)`, or `{"refetch": "q"}`) refetches a query for a reason no action can declare. +- **`refreshEvery`** polls, for data that changes with time rather than through actions, such as a clock or an + external feed. + +What a screen may not declare: `invalidates`, `patch` and `refreshOn`. A document that names one is refused, pointing +at the key. They would restate, per screen, what the action's effects already say once for every client, and the two +would drift apart. The C++ builders offer none of them, so authored screens cannot name them. + +## Checklist for a new or changed action + +Work through it every time you add an action or change what one does. A coding agent should treat it as required. + +1. **Does the action change persistent data?** If not, declare nothing: it is a read. If a screen runs it as a + mutation anyway, declare `none`. +2. **Which record types does it change?** Include the knock-on ones: counts, summaries and other views of the same + rows. Each needs a `BRIDGE_REGISTER_RECORD` (``). +3. **For each, does the reply hold the changed rows with their keys?** It does when the result type is the record, or + holds it through members, `std::vector`, `std::optional` or `std::map`, and the record has a key. Then + `updates`. Otherwise `changes`. When unsure, `morph::effects::recordsOf("Action")` says what it holds. +4. **Is a record computed from others?** Register it with `keyless` and `derivedFrom` instead of adding it to + every action's effects. +5. **Would a key alone leak something to another user?** Add `privateKeys` to the record. +6. **Write `BRIDGE_REGISTER_EFFECTS` (``) directly under the action's + `BRIDGE_REGISTER_ACTION`.** +7. **Remove any refresh the screens did by hand**: an `onSuccess` refetch that the effects now cover. +8. **Build and run the screen tests.** A bad declaration fails when the server starts. Check that the bundle loads + with no warnings ([Testing](#testing)). + +## Errors and warnings, and their fixes + +**When the server starts.** Records and effects are validated together on the first read of the registries: in +`RemoteServer`'s constructor, or in local mode when the first screen is registered. A problem throws +`morph::model::EffectsRegistrationError`, whose message lists every problem, one per line. + +| Message (abridged) | Fix | +|---|---| +| ``action `A` declares updates on record `R`, which has no key`` | Give `R` a key (an `id` member, or `key<&R::member>`), or declare `changes`. | +| ``action `A` declares updates on record `R`, but its result holds no `R` `` | The reply does not carry the rows: declare `changes`. | +| ``action `A` declares changes, which is not a registered record`` (or `updates`; `T` is the compiler's type name) | Register `T` with `BRIDGE_REGISTER_RECORD`, or name the record type you meant. | +| ``action `A` declares different effects in two registrations`` | Keep one `BRIDGE_REGISTER_EFFECTS` per action. | +| ``action `A` declares none together with other effects`` | `none` stands alone. Drop it, or drop the others. | +| ``record `D` is derived from other records but has a key`` | A derived record is keyless: add `keyless` and remove the key. | +| ``record `D` is derived from S, which is not a registered record`` | Register `S`, or fix the `derivedFrom` type. | +| ``record `D` is derived from itself: D -> … -> D`` | Break the `derivedFrom` cycle. | +| ``record name `N` is registered for two types`` | Give each record type its own wire name. | +| ``type T is registered under two record names`` | Register each type once, under one name. | +| ``record `N` is registered twice with different options`` | The same record is registered with different options in two places, such as two headers. Keep one registration. | +| ``action `A` is registered by two action types; give each its own id`` | Two action types share an id in `BRIDGE_REGISTER_ACTION`. | + +**When it compiles.** A record with two `key<>` options, with both `keyless` and `key<>`, or whose key or `id` member is +not a `ModelKey`, fails a `static_assert`. So does an effects declaration that is empty or names something other than +`updates`, `changes`, `changesInstance` or `none`. + +**When a screen loads.** These come from `ui::loadDocument`, which also runs when a screen is registered on the +server, so a refused screen shows up in the server's own tests. + +| Message (abridged) | Fix | +|---|---| +| ``a query reads, so its action declares no effects; `A` declares …`` | Run `A` as a mutation, or, if it really only reads, remove its effects. | +| ``a document does not declare `invalidates` `` (or `patch`, or `refreshOn`) | Delete the key. Declare the action's effects instead, or use `refetch` for a refresh no action declares. | + +**Warnings.** A warning never refuses a document. Registration logs each one at `warn`. + +| Warning | Fix | +|---|---| +| ``nothing refreshes after `m` succeeds: `A` declares no effects, and no `refetch` follows it`` | Declare `A`'s effects, or `effects::none` if it changes nothing. | +| A `watch` whose `on` reads a query and whose `do` runs a mutation with effects | Another client's change would refetch the query, and the watch would run the mutation again, possibly in a loop between clients. Add `"source": "user"` so it runs only on the user's own edits. | + +## Testing + +- **The bundle loads cleanly.** Each rung has a test that loads every screen against the catalog the server serves + and expects no refusal and no warning + ([`examples/common/testkit/bundle_warnings.hpp`](../examples/common/testkit/bundle_warnings.hpp)): + + ```cpp + auto const bundle = bank::screens::bundle(); + REQUIRE(bundle.has_value()); + CHECK(morph::examples::testing::bundleLoadFindings(**bundle).empty()); + ``` + +- **Effects are declared.** Read them back in a unit test with `morph::effects::effectsOf("RenameTag")`, see what a + result holds with `morph::effects::recordsOf("ListBookmarks")`, or derive a change from a sample result with + `morph::effects::deriveChanges(actionId, resultJson, ranOn)`. +- **Two clients see each other.** + [`tests/net/test_push_two_clients.cpp`](../tests/net/test_push_two_clients.cpp) runs two sessions against one server + over a real socket: client A's mutation refreshes client B's mounted screen, with no refresh declared anywhere. +- **Check that a test can fail.** Remove one effect and confirm that the screen test which depends on it fails. A + passing test proves nothing until you have seen it fail. + +## Agents over MCP + +An AI agent acts on the application through the same server, on a connection of its own. Its calls get every check a +GUI's calls get, and every open screen watching what it changes refreshes. `morph::ui::McpAdapter` lists one tool +per action the agent's principal may run, plus an `evaluate` dry run. `serveStdio` serves it over MCP's stdio +transport. + +Bank's server serves one with `--mcp-stdio`: + +```sh +claude mcp add bank -e MORPH_MCP_PRINCIPAL=alice \ + -e BANK_DB="DRIVER=SQLite3;Database=$PWD/bank.db" -e BANK_PORT=0 \ + -- ./build/examples/bank/ladder_bank_server --mcp-stdio +``` + +See [`examples/bank/README.md`](../examples/bank/README.md) for the full setup, and +[`docs/spec/ui/agent.md`](spec/ui/agent.md) for the adapter. + +## Seeing it in the explorer + +The app explorer's facts for an action show its effects (`changes Bank_Account`, `updates Lims_Result`, +`changes its instance`, `none`, or `reads`) and the records its result holds, each with its key field and paths. Use +it to check what a screen will refresh before wiring one up. + +## Limits + +- **More than 4096 changed keys** from one execute are reported as a whole-type change (`kMaxChangeKeys`). +- **A connection watches at most 4096 topics** (`wire::kMaxWatchTopics`). Past that, a new topic is refused, and the + client is not told which. Its entries then refresh only from that client's own replies, on a `resync` and on + reconnect. +- **`updates` cannot move a row to a new key.** If an action replaces a row under a new key, declare `changes`, + as lims's re-capture does. +- **Effects are declared, not computed.** An action that changes different records depending on its input declares + all of them. +- **A keyed row without its key member** in a reply is skipped when the changed keys are collected. + +## Where the rules live + +| Question | Reference | +|---|---| +| Exactly what the record and effects macros accept and refuse | [`docs/spec/core/registry.md`](spec/core/registry.md), "Record registry" and "Effects" | +| The `watch`, `unwatch` and `event` wire kinds | [`docs/spec/core/wire.md`](spec/core/wire.md), "The `event` kind" | +| Fan-out, coalescing, the backlog and re-authorisation on the server | [`docs/spec/core/backend.md`](spec/core/backend.md), "Fan-out" | +| How the client cache applies replies and events | [`docs/spec/ui/interpreter.md`](spec/ui/interpreter.md), "Queries and the cache" | +| The document rules and warnings | [`docs/spec/ui/document.md`](spec/ui/document.md) | +| The MCP adapter | [`docs/spec/ui/agent.md`](spec/ui/agent.md) | +| The design, and why it is shaped this way | [`docs/superpowers/specs/2026-10-16-server-push-design.md`](superpowers/specs/2026-10-16-server-push-design.md) | diff --git a/docs/spec/core/registry.md b/docs/spec/core/registry.md index 00f2d68b3..7fbb2d343 100644 --- a/docs/spec/core/registry.md +++ b/docs/spec/core/registry.md @@ -1751,8 +1751,9 @@ for every action, which records its result holds. asserts the registration latch, compiles out under `MORPH_CLIENT_ONLY`, and defines no specialisation, so a translation unit that does not see it is consistent with one that does (the `morph_effects_unseen_links` probe links - two such halves). It belongs beside the action's registration, not in a - header, where it would run once per translation unit. + two such halves). It belongs beside the action's registration, in a header + or a source file: each translation unit that sees it registers the same + declaration, which is one. - **Idempotent per action.** Each action id keeps its distinct declarations; one equal to a stored one, in any order, is not added again. Two distinct declarations are a conflict, recorded rather than thrown so the registrar @@ -1786,9 +1787,10 @@ for every action, which records its result holds. two types, or one type registered twice with different options (key, sources, `privateKeys`). An identical registration repeated is one registration, as `registerActionOnce` treats a repeated action, so - `BRIDGE_REGISTER_RECORD` may sit in a model header. `BRIDGE_REGISTER_EFFECTS` - may not: it is also idempotent, but in a header it runs once per - translation unit for nothing; + `BRIDGE_REGISTER_RECORD` may sit in a model header. So may + `BRIDGE_REGISTER_EFFECTS`, beside the action's registration: a repeated + identical declaration is one, and a translation unit that does not see it + reads the same effects; - a keyed record with `derivedFrom`: a key-level change to a source does not say which derived rows moved, so a derived record is keyless; - a `derivedFrom` source that is not a registered record: it never changes, diff --git a/examples/IMPLEMENTATION.md b/examples/IMPLEMENTATION.md index c219715c3..652498b39 100644 --- a/examples/IMPLEMENTATION.md +++ b/examples/IMPLEMENTATION.md @@ -95,6 +95,14 @@ not building UIs; we are proving morph can drive them. - Screens follow [`TESTING.md`](TESTING.md) exactly: toolkit-free, served by the rung's server, drawn by the generic client. A screen names models, queries and mutations; it never decides. +- **Screens never say what to refresh.** Every action that changes data + declares its effects with `BRIDGE_REGISTER_EFFECTS` beside its registration, + over records registered with `BRIDGE_REGISTER_RECORD`, and every screen on + every client showing that data refreshes from them. An action a screen runs + that changes nothing declares `effects::none`. The rung's bundle loads with + no refusal and no warning. How to choose an effect, and the checklist for a + new action, are in + [`docs/KEEPING-SCREENS-CURRENT.md`](../docs/KEEPING-SCREENS-CURRENT.md). - **Zero styling effort.** Default Qt Quick controls, default fonts, no theming, no animations, no custom drawing. A rung that looks pretty has spent effort in the wrong place. @@ -296,3 +304,5 @@ Every rung PR states, in its description: documented, any shortfall recorded in the rung README, matrix green. 6. The rung README's design questions are resolved in writing ([`LADDER.md`](LADDER.md) discipline rule). +7. Every new or changed action that changes data declares its effects, and the + rung's bundle loads with no refusal and no warning (rule 2). diff --git a/examples/bank/screens/tests/test_bank_screens.cpp b/examples/bank/screens/tests/test_bank_screens.cpp index f132679e8..89dd3b4f5 100644 --- a/examples/bank/screens/tests/test_bank_screens.cpp +++ b/examples/bank/screens/tests/test_bank_screens.cpp @@ -105,8 +105,8 @@ TEST_CASE("bank screens: every screen validates, and the bundle builds", "[bank- REQUIRE(bundle.has_value()); } -// Every document loads against the catalog the server serves with no warning: each mutation's action declares its -// effects (or `none`), and no refresh declaration repeats what the effects already cover. +// Every document loads against the catalog the server serves with no refusal and no warning: each mutation's action +// declares its effects, or `none`. TEST_CASE("bank_bundle_loads_without_warnings", "[bank-screens]") { auto const bundle = bank::screens::bundle(); INFO((bundle ? std::string{} : bundle.error().message())); diff --git a/examples/bookmarks/screens/tests/test_bookmarks_screens.cpp b/examples/bookmarks/screens/tests/test_bookmarks_screens.cpp index 68959e0e7..0b7e73330 100644 --- a/examples/bookmarks/screens/tests/test_bookmarks_screens.cpp +++ b/examples/bookmarks/screens/tests/test_bookmarks_screens.cpp @@ -67,8 +67,8 @@ TEST_CASE("bookmarks screens: the screen validates, and the bundle builds", "[bo REQUIRE(bundle.has_value()); } -// Every document loads against the catalog the server serves with no warning: each mutation's action declares its -// effects (or `none`), and no refresh declaration repeats what the effects already cover. +// Every document loads against the catalog the server serves with no refusal and no warning: each mutation's action +// declares its effects, or `none`. TEST_CASE("bookmarks_bundle_loads_without_warnings", "[bookmarks-screens]") { auto const bundle = bookmarks::screens::bundle(); INFO((bundle ? std::string{} : bundle.error().message())); diff --git a/examples/lims/screens/tests/test_lims_screens.cpp b/examples/lims/screens/tests/test_lims_screens.cpp index 768de9953..bc405227f 100644 --- a/examples/lims/screens/tests/test_lims_screens.cpp +++ b/examples/lims/screens/tests/test_lims_screens.cpp @@ -62,8 +62,8 @@ TEST_CASE("lims screens: every screen validates, and the bundle builds", "[lims- REQUIRE(bundle.has_value()); } -// Every document loads against the catalog the server serves with no warning: each mutation's action declares its -// effects (or `none`), and no refresh declaration repeats what the effects already cover. +// Every document loads against the catalog the server serves with no refusal and no warning: each mutation's action +// declares its effects, or `none`. TEST_CASE("lims_bundle_loads_without_warnings", "[lims-screens]") { auto const bundle = lims::screens::bundle(); INFO((bundle ? std::string{} : bundle.error().message())); From eb52b96bc6120e93ddc75c09a7b0a629983dc0ae Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 08:39:04 +0200 Subject: [PATCH 67/68] Address code review feedback - core/remote.hpp: a coalesced `changes` event over the envelope cap is no longer dropped. It is sent with its records as whole-type changes, and as a `resync` if even that does not fit, so a watcher is never missed; test coalesced_event_over_the_cap_goes_as_whole_types (per Copilot). - examples/common/app: the connect listeners run in a task of their own, after the bridge's reconnect handler, so a mounted query's refetch waits for the rebind instead of using the old connection's model id; the ui-hello still goes first. Test remote_source_refetches_after_the_bridge_ rebinds replays a transport's reconnect over a populated query (per Copilot). - CMake: bank's screens are built whenever the bank server is, with MORPH_BUILD_EXAMPLES off too; their tests need the generic app and are built only with it (per Copilot). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- CMakeLists.txt | 4 ++ docs/spec/core/backend.md | 10 ++- examples/bank/screens/CMakeLists.txt | 5 +- examples/common/app/app_source.cpp | 17 ++++- examples/common/app/tests/test_sources.cpp | 81 +++++++++++++++++++++- include/morph/core/remote.hpp | 29 ++++++-- tests/test_push_watch.cpp | 20 ++++++ 7 files changed, 152 insertions(+), 14 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index e31deab37..0e25e4df6 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -1164,6 +1164,10 @@ if(MORPH_BUILD_EXAMPLES) if(MORPH_BUILD_QT_QUICK OR MORPH_BUILD_TUI) add_subdirectory(examples/client) endif() +elseif(TARGET ladder_bank_server) + # The bank server serves bank's screens, so it needs them without the generic examples; their tests, which use + # the generic app, are built only with them. + add_subdirectory(examples/bank/screens) endif() # ── Documentation ─────────────────────────────────────────────────────────── diff --git a/docs/spec/core/backend.md b/docs/spec/core/backend.md index 0c1975288..ac32d7f57 100644 --- a/docs/spec/core/backend.md +++ b/docs/spec/core/backend.md @@ -1235,8 +1235,9 @@ through `dispatchExecute`. `RemoteServer::reauthorise`, the one place a recipient's topics are narrowed. A body left with no topic is not sent. What is left is queued on the connection and sent merged with the connection's other changes - ([Coalescing](#coalescing)), written as `post` writes: dropped above - `wire::kMaxEnvelopeBytes`, held back behind a full backlog. + ([Coalescing](#coalescing)), written as `post` writes, held back behind a + full backlog. A merged event that would exceed `wire::kMaxEnvelopeBytes` is + never dropped: see *Size* under [Coalescing](#coalescing). - **Re-authorised at send time, by the rule that admitted it.** For each watched topic in a recipient's body: 1. the watch's stored session must still `authenticate` as the principal it @@ -1287,6 +1288,11 @@ through `dispatchExecute`. `detail::kCoalesceKeyLimit` (256: one per key, one per whole-type record, one per instance topic), and before a `resync` a fan-out owes the connection. A window of zero sends each fan-out at once. + - *Size.* The key count is bounded and the keys' size is not, so a merged + event can exceed `wire::kMaxEnvelopeBytes`. It is then sent with every + record as a whole-type change, which names what changed in a few bytes per + record type; one that still does not fit is sent as a `resync`. A watcher + is over-notified, never missed. - *What is not queued.* Nothing is queued for a connection with no writer, or one held back behind its backlog: its drain sends a `resync`, which covers every change since. A closed connection, and one whose writer is diff --git a/examples/bank/screens/CMakeLists.txt b/examples/bank/screens/CMakeLists.txt index d5fb29a30..2cb91be29 100644 --- a/examples/bank/screens/CMakeLists.txt +++ b/examples/bank/screens/CMakeLists.txt @@ -26,8 +26,9 @@ if(AF_COVERAGE) apply_coverage(bank_screens) endif() -# The screen harness lives in the ladder's testkit, so the screen tests are built with the ladder. -if(MORPH_BUILD_TESTS AND TARGET morph_ladder_testkit) +# The screen harness lives in the ladder's testkit, and the tests open screens through the generic app, so they are +# built with both. +if(MORPH_BUILD_TESTS AND TARGET morph_ladder_testkit AND TARGET morph::examples_common) include(Catch) add_executable(bank_screen_tests tests/test_bank_screens.cpp "${PROJECT_SOURCE_DIR}/examples/common/testkit/testkit_main.cpp") diff --git a/examples/common/app/app_source.cpp b/examples/common/app/app_source.cpp index 1f843f9da..5f641d96e 100644 --- a/examples/common/app/app_source.cpp +++ b/examples/common/app/app_source.cpp @@ -443,10 +443,21 @@ void ClientSource::connected() { hello.body = ui::detail::writeMessage(clientHello()); _exchange(std::move(hello), [](ui::ExchangeReply const&) {}); } - std::vector> const listeners = _connectListeners; - for (auto const& listener : listeners) { - listener(); + // The listeners re-watch and refetch. Posted, not run here: a transport posts the bridge's reconnect handler to + // this same owner right after the connect hook, so it runs first, clears the old connection's model ids and starts + // rebinding, and a refetch then waits for its new binding. The `ui-hello` above still goes out before any watch. + if (_bridge == nullptr) { + return; } + _bridge->owner().post([this, alive = std::weak_ptr{_alive}] { + if (auto const live = alive.lock(); !live || !*live) { + return; + } + std::vector> const listeners = _connectListeners; + for (auto const& listener : listeners) { + listener(); + } + }); } void ClientSource::onConnected(std::function const& listener) { _connectListeners.push_back(listener); } diff --git a/examples/common/app/tests/test_sources.cpp b/examples/common/app/tests/test_sources.cpp index f94cb8a4d..f1f6ce570 100644 --- a/examples/common/app/tests/test_sources.cpp +++ b/examples/common/app/tests/test_sources.cpp @@ -6,6 +6,7 @@ #include // clang-format on +#include #include #include #include @@ -14,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -21,6 +23,7 @@ #include #include #include +#include #include #include #include @@ -474,15 +477,58 @@ TEST_CASE("Client shell: the frame's navigator opens a screen as a tab", "[examp namespace { -// A local backend whose connect hook the test fires, as a transport does after each connect. +// A local backend whose connect hook the test fires, as a transport does after each connect. `reconnect` replays a +// transport's reconnect: the connect hook, then the bridge's reconnect handler posted to its executor. It logs each +// bind and execute, in order. class ConnectingBackend final : public morph::backend::LocalBackend { public: using LocalBackend::LocalBackend; void setConnectHandler(std::function const& handler) override { *_handler = handler; } + void setReconnectHandler(std::function handler, morph::exec::IExecutor* exec) override { + _reconnect = std::move(handler); + _reconnectExec = exec; + } [[nodiscard]] std::shared_ptr> handler() const { return _handler; } + morph::async::Completion bindModel(morph::backend::detail::BindRequest request, + morph::exec::IExecutor& cbExec) override { + note("bind"); + return LocalBackend::bindModel(std::move(request), cbExec); + } + void executeInto(morph::exec::detail::ModelId mid, morph::backend::detail::ActionCall call, + morph::exec::IExecutor* cbExec, + std::shared_ptr sink) override { + note("execute"); + LocalBackend::executeInto(mid, std::move(call), cbExec, std::move(sink)); + } + + void reconnect() { + (*_handler)(); + if (_reconnect && _reconnectExec != nullptr) { + _reconnectExec->post(_reconnect); + } + } + + std::vector calls() { + std::scoped_lock const lock{_mutex}; + return _calls; + } + void clearCalls() { + std::scoped_lock const lock{_mutex}; + _calls.clear(); + } + private: + void note(std::string call) { + std::scoped_lock const lock{_mutex}; + _calls.push_back(std::move(call)); + } + std::shared_ptr> _handler = std::make_shared>(); + std::function _reconnect; + morph::exec::IExecutor* _reconnectExec = nullptr; + std::mutex _mutex; + std::vector _calls; }; // The probe application behind a server-like transport: the UI service answers the `ui-*` kinds in process, the @@ -573,3 +619,36 @@ TEST_CASE("remote_source_rehellos_then_rewatches_on_reconnect", "[examples-commo } source.close(); } + +// Mutation: run the connect listeners in the connect hook's own task, ahead of the bridge's reconnect handler — the +// mounted query refetches through the old connection's binding before the rebind. +TEST_CASE("remote_source_refetches_after_the_bridge_rebinds", "[examples-common][source][push]") { + FakeAppContext context; + morph::ui::testing::RecordingBackend screens; + ReconnectingSource source; + ShellLog log; + { + auto const shell = + morph::examples::mountClientShell(context, screens, openAndWait(context, source), source, log.options()); + REQUIRE(log.mounted == std::vector{"probe.home"}); + REQUIRE(source.backend != nullptr); + // The screen's query has run, through a bound private instance. + REQUIRE(pumpUntil(context.owner(), [&] { + auto const calls = source.backend->calls(); + return std::ranges::find(calls, "execute") != calls.end(); + })); + + source.backend->clearCalls(); + source.backend->reconnect(); + REQUIRE(pumpUntil(context.owner(), [&] { + auto const calls = source.backend->calls(); + return std::ranges::find(calls, "execute") != calls.end(); + })); + // The refetch waits for the new binding. + auto const calls = source.backend->calls(); + REQUIRE_FALSE(calls.empty()); + CHECK(calls.front() == "bind"); + CHECK(log.problems.empty()); + } + source.close(); +} diff --git a/include/morph/core/remote.hpp b/include/morph/core/remote.hpp index 0025ea6cb..30b6daae3 100644 --- a/include/morph/core/remote.hpp +++ b/include/morph/core/remote.hpp @@ -1328,13 +1328,30 @@ class RemoteServer : public std::enable_shared_from_this { if (outbox.queued.empty()) { return; } - std::string payload; - (void)glz::write_json(outbox.queued.take(), payload); - auto envelope = - ::morph::wire::encode(::morph::wire::makeEvent(::morph::wire::kChangesTopic, std::move(payload))); - if (!oversized(cid, envelope)) { - postHere(cid, std::move(envelope)); + ::morph::wire::ChangesBody body = outbox.queued.take(); + std::string envelope = changesEnvelope(body); + // The key count is bounded, the keys' size is not: past the envelope cap, the same records go as whole-type + // changes, which name what changed in a few bytes each. A watcher must never miss a change, so what still + // does not fit is a `resync`. + if (envelope.size() > ::morph::wire::kMaxEnvelopeBytes) { + for (::morph::effects::RecordChange& change : body.records) { + change.keys.reset(); + } + envelope = changesEnvelope(body); + } + if (oversized(cid, envelope)) { + envelope = ::morph::wire::encode(::morph::wire::makeEvent(::morph::wire::kResyncTopic, "{}")); } + postHere(cid, std::move(envelope)); + } + + /// @brief The encoded `changes` event carrying @p body. + /// @param body The changes. + /// @return The envelope. + [[nodiscard]] static std::string changesEnvelope(::morph::wire::ChangesBody const& body) { + std::string payload; + (void)glz::write_json(body, payload); + return ::morph::wire::encode(::morph::wire::makeEvent(::morph::wire::kChangesTopic, std::move(payload))); } /// @brief Body of `post`. On the server strand. diff --git a/tests/test_push_watch.cpp b/tests/test_push_watch.cpp index b22dc48b5..f75de2c2b 100644 --- a/tests/test_push_watch.cpp +++ b/tests/test_push_watch.cpp @@ -1553,6 +1553,26 @@ TEST_CASE("updates_keys_merge_once_each", "[push][watch]") { CHECK(keysOf(body.records.front()) == std::vector{"1", "2", "3"}); } +// Mutation: drop a coalesced event over the envelope cap, as an oversized post is — the watcher never hears of +// either change. +TEST_CASE("coalesced_event_over_the_cap_goes_as_whole_types", "[push][watch]") { + constexpr auto kWindow = std::chrono::milliseconds{20}; + FanRig rig{std::chrono::milliseconds{0}, nullptr, kWindow}; + auto const cid = rig.connect(); + rig.watch(cid, records({"T_FanRow"})); + // Two keys that each fit an envelope, and together do not. + std::string const big = "\"" + std::string((::morph::wire::kMaxEnvelopeBytes / 2) + 16, 'k') + "\""; + rig.fanOut({rowChange({big}), rowChange({big + " "})}, cid); + REQUIRE(::morph::testing::waitUntil([&] { return !rig.wire(cid).snapshot().empty(); })); + rig.flush(); + + auto const bodies = rig.wire(cid).changes(); + REQUIRE(bodies.size() == 1U); + REQUIRE(bodies.front().records.size() == 1U); + CHECK(bodies.front().records.front().record == "T_FanRow"); + CHECK_FALSE(bodies.front().records.front().keys.has_value()); +} + TEST_CASE("flush_at_256_keys_without_waiting", "[push][watch]") { FanRig rig{std::chrono::milliseconds{0}, nullptr, kForever}; auto const cid = rig.connect(); From 5a0a61a67eb397c04b661a93033bdc477a6f88ab Mon Sep 17 00:00:00 2001 From: Yaraslau Tamashevich Date: Mon, 19 Oct 2026 10:06:38 +0200 Subject: [PATCH 68/68] coverage: move the document.hpp allowlist hints to their current lines Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01A4hwcANrTbPYnqmWX4iyMZ --- scripts/branch_partial_allowlist.json | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/scripts/branch_partial_allowlist.json b/scripts/branch_partial_allowlist.json index 355035fef..5a4c64fa5 100644 --- a/scripts/branch_partial_allowlist.json +++ b/scripts/branch_partial_allowlist.json @@ -121,28 +121,28 @@ }, { "file": "include/morph/ui/document.hpp", - "line": 1643, + "line": 1604, "source": "default:", "context": "download(cmd, fields, primary, path);", "reason": "The default arm is unreachable: the switch names every enumerator of CommandKind, so no value reaches `default`. The project's strict warnings require both at once -- -Wswitch-enum wants every enumerator listed and -Wswitch-default wants a default label -- so the default shares the last enumerator's body and is never the label taken. It is DocumentParser::command's switch over the command kinds." }, { "file": "include/morph/ui/document.hpp", - "line": 1969, + "line": 1930, "source": "default:", "context": "case Shape::Keys:", "reason": "The default arm is unreachable: the switch names every enumerator of detail::Shape, so no value reaches `default`. The project's strict warnings require both at once -- -Wswitch-enum wants every enumerator listed and -Wswitch-default wants a default label -- so the default shares the last enumerator's body and is never the label taken. It is DocumentParser::entry's switch over a key's shape." }, { "file": "include/morph/ui/document.hpp", - "line": 2957, + "line": 2887, "source": "default:", "context": "if (cmd.value) {", "reason": "The default arm is unreachable: the switch names every enumerator of CommandKind, so no value reaches `default`. The project's strict warnings require both at once -- -Wswitch-enum wants every enumerator listed and -Wswitch-default wants a default label -- so the default shares the last enumerator's body and is never the label taken. It is DocumentValidator::checkCommand's switch." }, { "file": "include/morph/ui/document.hpp", - "line": 3363, + "line": 3222, "source": "default: {", "reason": "The default arm is unreachable: the switch names every enumerator of NameCategory, so no value reaches `default`. The project's strict warnings require both at once -- -Wswitch-enum wants every enumerator listed and -Wswitch-default wants a default label -- so the default shares the last enumerator's body and is never the label taken. It is ScopeResolver::ref's switch over what a name declares." },