From 85d8b577c2a56e5cf45320156711a9390d33c2db Mon Sep 17 00:00:00 2001 From: ditadi Date: Thu, 22 Oct 2026 15:00:35 +0100 Subject: [PATCH 1/3] docs: design explicit database route operations Signed-off-by: ditadi --- designs/002_DatabaseRouteOperations.md | 684 +++++++++++++++++++++++++ 1 file changed, 684 insertions(+) create mode 100644 designs/002_DatabaseRouteOperations.md diff --git a/designs/002_DatabaseRouteOperations.md b/designs/002_DatabaseRouteOperations.md new file mode 100644 index 000000000..f80016820 --- /dev/null +++ b/designs/002_DatabaseRouteOperations.md @@ -0,0 +1,684 @@ +# 002 - Database route operations + +| | | +|---|---| +| **Status** | Draft, awaiting operator validation | +| **Owner** | Victor Ditadi | + +[Google Doc](https://docs.google.com/document/d/1zO0X7bCwYtmX6m-p7DFGZJneSa5OsgVd5nctGhCdVZY/edit) +is the comment copy. Approved edits there are synced to this file. + +This proposal lets an app author expose a database operation without duplicating column types or Zod validators. +Authorization stays explicit. The database plugin exposes nothing by default. For an authored operation, the table +and HTTP method infer the ordinary input and public response. Custom contracts are optional overrides. + +Every new API below is proposed. No example has been compiled or run. This PR changes only this design document. +Implementation planning and implementation wait for operator validation. + +## Background and motivation + +At baseline `670764d742db6963da9d6929da58560cce18e228`, `database()` loads `config/database/schema.ts` and publishes +`appkit.database`. There is no `config/schema.ts` or `appkit.db`. Its HTTP registration has a different safety model +from its trusted server API. + +| Area | Current behavior | Source | +|---|---|---| +| Exposure | An omitted `api`, `true`, or `{}` enables generated CRUD for all declared tables. `{ tables }` also enables writes unless narrowed separately. `false` disables HTTP only. | `packages/appkit/src/plugins/database/crud/exposure.ts:93-146` | +| Routes | `injectRoutes` registers list, detail, create, update, and delete handlers without per-user authorization. | `packages/appkit/src/plugins/database/database.ts:126-184`, `packages/appkit/src/plugins/database/crud/routes.ts` | +| Credentials | One Lakebase pool uses the app's service principal. Request context never implicitly changes that credential. | `packages/appkit/src/connectors/lakebase/index.ts:29-62` | +| Identity | `createRequestScope` copies forwarded headers and has development fallbacks. It does not establish the proxy's header-stripping contract. | `packages/appkit/src/context/request-scope.ts:26-73` | +| OBO | `database` and `postgres` are app-only resources. Database execution inside an ordinary caller scope fails with `APP_ONLY_RESOURCE`. | `packages/shared/src/schemas/manifest.ts:69-73`, `packages/appkit/src/context/resource-capabilities.ts:63-77` | +| Validation | Trusted insert and update schemas admit private columns. The narrower HTTP capabilities still grow with public columns. | `packages/appkit/src/database/schema-builder/validators.ts:46-71`, `packages/appkit/src/plugins/database/crud/capabilities.ts:32-55` | +| Execution | Interceptors run cache, retry, timeout, telemetry, then the callback. Cache can bypass the callback. Database defaults disable retry and do not enable a result cache. | `packages/appkit/src/plugin/plugin.ts:556-611`, `packages/appkit/src/plugins/database/defaults.ts` | +| Errors | The shared database error vocabulary lacks 401. Generic execution and telemetry can observe an exception before the HTTP handler receives its result. | `packages/shared/src/database/errors.ts`, `packages/appkit/src/plugin/plugin.ts:488-518`, `packages/appkit/src/plugin/interceptors/telemetry.ts` | +| Types | Database typegen loads the schema and writes `shared/appkit-types/database.d.ts`. Server and browser types use separate registry bindings. | `packages/appkit/src/type-generator/database/generate.ts`, `packages/appkit/src/type-generator/database/walk-schema.ts` | +| Packaging | The server preset builds the server entry and agent entries with `unbundle: true`. It has no database-operation discovery. | `packages/appkit/src/tsdown/index.ts:43-81` | +| Browser | The client resolves URLs from registered endpoints and raises `NOT_EXPOSED` without a request for a missing entry. | `packages/appkit-ui/src/js/database/client.ts:175-198` | + +The playground currently limits writes to `boards` and `notes`, but that setting still leaves reads on `note_events` +(`apps/dev-playground/server/index.ts:74-81`). A policy on a new route would not protect the generated route beside it. + +Public PR [#682](https://github.com/databricks/appkit/pull/682) is an unmerged operations-first POC with +`defineQuery`, `defineMutation`, and `definePolicy`. This proposal uses it as prior art, not as the baseline. It closes +legacy exposure, adds schema-derived contracts and client types, and specifies production packaging. +Design [001](https://github.com/databricks/appkit/pull/677) separately addresses database development and migrations. +This proposal preserves its service-principal execution model. + +## Goals + +1. The database plugin registers only explicitly authored operations with authorization. Custom Express routes remain + the application author's responsibility. +2. Authentication precedes policy lookup and execution. Authorization always precedes `execute`. +3. Column metadata supplies types, defaults, and validation without a duplicate Zod schema for ordinary writes. +4. Reuse the schema's public and writable field metadata instead of requiring a second field list per operation. +5. Source development and a source-less production build use the same selected operation modules. +6. Custom operations have typed browser calls and generic React hooks without importing server code. +7. Turning operations off never restores generated CRUD. + +## Out of scope + +- RLS and per-user database credentials. The application server enforces authorization. +- A framework-wide authorization DSL, role store, policy combinators, or migration of every plugin's policy API. +- A server-code sandbox, a new read-only fluent database API, or an automatic locking abstraction. +- Custom URL patterns, uploads, streaming, batch operations, and automatic idempotency. +- Operation result caches and automatic retries. +- Automatic user-scoped data access or forwarding a user token into an operation context. Resolving caller identity + during authentication does not change the database credential. + +## Product decisions + +### No implicit or legacy database HTTP routes + +Remove generated table CRUD and the `api` field. There is no migration guard, compatibility flag, or deprecated alias. +`database({ routes: databaseRoutes })` exposes operations. Absent `routes`, or `routes: false`, exposes nothing. +An untyped old option cannot enable a route because the old registration loop is removed. +Trusted `appkit.database` access and explicit transactions remain. Entity lifecycle hooks and `serialize` are removed. +The operator approved this removal and accepted breaking changes without a compatibility layer. + +The operator accepted removal without compatibility work and requires first-party database apps to adopt the new +operation routes. The app cutover and deletion of their generated CRUD paths happen together, not through a permanent +compatibility router. No configuration of the removed API can restore an unauthenticated alternate. + +### Required authorization has runtime enforcement too + +TypeScript requires `authorize`. For untyped declarations, a missing or invalid `authorize` skips the operation. +Boot logs its file and export name. It never supplies an allow-all default. Recommend making this condition fatal in +production builds and CI while preserving the required skip-and-warn behavior at runtime. + +### Infer ordinary contracts and make overrides optional + +The operator confirmed that the table and method infer CRUD input and output. GET takes the public key. POST takes +the create body with schema-derived required fields, optional fields, and defaults. PATCH takes the key and changes. +No `input`, `output`, field list, or `name` is required for those operations. + +An explicit Zod `input` overrides the conventional request contract. An explicit Zod `output` describes and validates +a non-default response. `execute` owns response mapping. A non-row response needs that output override unless a +future generator can prove its client type by inspecting the executor. V1 makes no such arbitrary-return claim. + +The schema's HTTP capabilities exclude private, generated, and non-updatable fields. They are not authorization. +Adding a public column can expand inferred contracts. App authors must mark server-owned data appropriately and use +an optional narrower input when an operation should accept only a subset, such as a title-only rename. + +### Keep one explicit place for business behavior + +Remove `beforeCreate`, `afterCreate`, `beforeUpdate`, `afterUpdate`, `beforeUpsert`, `afterUpsert`, `beforeDelete`, +`afterDelete`, and `serialize`, including their public types and serializer-only client overrides. Keep React data +hooks, wire encoding, and private-column projection. They have independent responsibilities. + +Operation-specific behavior belongs in `execute`. Shared behavior belongs in an ordinary domain function called by +operations and trusted server callers. Writes and audit records that need atomicity use `db.transaction` explicitly. +There is no replacement lifecycle framework or automatic transaction around every executor. + +This removal is approved. A direct entity write no longer invokes normalization or audit callbacks. Removed fields +simply disappear from the type and implementation. There is no special runtime handling of a leftover `hooks` key. + +The empty events panel in the subtraction slice is not the final app migration. The playground's new note-creation +operation must explicitly create the note and its `note_events` row in one transaction. Migrate every first-party +app that uses database to the operation API. Do not recreate hooks or add a one-off endpoint outside that API. + +### Conceal resource denials by default + +An absent actor gets 401. A resource policy's missing resource, denial, malformed result, or exception gets the same +opaque 404. A policy without a resource lookup denies with 403. An explicit fixed denial can choose 403 or 404. +No denial includes a caller-supplied error message. The concealment default awaits validation. + +## Architecture decisions + +### One registry owns registration + +The organizing data shape is an immutable `OperationRegistry`, indexed by operation id and canonical method plus +URL. The exact public id and URL format remains open. Each entry binds its schema-derived contract, authorization, +executor, and diagnostic source location. +`Policy` separates the actor, validated input, and optional resource snapshot. There is one authorization evaluator. + +The explicit `name` property is removed. One candidate is to derive `.`, so `export const +renameTask` for `tasks` would become `tasks.renameTask` at `/api/database/operations/tasks.renameTask`. Under that +candidate, moving files would preserve identity while renaming the export or entity would change the API. The operator +has not confirmed this public format. Examples below illustrate the candidate, not an approved routing contract. + +Validate and bound identifiers, compare canonical routes with Express-equivalent case and slash semantics, and reject +duplicate ids or aliases of the same descriptor before anything mounts. No entity-based CRUD aliases remain. + +Rejected alternatives: + +- A policy map on generated CRUD needs a separate mechanism for custom actions and can preserve implicit exposure. +- Separate `defineQuery` and `defineMutation` builders duplicate the method distinction without solving inference. +- A mandatory explicit `name` repeats identity already present in the export. The operator requested its removal. + No replacement public-id format or stable-alias configuration is approved yet. +- An `action` field duplicates the conventional method mapping. A custom action supplies an optional input override. +- A legacy `api` flag violates the off-state guarantee. + +### Table and method infer the default request and response + +| Method without overrides | Validated input | Default output | +|---|---|---| +| GET | `{ id }` using the public key's canonical value type | One public row, or 404 | +| POST | The HTTP create body, preserving required fields, optionality, and defaults | The created public row | +| PATCH | `{ id, changes }` with a non-empty HTTP update payload | The updated public row, or 404 | +| PUT | `{ id, values }` for a full public replacement payload | The resulting public row, or 404 | +| DELETE | `{ id }` | 204 on true, or 404 on false | + +These contracts use the schema's HTTP facets, not the trusted validators that admit private fields. Entities without +an addressable public key need an explicit input for methods that otherwise require one. List and summary operations +are custom GETs with an input override and a non-row output override. Their export names do not change inference. +PUT replacement and schema-default semantics must be pinned in the contract prototype before implementation. + +Wire input and callback input differ. Reuse the existing codecs in +`packages/appkit/src/plugins/database/crud/codecs.ts` and the `WireInput` and `WireOutput` adapters in +`packages/shared/src/database/api-types.ts`. Bigint inputs are decimal strings or safe integers on the wire and +`bigint` after decoding. Bigint outputs are decimal strings. The browser never stringifies native bigint. + +Optional Zod input replaces the conventional contract. Object inputs must be strict, including nested contracts. +Unknown fields are rejected. V1 supports JSON-renderable structural schemas and representable input defaults. +Transforms, preprocessors, coercion, `any`, `unknown`, and unsupported constructs fail generation. + +### Output is an optional contract, not a serializer callback + +Without `output`, runtime rebuilds the conventional response through the table's public projection and codecs. It +never forwards the trusted row object directly. Existing safe HTTP encoding remains after callback removal. + +With `output`, the executor maps to the custom value and the runtime validates that value against the supplied Zod +schema. V1 requires JSON-value output with strict objects and no transforms, coercion, defaults, or catches. Client +types derive from that schema. Invalid executor output is opaque 500. Importing server modules or emitting `unknown` +is not a fallback for an unsupported custom schema. + +Output validation after commit does not undo a write. If a malformed result must roll back, map and validate it inside +the explicit transaction. SQL rollback never undoes an LLM request, email, or other external side effect. + +### Callbacks and reusable policies share one evaluator + +`authorize` accepts an async callback or `definePolicy({ resource?, allow })`. An inline callback normalizes to a +no-resource policy. It receives `resource: undefined`. A resource policy runs its lookup once after authentication +and parsing, skips `allow` on null, and passes the same non-null resource to `allow` and `execute`. + +Only literal `true` grants. `false`, malformed values, rejected promises, and thrown exceptions deny. A policy can +return `{ deny: "forbidden" }` or `{ deny: "not-found" }`. Internal diagnostics use fixed reasons, not arbitrary +client-facing text. + +The resource is a snapshot, not a lock. A state-dependent mutation must repeat its complete authorization predicate +in a conditional write or re-read and lock the relevant rows in a transaction. A transaction at the default isolation +level does not automatically make an earlier lookup authoritative. + +Policy descriptors expose readonly `resource` and `allow` functions. Application tests can call them with explicit +actor and database fixtures without enabling an HTTP actor override. The dispatcher and actor injection seam remain +internal. Supporting direct policy tests does not add a network bypass. + +The Files plugin already has async boolean policies and `FilePolicyUser.id` +(`packages/appkit/src/plugins/files/policy.ts`). Use the same `actor.id` spelling. Database policies additionally need +validated operation input, resource loading, and concealed denials. Reuse those concepts, not Files' SP fallback or +its action and volume-specific combinators. Unifying both APIs is outside this change. + +### Actor identity is not a database credential + +`Actor` contains an authenticated end user's canonical workspace SCIM `id` and optional email. Email is not an +ownership key. The registry's dispatcher never turns an actor into `CallerContext`, never chooses an OBO pool, and +rejects an inherited caller scope rather than silently elevating it to app execution. + +Context contains the ordinary service AppKit handle and `db`, an alias for `appkit.database`. It contains neither the +request nor a user token. Trusted server code can still close over other handles or deliberately call OBO APIs for +other resources. This is not a sandbox or a claim that such calls are impossible. The database credential remains the +service principal, and its app-only guard stays in force. + +Recommended v1 actor integrations: + +- On Apps, do not assume `X-Forwarded-User` is a SCIM id. The public header contract describes an IdP identifier. + A supported canonical path is to validate the request's forwarded user credential through the workspace Current + User `Me` API, using identity-read scopes, and use the returned id. This authenticates the caller; database queries + still use the service principal. It does not open `CallerContext`. Missing or invalid request credentials deny + before lookup or execution. The verified resolver and actual Apps integration still require implementation proof. + Header presence and deployment environment variables alone never establish an actor. +- In explicit local development, resolve the signed-in developer through the existing local identity loader in + `packages/shared/src/dev-obo.ts`. Copy only `id` and email into the actor, never its token. Check the socket, + host, origin, and fetch metadata on every operation request, even if forwarded headers exist. Extract the existing + loopback predicate from `packages/appkit/src/plugins/server/dev-obo-middleware.ts`, not its header short-circuit. + Follow its explicit credential and `APPKIT_DEV_OBO` enable rules. Missing credentials gives 401. Forwarded headers + never choose the local actor. A local process or tunnel with loopback access is within the developer's trust + boundary, so this mode must not be exposed through forwarding or remote tunnels. + +A recognized Apps boundary takes precedence over development mode. There is no `dev-user` or service-principal actor. +Canonical identity must come from the same authority in Apps and local development. The public sources are +[Apps authorization](https://docs.databricks.com/aws/en/dev-tools/databricks-apps/auth), +[forwarded headers](https://docs.databricks.com/aws/en/dev-tools/databricks-apps/http-headers), and +[Current User Me](https://docs.databricks.com/api/workspace/currentuser/me). Forwarding a user credential for identity +validation is not OBO database execution. Support for other hosting remains an explicit scope question. + +### Generate one server-only static registry + +Discovery occurs in `appkit generate-types`, not in plain Node at startup. It scans `server/database/routes/*.ts` +in deterministic order, imports each module for contract inspection, and writes a committed server registry. +Named tagged declarations are inspected. Ordinary helper exports are ignored, and a tagged default export is rejected. +Modules must be safe to import without a running app. + +```ts +// server/database/routes.gen.ts, generated +import { schema } from "../../config/database/schema"; +import * as tasks from "./routes/tasks"; + +export const databaseRoutes = { + schema, + files: { "server/database/routes/tasks.ts": tasks }, +} as const; +``` + +The app statically imports `databaseRoutes` and passes it to `database({ routes: databaseRoutes })`. The registry +imports the schema too, so the intended production graph does not depend on source files or runtime Jiti. +`routes.schema` supplies the plugin schema. A distinct explicit `schema` is an error. + +Generation alone writes the registry and client declarations. Setup is read-only and runs the same descriptor +validator. Import namespaces deliberately retain tagged invalid declarations so boot can name and warn about a missing +`authorize`, even when typegen already omitted that operation from client types. Only valid entries reach endpoints. +The generated file shows the eligible modules. The operation source and generated declaration show callable exports. + +Use existing atomic file replacement, not a new manifest store, lease, or generation-pointer protocol. CI and +production `prebuild` run proposed `appkit generate-types --wait --strict --check`. Check mode verifies committed +artifacts without replacing them. Drift, invalid authorization, or generation errors fail the build. Development +writes current artifacts and warns for skipped authorization. A broken generation clears client operation types and +must not publish a candidate dispatcher. + +In dev, the watcher observes schema, routes, additions, removals, and imported dependencies. It stops dispatch before +rebuilding or restarting. It resumes only after the current registry validates. A failed generation cannot keep an +old authorized operation serving. Existing Vite watching and `tsx watch` are starting points, not proof that this +lifecycle already works. There is no production hot reload. + +A static import graph, schema inclusion, exact emitted paths, and guarded startup diagnostics need a source-less +bundle proof. Static import failures can occur before plugin setup. Do not assume the current dynamic schema loader +sanitizes those Node startup errors. The production bootstrap must report a bounded startup failure without printing +module-thrown secrets before the privacy contract is accepted. + +Rejected alternatives are runtime TS scanning, a manual module list, a second `routes: true` mode, and a registry +under the client's `shared/appkit-types` directory. The explicit generated import is the only opt-in. + +### Authentication and error handling own the HTTP boundary + +A namespace adapter covers `/api/database/operations` before the global body parser and frontend fallback. It +resolves the actor before decoding or dispatch, including unknown operation names and wrong methods. Unauthenticated +calls get 401. Authenticated calls to an unknown name or method get opaque 404. + +The current `skipBodyParsing` set matches exact strings, while Express normally accepts case and trailing-slash +variants (`packages/appkit/src/plugins/server/index.ts:144-153`). It is insufficient by itself. The server integration +must exclude the operation namespace with matching semantics, then apply its bounded parser after authentication. +Use the configured body limit. Tests cover canonical, mixed-case, trailing-slash, and unknown routes. Other plugin +namespaces keep their behavior. + +GET carries one bounded JSON `input` query value. Reject repeated values, unexpected query parameters, invalid JSON, +and oversized input through one operation decoder. Other methods use bounded JSON bodies and require +`application/json`. GET executors must be side-effect-free by API contract. The library cannot prove arbitrary +callback purity. Sensitive input belongs in a non-GET operation because URLs also reach browser and proxy logs. + +Cookie-backed mutations require a verified same-origin or equivalent CSRF boundary. JSON content type alone is not +that proof. Use a trusted deployment origin, not unverified forwarded headers. Missing platform confirmation blocks +release of the integration. + +All operation failures end in the operation response adapter. They are not forwarded as raw exceptions to generic +Express error handling. Unknown properties and invalid contracts yield value-free validation details. + +### Interceptors cannot bypass authorization + +The sequence is actor, parse, resource, allow, execute, output. Authorization stays outside `Plugin.execute`. +The executor wrapper forces cache and retry off and enables telemetry. It has no generic timeout or user override. +Entity operations continue using their existing execution defaults and explicit transaction context. No mutation +hook or serializer is invoked. + +Crucially, the callback passed into `Plugin.execute` catches and classifies errors before that plugin's logger and +telemetry interceptor observe them. Preserve a known `DatabasePluginError` category. Convert other executor errors +to a safe operation error without the original message, cause, or row data. Retain only bounded diagnostics outside +the exception. Mapping the returned status and discarding its message afterward would be too late. + +A lookup or `allow` exception always denies, including a classified database outage. This is the operator's +fail-closed rule. The internal stage distinguishes an outage from a deliberate denial without changing the response. +An executor's database error keeps its normal category. + +The database's statement and transaction limits are not an end-to-end request deadline. A callback awaiting non-SQL +I/O can still hang in v1 and must supply its own dependency deadline. No JavaScript timeout, client disconnect, or +AbortSignal is claimed to cancel SQL. Automatic retries could repeat committed mutations and remain disabled. + +### Flag strategy + +`routes?: DatabaseRoutes | false` is the only behavior switch. Its default is off. An empty validated registry also +serves nothing. Turning it off removes operations at the next start, publishes no endpoints, and never enables CRUD. +Types, policy helpers, code generation, and unreachable internals need no feature flag. The first user-reachable +slice must introduce this switch together with authorization enforcement. Strict generation is a build check, not a +second runtime exposure flag. Actor-integration selection is an authentication boundary, not an enable flag. + +## Main design + +### What an app author writes + +Assume a schema entity `tasks` with numeric `id`, `title`, `done`, and private server-controlled `owner_id`. These +are proposed examples, not the current playground schema. + +```ts +// server/database/policies.ts +import { definePolicy, type PolicyContext } from "@databricks/appkit/beta"; + +export const authenticated = definePolicy({ allow: () => true }); + +export const ownerOnly = definePolicy({ + resource: ({ actor, db, input }: PolicyContext<{ id: number }>) => + db.tasks.where({ owner_id: actor.id }).find(input.id), + allow: () => true, +}); +``` + +```ts +// server/database/routes/tasks.ts +import { defineOperation } from "@databricks/appkit/beta"; +import { z } from "zod"; +import { authenticated, ownerOnly } from "../policies"; + +export const createTask = defineOperation({ + entity: "tasks", + method: "POST", + authorize: authenticated, + execute: ({ actor, db, input }) => db.tasks.create({ ...input, owner_id: actor.id }), +}); + +export const renameTask = defineOperation({ + entity: "tasks", + method: "PATCH", + authorize: ownerOnly, + execute: ({ actor, db, input, resource }) => + db.tasks.where({ owner_id: actor.id }).update(resource.id, input.changes), +}); + +export const listTasks = defineOperation({ + entity: "tasks", + method: "GET", + input: z.strictObject({ limit: z.number().int().min(1).max(100).default(20) }), + output: z.array(z.strictObject({ id: z.number(), title: z.string() })), + authorize: authenticated, + execute: async ({ actor, db, input }) => { + const rows = await db.tasks.where({ owner_id: actor.id }).limit(input.limit).toArray(); + return rows.map(({ id, title }) => ({ id, title })); + }, +}); +``` + +`ownerOnly` can serve any compatible input containing `id`, including a custom action with an explicit Zod schema. +Its lookup already filters by the actor's owner id. Only a non-null owned row reaches `allow`. The owner condition is +repeated in the mutation. If ownership changes after the lookup, a null result becomes 404. +A policy checking additional mutable state must also protect that state at write time. `renameTask` accepts the +schema's ordinary update fields, not only `title`. A title-only contract would supply an optional narrower input. + +```ts +// server/server.ts +import { database } from "@databricks/appkit/beta"; +import { databaseRoutes } from "./database/routes.gen"; + +const registration = database({ routes: databaseRoutes }); +``` + +The existing app passes this registration to `createApp`. One file can export several operations. The folder and +registry do not register helpers or automatically create an operation per table. + +### API changes + +The intended signature sketch is not compiler-verified. Registry-derived helper types follow the input and output +tables above. Inference, variance, and contextual typing are proof obligations, not established behavior. + +```ts +type Awaitable = T | Promise; +type Actor = Readonly<{ id: string; email?: string }>; +type Decision = boolean | { readonly deny: "forbidden" | "not-found" }; +type HttpMethod = "GET" | "POST" | "PUT" | "PATCH" | "DELETE"; + +type PolicyContext = Readonly<{ + actor: Actor; + input: I; + appkit: AppKitHandle; + db: DatabaseExports; +}>; +type OperationContext = PolicyContext & { readonly resource: R }; + +declare function definePolicy(spec: { + resource(ctx: PolicyContext): Awaitable; + allow(ctx: OperationContext>): Awaitable; +}): ResourcePolicy>; +declare function definePolicy(spec: { + allow(ctx: OperationContext): Awaitable; +}): PlainPolicy; + +declare function defineOperation< + const E extends EntityName, + const M extends HttpMethod, + I extends ZodType | undefined = undefined, + O extends ZodType | undefined = undefined, + R = undefined, +>(spec: { + readonly entity: E; + readonly method: M; + readonly input?: I; + readonly output?: O; + readonly authorize: Policy, R> | InlineAuthorize>; + execute(ctx: OperationContext, R>): Awaitable>; +}): Operation; +``` + +An inline callback has no resource. A resource policy supplies its inferred non-null resource. The final overloads +must enforce that distinction. `InputOf` and `ResultOf` choose the method/table default when no override exists and +the Zod contract otherwise. Discovery supplies export identity after definition. `entity` is a generated schema key. +With custom contracts it is the primary entity label, not a claim that execution touches only that table. + +`defineOperation` brands a descriptor without throwing for missing authorization. Otherwise import would abort +before discovery could report the ignored export. Shared validation runs during generation and setup. Other malformed +contracts fail the candidate registry instead of receiving defaults. + +Add server helpers and types to `@databricks/appkit/beta`, add `routes?: DatabaseRoutes | false`, and add +`UNAUTHENTICATED` to the shared database error vocabulary. Determine the typed AppKit handle available through plugin +context before implementing the public context. No `appkit.db` API is introduced. Remove `hooks`, its lifecycle and +serializer types, `serialized()`, and serializer-only `shape` options. Do not add compatibility checks or migration +warnings for removed options. Existing structural config validation remains independent of this removal. + +### Request outcomes + +| Condition | HTTP result | May execute? | +|---|---|---| +| Missing or unverified actor under a supported integration | 401 `UNAUTHENTICATED` | No | +| Authenticated unknown operation or wrong method | 404 `NOT_FOUND` | No | +| Bad encoding, oversized input, or wrong content type | 400, 413, or 415 | No | +| Schema failure, extra field, or empty update | 422 `VALIDATION_FAILED` | No | +| Resource missing, or lookup throws | Opaque 404 | No | +| Resource policy denies, throws, or returns a malformed decision | Opaque 404 by default | No | +| No-resource authorization denies or throws | 403 `FORBIDDEN` by default | No | +| Explicit fixed deny | 403 or 404 | No | +| Conventional POST succeeds | 201 with the public created row | Once | +| Conventional DELETE succeeds | 204 without a body | Once | +| Other allowed callback succeeds | 200 with its inferred or custom response | Once | +| Conventional row output receives null | 404 `NOT_FOUND` | Already ran | +| Executor throws a known database error | Its safe database category | Already ran | +| Other executor error or invalid output | 500 `INTERNAL` | Already ran | + +Every response uses `Cache-Control: no-store`. Reusing `FORBIDDEN` covers both a policy denial and database privilege +failure, but stage and reason distinguish them internally. No raw error text reaches the response adapter. + +### Boot and reload outcomes + +| Condition | Required behavior | +|---|---| +| `routes` absent or false | Zero database endpoints. Trusted server API remains. | +| Untyped removed `api` option present | It cannot enable anything. The old route registration path does not exist. | +| Source routes folder missing or empty | Generation produces an explicit empty registry. | +| Tagged operation has missing or invalid authorization | Retain diagnostic declaration, skip HTTP registration, warn with `file#export`. Strict generation fails. | +| Ordinary helper export | Ignore. | +| Bad contract, unknown entity, duplicate, or import failure | Fail generation or setup. Publish no candidate router. | +| Expected generated registry or compiled module missing | Fail build or boot. Never reinterpret it as an empty source folder. | +| Failure after opening a pool | Close the candidate pool and publish nothing. | +| Dev rebuild fails or authorization is removed | Disable dispatch before rebuild. No last-good serving registry. | +| Production source changes on disk | No hot reload. Serving state changes only through a new validated start. | + +Boot can log, for example, `server/database/routes/tasks.ts#purgeTasks skipped: authorize is missing`. +`createApp` already awaits setup before listening (`packages/appkit/src/core/appkit.ts:304-327`). That ordering does +not prove the proposed watcher or build integration. + +### Generate browser types without server imports + +Derive the browser contract from the same inspected exports, entity/method defaults, and optional Zod overrides. Emit a +global operation registry in `shared/appkit-types/database.d.ts`, analogous to the current entity bridge. It contains +wire input, serialized output, and method per operation. No declaration imports a route, server schema, or policy. + +Default types use existing column metadata and wire adapters. Custom schemas use Zod's JSON Schema conversion +plus a bounded TypeScript renderer. Unsupported constructs fail generation. Do not emit `unknown` or infer arbitrary +executor return types as a fallback. + +Boot metadata publishes only registered names, methods, and URLs. The existing endpoint map stores paths, so method +metadata is new. It never publishes policies, schema internals, tokens, private fields, or the server registry. + +Proposed browser calls are: + +```ts +const tasks = await databaseApi.call("tasks.listTasks", { limit: 20 }); +const renamed = await databaseApi.call("tasks.renameTask", { id: 42, changes: { title: "Reviewed" } }); +``` + +`useDatabaseQuery(name, input)` accepts GET names. `useDatabaseMutation(name)` accepts non-GET names. These generic +hooks replace the CRUD hooks and reuse cancellation and mutation lifecycle code where its guarantees fit. No hook is +generated per operation. A missing endpoint raises `NOT_EXPOSED` without a request. + +V1 does not share results or pending requests across hooks. On a session transition, clear operation state and reject +late results from the previous session. A supported session-change signal is an integration proof obligation, not an +assumption that an in-page identity never changes. Successful mutations invalidate active queries broadly instead of +introducing a dependency DSL. Mutations never automatically retry. A lost response reports proposed +`OUTCOME_UNKNOWN`, not permission to resend. + +## Observability + +One `database.operation` span covers actor resolution through output handling, including denials. It records +operation, entity, method, canonical route, stage, outcome, and a fixed denial reason. Logs use those fields and a +trace id. Skips log file and export at warn. Exceptions log bounded class or category information, not their message, +cause, actor, email, input, or resource. + +This requires more than an allowlist on the custom span: + +- Sanitize executor failures inside the execution callback before `Plugin.execute` and `TelemetryInterceptor` log or + record them. A sentinel-secret test must inspect actual exported logs and spans. +- Redact query-bearing `url.query`, `http.target`, `http.url`, and equivalent HTTP attributes for operation traffic. + Renaming the span does not remove attributes. The current HTTP instrumentation changes names only + (`packages/appkit/src/telemetry/instrumentations.ts`). +- Prevent Lakebase SQL text and raw driver exception details from leaking operation data. The current connector + records SQL and exceptions (`packages/lakebase/src/pool.ts:87-117`). Operation-path redaction is a release gate. +- Bound startup diagnostics for import failures before claiming sanitized boot errors. + +Do not claim repository-wide telemetry is already sanitized. Other plugins and external proxy logs need their own +contracts. GET URLs must contain no secrets even after AppKit's own trace redaction. + +## Security considerations + +The browser, its input, and any client-sent identity fields are untrusted. The server author, policy code, and +executor are trusted and have the service principal's database privilege. Required authorization prevents omission, +not a logically incorrect policy. RLS is explicitly excluded, so PostgreSQL supplies no per-user safety net. + +| Threat | Enforcement or limitation | +|---|---| +| Omitted policy or forgotten route restriction | Required `authorize`, runtime skip, strict build, and no generated CRUD. | +| Spoofed actor | Verified deployment integration, no header-presence inference, no local synthetic actor. | +| Another user's row | Resource policy and conditional writes. List executors must also constrain their rows. | +| New public owner or role field | HTTP schema metadata excludes private and generated fields. Public writable additions widen defaults and require schema review or a narrower override. | +| Private response field | Runtime public projection or strict custom output. Public columns widen default responses. An author can still copy a secret into a declared custom field. | +| Cache bypass or cross-user reuse | Authorization outside interceptors, no server cache, no cross-hook sharing, session-bound client state. | +| Duplicate mutation | No automatic client or server retry. Repeated user submission still needs application idempotency. | +| Ownership or state changes after lookup | Resource is a snapshot. Protect the complete predicate in the write or transaction. | +| Authorization outage | Lookup and policy exceptions deny. Logs distinguish the failure without revealing it to callers. | +| Stale dev authorization | Dispatch stops before rebuild and restarts only with a valid current registry. | +| CSRF or GET mutation | Verified mutation origin boundary and side-effect-free GET contract. JSON alone is insufficient. | +| Expensive or hung callbacks | Bounded transport and SQL limits. Non-SQL dependency deadlines remain the author's responsibility in v1. | +| Server secrets in the browser | Server-only registry and structural declarations. Add a client import restriction and inspect the actual bundle. | +| Unprotected custom Express routes | Outside the database plugin's guarantee. Trusted server code can bypass these building blocks. | + +## Platform dependencies + +| Ask | Why | +|---|---| +| Confirm stripping and injection of Apps user headers, including SP-only deployments, and name the supported deployment boundary. | Client-supplied headers or environment presence cannot establish the actor. | +| Verify the forwarded user credential and identity-read scopes needed for workspace `Me`, or provide an equally verified canonical identity adapter. | IdP header strings are not documented as SCIM ids. Ownership needs the same authority in dev and production. | +| Confirm the trusted external origin, cookie and CSRF protections, and session transition signal. | Mutation authentication and browser state must remain bound to the current user. | + +These are engineering release prerequisites for the Apps integration. They are not facts established by this repo and +not product preferences for the operator to guess. + +## Rollout plan + +1. Apply the approved subtraction first. Remove hooks, `serialize`, serializer-only client helpers, and their demo + wiring in an independent slice. Preserve explicit transactions, ordinary CRUD contracts, and HTTP protections. + The operator accepted breaking changes and requires no compatibility layer. Other operation behavior still waits + for validation of the remaining design questions. +2. Run the type, Zod, packaging, identity, transport, and reload proofs below. Record artifacts and revise the design + if a proof fails. Do not silently replace a missing proof with a type assertion or fallback router. +3. After validation, plan independently verifiable slices. Types and unreachable internals need no behavior flag. + The first reachable operation path must have the explicit registry opt-in and no unprotected legacy alternate. +4. Migrate repository callers, docs, template, playground, and browser APIs to the current operation model. Preserve + the playground's note-plus-event behavior through an explicit executor transaction. Do not add legacy adapters or + migration-only handling. Never release an intermediate version that reopens CRUD when operations are off. +5. Release only after the negative behavior matrix, source-less bundle, real Apps identity, and telemetry privacy + gates pass. Production rollout is a separate approved plan. + +Required implementation evidence includes: + +| Area | Later proof | +|---|---| +| Authentication | Missing identity, forged headers, dev disable rules, unsupported hosting, and malformed body all fail before lookup or execute. | +| Authorization | False, malformed return, null lookup, thrown lookup, thrown allow, async allow, explicit deny, and reused resource all have the specified status and execute count. | +| Defaults | Missing policy is absent from routes and warns at boot. Strict CI fails. Neither flag state can enable legacy CRUD. | +| Inference | Method/table defaults without required name/input/output, discovery ids after format validation, incompatible policy input, non-null resource, bigint keys and writes, strict overrides, and unsupported Zod schemas. | +| Output | Default public projection excludes private columns. Explicit custom contracts reject undeclared fields. Invalid custom output is opaque 500. | +| Subtraction | Hooks, serializer exports, and removed fields are gone without compatibility guards. Explicit transaction commit/rollback/join/expiry/isolation and deadline remain covered. | +| Interceptors | Denials never enter execute, a transient mutation runs once, and no actor receives another actor's cached result. | +| Transport | Case and slash variants, unknown names and methods, repeated query values, limits, content type, and CSRF boundary. | +| Discovery | Add, remove, remove authorization, import error, duplicate, stale artifact, failed restart, and generated-file drift. | +| Production | Plain Node serves compiled operations with `server/` and `config/` source trees absent. Inspect client and server bundles. | +| Client | Missing endpoint sends nothing. Session switch rejects old pending results. Lost mutation response never retries. | +| Privacy | Sentinel input, row values, and exception messages do not reach exported operation, HTTP, execution, or Lakebase telemetry. | +| Integration | Dev playground exercises allowed and denied users through the actual HTTP path. Vitest assertions alone are not live identity proof. | + +After implementation changes, run `pnpm build && pnpm docs:build` and +`pnpm check:fix && pnpm -r typecheck`, alongside the relevant Vitest and integration checks. None ran for this design. + +## PR stack + +1. `00-design`. This document only. Its draft pull request is the review entry point. +2. `01-remove-hooks`. Approved removal of entity hooks, `serialize`, serializer-only client APIs, and their callers. + This does not introduce or enable operation routes. +3. `02-operation-definitions`. Typed descriptors, inferred method/table contracts, reusable policy definitions, and + real package-consumer compile probes. No loader, actor resolver, or HTTP registration. +4. Operation execution, discovery, client generation, and the app cutover follow as independently verified slices. + The final cutover migrates every first-party database app and removes its old route path. + +The exact public routing format remains open. Defining metadata and types does not settle or expose that format. + +## Open questions + +The settled requirements are mandatory authorization, skip-and-warn for missing authorization at boot, 401 for an +unauthenticated caller, no execute after a lookup or authorization exception, and no RLS. The operator also confirmed +removal of explicit `name`, inferred CRUD input/output with optional overrides, and removal of hooks and `serialize`. +Breaking changes and removal without compatibility shims are accepted. The operator explicitly rejected special +runtime handling of removed fields and requires apps to move to the new routes, not stay as an empty demo. + +### Product decisions for validation + +1. Use concealed 404 for resource-policy denials and 403 otherwise, with fixed explicit overrides? Recommend yes. + Required reviewers are AppKit maintainers and security. +2. Make strict, non-mutating generated-artifact verification mandatory in production builds and CI? Recommend yes, + while runtime retains the required skip-and-warn behavior. Required reviewers are AppKit maintainers. +3. Limit v1 actor integrations to verified Apps and explicit local development, failing setup elsewhere? Recommend + yes initially, without claiming other hosting is impossible. Required reviewers are AppKit and Apps maintainers. + A concrete additional deployment can justify a verified adapter later. +4. What exact public operation id and URL should replace the explicit `name`? The export-derived candidate is not + confirmed. Required reviewers are the operator and AppKit maintainers. This choice does not block hook removal. + +### Engineering proof obligations + +Local throwaway compiler experiments support one generic method-indexed signature for GET, POST, PATCH, and DELETE, +optional Zod overrides, and an annotated reusable resource policy. Unsuppressed negative controls reject missing +`authorize`, private default input, and incompatible policy inputs. These experiments used narrow contract mappings, +not the published package entry, so package-consumer proof is still required. One overload experiment was inconclusive. + +Do not copy the experiments' gaps into the API. Separate nullable executor results from successful wire rows, map +bigint on the wire, and use an empty DELETE response rather than its internal boolean. Keep resource and `allow` +types correlated; do not accept an unrelated callback through a `never` parameter. A standalone reusable lookup may +need one input-context annotation. Runtime projection is required even when the type names only public columns. + +Before release, prove Zod wire rendering, source-less imports, the actual AppKit context, pre-parser ordering, +fail-closed watching, canonical actor resolution, session behavior, and exported telemetry redaction. Those outcomes +are not established by type-checking a scratch model and are not operator preferences to guess. + +The root `designs/` convention comes from open PR #677 rather than merged main. Recheck the design number in open PRs +before publication and merge. From 834001a57eb991af732b5ab223bd73a7d3a9af09 Mon Sep 17 00:00:00 2001 From: ditadi Date: Tue, 3 Nov 2026 15:16:47 +0000 Subject: [PATCH 2/3] docs: align database operation design with implemented experience Signed-off-by: ditadi --- designs/002_DatabaseRouteOperations.md | 925 ++++++++++--------------- 1 file changed, 350 insertions(+), 575 deletions(-) diff --git a/designs/002_DatabaseRouteOperations.md b/designs/002_DatabaseRouteOperations.md index f80016820..d4265ebb8 100644 --- a/designs/002_DatabaseRouteOperations.md +++ b/designs/002_DatabaseRouteOperations.md @@ -2,683 +2,458 @@ | | | |---|---| -| **Status** | Draft, awaiting operator validation | +| **Status** | In review; implemented in the open operation stack | | **Owner** | Victor Ditadi | +| **Design PR** | [#683](https://github.com/databricks/appkit/pull/683), first in the stack | [Google Doc](https://docs.google.com/document/d/1zO0X7bCwYtmX6m-p7DFGZJneSa5OsgVd5nctGhCdVZY/edit) -is the comment copy. Approved edits there are synced to this file. +is the linked comment copy. This repository document records the current implementation and supersedes its earlier +proposal for inferred CRUD inputs, export-derived names, and mandatory registry imports. -This proposal lets an app author expose a database operation without duplicating column types or Zod validators. -Authorization stays explicit. The database plugin exposes nothing by default. For an authored operation, the table -and HTTP method infer the ordinary input and public response. Custom contracts are optional overrides. - -Every new API below is proposed. No example has been compiled or run. This PR changes only this design document. -Implementation planning and implementation wait for operator validation. +An app exposes database access to the browser through named, authorized operations. `database()` discovers operations +in the conventional directory; declaring tables alone exposes no HTTP endpoint. Inputs are explicit schemas or absent. +Outputs retain useful entity/method defaults, with explicit schemas for lists and custom responses. Browser calls and +React hooks are typed without importing server modules. ## Background and motivation -At baseline `670764d742db6963da9d6929da58560cce18e228`, `database()` loads `config/database/schema.ts` and publishes -`appkit.database`. There is no `config/schema.ts` or `appkit.db`. Its HTTP registration has a different safety model -from its trusted server API. +The earlier database beta generated table CRUD routes from schema declarations. Those routes used the app's service +principal and had no required per-operation user authorization. Adding a protected custom route beside them did not +protect the generated route. Lifecycle hooks also spread business behavior between table declarations and callers. -| Area | Current behavior | Source | -|---|---|---| -| Exposure | An omitted `api`, `true`, or `{}` enables generated CRUD for all declared tables. `{ tables }` also enables writes unless narrowed separately. `false` disables HTTP only. | `packages/appkit/src/plugins/database/crud/exposure.ts:93-146` | -| Routes | `injectRoutes` registers list, detail, create, update, and delete handlers without per-user authorization. | `packages/appkit/src/plugins/database/database.ts:126-184`, `packages/appkit/src/plugins/database/crud/routes.ts` | -| Credentials | One Lakebase pool uses the app's service principal. Request context never implicitly changes that credential. | `packages/appkit/src/connectors/lakebase/index.ts:29-62` | -| Identity | `createRequestScope` copies forwarded headers and has development fallbacks. It does not establish the proxy's header-stripping contract. | `packages/appkit/src/context/request-scope.ts:26-73` | -| OBO | `database` and `postgres` are app-only resources. Database execution inside an ordinary caller scope fails with `APP_ONLY_RESOURCE`. | `packages/shared/src/schemas/manifest.ts:69-73`, `packages/appkit/src/context/resource-capabilities.ts:63-77` | -| Validation | Trusted insert and update schemas admit private columns. The narrower HTTP capabilities still grow with public columns. | `packages/appkit/src/database/schema-builder/validators.ts:46-71`, `packages/appkit/src/plugins/database/crud/capabilities.ts:32-55` | -| Execution | Interceptors run cache, retry, timeout, telemetry, then the callback. Cache can bypass the callback. Database defaults disable retry and do not enable a result cache. | `packages/appkit/src/plugin/plugin.ts:556-611`, `packages/appkit/src/plugins/database/defaults.ts` | -| Errors | The shared database error vocabulary lacks 401. Generic execution and telemetry can observe an exception before the HTTP handler receives its result. | `packages/shared/src/database/errors.ts`, `packages/appkit/src/plugin/plugin.ts:488-518`, `packages/appkit/src/plugin/interceptors/telemetry.ts` | -| Types | Database typegen loads the schema and writes `shared/appkit-types/database.d.ts`. Server and browser types use separate registry bindings. | `packages/appkit/src/type-generator/database/generate.ts`, `packages/appkit/src/type-generator/database/walk-schema.ts` | -| Packaging | The server preset builds the server entry and agent entries with `unbundle: true`. It has no database-operation discovery. | `packages/appkit/src/tsdown/index.ts:43-81` | -| Browser | The client resolves URLs from registered endpoints and raises `NOT_EXPOSED` without a request for a missing entry. | `packages/appkit-ui/src/js/database/client.ts:175-198` | - -The playground currently limits writes to `boards` and `notes`, but that setting still leaves reads on `note_events` -(`apps/dev-playground/server/index.ts:74-81`). A policy on a new route would not protect the generated route beside it. - -Public PR [#682](https://github.com/databricks/appkit/pull/682) is an unmerged operations-first POC with -`defineQuery`, `defineMutation`, and `definePolicy`. This proposal uses it as prior art, not as the baseline. It closes -legacy exposure, adds schema-derived contracts and client types, and specifies production packaging. -Design [001](https://github.com/databricks/appkit/pull/677) separately addresses database development and migrations. -This proposal preserves its service-principal execution model. +This design replaces that HTTP surface while preserving trusted `appkit.database` access, schema-derived server types, +private-column projection, wire codecs, and explicit transactions. The operations-first POC in +[#682](https://github.com/databricks/appkit/pull/682) is prior art, rather than the implementation baseline. + +Design [001](https://github.com/databricks/appkit/pull/677) covers database branches, migrations, and deployment. Its +migration chain and this operation chain are separate. This stack checks that declared tables and columns exist at +startup; it does not create or migrate them. Integrating the migration flow is separate work before both ship together. ## Goals -1. The database plugin registers only explicitly authored operations with authorization. Custom Express routes remain - the application author's responsibility. -2. Authentication precedes policy lookup and execution. Authorization always precedes `execute`. -3. Column metadata supplies types, defaults, and validation without a duplicate Zod schema for ordinary writes. -4. Reuse the schema's public and writable field metadata instead of requiring a second field list per operation. -5. Source development and a source-less production build use the same selected operation modules. -6. Custom operations have typed browser calls and generic React hooks without importing server code. -7. Turning operations off never restores generated CRUD. +1. Register only authored operations with authorization, with no alternate generated CRUD endpoint. +2. Verify the caller before parsing input, then authorize before executing database work. +3. Make no-parameter operations simple to declare and call, without an empty Zod object. +4. Keep useful row-output defaults while making list, aggregate, and custom responses explicit. +5. Use conventional discovery in development and compiled registries in production. +6. Generate browser contracts and provide generic React hooks with clear read, write, and session behavior. +7. Keep transactional business rules visible in application code. ## Out of scope -- RLS and per-user database credentials. The application server enforces authorization. -- A framework-wide authorization DSL, role store, policy combinators, or migration of every plugin's policy API. -- A server-code sandbox, a new read-only fluent database API, or an automatic locking abstraction. -- Custom URL patterns, uploads, streaming, batch operations, and automatic idempotency. -- Operation result caches and automatic retries. -- Automatic user-scoped data access or forwarding a user token into an operation context. Resolving caller identity - during authentication does not change the database credential. +- RLS, per-user database credentials, and automatic user-scoped database queries. +- A framework-wide role store, authorization DSL, policy combinators, or a server-code sandbox. +- Custom URL patterns, uploads, streaming, batching, automatic idempotency, and shared operation caches. +- Automatic retries, an automatic transaction around every operation, or rollback of external side effects. +- A replacement lifecycle-hook or response-serializer framework. ## Product decisions -### No implicit or legacy database HTTP routes +### Operations are the only database HTTP surface -Remove generated table CRUD and the `api` field. There is no migration guard, compatibility flag, or deprecated alias. -`database({ routes: databaseRoutes })` exposes operations. Absent `routes`, or `routes: false`, exposes nothing. -An untyped old option cannot enable a route because the old registration loop is removed. -Trusted `appkit.database` access and explicit transactions remain. Entity lifecycle hooks and `serialize` are removed. -The operator approved this removal and accepted breaking changes without a compatibility layer. +Remove the legacy `api` option, AutoCRUD routes, entity-based browser CRUD calls and hooks, lifecycle hooks, +`serialize`, and serializer-only shape overrides. There is no compatibility router or alias. Disabling operations never +restores table CRUD. Trusted server entity clients, `sql`, and `transaction` remain. -The operator accepted removal without compatibility work and requires first-party database apps to adopt the new -operation routes. The app cutover and deletion of their generated CRUD paths happen together, not through a permanent -compatibility router. No configuration of the removed API can restore an unauthenticated alternate. +Business behavior belongs in `execute` or an ordinary domain function shared with trusted server callers. Related +writes and audit records use an explicit transaction. Constraints that must hold for every writer belong in PostgreSQL. -### Required authorization has runtime enforcement too +### Conventional routes require no registry import -TypeScript requires `authorize`. For untyped declarations, a missing or invalid `authorize` skips the operation. -Boot logs its file and export name. It never supplies an allow-all default. Recommend making this condition fatal in -production builds and CI while preserving the required skip-and-warn behavior at runtime. +The three supported configurations are: -### Infer ordinary contracts and make overrides optional +```ts +database(); // Discover the conventional generated registry. +database({ routes: false }); // Trusted server database access only. +database({ routes: customRoutes }); // Use an explicit registry for another layout. +``` -The operator confirmed that the table and method infer CRUD input and output. GET takes the public key. POST takes -the create body with schema-derived required fields, optional fields, and defaults. PATCH takes the key and changes. -No `input`, `output`, field list, or `name` is required for those operations. +The conventional source is `server/database/routes/*.ts`, through `server/database/routes.gen.ts`. The registry is +still generated and committed; app authors do not need to import it into the server entry. No operations means no HTTP +endpoint. Outside development, an existing routes directory without its registry fails setup with generation guidance. +Import failures and invalid registries never fall back to a different router. -An explicit Zod `input` overrides the conventional request contract. An explicit Zod `output` describes and validates -a non-default response. `execute` owns response mapping. A non-row response needs that output override unless a -future generator can prove its client type by inspecting the executor. V1 makes no such arbitrary-return claim. +### Public names are explicit and stable -The schema's HTTP capabilities exclude private, generated, and non-updatable fields. They are not authorization. -Adding a public column can expand inferred contracts. App authors must mark server-owned data appropriately and use -an optional narrower input when an operation should accept only a subset, such as a title-only rename. +Every operation requires `name`, such as `"notes.list"`, and answers its declared method at +`/api/database/operations/`. Moving a file or renaming its export does not change the public API. +Names contain dot-separated parts made of letters, digits, `_`, and `-`; each part starts with a letter and has at most +64 characters, with at most 255 characters in total. `constructor` and `prototype` are reserved names. Duplicate names +fail validation, including a duplicate whose declaration lacks authorization. -### Keep one explicit place for business behavior +### No input schema means no parameters -Remove `beforeCreate`, `afterCreate`, `beforeUpdate`, `afterUpdate`, `beforeUpsert`, `afterUpsert`, `beforeDelete`, -`afterDelete`, and `serialize`, including their public types and serializer-only client overrides. Keep React data -hooks, wire encoding, and private-column projection. They have independent responsibilities. +An absent `input` gives callbacks `undefined` and generates a browser input type of `void`. It never infers an ID, +create body, or patch from the entity or method. An operation that needs parameters declares its Zod schema. -Operation-specific behavior belongs in `execute`. Shared behavior belongs in an ordinary domain function called by -operations and trusted server callers. Writes and audit records that need atomicity use `db.transaction` explicitly. -There is no replacement lifecycle framework or automatic transaction around every executor. +```tsx +useDatabaseQuery("alerts.queue"); +useDatabaseQuery("alerts.get", { id: "123" }); -This removal is approved. A direct entity write no longer invokes normalization or audit callbacks. Removed fields -simply disappear from the type and implementation. There is no special runtime handling of a leftover `hooks` key. +const refresh = useDatabaseMutation("alerts.refresh"); +await refresh.mutate(); +``` -The empty events panel in the subtraction slice is not the final app migration. The playground's new note-creation -operation must explicitly create the note and its `note_events` row in one transaction. Migrate every first-party -app that uses database to the operation API. Do not recreate hooks or add a one-off endpoint outside that API. +No-parameter requests may omit input or send an empty object, which normalizes to `undefined`. Other input fails before +policy lookup. An explicit schema controls parsing, defaults, optional properties, and unknown-key handling according to +its Zod semantics. Every object is not implicitly made strict. Server-owned values, such as ownership, come from the +verified actor and domain logic rather than client input. -### Conceal resource denials by default +### Output defaults remain -An absent actor gets 401. A resource policy's missing resource, denial, malformed result, or exception gets the same -opaque 404. A policy without a resource lookup denies with 403. An explicit fixed denial can choose 403 or 404. -No denial includes a caller-supplied error message. The concealment default awaits validation. +| Method | Executor result without `output` | Successful response | +|---|---|---| +| `GET` | Public entity row or `null` | Row, 200; `null` gives 404 | +| `POST` | Created public entity row | Row, 201 | +| `PATCH` / `PUT` | Updated public entity row or `null` | Row, 200; `null` gives 404 | +| `DELETE` | Boolean | `true` gives 204 with no body; `false` gives 404 | -## Architecture decisions +The default row projection excludes private columns and relations. Existing codecs encode bigint as decimal strings +and timestamps as RFC 3339 text. Adding a public column can widen a default response, so schema review still matters. +An explicit output schema provides a narrower or different response; `execute` owns the mapping into it. -### One registry owns registration +Lists, aggregates, custom responses, and successful nullable responses declare `output`. The presence of `{ id }` in +input never determines whether the answer is one object or an array. A GET list with no input needs only its array +output schema. A nullable output can return `null` with 200; otherwise a null GET/PATCH/PUT result gives 404. -The organizing data shape is an immutable `OperationRegistry`, indexed by operation id and canonical method plus -URL. The exact public id and URL format remains open. Each entry binds its schema-derived contract, authorization, -executor, and diagnostic source location. -`Policy` separates the actor, validated input, and optional resource snapshot. There is one authorization evaluator. +### Authorization is required at both boundaries -The explicit `name` property is removed. One candidate is to derive `.`, so `export const -renameTask` for `tasks` would become `tasks.renameTask` at `/api/database/operations/tasks.renameTask`. Under that -candidate, moving files would preserve identity while renaming the export or entity would change the API. The operator -has not confirmed this public format. Examples below illustrate the candidate, not an approved routing contract. +TypeScript requires `authorize`. If an otherwise valid untyped declaration omits it, runtime setup and normal generation +skip it and warn with `file#export`. Production generation fails instead. An invalid authorization value, malformed +contract, unknown entity, duplicate name, or module import failure blocks the registry. A missing policy is never an +allow-all default, and skipping a policy does not hide another malformed field. -Validate and bound identifiers, compare canonical routes with Express-equivalent case and slash semantics, and reject -duplicate ids or aliases of the same descriptor before anything mounts. No entity-based CRUD aliases remain. +## Author experience -Rejected alternatives: +The following examples assume declared `notes` and `note_events` tables, with a numeric note ID and string author/body +columns. Generate the table types before authoring operations. -- A policy map on generated CRUD needs a separate mechanism for custom actions and can preserve implicit exposure. -- Separate `defineQuery` and `defineMutation` builders duplicate the method distinction without solving inference. -- A mandatory explicit `name` repeats identity already present in the export. The operator requested its removal. - No replacement public-id format or stable-alias configuration is approved yet. -- An `action` field duplicates the conventional method mapping. A custom action supplies an optional input override. -- A legacy `api` flag violates the off-state guarantee. +```ts +// server/database/policies.ts +import { definePolicy, type PolicyContext } from "@databricks/appkit/beta"; -### Table and method infer the default request and response +export const signedIn = definePolicy({ allow: () => true }); -| Method without overrides | Validated input | Default output | -|---|---|---| -| GET | `{ id }` using the public key's canonical value type | One public row, or 404 | -| POST | The HTTP create body, preserving required fields, optionality, and defaults | The created public row | -| PATCH | `{ id, changes }` with a non-empty HTTP update payload | The updated public row, or 404 | -| PUT | `{ id, values }` for a full public replacement payload | The resulting public row, or 404 | -| DELETE | `{ id }` | 204 on true, or 404 on false | - -These contracts use the schema's HTTP facets, not the trusted validators that admit private fields. Entities without -an addressable public key need an explicit input for methods that otherwise require one. List and summary operations -are custom GETs with an input override and a non-row output override. Their export names do not change inference. -PUT replacement and schema-default semantics must be pinned in the contract prototype before implementation. - -Wire input and callback input differ. Reuse the existing codecs in -`packages/appkit/src/plugins/database/crud/codecs.ts` and the `WireInput` and `WireOutput` adapters in -`packages/shared/src/database/api-types.ts`. Bigint inputs are decimal strings or safe integers on the wire and -`bigint` after decoding. Bigint outputs are decimal strings. The browser never stringifies native bigint. - -Optional Zod input replaces the conventional contract. Object inputs must be strict, including nested contracts. -Unknown fields are rejected. V1 supports JSON-renderable structural schemas and representable input defaults. -Transforms, preprocessors, coercion, `any`, `unknown`, and unsupported constructs fail generation. - -### Output is an optional contract, not a serializer callback - -Without `output`, runtime rebuilds the conventional response through the table's public projection and codecs. It -never forwards the trusted row object directly. Existing safe HTTP encoding remains after callback removal. - -With `output`, the executor maps to the custom value and the runtime validates that value against the supplied Zod -schema. V1 requires JSON-value output with strict objects and no transforms, coercion, defaults, or catches. Client -types derive from that schema. Invalid executor output is opaque 500. Importing server modules or emitting `unknown` -is not a fallback for an unsupported custom schema. - -Output validation after commit does not undo a write. If a malformed result must roll back, map and validate it inside -the explicit transaction. SQL rollback never undoes an LLM request, email, or other external side effect. - -### Callbacks and reusable policies share one evaluator - -`authorize` accepts an async callback or `definePolicy({ resource?, allow })`. An inline callback normalizes to a -no-resource policy. It receives `resource: undefined`. A resource policy runs its lookup once after authentication -and parsing, skips `allow` on null, and passes the same non-null resource to `allow` and `execute`. - -Only literal `true` grants. `false`, malformed values, rejected promises, and thrown exceptions deny. A policy can -return `{ deny: "forbidden" }` or `{ deny: "not-found" }`. Internal diagnostics use fixed reasons, not arbitrary -client-facing text. - -The resource is a snapshot, not a lock. A state-dependent mutation must repeat its complete authorization predicate -in a conditional write or re-read and lock the relevant rows in a transaction. A transaction at the default isolation -level does not automatically make an earlier lookup authoritative. - -Policy descriptors expose readonly `resource` and `allow` functions. Application tests can call them with explicit -actor and database fixtures without enabling an HTTP actor override. The dispatcher and actor injection seam remain -internal. Supporting direct policy tests does not add a network bypass. - -The Files plugin already has async boolean policies and `FilePolicyUser.id` -(`packages/appkit/src/plugins/files/policy.ts`). Use the same `actor.id` spelling. Database policies additionally need -validated operation input, resource loading, and concealed denials. Reuse those concepts, not Files' SP fallback or -its action and volume-specific combinators. Unifying both APIs is outside this change. - -### Actor identity is not a database credential - -`Actor` contains an authenticated end user's canonical workspace SCIM `id` and optional email. Email is not an -ownership key. The registry's dispatcher never turns an actor into `CallerContext`, never chooses an OBO pool, and -rejects an inherited caller scope rather than silently elevating it to app execution. - -Context contains the ordinary service AppKit handle and `db`, an alias for `appkit.database`. It contains neither the -request nor a user token. Trusted server code can still close over other handles or deliberately call OBO APIs for -other resources. This is not a sandbox or a claim that such calls are impossible. The database credential remains the -service principal, and its app-only guard stays in force. - -Recommended v1 actor integrations: - -- On Apps, do not assume `X-Forwarded-User` is a SCIM id. The public header contract describes an IdP identifier. - A supported canonical path is to validate the request's forwarded user credential through the workspace Current - User `Me` API, using identity-read scopes, and use the returned id. This authenticates the caller; database queries - still use the service principal. It does not open `CallerContext`. Missing or invalid request credentials deny - before lookup or execution. The verified resolver and actual Apps integration still require implementation proof. - Header presence and deployment environment variables alone never establish an actor. -- In explicit local development, resolve the signed-in developer through the existing local identity loader in - `packages/shared/src/dev-obo.ts`. Copy only `id` and email into the actor, never its token. Check the socket, - host, origin, and fetch metadata on every operation request, even if forwarded headers exist. Extract the existing - loopback predicate from `packages/appkit/src/plugins/server/dev-obo-middleware.ts`, not its header short-circuit. - Follow its explicit credential and `APPKIT_DEV_OBO` enable rules. Missing credentials gives 401. Forwarded headers - never choose the local actor. A local process or tunnel with loopback access is within the developer's trust - boundary, so this mode must not be exposed through forwarding or remote tunnels. - -A recognized Apps boundary takes precedence over development mode. There is no `dev-user` or service-principal actor. -Canonical identity must come from the same authority in Apps and local development. The public sources are -[Apps authorization](https://docs.databricks.com/aws/en/dev-tools/databricks-apps/auth), -[forwarded headers](https://docs.databricks.com/aws/en/dev-tools/databricks-apps/http-headers), and -[Current User Me](https://docs.databricks.com/api/workspace/currentuser/me). Forwarding a user credential for identity -validation is not OBO database execution. Support for other hosting remains an explicit scope question. - -### Generate one server-only static registry - -Discovery occurs in `appkit generate-types`, not in plain Node at startup. It scans `server/database/routes/*.ts` -in deterministic order, imports each module for contract inspection, and writes a committed server registry. -Named tagged declarations are inspected. Ordinary helper exports are ignored, and a tagged default export is rejected. -Modules must be safe to import without a running app. +export const noteAuthor = definePolicy({ + resource: ({ db, input }: PolicyContext<{ readonly id: number }>) => + db.notes.find(input.id), + allow: ({ actor, resource }) => resource.author === actor.id, +}); +``` ```ts -// server/database/routes.gen.ts, generated -import { schema } from "../../config/database/schema"; -import * as tasks from "./routes/tasks"; - -export const databaseRoutes = { - schema, - files: { "server/database/routes/tasks.ts": tasks }, -} as const; -``` +// server/database/routes/notes.ts +import { defineOperation } from "@databricks/appkit/beta"; +import { z } from "zod"; -The app statically imports `databaseRoutes` and passes it to `database({ routes: databaseRoutes })`. The registry -imports the schema too, so the intended production graph does not depend on source files or runtime Jiti. -`routes.schema` supplies the plugin schema. A distinct explicit `schema` is an error. +import { noteAuthor, signedIn } from "../policies"; -Generation alone writes the registry and client declarations. Setup is read-only and runs the same descriptor -validator. Import namespaces deliberately retain tagged invalid declarations so boot can name and warn about a missing -`authorize`, even when typegen already omitted that operation from client types. Only valid entries reach endpoints. -The generated file shows the eligible modules. The operation source and generated declaration show callable exports. +export const listNotes = defineOperation({ + name: "notes.list", + entity: "notes", + method: "GET", + output: z.array(z.object({ id: z.number(), body: z.string() })), + authorize: signedIn, + execute: ({ db }) => db.notes.select(["id", "body"]).limit(100).toArray(), +}); -Use existing atomic file replacement, not a new manifest store, lease, or generation-pointer protocol. CI and -production `prebuild` run proposed `appkit generate-types --wait --strict --check`. Check mode verifies committed -artifacts without replacing them. Drift, invalid authorization, or generation errors fail the build. Development -writes current artifacts and warns for skipped authorization. A broken generation clears client operation types and -must not publish a candidate dispatcher. +export const getNote = defineOperation({ + name: "notes.get", + entity: "notes", + method: "GET", + input: z.object({ id: z.number().int() }), + authorize: noteAuthor, + execute: ({ resource }) => resource, +}); -In dev, the watcher observes schema, routes, additions, removals, and imported dependencies. It stops dispatch before -rebuilding or restarting. It resumes only after the current registry validates. A failed generation cannot keep an -old authorized operation serving. Existing Vite watching and `tsx watch` are starting points, not proof that this -lifecycle already works. There is no production hot reload. +export const createNote = defineOperation({ + name: "notes.create", + entity: "notes", + method: "POST", + input: z.object({ body: z.string().trim().min(1) }), + authorize: signedIn, + execute: ({ actor, db, input }) => + db.transaction(async (tx) => { + const note = await tx.notes.create({ author: actor.id, body: input.body }); + await tx.note_events.create({ note_id: note.id, action: "created" }); + return note; + }), +}); +``` -A static import graph, schema inclusion, exact emitted paths, and guarded startup diagnostics need a source-less -bundle proof. Static import failures can occur before plugin setup. Do not assume the current dynamic schema loader -sanitizes those Node startup errors. The production bootstrap must report a bounded startup failure without printing -module-thrown secrets before the privacy contract is accepted. +`name`, `entity`, `method`, `authorize`, and `execute` are required. `input` and `output` are optional, independently. +An operation descriptor does not execute a policy or database query during declaration or generation. -Rejected alternatives are runtime TS scanning, a manual module list, a second `routes: true` mode, and a registry -under the client's `shared/appkit-types` directory. The explicit generated import is the only opt-in. +Callbacks receive the verified `actor: { id, email? }`, parsed `input`, service-principal `db`, and `appkit` exports. +`allow` and `execute` also receive the loaded `resource`, or `undefined` for a policy without a lookup. The callback +context contains no request, headers, or user token. At runtime `appkit` contains registered plugin exports; its static +operation type makes other SDK plugins optional because a route module cannot know the app's registration list. -### Authentication and error handling own the HTTP boundary +A reusable resource policy annotates the input fields its lookup reads. Operations with those fields can reuse it; +incompatible input is a type error. `allow` and `execute` receive the lookup's non-null resource type. A literal in a +custom output may need `as const` in the returned object to prevent TypeScript from widening it to `string`. -A namespace adapter covers `/api/database/operations` before the global body parser and frontend fallback. It -resolves the actor before decoding or dispatch, including unknown operation names and wrong methods. Unauthenticated -calls get 401. Authenticated calls to an unknown name or method get opaque 404. +## Authentication and authorization -The current `skipBodyParsing` set matches exact strings, while Express normally accepts case and trailing-slash -variants (`packages/appkit/src/plugins/server/index.ts:144-153`). It is insufficient by itself. The server integration -must exclude the operation namespace with matching semantics, then apply its bounded parser after authentication. -Use the configured body limit. Tests cover canonical, mixed-case, trailing-slash, and unknown routes. Other plugin -namespaces keep their behavior. +### Actor identity and database credentials are distinct -GET carries one bounded JSON `input` query value. Reject repeated values, unexpected query parameters, invalid JSON, -and oversized input through one operation decoder. Other methods use bounded JSON bodies and require -`application/json`. GET executors must be side-effect-free by API contract. The library cannot prove arbitrary -callback purity. Sensitive input belongs in a non-GET operation because URLs also reach browser and proxy logs. +The database pool always uses the app's service principal. PostgreSQL sees one role, so an operation's authorization +and row predicates enforce user access. Custom Express routes and trusted server writes are outside that guarantee. -Cookie-backed mutations require a verified same-origin or equivalent CSRF boundary. JSON content type alone is not -that proof. Use a trusted deployment origin, not unverified forwarded headers. Missing platform confirmation blocks -release of the integration. +The actor boundary is selected from deployment configuration, then verifies requests: -All operation failures end in the operation response adapter. They are not forwarded as raw exceptions to generic -Express error handling. Unknown properties and invalid contracts yield value-free validation details. +| Deployment | Identity source and behavior | +|---|---| +| Databricks Apps, `DATABRICKS_APP_NAME` set outside development | Verify `x-forwarded-access-token` with workspace Current User Me on each request. User/email headers are not trusted. Rejected credentials, inactive users, and service principals give 401; workspace identity-service failure can give 503. | +| Explicit local development | `NODE_ENV=development` plus `DATABRICKS_TOKEN` or `DATABRICKS_CONFIG_PROFILE` uses the verified local OBO identity loader. Requests must satisfy the loopback and origin boundary. `APPKIT_DEV_OBO=false` disables it. Forwarded identity headers are ignored. | +| Other hosting/configuration | Resolve no actor, answer 401, and warn during setup. | -### Interceptors cannot bypass authorization +`actor.id` is the workspace user ID. No synthetic local actor or service-principal fallback is supplied. Actor +resolution occurs before route lookup and input parsing, including unknown names and wrong methods inside the operation +namespace. The operation passes no user credential to database execution. -The sequence is actor, parse, resource, allow, execute, output. Authorization stays outside `Plugin.execute`. -The executor wrapper forces cache and retry off and enables telemetry. It has no generic timeout or user override. -Entity operations continue using their existing execution defaults and explicit transaction context. No mutation -hook or serializer is invoked. +### Policy evaluation -Crucially, the callback passed into `Plugin.execute` catches and classifies errors before that plugin's logger and -telemetry interceptor observe them. Preserve a known `DatabasePluginError` category. Convert other executor errors -to a safe operation error without the original message, cause, or row data. Retain only bounded diagnostics outside -the exception. Mapping the returned status and discarding its message afterward would be too late. +A function policy or a policy's `allow` must return literal `true` to admit. A resource policy runs its lookup once, +then reuses the result in `allow` and `execute`. -A lookup or `allow` exception always denies, including a classified database outage. This is the operator's -fail-closed rule. The internal stage distinguishes an outage from a deliberate denial without changing the response. -An executor's database error keeps its normal category. +| Result | HTTP behavior | +|---|---| +| No verified actor | 401; no policy or executor runs | +| Literal `true` | Execute once | +| `false`, malformed return, or exception without a resource policy | 403; no execute | +| Resource returns `null` or `undefined` | 404; neither `allow` nor execute runs | +| Resource lookup/allow fails or refuses | 404 by default; no execute | +| `{ deny: "forbidden" }` / `{ deny: "not-found" }` | Explicit fixed 403 / 404 | -The database's statement and transaction limits are not an end-to-end request deadline. A callback awaiting non-SQL -I/O can still hang in v1 and must supply its own dependency deadline. No JavaScript timeout, client disconnect, or -AbortSignal is claimed to cancel SQL. Automatic retries could repeat committed mutations and remain disabled. +A denial's optional `reason` is neither sent nor logged. Callers receive stable messages without resource values or +policy exception text. A resource is a snapshot, not a lock or a transaction. -### Flag strategy +### Mutable rules and transactions -`routes?: DatabaseRoutes | false` is the only behavior switch. Its default is off. An empty validated registry also -serves nothing. Turning it off removes operations at the next start, publishes no endpoints, and never enables CRUD. -Types, policy helpers, code generation, and unreachable internals need no feature flag. The first user-reachable -slice must introduce this switch together with authorization enforcement. Strict generation is a build check, not a -second runtime exposure flag. Actor-integration selection is an authentication boundary, not an enable flag. +Authorization establishes permission at the time of the check. An operation that depends on a case remaining open, +stock remaining available, or an ownership value remaining unchanged must enforce that predicate again inside its +write transaction, using a locked read, conditional write, or database constraint. -## Main design +Use database-generated IDs instead of `count() + 1`. Use uniqueness constraints for invariants such as one open case +per alert, and database enforcement when all writers must honor a rule. Write the business change and its accurate audit +record in the same transaction. Do not reconstruct a prior value from a stale browser or policy snapshot. -### What an app author writes +Transactions retain their existing commit, rollback, nested-join, expiry, statement-budget, and isolation semantics. +There is no automatic executor transaction. A transaction cannot undo another plugin's write or arbitrary external +work, and its deadline does not cancel arbitrary JavaScript. -Assume a schema entity `tasks` with numeric `id`, `title`, `done`, and private server-controlled `owner_id`. These -are proposed examples, not the current playground schema. +## Discovery, generation, and packaging -```ts -// server/database/policies.ts -import { definePolicy, type PolicyContext } from "@databricks/appkit/beta"; +### One inspected registry supplies runtime and browser contracts -export const authenticated = definePolicy({ allow: () => true }); +The generator scans direct `*.ts` children of `server/database/routes/`, excluding `*.test.ts`, `*.d.ts`, and +`*.gen.ts`. Tagged `defineOperation` exports are inspected; ordinary helper exports are ignored. Recursive discovery +and export-derived public names are not part of the contract. -export const ownerOnly = definePolicy({ - resource: ({ actor, db, input }: PolicyContext<{ id: number }>) => - db.tasks.where({ owner_id: actor.id }).find(input.id), - allow: () => true, -}); -``` +The existing `appkit generate-types` command and AppKit Vite plugin produce: -```ts -// server/database/routes/tasks.ts -import { defineOperation } from "@databricks/appkit/beta"; -import { z } from "zod"; -import { authenticated, ownerOnly } from "../policies"; +| Artifact | Purpose | +|---|---| +| `server/database/routes.gen.ts` | Lazy schema/module imports for runtime discovery and server packaging | +| `shared/appkit-types/database-operations.d.ts` | Structural operation-name, method, JSON input, and JSON output registry for browser consumers | +| `shared/appkit-types/database.d.ts` | Existing trusted server entity-type bridge | -export const createTask = defineOperation({ - entity: "tasks", - method: "POST", - authorize: authenticated, - execute: ({ actor, db, input }) => db.tasks.create({ ...input, owner_id: actor.id }), -}); +Commit generated artifacts. Browser declarations import no operation, schema, or policy module. Runtime boot metadata +publishes registered operation names, URLs, and methods, without policies or schema internals. -export const renameTask = defineOperation({ - entity: "tasks", - method: "PATCH", - authorize: ownerOnly, - execute: ({ actor, db, input, resource }) => - db.tasks.where({ owner_id: actor.id }).update(resource.id, input.changes), -}); +Explicit contracts use Zod 4 JSON Schema conversion and a bounded TypeScript renderer. JSON strings, numbers, +booleans, null, literals, enums, arrays, objects, string-key records, unions, and `z.unknown()` are supported, including +supported optional/default/nullable wrappers. `z.unknown()` intentionally generates `unknown`; it is not a fallback for +an unrenderable contract. Unsupported schemas such as dates, tuples, `z.json()`, transforms, and coercing or +preprocessing inputs fail generation with a source/slot diagnostic rather than silently widening the contract. +Optional object properties work; schemas that require top-level undefined without an exact JSON form do not. -export const listTasks = defineOperation({ - entity: "tasks", - method: "GET", - input: z.strictObject({ limit: z.number().int().min(1).max(100).default(20) }), - output: z.array(z.strictObject({ id: z.number(), title: z.string() })), - authorize: authenticated, - execute: async ({ actor, db, input }) => { - const rows = await db.tasks.where({ owner_id: actor.id }).limit(input.limit).toArray(); - return rows.map(({ id, title }) => ({ id, title })); - }, -}); -``` +Production generation is selected by `NODE_ENV=production` and rejects missing authorization. This uses the existing +generation/build commands; there are no new `--wait`, `--strict`, or `--check` command flags. -`ownerOnly` can serve any compatible input containing `id`, including a custom action with an explicit Zod schema. -Its lookup already filters by the actor's owner id. Only a non-null owned row reaches `allow`. The owner condition is -repeated in the mutation. If ownership changes after the lookup, a null result becomes 404. -A policy checking additional mutable state must also protect that state at write time. `renameTask` accepts the -schema's ordinary update fields, not only `title`. A title-only contract would supply an optional narrower input. +### Development -```ts -// server/server.ts -import { database } from "@databricks/appkit/beta"; -import { databaseRoutes } from "./database/routes.gen"; +A schema-only app receives an empty registry and empty browser operation declarations before its first route. During +development, database setup refreshes the registry before importing it, so a newly added or deleted route does not start +from a stale module list. Native imports let the standard `tsx watch` process track the registry and its dependencies. -const registration = database({ routes: databaseRoutes }); -``` +The Vite generator watches `config/database/` and all of `server/database/`, including shared policies and DTOs. +Operation, schema, and shared-contract edits refresh browser types and reload the backend through the standard +workflow. First-route addition and last-route deletion use the same flow. Generated outputs are ignored as generation +triggers to prevent loops. -The existing app passes this registration to `createApp`. One file can export several operations. The folder and -registry do not register helpers or automatically create an operation per table. +Invalid imports or declarations fail generation/setup. The implementation uses process watching and startup validation; +it does not introduce a separate atomic router-swap protocol or a promise that an already running process stops dispatch +before a filesystem rebuild begins. -### API changes +### Production -The intended signature sketch is not compiler-verified. Registry-derived helper types follow the input and output -tables above. Inference, variance, and contextual typing are proof obligations, not established behavior. +Production startup never generates or hot reloads operations. `appkitServerConfig()` includes the generated registry, +schema, and operation modules. Plain Node first looks for a compiled registry beside the server entry, such as +`dist/database/routes.gen.js`; a source-less server bundle can therefore serve operations without `server/` or +`config/` TypeScript sources. -```ts -type Awaitable = T | Promise; -type Actor = Readonly<{ id: string; email?: string }>; -type Decision = boolean | { readonly deny: "forbidden" | "not-found" }; -type HttpMethod = "GET" | "POST" | "PUT" | "PATCH" | "DELETE"; - -type PolicyContext = Readonly<{ - actor: Actor; - input: I; - appkit: AppKitHandle; - db: DatabaseExports; -}>; -type OperationContext = PolicyContext & { readonly resource: R }; - -declare function definePolicy(spec: { - resource(ctx: PolicyContext): Awaitable; - allow(ctx: OperationContext>): Awaitable; -}): ResourcePolicy>; -declare function definePolicy(spec: { - allow(ctx: OperationContext): Awaitable; -}): PlainPolicy; - -declare function defineOperation< - const E extends EntityName, - const M extends HttpMethod, - I extends ZodType | undefined = undefined, - O extends ZodType | undefined = undefined, - R = undefined, ->(spec: { - readonly entity: E; - readonly method: M; - readonly input?: I; - readonly output?: O; - readonly authorize: Policy, R> | InlineAuthorize>; - execute(ctx: OperationContext, R>): Awaitable>; -}): Operation; -``` +A custom build must include the registry beside its server with the same extension, or import it and pass +`database({ routes: databaseRoutes })`. Without a compiled registry, conventional source loading remains available; +production TypeScript loading uses Jiti. An explicit schema must be the same schema object used by an explicit registry. -An inline callback has no resource. A resource policy supplies its inferred non-null resource. The final overloads -must enforce that distinction. `InputOf` and `ResultOf` choose the method/table default when no override exists and -the Zod contract otherwise. Discovery supplies export identity after definition. `entity` is a generated schema key. -With custom contracts it is the primary entity label, not a claim that execution touches only that table. +Setup loads and validates declarations before opening a pool. Missing authorization skips only the affected otherwise +valid operation at runtime. Malformed or duplicate declarations, unknown entities, import errors, and invalid registry +exports fail setup. A missing/empty source directory is distinct from a broken expected artifact. Nothing restores CRUD. -`defineOperation` brands a descriptor without throwing for missing authorization. Otherwise import would abort -before discovery could report the ignored export. Shared validation runs during generation and setup. Other malformed -contracts fail the candidate registry instead of receiving defaults. +## HTTP contract -Add server helpers and types to `@databricks/appkit/beta`, add `routes?: DatabaseRoutes | false`, and add -`UNAUTHENTICATED` to the shared database error vocabulary. Determine the typed AppKit handle available through plugin -context before implementing the public context. No `appkit.db` API is introduced. Remove `hooks`, its lifecycle and -serializer types, `serialized()`, and serializer-only `shape` options. Do not add compatibility checks or migration -warnings for removed options. Existing structural config validation remains independent of this removal. +The operation namespace bypasses the server's global JSON parser using the same routed-path semantics as dispatch. +The operation handler controls ordering and limits: -### Request outcomes +1. Set `Cache-Control: no-store`, resolve the actor, and check the browser origin boundary. +2. Match the explicit name and exact declared method. Unknown names or wrong methods give 404 after authentication. +3. Decode and validate input, then evaluate authorization. +4. Execute at most once and parse/project the output before sending it. -| Condition | HTTP result | May execute? | -|---|---|---| -| Missing or unverified actor under a supported integration | 401 `UNAUTHENTICATED` | No | -| Authenticated unknown operation or wrong method | 404 `NOT_FOUND` | No | -| Bad encoding, oversized input, or wrong content type | 400, 413, or 415 | No | -| Schema failure, extra field, or empty update | 422 `VALIDATION_FAILED` | No | -| Resource missing, or lookup throws | Opaque 404 | No | -| Resource policy denies, throws, or returns a malformed decision | Opaque 404 by default | No | -| No-resource authorization denies or throws | 403 `FORBIDDEN` by default | No | -| Explicit fixed deny | 403 or 404 | No | -| Conventional POST succeeds | 201 with the public created row | Once | -| Conventional DELETE succeeds | 204 without a body | Once | -| Other allowed callback succeeds | 200 with its inferred or custom response | Once | -| Conventional row output receives null | 404 `NOT_FOUND` | Already ran | -| Executor throws a known database error | Its safe database category | Already ran | -| Other executor error or invalid output | 500 `INTERNAL` | Already ran | - -Every response uses `Cache-Control: no-store`. Reusing `FORBIDDEN` covers both a policy denial and database privilege -failure, but stage and reason distinguish them internally. No raw error text reaches the response adapter. - -### Boot and reload outcomes - -| Condition | Required behavior | +GET accepts no input or one URL-encoded JSON `input` query parameter, with an 8 KiB query limit. Other methods accept +an optional JSON body up to 1 MiB and no query parameters. Cross-origin browser requests are refused. A GET operation +must remain side-effect-free; the method cannot mechanically prove its executor is a read. + +| Condition | Response | |---|---| -| `routes` absent or false | Zero database endpoints. Trusted server API remains. | -| Untyped removed `api` option present | It cannot enable anything. The old route registration path does not exist. | -| Source routes folder missing or empty | Generation produces an explicit empty registry. | -| Tagged operation has missing or invalid authorization | Retain diagnostic declaration, skip HTTP registration, warn with `file#export`. Strict generation fails. | -| Ordinary helper export | Ignore. | -| Bad contract, unknown entity, duplicate, or import failure | Fail generation or setup. Publish no candidate router. | -| Expected generated registry or compiled module missing | Fail build or boot. Never reinterpret it as an empty source folder. | -| Failure after opening a pool | Close the candidate pool and publish nothing. | -| Dev rebuild fails or authorization is removed | Disable dispatch before rebuild. No last-good serving registry. | -| Production source changes on disk | No hot reload. Serving state changes only through a new validated start. | +| Success | 200; POST uses 201; default DELETE uses 204 without a body | +| Malformed transport or explicit input-schema failure | 400 | +| No actor / denied / missing operation or resource | 401 / 403 / 404 as specified above | +| Database constraint conflict | 409 | +| Oversized body / unsupported media type | 413 / 415 | +| Database entity validation failure | 422; distinct from operation input parsing | +| Transient service/database failure | 503 | +| Invalid or unusable output | Opaque 500; a write that returned from execute carries `outcome: "unknown"`, except its normal not-found result | + +Errors use `{ error: "" }`. Input-schema failures can add bounded `details: [{ path, message }]` for +forms. Paths include declared property names and array indices, not submitted dictionary keys or unknown fields. +Messages use safe fixed descriptions rather than raw Zod/refinement messages. Input values, resource contents, and raw +exception text are not returned. An unknown write outcome is not permission to retry automatically. + +## Browser client and React hooks + +```tsx +import { databaseApi } from "@databricks/appkit-ui/js/beta"; +import { + skipToken, + useDatabaseMutation, + useDatabaseQuery, +} from "@databricks/appkit-ui/react/beta"; + +const all = useDatabaseQuery("notes.list"); +const selected = useDatabaseQuery( + "notes.get", + selectedId === null ? skipToken : { id: selectedId }, +); +const create = useDatabaseMutation("notes.create"); +const { error } = await create.mutate({ body: draft }); +if (error === null) setDraft(""); + +// Imperative callers use the same generated contract. +const note = await databaseApi.execute("notes.get", { id: 7 }); +``` -Boot can log, for example, `server/database/routes/tasks.ts#purgeTasks skipped: authorize is missing`. -`createApp` already awaits setup before listening (`packages/appkit/src/core/appkit.ts:304-327`). That ordering does -not prove the proposed watcher or build integration. +The snippets illustrate hook and submit-handler calls; hooks run in React components and `mutate` in the handler. +Names and payloads come from `database-operations.d.ts`. Query hooks accept GET names; mutations accept non-GET names. +A no-parameter operation needs only its name, or `mutate()` for a write. Pass `undefined` in the input slot when supplying +query/request options. A name absent from boot metadata fails with `NOT_EXPOSED` without a request. -### Generate browser types without server imports +### Queries -Derive the browser contract from the same inspected exports, entity/method defaults, and optional Zod overrides. Emit a -global operation registry in `shared/appkit-types/database.d.ts`, analogous to the current entity bridge. It contains -wire input, serialized output, and method per operation. No declaration imports a route, server schema, or policy. +`useDatabaseQuery` returns `data`, `error`, `isLoading`, and `refetch`. A new name, input, or session clears state and +replaces the read. An equal encoded input preserves the current request. Refetch keeps current data visible. Unmount +aborts the read, and late responses from superseded requests cannot populate the hook. Each mounted hook owns its +request; results and pending calls are not deduplicated or cached between hooks. -Default types use existing column metadata and wire adapters. Custom schemas use Zod's JSON Schema conversion -plus a bounded TypeScript renderer. Unsupported constructs fail generation. Do not emit `unknown` or infer arbitrary -executor return types as a fallback. +For a dependent query, `skipToken` represents an input that is not ready. It clears data/error, stays idle, sends no +request, and makes refetch a no-op. Entering that state aborts an old read and ignores its late answer. It is accepted +only by query hooks; imperative calls and mutations still require their declared payloads. -Boot metadata publishes only registered names, methods, and URLs. The existing endpoint map stores paths, so method -metadata is new. It never publishes policies, schema internals, tokens, private fields, or the server registry. +### Mutations -Proposed browser calls are: +`useDatabaseMutation` sends nothing until `mutate`. It resolves `{ data, error }` and never rejects, retries, queues, +deduplicates, or aborts a write. Only the latest call updates hook state; `reset` returns to idle. Disable repeated +submissions while loading when the application requires it, and retain the user's draft on failure. -```ts -const tasks = await databaseApi.call("tasks.listTasks", { limit: 20 }); -const renamed = await databaseApi.call("tasks.renameTask", { id: 42, changes: { title: "Reviewed" } }); -``` +Mutation state belongs to the operation identity and session. Switching A to B, including switching back to A, does not +revive A's old pending state or result. Earlier writes finish; their result does not populate the newly selected hook. +Same-session success or unknown outcome refreshes mounted reads even when the originating hook changed or unmounted. +All mounted queries are invalidated, without a dependency DSL; active queries retain their data during refresh and +skipped/disabled queries remain gated. -`useDatabaseQuery(name, input)` accepts GET names. `useDatabaseMutation(name)` accepts non-GET names. These generic -hooks replace the CRUD hooks and reuse cancellation and mutation lifecycle code where its guarantees fit. No hook is -generated per operation. A missing endpoint raises `NOT_EXPOSED` without a request. +### Sessions and imperative use -V1 does not share results or pending requests across hooks. On a session transition, clear operation state and reject -late results from the previous session. A supported session-change signal is an integration proof obligation, not an -assumption that an in-page identity never changes. Successful mutations invalidate active queries broadly instead of -introducing a dependency DSL. Mutations never automatically retry. A lost response reports proposed -`OUTCOME_UNKNOWN`, not permission to resend. +Authenticated responses carry an opaque actor discriminator derived from the workspace and verified actor ID. A new +actor or a 401 advances the client's session state, clears hooks, and rejects late results from an earlier session as +`SESSION_CHANGED`. Detection is response-driven; this is not a push notification of an identity change before any +response arrives. -## Observability +`databaseApi.execute` rejects with `DatabaseApiError` on failure and supports an optional AbortSignal. A write with a +lost or unusable response can report `OUTCOME_UNKNOWN`; cancellation cannot undo a committed write. React callers +should prefer hooks for rendered database state. Imperative callers own request identity, state updates, cancellation, +and refresh after writes. -One `database.operation` span covers actor resolution through output handling, including denials. It records -operation, entity, method, canonical route, stage, outcome, and a fixed denial reason. Logs use those fields and a -trace id. Skips log file and export at warn. Exceptions log bounded class or category information, not their message, -cause, actor, email, input, or resource. +## Observability and boundaries -This requires more than an allowlist on the custom span: +The handler creates a `database.operation` span across actor resolution, routing, input, policy, execution, and output. +It records the method, matched operation name, outcome, failure stage, and status. Lower work runs with tracing +suppressed to prevent SQL, row values, tokens, and raw exception messages from entering nested spans. Operation HTTP +instrumentation redacts query-bearing attributes. Failure logging uses bounded categories rather than raw callback or +driver messages, and policy denial reasons are excluded. -- Sanitize executor failures inside the execution callback before `Plugin.execute` and `TelemetryInterceptor` log or - record them. A sentinel-secret test must inspect actual exported logs and spans. -- Redact query-bearing `url.query`, `http.target`, `http.url`, and equivalent HTTP attributes for operation traffic. - Renaming the span does not remove attributes. The current HTTP instrumentation changes names only - (`packages/appkit/src/telemetry/instrumentations.ts`). -- Prevent Lakebase SQL text and raw driver exception details from leaking operation data. The current connector - records SQL and exceptions (`packages/lakebase/src/pool.ts:87-117`). Operation-path redaction is a release gate. -- Bound startup diagnostics for import failures before claiming sanitized boot errors. +Operations do not use automatic execution cache/retry interceptors. Authorization cannot be skipped by a cache hit, +and a transient write is not automatically replayed. An executor can still perform multiple actions or external side +effects; at-most-once invocation is not exactly-once delivery or application idempotency. -Do not claim repository-wide telemetry is already sanitized. Other plugins and external proxy logs need their own -contracts. GET URLs must contain no secrets even after AppKit's own trace redaction. +The privacy contract applies to AppKit's operation path. App-authored logs, other plugins, and proxy telemetry need +their own handling. GET URLs should contain no secrets. A trusted executor can still copy private data into a declared +custom output or write an incorrect policy; neither the schema nor the SDK is a sandbox. -## Security considerations +## Validation and remaining integration work -The browser, its input, and any client-sent identity fields are untrusted. The server author, policy code, and -executor are trusted and have the service principal's database privilege. Required authorization prevents omission, -not a logically incorrect policy. RLS is explicitly excluded, so PostgreSQL supplies no per-user safety net. +The implementation PRs record build, docs build, lint/format, and monorepo typecheck results, plus focused server, UI, +contract-generation, and playground suites. Coverage includes missing/malformed policies, concealed denials, safe input +errors, private projection, unsupported contracts, exact methods, transport limits, mutation/session identity, +`skipToken`, shared-file watching, first/last route changes, and a plain-Node source-less server bundle. -| Threat | Enforcement or limitation | -|---|---| -| Omitted policy or forgotten route restriction | Required `authorize`, runtime skip, strict build, and no generated CRUD. | -| Spoofed actor | Verified deployment integration, no header-presence inference, no local synthetic actor. | -| Another user's row | Resource policy and conditional writes. List executors must also constrain their rows. | -| New public owner or role field | HTTP schema metadata excludes private and generated fields. Public writable additions widen defaults and require schema review or a narrower override. | -| Private response field | Runtime public projection or strict custom output. Public columns widen default responses. An author can still copy a secret into a declared custom field. | -| Cache bypass or cross-user reuse | Authorization outside interceptors, no server cache, no cross-hook sharing, session-bound client state. | -| Duplicate mutation | No automatic client or server retry. Repeated user submission still needs application idempotency. | -| Ownership or state changes after lookup | Resource is a snapshot. Protect the complete predicate in the write or transaction. | -| Authorization outage | Lookup and policy exceptions deny. Logs distinguish the failure without revealing it to callers. | -| Stale dev authorization | Dispatch stops before rebuild and restarts only with a valid current registry. | -| CSRF or GET mutation | Verified mutation origin boundary and side-effect-free GET contract. JSON alone is insufficient. | -| Expensive or hung callbacks | Bounded transport and SQL limits. Non-SQL dependency deadlines remain the author's responsibility in v1. | -| Server secrets in the browser | Server-only registry and structural declarations. Add a client import restriction and inspect the actual bundle. | -| Unprotected custom Express routes | Outside the database plugin's guarantee. Trusted server code can bypass these building blocks. | - -## Platform dependencies - -| Ask | Why | -|---|---| -| Confirm stripping and injection of Apps user headers, including SP-only deployments, and name the supported deployment boundary. | Client-supplied headers or environment presence cannot establish the actor. | -| Verify the forwarded user credential and identity-read scopes needed for workspace `Me`, or provide an equally verified canonical identity adapter. | IdP header strings are not documented as SCIM ids. Ownership needs the same authority in dev and production. | -| Confirm the trusted external origin, cookie and CSRF protections, and session transition signal. | Mutation authentication and browser state must remain bound to the current user. | - -These are engineering release prerequisites for the Apps integration. They are not facts established by this repo and -not product preferences for the operator to guess. - -## Rollout plan - -1. Apply the approved subtraction first. Remove hooks, `serialize`, serializer-only client helpers, and their demo - wiring in an independent slice. Preserve explicit transactions, ordinary CRUD contracts, and HTTP protections. - The operator accepted breaking changes and requires no compatibility layer. Other operation behavior still waits - for validation of the remaining design questions. -2. Run the type, Zod, packaging, identity, transport, and reload proofs below. Record artifacts and revise the design - if a proof fails. Do not silently replace a missing proof with a type assertion or fallback router. -3. After validation, plan independently verifiable slices. Types and unreachable internals need no behavior flag. - The first reachable operation path must have the explicit registry opt-in and no unprotected legacy alternate. -4. Migrate repository callers, docs, template, playground, and browser APIs to the current operation model. Preserve - the playground's note-plus-event behavior through an explicit executor transaction. Do not add legacy adapters or - migration-only handling. Never release an intermediate version that reopens CRUD when operations are off. -5. Release only after the negative behavior matrix, source-less bundle, real Apps identity, and telemetry privacy - gates pass. Production rollout is a separate approved plan. - -Required implementation evidence includes: - -| Area | Later proof | -|---|---| -| Authentication | Missing identity, forged headers, dev disable rules, unsupported hosting, and malformed body all fail before lookup or execute. | -| Authorization | False, malformed return, null lookup, thrown lookup, thrown allow, async allow, explicit deny, and reused resource all have the specified status and execute count. | -| Defaults | Missing policy is absent from routes and warns at boot. Strict CI fails. Neither flag state can enable legacy CRUD. | -| Inference | Method/table defaults without required name/input/output, discovery ids after format validation, incompatible policy input, non-null resource, bigint keys and writes, strict overrides, and unsupported Zod schemas. | -| Output | Default public projection excludes private columns. Explicit custom contracts reject undeclared fields. Invalid custom output is opaque 500. | -| Subtraction | Hooks, serializer exports, and removed fields are gone without compatibility guards. Explicit transaction commit/rollback/join/expiry/isolation and deadline remain covered. | -| Interceptors | Denials never enter execute, a transient mutation runs once, and no actor receives another actor's cached result. | -| Transport | Case and slash variants, unknown names and methods, repeated query values, limits, content type, and CSRF boundary. | -| Discovery | Add, remove, remove authorization, import error, duplicate, stale artifact, failed restart, and generated-file drift. | -| Production | Plain Node serves compiled operations with `server/` and `config/` source trees absent. Inspect client and server bundles. | -| Client | Missing endpoint sends nothing. Session switch rejects old pending results. Lost mutation response never retries. | -| Privacy | Sentinel input, row values, and exception messages do not reach exported operation, HTTP, execution, or Lakebase telemetry. | -| Integration | Dev playground exercises allowed and denied users through the actual HTTP path. Vitest assertions alone are not live identity proof. | - -After implementation changes, run `pnpm build && pnpm docs:build` and -`pnpm check:fix && pnpm -r typecheck`, alongside the relevant Vitest and integration checks. None ran for this design. +The local AML evaluation exercises more complex domain and UI scenarios, including transactional audit and concurrency +rules. AML is a local ignored test application and is excluded from every PR. Its local database checks do not establish +live Databricks Apps proxy behavior. + +Before production release, validate forwarded-token scopes and proxy/header behavior in the actual Apps deployment, +verify the deployed origin/session boundary, and integrate the separate migration flow if shipping both designs. +Repository tests and packaging proofs do not substitute for those platform checks. No current product decision depends +on inferring output cardinality from input or reintroducing legacy CRUD. ## PR stack -1. `00-design`. This document only. Its draft pull request is the review entry point. -2. `01-remove-hooks`. Approved removal of entity hooks, `serialize`, serializer-only client APIs, and their callers. - This does not introduce or enable operation routes. -3. `02-operation-definitions`. Typed descriptors, inferred method/table contracts, reusable policy definitions, and - real package-consumer compile probes. No loader, actor resolver, or HTTP registration. -4. Operation execution, discovery, client generation, and the app cutover follow as independently verified slices. - The final cutover migrates every first-party database app and removes its old route path. - -The exact public routing format remains open. Defining metadata and types does not settle or expose that format. - -## Open questions - -The settled requirements are mandatory authorization, skip-and-warn for missing authorization at boot, 401 for an -unauthenticated caller, no execute after a lookup or authorization exception, and no RLS. The operator also confirmed -removal of explicit `name`, inferred CRUD input/output with optional overrides, and removal of hooks and `serialize`. -Breaking changes and removal without compatibility shims are accepted. The operator explicitly rejected special -runtime handling of removed fields and requires apps to move to the new routes, not stay as an empty demo. - -### Product decisions for validation - -1. Use concealed 404 for resource-policy denials and 403 otherwise, with fixed explicit overrides? Recommend yes. - Required reviewers are AppKit maintainers and security. -2. Make strict, non-mutating generated-artifact verification mandatory in production builds and CI? Recommend yes, - while runtime retains the required skip-and-warn behavior. Required reviewers are AppKit maintainers. -3. Limit v1 actor integrations to verified Apps and explicit local development, failing setup elsewhere? Recommend - yes initially, without claiming other hosting is impossible. Required reviewers are AppKit and Apps maintainers. - A concrete additional deployment can justify a verified adapter later. -4. What exact public operation id and URL should replace the explicit `name`? The export-derived candidate is not - confirmed. Required reviewers are the operator and AppKit maintainers. This choice does not block hook removal. - -### Engineering proof obligations - -Local throwaway compiler experiments support one generic method-indexed signature for GET, POST, PATCH, and DELETE, -optional Zod overrides, and an annotated reusable resource policy. Unsuppressed negative controls reject missing -`authorize`, private default input, and incompatible policy inputs. These experiments used narrow contract mappings, -not the published package entry, so package-consumer proof is still required. One overload experiment was inconclusive. - -Do not copy the experiments' gaps into the API. Separate nullable executor results from successful wire rows, map -bigint on the wire, and use an empty DELETE response rather than its internal boolean. Keep resource and `allow` -types correlated; do not accept an unrelated callback through a `never` parameter. A standalone reusable lookup may -need one input-context annotation. Runtime projection is required even when the type names only public columns. - -Before release, prove Zod wire rendering, source-less imports, the actual AppKit context, pre-parser ordering, -fail-closed watching, canonical actor resolution, session behavior, and exported telemetry redaction. Those outcomes -are not established by type-checking a scratch model and are not operator preferences to guess. - -The root `designs/` convention comes from open PR #677 rather than merged main. Recheck the design number in open PRs -before publication and merge. +Design first, then the existing implementation layers, bottom to top: + +| Order | PR | Responsibility | +|---|---|---| +| 00 | [#683](https://github.com/databricks/appkit/pull/683) | This design and the current author experience | +| 01 | [#687](https://github.com/databricks/appkit/pull/687) | Remove lifecycle hooks, serialize, and their callers | +| 02 | [#688](https://github.com/databricks/appkit/pull/688) | Typed operation and reusable policy definitions; explicit input, retained output defaults | +| 03 | [#689](https://github.com/databricks/appkit/pull/689) | Actor, authorization, HTTP execution, and safe field-validation details | +| 04 | [#690](https://github.com/databricks/appkit/pull/690) | Discovery, generated browser contracts, empty registry, and shared-file generation | +| 05 | [#691](https://github.com/databricks/appkit/pull/691) | Typed browser client, query/mutation hooks, session/operation identity, and skipToken | +| 06 | [#692](https://github.com/databricks/appkit/pull/692) | Default route discovery, serving/build cutover, real watcher integration, and first-party app/docs migration | + +Each implementation PR remains independently reviewable against its preceding layer. The final layer replaces legacy +HTTP exposure and first-party callers together. The design-only bottom layer does not enable operations by itself. + +## Implementation source map + +These paths refer to the complete operation stack: + +- `packages/appkit/src/plugins/database/operations.ts`: descriptor, policy, context, and wire types. +- `packages/appkit/src/plugins/database/operation-metadata.ts`: shared declaration validation. +- `packages/appkit/src/plugins/database/operation-actor.ts`: verified identity and origin boundary. +- `packages/appkit/src/plugins/database/operation-runtime.ts` and `operation-validation.ts`: request execution and safe errors. +- `packages/appkit/src/plugins/database/load-routes.ts`: default source/compiled registry loading. +- `packages/appkit/src/type-generator/database/`: discovery and browser contract rendering. +- `packages/appkit/src/type-generator/vite-plugin.ts` and `packages/appkit/src/tsdown/index.ts`: watch and build integration. +- `packages/appkit-ui/src/js/database/` and `packages/appkit-ui/src/react/hooks/`: browser and hook behavior. +- `docs/docs/plugins/database.md`: current user guide and complete client error table. From f25524bff78dab23c946642fd260de40a250fdbb Mon Sep 17 00:00:00 2001 From: ditadi Date: Wed, 4 Nov 2026 17:56:50 +0000 Subject: [PATCH 3/3] docs: drop operation entities and output defaults Design 002 no longer gives defineOperation an entity field or infers a response body from a table. An operation without an output schema answers with no body, every example declares its output, and the author experience states that the resource lookup receives the actor like every other callback. Co-authored-by: Isaac Signed-off-by: ditadi --- designs/002_DatabaseRouteOperations.md | 108 +++++++++++++++---------- 1 file changed, 66 insertions(+), 42 deletions(-) diff --git a/designs/002_DatabaseRouteOperations.md b/designs/002_DatabaseRouteOperations.md index d4265ebb8..f39d14746 100644 --- a/designs/002_DatabaseRouteOperations.md +++ b/designs/002_DatabaseRouteOperations.md @@ -8,12 +8,12 @@ [Google Doc](https://docs.google.com/document/d/1zO0X7bCwYtmX6m-p7DFGZJneSa5OsgVd5nctGhCdVZY/edit) is the linked comment copy. This repository document records the current implementation and supersedes its earlier -proposal for inferred CRUD inputs, export-derived names, and mandatory registry imports. +proposal for inferred CRUD inputs, entity-derived outputs, export-derived names, and mandatory registry imports. An app exposes database access to the browser through named, authorized operations. `database()` discovers operations -in the conventional directory; declaring tables alone exposes no HTTP endpoint. Inputs are explicit schemas or absent. -Outputs retain useful entity/method defaults, with explicit schemas for lists and custom responses. Browser calls and -React hooks are typed without importing server modules. +in the conventional directory; declaring tables alone exposes no HTTP endpoint. Inputs and outputs are explicit schemas +or absent. No input schema means no parameters, and no output schema means no response body. Browser calls and React +hooks are typed without importing server modules. ## Background and motivation @@ -22,7 +22,8 @@ principal and had no required per-operation user authorization. Adding a protect protect the generated route. Lifecycle hooks also spread business behavior between table declarations and callers. This design replaces that HTTP surface while preserving trusted `appkit.database` access, schema-derived server types, -private-column projection, wire codecs, and explicit transactions. The operations-first POC in +the entity client's default private-column projection, and explicit transactions. Operation responses are explicit Zod +schemas, which replace the earlier HTTP column codecs. The operations-first POC in [#682](https://github.com/databricks/appkit/pull/682) is prior art, rather than the implementation baseline. Design [001](https://github.com/databricks/appkit/pull/677) covers database branches, migrations, and deployment. Its @@ -34,7 +35,7 @@ startup; it does not create or migrate them. Integrating the migration flow is s 1. Register only authored operations with authorization, with no alternate generated CRUD endpoint. 2. Verify the caller before parsing input, then authorize before executing database work. 3. Make no-parameter operations simple to declare and call, without an empty Zod object. -4. Keep useful row-output defaults while making list, aggregate, and custom responses explicit. +4. Make every response body an explicit schema, so a table change never widens an answer. 5. Use conventional discovery in development and compiled registries in production. 6. Generate browser contracts and provide generic React hooks with clear read, write, and session behavior. 7. Keep transactional business rules visible in application code. @@ -84,7 +85,7 @@ fail validation, including a duplicate whose declaration lacks authorization. ### No input schema means no parameters An absent `input` gives callbacks `undefined` and generates a browser input type of `void`. It never infers an ID, -create body, or patch from the entity or method. An operation that needs parameters declares its Zod schema. +create body, or patch from the method or a table. An operation that needs parameters declares its Zod schema. ```tsx useDatabaseQuery("alerts.queue"); @@ -99,29 +100,32 @@ policy lookup. An explicit schema controls parsing, defaults, optional propertie its Zod semantics. Every object is not implicitly made strict. Server-owned values, such as ownership, come from the verified actor and domain logic rather than client input. -### Output defaults remain +### No output schema means no body -| Method | Executor result without `output` | Successful response | -|---|---|---| -| `GET` | Public entity row or `null` | Row, 200; `null` gives 404 | -| `POST` | Created public entity row | Row, 201 | -| `PATCH` / `PUT` | Updated public entity row or `null` | Row, 200; `null` gives 404 | -| `DELETE` | Boolean | `true` gives 204 with no body; `false` gives 404 | +An absent `output` means the operation answers with no body, and the browser output type is `void`. Nothing infers a +response from a table or the method. + +| `execute` result without `output` | Response | +|---|---| +| `undefined` | 204 with no body | +| `null` on `GET`, `PATCH`, or `PUT` | 404 | +| Anything else | Opaque 500; a write carries `outcome: "unknown"` | -The default row projection excludes private columns and relations. Existing codecs encode bigint as decimal strings -and timestamps as RFC 3339 text. Adding a public column can widen a default response, so schema review still matters. -An explicit output schema provides a narrower or different response; `execute` owns the mapping into it. +Every operation that answers a body declares `output`, rows included. `z.object` strips undeclared keys, so the schema +is the projection and a private column stays out of the answer by not being named. `.passthrough()`, `z.record`, +`z.any()`, `z.unknown()`, transforms, and naming a private field all defeat that projection; `.private()` on a column +does not sanitize an operation's `output`. `execute` owns the mapping into the schema. -Lists, aggregates, custom responses, and successful nullable responses declare `output`. The presence of `{ id }` in -input never determines whether the answer is one object or an array. A GET list with no input needs only its array -output schema. A nullable output can return `null` with 200; otherwise a null GET/PATCH/PUT result gives 404. +The presence of `{ id }` in input never determines whether the answer is one object or an array. A GET list with no +input needs only its array output schema. A nullable output can return `null` with 200; otherwise a null GET/PATCH/PUT +result gives 404. A DELETE that must report a missing row uses a resource policy, which answers 404 before `execute`. ### Authorization is required at both boundaries TypeScript requires `authorize`. If an otherwise valid untyped declaration omits it, runtime setup and normal generation skip it and warn with `file#export`. Production generation fails instead. An invalid authorization value, malformed -contract, unknown entity, duplicate name, or module import failure blocks the registry. A missing policy is never an -allow-all default, and skipping a policy does not hide another malformed field. +contract, duplicate name, or module import failure blocks the registry. A missing policy is never an allow-all +default, and skipping a policy does not hide another malformed field. ## Author experience @@ -139,6 +143,13 @@ export const noteAuthor = definePolicy({ db.notes.find(input.id), allow: ({ actor, resource }) => resource.author === actor.id, }); + +// The lookup receives the same context as every other callback, actor included. +export const ownNote = definePolicy({ + resource: ({ actor, db, input }: PolicyContext<{ readonly id: number }>) => + db.notes.where({ id: input.id, author: actor.id }).first(), + allow: () => true, +}); ``` ```ts @@ -148,46 +159,59 @@ import { z } from "zod"; import { noteAuthor, signedIn } from "../policies"; +const note = z.object({ id: z.number(), author: z.string(), body: z.string() }); + export const listNotes = defineOperation({ name: "notes.list", - entity: "notes", method: "GET", - output: z.array(z.object({ id: z.number(), body: z.string() })), + output: z.array(note.pick({ id: true, body: true })), authorize: signedIn, execute: ({ db }) => db.notes.select(["id", "body"]).limit(100).toArray(), }); export const getNote = defineOperation({ name: "notes.get", - entity: "notes", method: "GET", input: z.object({ id: z.number().int() }), + output: note, authorize: noteAuthor, execute: ({ resource }) => resource, }); export const createNote = defineOperation({ name: "notes.create", - entity: "notes", method: "POST", input: z.object({ body: z.string().trim().min(1) }), + output: note, authorize: signedIn, execute: ({ actor, db, input }) => db.transaction(async (tx) => { - const note = await tx.notes.create({ author: actor.id, body: input.body }); - await tx.note_events.create({ note_id: note.id, action: "created" }); - return note; + const created = await tx.notes.create({ author: actor.id, body: input.body }); + await tx.note_events.create({ note_id: created.id, action: "created" }); + return created; }), }); + +export const deleteNote = defineOperation({ + name: "notes.delete", + method: "DELETE", + input: z.object({ id: z.number().int() }), + authorize: noteAuthor, + execute: async ({ db, input }) => { + await db.notes.delete(input.id); + }, +}); ``` -`name`, `entity`, `method`, `authorize`, and `execute` are required. `input` and `output` are optional, independently. -An operation descriptor does not execute a policy or database query during declaration or generation. +`name`, `method`, `authorize`, and `execute` are required. `input` and `output` are optional, independently. An +operation descriptor does not execute a policy or database query during declaration or generation. -Callbacks receive the verified `actor: { id, email? }`, parsed `input`, service-principal `db`, and `appkit` exports. -`allow` and `execute` also receive the loaded `resource`, or `undefined` for a policy without a lookup. The callback -context contains no request, headers, or user token. At runtime `appkit` contains registered plugin exports; its static -operation type makes other SDK plugins optional because a route module cannot know the app's registration list. +Every callback, the `resource` lookup included, receives the verified `actor: { id, email? }`, parsed `input`, +service-principal `db`, and `appkit` exports. `allow` and `execute` also receive the loaded `resource`, or `undefined` +for a policy without a lookup. The lookup runs once per request, so `execute: ({ resource }) => resource` answers +without a second query. The callback context contains no request, headers, or user token. At runtime `appkit` contains +registered plugin exports; its static operation type makes other SDK plugins optional because a route module cannot know +the app's registration list. A reusable resource policy annotates the input fields its lookup reads. Operations with those fields can reuse it; incompatible input is a type error. `allow` and `execute` receive the lookup's non-null resource type. A literal in a @@ -299,8 +323,8 @@ A custom build must include the registry beside its server with the same extensi production TypeScript loading uses Jiti. An explicit schema must be the same schema object used by an explicit registry. Setup loads and validates declarations before opening a pool. Missing authorization skips only the affected otherwise -valid operation at runtime. Malformed or duplicate declarations, unknown entities, import errors, and invalid registry -exports fail setup. A missing/empty source directory is distinct from a broken expected artifact. Nothing restores CRUD. +valid operation at runtime. Malformed or duplicate declarations, import errors, and invalid registry exports fail +setup. A missing/empty source directory is distinct from a broken expected artifact. Nothing restores CRUD. ## HTTP contract @@ -318,7 +342,7 @@ must remain side-effect-free; the method cannot mechanically prove its executor | Condition | Response | |---|---| -| Success | 200; POST uses 201; default DELETE uses 204 without a body | +| Success | 200; POST uses 201; an operation without `output` uses 204 without a body | | Malformed transport or explicit input-schema failure | 400 | | No actor / denied / missing operation or resource | 401 / 403 / 404 as specified above | | Database constraint conflict | 409 | @@ -415,7 +439,7 @@ custom output or write an incorrect policy; neither the schema nor the SDK is a The implementation PRs record build, docs build, lint/format, and monorepo typecheck results, plus focused server, UI, contract-generation, and playground suites. Coverage includes missing/malformed policies, concealed denials, safe input -errors, private projection, unsupported contracts, exact methods, transport limits, mutation/session identity, +errors, explicit output projection, unsupported contracts, exact methods, transport limits, mutation/session identity, `skipToken`, shared-file watching, first/last route changes, and a plain-Node source-less server bundle. The local AML evaluation exercises more complex domain and UI scenarios, including transactional audit and concurrency @@ -425,7 +449,7 @@ live Databricks Apps proxy behavior. Before production release, validate forwarded-token scopes and proxy/header behavior in the actual Apps deployment, verify the deployed origin/session boundary, and integrate the separate migration flow if shipping both designs. Repository tests and packaging proofs do not substitute for those platform checks. No current product decision depends -on inferring output cardinality from input or reintroducing legacy CRUD. +on inferring an output from a table or from input, or on reintroducing legacy CRUD. ## PR stack @@ -435,8 +459,8 @@ Design first, then the existing implementation layers, bottom to top: |---|---|---| | 00 | [#683](https://github.com/databricks/appkit/pull/683) | This design and the current author experience | | 01 | [#687](https://github.com/databricks/appkit/pull/687) | Remove lifecycle hooks, serialize, and their callers | -| 02 | [#688](https://github.com/databricks/appkit/pull/688) | Typed operation and reusable policy definitions; explicit input, retained output defaults | -| 03 | [#689](https://github.com/databricks/appkit/pull/689) | Actor, authorization, HTTP execution, and safe field-validation details | +| 02 | [#688](https://github.com/databricks/appkit/pull/688) | Typed operation and reusable policy definitions; explicit input and output schemas | +| 03 | [#689](https://github.com/databricks/appkit/pull/689) | Actor, authorization, HTTP execution, bodiless replies, and safe field-validation details | | 04 | [#690](https://github.com/databricks/appkit/pull/690) | Discovery, generated browser contracts, empty registry, and shared-file generation | | 05 | [#691](https://github.com/databricks/appkit/pull/691) | Typed browser client, query/mutation hooks, session/operation identity, and skipToken | | 06 | [#692](https://github.com/databricks/appkit/pull/692) | Default route discovery, serving/build cutover, real watcher integration, and first-party app/docs migration |