Skip to content

Spike: stream partial TSX drafts in the design plugin - #708

Merged
MarioCadenas merged 3 commits into
work/1-design-plugin-dev-only-design-from
work/40-spike-stream-partial-tsx-draft
Nov 5, 2026
Merged

MarioCadenas merged 3 commits into
work/1-design-plugin-dev-only-design-from
work/40-spike-stream-partial-tsx-draft

Conversation

@MarioCadenas

Copy link
Copy Markdown
Collaborator

The spike works and is committed (2ce2bfe2a on work/40-spike-stream-partial-tsx-draft), but the ~5s first-meaningful-paint target fails on all 3 apps. Nothing is missing any more; the checks, report and closing note are done.

Try it (from the worktree):

  1. pnpm install && pnpm --filter @databricks/appkit-ui build:package
  2. pnpm --filter tsx-stream-spike dev
  3. Open http://localhost:8040/replay.html, pick taxi, tetris or form, and click Replay. No Databricks login is needed.

How it works:

  • The plugin watches client/src/design/.staging/<name>.tsx as the writer appends to it and never writes draft files itself.
  • On each write it drops the unfinished last line, closes whatever is still open, and pushes the result to the browser as a hot update of src/design/pages/<name>.tsx. React Fast Refresh swaps it in place.
  • The replay plays App Builder's part: it writes the streams I recorded with KIT-37's writer on dogfood to the staging files at their real timing. When the recording says a file type-checked, it writes pages/<name>.tsx and mock.ts.

The three checks, per app (seconds after the draft's first chunk):

check taxi Tetris form
meaningful paint within ~5s no: 6.4–9.5s no: 7.5–11.1s no: 19.7–45s
no visibly broken frame yes, 0 yes, 0 yes, 0
final render identical to the compiled page yes, 3/3 yes, 3/3 yes, 3/3
  • File-watch latency: median 12ms, p95 about 113ms (one 100ms write batch). From staging write to painted frame takes 0.1–0.2s.
  • Why the 5s check fails: the delay is in the order the writer produces the file, not in the renderer.
    • In taxi and Tetris, the page's return ( only streams 3.5–5.6s after the first chunk.
    • In the form drafts, 144–170 lines of logic come first, so it streams after 15–18s.
    • The form also can't render until late: values it reads as objects (its initial form state) arrive late in mock.ts, so every version before that throws and is held back.
  • I did not build the component-boundary fallback, even though the 5s check triggers it. It publishes a subset of the same versions, so it can only paint later. Say if you want it built anyway for comparison.

Recommendation for the App Builder writer prompt (not changed):

  1. Imports, then export default function Page() with return ( straight away; only useState calls before it.
  2. Derived values, handlers and validation after the return, as hoisted functions or a hook defined below the page.
  3. Layout and section headers first, details after.
  4. Short helpers, written after the page.
  5. In mock.ts, small values pages read as objects first, long arrays last.

Tests: 9 new unit tests pass, and pnpm check and typecheck are clean. One existing design-plugin test fails, but it also fails on the base commit in this environment. Not covered: dark mode, and only one writer run per app.

The full report covers how App Builder would feed the stream, plus videos and screenshots per app. It's in /Users/mario.cadenas/.herdr-projects/reports/app-kit/40/: report.md, <app>-replay.webm, shots/.

Also noticed:

  • packages/appkit/src/plugins/design/tests/design.test.ts: "uses the router only when the app has react-router 7+" fails on the base commit too.
  • In the recorded taxi drafts, the "Trips per day" line charts are blank in the compiled page as well. That's the writer's output, probably labels instead of ISO dates on the x axis.

Task: KIT-40 (app-kit/40)

The design plugin watches client/src/design/.staging/<name>.tsx (and
.staging/mock.ts) as a builder appends a draft's source to it, closes the
partial source at the last line that parses (open JSX, brackets, template
literals; undeclared components stubbed, missing mock values stubbed) and
hot-updates src/design/pages/<name>.tsx with it, so Fast Refresh swaps it
in place. /design/__stream/<name> renders it and keeps the last good frame
when a version throws. Once the finished file is written there it loads
from disk and is remounted, ending identical to /design/<name>.

apps/tsx-stream-spike replays writer streams recorded with KIT-37's writer
(taxi, Tetris, onboarding form) through the staging files at their real
timing, 3 drafts side by side, and measures them. Spike only (KIT-40).

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
@MarioCadenas

Copy link
Copy Markdown
Collaborator Author

Review 1 · Fail · 3 major, 3 minor, 1 pre-existing

Does the change do what the task asked? No. Gate: major and worse must be fixed.

The streaming renderer works as a spike: on replay of all three apps there were no broken frames, the final DOM matched the compiled page every time, and file-watch latency was about 10ms. The ~5s first-paint target is still missed on all apps (taxi 6.3-9.5s, Tetris about 11s, form 20-45s), and the component-boundary fallback the decisions called for when that happens was not tried. The dev server also crashed with an unhandled ENOENT during one form replay.

Required

  • major: The first-meaningful-paint success criterion (~5s) is not met on any of the 3 apps; form-a shows nothing until its file is already finished. (apps/tsx-stream-spike (replay results))
    Proof: node scripts/measure.ts per app: taxi meaningful 6.3/8.1/9.5s, tetris 11.1-11.3s, form 19.9/38.0/44.9s; form-a finalAt-firstChunk=44.66s and first paint at 44.95s, so it never streamed. See replay-measurements.txt.
  • major: The decided fallback (component-boundary updates when first paint is over ~5s) was never built or measured, even though its trigger condition was hit on every app; the claim that the writer's file order is the only cause is argued but not tested. (packages/appkit/src/plugins/design/stream.ts, vite-plugin.ts)
    Proof: Diff has only the any-cut-point hot-update path; no boundary-update mode or measurement of one.
  • major: onFinal is fire-and-forget (void onFinal(name)) and calls fs.statSync on the final file; if the file is gone (reset or a rewrite that deletes first) the rejection is unhandled and the whole Vite dev server exits. (packages/appkit/src/plugins/design/vite-plugin.ts:863-881, 1033)
    Proof: During the first form replay the server died: 'Error: ENOENT ... stat .../client/src/design/mock.ts at onFinal (vite-plugin.ts:879)', ELIFECYCLE exit 1; the measure run timed out after 300s and later requests got connection refused. Before that the log shows repeated 'page reload src/design/mock.ts'. See dev-server-crash.txt.

Not required (below the gate)

  • minor: Replay harness: POST /__spike/reset does not cancel the setInterval writers of a replay still running, so a replay page that reloads (mock.ts updates trigger full reloads) and autostarts again races the old writers. This is the likely trigger of the crash above. (apps/tsx-stream-spike/dev.ts:31-70)
    Proof: Code reading: reset() deletes files only, and replay() intervals are never tracked or cleared. Matches the crash log sequence. Not reproduced in isolation.
  • minor: 'Final render identical 3/3' holds for the DOM but not always for pixels: taxi-a, taxi-c and form-a final screenshots are not byte-identical (taxi-c has 2 pixels off by more than 8). (apps/tsx-stream-spike/scripts/measure.ts:96-97)
    Proof: domIdentical true for all 9 drafts; pixelsIdentical false for taxi-a (0 differing), taxi-c (2 differing), form-a (0 differing).
  • minor: The measure script is flaky on a cold dev server: the first taxi run timed out waiting for the compiled /design/ page to render. (apps/tsx-stream-spike/scripts/measure.ts:87)
    Proof: First taxi run: 'page.waitForSelector: Timeout 30000ms exceeded ... #root > *'. A rerun passed.

Not required (there before this change)

  • trivial: design.test.ts 'uses the router only when the app has react-router 7+' fails in this checkout, and fails the same way with the base commit's vite-plugin.ts. (packages/appkit/src/plugins/design/tests/design.test.ts:408)
    Proof: vitest: 1 failed | 122 passed in the design folder; swapping in b987252's vite-plugin.ts still fails, so it comes from the environment, not this change.

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

🤖 AppKit PR bot

🔬 Run evals

Start an eval for this PR from the evals-monitor app: Go to Evals Monitor →

📦 Try this PR's app template

Scaffolds a new app from this PR's SDK build. Run it in any folder (requires the GitHub CLI — gh auth login — and the Databricks CLI):

gh run download 42056296184 -R databricks/appkit -n appkit-template-0.90.0-pr.cb83aec-work-40-spike-stream-partial-tsx-draft-708 -D appkit-pr-708 \
  && unzip -o "appkit-pr-708/appkit-template-0.90.0-pr.cb83aec-work-40-spike-stream-partial-tsx-draft-708.zip" -d "appkit-pr-708" \
  && databricks apps init --template "appkit-pr-708"

The template pins @databricks/appkit and @databricks/appkit-ui to tarballs built from this branch, so the scaffolded app runs against this PR's code.

…etons

appkit-ui (react/beta) gets DesignPage, the shell every /design page
renders through (streamed, finished, locked in): the app's theme with dark
mode from the first paint (?theme= or the system setting), and an error
boundary that keeps the last good frame on screen and holds readiness back
(useDesignReady). The page entry and the stream host both use it instead of
their own boundary and snapshot code. DraftPlaceholder (chart, table, card,
generic skeleton) stands in for components a streamed draft uses before it
defines them; mock values not written yet are a value that can't throw.

Fixes from review: a deleted finished file no longer crashes the dev server
(watcher handlers are caught), generic arrows (<K extends X>(...)) no longer
read as JSX (this kept the form drafts from rendering), a finished file
nothing imports yet doesn't reload every page, and the replay reset stops
the writers of a replay still running. APPKIT_DESIGN_STREAM_MODE=boundary
measures component-boundary updates; scripts/bound.ts gives the earliest
paint any renderer could reach from each recording. The spike app pins the
repo's React so the root test setup keeps one copy.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
@MarioCadenas

Copy link
Copy Markdown
Collaborator Author

Review 2 · Fail · 1 major

Does the change do what the task asked? No. Gate: major and worse must be fixed.

Round-1 fixes F2-F6 are in: the boundary fallback is built and measured, onFinal rejections are caught, reset clears running writers, measure.ts retries on a cold server and counts pixels instead of comparing bytes. The ~5s first-meaningful-paint target is still missed on all 3 apps (taxi 6.4-6.7s, Tetris 7.5-10.8s, form 17.4-21s), as the agent itself reports, so F1 stays open.

Required

  • major: The ~5s first-meaningful-paint success criterion is still not met on any of the 3 apps (form 17-21s). (apps/tsx-stream-spike (replay results, closing-note measurements table))
    Proof: Closing note table: meaningful paint taxi 6.4-6.7s, Tetris 7.5-10.8s, form 17.4-21.0s; agent states the target misses on all 3.

@MarioCadenas

Copy link
Copy Markdown
Collaborator Author

Review 3 · Pass · 1 minor, 1 pre-existing

Does the change do what the task asked? Yes. Gate: major and worse must be fixed.

The spike builds the streaming TSX renderer, a 3-up replay page, recordings for taxi, Tetris and form, and a measured report. Across 9 drafts it has 0 broken frames and the final render matches the compiled page. It misses the ~5s first-paint target (6.4-21s), but the human accepted that as the spike's finding. Design plugin and DesignPage tests pass except one react-router test that the agent says also fails on the base commit.

Not required (below the gate)

  • minor: The ~5s first meaningful paint target is missed on all 3 app types (taxi 6.4-6.7s, Tetris 7.5-10.8s, form 17.4-21s); the human accepted this as the spike's finding, and the report says so. (reports/app-kit/40/report.md)
    Proof: Measurements and checks tables in report.md

Not required (there before this change)

  • trivial: The design.test.ts react-router page-entry test fails; the agent says it fails on the base commit too, and I did not confirm that on base. (packages/appkit/src/plugins/design/tests/design.test.ts)
    Proof: vitest: 1 failed / 131 passed (tests.txt)

The spike app inherited react-router 7 from KIT-37's app without using it.
Having it in the workspace made the design plugin's router test find a
react-router 7 for a fixture app without one, failing Unit Tests in CI.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
@MarioCadenas

Copy link
Copy Markdown
Collaborator Author

Review 4 · Pass · 1 minor

Does the change do what the task asked? Yes. Gate: major and worse must be fixed.

Round 4 only drops the unused react-router 7 dependency from the spike app; the lockfile no longer adds any react-router entry versus the base, and the previously failing design router test now passes along with all design plugin and DesignPage tests. The rest of the spike was accepted in the prior round, including the human-accepted ~5s paint miss. CI on this commit was still queued when I checked.

Not required (below the gate)

  • minor: The ~5s first meaningful paint target is missed on all 3 apps (taxi 6.4-6.7s, Tetris 7.5-10.8s, form 17.4-21s); the human accepted this as the spike's finding and the closing note says so. (closing-note.md, Measurements)
    Proof: Closing note measurements table and Decision section.

Base automatically changed from work/38-serve-tsx-drafts-from-the-desi to work/1-design-plugin-dev-only-design- November 5, 2026 17:03
@MarioCadenas
MarioCadenas merged commit 1289cd2 into work/1-design-plugin-dev-only-design- Nov 5, 2026
9 checks passed
@MarioCadenas
MarioCadenas deleted the work/40-spike-stream-partial-tsx-draft branch November 5, 2026 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant