Skip to content

chore(playground): prototype TSX drafts in an AppKit playground - #706

Closed
MarioCadenas wants to merge 3 commits into
mainfrom
work/37-prototype-tsx-drafts-in-an-app
Closed

MarioCadenas wants to merge 3 commits into
mainfrom
work/37-prototype-tsx-drafts-in-an-app

Conversation

@MarioCadenas

Copy link
Copy Markdown
Collaborator

The prototype answers yes: drafts written directly as appkit-ui TSX pages worked for all three app types. All 9 drafts compiled and rendered with no runtime errors, and all 3 drafts for an app were ready in 25–59s, against today's ~94s for the HTML drafts. It's on branch work/37-prototype-tsx-drafts-in-an-app (commit d63d0e1ca, new app in apps/tsx-drafts), and pnpm check passes.

Try it yourself

  1. Setup, once, from the repo root: pnpm install && pnpm build. The build is needed because the playground uses appkit-ui's built output.
  2. Profile: a Databricks CLI profile with a valid login on a workspace that has the AI Gateway Anthropic route. I used ug-oauth-dbc-a5d4177a-49dc…; the plain dbc-a5d4177a-49dc profile's login has expired. databricks auth profiles shows which ones are valid.
  3. Command, from apps/tsx-drafts:
    DATABRICKS_CONFIG_PROFILE=<profile> pnpm try --name myapp --prompt "..." --answers "..."
    Optional: --shots <dir> saves light and dark screenshots, and --effort / --model change the model settings.
  4. URLs: it prints the timing table and http://localhost:8000/design/myapp-a, -b and -c. Add ?theme=dark for dark mode. It starts the dev server (or reuses one already on port 8000) and keeps it running until you press Ctrl+C.

Timings

Seconds from the start of the run (prompt and answers in). Sonnet 5.5, low effort, one run per prompt.

App mock.ts done Draft a Draft b Draft c All 3 ready
Taxi dashboard 20.6 23.0 25.5 23.9 25.5
Tetris 15.5 25.5 35.2 44.3 44.3
Onboarding form (5 steps) 21.6 50.8 59.0 51.1 59.0

Two changes made the difference. Run fully in parallel, every draft guessed the mock data's export names wrong and needed a fix round (95s). Now the mock writer sends its list of exports first, and the drafts start from that about 5s in. The other change was low effort: by default Sonnet 5.5 spends 10–25s thinking before it writes anything. The gateway rejects turning thinking off directly, so low effort was the only way I found to skip it. Code quality held up at low effort.

Success criteria

  1. Lock-in under 10s: met. Copying one page and its mock file elsewhere and type-checking it took 1.4s.
  2. Within ~20% of today's time: met by a wide margin. The 94s baseline comes from App Builder, a different setup, so the comparison is rough.
  3. Visual quality at least as good as the HTML drafts: partly verified. The drafts look polished in both themes, use no hard-coded colors, and Tetris is playable. I didn't have the HTML drafts to compare against, so that judgment is yours from the screenshots. One weak spot: in dark mode, onboard-b's header card turns light gray and looks jarring.
  4. All 3 app types: met.

What differs from the brief

  • There's no components.schema.json in this repo, so I wrote a stand-in from appkit-ui's real props.
  • I don't know which model or cache settings App Builder's draft writers use, so I used Sonnet 5.5 with a cached system prompt. Cache reads were confirmed.
  • A new prompt replaces the previous drafts, because they all share one mock file.

The full report and 21 screenshots (light, dark, and Tetris mid-game) are in ~/.herdr-projects/reports/app-kit/37/ (report.md and shots/). The closing note is written.

Also noticed

  • packages/appkit-ui/src/react/charts/normalize.ts: any x field named date becomes a time axis, so display labels like "Jun 3" quietly produce blank charts. The drafts avoid it with a prompt rule.
  • @tailwindcss/vite doesn't generate classes for newly added files until the stylesheet changes. The script works around it by touching index.css.

Task: KIT-37 (app-kit/37)

Adds apps/tsx-drafts: a plain AppKit app plus a script that writes a shared
mock.ts and 3 appkit-ui TSX design pages in parallel, type-checks them with
up to 2 fix rounds, renders each at /design/<name> behind its own error
boundary, and prints per-draft timings. 'pnpm try' runs it end to end.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
@MarioCadenas

Copy link
Copy Markdown
Collaborator Author

Review 1 · Pass · 2 minor, 1 trivial

Does the change do what the task asked? Yes. Gate: major and worse must be fixed.

The prototype does what was asked. I ran it live on a fresh Tetris prompt: the mock contract streamed first, all 3 drafts type-checked on the first try and rendered in light and dark, and everything was ready in 31.4s against the ~94s baseline. The only problems I found are small: one clipped layout in a generated draft, and a failed mock.ts takes down the whole run instead of failing cleanly.

Not required (below the gate)

  • minor: If mock.ts fails after its 2 fix rounds, the mockDone rejection goes up through every writer's Promise.all, so the script crashes with no timing table and leaves the browser open, instead of marking the drafts failed. (apps/tsx-drafts/scripts/draft.ts (mockDone .then throw; write(..., () => mockDone)))
    Proof: Code reading: write() awaits before() = mockDone, which throws when ok is false; nothing catches it before await Promise.all(results.map(...)), and browser.close() and the table print come after that.
  • minor: A generated draft (tetris-b) clips its big score number out of its card, so the 'visual quality' result depends on luck and nothing checks for this. (generated client/src/design/pages/tetris-b.tsx)
    Proof: tetris-b-light.png: '12,450' is cut off at the right edge of the Score card.
  • trivial: try.ts and draft.ts hardcode localhost:8000 and reuse any server already listening there, even one from another app. (apps/tsx-drafts/scripts/try.ts)
    Proof: Code reading: up() only checks that fetch(url) returns ok.

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

🤖 AppKit PR bot

🔬 Run evals

Start an eval for this PR from the evals-monitor app: Go to Evals Monitor →

📦 Try this PR's app template

Scaffolds a new app from this PR's SDK build. Run it in any folder (requires the GitHub CLI — gh auth login — and the Databricks CLI):

gh run download 41922071930 -R databricks/appkit -n appkit-template-0.90.0-pr.9d3c840-work-37-prototype-tsx-drafts-in-an-app-706 -D appkit-pr-706 \
  && unzip -o "appkit-pr-706/appkit-template-0.90.0-pr.9d3c840-work-37-prototype-tsx-drafts-in-an-app-706.zip" -d "appkit-pr-706" \
  && databricks apps init --template "appkit-pr-706"

The template pins @databricks/appkit and @databricks/appkit-ui to tarballs built from this branch, so the scaffolded app runs against this PR's code.

The playground home page now takes a prompt and refinement answers, runs the
3 draft writers through POST /api/drafts, shows live progress per draft, and
previews each option in an A/B/C selector (with a dark-mode toggle) as soon
as it compiles. pnpm try now stops the whole dev server process group.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
@MarioCadenas

Copy link
Copy Markdown
Collaborator Author

Review 2 · Fail · 1 major, 1 minor, 1 trivial

Does the change do what the task asked? Yes. Gate: major and worse must be fixed.

The playground builds what the task asked for: a shared mock.ts written in parallel with 3 tool-less cached writers, tsc with 2 fix rounds, a per-draft error boundary, compile-gated display, a one-command try script, a browser builder, and timings for all 3 app types that come in well under the 94s baseline. The live server works: the builder and drafts render with no browser errors and no color literals. The remaining gaps are an untidy failure path when the mock writer fails, and a visual-quality criterion that has screenshots but no side-by-side comparison with the HTML drafts.

Required

  • major: If mock.ts fails all its fix rounds, mockDone rejects inside every page writer's before() hook, so Promise.all throws, the browser and ticker are never cleaned up, and status.json is never marked done; the builder only shows a stderr tail. (apps/tsx-drafts/scripts/draft.ts (mockDone .then throw, write() before()))
    Proof: Code reading only; not reproduced because it needs the model to fail 3 times.

Not required (below the gate)

  • minor: Criterion 3 (visual quality at least as good as the HTML drafts) is claimed only as 'partly verified': there are light and dark screenshots, but no HTML-draft screenshots to compare against. (closing-note.md, Success criteria)
    Proof: Read the closing note; the shots/ folder contains only TSX drafts.
  • trivial: Timings come from one run per prompt, against a baseline measured on a different stack (the note says so itself). (closing-note.md, Timings)

A failed mock.ts used to reject inside every page writer, so Promise.all threw,
the browser and status ticker leaked, and status.json was never marked done.
Each draft now catches its own failure, the contract always resolves, and the
run ends with done: true and the reason on each failed draft.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
@MarioCadenas

Copy link
Copy Markdown
Collaborator Author

Review 3 · Pass · 1 minor

Does the change do what the task asked? Yes. Gate: major and worse must be fixed.

F4 is fixed: the mock writer's promise no longer rejects into Promise.all, each draft catches its own failure, and cleanup plus saveStatus(true) always run. F6 is addressed with a second run per prompt. F5 is still open as a minor gap, because there are still no HTML-draft screenshots to compare against.

Not required (below the gate)

  • minor: Criterion 3 (visual quality at least as good as the HTML drafts) is still only partly verified, with no HTML-draft screenshots to compare against. (closing-note.md, Success criteria)
    Proof: The agent's own summary says visual quality is 'still only partly verified' and only describes how to do the comparison.

@MarioCadenas MarioCadenas changed the title Prototype TSX drafts in an AppKit playground chore(playground): prototype TSX drafts in an AppKit playground Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant