Skip to content

Keep a VM's work inside the VM: browser and computer use run in the guest (GODM-22) - #24

Merged
danielehrhardt merged 3 commits into
mainfrom
agent/c-pro/d45f27ead815
Sep 29, 2026
Merged

danielehrhardt merged 3 commits into
mainfrom
agent/c-pro/d45f27ead815

Conversation

@danielehrhardt

Copy link
Copy Markdown
Contributor

What does this change?

When a VM is attached (to a chat, an agent or a workspace), the run now does all of its work inside the VM. No browser starts on the Mac anymore, and the Mac's screen is never used.

  • Godmode's agent in the VM (packages/core/src/vm/guest.ts). On first use, Godmode installs these into the guest:

    • uv, copied from the host (the official installer is the fallback)
    • Google Chrome, from Google's disk image
    • the pinned browser-use and Cua Driver packages

    Claude Code starts two MCP servers inside the guest, talking over stdio through tart exec -i:

    • browser: browser-use, driving the VM's Chrome.
    • cua: Cua Driver, which controls the VM's apps and windows. The Cirrus images already give the Tart guest agent Accessibility and Screen Recording.

    Setup runs once per VM. After that, getting a VM ready for a run takes about 0.4 s.

  • Runner

    • A run in a VM never builds the host browser server or gets host computer use, not even for a screen shared in the chat.
    • Runs in the same VM take turns: one screen, one Chrome.
    • If a tool can't be set up in the guest, it is left out and the chat shows a notice. It is never swapped for the host's version.
    • browser-use's LLM tools are hidden in a VM, because the OpenAI key never enters the VM.
    • Cua Driver's own browser tools and its self-maintenance tools are hidden.
  • Logins: vault_fill_login / vault_fill_totp fill the VM's Chrome over CDP through an SSH port forward, and stay bound to the login's site. This uses the same opt-in as Let agents use vault logins and 2FA codes inside their VM (GODM-18) #19 (settings.vm.vaultFill, off by default), because the agent's shell shares the VM with that Chrome. The prompt tells the agent to use vault_fill_* for websites in Chrome and fill_login / fill_totp for other apps.

  • UI

    • A chat that works in a VM now shows a VM panel instead of the host browser panel: a live view of the VM's screen, full size on click, and Take control (opens Screen Sharing).
    • The host "share screen" chip is hidden in VM chats.
    • vm and cua tool calls get readable labels.

How was it tested?

  • pnpm -r typecheck is clean, and bun test in packages/core passes: 645 pass, 0 fail.
  • New tests in vms.test.ts run against a fake guest (Tart shims, a fake uv):
    • everything is installed on first use and reused afterwards
    • both MCP servers start through tart exec -i and answer from inside the guest
    • no computer server, even when a screen is shared in the chat
    • a failed install leaves that tool out and is not retried right away
    • fills are refused unless allowed, and never reach the host browser
    • runs in the same VM take turns
  • Real VM, end to end. I used an APFS clone of a downloaded macOS Tahoe template in a separate Tart home.
    • Boot plus the first-time setup took 25 s + 42 s.
    • browser-use navigated the VM's Chrome.
    • A site-bound fill typed into GitHub's login field inside the VM, and a fill for a foreign site was refused.
    • Cua Driver reported Accessibility and Screen Recording as granted and listed the windows.
    • One real Claude run (Sonnet 5) opened GitHub in the VM and listed the VM's windows through Cua. No Chrome started on the host.
  • Screenshots of the VM panel and the full-size view are attached to GODM-22.

Not changed: custom stdio MCP servers that the human configured for an agent still run on the Mac, as before.

Checklist

  • pnpm typecheck and pnpm test pass
  • cargo clippy / cargo test pass (if apps/desktop/src-tauri changed) — not changed
  • API changes are reflected in packages/shared and apps/desktop/src/lib/api.ts (new CUA_MCP_NAME; no API changes)
  • No secrets, tokens or personal data in code, fixtures, logs or screenshots
  • Docs updated where behaviour changed (README, SECURITY, ARCHITECTURE)

Fills into the VM's Chrome follow the same opt-in as fill_login / fill_totp
(settings.vm.vaultFill) and stay bound to the login's site; the prompt tells
the agent which to use where. Review fixes: the run works in the VM the queue
locked, the SSH tunnel's stderr is drained, the uv installer and the install
probe fail loudly, in-guest MCP servers start without shell startup files,
and runs in the same VM taking turns is tested.
# Conflicts:
#	packages/core/src/runner/prompt.ts
@danielehrhardt
danielehrhardt merged commit bfbbf5c into main Sep 29, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant