Keep agents' browser from being flagged as a bot, with a bot check in Settings (GODM-34)

[danielehrhardt]

Closes GODM-34.

Many sites block browsers that look automated, or bury them in CAPTCHAs. Godmode's browser now hides the signals they look for, so to those sites the agents' browser looks like a normal Chrome. Settings → Browser → Bot detection has a switch for this (on by default). It also has a bot check that shows, signal by signal, what bot detection sees in the agents' browser.

What changes in the browser

• Automation flag: every Chromium Godmode starts gets --disable-blink-features=AutomationControlled, so navigator.webdriver stays false on every build. The Chrome inside a VM gets it too.
• Headless user agent: a headless Chrome used to introduce itself as HeadlessChrome/154…, and many sites block that on sight. Godmode now asks the installed Chrome once which user agent it sends with a window, and starts headless Chrome with that one (--user-agent). It learns this from a throwaway headless launch, once per executable and version. Because it's a launch flag, requests, frames, workers and service workers all agree.
• Headless screen: headless Chrome reported an 800×600 screen with a 1280×900 window. It now gets a desktop screen: 1920×1080, minus the menu bar on macOS or the taskbar on Windows.
• Viewport: in headless mode, browser-use used to emulate the display size as the viewport, so the page was bigger than its own window, which also gives headless away. With the switch on, browser-use leaves the viewport alone.
• What's left: while --user-agent is set, Chrome withholds the detailed client hints (full version). Only sites that ask for them notice, and the check reports it as a warning. A visible window passes all 10 checks.

Bot check

POST /api/browser/profiles/:id/bot-check opens a page in a background window of the profile's browser. The page is served by a throwaway server on 127.0.0.1. The check then judges ten signals the way common bot detection does, and marks each pass, warn or fail:

• navigator.webdriver
• the user agent, in the page and in the request header
• client hints
• a web worker
• window versus screen size
• WebGL renderer
• plugins
• languages
• the Notification and Permissions APIs agreeing
• window.chrome

In Settings you get a verdict, a 10-segment meter and one row per signal.

• If you change the switch while the browser is still running with the old value, a note offers Restart & check. It keeps the browser's current mode (headless or visible).
• If the browser isn't running, the check starts it and stops it again afterwards. It's only stopped if nothing else started using it in the meantime: bot checks and cookie imports only borrow a browser.

Testing

• pnpm typecheck passes for shared, core and desktop, and the desktop build passes.

• All core tests pass on the merged main: 755 pass, 10 skip, 0 fail.

• The browser-use end-to-end test (GODMODE_E2E=1, real browser-use against headless Chrome) passes.

• The new browser-stealth.test.ts covers:

  • the launch flags per platform
  • replacing the headless user agent (Edge keeps its Edg/ part)
  • probe caching, including a failed probe never throwing
  • browser-use's viewport setting
  • every pass/warn/fail rule of the check

• cdp-integration.test.ts against real Chrome:

  • a headless browser with the switch on passes the automation-flag, user-agent, worker and window checks, and leaves no tab behind
  • with the switch off, the check catches HeadlessChrome
  • a browser started for the check is stopped again, unless someone else got it meanwhile (a join during the check, two concurrent checks)

• Two independent code reviews. The first found one high and two medium issues:

  • a failed user-agent probe could crash the core
  • the check could stop a browser an agent had just started using
  • the restart note was misleading

  All three and most low findings are fixed. The follow-up review confirmed the fixes and the merge with GODM-28/GODM-30 (per-chat tabs, browsers started on demand).

• Manually tested in an isolated dev instance, headless and visible, in light and dark mode:

  • the switch
  • both notes, including Restart & check keeping the mode
  • the check window staying open as a blank spare page when it's the browser's last page