Browser Tools

Use browser_tool to control built-in browser windows (Chromium) inside Vorno.

Quick start: Run browser_tool --help to see all available commands and usage examples.

Browser usage paths

Primary and only in-session tool surface: browser_tool
Secondary helper CLI: bun run browser-tool --help for command discovery/templates, and bun run browser-tool parse-url <url> for safe URL diagnostics outside agent turns

Browser as an Alternative to Source Setup

Use browser workflows when creating a source would add unnecessary overhead for the current task.

Good fit for browser-first:

One-off tasks that don’t need reusable integration
UI-only workflows where API/MCP coverage is poor
Fragile source setup/auth cases where user needs results now

Still prefer sources when:

Work is repeatable and automation/reporting is needed
Team-wide reuse and stable tooling matter

Core workflow

If you’re unsure which window to use, run:

browser_tool({ command: "windows" })

Recommended flow:

open — ensure browser window exists (background by default)
navigate <url> — load a URL
snapshot — inspect accessible elements and get refs (@e1, @e2, …)
find <query> — quickly narrow to matching refs by keyword
click / fill / select — interact using refs
screenshot --annotated (or screenshot-region) — visual verification when needed

`browser_tool` command examples

browser_tool({ command: "--help" })
browser_tool({ command: "open" })
browser_tool({ command: "open --foreground" })
browser_tool({ command: "navigate https://example.com" })
browser_tool({ command: "snapshot" })
browser_tool({ command: "find login button" })
browser_tool({ command: "click @e12" })
browser_tool({ command: "click-at 350 200" })
browser_tool({ command: "drag 100 200 300 400" })
browser_tool({ command: "fill @e5 user@example.com" })
browser_tool({ command: "type Hello World" })
browser_tool({ command: "select @e3 optionValue" })
browser_tool({ command: "select @e75 CNAME --assert-text Target --timeout 3000" })
browser_tool({ command: "upload @e3 /absolute/path/to/file.pdf" })
browser_tool({ command: "set-clipboard Name\tAge\nAlice\t30" })
browser_tool({ command: "get-clipboard" })
browser_tool({ command: "paste Name\tAge\nAlice\t30" })
browser_tool({ command: "scroll down 800" })
browser_tool({ command: "evaluate document.title" })
browser_tool({ command: "console 50 warn" })
browser_tool({ command: "screenshot" })
browser_tool({ command: "screenshot --annotated" })
browser_tool({ command: "screenshot-region --ref @e12 --padding 8" })
browser_tool({ command: "window-resize 1280 720" })
browser_tool({ command: "network 50 failed" })
browser_tool({ command: "wait network-idle 8000" })
browser_tool({ command: "key Enter" })
browser_tool({ command: "downloads wait 15000" })
browser_tool({ command: "focus" })
browser_tool({ command: "windows" })
browser_tool({ command: "release" })
browser_tool({ command: "hide" })
browser_tool({ command: "close" })

The wrapper validates commands and returns actionable errors when arguments are missing or invalid.

It also returns rich execution feedback for most commands, including before/after state where available (scroll positions, active element, URL/title transitions, resize clamping, request/error summaries, and window ownership/visibility details).

You can batch commands with semicolons, for example: fill @e1 user@example.com; fill @e2 password123; click @e3

Batches run left-to-right and stop automatically after navigation commands (navigate, click, back, forward) so refs don’t go stale silently.

Quoting and escaping

browser_tool supports quoted arguments:

Double quotes: fill @e5 "Hello world"
Single quotes: wait text 'welcome back' 5000

Semicolons inside quotes are treated as literal text (not batch separators):

fill @e1 "a;b;c"; click @e2
screenshot-region --selector "div[data-x='a;b']" --padding 8

Use backslash escaping when needed:

\; for a literal semicolon outside quotes
\" for a literal " inside double-quoted text

Key commands

`open [--foreground|-f]`

Create or reuse the session browser window.

Default: opens in background
--foreground / -f: focuses in foreground

`snapshot`

Returns an accessibility tree with refs and element metadata.

`find <query>`

Performs keyword search over the snapshot accessibility nodes (role, name, value, description) and returns matching refs.

`click <ref> [waitFor] [timeoutMs]`

Click an element ref from snapshot. Optional wait modes: none, navigation, network-idle.

`click-at <x> <y>`

Click at raw pixel coordinates. Use this for canvas-based UIs (e.g., Google Sheets cells, map elements, chart data points) where snapshot can’t produce element refs. Get coordinates from screenshot or screenshot-region.

`drag <x1> <y1> <x2> <y2>`

Drag from pixel coordinates (x1, y1) to (x2, y2). Performs mousedown, interpolated mousemove events, and mouseup. Use this for:

Moving charts or objects in canvas-based UIs (e.g., Google Sheets charts)
Reordering items via drag-and-drop
Resizing elements by dragging handles
Drawing or selecting regions

Get coordinates from screenshot or screenshot --annotated.

`fill <ref> <value>` / `select <ref> <value> [--assert-text <text>] [--assert-value <value>] [--timeout <ms>]`

Fill text inputs or select dropdown values. Requires an element ref from snapshot.

For modern React/portal combobox UIs, select now performs additional verification and may return a warning when interaction succeeds but form state does not appear to mutate.

Useful flags:

--assert-text <text>: verify downstream UI mutation (for example field label changes to Target)
--assert-value <value>: verify selected control reflects expected value
--timeout <ms>: verification timeout (default 2000ms)

`upload <ref> <path> [path2...]`

Attach local file(s) to a file input (<input type="file">) using a ref from snapshot.

Notes:

Use absolute file paths.
Multiple files are supported: upload @e3 /path/a.pdf /path/b.jpg
Files must exist and pass safety validation (sensitive paths are blocked).

`type <text>`

Type text character-by-character into the currently focused element without needing a ref. Use this when:

The target is a canvas-based input (no DOM ref available)
You’ve already focused an element via click or click-at
The application uses a custom input mechanism

Difference from fill: fill focuses a ref and replaces its value. type sends keystrokes to whatever is currently focused.

`set-clipboard <text>` / `get-clipboard`

Read or write the page clipboard programmatically.

set-clipboard writes text and interprets common escape sequences:
- \t → tab
- \n → newline
- \r → carriage return
- \\ → literal backslash
Unknown escapes are preserved literally (example: \\x stays \\x)
get-clipboard reads the current clipboard text content as raw text (tabs/newlines are returned as actual characters)

`paste <text>`

Convenience command: writes text to clipboard then triggers Ctrl+V (or Cmd+V on Mac). Equivalent to set-clipboard <text> followed by key v meta/key v control. Escape handling is identical to set-clipboard, which makes TSV-style bulk data entry reliable.

`screenshot` / `screenshot --annotated` / `screenshot-region ...`

Capture full-window or targeted screenshots. --annotated overlays @eN labels on interactive elements for easier ref debugging.

`console`, `network`, `wait`, `downloads`

Debug runtime issues, requests, synchronization points, and download progress.

downloads output includes the resolved local savePath when available so you can reference the downloaded file directly.

`focus [windowId]` / `windows`

Manage and inspect browser window ownership and visibility.

Lifecycle commands

release — dismiss agent overlay, keep window visible for user
hide — hide window but preserve session state
close — close and destroy window

Per-session isolation and lifecycle

Each agent session’s browser window has its own private storage partition — cookies, logins, localStorage, and permission grants are isolated per session. Windows the user opened manually keep a separate shared partition.

Consequences worth planning around:

Sessions start logged out. A login the user (or another session) holds in their own browser is not visible here. Log in within this session’s window when a task needs authenticated access; the login persists for this session across turns.
Windows are session-sticky. Across turns a session re-binds the same window it used last turn; it never adopts another session’s window. A user-opened window with no session history can still be adopted.
Hidden windows are reaped. A hidden, unbound session window is destroyed after the workspace’s idle TTL (Workspace Settings → Advanced → idle browser window timeout; default 60 minutes, 0 disables). User-opened windows are never reaped. After a reap, the next open simply creates a fresh window — but page state and logins in that window’s partition survive only if the partition persists (it does; only the window is destroyed).
Closed windows fail clearly. Commands against a window that was closed return Browser window was closed (instance: …) — re-run open to get a fresh window.

Common validation errors

Missing command... → pass a command string (try --help)
Unknown browser_tool command ... → typo/unsupported verb; check help
...requires ... → required argument is missing for that command
...must be numbers → numeric argument parse failed

Secondary helper: `browser-tool parse-url`

Use this for safe URL debugging in Explore mode without running a generic interpreter snippet:

bun run browser-tool parse-url https://example.com/path?q=1#hash
bun run browser-tool parse-url file:///Users/me/Desktop/report.html

Output is deterministic JSON (href, protocol, host, hostname, pathname, search, hash, origin, plus basename for file:// URLs).

Behavior notes

Browser tools are allowed in Explore/Safe mode by default.
Before first browser tool usage, the agent must read this guide (~/.craft-agent/docs/browser-tools.md).
Closing browser UI via OS controls may hide the window; use browser_tool close for explicit teardown.

Recipe: Canvas-based UIs (Google Sheets, etc.)

Canvas-based web apps (Google Sheets, Google Docs, some map/chart UIs) render content as pixels on <canvas> — individual cells or elements are not DOM nodes and won’t appear in snapshot. Use these patterns instead:

Google Sheets workflow

# 1. Navigate and wait for load
navigate https://docs.google.com/spreadsheets/d/{id}/edit
wait selector [aria-label="Name Box"] 10000

# 2. Navigate to a cell via Name Box (a DOM element — snapshot finds it)
snapshot
click @nameBoxRef
type A1
key Enter

# 3. Edit a cell
key F2
type Hello World
key Enter

# 4. Bulk write via TSV clipboard paste
snapshot
click @nameBoxRef
type A1
key Enter
paste Name\tAge\tCity\nAlice\t30\tNYC\nBob\t25\tLA

# 5. Read data via clipboard
key a meta           # Select all (Cmd+A)
key c meta           # Copy (Cmd+C)
get-clipboard        # Returns TSV string

# 6. Click a canvas cell by coordinates (from screenshot)
click-at 350 200

# 7. Move a chart by dragging (coordinates from screenshot)
drag 400 300 100 50

# 8. Read data via export URL (no editing needed)
navigate https://docs.google.com/spreadsheets/d/{id}/export?format=csv&gid=0

Key principles for canvas UIs

Name Box and formula bar are DOM elements — snapshot can find them
Cells are canvas pixels — use click-at or keyboard navigation, not click
Charts and objects are moveable — use drag to reposition elements on the canvas
Keyboard shortcuts are more reliable than clicking — use key for navigation
Clipboard TSV is the fastest bulk data path — paste with tab-separated values
Export URLs work with session cookies — no API key needed for reads

Troubleshooting

“Browser window controls are not available”

The desktop browser manager isn’t wired for this runtime/session. Ensure you’re in the Electron desktop app and session is initialized.

“Element @eX not found”

Refs are stale. Re-run snapshot and use fresh refs.

Interaction feels flaky

Wait for page readiness and retry using: open → snapshot → interaction

Browser Tools

Browser usage paths

Browser as an Alternative to Source Setup

Core workflow

browser_tool command examples

Quoting and escaping

Key commands

open [--foreground|-f]

snapshot

find <query>

click <ref> [waitFor] [timeoutMs]

click-at <x> <y>

drag <x1> <y1> <x2> <y2>

fill <ref> <value> / select <ref> <value> [--assert-text <text>] [--assert-value <value>] [--timeout <ms>]

upload <ref> <path> [path2...]

type <text>

set-clipboard <text> / get-clipboard

paste <text>

screenshot / screenshot --annotated / screenshot-region ...

console, network, wait, downloads

focus [windowId] / windows