Automation

Peekaboo's automation surface is small but covers the whole macOS UI graph. Each command is documented separately under commands/; this page is the map.

#Targeting model

Every input command accepts one of three target shapes:

Element ID — --id E12 (from peekaboo see); the most reliable.
Label / role / app — --label "Send" --app Mail; resolved via the AX tree.
Coordinates — --at 480,120; the fallback when the AX tree lies.

Prefer IDs when you can capture them, labels when you can't, and coordinates only as a last resort. The agent and MCP tooling default to the first two.

#Input primitives

Command	Use it for
click	mouse clicks, double/triple, right/middle, hold
type	typing strings into focused fields
press	individual key presses (return, escape, arrows, etc.)
hotkey	shortcut combos, including background apps
scroll	wheel scrolling at a point or on a target
drag	press, move, release — files, sliders, selections
swipe	trackpad-style multi-finger gestures
move	warp the mouse without clicking
set-value	write to text fields without typing
perform-action	trigger any AX action (`AXPress`, `AXShowMenu`, …)
sleep	wait between steps with deterministic timing

For UX parity with humans (jitter, easing, dwell), see human-typing.md and human-mouse-move.md.

#Surfaces

Surface	Command	Notes
App lifecycle	app	launch, quit, focus, hide
Windows	window	move, resize, focus, minimize, fullscreen
Spaces & Stage Manager	space	enumerate and switch Spaces
Menus	menu	walk app menus by path
Menu bar / status items	menubar.md	extra-fiddly popovers
Dialogs	dialog	sheets, alerts, save panels
Dock	dock	inspect/click dock items
Clipboard	clipboard	read/write pasteboard contents
Open files / URLs	open	with focus controls
Visual feedback	visualizer	overlay so a human can follow what the agent is doing

#Recipe: click a button by label

# 1. Inspect first to find a stable label.
peekaboo see --app Safari --annotate --output safari.png

# 2. Click it.
peekaboo click --label "Reload" --app Safari

#Recipe: a small flow

peekaboo app focus --name "Notes"
peekaboo hotkey cmd+n
peekaboo type "Standup notes\n\n- Shipped Peekaboo docs\n- Reviewed PR #42\n"
peekaboo hotkey cmd+s

Three primitives, four lines. The agent does the same thing under the hood — it just plans the sequence for you.

#Resilience tips

Always run peekaboo see when an element is unreachable. The AX tree refreshes after focus changes; capture again if a click fails.
Use focus and application-resolving for tricky cases (multiple windows, helper apps, processes that hide on activation).
Wrap risky sequences with peekaboo sleep 0.2 — humans don't fire ten clicks in a single frame, and neither should you.
Prefer hotkey --focus-background when you need to drive an app without stealing focus from the user.

#Going further

Agent overview — let Peekaboo plan input sequences from a goal.
MCP — expose all of the above to Codex, Claude Code, and Cursor.
Architecture — how the input pipeline routes through Bridge and Daemon.