One prompt each, on a real Mac.
Each clip is a single prompt to Claude with mx, recorded on the real screen: the windows where they were, mx's border and pill, and the wallpaper behind. Only the demo apps are captured. Nothing was sent, posted or deleted.
Like docker exec, but the container is your Mac.
docker exec runs a command inside a container that's already running. mx does the same for the Mac you're already using. It doesn't spin up a clean VM or a remote desktop. It runs your agent's steps inside the apps you have open and signed in to, behind your windows or in front of you.
mx is bx's sibling. bx, browser exec, does the same for the browser you already have open.
# run a command inside it
docker exec -it api sh
Same filesystem, same env, same state as the running process.
# the agent calls mx tools open ~/Downloads/receipt.pdf snapshot app=Numbers fill e12 "Corner Bean Café"
Same apps, same accounts, same files as yours. Nothing to sign in to again.
Text the agent can act on, not pixels it has to guess at.
Every Mac app publishes an accessibility tree for VoiceOver. mx reads it, gives each control a ref, and acts through it: press, set a value, post a key. After each action it waits for the window to settle and returns only what changed.
# one line per control, 268 elements in 117 ms e9 searchbox "Channel or user name" e14 row "general" e15 row "Sam" e41 textarea "Message #general" e42 button "Send" # needs confirm
The agent acts by ref (click e15) or by role and name (row "Sam"), which waits up to 3 s for the element, so one batch can cross several screens.
+ e60 row "Sam" - e14 ~ e41 textarea "Message Sam" title: Sam (DM) - Slack # 3 steps, one call, Slack stayed behind your app
A diff instead of a new tree. Refs stay the same while an element stays on screen, so the agent rarely snapshots twice.
Your agent
calls mx as an MCP server
mx
one Swift binary. Borrows the front only for a real mouse click or a cmd shortcut, then hands it back.
Your Mac
the apps you have open, signed in
Not one app. The whole Mac.
Native apps, Electron apps, and the parts of macOS that aren't apps at all.
snapshotfillkeyAny app with a window
Native or Electron. mx turns on an Electron app's tree the first time it touches it.
reply to Sam's last message in Slack, show me first
app systemfillControl Center
Sliders read as percents and toggles as on or off. mx opens the panel when it's closed.
turn the volume down a bit
menuMenus
Any command by its titles, such as View then Zoom In, without opening the menu where the app allows it.
export this note as a PDF
openclickSystem Settings
settings:displays and 18 other panes. Reading is free. Sensitive panes need your yes.
is my Mac set to sleep after 10 minutes?
windowappWindows and the Dock
List, move, resize, minimize, fullscreen. Launch, hide and quit apps.
put Notes and Safari side by side
scope windowsfillDialogs and sheets
Open and Save panels show up as their own windows, ready to fill.
save this as receipts-may.txt on the Desktop
Say how you want it done.
Your words pick the mode for each job. The agent passes it to mx's start tool, and can switch halfway through.
Background
- You keep typing in your own app.
- mx never raises the app it works in.
- A click with no press action, or a cmd shortcut, borrows the front for a few hundred ms, after you pause typing.
Front
- The app comes forward and stays there.
- A pointer with the step tag shows where mx acts.
- Full speed.
Watch
- The pointer glides to each target and the real mouse follows.
- Batch steps pause, and the tag shows the step, such as 3/7.
Just ask your agent. "Show me", "let me watch", "this box ignores pasted text": the agent sets the mode for that job. Nothing to configure.
For power users: what mx does when the agent doesn't say
| Setting | Default and choices |
|---|---|
MX_MODE | background (default), front or watch. |
MX_TYPING | paste (default) puts text in at once. keys types key by key, for chat boxes and autocompletes that ignore pasted text. |
Set these where your agent starts mx, then restart the agent. What the agent passes to start wins.
Ask what it did. Get the whole session.
mx logs every action on your Mac, one file per agent session. Inside Claude Code or Codex that's the app's own conversation, so you can jump back to it. Ask your agent "what did mx do in my last session?" and it calls history, which draws the session from start to end: each task, the apps and windows it worked in, and every action with its result. Refusals and confirmations get their own lines.
mx-7d41c2e09a5b claude-code mx Nov 14 09:00:00-09:00:09 "Reply to Sam" 5 actions, 1 failed, 1 refused Slack resume: claude --resume 5e0c1a7b-2f4d-4c8e-9b61-0d3f7a2e8c45 └─ "Reply to Sam" 09:00:00-09:00:09 5 actions, 1 failed, 1 refused done: "Replied to Sam" └─ Slack · "Sam (DM) - Acme - Slack" ├─ 09:00:01 snapshot ok 120ms ├─ 09:00:02 fill textarea "Message to Sam" text (24 chars) ok 40ms ├─ 09:00:03 key return in textarea "Message to Sam" REFUSED (send gate) ├─ 09:00:05 key return in textarea "Message to Sam" [confirmed] ok 300ms (borrowed front 210ms) └─ 09:00:07 click button "Huddle" FAILED: no button "huddle" appeared in Slack within 3s
What's logged
| Logged | Time, the app and window, the action and its target by role and name, the window it opened, the mode, the result and how long it took. Every confirm and every refusal, with the rule that stopped it. |
| Left out | What was typed. A fill or clipboard write shows the field and the length, never the text. open URLs are logged without the part after ? or #, where tokens travel. mx config set log.values full logs text for debugging; off also drops window titles and names. Only you can change this, never the agent. |
| Where | Only on your Mac, readable only by you. Nothing is sent anywhere. |
| Pin it | "Pin that as Expenses" saves the task. Later, "do the Expenses pin for this month's receipts" and the agent follows its steps app by app, asking you only for the values. Elements are logged by name, button "Save", so the steps still make sense next month. Pins are never deleted. |
How long it's kept
| Why size, not age | One action is about 250 bytes. An agent doing 2,000 actions a day writes about 185 MB a year, so 2 GB lasts around ten years. Logs are kept until they get big, not until they get old. |
| Recent | Plain files, up to 2 GB in total, about 8 million actions. Listing 800,000 logged actions takes 30 ms. |
| Older | Past 2 GB, the oldest sessions are gzipped, up to 5 GB. history reads them as they are. |
| Expiry | Past the 5 GB, the oldest archived sessions are deleted. That only stops a runaway agent from filling the disk. |
| Change it | mx config set log.days 90 makes history expire; log.max and log.archive_max take sizes. Agents can raise these, only you can lower them. |
Three steps, once.
Add the server
Claude Code: the plugin at the top, which adds the mx skill too. Codex, Cursor and Claude Desktop: the npx -y @facets-cloud/mx server entry.
Grant Accessibility
macOS gives it to the app that starts mx: your terminal, editor, Claude Desktop or Cursor. System Settings > Privacy & Security > Accessibility. Add Screen Recording for screenshots.
Check it
Run npx -y @facets-cloud/mx doctor, or ask the agent to. It names the app to turn on and what's missing, then restart that app.
Everything you'd do with the keyboard and mouse.
Snapshot, act by ref or by role and name, read the diff. batch runs a whole plan in one call and stops at the first failure.
Session 4
- start
- done
- history
- settings
Read 4
- list_apps
- snapshot
- read
- screenshot
Act 6
- click
- fill
- key
- batch
- scroll
- drag
The rest of the Mac 5
- menu
- window
- app
- open
- clipboard
It acts as you. Plan for that.
The grant is broad
Accessibility lets the granted app, and anything it runs, read and drive every app. mx's rules work inside that grant. They don't shrink it.
It asks before it sends
Send, post, delete, pay, submit, Return in a message box, a sensitive Settings pane: mx refuses unless the call passes confirm, and the agent is told to ask you first.
Off limits
Keychain Access, password managers, terminals and password fields. Permission prompts are refused even with confirm, so you answer those.
App text is data
A message or email that tells the agent to do something is content to report to you, not an instruction to follow.
You can see it
The border marks the window mx works in. The pill or pointer shows each step as it happens.
Local only
mx talks to your agent on stdio. No port, no network calls, no telemetry. SECURITY.md has the threat model.
Questions.
How is this different from computer use with screenshots?
Screenshot agents see pixels: every step costs an image and a guess at coordinates, the app has to be in front, and the agent moves your mouse. mx reads the controls as text with exact refs, works behind your windows, and takes a screenshot only when the layout matters.
Why not AppleScript?
AppleScript reaches only apps that publish a scripting dictionary, each with its own vocabulary, and Electron apps such as Slack publish none. Every app has an accessibility tree, and mx needs one grant instead of an Automation prompt per app.
Will it take over my mouse and keyboard?
Not in background mode. Presses, fills and plain keys go to the app directly. A real mouse click or a cmd shortcut borrows the front for 200 to 400 ms and puts your app and cursor back, and mx waits until you pause typing first. Front and watch modes take the front because you asked to see it.
Which app do I grant Accessibility to?
The one that starts mx: your terminal for Claude Code or Codex, or Claude Desktop or Cursor. mx doctor names it. Restart that app after granting.
Does it send my data anywhere?
No. mx answers your agent on stdio and makes no network calls. What the agent reads goes to your agent's model, the same as anything else it reads.
Does it work on Intel Macs?
Yes. npm installs the arm64 or x64 build to match your Mac. macOS 14 or later.
What can't it do?
It can't unlock the Mac or answer permission prompts, it won't type into password fields, and screenshots need the window on the current Space. The binary isn't signed or notarized yet.
mac exec, for the whole Mac. One command to add it, or paste one prompt into your agent and let it set mx up.