diff --git a/.github/mutant-porte.py b/.github/mutant-porte.py index e17a659f..f8ccc593 100644 --- a/.github/mutant-porte.py +++ b/.github/mutant-porte.py @@ -18,10 +18,14 @@ k = subprocess.run(['node', proeve], capture_output=True, text=True, timeout=600, env={**os.environ, 'TMPDIR': tmp + '/'}) groen[proeve] = k.returncode == 0 except subprocess.TimeoutExpired: - groen[proeve] = False + groen[proeve] = False; k = None finally: shutil.rmtree(tmp, ignore_errors=True) - if not groen[proeve]: print(f"::error::proeven {proeve} er ROED uden mutation - dens mutanter er ikke bevist") + if not groen[proeve]: + print(f"::error::proeven {proeve} er ROED uden mutation - dens mutanter er ikke bevist") + # 30/10 (0.4.1-LISTE #4): uden hvorfor kunne ingen se, at baggrund-mus/stop-alle kun var roede her - vis halen. + ud = ((k.stdout or '') + (k.stderr or '')).splitlines() if k else ['timeout efter 600 s'] + print(f"::group::{proeve} umuteret - de sidste linjer"); print('\n'.join(ud[-40:])); print("::endgroup::") for m in MUTANTER: if not groen.get(m['proeve']): fejl.append(f"{m['navn']}: proeven {m['proeve']} er roed uden mutation"); continue diff --git a/.github/workflows/fremmed-mac.yml b/.github/workflows/fremmed-mac.yml index 1116c052..5aa45490 100644 --- a/.github/workflows/fremmed-mac.yml +++ b/.github/workflows/fremmed-mac.yml @@ -167,6 +167,9 @@ jobs: - name: Mange agenter samtidig id: par continue-on-error: true + # 1/11 (panel R2-R3): parallelproeven hang til jobfristen (41077418047 forsoeg 1); mutationsbeviset blev sprunget + # over, og slutporten afviste koerslen. Et normalt trin tager ~4 min. + timeout-minutes: 30 run: | killall Chess Calculator Contacts "System Settings" "Activity Monitor" Notes Safari "Script Editor" 2>/dev/null || true CMCP_FILM="$RUNNER_TEMP/film" CMCP_PARALLEL_RAPPORT="$RUNNER_TEMP/parallel.json" node test/parallel.mjs diff --git a/CHANGELOG.md b/CHANGELOG.md index 19c072e1..36f5abb9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,7 +1,67 @@ # Changelog -Dates are the day the version was tagged. Everything here was measured before it -was written down; where a claim has a test, the test is named. +Newest first. Where a claim has a test, the test is named; what is not measured +says so. + +## 0.4.1 (in progress) + +**The consent box no longer says password apps ask while the agent works in the +background.** In the background they are refused outright, but the box said +"Password apps still always ask". It now says "Password apps stay refused while +it works in the background." The text shown in the foreground was already true. +`test/ikon-godkend.mjs` D2 checks the wording, and fails with the old text. + +**The MIT license ships with the package.** The npm package did not carry +`LICENSE`; it does now (`npm pack` lists it), and the MCPB bundle copies the same +file list. + +**The lockfile was renewed on 1 November.** An `npx` install has no lockfile and +resolves the newest versions in range; on 1 November that was MCP SDK 1.32.2, +with no `npm audit` findings. 0.4.0's lockfile, which the tests and the MCPB +bundle install from, had SDK 1.32.1 and five audit findings (zod, body-parser, +@hono/node-server, fast-uri, qs). It was replaced that day by a fresh +resolution of this version's `package.json`; newer releases in range can +appear after it. The SDK also has to be at least 1.32.1. + +**Several agents at once.** In `test/parallel.mjs`, 5, 10 and 15 agents work in +the background while another test types into TextEdit. An agent that opens a +Finder window brings Finder forward for a moment, and the front is handed +straight back - the longest one that was handed back lasted 2.8 seconds. In +eleven completed runs on GitHub's Mac, on macOS 15.7 and 15.8 alike, it happened twice at +32 of 33 levels; the recording on the front page shows the same thing. In one +of 65 such moments the front could not be handed back: Finder stayed in front +for the rest of that round (at least 40 seconds), and the reply said so +(`gave_back: false`). Under load an agent can also stop and wait when it cannot +tell which app the person is in - it then does nothing - and one run hung until +GitHub's time limit; the cause of the hang is not found. None of this is fixed +in this version. Every word the typing test sent into TextEdit was verified by +the helper (4,939 of 4,939 across the eleven runs); whether keys typed by hand +during such a moment land in Finder is not measured. The test used to blame +whichever agent happened to start during the moment - even one that only +reads - and now judges it on GitHub's Mac the way it already did on a person's +own Mac. + +**Tests.** `test/stille-vej.mjs` gives `scroll --app` a point, as 0.4.0's helper +requires; on a foreign Mac that check had failed on 0.4.0. The mutation gate now +prints the last lines of a test that fails before anything is mutated. +`scripts/release.sh` looks for the CI run in the last 100 runs, not 20. + +**Known limits found while measuring 0.4.1.** The mutation gate on GitHub's Mac +runs `test/baggrund-mus.mjs` after the many-agents test, and there a menu +opened by a click behind is not closed again (`menu_closed: false` in the +reply) - with Finder in front, and in two runs that put Script Editor in front +first, with Script Editor in front - though these menu checks pass in the +suite. The cause is not found. Its 38 mutants are therefore not proven on a +foreign Mac. In `test/sikker-frigivelse.mjs`, the check that the helper process +ends within 500 ms of a stop (F2b) failed in 4 of 13 completed runs by +1 November (517-546 ms), on 0.4.0 and 0.4.1 alike; the mouse button was +released in all of them (F2). + +**Corrected.** The 0.2.1 notes below said "Password managers ask every single +time, in every mode." In background mode they are refused, not asked; in the +foreground they ask every time. The 0.4.0 notes say "A click behind never leaves +a menu over your work." In the mutation gate on GitHub's Mac a menu opened by a +click behind was left open, and the reply said so (`menu_closed: false`). ## 0.4.0 (in progress) @@ -447,7 +507,8 @@ that it is leaving and the helper finishes the file on its own. A server killed anything. Needs macOS 15. **Consent, as it actually works.** `allow` is the default, so an agent can be -left running. Password managers ask every single time, in every mode. Terminals +left running. Password managers ask every single time, in every mode (corrected +in 0.4.1: in background mode they are refused). Terminals and editors ask once per app per session. Quitting, closing a window, switching Space and destructive-looking menu items ask every time. Several pages said otherwise; they now match the code, and a test derives the forbidden phrasings diff --git a/README.md b/README.md index e8a14efd..3bee9215 100644 --- a/README.md +++ b/README.md @@ -13,8 +13,11 @@ in the hours you are not watching. - **It does not take over your Mac.** It presses buttons and fills fields in windows that stay behind the one you are in, and it leaves your pointer where you put it. When an app pulls itself to the front as it opens, the front is - handed straight back and the reply says so - nothing is left stealing what you - are typing. + handed straight back and the reply says so. In one of 65 measured moments like + it (GitHub's Mac, many agents at once) it could not be, and the reply said that too (`gave_back: + false`): until you click back, what you type goes to that app. It also leaves + the front alone when you clicked, scrolled or pressed a modifier key such as + Shift in the last 1.5 seconds - that is taken to be you. - **It does not pretend.** When something is refused or fails, it says so and says why. The most common complaint about agents driving a computer is that they carry on as if it worked. @@ -133,11 +136,11 @@ and that its chain holds. ```jsonl -{"ts":"2026-10-20T18:52:45.701Z","session":"fabb2663","call":"746fffee","tool":"computer_type","tier":"write","args":{"app":{"length":8,"sha256_12":"1b330dc3a979","salted":true},"text":{"length":37,"sha256_12":"51f664abcb74","salted":true}},"target":"com.apple.TextEdit","mode":"allow","asker":"custom","decision":"allowed","asked":false,"reason":"CMCP_MODE=allow","l":1,"kaede":"e02a3d1b3560ea17"} -{"ts":"2026-10-20T18:52:45.705Z","session":"fabb2663","call":"746fffee","tool":"computer_type","tier":"write","target":"com.apple.TextEdit","phase":"executing","l":1,"kaede":"c1e763106513c3e7"} -{"ts":"2026-10-20T18:52:46.112Z","session":"fabb2663","call":"746fffee","tool":"computer_type","outcome":"ok","effect":"sent","l":1,"kaede":"b053781eb3511c4d"} -{"ts":"2026-10-20T18:52:47.761Z","session":"fabb2663","call":"88f2d9ae","tool":"computer_menu","tier":"write","args":{"app":{"length":6,"sha256_12":"0771e2f86fa0","salted":true},"path":"File > Move to Trash"},"target":"com.apple.finder","mode":"allow","asker":"menubar","presence":{"idle_at_ask":600,"idle_at_end":600,"server":"0.4.0","icon":"0.4.0","box":"on","sessions":1,"surface_derived":"box+menu"},"decision":"denied","asked":true,"reason":"background mode: nobody answered in the menu bar in time","l":1,"kaede":"9b82d6ae5a36942d"} -{"ts":"2026-10-20T18:52:48.229Z","session":"fabb2663","call":"0ba79155","tool":"computer_type","tier":"write","args":{"app":{"length":9,"sha256_12":"84b5a01fae6a","salted":true},"text":{"length":7,"sha256_12":"b56e1d9fbfd2","salted":true}},"target":"com.1password.1password","mode":"allow","asker":"custom","decision":"denied","asked":false,"reason":"background mode: this would need a dialog, and a dialog takes the screen (a password app can never be approved from the menu bar)","l":1,"kaede":"1be981d7eeaf7445"} +{"ts":"2026-10-31T22:35:10.230Z","session":"56f00545","call":"02bc741b","tool":"computer_type","tier":"write","args":{"app":{"length":8,"sha256_12":"78743a52a54c","salted":true},"text":{"length":37,"sha256_12":"cdde26c85cca","salted":true}},"target":"com.apple.TextEdit","mode":"allow","asker":"custom","decision":"allowed","asked":false,"reason":"CMCP_MODE=allow","l":1,"kaede":"4fb5f61a6e442962"} +{"ts":"2026-10-31T22:35:10.231Z","session":"56f00545","call":"02bc741b","tool":"computer_type","tier":"write","target":"com.apple.TextEdit","phase":"executing","l":1,"kaede":"13cd03c71bfb3b38"} +{"ts":"2026-10-31T22:35:10.321Z","session":"56f00545","call":"02bc741b","tool":"computer_type","outcome":"ok","effect":"sent","l":1,"kaede":"79696176e8603981"} +{"ts":"2026-10-31T22:35:11.481Z","session":"56f00545","call":"ac33c074","tool":"computer_menu","tier":"write","args":{"app":{"length":6,"sha256_12":"2c65732d0d4e","salted":true},"path":"File > Move to Trash"},"target":"com.apple.finder","mode":"allow","asker":"menubar","presence":{"idle_at_ask":600,"idle_at_end":600,"server":"0.4.1","icon":"0.4.1","box":"on","sessions":1,"surface_derived":"box+menu"},"decision":"denied","asked":true,"reason":"background mode: nobody answered in the menu bar in time","l":1,"kaede":"ad20dddb1195fa7a"} +{"ts":"2026-10-31T22:35:11.578Z","session":"56f00545","call":"5b15b900","tool":"computer_type","tier":"write","args":{"app":{"length":9,"sha256_12":"ac0afc1b1e97","salted":true},"text":{"length":7,"sha256_12":"f0b579393774","salted":true}},"target":"com.1password.1password","mode":"allow","asker":"custom","decision":"denied","asked":false,"reason":"background mode: this would need a dialog, and a dialog takes the screen (a password app can never be approved from the menu bar)","l":1,"kaede":"2332aa9606dd896e"} ``` @@ -237,13 +240,14 @@ the same way - including when the call failed halfway, because a half-done write that took the screen must not be the one line that stays silent. `press` and `menu` carry it too: an app can bring itself forward in response (measured: File > New Finder Window did), and when it does, the answer says so -and the front is handed straight back to the app you were in (`gave_back`). +and the front is handed straight back to the app you were in - `gave_back` +says whether that worked. `set_value`, `drag`, `paste` and `move` do not carry the field yet. `computer_launch` joins them with `background: true`: the app starts behind what you are doing. Many apps bring themselves forward as they start - six of eight did on a clean Mac - and when one does, the front is handed straight back -and the answer says `took_screen` and `gave_back`. It is a moment, not nothing. Seven are still held back: +and the answer says `took_screen`, and in `gave_back` whether that worked. It is a moment, not nothing. Seven are still held back: `move`, `activate`, `quit`, `space`, `window`, `drag` and `paste` - until you lend the screen: `computer_request_screen` asks you in the box in the corner of your screen (and in the menu bar), one agent at a time, approved with Touch ID, for at diff --git a/bundle/manifest.json b/bundle/manifest.json index e7eb8798..8d2f2313 100644 --- a/bundle/manifest.json +++ b/bundle/manifest.json @@ -2,7 +2,7 @@ "manifest_version": "0.3", "name": "computer-mcp", "display_name": "Computer MCP", - "version": "0.4.0", + "version": "0.4.1", "description": "macOS computer use you can leave running: it stays behind your window and never touches passwords.", "long_description": "A macOS computer-use MCP server built for the hours you are not watching. It acts in windows behind the one you are in and leaves your pointer alone, it says so when something fails, several can run at once, and it will not go near your password manager. Every action is written to an append-only audit log. Runs in any MCP client, macOS, MIT, no account.", "author": { diff --git a/docs/docs/capability-matrix/index.html b/docs/docs/capability-matrix/index.html index 5a709556..4276e6de 100644 --- a/docs/docs/capability-matrix/index.html +++ b/docs/docs/capability-matrix/index.html @@ -221,7 +221,7 @@
| Wall | What actually happens | |
|---|---|---|
| Redacting secrets | measured | Only what macOS marks as secure is painted out. Measured on a real web page: a password input carries role AXTextField with subrole AXSecureTextField and is redacted; a plain text field is not. A password in a text editor, or a token printed in a terminal, carries no marker and will not be hidden. Use readonly when the screen holds something the system cannot know about. |
| Working without taking over the screen | measured | Three kinds, and the difference is how the action is delivered, not what the tool is called. Quiet by delivery: computer_press, computer_set_value and computer_menu fire the element's own accessibility action, so they work on a window behind the one you are in and never touch your pointer. The app can still bring itself forward in response - measured: File > New Finder Window did - and then press and menu answer took_screen and hand the front straight back. Quiet when you name the app: computer_click, computer_type, computer_key and computer_scroll are delivered into that app's own event queue - measured: the text arrived, the pointer stayed where it was left, and the front window did not change. computer_launch joins them with background: true - measured on a clean Mac, six of eight apps brought themselves forward as they started, and the front is handed straight back. Without an app they go through the system input tap and land wherever your keyboard focus is, so in background mode they are refused. One measured limit: keystrokes do not land in a Chromium window that is not focused, so for anything Chromium-based use computer_press and computer_set_value. Held back in background mode: computer_move, computer_drag, computer_paste, computer_activate, computer_quit, computer_space and computer_window. Each either moves your pointer, brings something to the front, or changes what you see - a window moved on a screen you are looking at is a quiet channel with a very visible result. computer_ask_user left this list on 29 September: in background mode its question waits in the menu bar icon instead of a dialog. And the held-back seven are no longer out of reach: computer_request_screen asks you to lend this one agent the screen, approved with Touch ID in the menu bar, for at most 15 minutes. While it has it, each step waits while you are using the keyboard or mouse (a step already running finishes), and one click in the menu takes the screen back and stops a step still running. Set CMCP_BACKGROUND and nothing can borrow it. |
| Working without taking over the screen | measured | Three kinds, and the difference is how the action is delivered, not what the tool is called. Quiet by delivery: computer_press, computer_set_value and computer_menu fire the element's own accessibility action, so they work on a window behind the one you are in and never touch your pointer. The app can still bring itself forward in response - measured: File > New Finder Window did - and then press and menu answer took_screen and hand the front straight back - gave_back says whether that worked (once in 65 moments on GitHub’s Mac it did not). Quiet when you name the app: computer_click, computer_type, computer_key and computer_scroll are delivered into that app's own event queue - measured: the text arrived, the pointer stayed where it was left, and the front window did not change. computer_launch joins them with background: true - measured on a clean Mac, six of eight apps brought themselves forward as they started, and the front is handed straight back - gave_back says whether that worked. Without an app they go through the system input tap and land wherever your keyboard focus is, so in background mode they are refused. One measured limit: keystrokes do not land in a Chromium window that is not focused, so for anything Chromium-based use computer_press and computer_set_value. Held back in background mode: computer_move, computer_drag, computer_paste, computer_activate, computer_quit, computer_space and computer_window. Each either moves your pointer, brings something to the front, or changes what you see - a window moved on a screen you are looking at is a quiet channel with a very visible result. computer_ask_user left this list on 29 September: in background mode its question waits in the menu bar icon instead of a dialog. And the held-back seven are no longer out of reach: computer_request_screen asks you to lend this one agent the screen, approved with Touch ID in the menu bar, for at most 15 minutes. While it has it, each step waits while you are using the keyboard or mouse (a step already running finishes), and one click in the menu takes the screen back and stops a step still running. Set CMCP_BACKGROUND and nothing can borrow it. |
| Several agents at once | measured | Each MCP client runs its own server. Measured: two servers, 50 interleaved writes, zero torn lines, and every line carries a session mark. Writes in the same app take turns: each holds a lock for that app, so keys and clicks never interleave within an action. Between actions they still share one pointer and one focused window; give background work to computer_press. |
| Knowing where the cursor is | measured | computer_focused reports the focused element and whether it is secure. The system-wide lookup alone returns nothing - measured - so it goes through the focused application, then the frontmost app. |
| A Save / Don't Save sheet | measured | The sheet you meet every time you close something with unsaved work. Measured: the sheet is found as AXSheet, all three buttons come back with a center and a pressable flag, and computer_find with contains: "Don't Save" returns exactly that one button - an agent looks for the words, not for a position. It is also the clearest case for the gate: a sheet can throw away work on your behalf, so in read-only mode pressing it is refused and nothing reaches the machine. Measured against a sheet the test puts up itself, in a fully transparent window, so proving it costs nobody their screen. |