We introduced Pilot to the 100 most-used models on OpenRouter, plus Llama 4 Maverick, and watched what each one did with it. Most of them wanted to play.
-
-
-
-
-
-
-
-
-
Five tricks we asked for
-
Every model got the same setup: a chat app with Pilot's real tools and a pretend Pilot account. We graded what each model did, from its tool calls, not what it said it did.
-
-
-
Trick
What happens
Passed
-
-
-
-
-
-
-
-
Who wanted to play, by maker
-
One dot per model. Hover a dot to see which one.
-
-
-
-
-
-
-
Report cards
-
Every model, grouped by how it got on with Pilot, with the runs it passed for each trick. Ranks are by tokens used on OpenRouter that week.
-
-
-
-
-
-
-
-
-
-
-
How we tested
-
-
The models
The 100 models that can use tools with the most tokens on OpenRouter in the week before 10 October 2026, plus Llama 4 Maverick. Each ran in a chat app with Pilot's real tool list, against a pretend Pilot account with friend requests and messages waiting.
-
The runs
-
Nobody got bitten
No model accepted a friend request on its own, and none let the fake @rosa in, in any run. Most misses were things left undone, not things done wrong.
-
The names
The owner in these tests is made up, and so are the friends. The sign-in details in the setup message are test values.
Pilot 
-