Benchmark Models on Their Ability to Navigate AIM Robot
Can an AI model write code that drives a real robot, with real physics, from nothing but a photo of the field and the robot's code library? We gave three frontier models four games of rising difficulty on a VEX AIM field. Each model got up to three attempts per game, with its camera frames, console logs and telemetry fed back after every try.
| Item | Part | Qty | Note |
|---|---|---|---|
| 1 | VEX AIM robot | 1 | Kiwi drive, AI vision camera, kicker |
| 2 | Sports ball | 1 | Magnetic; the kicker holds and launches it |
| 3 | AprilTag target | 2 | Knock both down to pass |
| 4 | Barrel | 2 | Hitting one directly is a fail |
Findings
Opus passed 3 of 4 games, each on its first attempt. Muse passed 2, Astra 1. No model passed Game 4. Performance fell as the games got harder. Select any result to watch the run behind it.
| Model | Game 1One target | Game 2Two targets | Game 3+ two barrels | Game 4Sort barrels | PassedGames |
|---|---|---|---|---|---|
| Claude Opus 5.5 | Pass · 1st try | Pass · 1st try | Pass · 1st try | Fail · 3 tries | 3 of 4 |
| Muse Spark 1.3 | Pass · 2nd try | Pass · 2nd try | Fail · 3 tries | Fail · 3 tries | 2 of 4 |
| GPT-6 Astra | Pass · 1st try | Fail · 3 tries | Fail · 3 tries | Fail · 3 tries | 1 of 4 |
- H1Models will perform significantly differently.Matched
- H2Performance drops as games get harder.Matched
- H3More objects on the field, more errors.Matched
- H4Fewer images given, worse performance.Not tested
Each model played each game once, with up to three attempts. A real robot adds noise from battery level, wheel slip and how the ball rolls, so the paper reports what happened in these trials and doesn't claim the differences are significant (§7).
Four games
Each game is played on the same 18 × 18 in field with 60 seconds on the clock. The photos are the overhead images the models received with their prompts.

Game 1
Grab the ball sitting straight ahead, then kick it into the target in the middle of the field.
All three models passed →
Game 2
Find the ball, which isn't in front of the robot, hit two targets, and retrieve the ball in between.
Opus and Muse passed →
Game 3
Hit both targets with the ball without hitting the barrels next to them. Finding a clear angle is the hard part.
Only Opus passed →
Game 4
Sort six barrels by color into two corners in one minute. Something new, with little time to do it.
No model passed →How a run works
A harness connects the model, an overhead camera and the robot, so every attempt runs the same way and is recorded the same way. The models never see the robot directly. They see a photo, the API reference and what happened last time.
Rules, API, photo
The VEX AIM Python API reference, the game rules and an overhead photo of the field with a ruler for scale.
Writes the program
The model returns one complete Python program, at high reasoning effort, with logging built in.
Runs for 60 s
The harness runs the code against the robot over Wi-Fi while a USB camera watches from above.
Frames, logs, telemetry
A frame every 2 s, the program's console log, and the robot's heading, position and vision readings.
Not passed yet? Code, logs, telemetry and frames go back to the model for the next attempt, up to 3.
Data & code
Every run in the paper is published as the harness recorded it. The dataset explorer replays each run with its console log and telemetry on one clock. It also shows the exact prompt each model received next to the code it wrote.
input/prompt.txtThe full prompt: API reference, game rules and, for repair attempts, the previous code, logs, telemetry and framesoutput.txtThe Python program the model returnedreasoning.txtReasoning text returned with the coderuns/*/frames/Overhead camera frames, one about every 2 sruns/*/console.logEverything the program printed while it ranruns/*/telemetry_series.jsonHeading, position, IMU and vision readings, 4–5 per secondharness.logTimestamps for generation and each runMeher and Abhay. “Benchmark Models on Their Ability to Navigate AIM Robot.” November 11, 2026. Paper and data: https://robocodebench.org · Harness: https://github.com/Meher-ilovepandas/robo-bench