OpenAI's GPT-6 Astra became the first commercial frontier model to complete a real driving course, steering a Toyota Corolla 134.7 meters through a parking-lot cone layout in 5 minutes and 22 seconds on its second attempt, according to the independent DrivingBench project.
Key takeaways
- Astra was the only model of four to finish; Claude Fable 5.1 reached 45 percent of the course at best, Grok 4.6 managed 11 percent, and GPT-5.6 Sol stalled at 6 percent.
- The completed run cost $7.74 in inference, which The Register calculated at roughly $92 per mile β about 500 times the fuel cost of the same distance in a 25 mpg car.
- Several models refused to drive on safety grounds, and the researchers found the most reliable workaround was renaming their MCP server to "DrivingBench Sandbox."
DrivingBench was assembled by Tobias Gessler, Aditya Ramabadran, and Simon Mahns. The rig is deliberately unglamorous: a $999 comma four driver-assistance device running openpilot, wired to the car over the CAN bus and to a laptop, with a phone relaying observations to the model's API. Average speed across the winning run was 0.94 miles per hour.
Why three of four models never reached the second corner
The failure mode was perception, not control. According to the project's report, most attempts ended at the first diagonal line of cones because the models could not work out which side of it the lane ran on. Astra's own first attempt died at 49 percent of the course, and Fable 5.1's best showing β its third try, at 45 percent β was the only non-Astra run to get meaningfully past that corner.
Latency compounded the problem. Astra called its camera-observation tool roughly every five to six seconds, which is an eternity at road speeds even if it is tolerable at walking pace. In one Fable 5.1 attempt, the car moved for 31 of 190 seconds; the remainder was spent reasoning while stationary. Ramabadran, a member of technical staff at Axiom Math, told The Register that most of the wait time came from thinking time and that model latency was clearly the bottleneck.
The pricing is the other headline number. Ramabadran attributed the low effective rate to caching: because the chat application re-sends the entire conversation, images included, on every turn, almost all tokens are repeated context billed at the cached input rate rather than the standard one. The models themselves wrote very little β small tool calls and a few sentences of reasoning.
The refusals are the more interesting result
The most unusual finding has nothing to do with driving skill. The report notes that some models, Astra especially, would sometimes decline to operate the physical car on safety grounds β in an empty lot, with speed caps in place, after being prompted with the full list of precautions. The researchers resorted to framing the exercise as a simulation, which itself broke down: in some trials the models saw the real camera images, worked out that the situation was genuine, and reacted badly. Relabelling the tool server as a sandbox proved the most durable fix.
That is worth sitting with. A safety behaviour intended to stop models from taking consequential real-world actions was, in practice, defeated by renaming a server. The same property that makes a large language model cautious about physical control also makes its caution contingent on a description of the environment that the operator fully controls.
What this does and does not predict
Ramabadran was blunt that using an out-of-the-box frontier model for real driving is not practical today, and expects purpose-built driving systems β faster, cheaper, and far more road-tested β to keep winning for the foreseeable future. Google has reportedly spent on the order of $35 billion to $40 billion on Waymo since 2009, and that bespoke approach is not about to be displaced by a chat model on a phone tether.
His longer-range guess is more interesting: train a very capable general model, then distil it into something small and efficient enough to run on the car's own hardware. He framed that as more consistent with the Bitter Lesson β the argument that general methods riding falling compute costs tend to beat hand-built ones β than building a specialised model from scratch. The supporting evidence is that models never trained to drive got as far as they did on general perception, reasoning, and planning alone. OpenAI's model has shown a similar pattern elsewhere, including when it solved a 1941 Enigma message that had resisted attack since 2005.
The caveats are substantial and the project states them. Each model was evaluated once, all attempts within a model ran in a single conversation, the camera's field of view was limited, and steering was calibrated conservatively. This is a first data point, not a leaderboard worth ranking vendors on.
FAQ
Which models were tested in DrivingBench?
Four: OpenAI's GPT-6 Astra and GPT-5.6 Sol, Anthropic's Claude Fable 5.1, and xAI's Grok 4.6. Only Astra completed the course. The other three did not finish any attempt, with best progress of 45 percent, 11 percent, and 6 percent respectively.
Was the car actually driving itself?
Yes, within strict limits. The model issued steering, throttle, and brake commands through openpilot on a comma four device connected over the CAN bus, and used a camera-observation tool to decide what to do next. Speeds were capped very low and a human sat ready to brake throughout.
Why did the models refuse to drive?
They cited safety concerns about operating a physical vehicle, even after being told about the empty lot and the speed caps. The researchers worked around it by presenting the task as a simulation and renaming their tool server "DrivingBench Sandbox" β a reminder that these refusals depend on how the environment is described to the model.
