GPT-6 Edits Qwen Weights, Math Score Hits 48%

Four MIT CSAIL researchers said on Nov. 2 that OpenAI's GPT-6 Astra can read the raw weights of small neural networks and rewrite them by hand, with no gradient descent or training run. In their headline test, the model edited 0.016% of the weights in Alibaba's Qwen3.5-0.8B and lifted its MATH-500 score from 7.9% to 48.1%.

The work by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab appeared as a blog post, "Sparks of Weight-Space Programming," not a peer-reviewed paper. Full logs and code are on GitHub.

Key takeaways

• GPT-6 Astra traced a looping failure in Qwen3.5-0.8B to one direction in two layers and subtracted it from 131,072 weights, raising MATH-500 from 7.9% to 48.1% and GSM8K from 13.8% to 67.1%.
• Starting from all-zero weights, Astra wrote a Breakout-playing network that scored 517.4, against 25.3 for a network trained with PPO for 500 million steps.
• The authors call the results "very early": the same edit gave no real gain on a model without the looping flaw, and Astra could not spot a sycophantic model from weights alone.

What is weight-space programming?

Networks are normally shaped by an optimizer that nudges millions of numbers toward lower error. The MIT team instead gave a language model a plain-English spec and let it read and write weight tensors directly, sometimes with no environment feedback.

According to the project's GitHub README, the model was "gpt-6-astra" run through OpenAI's Codex CLI with its built-in tools disabled, and all 1,125 turns were audited as tool-free. In some tests Astra got a small budget of "forward passes," chances to run the network and watch the output. In the Qwen test, one pass meant answering up to 512 practice problems.

How GPT-6 Astra fixed Qwen3.5-0.8B

The base Qwen model, with thinking mode on, hit its 32,768-token output limit on 93% of answers because it looped. Astra traced the loop to a single direction written by the MLP blocks of layers 15 and 19 and subtracted it from 64 output columns in each layer.

The same approach failed on Google's Gemma 3 270M, which moved from 6.8% to at most 7.7% on MATH-500. Only 8.5% of Gemma's answers hit the token limit. "Editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked," the authors wrote.

Networks written from scratch

In other tests, Astra wrote small game-playing networks starting from zero weights, given only the rules and source code. Its one-shot Breakout network scored 517.4. On Seaquest it scored 53.1, below the 88.9 of a 500-million-step PPO network, and on Asterix 53.2 against 55.2.

Astra also fixed a stalled car-on-ramp control policy with one edit to 3.5% of its weights, going from 0 of 256 successful episodes to 1,024 of 1,024.

Limits the authors acknowledge

Astra named the right game for four policy networks from their weights alone. But when asked which of two fine-tuned Gemma models was sycophantic, it failed all four tries using weights only. It succeeded on all three pairs only after seeing attention patterns.

The researchers call the capabilities "exciting but still limited," citing context length and harness inefficiency. The largest model tested had 0.8 billion parameters.

What's next

The team says it is building a benchmark and harness for weight-space programming. Future directions it lists include auditing models for hidden backdoors and unlearning without data. No independent replication on larger models had been reported as of Nov. 3.

FAQ

Did GPT-6 Astra retrain Qwen?
No. It changed 131,072 weights by reasoning about them directly, without gradient descent or a training loop.

Can this make any model smarter?
Not on current evidence. The gain came from removing a looping flaw; a Gemma model without that flaw saw no meaningful improvement.

References

• Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights — Takehi, Teoh, Zhang, Khattab, MIT CSAIL (Nov. 2, 2026)
• RikiyaT/lm_understand-logs — GitHub (accessed Nov. 3, 2026)
• MIT Researchers Show GPT-6 Astra Can Rewrite Another AI's Brain Without Training It — Startup Fortune (Nov. 3, 2026)