1e:I[70616,["/_next/static/chunks/0kcf5dt6yxdox.js","/_next/static/chunks/1su5n1uv1ti4s.js","/_next/static/chunks/0gvke1wv6udfg.js","/_next/static/chunks/1dfrle3jgmlzh.js","/_next/static/chunks/162lgvduyr-fa.js","/_next/static/chunks/2gcnnopam8e9w.js","/_next/static/chunks/2g60iab5a1pkg.js"],"Breadcrumb"] 1f:I[90762,["/_next/static/chunks/0kcf5dt6yxdox.js","/_next/static/chunks/1su5n1uv1ti4s.js","/_next/static/chunks/0gvke1wv6udfg.js","/_next/static/chunks/1dfrle3jgmlzh.js","/_next/static/chunks/162lgvduyr-fa.js","/_next/static/chunks/2gcnnopam8e9w.js","/_next/static/chunks/2g60iab5a1pkg.js"],"ArticleGrid"] 20:T19c8,

Turn a language model loose on code that already works and it will, with some regularity, break it. A new report on arXiv puts arithmetic behind that intuition: in one configuration, an automated repair loop damaged correct programs roughly 26% of the time while repairing genuinely broken ones about 2.3% of the time — a ratio of more than ten to one in the wrong direction.

Key takeaways

Measured on competitive programming submissions, an LLM repair loop damaged working code at a rate of 0.261 while fixing buggy code at only 0.023.
Left to iterate, the models fell into pseudo-bug-fixing cycles, adding and removing the same edit indefinitely, with search-and-replace edits cycling more than whole-file rewrites.
Probing found an internal direction that predicts editing propensity with near-perfect separation by layer 19; suppressing it stopped the destructive loop entirely — and stopped all real repairs too.

The measurement

The paper, If It's Not Buggy, Don't Fix It by Xietao Wang-Lin, Anton Isopoussu and Louis Mahon, examines what the authors call blind iterative use: handing a model a program, accepting whatever edit comes back, and repeating. It is a fair description of how a growing number of teams wire repair bots into code review.

The test bed was CodeContests+ — 20 problems, 40 C++ submissions each, averaging 23 test cases per problem — run against Google's Gemini 2.5 Flash-Lite and Alibaba's Qwen2.5-7B-Instruct. With greedy decoding and search-and-replace editing, Gemini's repair rate came in at 0.023 ± 0.002 against a damage rate of 0.261 ± 0.032; a second configuration put damage at 0.293 ± 0.011. The authors' summary is blunt: the damage rate can end up significantly higher than the repair rate.

Two caveats belong next to those figures. The models under test are small, fast ones rather than frontier systems, and competitive programming submissions are not a production codebase with its own test suite, review gates and CI. The benchmark measures a loop with no stopping condition and no human in it, which is the point — but it is not the same as measuring a well-instrumented repair pipeline.

The loop that will not settle

The more unsettling finding is what happens when you let the process run. Rather than converging on a stable version, the models frequently entered what the paper calls a pseudo-bug-fixing cycle: the same change is introduced, removed, and introduced again, with the program oscillating between passing and failing states. Search-and-replace blocks produced significantly more of these cycles, and longer ones, than whole-file edits did.

That matters operationally because a cycling agent looks busy. It generates diffs, it reports activity, it consumes tokens — and, absent a test oracle wired into the loop, nothing in its output signals that it is oscillating rather than progressing.

An internal direction for "this looks buggy"

The mechanistic section is where the paper gets genuinely interesting. Probing the model's activations, the authors constructed a steering vector that tracks editing propensity, with separation rising from an AUC of 0.7–0.8 in early layers to near 1.0 by around layer 19. The implication is that the model carries an internal representation of "buggy code" — and that this representation is being falsely triggered on programs that are perfectly fine, which is hallucination with a concrete address in the network.

Steering against that direction at γ = -0.5 shut the destructive loop down completely, preserving every correct submission. It also fixed nothing. That is not a deployable knob so much as a clean demonstration that willingness to edit and ability to repair are, in these models, entangled — you cannot simply dial down the false positives and keep the true ones.

What this changes for teams running repair bots

The practical reading is not that automated repair is useless but that the stopping condition is the product. A repair tool without a reliable oracle for "there is nothing here to fix" will spend most of its edits on code that did not need them, and the arithmetic above suggests those edits are net negative.

It also lends weight to an architectural trend already visible in shipping tools. Review-and-revise patterns that put an independent critic between a draft and the working tree — including the cross-family critique step in GitHub's newly previewed orchestration layer, which we covered in our report on Project HydraFusion — exist precisely to interrupt the blind loop this paper measures. Bounded iteration, tool-less reviewers and fail-safe patch application stop being conservative design choices once damage outpaces repair by an order of magnitude.

Outlook

Whether these ratios hold for frontier models on real repositories is the obvious follow-up, and nothing here settles it. What the report does establish is that the failure mode is structural rather than incidental: the propensity to edit is represented internally, it activates on healthy code, and iteration amplifies it instead of washing it out. Anyone building fully autonomous repair should be designing around that rather than waiting for the next model to make it go away.

FAQ

Which models were tested in the study?

Gemini 2.5 Flash-Lite and Qwen2.5-7B-Instruct. Both are small, inexpensive models rather than frontier systems, so the reported rates should not be read as measurements of what the largest coding models do.

Does this mean AI code review tools make code worse?

Not on its own. The study measures blind iteration with no stopping condition and no human review, which is the worst case rather than the typical deployment. Its relevance grows in proportion to how autonomously a repair tool is allowed to run.

Can the false bug detection simply be turned off?

Not cleanly. The authors found a steering vector that eliminates the destructive editing loop, but applying it also eliminated every successful repair — suggesting the same internal representation drives both behaviours.

a:["$","div",null,{"className":"mx-auto max-w-7xl px-4 py-8 sm:px-6","children":[["$","script",null,{"type":"application/ld+json","dangerouslySetInnerHTML":{"__html":"{\"@context\":\"https://schema.org\",\"@type\":\"BreadcrumbList\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https://ainewsway.com\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"#automated-program-repair\",\"item\":\"https://ainewsway.com/tag/automated-program-repair\"}]}"}}],["$","script",null,{"type":"application/ld+json","dangerouslySetInnerHTML":{"__html":"{\"@context\":\"https://schema.org\",\"@type\":\"ItemList\",\"@id\":\"https://ainewsway.com/tag/automated-program-repair#itemlist\",\"name\":\"#automated-program-repair Articles\",\"numberOfItems\":1,\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"url\":\"https://ainewsway.com/articles/llm-bug-fixers-broke-working-code-ten-times-more-often-than-they-fixed-broken-code-mtzlmrg2\",\"name\":\"LLM Bug-Fixers Broke Working Code Ten Times More Often Than They Fixed Broken Code\"}]}"}}],["$","$L1e",null,{"items":[{"label":"# automated-program-repair"}]}],["$","h1",null,{"className":"mb-8 text-3xl font-bold text-zinc-900 dark:text-zinc-100","children":["#","automated-program-repair"]}],[["$","$L1f",null,{"articles":[{"id":"3225232f-0c0a-4f4e-b2e9-8a9478335bf6","slug":"llm-bug-fixers-broke-working-code-ten-times-more-often-than-they-fixed-broken-code-mtzlmrg2","title":"LLM Bug-Fixers Broke Working Code Ten Times More Often Than They Fixed Broken Code","subtitle":"A new arXiv report measures blind iterative repair — and finds a steering vector that switches the damage off, along with every real fix","content":"$20","excerpt":"An arXiv study clocked an LLM repair loop damaging correct programs at 0.261 while fixing buggy ones at 0.023, then found the internal direction driving it.","coverImage":"https://cdn.ainewsway.com/uploads/2026/09/727196d3-6222-454d-a239-f06cda0d0527.jpg","coverImageCaption":"Source code under review: the study measured what happens when a language model is handed working code and asked to fix it anyway.","authorId":"6773f67d-d7fa-4b19-8f38-384f10ac761b","status":"PUBLISHED","publishedAt":"2026-09-13T09:16:29.283Z","viewCount":0,"likeCount":0,"readingTime":5,"isFeatured":false,"isBreakingNews":false,"metaTitle":"Study: AI Bug-Fixers Break More Code Than They Repair","metaDescription":"An arXiv study measured an LLM repair loop damaging working code at 0.261 versus a 0.023 repair rate, and traced it to an internal steering vector.","metaKeywords":["automated program repair","LLM code review","AI coding agents","arXiv study","Gemini 2.5 Flash-Lite","Qwen2.5","steering vector","AI hallucination"],"ogImage":null,"aiSummary":"A new arXiv report finds that language models used as blind iterative bug-fixers damage correct programs far more often than they repair broken ones, measuring a 0.261 damage rate against a 0.023 repair rate for Gemini 2.5 Flash-Lite on competitive programming submissions. The models also fall into endless pseudo-bug-fixing cycles. Probing revealed an internal direction controlling editing propensity, which when suppressed stopped the damage and all successful repairs alike.","aiTags":[],"sentiment":null,"articleType":"ANALYSIS","videoEmbed":null,"videoType":null,"seoScore":85,"isAutoGenerated":true,"isBackfill":false,"pipelineStage":null,"copyrightScore":null,"similarityScore":null,"matchedSourceId":null,"createdAt":"2026-09-13T09:15:47.956Z","updatedAt":"2026-09-15T11:44:50.144Z","contentUpdatedAt":"2026-09-15T11:44:50.143Z","deletedAt":null,"author":{"id":"6773f67d-d7fa-4b19-8f38-384f10ac761b","name":"Seung Jung","username":"Seung Jung","avatar":null},"categories":[{"id":"32f2303b-b593-473f-bf9c-6a99acbe6ce1","name":"Developer Tools","slug":"developer-tools"}],"tags":[{"id":"244fd9e0-6680-4dd0-9e6a-4e4893c5fbc0","name":"Automated Program Repair","slug":"automated-program-repair"},{"id":"18cdd61e-c75a-4aff-843e-db288dddc418","name":"Coding Agents","slug":"coding-agents"},{"id":"11d8a572-c077-4bf0-b5d3-993830882cbf","name":"LLM","slug":"llm"},{"id":"75a3bf99-8d8d-4109-a75c-238ead02890e","name":"Gemini","slug":"gemini"},{"id":"971d527f-0508-47c9-aac1-39914a5f883c","name":"AI Agents","slug":"ai-agents"}],"translations":[{"id":"a574c57c-29f3-4dfe-91d4-86b133944cc2","language":"ko","title":"LLM 버그 수정기, 멀쩡한 코드를 고친 것보다 열 배 더 망가뜨렸다","subtitle":"arXiv 신규 보고서가 맹목적 반복 수리를 측정했다. 손상을 끄는 조향 벡터를 찾았지만, 진짜 수리까지 함께 꺼졌다","excerpt":"arXiv 연구에서 LLM 수리 루프의 정상 코드 손상률은 0.261, 버그 코드 수리율은 0.023으로 나왔다. 연구진은 이를 몰고 가는 내부 방향까지 찾아냈다.","metaTitle":"연구: AI 버그 수정기, 고치는 것보다 망가뜨리는 게 많다","metaDescription":"arXiv 연구가 LLM 수리 루프의 정상 코드 손상률 0.261, 수리율 0.023을 측정하고 이 행동을 내부 조향 벡터로 추적했다."},{"id":"6a1ba540-d22b-404d-b3bb-dc13f5ab1103","language":"ja","title":"LLMのバグ修正は、直した数の10倍も動くコードを壊していた","subtitle":"arXivの新報告が盲目的な反復修復を測定。破壊を止める操舵ベクトルも見つかったが、成功した修復まで一緒に消えた","excerpt":"arXivの研究で、LLM修復ループが正しいプログラムを壊す率は0.261、バグを直す率は0.023だった。著者らはその挙動を駆動する内部方向まで突き止めた。","metaTitle":"研究: AIのバグ修正は直すより壊す方が多い","metaDescription":"arXivの研究がLLM修復ループの破壊率0.261と修復率0.023を測定し、その挙動を内部の操舵ベクトルにまで追跡した。"},{"id":"7508aeb0-14b8-470b-b080-16a2c2d54e21","language":"zh","title":"LLM修復程式碼時弄壞可用程式碼的次數，是真正修好次數的十倍","subtitle":"arXiv新報告測量了盲目迭代修復，並找到一個能關掉破壞行為的引導向量——連同所有真正的修復一起關掉","excerpt":"一項arXiv研究測得LLM修復迴圈破壞正確程式的比率為0.261，修復缺陷程式的比率僅0.023，並找到了驅動這一行為的內部方向。","metaTitle":"研究：AI修復程式碼，弄壞的比修好的更多","metaDescription":"一項arXiv研究測得LLM修復迴圈破壞可用程式碼的比率為0.261，而修復率僅0.023，並將該行為追溯到模型內部的一個引導向量。"},{"id":"47b3b953-dc23-4166-b800-4c02b14ec7bb","language":"es","title":"Los reparadores con LLM rompieron código que funcionaba diez veces más a menudo de lo que arreglaron el defectuoso","subtitle":"Un nuevo informe en arXiv mide la reparación iterativa ciega y encuentra un vector de dirección que apaga el daño, junto con todas las reparaciones reales","excerpt":"Un estudio en arXiv midió un bucle de reparación con LLM dañando programas correctos a 0,261 y arreglando los defectuosos a 0,023, y halló la dirección interna que lo impulsa.","metaTitle":"Estudio: la IA rompe más código del que repara","metaDescription":"Un estudio en arXiv midió un bucle de reparación con LLM dañando código funcional a 0,261 frente a una tasa de reparación de 0,023, y lo rastreó hasta un vector interno."}],"celebrities":[{"id":"bccb3daa-91a5-42d2-b58d-a641f2e3d534","nameKo":"구글 딥마인드","nameEn":"Google DeepMind","slug":"google-deepmind","profileImage":null}]}]}],"$L21"]]}]

#automated-program-repair

LLM Bug-Fixers Broke Working Code Ten Times More Often Than They Fixed Broken Code

Subscribe to Newsletter

#automated-program-repair

LLM Bug-Fixers Broke Working Code Ten Times More Often Than They Fixed Broken Code