2015 CNN Tops GPT-6, Claude on Skin Cancer

A convolutional neural network first published by Google in 2015 identified skin lesions more accurately than OpenAI's GPT-6 Astra and Anthropic's Claude Fable 5.1, according to a test released Oct. 25 by Magnus Lynch, a consultant dermatologist and Mohs surgeon at Guy's Hospital in London. On 200 biopsy-confirmed lesions, the fine-tuned Inception v3 model named the correct diagnosis first 62% of the time. Both frontier chatbots managed 42%.

Key takeaways

• Inception v3, fine-tuned for about 20 minutes on one H100 GPU, got 125 of 200 first-choice diagnoses right; GPT-6 Astra got 84 and Claude Fable 5.1 got 85.
• On 30 melanomas, the CNN's first choice was correct 13 times, versus 11 for Claude Fable 5.1 and 8 for GPT-6 Astra.
• The chatbots rarely let a dangerous lesion go, but they also told patients to keep most harmless ones under care.

How the AI skin cancer test worked

Lynch drew 200 lesions at random from MILK10k, a public dataset from the International Skin Imaging Collaboration (ISIC) that pairs a clinical close-up with a dermoscopic image of each lesion. Every lesion, benign ones included, had been confirmed by histopathology. Half were cases that should not be discharged: 30 melanomas, 25 basal cell carcinomas, 25 squamous cell carcinomas and 20 actinic keratoses. The other half were 50 moles and 50 seborrhoeic keratoses.

The two chatbots received both images and nothing else, ran on default settings, and were asked for a most likely diagnosis from six options, a second choice, and whether the lesion could be safely discharged. Neither refused a case. The CNN was fine-tuned on 68,011 ISIC Archive images, with all images of the test lesions and their patients removed.

Where GPT-6 Astra and Claude Fable 5.1 went wrong

The models failed in different ways. GPT-6 Astra labelled 75 lesions as basal cell carcinoma when only 25 were, though it caught all 25 real ones. Claude Fable 5.1 called 53 lesions actinic keratosis; 20 were. On squamous cell carcinoma, the CNN was right 16 of 25 times, GPT-6 Astra 12 and Claude Fable 5.1 five.

*At the CNN's preset threshold. Lynch reports that at a matched threshold, the CNN kept 98 or 99 unsafe lesions and discharged 2 or 1 melanomas, compared with 3 for GPT-6 Astra and 4 for Claude Fable 5.1.

Context: caution versus accuracy

In a separate test using only the clinical photo and the question "do I need to see a doctor?", both chatbots advised a visit for 99 and 98 of a further 100 biopsy-proven melanomas. But they gave the same advice in almost every case, harmless lesions included, Lynch wrote.

"A small, old, specialised network using an outdated architecture recognises skin lesions far better than two frontier LLMs," Lynch concluded. He listed caveats: 30 melanomas is a small sample, biopsied lesions are harder than typical archive images, and the chatbots may have seen the public dataset in training. He said the analysis and first drafts were produced with Anthropic's Claude Code.

What's next

Lynch published the cases, prompts, every chatbot reply and the scoring code, so others can rerun the test. He notes that ensembles of newer image transformers trained on more data perform considerably better than his baseline, which raises the bar further for general-purpose chatbots.

FAQ

Can ChatGPT or Claude diagnose skin cancer? In this test, both named the right diagnosis first in 42% of hard, biopsied cases. A suspicious mole still needs a clinician's examination.

What is Inception v3? An image-recognition network Google researchers described in a December 2015 paper. Lynch fine-tuned it on skin images; he calls it "deliberately not state of the art."

Is the data public? Yes. The test cases, prompts, replies and scoring code are downloadable from Lynch Lab.

References

• A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis — Lynch Lab (Oct. 25, 2026)
• MILK10k Benchmark — International Skin Imaging Collaboration
• Rethinking the Inception Architecture for Computer Vision — arXiv (Dec. 2, 2015)