#pretraining-data

After FineWeb filtering, 31.1% of August 2026 web tokens were AI-written. An 800-model study shows their value flips sign — and Chinchilla can't model it.