OpenAI, Google, Microsoft Back AI Opt-Out Tag

OpenAI, Google and Microsoft said on Oct. 30 that their AI systems will respect a single robots.txt entry, __usage_ai-training, that lets a website refuse use of its content for AI model training across all of them at once. Anthropic, whose engineer wrote the underlying proposal, has already begun implementing it, The Verge reported.

• One shared token replaces the current practice of blocking each company's training crawler by name.
• Microsoft starts honoring it Feb. 1, 2027; Google targets 2027; OpenAI says "in the coming months."
• The signal is a preference, not a lock: it rests on an individual IETF draft with no formal standing and cannot stop scrapers that ignore it.

How the __usage_ai-training robots.txt token works

The mechanism is defined in an Internet-Draft titled "Machine-Readable Training Preference Signal," filed by Kevin Kelley of Anthropic and dated Oct. 30, 2026. Rather than inventing new syntax, it treats the token as a stand-in for a crawler name in an ordinary robots.txt group.

A site that writes User-Agent: __usage_ai-training followed by Disallow: / expresses a preference against AI training everywhere. An Allow line can carve out exceptions, such as a press-kit folder. If no group names the token, no preference is expressed, and the draft says catch-all User-Agent: * groups do not count.

The token governs how content is used, not whether it is fetched. A crawler may still visit pages for search indexing.

Microsoft, Google and OpenAI timelines

Microsoft gave the most detail. In an Oct. 30 Bing blog post, principal product manager Krishna Madhavan said Bing will begin honoring the new preferences as it crawls from Feb. 1, 2027. After March 1, 2027, Bing will stop reading the older NoArchive and NoCache tags as training or answer signals.

Microsoft also added a separate, page-level setting for "grounding," the use of content to inform live AI answers, with Allow, DisallowExtended and Disallow options. With nothing published, both uses default to allowed. "If no grounding preference is expressed, your content may be used to help inform generated responses," the post says.

Google's John Mueller wrote, as quoted by Search Engine Roundtable: "We aim to support this new control along with other AI providers in 2027, and will provide updates on our progress." Google will treat a token opt-out as a block on Google-Extended unless a site addresses Google-Extended directly. OpenAI said it aims to implement the signal in the coming months.

Why a shared AI training opt-out matters

Until now, publishers had to track a growing list of company-specific agents. A one-line rule lowers that burden, especially for small sites. It also follows Cloudflare's push for crawler accountability, which AI Newsway has covered alongside its effort to rebuild the web for agent traffic.

The limits are significant. The draft states it has no formal standing in the IETF standards process, and it lets private agreements override the signal. Nothing in it removes material already in trained models, and compliance is voluntary.

What's next

The draft expires May 3, 2027, unless revised, and discussion moves to the IETF's AI preferences mailing list. Microsoft opens temporary grounding settings in Bing Webmaster Tools on Dec. 15 for sites still using the old tags. Whether other crawler operators adopt the token is the open question.

FAQ

Does blocking AI training remove my site from search?
No. The token addresses training use only. Microsoft says grounding choices do not affect eligibility for traditional Bing results.

Do I still need Google-Extended or GPTBot rules?
Existing controls keep working. Google says company-specific controls "will continue to function alongside this new joint signal."

Will it delete my content from existing models?
No. It expresses a preference for future use and does not alter models already trained.

References

• Machine-Readable Training Preference Signal (draft-kelley-aipref-training-preference-signal-00) — IETF Datatracker (Oct. 30, 2026)
• Your content, your choice: clearer preferences for training and grounding — Microsoft Bing Blogs (Oct. 30, 2026)
• Supporting a shared AI training control for websites — Google Search Central Blog (October 2026)