According to The Verge, the signal, a shared "policy token", was proposed at the IETF by Kevin Kelley, a standards technical lead at Anthropic, and it lives in a site's robots.txt file. Today sites have to signal their training preferences to each company separately. The companies said their crawlers will not access a site where they find the token.
OpenAI said in its announcement that it will honor the signal to help website operators communicate their training preferences to participating providers. Microsoft said it will offer more granular controls, letting sites separately opt out of training or of grounding, and that sites will still appear in Bing results either way.
Meta spokesperson Andy Stone expressed support for the idea but did not say whether the company will honor the signal. The announcement follows earlier efforts to limit AI crawlers, such as the RSL standard, which lets publishers charge crawlers, and Cloudflare blocking AI crawlers by default.