Credit: Sam on Unsplash
OpenAI will add an invisible watermark to text from ChatGPT and Codex in the EU over the coming weeks. The change covers eligible users on all plans, but only in the EU, the company said in a blog post on Monday. OpenAI said it is acting in response to the EU AI Act, which requires AI-generated text to be identifiable by machines.
OpenAI is not making text watermarking a global default at launch. The regional approach gives it room to learn from real-world use and feedback, it said. API customers anywhere can opt in for select models from today. Watermarking stays off by default in the API. OpenAI said it is also working with cloud partners to offer it on its models through their services.
How textGrain works
The system, called textGrain, adds a hidden statistical signal to the model’s word choices. A detector then looks for that signal. OpenAI said textGrain matched or beat other approaches it tested, including Google’s SynthID for text. It published a technical report written with researchers from the University of Pennsylvania and Yale.
Benchmark scores for its Astra model showed no meaningful difference with the watermark on, OpenAI said. It also plans to release the technology as open source.
Where it fails
OpenAI also set out the limits. At a target false positive rate of 1%, its detector found the watermark in about 80% of 200-token passages on topics such as psychology. For 400-token passages, the rate was about 95%. Replacing 10% of words with synonyms cut detection from about 92% to 66%. Replacing a quarter of the words cut it to 17%. Detection was substantially lower for maths, where word choice is less flexible, the company said.
The watermark does not identify the user, measure human contribution, establish ownership or verify accuracy, OpenAI said. A missing watermark does not prove a human wrote the text either. The text could be too short, edited or translated, or come from another company’s tools.
“These limitations contribute to our decision to provide initial detector access only to approved researchers and expert organizations, who can help us evaluate reliability and responsible uses,” OpenAI wrote.
Applications open today, and access will be granted case by case, in line with the EU’s Code of Practice, OpenAI said. The detector reports whether it finds an OpenAI watermark without identifying the user or revealing prompts. OpenAI said it will widen access once results can be interpreted responsibly. Its tools for checking images and audio remain public.
The EU deadline
Article 50 of the AI Act requires generative AI providers to make their text output machine-readable. Providers already on the market have until 2 December to comply, Engadget reported.
Anthropic began watermarking Claude’s text worldwide in August, with some exemptions. Tools to strip AI watermarks have since appeared. Google DeepMind has also watermarked AI-designed proteins with SynthID.
Get the TNW newsletter
Get the most important tech news in your inbox each week.
