SynthID-Text watermarking drifts token selection and can change whether AI agents refuse or call tools

Watermarking sounds like a good idea, but it isn't. Token drift will degrade output quality and could let clever people circumvent guardrails. I'd just assume all text is AI generated and test people with pencil and paper.

Hold on, changing a random seed can improve or degrade the output too. Better and worse outputs should be equally probable, just depends on your luck with the seed.

It's a horrible idea regardless. These companies can embed unique identifiers in content to forever track you and where your content diffuses across the web.

This is getting tiring. Watermarking has zero effect on output quality when implemented correctly, it's like swapping the RNG seed for seed 42 and then detecting that seed from the random sequence.

Model companies are doing this for themselves anyway, so they don't feed generated content back into the slopper and collapse the model. From that angle it slowly contributes to better model quality.

Your statement reads pretty dogmatic next to a decent summary of the watermarking algo. What if seed 42 specifically causes poor quality behaviour in some contexts? Normally that's washed out because the seed is random, and now it's fixed, so shouldn't we check behaviour under this specific seed?

That's not true. Watermarks are messing with the next token generation probabilities based on some random seed. Quality is necessarily lower, the difference is simply too small to notice typically.

The 'when implemented correctly' part is probably what people are complaining about. Opus 5 started adding a bunch of comments to my code even when I told it not to, for trivial changes where the comment was longer than the code change. Was that so there are enough tokens outputted for watermarking?

Benchmarked output quality and actual output quality are very different things. Some use cases sit at the very fringe of model intelligence and depth of logic suffers there.

You're using the subjective definition of quality. The thing you're missing is that the tokens aren't actually random. Tool names, syntax, and prompting style become incredibly deterministic in the areas that matter, like tool calling and parameters. I tune thousands of names and return formats until a call is exactly correct 100% of the time, and what works on Opus 4.7 won't work on 4.8.

The caveat is they have to be a little more sophisticated than just changing the rng on sampling, since they need to detect snippets in the middle of a long chat with thinking removed. I don't know how much impact that part has though.

Over a certain token threshold, roughly 300 to 400 words, yes, there are zero negative effects. At the boundary and below it does affect response quality, so they shouldn't do it. It also incentivizes padding tokens in short responses so they can be watermarked, which is its own quality issue.

Even accepting the seed analogy at face value, fixing a specific seed can change things. Recurrent PRNGs often have degenerate cycles in state space, so a different starting point is just further along the same trajectory. LLMs may not have that failure mode, but recurrence in high dimensional spaces makes me nervous.

Am I missing something, or did they actually completely misunderstand how this technology works?

More likely you misunderstood it than them.

Hard to tell, the writing quality is garbage. 'SynthID-Text changes the process by which the model generates each next token' is a stretch, the refusal claim support is pages later, and 'prompt injection connects these two settings' is a straight non-sequitur. They should have used an LLM for writing help.

The article and most of the comments so far are a dumpster fire.

All these attempts to control language are fundamentally misguided at best and genuinely immoral at worst, same pattern as Newspeak in 1984 or the repressions described in psychoanalysis. Severe unintended consequences either way.