As of August 2, the transparency obligations of Article 50 of the EU’s Artificial Intelligence Act have come into effect. Providers of AI systems that generate text, images, audio, and video must ensure that outputs carry machine-readable markings and can be identified as human-generated or human-modified; the publication of deepfake content, as well as AI text involving the public interest that has not been reviewed and edited under human responsibility, must also be disclosed to the public. The transparency code of conduct drafted by the EU is voluntary to participate in, but the Article 50 requirement itself is legally binding.
With the new rules now in force, Anthropic announced that it would add watermarks difficult for the human eye to discern but detectable by machines to the text generated by Claude, and planned to apply this mechanism to a range of products including API and Claude Code. Images can record provenance through file metadata, but plain text makes it hard to preserve a marker independent of the body of the text over the long term; companies thus have no choice but to consider embedding statistical signals directly into the text-generation process.
The independent scholar Hu Yilin believes that the problem is not merely whether such a watermark can be stripped away by another model, nor even whether it will lower average text quality. The more fundamental dispute is this: does the government have the right to require AI to insert, inside the user’s text, a hidden function that serves not expression, but future regulation and identification?
The watermark is not pasted beside the text; it participates in deciding the text itself
Plain-text watermarks are usually not just a few zero-width characters sneaked in. Take SynthID-Text, developed by Google DeepMind, for example: as the system generates tokens one by one, it alters the sampling method, so that the entire passage exhibits a set of statistical regularities recognizable to those who hold the detection key.
DeepMind’s study published in Nature says that this scheme can be configured in a “non-distorting” mode; across nearly 20 million real Gemini responses, text with watermarks and text without them showed no significant difference in user feedback or standard capability tests, and the additional latency was very small. At the same time, the paper also acknowledges that stronger detectability may involve trade-offs with quality, diversity, and computational complexity, and that its approach will have some effect on the diversity among different responses.
In Hu Yilin’s view, the fact that the average user does not notice a decline in quality does not prove that a specific piece of writing has not changed at all in its expression.
He stresses, “I do not believe that generating watermarks by adjusting the text is fully lossless to expression, because human language, in some fundamental sense, has no ‘synonyms.’” Two words may be close in meaning in the dictionary, yet still carry different tones, rhythms, historical associations, and positive or negative valences; for ordinary question-and-answer use, these differences may be negligible, but for fiction, poetry, philosophical argument, and stylized writing, which word is chosen is itself part of the work.
Strictly speaking, a statistical watermark is not necessarily a second-round modification made to an already finished “clean draft.” The model generates the text from the outset under watermarked sampling rules, and there is no single, definite unwatermarked version to compare against. But this does not affect Hu Yilin’s core criticism: the model was originally supposed to serve the user’s expressive aims as much as possible, yet it is now being forced to take on a second objective—so that a third party can identify the text in the future.
More transparent to machines, yet more opaque to the author
In the name of transparency, the EU’s new rules require generated content to remain recognizable to detection systems. But Hu Yilin believes that this transparency belongs first and foremost to regulators, platforms, and model providers—not to the author themselves.
The author can see every final character, yet does not know which wording was chosen because it served expression, and which wording was chosen to preserve a machine-detectable statistical signal. The watermark is not “invisible”; rather, humans can see the textual result it produces, yet do not realize that another set of reasons for selection is at work within it.
He compares this to the relationship between writer and editor:
Imagine a writer wants to publish his own novel and hands it over to an editor for revision—of course he would want every change the editor makes to be transparent, something he can approve line by line. And he would want the editor’s changes to be made for beneficial reasons such as correcting errors or polishing the language, while not undermining the author’s style. If an editor makes certain changes not to improve the manuscript, but for purposes that do the author no good, and the author cannot even tell exactly what was changed, that is of course abhorrent. Even in the case where AI is not acting as an editor or polisher but as a collaborator, the human as first author or corresponding author would still hope the collaborator could be as open and candid as possible.
This analogy shifts the dispute from “has text quality declined?” to “who has decision-making authority over the text?” Even if the watermark has almost no effect on most people’s reading experience, it still writes into the work a institutional purpose that the author cannot review and cannot reject item by item.
Hu Yilin thus argues that “this watermark policy is nominally for so-called transparency, but it itself creates opacity.” It makes works easier for machines to classify, yet makes the process by which machines generate works harder for authors to understand.
Private companies may choose watermarks; state compulsion requires a much higher threshold
Hu Yilin does not advocate that the law forbid private AI companies from using text watermarks. He even believes that, so long as a company has not clearly promised “no watermarks,” it may in principle decide for itself how to generate output; even if it has not proactively informed every user, this is first and foremost a matter of market choice between enterprises and consumers.
This does not contradict his opposition to the EU’s mandatory rule. In his framework, private firms may set product rules that are imperfect, or even annoying; users may refuse to use them, turn to competitors, or criticize them publicly. Public power, by contrast, possesses punitive, access-control, and general coercive capacity, and must bear a much higher burden of justification.
“Public power should be cautious and restrained; if there are not sufficiently just reasons, it should not interfere in the market, but free participants in the market are not held to such a high standard of justice,” he says.
Therefore, the key issue is not whether watermark technology itself is evil, but who is making the decision. A company voluntarily offering a watermarked model is a market solution; a government requiring all major models entering the market to implant machine signals in users’ text elevates a commercial choice into a universal system.
If only the government and a few companies control the detection algorithm, people accused of using AI will find it very difficult to independently verify the evidence of detection; if detection tools are made broadly available to schools, employers, publishers, and the general public, then AI-assisted text may thereby acquire a permanently scannable special identity. Regulators can declare that the markings are merely neutral provenance information, but real society may not use them in a neutral way.
What watermarks actually achieve: what they most easily constrain are precisely those least inclined to deceive
Text watermarks are not entirely without technical effect. When model responses are directly copied or only lightly edited, the statistical signals may persist. They can help platforms identify large-scale automated posting, and are also more resilient than ordinary file metadata, which tends to disappear when copied.
But research on existing watermarks has also repeatedly shown that model paraphrasing, expansion, compression, back-translation, and targeted word replacement can all significantly weaken detection. A 2026 ACL study has already been able, in a black-box manner, to locate and perturb tokens suspected of carrying a watermark while preserving semantic similarity and linguistic naturalness. Watermark research therefore always has to juggle, again and again, between text quality, detection accuracy, and resistance to adversarial rewriting.
This is exactly what Hu Yilin means by “it can keep honest people in line, but not the villains.”
Ordinary users who directly use Claude to polish a draft, without deliberately hiding its origin, will preserve the watermark intact; those who truly want to disguise AI text as a human work can instead call up another local model to rewrite it paragraph by paragraph, translate it across languages, or reorganize it from scratch. The gentler the watermark, the easier it is to wash away; the more it tries to resist thorough rewriting, the more deeply it may constrain concrete expression.
In his view, this makes mandatory watermarking fail on both justice and efficacy: those who comply bear the costs of hidden markings, expressive interference, and possible discrimination, while those who intend to deceive can still get around it.
Such a system also cannot solve genuinely dangerous content. Whether a set of instructions for making poison is dangerous depends on what knowledge it provides and how it is used, not on whether it came from a human or from AI. A harmful guide written by a person and the same kind of guide generated by a model should be subject to the same content and conduct rules. Proving that “it may have come from a certain model” proves neither who the author is nor whether the text is illegal or harmful.
The openness of Chinese AI does not mean the Chinese government is freer
In an earlier discussion, Hu Yilin used “Chinese AI tends toward openness and freedom” to contrast with the growing emphasis on censorship, compliance, and tracing in AI in Europe and the United States. On this point, he especially stresses that he is talking about the AI industry and the technical route of models, not the overall style of governance of the Chinese government.
“What I am saying is that Chinese AI leans more toward openness and freedom, not that the Chinese government leans more toward openness and freedom,” he says.
The key to this openness is not merely that companies publish technical reports, but that users can obtain the model weights and independently deploy them on local servers, personal devices, or in other jurisdictions. Qwen once released multiple Qwen3 models of different sizes at once, under the Apache 2.0 license; DeepSeek also made it clear that it publicly releases model weights and inference code, enabling users to download and deploy them on their own.
Local models do not guarantee that users will be virtuous, nor do they mean the country of their development lacks content regulation. But they provide a technical “right of exit”: users need not route every prompt, generation, and rewrite through the original model company’s servers, nor permanently accept watermarks, censorship rules, prices, and regional restrictions later added by the original service provider.
For that reason, the EU’s mandatory watermarking may create a reverse incentive: closed services that are easiest to regulate become ever more traceable, while users who wish to protect anonymous writing, creative autonomy, or avoidance of markings will be more motivated to migrate to open models that cannot be continuously controlled by the original vendor.
China’s identification system is taking a different technical path
China’s Measures for Labeling Artificial Intelligence-Generated Synthetic Content, implemented in September 2025, likewise require generated content to be labeled, but the mandatory path for pure text is not the same as the EU’s.
The Chinese rules require noticeable prompts to be added at the beginning, end, or in the middle of text, or around the user interface and text; mandatory implicit labeling is mainly written into the metadata of downloadable files, while digital watermarks inside the content are only encouraged. In its official Q&A, the Cyberspace Administration of China explicitly explained that because there are still technical difficulties in adding implicit labels to textual content, and doing so might also increase enterprise costs, it has not made this a mandatory requirement.
This at least prevents the Chinese system from having the government uniformly require models to rewrite pure text through covert word choice. But it still requires users to make an active declaration when they publicly disseminate generated content on online platforms, and Hu Yilin likewise believes this scope is too broad. He argues that when a user is merely passing along model output unchanged, the platform may require an honest disclosure; when the user has already sufficiently reviewed and substantially edited the text and is willing to bear full responsibility for it, it should no longer be classified merely as a production tool’s work.
This continues his long-standing judgment on AI creation: the law should look for the person who can bear responsibility, rather than establishing a permanent identity for a tool.
Machine credentials cannot replace human integrity
In publishing, education, competitions, and academic research, there are indeed scenarios that require explanation of the degree to which AI was involved. Platforms and publication venues can also write disclosure requirements into contracts: if a competition explicitly forbids generative AI and a contestant conceals its use, that is first and foremost a breach of contract; if a student submits an assignment meant to test their own writing ability, they too should honestly disclose it in accordance with the exam rules.
But Hu Yilin opposes the government preinstalling a uniform machine proof for all text. He believes that even if one uses independent cryptographic signed receipts capable of proving that a certain piece of content was once generated by a certain model, that still cannot prove how much humans ultimately contributed to the final work. An attempt to fake it could simply have another AI play the role of a human, debate back and forth with the first model, and then leave behind a complete record that appears to contain extensive “human participation.”
Machines can preserve the process of interaction, but they cannot judge for society where creativity truly came from, much less bear responsibility for a work on behalf of a person.
“These kinds of needs still have to respect human integrity; there’s no need for machines to provide any extra guarantee,” he said.
This does not mean believing that everyone will never lie, but rather holding that a civilized society should not, because some deceivers exist, first turn every ordinary person’s work into an object for surveillance machines to scan. People can make declarations, institutions can verify specific disputes, and fraud can be pursued after the fact; but administrative convenience by itself is not enough to prove the legitimacy of universal tracking.
What, exactly, does transparency let whom see?
The starting point of the EU is not hard to understand. Generative AI can produce large amounts of content at low cost, and machine-readable markers can help platforms identify synthetic information, while also potentially reducing the risks of misidentification and impersonation. The EU’s official explanation of Article 50 is precisely that it is intended to prevent deception and manipulation and to maintain the integrity of the information ecosystem.
Hu Yilin’s objection is not that he denies the need for all transparency. He accepts that enterprises may choose watermarks on their own, accepts that publishers may require authors to make declarations, accepts that competitions and contracts may set rules according to the method of production, and also supports holding people accountable for forged scenes, identity theft, and specific fraud under the same standards.
His core objection lies in the direction of transparency.
A system may require authors to be honest with readers, and it may require platforms to disclose their own generation and detection methods; but when, in order to make text transparent to machines, it instead allows machines to alter the text in ways the author cannot discern and cannot review item by item, then “transparency” itself needs to be subjected to scrutiny.
The dispute over text watermarking is ultimately not just about AI. It raises once again an old liberal question: should society first treat people as subjects capable of making declarations, bearing responsibility, and answering questions, or should it first treat them as possible liars, and therefore as objects for which machines must leave evidence in advance?
The former system inevitably tolerates some deception; the latter may, in order to manage deception, turn everyone’s work into an ID card waiting to be scanned.
Translated from the Chinese original with AI assistance. The original text is authoritative.
Leave a Reply