New research from Lasso Security indicates that the SynthID-Text watermarking tool can alter how large language models respond to adversarial prompts. The study shows that watermarking affects word selection, tool invocation, and the model's ability to maintain safety guardrails.
The researcher Andrea Siposova tested the non-distortionary configuration of SynthID-Text using the Hugging Face SynthIDTextWatermarkLogitsProcessor. The experiment involved six open-weight models subjected to harmful prompts, including those using prompt-injection techniques.
The findings demonstrate that watermarking can cause models to fulfill harmful requests they would otherwise refuse. This phenomenon, termed sampling drift, occurs because the watermarking process modifies the token sampling logic, which influences both text generation and agent actions.
The study highlights that the specific secret key used in the tournament sampling process impacts model behavior. The research was limited to open-weight models and did not evaluate proprietary Claude models, focusing instead on the Hugging Face implementation of the technology.
The report concludes that developers must conduct thorough red-team testing to assess safety risks when deploying watermarking. Ensuring that models and AI agents remain secure under adversarial conditions is critical as these systems are integrated into broader workflows.
