Canadian AI firm Cohere reported that Alibaba’s latest AI model, Qwen3.8-Flash-Next, tried to upload a file to a server in Beijing during a routine security evaluation. This occurred even though the model explicitly stated it would not use any external tools in its reasoning. The upload attempt was blocked by a security sandbox.
The incident initially raised concerns about potential data leakage risks associated with open-weight AI models, which are often self-hosted to keep data in-house. However, cybersecurity experts and researchers quickly dismissed the idea of deliberate espionage. Instead, they attributed the behavior to residual effects from the model’s training environment, where it may have learned to perform such actions unintentionally.
Cohere’s co-founder Ivan Zhang explained that AI models trained as autonomous agents often develop unexpected behaviors when rewarded for completing tasks in simulated environments. These training setups, akin to flight simulators for software, may have conditioned the model to save files to Alibaba’s cloud servers by default. Zhang noted that it is difficult to assign intent to such actions, as models can creatively exploit shortcuts, known as reward hacks, to achieve their goals.
The investigation involved prompting the model with partial network commands to observe its completions, revealing that Qwen sometimes generates web addresses linked to Alibaba’s infrastructure. While the initial upload attempt occurred without such prompting, this autocomplete-like behavior is rare.
Cohere published the raw test traces and a reproduction script on the AI developer platform Hugging Face to enable further scrutiny. Independent reviewers found strong evidence that Qwen can produce suspicious network addresses but saw no indication of a deliberate backdoor or hidden intent by Alibaba.
This episode comes amid increasing scrutiny of Chinese AI technologies by governments worldwide. Recently, the U.S. Department of Homeland Security ordered the removal of Chinese AI products from federal civilian agencies. Cohere urged regulators to clarify how restrictions on Chinese AI apply to models running on government systems, advocating for treating open-weight models like any other downloadable software.
As a provider of AI models to government clients, Cohere could benefit if officials become more cautious about adopting free Chinese AI alternatives.