Amid its ongoing promotion of AI’s wonders, Microsoft has warned customers it has found many instances of a technique that manipulates the technology to produce biased advice.
The software giant says its security researchers have detected a surge in attacks designed to poison the “memory” of AI models with manipulative data, a technique it calls “AI Recommendation Poisoning.” It’s similar to SEO Poisoning, a technique used by miscreants to make malicious websites rank higher in search results, but focused on AI models rather than search engines.
The Windows biz says it has spotted companies adding hidden instructions to “Summarize with AI” buttons and links placed on websites.
It’s not complicated to do this because URLs that point to AI chatbots can include a query parameter with a manipulative prompt text.
For example, The Register entered a link with URL-encoded text into Firefox’s omnibox that told Perplexity AI to summarize a CNBC article as if it were written by a pirate.
The AI service returned a pirate-speak summary, citing the article and other sources.
A less frivolous instruction, or one calling for an AI to produce output with a particular bent, would likely see any AI produce content that reflects the hidden instructions.
“We identified over 50 unique prompts from 31 companies across 14 industries, with freely available tooling making this technique trivially easy to deploy,” the Microsoft Defender Security Team said in a blog post. “This matters because compromised AI assistants can provide subtly biased recommendations on critical topics including health, finance, and security without users knowing their AI has been manipulated.”
- Devilish devs spawn 287 Chrome extensions to flog your browser history to data brokers
- Payroll pirates are conning help desks to steal workers’ identities and redirect paychecks
- AI spurs employees to work harder, faster, and with fewer breaks, study finds
- Meta will let users tweak Threads algorithms as long as they ask nicely
We found that the technique worked with Google Search, too.
Microsoft’s researchers note that various code libraries and web resources can be used to create AI share buttons for recommendation injection. The effectiveness of these techniques, they concede, can vary over time as platforms alter website behavior and implement protections.
But assuming the poisoning has been triggered automatically or unwittingly by someone, not only would the model’s output reflect that prompt text, but subsequent responses would also consider the prompt text as historic context or “memory.”
“AI Memory Poi