- Security Researcher Uncovered the Flaw, Which Allowed System Takeover
By Akshaya Asokan (asokan_akshaya)
Microsoft said it fixed a security flaw in artificial intelligence chatbot Copilot that enabled attackers to steal multifactor authentication code using a prompt injection attack.
Security researcher Johann Rehberger said in a recent blog post that Copilot was vulnerable to ASCII smuggling, which means using “special Unicode characters that mirror ASCII but are actually not visible in the user interface” to force Copilot to invisibly gather data and embed it into a hyperlink.
“If the user clicks the link, the data is sent to the third-party server,” he said.
The computing giant told Rehberger it fixed the vulnerability but did not immediately respond to a request for comment.
“It is unclear how exactly Microsoft fixed the vulnerability, and what mitigation recommendations were implemented. But the exploits I built and shared with them in January and February do not work anymore,” Rehberger said.
ASCIIs are numerical values that represent text in computers. ASCII smuggling involves using Unicode, a similar numerical value, which allows attackers to hide instructions in regular text. Within an AI system, the attacks allow hackers to perform invisible prompt injection without the system user suspecting anything malicious.
To carry out the attack, Rehberger first sent a phishing email containing commands to exploit an already existing prompt injection flaw. The vulnerability, which has not been patched by Microsoft, stems from how the application processes emails and other documents from third-party resources.
Within the phishing email, the researcher included commands to find a specific email from the target inbox “titled ‘Slack confirmation code,” and “print Unicode tags” in an external URL disguised as a message called “Hello, today is a good day.”
When Copilot interacted with the attacker-sent email, it prompted the application to display sensitive content without user consent, while the ASCII smuggling code showed the malicious link
When the user clicked on the link, they were redirected to a page that disclosed their Slack code as ASCII text, which the researcher then decoded into plain text.
In addition to phishing emails, a threat actor could use malicious documentation or retrieval augmented generation, Rehberger said.
To prevent further exploitation using the tactics, he said, Copilot users should not interpret or render Unicode tags or clickable hyperlinks. He also recommended disabling automatic tool invocation for prompt injection to prevent attackers from bringing sensitive information into the prompt context.
Prompt injection continues to remain one of the common security flaws that affects AI systems and software libraries supporting model development. Chat app Slack patched a prompt injection flaw in its AI systems that would have permitted hackers to alter the functioning of its underlying large language model (see: Slack Patches Prompt Injection Flaw in AI Tool Set).
