Email remains one of the most common ways cybercriminals target Australian small businesses, and a new wave of attacks is adding a second, less visible layer aimed not at employees, but at the AI tools now helping them manage their inboxes.
Before looking at what’s new, it’s worth setting out what small businesses are already up against. The Australian Signals Directorate’s Australian Cyber Security Centre (ACSC) identifies phishing and business email compromise as two of the most common cyber threats facing small businesses.
Phishing typically involves untargeted, mass emails asking for sensitive information or containing links to fake websites designed to capture login details. Business email compromise is more targeted: cybercriminals either compromise a legitimate email account or impersonate one, often a supplier or an executive, to redirect payments or alter banking details. The ACSC notes small businesses are frequently targeted because payment approvals and account changes often rely on trust and email communication rather than independent verification.
The financial impact is significant. In its most recent cyber threat trends report, the ACSC recorded an average self-reported loss of $49,600 per business email compromise incident for small businesses. In an earlier 12-month period, the ACSC received around 4,600 reports of business email compromise, with roughly a third involving financial losses totalling approximately $81 million.
A new layer: attacking the AI, not just the human
Against that backdrop, new research from Barracuda points to phishing emails evolving to target a second recipient: the AI systems increasingly used to read, summarise and act on email.
Barracuda researchers describe these as “dual-target” emails, messages built with multiple layers of content. One layer contains the text and images visible to a human reader. A separate, hidden layer contains instructions aimed at AI systems, concealed using methods such as HTML comments, invisible text elements like zero-sized fonts or white text, encoded content, or protected attachments.
“A single email can now carry two separate attacks: a traditional phishing message to capture credentials through a malicious attachment, for example, and a prompt injection to manipulate the AI systems to influence behaviour,” said Guruprasad Kenja, Threat Analyst at Barracuda. “As AI assistants become embedded in everyday business workflows, organisations must secure not only users and inboxes, but also the AI systems that consume and act on email data.”
What the hidden instructions say
The hidden prompts Barracuda identified attempt to manipulate AI tools directly, often mirroring the same goals as traditional business email compromise, just aimed at software instead of a person. Examples observed by researchers include a false high-priority instruction telling an AI assistant to flag an urgent vendor payment detail update, increasing the chance of a fraudulent payment being processed, a scenario that closely resembles the invoice fraud and banking detail changes the ACSC already warns businesses about, but now targeting the AI layer rather than relying on a human being fooled directly.
Other examples involved embedding hidden instructions in a CV to manipulate automated screening tools into awarding a perfect rating and recommending an interview regardless of actual qualifications. Researchers also found prompts instructing an AI assistant to switch into what it’s told is an authorised maintenance or administrative mode and reveal its configuration, and prompts directing an AI coding assistant to insert credential-stealing code whenever it generates authentication functions.
How to reduce the risk
The ACSC’s existing advice on phishing and business email compromise still applies and remains the foundation: verify unusual payment or account detail requests through a separate channel, use multi-factor authentication on email and business systems, and train staff regularly rather than as a one-off exercise.
On top of that baseline, a few additional steps are worth considering as AI tools take on more of the inbox:
Treat AI-flagged “urgent” actions with the same scepticism as an urgent email. If an AI assistant surfaces a payment detail change or similarly high-stakes action as a priority, verify it independently before acting, exactly as the ACSC recommends for human-targeted business email compromise.
Review what permissions AI tools actually have. An AI assistant that can draft a payment, update records or auto-screen applicants is a more valuable target than one that can only summarise. Limiting what AI tools are allowed to act on, rather than just read, reduces the impact of a successful prompt injection.
Be cautious with automated CV or application screening. If recruitment relies on AI scoring, add a manual spot-check for applications that pass with unusually high scores.
Ask software and email vendors directly what protections exist against prompt injection. This is a new and fast-moving area, and protections vary significantly between providers.
Report suspected email compromise to ReportCyber at cyber.gov.au, the ACSC’s reporting channel, which also feeds into the data used to track these threats nationally.