Overcoming the challenge of AI-driven disinformation
By Matthew Lowe (pictured), Regional Director – Pacific, Anomali
In a world increasingly filled with digital noise, separating fact from fiction has never been more difficult.
The rise of AI-generated disinformation and deepfakes is not just an ethical or social concern but also a growing business risk, undermining trust in markets, media and even the security of global institutions.
The challenge lies partly in human nature. People rarely recognise how vulnerable they are to influence or manipulation, especially when their emotions are engaged. That makes AI-powered misinformation especially effective as it hijacks instinct, confidence and bias to spread falsehoods at speed and scale.
For business, the implications are profound. Corporate reputations can be destroyed overnight by a convincing fake video and political narratives can be rewritten in minutes. At the same time, fraudsters are no longer confined to crude scams and can now impersonate CEOs, executives or regulators with unnerving precision.
The threat in focus
AI-generated “deepfakes” (fabricated images, video or audio that appear authentic) are growing at an alarming rate. Analysts report[1] a tenfold rise in deepfake-related fraud between 2022 and 2023, while another study[2] found a 3,000% jump in attempted incidents during that same year.
The use cases vary widely. Criminal groups use deepfakes to commit financial crimes, compromise systems or execute social engineering attacks. State-aligned actors deploy them for propaganda and disinformation, seeking to influence elections or undermine trust in institutions. Even individuals have used AI-generated content to intimidate, defame or blackmail.
Human perception remains both our greatest strength and our greatest weakness. The phrase “seeing is believing” reflects a hardwired trust in visual and auditory evidence – the very senses deepfakes manipulate. When a familiar face appears in a video or a trusted voice delivers a message, our brains tend to accept it as real, often before critical reasoning has a chance to intervene.
Adding to the problem is misplaced confidence. Studies show that most people believe they are good at spotting manipulation, but their actual detection rates are far lower. This overconfidence leaves users exposed, especially when disinformation triggers emotions such as anger, fear, outrage or excitement. These emotional surges cloud judgement and override logic, which is precisely what threat actors exploit.
The political sphere demonstrates the danger vividly. Falsified clips of public figures can sway opinion or damage reputations even after being debunked. Researchers have found that deepfake videos can shape perceptions almost as strongly as genuine footage, even among viewers who are aware that deepfakes exist.
At the same time, the growing prevalence of synthetic media fuels what experts call the “liar’s dividend” which is a new kind of plausible deniability. As fabricated content becomes more common, genuine evidence can be dismissed as fake and, for powerful figures, that uncertainty is an asset.
Defending against the threat
Technology offers partial solutions as AI detection tools, reverse image searches and forensic software can help identify manipulated media. However access to these tools is uneven, and their accuracy is inconsistent. For most people and organisations, the first line of defence is still awareness.
Critical habits, developed consciously and practised consistently, are essential. The process begins with verification by cross-checking dramatic or suspicious content against credible, independent sources. Genuine news rarely appears in isolation and so, if a claim exists only on fringe platforms, it’s likely unreliable.
The next step is contextual scrutiny. Who is sharing the content? Does the account have a history of credible activity, or is it newly created and anonymous? Does the scenario in the video seem plausible, or are there inconsistencies that don’t align with what’s known about the subject? A few seconds of background checking can expose even sophisticated fabrications.
Visual analysis can also help. Despite improvements in AI, many generated images still contain subtle errors such as distorted fingers, irregular jewellery, warped text or unnatural lighting.
The same principles apply to audio. Voice-cloning software can replicate tone and rhythm with uncanny accuracy, but there are often giveaways such as uneven pacing, robotic emphasis, or shifts in tone that don’t match a speaker’s normal style.
Even text can be deceptive as generative AI models can produce polished, plausible-sounding posts or articles that conceal false claims or fabricated sources. Signs of synthetic writing include unnatural uniformity, abrupt shifts in tone, and overuse of generalities. Fast, fluent responses from newly created accounts can also indicate AI involvement.
A human solution
Ultimately, deepfake defence is not purely a technological problem but a behavioural one. The same instincts that make humans vulnerable to manipulation can also be retrained. The key is to slow down. Verification, context-checking and deliberate analysis interrupt the emotional triggers that disinformation relies upon.
As AI-generated content grows more convincing, it becomes less about whether a viewer can tell the difference, and more about whether they pause to question it. For individuals, that means practising scepticism without cynicism. For organisations, it means embedding digital literacy into risk management frameworks and undertaking regular employee training.
[1] https://www.cyber.gov.au/about-us/view-all-content/reports-and-statistics/annual-cyber-threat-report-2023-2024
[2] Australian Cyber Security Centre, Annual Cyber Threat Report 2023–2024
