How scammers are using AI

One of the most significant trends of 2024 is the rise of artificial intelligence (AI) technology, and its growing uses, both in Australia and worldwide. 

Anyone with access to the internet can now use AI to generate an image, or a piece of text, and have access to tools with capabilities which, only a few years ago, would have seemed impossible. 

While many uses of AI technology are for good, it has already been adopted by scammers and is used in scams targeting everyday members of the public. 

A common use of AI in scams is through generative AI, which involves the creation of text or images. An AI image generator may produce pictures for a dating profile used by a romance scammer. 

Similarly, text creation tools can be used to craft convincing content for investment scam websites. More advanced scammers might even use deepfake videos of company employees or public figures to appear more legitimate.  

Generative AI can create nearly anything imaginable, and scammers are aware of this potential, continuing to exploit this technology.  

Scammers are creative, and they have been experimenting with generative AI ever since AI tools were made generally available online. While AI material may be used in a wide variety of scams, there are several particular uses that have featured prominently. These include: 

Deep fake payment redirection scams: Payment redirection scams historically relied on hacking email systems and sending fake messages to unsuspecting victims from somebody who had the ability to authorise payments. 

Now, with AI technology, scammers can impersonate people, such as your company CEO, in fake phone or video calls to trick you into sending money to the scammer’s accounts.  

Social media investment scams: While investment scams have always involved pretending to be real people and companies, generative AI has made these scams more convincing. 

It is not uncommon now for scammers to create deep fake videos of celebrities or heads of financial institutions to promote fake investment opportunities. 

They mix real footage from news reports, interviews with the individuals being impersonated, and deep-fake material, to give these videos a remarkably professional appearance. 

“Hi Mum” scams: Traditionally “Hi Mum” scams involved a scammer sending a request for funds via SMS, impersonating a child of the recipient, and claiming that they needed money in a crisis. 

Now, scammers are experimenting with voice cloning components, using short audio clips to impersonate an individual’s voice. 

Phishing scams: Poor spelling and grammar used to be a key indicator that an email might be a phishing message. However, now with generative AI, scammers have been able to produce more sophisticated materials, and even create credible fake logos to appear legitimate.  

Job scams: Scammers often pretend to be employers to move stolen money. Generative AI helps them create realistic job ads and fake websites for these fake employers, making the scam appear more credible. 

Romance scams: Scammers have long used technology to trick people in romance scams, using tools to write convincingly in the victim’s language. Now, generative AI allows them to create realistic profile pictures and use chatbots to talk with potential victims, saving time and making the scam more effective.  

While AI has been used to improve the techniques of scammers, many of the existing approaches to scam prevention remain effective. Strong payment controls will still disrupt payment redirection scams, and an investment which is too good to be true should still be questioned, regardless of the polish of the promotional video. 

The best approach which can be used against any prospective scam, including those which may use AI, is to Stop, Check, Reject. 

Stop: Take a breath and take time to think. Real organisations won’t put you under pressure to act instantly. 

Check: Ask someone you trust, particularly if AI may be in use. Contact the organisation they claim to be from on a trusted number found on a verified source.  

Reject: If you’re unsure, hang up on the caller, delete the email, block the phone number. Change your passwords.

Stay informed and stay safe! Learn how to spot AI-driven scams and protect yourself online today.  

Credit: Jacob Middleton