Fraud Thoughts (Part II)

What’s New? GenAI Edition

GenAI is a scorching hot topic. Before we dive into the fraud impacts, let’s get grounded in what it is…

---

Below are some real-world examples across these categories…

---

What does this all mean for fraud?

It is simple – GenAI is another instrument in the fraudster tool belt.

GenAI accelerates the effectiveness and sophistication of social engineering – spanning phishing, deep fakes, and more!

Social engineering has traditionally been a successful pathway for bad actors to solicit sensitive information or to convince the victim to complete an urgent act, such as sending money.

With GenAI’s help, these attacks will become even more successful – for example, more sophisticated impersonation schemes, phishing messages, or an enhanced ability to bypass voice or facial recognition.

Let’s go ahead and explore a few examples below…

---

AI-Generated Crypto Invoice Scam

This AI-generated crypto invoice scam almost got me, and I’m a security pro

In this article, Jason Perlow shares his experience of almost falling for an AI-generated phishing email scam that closely resembled an invoice from Stripe, a payment processor often used for cryptocurrency transactions. The language and invoice were so well-written and formatted, Jason states….

I’m used to seeing phishing emails that are far less convincing because they have easily detectable formatting, phrasing, and spelling errors.

In this instance, Gmail didn’t flag the phishing attempt as spam. The invoice and email language were so well written and formatted that it is very likely that AI was used to mimic what one of these invoices from Stripe might look like to evade Gmail’s and human filters. Perlow called the support number in the email, believing it to be PayPal’s, and connected to a busy call center in India that knew enough details about him to sound authentic. He sent codes associated with his emails attached to his Amazon account before he ‘woke up’; he then hung up the phone and reset his passwords.

---

GenAI Fraud-for-Hire

On the dark web, there is a fraud-as-a-service industry run by international cyber gangs from all over the world, including Russia, Nigeria, and China, among dozens of others.

The one depicted in the video is called Mega Darknet Market, one of the world’s biggest enterprises.

“Yes, I sell Chase bank accounts. Yes, I am one of the first people to sell fake bank accounts four years ago,” the man who calls himself “Sanchez” said. “We started with my partner four years ago. Now we are about 30 people in one office.”

This video gave the first glimpse into how these organizations sell “mule accounts,” bank accounts set up with stolen identities, and GenAI and “deepfake” tools to other criminals.

---

Want to dive deeper? Check out this recent article … ‘Hackers Are Weaponizing AI to Improve a Favorite Attack – Phishing attacks are already devastatingly successful. What happens when artificial intelligence makes them even harder to spot?‘

---

How can you protect your business from GenAI-enabled fraud?

GenAI can be compared to other disruptors, such as the COVID-19 pandemic. To prepare for the impact of GenAI, it is crucial to implement a comprehensive anti-fraud strategy that includes an ongoing process to identify emerging risks, like the accelerated threats GenAI poses. This foresight can allow your organization to prepare and implement mitigating actions proactively, both preventive and detective.

In the case of the pandemic, we saw reactive vs. proactive actions or a lack of action entirely. However, proactive steps could have been taken if emerging risks were understood. Similarly, you can proactively prepare for the impact of GenAI by implementing measures now.

Key measures to take include…

Assess Your Risks – Are there areas of vulnerability where AI-enabled fraud could occur across your business? What types of attacks do you see today that will be accelerated with the help of GenAI? Do you have the proper controls to mitigate those risks, and if not, how can you define a path to get there now before a more significant problem arises?

If you don’t have it, now is also an excellent time to implement a process for ongoing monitoring of emerging risks. This is usually a component of a broader fraud risk assessment program – ongoing, ad hoc, and periodic assessment – which feeds into your fraud strategy so the fraud program can adapt swiftly as your threat landscape changes when the next disruption occurs.

Evaluate Your Fraud Tech Stack – Understand your current fraud tech stack and where there may be gaps as GenAI accelerated threats emerge and evolve. It would be best to focus on partners who can adapt as the fraud landscape shifts and those who can integrate into your broader tech ecosystem.

For example, do you use Voice ID (e.g., my voice is my password) to authenticate callers in your call center? How is that partner adapting their technology for enhanced or more sophisticated voice cloning and deep fakes?

Focus on Your Controls – Systematic and operational controls will continue to play an essential role in the fight against fraud – and GenAI-enabled fraud. Ensure you have the appropriate controls across activities with a higher risk or vulnerability to accelerated social engineering attempts or GenAI-enabled fraud.

Update Training – Now is the time to prepare your workforce and customer base for this new threat landscape. Update and roll out further training for your employees and customers that details the accelerated threats GenAI poses and how to keep the business or themselves secure. For example, if misspellings are no longer the tell-tale sign of a phishing email – what other red flags should employees or customers look for?

---

Accelerated fraud threats…and fraud tools?

GenAI may enhance or accelerate the fraud threats of today and tomorrow. However, it also provides a new tool in the fight against fraud; it can help with the efficiency and effectiveness of investigations, analytics, and models – and support prevention and detection efforts.

For example, GenAI models can help generate new programming code with natural language prompts, complete partially written code with suggestions, or even translate code from one programming language to another. This can lead to more effective fraud models, quicker model development for emerging schemes, or more efficient fraud model tuning and management – all of which can support a more effective fraud management program.

Bottom line? As you think about how to protect your business from GenAI-enabled fraud, you should also consider how GenAI can act as a tool to help you more effectively combat fraud now and in the future.

---

How can you protect yourself from GenAI-enabled fraud?

Each of us needs to stay vigilant and protect ourselves and our loved ones – here are a couple of tips to keep in mind:

---

Want to learn more?

Check out Episode 69 of the AFERM Risk Chats podcast – we talked all about #GenAI and the impact on your #fraud risk landscape and broader fraud strategy. This is a federal government-focused podcast, but the advice is industry-agnostic.

Thanks to the Association for Federal Enterprise Risk Management (AFERM), Paul Marshall, CPA, PMP, and Dan Featherly for having me on!

---