Phishing Tests Are Getting Downright Mean
11:55 Share

This episode is brought to you by Shopify. Forget the frustration of picking commerce platforms when you switch your business to Shopify, the global commerce platform that supercharges your selling wherever you sell. With Shopify, you'll harness the same intuitive features, trusted apps, and powerful analytics used by the world's leading brands. Sign up today for your $1 per month trial period at shopify.com slash tech, all lowercase. That's shopify.com slash tech.

Welcome to Tech News Briefing. It's Monday, February 10th. I'm Pierre Bien-Aimé for The Wall Street Journal. How useful can artificial intelligence really be if it sometimes makes stuff up? Amazon is turning to so-called automated reasoning to cut down on hallucinations. And IT departments routinely try to fool their employees in order to get them to recognize hackers' phishing attempts. And some say they've gone a little too far.

Hackers who engage in phishing, sending deceptive emails aimed at stealing sensitive information, are cooking up some increasingly sophisticated scams. As a result, IT departments at companies and universities are throwing sensational tests at their employees and students. The idea is, if you opened this email and clicked on the link, you've failed the test. And failure comes at a cost.

Phishing, spelled with a PH, was the first step in about 14% of cyberattacks last year. That's according to an analysis of data breaches done by Verizon. Bob McMillan writes about computer security for The Wall Street Journal, and he reported on the value of these test traps. So Bob, how does phishing typically work? Well, they try to...

play in your mind. They try to get you in some kind of panic mode. So usually what happens with these phishing emails is there's some very, very important piece of information they promise, like your vacation days are being cut. And you're like, what? My vacation days are being cut? You click on the link, then you have to log in.

You don't even realize you're not on your corporate website. You're on some fake website the hackers set up. So you're giving them information right there. What are some of the examples of these kind of phishing traps set up by IT departments to lure people and maybe trick them into falling for it?

There was one email, it was about a lost puppy dog in a parking lot. There was a guy who sent an email to NASA staffers promising, you know, a chance to win a ticket to see the final space shuttle launch. And he apparently made a staffer there cry when she realized it was a fake. The craziest example that I heard of was the University of California, Santa Cruz, which last summer sent a phishing email test to

Ebola outbreak on campus. And it basically sent some people into a panic there, thinking that there was a case of Ebola on campus. There wasn't. They were just trying to do a phishing education test

What are IT departments learning as far as the most effective way to spread awareness and boost resilience against phishing, which is the whole idea? I did interview a guy from Google who was talking about some other approaches to curbing this. Like education is not a bad thing. It's the idea of embarrassing people and putting them in an adversarial position and then going like, now listen to me. There are other ways of doing education, having phishing awareness months and fun conversations

less shameful kinds of ways of teaching people to report phishing emails and to spot them. There's some research from the University of California in San Diego that basically...

looked at a variety of phishing email tests and then educational responses to them. And they found that basically the sort of classical approach to doing this yields negligible results. At best, they found sort of a 2% improvement in the likelihood of the targets to avoid phishing emails in the future.

Now, the phishing attempts of yore look pretty ridiculous. Things like free prizes, I love you in all caps, or, you know, the notorious email from a Nigerian prince. But what do really advanced phishing attempts look like now?

The hackers are getting very clever. They know how corporations work and they know which kind of emails are high priority. They know like an email from the CEO demanding some kind of immediate response about corporate facts or what's going on with this pitch or something like that. They know those kinds of emails are very successful.

At the open enrollment time of year, you know, when you're re-upping your medical, they know that an email with that theme that's sent around November, you know, gets a very high response rate. The problem is these phishing attacks, they lead to ransomware. They lead to like catastrophic consequences for some corporations, for some hospitals. And there's a sense of urgency around stopping them from working. That was WSJ reporter Bob McMillan.

Coming up, AI bots occasionally say the darndest things, giving flat-out wrong answers. We hear about the obscure field of research that could help solve that problem. That's after the break.

This episode is brought to you by Indeed. When your computer breaks, you don't wait for it to magically start working again. You fix the problem. So why wait to hire the people your company desperately needs? Use Indeed's sponsored jobs to hire top talent fast. And even better, you only pay for results. There's no need to wait. Speed up your hiring with a $75 sponsored job credit at indeed.com slash podcast. Terms and conditions apply.

Artificial intelligence is known to sometimes make up answers and to share these so-called hallucinations with confidence. Now, Amazon's cloud computing unit, Amazon Web Services, is looking to automated reasoning for hard mathematical proof that these errors can be stopped, at least in certain areas. Some analysts say that success could mean millions of dollars worth of AI deals with businesses. Belle Lin writes about AI and enterprise technology for The Wall Street Journal, and she joins me now.

Okay, so, Belle, how does this automated reasoning work, this mathematical concept that Amazon is turning to to solve hallucinations in part? Automated reasoning is actually a branch of AI. So in some ways, you can think of it as using AI and math to sort of fight back against a different form of AI's hallucinations or propensity to spit back this inaccurate data.

And automated reasoning is really using computers to automate the mathematical logic behind putting rules into AI and sort of hard coding it. So machine learning differs from automated reasoning in that it basically hoovers up a bunch of data and that can be structured, unstructured data, can be words or text, it could be numbers. And it teaches machines or computers how to capture patterns from that data. So how to separate a dog from a cat.

how to identify a number from a letter. And so that's how the machine captures or gets its intelligence. Whereas in automated reasoning, you're sort of hard coding a set of rules and logic into a system. So the AI is able to check itself for errors in a way? Yeah.

Yeah, that's right. So similar to the way that we've heard that some large language models can reason through problems, is the system working the way that it's intended? Is the model spitting out an answer that's accurate based on a preset defined set of rules? And those rules for a company can be a set of internal company guidelines for employees, or it could be a product guidebook for customers to know what sorts of services and products you have in your catalog.

Okay. So speaking of customers, has Amazon had much success taking this approach to market? It's relatively new and it's something that they call in preview. So they're certainly testing it and hoping that it really unlocks a lot of business deals for them because right now hallucinations are a big blocker, not just for consumers like you and I to

use chatbots more fully in our daily lives, but for businesses who need that reliability when they're doing things like creating advertisements for pharmaceuticals and they can't run afoul of regulations or at worst promote something that is completely inaccurate.

So, for instance, PricewaterhouseCoopers, the big audit, accounting and tax firm, is actually a customer of Amazon's. And so that's important because on one hand, the AI may be trying to help PricewaterhouseCoopers achieve the goal of creating really good advertising. But that runs up against the goal of ensuring that the regulations are met for how these drugs are marketed.

And so you need automated reasoning to come in and say, yes, we are adhering to the regulations or no, we're not adhering to the regulations. What are experts saying about automated reasoning? Is it something that's really going to put an end to hallucinations? Oh, absolutely not. When you pose this question to an automated reasoning system, the AWS scientist who told me about how they're using these systems said the answer is, quote, undecidable.

And so that's a really interesting answer that I interpreted as no, because when automated reasoning can't tell you something with accuracy 100% of the time, that means it's probably a no because it can't say that it's 100% a yes. Are there some other possible solutions?

The solution that Amazon is pushing and its competitors like Microsoft and Google also have something similar for reducing chatbot hallucinations. And so they're saying that maybe the hallucinations can be mitigated or chatbots might be taught to say, I don't know, rather than a

eliminating them altogether. There are actually really great uses for hallucinations in creative sectors and fields where you want a really wacky image because you are a painter or you want some out-of-the-box song lyric because you're a lyricist. So it's by design that these chatbots hallucinate, but we really do want them to not hallucinate in circumstances where it really, really matters. But until then, it seems like you'll always maybe want a human in the chain to check that things are right.

Yeah, that's right. Some of the big tech companies are saying that humans should be out of the loop altogether, that automated reasoning and other methods like retrieval augmented generation should supplant the need for a human to basically check the output of a chatbot or for a doctor to check the output of a medical question you input into the system. PricewaterhouseCoopers still has their legal team review the advertisements, for instance. So the chatbot and automated reasoning forms the first layer of checking.

That was our reporter, Bell Lin. And that's it for Tech News Briefing. Today's show was produced by Julie Chang with supervising producer Catherine Milsop. I'm Pierre Bien-Aimé for The Wall Street Journal. We'll be back this afternoon with TNB Tech Minute. Thanks for listening.