We're sunsetting PodQuest on 2025-07-28. Thank you for your support!
Export Podcast Subscriptions
cover of episode When the Autopilot Switched Off

When the Autopilot Switched Off

2022/5/6
logo of podcast Cautionary Tales with Tim Harford

Cautionary Tales with Tim Harford

AI Deep Dive AI Chapters Transcript
People
旁白
知名游戏《文明VII》的开场动画预告片旁白。
Topics
本集讲述了1994年俄罗斯航空593航班坠毁事故,该事故中,机长允许其子女在驾驶舱内操作控制杆,虽然飞机当时处于自动驾驶状态,但由于孩子无意中操作失误,导致自动驾驶仪部分失灵,最终导致飞机坠毁,机上75人全部遇难。该事故凸显了即使在有安全装置(如自动驾驶仪)的情况下,也可能因为人为因素而导致严重后果。同时,本集还探讨了风险补偿理论(Peltzman效应),即人们在感觉更安全时可能会承担更多风险。通过对戴通纳500大赛中安全装置使用情况的分析,以及对汽车安全带使用情况的回顾,本集论证了风险补偿理论的存在,并指出安全装置虽然能够降低事故风险,但也可能导致人们行为更加冒险,从而部分抵消安全装置带来的益处。但总的来说,安全装置仍然是必要的,关键在于理解其局限性,并做出明智的决策。

Deep Dive

Chapters
The episode begins with the story of Aeroflot Flight 593, where the autopilot was inadvertently switched off by a teenager, leading to a tragic crash. This sets the stage for a discussion on the dangers of overreliance on safety features.

Shownotes Transcript

Translations:
中文

So I have some big news for vegans and vegetarians everywhere. It's Hellman's plant-based mayo spread and dressing. Made for people with a plant-based diet or anyone really who wants to enjoy the great taste of Hellman's real without the eggs. Hellman's plant-based is perfect for sandwiches, salads, veggie burgers, or any of your family favorites.

To celebrate, Hellman's is sharing some easy, delicious plant-based recipes at hellmans.com. Hellman's plant-based mayo spread and dressing. Same great taste, plant-based. I'm Malcolm Gladwell, and I'd like to take a moment to talk about an amazing new podcast I'm hosting called Medal of Honor.

It's a moving podcast series celebrating the untold stories of those who protect our country. And it's brought to you by LifeLock, the leader in identity theft protection. Your personal info is in a lot of places that can accidentally expose you to identity theft. And not everyone who handles your personal info is as careful as you.

LifeLock makes it easy to take control of your identity and will work to fix identity theft if it happens. Join the millions of Americans who trust LifeLock. Visit LifeLock.com slash metal today to save up to 40% off your first year. The most innovative companies are going further with T-Mobile for Business.

Tractor Supply trusts 5G solutions from T-Mobile. Together, they're connecting over 2,200 stores with 5G business internet and powering AI so team members can match shoppers with the products they need faster. This is enriching customer experience. This is Tractor Supply with T-Mobile for Business. Take your business further at T-Mobile.com slash now. ♪

There are 63 passengers on board flight 593. Some are drinking beer, some are asleep, others are bored. There are no smartphones to play with because this is 1994. They left Moscow four time zones ago. Hong Kong is still several hours away. The flight is long. Time goes slowly. It's dark outside and even if it wasn't, there wouldn't be much to look at.

The Airbus A310 is inching its way over the vast expanse of southern Siberia. Grasslands and forests and hills. To the south lies Kazakhstan. Ahead, Mongolia. The weather is calm. Visibility is good. The plane is on autopilot. And the captain has gone back to the cabin for a snooze. An off-duty pilot, Vladimir Makarov, is travelling as a passenger.

He's ambled to the cockpit to pass the time with his Aeroflot colleagues, relief captain Yaroslav Kudrinsky and first officer Igor Piskarev. And he's brought guests, Kudrinsky's two teenage children. In this pre-911 era, attitudes to cockpit security were more relaxed. The kids have got free flights, a perk of their dad's job, and he's taking them to see Hong Kong. It's their first international journey.

Kudrinsky, Piskarev and Makarov shoot the breeze for a while, then Kudrinsky turns to his daughter. Come and sit here now in my seat. Would you like that? 13-year-old Yana takes his place at the controls alongside co-pilot Piskarev. Dad, raise me up. Kudrinsky adjusts the seat so Yana can see better. How high are we flying? Here, look. 10,100 metres. It's a lot, isn't it? It's a lot.

It's a cloudless night. Far above, the stars are shining. Down below, the land is covered in snow. Hey, Jana, you're going to fly the aeroplane a bit. Go ahead, take the controls. You don't think he's actually going to let Jana fly the plane, do you? Remember, the autopilot is switched on. Kudrinsky is only going to give Jana the illusion of being in control. He adjusts the autopilot so the plane will bank first to the left,

then back to the right, while he tells Jana to turn the control wheel to the left, then to the right. She'll think the plane's responding to her touch, but the autopilot will be keeping them safe the whole time. Hold the wheel, but don't push the buttons. He just needs to make sure that it doesn't accidentally get switched off. This button, the red one, don't touch it. I'm Tim Harford, and you're listening to...

to cautionary tales. 43 drivers are lined up for the Daytona 500, the most keenly anticipated race in the NASCAR calendar. This time, however, that anticipation was mixed with a sense of unease. In the previous nine months, three drivers had lost their lives in accidents.

2001 had been the deadliest season in decades. All three had been killed by the same injury, a bacillus skull fracture. Think about what happens when a car hits a wall. The car stops suddenly. The car's driver would keep moving forward if they weren't strapped into their seat. A bacillus skull fracture can happen when the driver's body is restrained, but their head is not.

If the head shoots forward with enough force and at the wrong angle, there's a risk of cracking the bones that connect the skull to the neck. So why not strap the driver's head in too? Racing driver Jim Downing asked himself that question when a fellow racer was killed by a bacillar skull fracture back in 1981.

Downing and his brother-in-law spent years perfecting a head and neck support device, HANS, or hands. When at long last they put it on the market, hardly anyone wanted to buy it. I ain't wearing that damn noose, said Dale Earnhardt Sr., nickname The Intimidator. Earnhardt was larger than life, a seven-time champion. His opinion was widely shared.

When another NASCAR driver started to wear a Hans device, he was called Sissy, Pansy, Candy Ass. After those three deaths in quick succession, attitudes started to soften, but not by much. Of the 43 drivers who lined up for the Daytona 500 in 2001, only five were wearing a head and neck support device.

Dale Earnhardt Sr. was not among them. If the drivers in Soussiennes seems puzzling, you have to remember that most accidents weren't fatal. With 25 laps to go in this Daytona 500, there came an example. A huge crash, a 20-car pileup. One car flipped through the air and landed upside down on top of another. It looked horrendous. But the driver got out and walked away.

They usually did. When the track was cleared of debris and the race resumed, one of Dale Earnhardt's teammates took the lead. His son, Dale Earnhardt Jr, was running second. Earnhardt himself was in third, driving defensively, determined to stop anyone else from getting past. On the fourth and final turn of the very last lap, the back of Earnhardt's car clipped the front of the car on his inside.

Earnhardt tried to keep control, but couldn't. His car veered to the right, diagonally up the banked turn. It took out a rival's car and the two slid into the wall at the top of the bank. At first glance, the crash didn't look as bad as the earlier one,

The television commentators didn't sound alarmed. Dale Earnhardt gets turned sideways. They noted the incident, then went back to describing the battle for the win. Coming down to the finish, though, it is Michael Waltrip trying to hold off Dale Earnhardt Jr. Earnhardt's car and that of his rival, Ken Schrader, slipped down the banking and came to rest on the grassy infield. Schrader got out and walked stiffly round to check that Earnhardt was OK.

He took one look into Earnhardt's car, then turned and gestured urgently to the medical crew who were just arriving. But there was nothing they could do. Dale Earnhardt had been killed by a basilar skull fracture. The inventor, Jim Downing, who'd struggled for years to interest anyone in his hands device, suddenly found that his telephone didn't stop ringing.

On Aeroflot Flight 593, 13-year-old Jana has done what her father told her. She's avoided the temptation to press the red button. While Captain Kudrinsky has been coaching his daughter to turn the plane left, then right, the autopilot has actually been flying the plane. Now it's the turn of Jana's older brother. 15-year-old Eldar gets into his dad's seat.

We might wonder how Kudrinsky's two colleagues are feeling about his impromptu bring-your-kids-to-work day. Is the co-pilot, Igor Piskarev, at all anxious? If so, we might expect him to be hunched over his own set of controls, ready to take charge at a moment's notice. But no, it seems not. He's pushed his seat right back, almost as far as it can go.

Kudrinsky goes through the same routine as he did with Jana. He adjusts the autopilot, then gives Eldar instructions to give him the illusion that he's in control. Eldar turns the control wheel. It seems to him that the plane turns in response. Is the plane turning? That's great! From the back of the cockpit, Jana says something. Kudrinsky turns to answer.

It's hard to make out what they're talking about. Something about going to sleep in the first class cabin. Eldar is at the controls. First officer Piskarev doesn't seem to be paying close attention because it's the 15-year-old who's the first to notice that something isn't right. Dad, I don't understand. Why is it turning? It's turning by itself. Yes.

AI might be the most important new computer technology ever. It's storming every industry, and literally billions of dollars are being invested. So, buckle up. The problem is that AI needs a lot of speed and processing power. So how do you compete without costs spiraling out of control? It's time to upgrade to the next generation of the cloud, Oracle Cloud Infrastructure, or OCI.

OCI is a single platform for your infrastructure, database, application development, and AI needs. OCI has 4 to 8 times the bandwidth of other clouds, offers one consistent price instead of variable regional pricing, and of course, nobody does data better than Oracle. So now you can train your AI models at twice the speed and less than half the cost of other clouds.

If you want to do more and spend less, like Uber, 8x8, and Databricks Mosaic, take a free test drive of OCI at oracle.com slash strategic. That's oracle.com slash strategic. oracle.com slash strategic. I love cycling, and I'm eager to get my kids cycling too. It's a great way for them to stay fit and move around our home city independently. But of course, I also want them to be confident and safe.

which is where Guardian Bikes comes in. The bike comes in a box and it's easy to assemble with all the tools you need and simple online instructions. My son and I unboxed his bike together, spent about 20 minutes working as a team to assemble it

And then he was on the bike and ready to ride. The bike looks great and with the SureStop braking system it brakes quickly and safely without locking the front wheel and sending you over the handlebars. Guardian bikes offer a 365-day money-back guarantee covering returns, repairs and spare parts. Join hundreds of thousands of happy families by getting a Guardian bike today.

Visit GuardianBikes.com to save up to 25% off bikes. No code needed. Plus, receive a free bike lock and pump with your first purchase after signing up for the newsletter. That's GuardianBikes.com. Happy riding! If you're listening to this right now, you probably like to stay on top of things, which is why I want to mention The Economist. Today, the world seems to be moving faster than ever. Climate and economics, politics and culture, science and technology, wherever you look,

Events are unfolding quickly, but now you can save 20% off an annual subscription to The Economist so you won't miss a thing. The Economist broadens your perspective with fact-checked, rigorous reporting and analysis. It's journalism you can truly trust. There is a lot going on these days, but with 20% off, you get access to in-depth, independent coverage of world events through podcasts, webinars, expert analysis, and even their extensive archives. So where

Whether you want to catch up on current events or dive deeper into specific issues, The Economist delivers global perspectives with distinctive clarity. Just to give an example, What's Next for Amazon as it turns 30? analyzes how Amazon's fourth decade looks like an area of integration for the company. Go beyond the headlines with The Economist. Save 20% for a limited time on an annual subscription. Go to economist.com and subscribe today.

It's tempting to question the judgement of Dale Earnhardt Senior for his casual dismissal of the safety device that could have saved his life. But perhaps we should equally question 98% of people who bought a new Ford in 1956. Back then, cars didn't come with seatbelts as standard. Ford offered the option for a modest fee.

Admittedly, these were simple seatbelts that fastened only across your lap. In an accident, you might still lurch forward and bash into the steering wheel or dashboard. But the belts were better than nothing. And Ford's modestly priced safety upgrade also included a padded dash. Still, only 2% of buyers were willing to pay for the upgrade.

Soon after, an engineer at Volvo invented the modern three-point seatbelt, with a second strap passing diagonally across the shoulder. Evidence mounted that these seatbelts dramatically improved your chance of surviving a crash.

In 1966, the US government required all new cars to have seatbelts and some other safety features too, such as toughened windscreens and shock-absorbing steering columns. A few years after that, an economist at the University of Chicago wondered how many lives these new regulations had saved.

Sam Peltzman looked at the numbers and published his answer in a paper called The Effects of Automobile Safety Regulation. And that answer was surprising. The regulations, concluded Peltzman, had saved no lives at all. So, was the evidence on seatbelts wrong? No. If you had a crash, they really would make you much less likely to die.

But, said Peltzman, you couldn't look only at how likely people were to die if they had an accident. You also had to look at how likely they were to have accidents. Drivers who feel safer because they're strapped into padded, shock-absorbing cars might drive more recklessly. Peltzman found that drivers were surviving more accidents, but also having more accidents. The two effects more or less cancelled each other out.

Particularly once you realise that some of the victims of these extra accidents weren't protected by seatbelts. We call these people pedestrians. The idea that people might take more risks when they feel more protected became known as risk compensation theory, or the Peltzman effect. It was immediately controversial.

The politically charged implication was that government regulations might be self-undermining. And there was another reason for the controversy. Peltzman's research methods were primitive. As he later admitted, he added that if an undergraduate today gave him that work, he'd tell them to redo it. Some other researchers looked at the same road safety data and said there was no effect.

Still others charted a middle ground. They thought risk compensation did happen, but it only partially cancelled out the good of the safety features. It was hard to get a definitive answer, because the world is messy.

You can't just compare data on traffic accidents before and after a change in regulations. Lots of other things might also be changing in that time. What kind of cars people buy, what journeys they make, how roads are maintained, how laws are enforced, how statistical bureaus count accidents, and so on. How do you adjust for all these things? Reasonable people can differ. Then, some researchers realised that there's a way to cut through the mess.

It's a natural experiment on car safety regulations that's as close to perfect as you can get. NASCAR. The cars are all very similar. The drivers have comparable skills. They drive around the same circuits year after year. And because it's such a popular sport, there's comprehensive data on accidents. Any differences in that data are quite likely to be down to changes in the safety regulations.

Dale Earnhardt's accident led to a big change. In the week that followed Earnhardt's death, Jim Downing sold more hands devices than he had in all of the previous ten years. Within months, NASCAR had mandated that all drivers must wear them. Two researchers looked at data on NASCAR crashes before and after this big rule change.

Earnhardt's fatal crash, remember, had been the fourth in less than a year. In the seasons after the HANS device became compulsory, no one died. But there were more accidents. The drivers were feeling more protected and taking more risks. Other researchers have looked at other NASCAR regulation changes over the years and tried to put a number on the size of the Peltzmann effect.

Various studies estimate that whenever you improve the safety of the cars, you'll lose about 20 to 40% of the benefits because the drivers will respond by pushing the envelope. So risk compensation looks like a real effect, but a partial one. In this case at least, it isn't pointless to mandate the use of safety devices.

Yes, they might tempt us to be a bit more reckless, but on the whole, they can still save lives. Indulge me for a moment to consider another car safety device, the speed limiter. You set a maximum speed, say 40 miles an hour. You drive as you normally would, slowing down in traffic and speeding up when the road is clear. But once you reach 40, the car will no longer accelerate further as your foot stays on the gas.

That means you can't absentmindedly go too fast and get a speeding ticket. It's a useful device, but it's also one you want to be able to override in a hurry. There are emergency situations where you need to accelerate, and you don't have time to think about which lever turns off the speed limiter. Instinctively, you'd just slam your foot to the floor. So the system is designed to account for that possibility.

Apply gentle pressure on the accelerator pedal and it ignores your input. Sudden hard pressure and it turns itself off. Back in the cockpit of Aeroflot Flight 593, 13-year-old Jana had been gentle at the control wheel. The autopilot had stayed in charge, just as Captain Kudrinsky had intended.

15-year-old Eldar, though, was more gung-ho. He applied more force. So much force that the plane's autopilot did what a car's speed limiter does when the driver slams the pedal to the metal. The autopilot turned itself off. At least in part. The part that controls the ailerons, the section of the wings that roll the plane one way or the other. Somewhere in the cockpit, a light came on.

The autopilot silently confirming that it had partially disengaged itself. Captain Kudrinsky didn't notice the light coming on. He was distracted, talking to Yana. Co-pilot Piskarev didn't notice the light. Off-duty pilot Makarov didn't notice. Eldar? Eldar was 15. How could he possibly have known what the light meant? Dad, I don't understand. Why is it turning? It's turning by itself. Yes. I...

I don't know why it's turning. The three pilots all believed that the autopilot was still fully in charge of the plane. So the mystery they thought they had to solve was why the autopilot had put them into a sharp turn. It looked very much like the start of a holding pattern when you're waiting above an airport for a slop to land. The pilots assumed the autopilot must somehow have glitched, but it hadn't.

Eldar was still holding his control wheel slightly off-centre. And that meant the plane was continuing to bank further and further to the right. It passed a tipping point. The plane banked so far that it couldn't maintain forward momentum. It started to fall out of the sky.

The sudden G-forces slammed everybody backwards. Kudrinsky and Makarov were pinned to the back of the cockpit. Eldar was trapped where he sat in the pilot's seat. Piskarev was pressed back into his co-pilot's chair, which he'd previously pushed back as far as it would go. He stretched forward, desperately reaching for the controls, but First Officer Piskarev was only five feet two inches tall. He could barely get a hand on them.

Only one person in the cockpit was sitting in a place where he could easily reach the controls. That person was Eldar. Cautionary Tales will return in a moment.

I love cycling and I'm eager to get my kids cycling too. It's a great way for them to stay fit and move around our home city independently. But of course, I also want them to be confident and safe. Which is where Guardian Bikes comes in. The bike comes in a box and it's easy to assemble, with all the tools you need and simple online instructions. My son and I unboxed his bike together, spent about 20 minutes working as a team to assemble it,

And then he was on the bike and ready to ride. The bike looks great and with the SureStop braking system it brakes quickly and safely without locking the front wheel and sending you over the handlebars. Guardian bikes offer a 365-day money-back guarantee covering returns, repairs and spare parts. Join hundreds of thousands of happy families by getting a Guardian bike today.

Visit GuardianBikes.com to save up to 25% off bikes. No code needed. Plus, receive a free bike lock and pump with your first purchase after signing up for the newsletter. That's GuardianBikes.com. Happy riding! If you're listening to this right now, you probably like to stay on top of things, which is why I want to mention The Economist. Today, the world seems to be moving faster than ever. Climate and economics, politics and culture, science and technology, wherever you look,

Events are unfolding quickly, but now you can save 20% off an annual subscription to The Economist so you won't miss a thing. The Economist broadens your perspective with fact-checked, rigorous reporting and analysis. It's journalism you can truly trust. There is a lot going on these days, but with 20% off, you get access to in-depth, independent coverage of world events through podcasts, webinars, expert analysis and even their extensive archives. So where

Whether you want to catch up on current events or dive deeper into specific issues, The Economist delivers global perspectives with distinctive clarity. Just to give an example, What's Next for Amazon as it turns 30? analyzes how Amazon's fourth decade looks like an area of integration for the company. Go beyond the headlines with The Economist. Save 20% for a limited time on an annual subscription. Go to economist.com and subscribe today.

Perhaps there's another reason Sam Peltzman's risk compensation theory has been so controversial. We feel uncomfortable about what it says about us. Do we really start to drive more recklessly when we buy a safer car? We'd like to think that we drive equally responsibly all the time, but we don't. The idea that we take more risks when we feel protected is just another way of saying that we behave more cautiously when we feel vulnerable.

And that's obviously true. The economist Gordon Tullock came up with a famous thought experiment to prove the point.

Imagine, Tulloch said, that instead of mandating seatbelts, the government instead said that every steering wheel must have a sharp metal spike sticking out of it, pointing straight at the driver's chest. You'd drive far more cautiously, of course. Remove the spike and you'd return to driving normally. Which is to say, more recklessly in comparison.

Researchers have found suggestions of the Peltzmann effect in all kinds of places, almost all of them fiercely controversial. One unexpected example concerns prophylactic drugs that protect you against catching HIV. Some researchers have found that when people start to take these drugs, they have more risky sex. Other studies say there's no effect. Or consider the COVID pandemic.

When mask mandates came in, some worried that people would compensate by getting two lakhs on social distancing. Researchers have looked into that question, again, with varying results. Sometimes we're well aware that we're choosing to live more dangerously because we feel more protected. Think of Covid vaccines. I know I took more social risks once I was fully dosed.

But that's only because I'd been cautious before, when seeing friends felt like driving with Tulloch's spike. It's easy to get moralistic about the Peltzmann effect, framing it in terms of recklessness, laziness or selfishness. But some safety measures work so well that we don't even realise we're relying on them to protect us.

In his book Factfulness, Hans Rosling recounts the tale of Swedish medical students visiting a hospital in India. One student was running late. As she dashed across the lobby to try to join her colleagues in the elevator, another student did something I've done before, and perhaps you have too. She stuck out her foot to stop the elevator doors from closing. But the elevator doors didn't stop closing.

They kept on pressing together, painfully trapping her leg. Then the elevator began to move upwards. The student's leg was about to be crushed on top of the doorway when their Indian host lunged for the emergency stop button. He was astonished by the student's stupidity. She'd just stuck a limb into a moving machine. She was astonished to discover that elevator doors in India didn't all behave like the ones in Sweden.

Every time we stick out a foot to hold an elevator, we're taking a risk. But some small risks are acceptable, and some make life worth living. When I started spending time again with vaccinated loved ones, that wasn't irresponsible behaviour that merits an apology. It's exactly what the Covid vaccines were for. Risk compensation can be a good thing.

Captain Kudrinsky was taking a small risk, in the sense that it wasn't especially likely that his children would somehow contrive to crash the plane. But it would be hard to describe that risk as acceptable. The downside was just too big, and now everything was spiralling out of control. Flight 593 was plunging towards the ground.

The three qualified pilots had been pinned back by the sudden g-forces. Only young Eldar had a proper grip on the controls. Hold it! Hold the steering wheel! Hold it! The speed! To the left! To the left! Left! The other way! I am turning it left! To the right! To the right! The autopilot is no longer controlling the ailerons. But remember, it hasn't turned off completely.

It responds to the loss of altitude by pitching the nose up, which lessens the forces in the cockpit just enough for Kudrinsky to reclaim his seat. Eldar, get out! Get out, Eldar! Get out! Get out! Eldar, scrambling out of the seat, accidentally stands on the pedal that controls the rudder. Or perhaps it's Kudrinsky who stands on it as he clambers in.

Meanwhile, Iskadev has instinctively overcompensated for the dive and pulled the control column back too far. The plane is now pointing almost directly upwards. That causes it to stall. The plane starts to nosedive and with the rudder deflected by the push of the pedal it starts to spin like a corkscrew too. The effect is disorientating. Full power! Full power! Full power! I gave it full power!

The way you recover a plane from a stall is by pointing the nose down and getting up enough speed that you can gradually pull it back up again, come out of the dive and level off. And it would have been if only they hadn't already fallen quite so far.

A moment after Captain Kudrinsky said, "Everything's fine!" the residents of the coal mining town of Mezderechensk were woken by the sound of a distant explosion. It was two minutes to one in the morning on a cold Thursday night in March. Air traffic controllers were mystified. They'd received no distress call from the pilots.

Flight 593 had simply disappeared from their radar. Was it a terrorist attack? An asteroid? They sent a search helicopter. Twelve miles from Mezhde Rechensk, the search helicopter found a raging fire on a hillside. But at night, in a forest, it couldn't land. The search party arrived at daylight. The snow lay thick on the ground.

All 75 passengers and crew had been killed instantly. Some bodies had burned, others could be recognised, including the ones in the cockpit. And the rescuers were puzzled. Why were the bodies of two teenage kids in the cockpit, along with those of three pilots? On the second day of the search, they found the plane's black box.

Investigators listened to the recording and understood what had happened and how quickly. From Eldar asking why the plane was turning to Kudrinsky saying they'd be fine took just two and a half minutes. What lesson might we learn from the cautionary tale of Aeroflot Flight 593? Don't let your kids fly a passenger jet, obviously. But there are other insights to be gained.

The investigation taught manufacturers of airplanes that a warning light is too easy to miss when the autopilot is partially overridden. If there'd been an audible alarm, the pilots might not have wasted those vital few seconds being confused about why the plane was turning. But for me, Flight 593 is the perfect illustration of the Peltzmann effect.

Would Captain Kudrinsky have let his teenage children sit at the plane's controls if the plane hadn't had autopilot? Of course not. It's unthinkable. He took the risk only because he felt protected. The lesson isn't that we shouldn't have safety devices. Autopilots are a good thing. So are seatbelts. Nor is it that we should never take any risks at all. The lesson, I think, is that whenever we rely on a safety feature to protect us...

We should notice that and try to understand how it works and what are its limits so we can take informed decisions. It's true of Covid vaccines. It's true of the sensors in elevator door mechanisms. And it's true of autopilots. And not only for the obvious reasons, because there's a final irony to Flight 593. In their panic,

the pilots forgot something that could have saved them. As the stalled plane corkscrewed downwards, the autopilot would have kicked in and recovered the situation if only it had detected no pilot's inputs at the controls. When Kudrinsky put his children in the pilot's chair, he was trusting the autopilot too much. But when he and Piskarev grappled to save the plane...

They weren't trusting the autopilot enough. If everyone had taken their hands off the controls of Flight 593, the autopilot would have saved them all. For a full list of our sources, please see the show notes at timharford.com.

Cautionary Tales is written by me, Tim Harford, with Andrew Wright. It's produced by Ryan Dilley, with support from Courtney Guarino and Emily Vaughan. The sound design and original music is the work of Pascal Wise. Julia Barton edited the scripts. It features the voice talents of Ben Crow, Melanie Gutteridge, Stella Harford and Rufus Wright.

The show also wouldn't have been possible without the work of Mia LaBelle, Jacob Weisberg, Heather Fane, John Schnarz, Carly Migliori, Eric Sandler, Royston Besserve, Maggie Taylor, Nicole Morano, Daniela Lacan and Maya Koenig. Cautionary Tales is a production of Pushkin Industries.

If you like the show, please remember to share, rate and review. And if you want to hear the show ad-free and listen to four exclusive Cautionary Tales shorts, then sign up for Pushkin Plus on the show page, in Apple Podcasts or at pushkin.fm slash plus.

I'm Malcolm Gladwell, and I'd like to take a moment to talk about an amazing new podcast I'm hosting called Medal of Honor. It's a moving podcast series celebrating the untold stories of those who protect our country. And it's brought to you by LifeLock, the leader in identity theft protection. Your personal info is in a lot of places that can accidentally expose you to identity theft.

and not everyone who handles your personal info is as careful as you. LifeLock makes it easy to take control of your identity and will work to fix identity theft if it happens. Join the millions of Americans who trust LifeLock. Visit LifeLock.com slash metal today to save up to 40% off your first year.

So I have some big news for vegans and vegetarians everywhere. It's Hellman's plant-based mayo spread and dressing. Made for people with a plant-based diet or anyone really who wants to enjoy the great taste of Hellman's real without the eggs. Hellman's plant-based is perfect for sandwiches, salads, veggie burgers, or any of your family favorites.

To celebrate, Hellman's is sharing some easy, delicious plant-based recipes at hellmans.com. Hellman's Plant-Based Mayo Spread and Dressing. Same great taste, plant-based.

The news isn't always good news, but when you're getting quality journalism and in-depth expert analysis that's held up for more than 180 years, that is definitely good news. So if you haven't already, save 20% with The Economist's summer sale today and stay on top of the stories that matter to you. You'll instantly gain unlimited digital access to daily articles, special reports, award-winning podcasts, subscriber-only events, and so much more. Now that's

Good news. Go to economist.com and subscribe today.