Deep Dive
Shownotes Transcript
We all live with it, don't we? Uncertainty. It's just there every day from like tiny decisions to the really big life stuff. Absolutely. It's this constant background hum. And I think a lot of us, you know, we want to understand it better, get a handle on it without feeling totally overwhelmed. Which is exactly what we're trying to do in this deep dive. Yeah. So today we're unpacking uncertainty using David Spiegelhalter's book, The Art of Uncertainty, as our guide. It's a fantastic book.
really gets into chance, risk, luck, even our own ignorance. We're aiming to pull out some surprising facts and maybe bring a bit more clarity. Sounds good. He covers so much ground. Let's jump in. Where does he start? Well, right at the beginning, he makes it really personal. He talks about his grandfather, Cecil, in World War I. Ah, yes, the story about the trench explosion. Exactly. His grandfather was injured, which sounds awful, of course. Terrible luck, you'd think. But here's the twist. That injury actually saved him.
It got him moved away from the front line. And his old battalion. Suffered massive losses later at the Somme. So that single, seemingly random event...
Well, it meant Spiegelhalter himself could eventually be born. Wow. That really hits home. He calls those micro-contingencies, doesn't he? Yeah, these tiny, unpredictable forks in the road that just stack up and can totally change a life's path. It shows how fragile things can be. And it's not just those huge, life-altering moments either. He makes it clear, unsurprisingly,
Uncertainty is baked into everything. Mundane stuff. Big existential questions. Right. From Roman times dealing with, you know, disease and shorter lifespans to us today worrying about jobs or the climate. It's always been part of the human condition. And crucially, he argues it's not some objective thing out there. No, exactly. It's personal. It's about your relationship with the world shaped by what you know, what you perceive.
What's uncertain for me might not be for you. OK, so let's dig into that a bit. He breaks down different kinds of uncertainty, which I found really useful. Yes, this is a really key distinction he makes between aleatory and epistemic uncertainty. Right. So aleatory is that's the future stuff, right? The cure chance element, like flipping a coin before it lands. You just can't know. Inherently unpredictable.
Randomness. But epistemic uncertainty is different. That's about what we don't know now, but theoretically could know. Exactly. So if I flip the coin, cover it, and then ask you heads or tails, the outcome is fixed. You just lack the information.
your uncertainty is epistemic. Okay, that makes sense. So you can reduce epistemic uncertainty by learning more, but aleatory, that's just chance. Precisely. And it loops back to his point about your probability versus the probability. It's about your state of knowledge. Which leads nicely into how we actually respond to uncertainty as humans.
Because we don't all react the same way, do we? Not at all. Spiegelhalter talks about this whole spectrum of emotions. Anxiety, obviously, maybe fear. But also curiosity, excitement even. Yeah, absolutely. But he also warns that a real intolerance for uncertainty can be problematic, can feed into anxiety, depression. He mentions his father's travel fever, that intense anxiety before trips. Right, which got so bad he just stopped traveling.
And when the author sells something similar, he used cognitive behavioral therapy, CBT. Trying to reframe that anxiety as excitement, kind of a mental trick. Sort of, yeah. Changing the perception, which raises that other question. How much do we actually want to know? Like, would you want to know the exact date you'll die?
Most people say no, right? Definitely not. Or like the final score of a game before it starts. Yeah. It kind of ruins the fun, the suspense. Sometimes ignorance is bliss. It reminds me of Richard Feynman, the physicist. He said he was comfortable with doubt, with not knowing. Maybe that's the key. Accepting it. It seems like a healthier approach. Yeah. But Spiegelhalter also pushes for quantifying uncertainty, moving beyond just vague words. Yeah. The problem with words like likely or possible, they mean different things to different people. Hugely different.
He uses the Bay of Pigs example. The CIA told Kennedy it had a fair chance of success. Which sounds okay. Maybe 50-50. Kennedy seemed to think so. But the CIA's internal number was more like 30%.
A big gap in understanding. Using the number might have changed things. And the medical examples are stark, too. Common side effects meaning one 10 percent. But people think it means closer to a third, around 34 percent. That's a massive difference when you're assessing risk. Even when bodies like the IPCC try to standardize terms like likely being 66, 100 percent probability.
People still often guess lower, maybe 60%. It really shows how tricky communication is. Numbers seem more reliable, less ambiguous. So how do we get better at judging our own certainty? He talks about calibration, right? Yes, the confidence quiz idea. It's clever. You answer questions and you rate how confident you are, say 5 out of 10, up to 10 out of 10.
And the scoring penalizes you if you're super confident but wrong. Exactly. It discourages overconfidence. It's about being well calibrated, knowing how much you actually know. And what does this quiz usually show? Generally, three types of people emerge. Those who know a lot and are pretty accurate about their confidence. Then those who are more cautious, maybe underestimate their knowledge a bit.
aware of what they don't know. And the third, the overconfident ones, often wrong, but very sure they're right. Being well calibrated is crucial for good judgment.
He links this to forecasting, doesn't he? The foxes and hedgehogs. Yeah, borrowing from Philip Telleck, foxes are adaptable, skeptical. They draw on lots of ideas, change their minds. Hedgehogs know one big thing and stick to it, often overconfidently. Like Nate Silver with the 2016 election prediction, giving Trump maybe a 29% chance. People said he was wrong afterwards. But he wasn't necessarily wrong. He gave a probability. Unlikely things do happen.
Foxes, like Silver in his method, tend to be better forecasters, precisely because they embrace that uncertainty. They aren't dogmatic. OK, let's shift gears slightly. The book also delves into the history and maths of probability itself. Where did all this start? Well, way back. Early forms of gambling, like using knuckle bones as dice in ancient times, people had an intuitive sense of chance. But the formal maths came later. Much later.
Think Cardano in the 16th century with his book of games of chance. And Fibonacci's introduction of Hindu-Arabic numerals to Europe was also key for calculations. And those early thinkers tackled problems like...
calculating dice odds. Yeah, basic stuff like that, but also more complex things like the problem of points, how to divide stakes fairly if a game is interrupted. Some of those ideas are ancestors of methods used today, like in Cricket's Duckworth-Lewis-Stern system. Fascinating. And he mentions things like the binomial distribution. Right, for figuring out probabilities in repeated events, like multiple coin flips. And DeMauve's work approximating that, it gets quite mathematical. There's
There's also a philosophical debate, isn't there? Is probability objective, like a frequency or subjective, like a degree of belief? Yes, that's a long running discussion. Spiegelhalter seems to lean towards the subjective personal view, tying back to the idea of your probability. OK, let's talk about something maybe more relatable. Coincidences, those moments that just seem too improbable to be chance. Ah, yes. He has some great stories.
Like the two brothers, Doug and Ron Biederman, who checked into the same hotel, same night, randomly assigned adjacent rooms, and both happened to be wearing the same distinctive striped shirt bought independently. That's uncanny. How do statisticians even define a coincidence? As a surprising concurrence of events perceived as meaningfully related with no obvious causal connection. There's even a Cambridge coincidence collection he mentions.
So how did these seemingly impossible things happen? Is it just statistics? Largely, yes. One key factor is the law of truly large numbers.
Basically, with enough opportunities, even incredibly unlikely things are bound to happen eventually. Like the birthday problem. Put enough people in a room. Exactly. With just 23 people, there's actually a greater than 50% chance that two share the same birthday. It feels counterintuitive, but the maths checks out. He had his own coincidence on the radio, didn't he? He did.
During an interview about birthdays, two callers phoned in who shared his birthday, January 27th. Three people connected to the show, same birthday. Seems amazing, but again, large numbers. Lots of listeners. But he also warns about misinterpreting coincidences, especially tragic ones. Yes, the case of Lucia DeBurke, the nurse wrongly convicted based on seemingly improbable clusters of deaths.
Or even simple things like assuming double-yolked eggs are rarer than they are because you haven't thought about how many eggs are laid daily worldwide. We need to question our assumptions. Okay, what about luck? That feels related but different from pure chance or coincidence. It's a slippery concept. Spiegelhalter uses examples like his friend who survived a plane crash because his father insisted he move seats. Was that luck?
Or the father's good sense. Or Julianne Kopech, the teenager who survived a crash in days in the Amazon. How much was chance? How much was her knowledge and resilience? It's often a mix. He also brings up huge historical events, like the assassination of Archduke Franz Ferdinand.
A series of unlikely events and poor decisions led to Gavrila Princip being in just the right place at the right time. Immense consequences from seemingly random turns. So are there different types of luck? He discusses a few. Resultant luck, how things turn out due to chance factors after an action.
Circumstantial luck, the situation you find yourself in. And constitutive luck, the luck of your birth, your genes, your upbringing. The things you don't control at all. That last one feels huge. And what about luck in sports or science? Is someone just lucky? Often it's more complex. Yes, chance plays a role, but skills, preparation, mindset, what some call serendipity, being prepared to notice the unexpected are crucial. Think Fleming and the penicillin mold. It was
It was partly chance, but he was prepared to see its significance. This leads into another big question: randomness versus determinism. Are things truly random or just too complex for us to predict? Spiegelhalter explores this through his own prostate cancer diagnosis. He had a 50 percent chance of inheriting the BRCA2 gene mutation from his mother. Like Mendelian inheritance, a coin flip, basically. Right. But is that true randomness at the biological level or is it just
deterministic physics and chemistry that's far too complex for us to model perfectly. The whole chaos theory idea. The butterfly effect. Tiny changes having huge unpredictable outcomes. Exactly. It blurs the line. Philosophically, you have Laplace's demon, the idea that if you knew the position and momentum of every particle, you could predict the future perfectly. Determinism. Versus quantum mechanics, which seems to suggest inherent randomness at the smallest scales,
Stochasticity. It's a deep debate. But Spiegel-Holter takes a pragmatic approach. We treat things differently depending on the scale. We might use randomness deliberately, like in the Monte Carlo methods for the Manhattan Project or the UK lottery machines. Even though those lottery balls follow physical laws, we treat the outcome as random for practical purposes. Precisely.
And he notes we're generally bad at intuiting randomness. We see patterns where there are none, think random events should look evenly spread out when they're often clumpy, like shuffled playlists still playing songs by the same artist close together sometimes. Yeah, that drives people crazy. Okay, another crucial concept, context and conditional probability. How important is the background information? Hugely important. He gives a striking example from the UK COVID pandemic in June 2021.
The data showed that most COVID deaths were occurring among people who were fully vaccinated. Which sounds completely paradoxical, like the vaccines weren't working or were even harmful. Exactly how it could be misinterpreted. But the crucial context was missing. At that point, a very high percentage of the most vulnerable population, the elderly, those with health conditions, were fully vaccinated. So interesting.
Even with vaccines being highly effective because almost all the vulnerable people had the vaccine, the smaller number of breakthrough infections and deaths among them still outnumbered the deaths among the much smaller group of unvaccinated vulnerable people. Precisely. It's about the base rates. You need to consider the probability of dying given you are vaccinated versus given you are unvaccinated within specific groups.
It's like the seatbelt analogy. Most people injured in car crashes wear seatbelts. But that doesn't mean seatbelts are useless. It means most people wear them. That's a really clear illustration of why context is vital. Don't look at numbers in isolation. Absolutely. And this leads into Bayes' theorem. Right. The famous formula for updating our beliefs. Yeah. It's a formal way of thinking about how new evidence should change our assessment of how likely something is. Hmm.
What was our prior belief and how does this new information adjust it? Can you give an example? He uses facial recognition technology, say, at King Charles' coronation. Imagine the system is 99.9% accurate at identifying a specific wanted person. Sounds great. Very accurate.
But if that wanted person is extremely unlikely to actually be in the crowd, say a one in a million chance, that's the prior probability, even a system this accurate will produce overwhelmingly more false positives than true hits.
Most alarms will be for innocent people who just happen to look similar enough. Because you're applying a tiny error rate, 0.1%, to a huge number of innocent people and a high hit rate, 99.9%, to maybe just one target if they're even there.
Exactly. Bayes' theorem helps calculate that posterior probability, the chance the person flagged is the wanted person given the alarm. It would be much lower than 99.9%. It shows how prior belief or the base rate is critical. Absolutely. He also mentions the likelihood ratio, a component of Bayes used by Alan Turing and his team in breaking the Enigma code.
How much more likely is this evidence if hypothesis A is true versus hypothesis B? And this Bayesian way of thinking encourages updating beliefs, not being too dogmatic. Right. Cromwell's rule. Avoid being absolutely certain 0% or 100% probability because then no amount of evidence can ever change your mind. Always leave a little room for doubt. This seems relevant to science itself.
We often think of science as delivering solid facts, but Spiegelhalter emphasizes it's often messy, right? Very much so. It's an evolving process. He uses the measurement of the speed of light as an example. Over centuries, the accepted value changed, and crucially, the estimates of the uncertainty around that value also changed, often revealing previous estimates were overconfident. So even seemingly objective measurements have uncertainty baked in. Definitely.
And it's not just uncertainty in the data, but also in the models we use to interpret it. He talks about the recovery trial during COVID, which found dexamethasone was effective.
but the size of the benefit depended on assumptions made in the statistical model. And common statistical tools can be misunderstood, like p-values or confidence intervals. Yes, he points out they're often misinterpreted. A p-value doesn't tell you the probability your hypothesis is true, and a 95% confidence interval doesn't mean there's a 95% chance the true value lies within that specific interval.
Bayesian methods offer an alternative way to express probability more directly. But science does strive for rigor when claims are made. Oh, absolutely. For major discoveries like the Higgs boson, scientists demand extremely high levels of certainty, the famous five sigma level, meaning the result is incredibly unlikely to be just a random fluke. They explicitly acknowledge and quantify the uncertainty.
OK, so we have probability, but you also mentioned confidence earlier. How does confidence relate to probability? Can you have the same probability but different levels of confidence? Yes, that's a key point. Imagine predicting a coin flip. It's 50% probability, heads or tails. You're pretty confident in that 50% number because you understand the physics of a fair coin. Right. Now imagine comparing the weights of two objects just by looking. You know, I guess there's a 50% chance object A is heavier.
But your confidence in that 50 percent estimate might be quite low because you're just eyeballing it. Ah, so the probability estimate is the same, but the certainty about that estimate is different. Exactly. Intelligence analysts often use this. They'll give a probability for an event, but also a confidence level, high, medium, low, based on the quality and quantity of the evidence they have. He gives the example of U.S. intelligence agencies assessing Russian interference in the 2016 election. Right.
Different agencies might have agreed on the likelihood, but expressed different levels of confidence based on their specific sources and methods. But is this always used consistently? Not always. Sometimes confidence scales seem to supplement probability. Other times they almost substitute for it, which can be confusing.
Spiegelhalter shares his own experience estimating hepatitis C infections from contaminated blood dealing with indirect uncertainty based on layers of assumptions requiring careful expression of confidence. The IPCC uses confidence scales, too, for climate change findings. Yes, alongside probability statements to reflect the level of scientific understanding or agreement.
But even these have limits when facing what he calls deep uncertainty. Let's talk about causality next. Figuring out if A actually causes B, that seems fraught with uncertainty too. Extremely. He uses a simple analogy. Your car won't start. Is it the battery?
The starter motor, fuel, lack of pixie dust, multiple potential causes. And we have to be careful not to assume that just because B happened after A, A must have caused B. Exactly. The post hoc ergo proctor hoc fallacy. Correlation doesn't equal causation. He gives the example of hormone replacement therapy, HRT.
Observational studies initially suggested it protected against heart disease. But then randomized controlled trials showed something different. Yes. The trials, which are better at establishing causality, actually showed HRT could slightly increase heart disease risk in some groups. The observational studies were likely confounded by other factors. Women choosing HRT might have been healthier or wealthier to begin with. So establishing causation is tricky. Some links are clear, like smoking and lung cancer. Very clear, based on multiple lines of strong evidence.
But other classifications, like the IRRC's categories for carcinogens, need nuance. Saying processed meat is in the same category as plutonium, group 1 carcinogenic to humans, doesn't mean it's equally dangerous. It just means the evidence for carcinogenicity is considered equally strong. The risk level is vastly different.
How do scientists attribute things like climate change impacts? They use models. They run simulations of a climate with human greenhouse gas emissions and without them and see how likely specific events like heat waves are in each scenario. He mentions the UK's record September 2023 temperatures being made vastly more likely by human-caused climate change.
There's also the legal side, like the attributable fraction or the prosecutor's fallacy. Right. The prosecutor's fallacy is confusing the probability of the evidence given innocence with the probability of innocence given the evidence. A classic error seen tragically in cases like Sally Clark's involving misinterpreted death statistics. OK, so understanding the present and past is hard enough. What about predicting the future? Even harder. Spiegelhalter points out humans are generally terrible at it.
We have examples like Isaac Newton predicting the end of the world. Didn't quite pan out. No. But then you have scientific predictions like Halley's Comet, which was remarkably accurate based on understanding celestial mechanics. So there's a tension. Deterministic models versus real-world randomness. Exactly. Even a coin flip is deterministic physics, but predicting it is practically impossible. And the further out you try to predict, the harder it gets. Weather forecasts are okay for a few days, less so for weeks.
Climate projections deal with long term averages, not specific weather on a given day decades from now. Pandemic modeling showed this too, right? The model struggled partly because human behavior is so unpredictable. Hugely unpredictable. Economic forecasts are notoriously tricky too. The Bank of England uses fan charts for inflation forecasts, explicitly showing the widening range of uncertainty over time. What are personal predictions like life expectancy? Those are statistical averages for populations.
Your personal risk might be very different. People often misunderstand average risk. Ultimately, Spiegelhalter suggests focusing on flexibility and resilience might be more useful than striving for pinpoint predictions that are likely to be wrong anyway. That makes sense. What about those really big, impactful, but rare events, extreme risks? He starts with the story of the MV Derbyshire.
A huge ship lost likely due to a massive rogue wave in a typhoon. An extreme, unexpected event. How do statisticians even model things like that? They're outliers. They use something called extreme value theory, often involving specific statistical distributions like the generalized Pareto distribution, designed to model the tails, the rare events, rather than the central bulk of the data. But it's not just about the numbers, is it? How we perceive risk matters, too. Absolutely. There's the quantitative side, the statistics, but also the qualitative.
Psychological factors, cultural views, political influences. Our perception of risk doesn't always match the statistical probability. And standard models can fail us here. Like assuming things follow a normal bell curve. Yes, especially in complex systems like finance.
Many models used before the 2007 financial crisis assumed nice, well-behaved distributions, underestimating the possibility of extreme fat-tailed events where outliers are much more common than expected. He mentions FN curbs, plotting frequency versus number of fatalities for catastrophes. Yeah, it used to compare risks like plane crashes versus nuclear accidents.
but they rely heavily on assumptions and can give a false sense of precision about incredibly uncertain events. And communicating these risks is hard, that one-in-hundred-year event phrasing. Terribly misleading. People think it happens only once every century. Better to say 1% chance each year. Clear language is crucial. So what happens when we face uncertainty that's even deeper, where we can't even list the possibilities, let alone assign probabilities? That's what he calls deep uncertainty, or sometimes nice uncertainty.
Examples might include Malthus worrying about famine centuries ago, the specifics of the MV Derbyshire sinking, the Fukushima nuclear disaster, or even policy failures like the cash for ash scheme in Northern Ireland, where the potential outcomes weren't fully grasped. Are these like perfect storms or black swans? They can be. Black swans are unexpected events with major impacts, which are often rationalized in hindsight.
Deep uncertainty also encompasses meta-ignorance, not knowing what we don't know. Or even ontological uncertainty, where our fundamental understanding of the system might be flawed. How can we even begin to navigate that? He suggests a framework based on how well we can define outcomes and assign probabilities.
For deep uncertainty, where both are difficult, approaches like scenario planning become important, exploring a range of plausible futures without assigning precise likelihoods. And trying to avoid groupthink. Definitely. Encouraging diversity of thought.
Using red teams to challenge assumptions, even consulting science fiction writers to imagine unexpected possibilities. Being humble about the limits of our foresight. Given all these challenges, how can we communicate uncertainty effectively and ethically? This is critical. Spiegelhalter contrasts honest communication with deliberate manipulation downplaying uncertainty, like claims about WMDs in Iraq.
or exaggerating it, like the tobacco industry, sowing doubt about smoking risks or COVID disinformation. Trustworthiness is key, isn't it? More than just demanding trust. Exactly. Citing Onora O'Neill, trustworthiness comes from competence, honesty, and reliability. He mentions John Krebs' strategy during the mad cow disease crisis.
Be open. Acknowledge uncertainty. Explain the evidence. Suggest action. But don't pretend to have all the answers. Tailoring the message is important, too. Visual aids. Yes. Like the Bank of England's fan charts, visually showing uncertainty, using precise language where possible, like the IPCC's calibrated terms, or giving ranges instead of single numbers. Avoiding misleading formats, like the 1 in 100 risk. And framing matters. Relative versus absolute risk.
Hugely. Saying something doubles your risk, relative risk, sounds alarming. But if the baseline risk, absolute risk, is tiny, the actual increase might be negligible. He uses a funny example about binge watching TV, potentially increasing bowel cancer risk. So what are the core principles for trustworthy communication? He boils it down to about five things. Aim to inform, not persuade. Present balance. Acknowledging different perspectives.
be upfront about uncertainties and limitations. Acknowledge potential misunderstandings. Basically, treat your audience with respect. He points to the communication around the AstraZeneca vaccine risks as a good example of transparency, building in trust despite uncertainty. Okay, bringing it all together, how do we make decisions when faced with all this uncertainty? Well, rational choice theory, like the Ramsey framework, provides a logical structure.
Assess probabilities, consider utilities, values of outcomes, choose the option maximizing expected utility. But real life isn't always that neat. Rarely. We're not perfectly rational. We tend to be risk-averse when facing potential gains, but risk-seeking when facing potential losses, even if the odds are the same. Daniel Kahneman and Amos Traversky show this. Bernoulli's idea of utility, the subjective value of an outcome, helps explain this partly. We also suffer from ambiguity aversion.
We prefer known risks to unknown ones. And many big decisions involve that deep uncertainty we talked about. Right. Often decisions aren't one big choice, but a series of smaller ones. Spiegelhalter suggests four broad strategies, full quantitative analysis if possible, semi-quantified approaches using scores or rankings, relying on heuristics, rules of thumb, or story-based reasoning narratives and analogies. What about policy decisions? Cost-benefit analysis? That's common, like the UK Treasury's Green Book trying to put monetary values on outcomes.
But it raises ethical issues, like valuing statistical lives versus identifiable lives. We react more strongly to saving one named person than preventing many anonymous future deaths. Regulatory frameworks like the UKHSE's ALAR principle, as low as reasonably practicable, try to balance risk and cost. And sometimes being overly cautious can backfire. The precautionary principle. Yes, he cautions against applying it too rigidly.
Germany's rapid nuclear phase-out after Fukushima might have increased reliance on coal. Overreacting to trace amounts of acrylamide and burnt toast could distract from bigger health risks. It needs careful judgment. So looking ahead, what's the future of uncertainty? Well, Spiegelhalter reflects on the sheer improbability of our own existence, the anthropic principle.
We exist, so the conditions must have been right, however unlikely. People try to quantify existential risks now, too. Nuclear war, AI, pandemics. Yes, groups like Toby Ord's try to estimate these probabilities, but it's incredibly difficult dealing with unprecedented events and deep uncertainty. Even AI currently struggles to understand and communicate its own uncertainty reliably. So what's the final takeaway? How should we live with uncertainty? He offers a kind of manifesto.
Own your uncertainty. Express it clearly. Use it to make better decisions. Be skeptical of models that seem too certain. Build flexibility to handle extreme risks and ultimately embrace it. Uncertainty isn't just a problem to be solved. It's part of what makes life interesting, what drives discovery. It's not about eliminating uncertainty, which is impossible anyway. Exactly. It's about navigating it with more awareness, more skill, and maybe a bit more humility.
understanding chance, risk, and the limits of our knowledge. That feels like a really solid place to land. It certainly makes you think differently about everyday choices and big global issues. It does. Which leads us to a final thought for everyone listening. Go on. Now that you've taken this deep dive with us into the art of uncertainty,
how will you approach the uncertainties that lie on your own path? Hmm. What new questions about chance, risk, and the unknown might you be asking yourself now? Something to ponder. Indeed. Thank you for joining us for this really insightful exploration.