The plan for NHS league tables is bad, and will lead to unintended consequences
26:09 Share

This podcast is sponsored by Talkspace. You know when you're really stressed or not feeling so great about your life or about yourself? Talking to someone who understands can really help. But who is that person? How do you find them? Where do you even start? Talkspace. Talkspace makes it easy to get the support you need.

Talkspace is here for you.

Plus, Talkspace works with most major insurers, and most insured members have a $0 copay. No insurance? No problem. Now get $80 off of your first month with promo code SPACE80 when you go to Talkspace.com. Match with a licensed therapist today at Talkspace.com. Save $80 with code SPACE80 at Talkspace.com. ♪♪

Welcome to the Medicine and Science Podcast from the BMJ. I'm Cameron Abassi, Editor-in-Chief. This week we'll be hearing why Wes Streeting's plan for hospital league tables is a bad idea. So we don't do this in the Netherlands. Some papers have done it in the past, but as a government regulator, we stay away from this because it's dangerous.

But first, ketamine addiction. Certain high-profile individuals have brought chronic ketamine use into the news, but the rise in addiction amongst young people in the UK has caused concern for some time. Shivali Phulchand, one of the BMJ's clinical editors, finds out more. Hello, I'm Shivali Phulchand, editorials and education editor at the BMJ, and I'm here today with Dr Irene Guarini and Dr Nikki Cork.

Please could you both introduce yourselves? I'll go with Nikki first. Hi, I'm Nikki Kolk. I'm a consultant addiction psychiatrist. I work in a general hospital, King's College Hospital with the addictions care team. Hi, I'm Irene. I'm a consultant and addiction psychiatrist based in Greater London. I'm the lead on the Ketamine clinical project in my area.

So in your recent editorial, you write about the growing problem of non-prescribed ketamine use, which we've also heard more about in the news recently. So start with why is it a problem now? So ketamine, because it has use medically, but also industrially and as a veterinary product, it is very difficult to control the flows of ketamine, for example, around Europe.

And for that reason, ketamine is relatively easy to obtain and it's relatively cheap. So a gram of ketamine, which is what regular users might use on a using day, is about the same as a coffee, as buying a takeaway coffee. And the other thing about it is because it's not banned from being transported across borders because of its industrial use, it tends to be reasonable quality coffee.

so people can buy it and they know they're going to get a reliable high. Other things I've heard young people talk about is particularly if they've had a history of trauma that it can be quite numbing and can really just take a lot of anxiety and difficult feelings away. Obviously it also causes euphoria, that's one of the things that's reinforcing about it. And finally they're

Particularly in women, I have heard that one of the things that it does is it doesn't cause weight gain in the way that, so for example, cannabis does. And some of them prefer it for that reason. Thank you. That's really interesting to note. And

Perhaps it's also unique in that it's seen as a party drug, almost a psychedelic-like drug, and also a psychiatric drug that can be used therapeutically. And so that poses a challenge of when does ketamine use become a problem? Well, you have to think about in terms of dose as well and frequency of use. Obviously, when we talk about ketamine used for treatment-resistant depression, we are talking about

milligrams, so a very little amount. When we are dealing with chronic users of ketamine, their use is measuring grams. So there is a very big difference in dosage. And also what we see, which is quite common of other drugs as well, but also common in ketamine, you see a crescendo from

Recreational use, usually in the context of party drugs with other peers, and then in some individuals, then can evolve in more frequent use up to, as I say, daily use, which then it cause a little bit of problem. Ketamine has a significant impact on the developmental tolerance. So they have to increase the dose in order to get the same effect.

which is the acute effect is a dissociative psychedelic effect. But then in the chronic part of when they develop an addiction, then the chronic effect disappeared. And in order to have a kind of dissociative effect, they have to increase the dose a massive amount.

massively, which also poses a risk of the physical health because the more frequently you use ketamine and the higher is the dose, the impact on your physical health and mental health is greater.

So that does pose a lot of difficulties. And the tolerance is a phenomenon that we do see very frequently from the very beginning. So as soon as they start using more than a weekend, then the risk of developing tolerance is important, significant.

I completely agree with you, Irene. I think there are some overarching, I mean, we've mentioned that ketamine use disorder, there's sort of an agreed diagnostic criteria for it. But as addiction psychiatrists, we realize there are sort of overarching concepts that tell us when someone has a problem with something. So Irene's mentioned tolerance, and some people have distress when they stop using ketamine as well, sort of dysphoric withdrawal state.

The other symptoms of an addiction are things like loss of control over use. So either people make rules for themselves about how much they're going to use and find they can't keep to that, or they just keep using until they're out of money, or making the decision to stop and then finding that they're unable to stop.

So that's the other thing. And then the third thing is usually the sense that they become really preoccupied with obtaining, using and recovering from ketamine use. So it takes over pretty much all of their life. So we observe people not being able to cope at school, not being able to turn up to work, using despite having terrible physical consequences. And the other thing that we're noticing more and more is that people stop eating.

and their BMI drops really, really low. And it doesn't seem necessarily that they've got an eating disorder. It's just that they're so preoccupied by using ketamine and being in this dissociative state and that they forget to do something as basic as eating. Obviously, that's towards the severe end-effect spectrum. Also mentioned in the editorial are some of the more kind of significant effects that can happen from ketamine use. Would you be able to elaborate a bit more on those?

Well, the effects are mainly on physical effect and mental effect. So physically, when somebody is using acutely, as I said, you have like a

dissociative state which is characterized by hallucination, distortion of perception, you're having a phenomenon like auditory hallucinations, then the higher is the dose, then you can have a phenomenon called keyhole which is again part of the dissociation. So the feeling that they are running into a rabbit hole and with no end.

So this kind of acute effect. Obviously, the chronic effects are greater. They can have auditory hallucinations, which they do persist in time. They develop significant memory problems, mainly related to chronic use.

They develop weight loss, as Nikki said, and also they develop problem with the urinary system going from the kidney down to the bladder. And also there are some evidences as also the liver is impaired as well.

But what happens, for example, if somebody tries to stop? So a chronic user decides to stop ketamine all of a sudden, and they usually have these terrible abdominal cramps, which are described as incredibly intense and excruciating. And sometimes they are poorly controlled by conventional non-steroid painkillers or non-codeine or non-opiate-based painkillers.

which is one of the reasons why they frequently attend A&E. The cramps seem to be part of the kind of withdrawal type of syndrome because the pain is relieved by the use of ketamine. So if they use ketamine again, the pain goes. And at the end of the day, ketamine is used as a pain management pharmacological option.

in medicine. So this is a quite complex type of scenario here. So they use, they chronically use, they develop chronic complication. If they try to stop or reduce drastically the amount, they are unwell with excruciating pain, predominantly abdominal pain, gastric pain. They do have palpitation, quite unpleasant pain.

And obviously all of these symptoms are reinforced, they're used, and they are in a kind of vicious circle, which is some patients it's quite difficult to break. And as you mentioned, these patients are showing up in A&E and more frequently as we've seen. So what can a doctor in the emergency department do when they see these patients, both for their physical symptoms and also longer term?

I think there are a couple of things that would be really important for them to do. I mean, firstly, if someone, if a young person particularly is presenting with abdominal pain that they find it difficult to explain or any urinary symptoms, and this pertains to primary care as well, it's really important to take a drug history and to ask about ketamine use and frequency is the important thing because that's what relates to physical harm.

If someone does have urinary symptoms and they're using ketamine regularly, then it would be appropriate to refer them to urologists for investigation and treatment of ketamine bladder. It's also really important that they know how to access treatment for their drug use. Now in the UK, drug treatment is borough-based and most treatment is provided on a walk-in basis.

So any emergency department doctor can look up where the patient lives, can just Google that, plus community drug service and can give the patient details about how to access treatment. And there are really, really low barriers to access treatment. They don't need a referral or anything. So that's the message they need to get, that there's pain, the symptoms are not going to get better, unless they're able to address the ketamine dependence.

The other thing is, is that because ketamine can affect the liver, if you have someone who's come in who's using ketamine regularly, it's important just to check their liver function tests and to make a referral to the liver clinic if there's an elevation. And that editorial, Non-Prescribed Ketamine Use is Rising in the UK, is available on bmj.com.

In November last year, West Streeting, the UK's health and social care minister, announced that he was planning to introduce league tables for hospitals and he'd be linking managers' pay and continued employment to those outcomes. Rachel Hinton, the BMJ's analysis editor, finds out why that would lead to unintended consequences.

I think it's reasonable to say that this announcement about the rollout of lead tables in the NHS didn't go down so well in the quality improvement world. So much so that a group of authors have written an analysis piece in the BMJ on why this won't work. I'm joined by two of these authors, Richard Lilford and Tim Hofer. Richard, could I please ask you to introduce yourself?

My role is Professor of Public Health at the University of Birmingham, and I have a long interest in safety research and in trying to make NHS institutions safer. Thank you. And Tim? My name's Tim Hofer. I'm a Professor of Internal Medicine at the University of Michigan Medical School, and also a Health Service Professor.

health service delivery researcher with a sort of focus over my career of really looking at methods of measuring and monitoring quality in large health systems. Richard, I'll start with you. So isn't it reasonable that the Secretary of State for Health and Social Care, who has to answer to the country for hospital care, should want to hold these hospitals to account through measurement? What would you say to that? So I would say it's a truism in life.

that a problem does not necessarily mean that you have a solution. So in the case of measurement of hospital or assessment of hospital performance, it would clearly be convenient, nice, if there was one single measure which the Secretary of State or the government could use to monitor the quality of hospitals.

The inconvenient truth, however, is that such a measure does not exist. And because it doesn't exist, insistence that hospitals are assessed by one simple measure will do more harm than good. It'll do more harm than good to patient care, and it is immoral.

And so rather than start with the problem, we should start with the solution. What are the things we can measure and how reliable are they, how valid, and work from there to how the monitoring should work rather than from the presumption that there has to be a measure because we'd like there to be a measure. Before I move on, what do you mean by could do more harm than good?

Yes. So it's a sort of a well-established tenet of economic theory that when you give a person a strong incentive, but not the means to reach that incentive by actually improving their care or quality, that when you do that, you introduce gaming.

With gaming, you get perverse incentives. You get situations such as ambulances waiting outside the hospital, being forced to wait outside the hospital, so that the hospital does not breach its four-hour waiting target. These incentives are very strong, and therefore they are like a surgical operation that is not well indicated. It can do considerable harm.

Tim, you're in the US and, you know, hospital league tables are created by, you know, a range of organizations, one of them in particular called the Leapfrog Group. How do these go down there? I mean, it's not just Leapfrog. There's a number of organizations that have gotten into it. It's, you know, I mean, clearly at some level, this is a, they see this as a

service that they can get paid for doing. And so they, you know, which is not to sort of totally belittle the rationale for trying to do this thing, which Richard's already covered.

So, you know, we have U.S. News and World Report, have hospital rankings, and, you know, they do have a big effect. We get, you know, it gets discussed and a lot of executive time is spent on thinking about really how to manage these ratings. There's certainly nobody who's there saying that these things actually measure what we want to measure very well, but they say that this is a political reality that we have to manage these ratings. So to the extent that people are spending time doing that, they're not...

not spending time doing things that might actually improve care. And that is a cost. Listening to this conversation is Ian Leistikoff, who's an inspector at the Dutch Health and Youth Care Inspectorate and also on the advisory committee of the international forum that the BMJ runs with IHI.

So, Ian, welcome to the podcast. What do you think about what you've been hearing? Yeah, thank you, Rachel. So my views are from like two perspectives, one being an inspector. So the Dutch Health and Youth Care Inspectorate is kind of like the CQC and the GMC together. So we inspect both organizations and professionals. And I work at the Erasmus School of Health Policy and Management, the Erasmus University, as a professor on governmental regulation of health and care quality.

So I've been listening to this with fascination and I'll be a little bit Dutch, if you will excuse me. So a little bit more blunt. And I would say that performance assessments with punishments are unwise, outdated and unrealistic. I think it's a really, really bad idea.

And it's a dangerous idea. I think it would be it's unwise because it's demoralizing at best. So basically, if my boss came to me and said, you know, Ian, I'm going to pay you more if you do a better job. Then basically she's saying I'm not doing a good job. It will lead to cover up behavior because if I can't do it properly, I'll try to look like I'm doing it properly.

And if you give financial rewards, it can actually corrupt leadership because then my focus will be on my own personal financial reward instead of the societal goal that I've been paid to help with. So that's why I think it's unwise. I think it's outdated, if I may, because the whole carrot and stick presumes that all will be well if people just behave the way that they should.

And that is kind of a hundred years ago safety thinking, whilst now we are much more aware of the whole systems thinking and how an individual on his or her own can't have much effect if the system in which that person works doesn't make it possible to do their work properly.

And I think it's unrealistic because as Richard and Tim said before, you can't score a whole hospital with one measure. There's so many things happening in a hospital. And if I have rheumatoid arthritis, I would want to know how the rheumatology department works and not how the obstetric department works. So those are all these different facets that you just can't catch in one measure. So we don't do this in the Netherlands. Some

Papers have done it in the past, but as a government regulator, we stay away from this because it's dangerous. How does accountability work in the Netherlands? Yeah, I think accountability is very important. So, I mean, I'm an inspector, so clearly I think some form of governmental regulation is important. So we have a system of quality indicators here.

which you could say are performance assessment measures, but we call them indicators because the measure is an indicator. It's the start of a conversation. It's not an assessment. And some of these indicators sometimes go wrong. So I'll give you an example because I think that's illustrative. So we had an indicator some time ago. So we were talking about the safety of a colorectal surgery.

And we have these discussions with the Dutch Society for Surgery, and they said, well, maybe we should look at unscheduled re-interventions after resection of a primary colorectal carcinoma. Because this can lead to serious complications like anastomotic leakage and people can die. And that could be an indicator for the quality of care. So this was a public indicator. This was a national indicator which was made public.

And at some point we looked at after a year, we looked at the data and we saw that one hospital had a huge decrease in endostomal leakage, but an increase of stomas. So the 100% way to prevent endostomal leakage and prevent the chance of unscheduled re-intervention is not to perform a suture, but to make a stoma.

So basically, this hospital was creating more stomas than probably needed, that they probably needed to, as a perverse incentive. And you must imagine this system, we didn't pay people for better performance. We didn't punish them for bad performance. But still, this created a perverse incentive that luckily we saw after a year.

Because we were talking, the inspectorate was collaborating with the Dutch Society for Surgery to help us understand what was going on. So we changed that indicator directly to an indicator about failure to rescue. So I'm just saying it is so dangerous. As soon as you if you put an indicator out there and say this is what we're going to score you on, there is going to be gaming. It's just going to happen.

Yeah, I mean, the ultimate challenge in designing a performance measure is that you want to design one where it's easier to do the right thing than to game the measure. And of course, that's a really hard thing to do. Maybe even not that possible in many cases. But that would be the optimum measure. And it sort of implies that

it would require a lot of system changes such that the provider just finds it so much easier to do the right thing that they do it because the system has made it really easy for them to do. Because we know that most providers are trying to do the right thing. They just are overwhelmed. They have tons of things to do. They can barely do the things they need to do now. And so if you want to ask them to do something better, you kind of have to give them the supports to do it better. And that would be the challenge to designate

designers and say, well, you know, you can put out a measure, but you have to make it easier for the provider to do the right thing than to game it. Richard, we started with you talking about, you know, the problems with measurement. And it sounds like from these conversations that if Wes Streeting does go ahead with league tables, all he'll actually be doing is annoying clinicians and potentially causing hospital scandals down the road. Is that right?

We think that's exactly right. When I came to this country from my country of origin, I was amazed at the autonomy and power of my clinical colleagues. And they just were a law unto themselves. But during my sojourn here, I've witnessed the pendulum swing to complete the other position.

where they now have very little autonomy, are at war with the management of their hospitals who are trying to carry out the government's business about meeting targets for everything from cancer care to the overall mortality rate of the hospital and so on.

They used to work in teams. That's been completely taken away from them. And my opinion is that the – and heroism, you're not allowed to have a hint of heroism anymore, except perhaps a little bit during COVID. So I think that the pendulum has swung far too much the other way and that the government should stop pretending it's got a kind of control, you know, which the founder of the NHS –

Nora and Bevan knew only too well it did not. And just loosen up a bit. You know, praise doctors. Don't necessarily pay them anymore. They're quite well paid by international comparison. But give us praise. Give us more control of our destinies and our services.

And take the heat off with all these endless targets and so on. I've studied a large number of targets now. Many of my studies published in BMJ Quality and Safety, they all produce a threshold effect, which is a sign of gaming. All of them do.

So let's stop all this gaming. Patients are treated, you know, cancer patients are treated according to whether they can get the hospital to just meet its target rather than on clinical need. It's there in the data. Let's try something else now, a little bit more of a supportive management style. And that analysis about hospital league tables, targets and performance incentives is online now.

That's it for this episode. We'll be back next time with a new investigation into a commonly used cardiology drug, but you'll have to wait and see what that's all about. Until then, I'm Cameron Abassi. Thanks for listening.

This podcast is sponsored by Talkspace. You know when you're really stressed or not feeling so great about your life or about yourself? Talking to someone who understands can really help. But who is that person? How do you find them? Where do you even start? Talkspace. Talkspace makes it easy to get the support you need.

Talkspace is here for you.

Plus, Talkspace works with most major insurers, and most insured members have a $0 copay. No insurance? No problem. Now get $80 off of your first month with promo code SPACE80 when you go to Talkspace.com. Match with a licensed therapist today at Talkspace.com. Save $80 with code SPACE80 at Talkspace.com.