AI for COVID-19 with Professor Matthew Lungren
34:12 Share

Welcome to Skynet Today's Let's Talk AI podcast, where you can hear from AI researchers about what's actually going on with AI and what is just clickbait headlines. On this special interview episode, you'll get to hear from an expert on the intersection of AI and medicine about

the many ways AI relates to the current coronavirus crisis, and about other questions and trends at the intersection of AI and medicine. I'm Sharon, a third-year PhD student in the machine learning group working with Andrew Ng. I do research on generative models, improving generalization of neural networks, and applying machine learning to healthcare and tackling the climate crisis. And with me for this interview is Professor Matt Lungren, one of our fantastic collaborators in the lab.

Matt is a faculty at Stanford, a co-director at the Stanford Center for Artificial Intelligence in Medicine and Imaging, or AMI, an interventional radiologist at Stanford University School of Medicine, and is on the machine learning committee at the Radiological Society of North America, or RSNA.

Thanks for making the time for this, Matt. I imagine it must be a hectic time to some extent. How have the last few weeks been going for you as the coronavirus crisis has really hit here in the U.S.? Yeah. Hi. Thanks. And thanks for having me. Yeah. You know, it has been a crazy few weeks. It's been pretty much nonstop. I will say that both as sort of a radiologist on the front lines in the procedural world, dealing with these patients, learning

practically every day more and more about this disease and what we are to do on the front lines to treat and manage these patients, but also to learn about how we can potentially apply some of our research and data science techniques to help make better decisions and potentially forecast what we're going to be expecting in the days and weeks to come.

Yeah, definitely. So given how it's very much what most of us are thinking about these days for this podcast, we'll start off by focusing on COVID-19 and then later step back to discuss less topical things.

So two weeks ago, there was an article called Can AI Diagnose COVID-19 on CT Scans? Can Humans? It concludes with, quote, AI tools built to diagnose COVID-19 infection need to be evaluated robustly since current radiological knowledge does not have specific features that can help identify COVID-19 infection on CT scans.

That said, AI tools have a vital role to play in quantifying disease progression and possibly predicting disease outcomes. So Matt, what are your thoughts on AI's role in diagnosis and other aspects of the response to coronavirus? Yeah, this is a very, very interesting article. And there have been, of course, many since. One of my colleagues kind of quipped that all

of the hundreds of emails a day about AI have switched to COVID emails. And it's kind of interesting that we still find ourselves at the center of a lot of this. But, you know, from our perspective, I think that there are several great opportunities and even a couple pitfalls that I think as a community, we have to be careful about. I think, you know, starting out, I think that, you know, as this crisis is sort of developing, we've seen groups that have

rapidly been overwhelmed by these cases and to the point where even the point of care testing, as we've seen in Italy and parts of China in certain circumstances, has really hampered their ability to identify these patients and properly triage them. And so in those settings, imaging actually played a vital role. And there have been a couple of great papers that have described opportunities for quick screening and simply triaging for patients who are coming and presenting with respiratory illness.

But looking more long term and where AI might fit in, I think that for the most part, it's kind of one of those questions where AI may be able to do something in terms of identifying these patients and potentially even quantifying the disease. But as many articles have pointed out, and certainly most medical societies have tried to emphasize, it's not an ideal screening test. And I think it's although tempting to potentially use it for that purpose, and then certainly

consider injecting AI into that screening process, it actually could potentially cause more harm than good. And let me explain why. I think that for the most part, not only do you have patients that are highly contagious and have the opportunity then as you bring them into the healthcare system to image them potentially infecting staff and other frontline care workers,

only to find that they have a disease for which you have a high pretest probability and likely have other ways to diagnose. It doesn't quite make sense and potentially could, as I mentioned, cause harm. The other issue is that even though there are some findings on imaging that are very specific for COVID-19 in the proper pretest grouping, if you imagine now take away the pandemic and we're in an endemic situation where we have this disease circulating, but also many other

common diseases that we typically see on imaging, then you have a situation where you're potentially over-diagnosing COVID-19, for example, particularly if you're training models to recognize what are, you know, in conglomerate, a very general condition.

group of findings. And so it can be challenging to sort of find where AI could fit in here. But in my view, there are some potential big opportunities for model development. I think the first I think will be as part of, as you sort of pointed out from that article, is trying to understand in conjunction with the clinical data, who is going to need an escalated level of care and who will not.

It's not entirely clear that we have a grip on this, both from the clinical front lines and certainly from the research perspective. And we're still learning about how to determine who is going to do well, that you can manage them at home safely. And then those that need to be hospitalized and potentially intubated on a mechanical ventilator to get them through the disease. And so that is...

is a place where I believe imaging, but also imaging in addition to clinical data could potentially play a huge role. And AI, of course, if you can come up with quantification of the disease or even a prediction of the outcome, could even potentially provide better clinical decision support from that perspective. So that's a really big opportunity.

I think another one that I don't see a lot of folks talking about, but I do think will be important. We have to remember that this pandemic and what we're calling the surge will eventually sort of and thankfully fade away into sort of a new environment of we have to live with this disease, at least until we have a greater herd immunity and and have the opportunity.

availability of vaccines that are effective. And so until that time, which again could be a year, maybe even two years from now, we have to figure out a way to identify these patients among patients that might be presenting with heart failure or the flu or bacterial pneumonia, atypical pneumonias. There's all kinds of other respiratory diseases that we're going to need to be better at identifying these patients and certainly triaging them out.

in a more usual population. And that is going to become particularly important next fall as we talk to many different epidemiologists and scientists who are telling us that this is likely to have a second peak to come in the fall. And hopefully we'll be ready, certainly more ready than we have been

during this endemic. But I do believe that this will be an opportunity to, uh, to use machine learning, to try to be more intelligent about our differential diagnosis, uh, for findings that may or may not be very specific, but also what we've been finding, which is very unique about this disease and something that I really want, um,

everyone to sort of think about is that we've seen such a high percentage of patients who have apparently been able to spread this disease or what we call viral shedding, even though they're asymptomatic. And that's a very unique hallmark of this particular disease and why it's been so difficult to control and to contain. And imaging actually plays, in that case, an incredibly important role.

And if you can develop an AI model, for example, that has the ability to flag imaging examinations on patients who are not being evaluated for respiratory distress, but instead being evaluated for other problems. You can imagine someone showing up after a car accident

and getting a chest CT. And incidentally, they also have findings of COVID-19 lung infection or someone coming in for a lung nodule or a lung cancer follow up. And they also happen to have findings for COVID-19. Now, this is important because number one, if you're asymptomatic, you're potentially exposing people around you

And we would like to be able to continue to control these outbreaks from a public health perspective. So an AI model that's running in the background on any imaging tests that may image those portions of the lung and have findings that could be sensitive for COVID-19 could potentially identify patients, even potentially alerting public health authorities of outbreaks, but certainly not allowing patients

outbreak to continue, you know, decontaminating the scanner, identifying the patients and certainly doing some contact tracing would be important follow-ups there. So those are just a couple examples, I think, in my mind of where I think the intersection of AI and healthcare, particularly as this crisis is kind of evolving, I think could play a big role. Wow. Thank you so much, Matt.

I think one thing that you've really stressed in there is how AI can be helpful in ways where it would do less harm, for example, in triaging and not necessarily direct diagnosis, but that kind of background in direct diagnosis for patients who come in for other things, not necessarily for COVID-19 that doctors are already quite equipped to handle.

focus on in terms of finding that signature in the image. Could you unpack a little bit how AI would be enabling less harm than diagnosis when it comes to helping with triaging?

Yeah, I think with triaging, if you were, let's say, to just simply deploy a model that was trained to recognize COVID, I think that, again, in this population, you might find that it's fairly useful in the sense that you can certainly, you know, again, identify the hallmark symptoms and as we talked about, maybe have a, you know, quantification measure that would be helpful clinically. But then imagine if your pre-test probability starts to change, it's

but your model hasn't. And, um, and then you can have patients coming in with, you know, for example, heart failure, uh, coming in with, you know, viral pneumonias of other etiologies or even bacterial pneumonia. And if we're relying on imaging alone, which again, I don't anticipate would ever be the case, but, but just playing out that scenario, like, you know, why not just take a model and start deploying it and using it for diagnosis on imaging and screening? Uh, again, the, the dangers there I think would be both in the sense that, um, you know, you would have, uh,

misapplication of this diagnosis to patients that need to be treated in other ways. But second, you know, public health actually has quite a bit of leeway to, at least in this country, to take away your freedom. And imagine that you're coming in with an exacerbation of

of congestive heart failure, you don't have COVID-19. But some of the imaging findings on the lungs might be similar enough for a machine learning model to misdiagnose that. And so if you're relying on simply that model to make that diagnosis on that single imaging examinations, you can actually quarantine that person, right? For 14 days, you might subject them to further testing that maybe they wouldn't necessarily need.

And you could also, of course, be alerting and maybe potentially alarming their contacts. And so, you know, there are certainly places where this application is not a one size fits all. And it definitely would require sort of synthesis of other information. That's where back to the concept of, you know, where can I really help in health care and how can I really have a bigger impact?

What we've seen in these early efforts, which have been great, is really just sort of taking a data set and training a model to perform a relatively narrow task at very high levels, and particularly as we benchmark against human experts. And that's terrific.

But remember that the human task at its core, particularly in medicine, and no matter which specialty you're in, still requires quite a bit of context. I mean, the 10 to 12 years of medical training leading up to being able to do this independently

isn't just about learning how to read an image, for example, or make a certain diagnosis. It's about understanding the whole patient, right, and the clinical context. And that's where I think there's some opportunities for some research I've seen in sort of multimodal and fusion modalities where you can potentially take, you know, other data like structured data or even unstructured text data and sort of bring that into the model along with the pixel data. And then you can maybe have a better, maybe even a holistic picture of

of what's happening in healthcare applications for a given patient, maybe making better diagnoses and certainly having a better chance of being clinically meaningful.

Yeah, so it sounds like there are many ways where AI could do much more than just imaging and come in and really integrate that information. It also sounds like that there is quite a bit of harm that could be done not only when there is a false positive, which we are probably aware of what the harm would be there, but also a false negative also.

Shifting a little bit away from that article, which, by the way, to listeners, was linked on the page of COVID-19 and imaging AI resources put out by Amy, the Stanford Center for Artificial Intelligence in Medicine and Imaging, of which Matt is the co-director.

So RSNA, the Radiological Society of North America, announced just a week ago that it's launching a COVID-19 imaging data repository in a bid to boost research on the novel disease. Matt, could you tell us a bit more about these efforts?

Absolutely. Yeah, this is something that I'm very, very passionate about, I think, in general, certainly as part of the Amy Center mission, but just more broadly in the research and education communities. I think that one thing that healthcare has not done as well as those in the computer science world is share.

Um, certainly we share results of, of things that we discover and certainly things that we work on, but, but we're not as, um, we're not as used to sharing, um, our models and, and our data sets. And, and obviously some of that has to do with, with privacy and other considerations. But, uh, as we've learned more, particularly in the medical imaging world, um,

We've realized that without the ability to share data freely across institutions and collaborations, we actually hurt ourselves in particularly trying to develop clinically useful models in our practices. And so as this COVID pandemic began to come forward, I think that it really provided an opportunity for alignment of interests across many different groups, the Radiologic Society of North America, of course, being the

the largest, but our partners in Europe, partners in Australia and Asia, and even our other cousin organizations in radiology have all really gotten together and said, how can we work together with all of our collective resources, partnering with both commercial and government entities to really make a resource available to anyone out there who both feels that they'd like to share data

more broadly, but also access data in order to learn more about this disease and potentially develop, in this case, models that might be clinically useful. It's very similar, I think, to the sort of effort that came on early in the pandemic, where researchers immediately sequenced the genome of COVID-19, the viral genome, and then released it, you know, almost as fast as

And because of that effort, we're we're months, if not years ahead of the normal timeline for vaccine development. And there are groups all over the world working independently and together, you know, alternating sort of discoveries and figuring out pathways to to create vaccines. And that is sort of the same spirit that we would like to bring from the medical imaging community. We recognize that.

The imaging findings are going to be at least in the discussion for the foreseeable future. And certainly imaging will play a role in the management of this disease going forward. And so we can't continue to have small data sets that are independently researched and have data.

the opportunity for us as a community to develop new approaches or have conclusions that actually are clinically relevant. So this effort is not just about creating this data set, but it's also about aligning our annotation efforts so that we don't have a Tower of Babel

as typically can happen when many different groups are attempting to work on the same problem. And so if we can achieve, and again, I think this is going to be potentially one of the biggest silver linings out of this current crisis, is that we've never seen groups work so quickly and so easily together. And in just about a week and a half, we have almost 100 different groups who have pledged to make data available. We have

partnered throughout Europe, and I mentioned partners in Asia and other groups, all working together right now on creating the annotation sort of rulebook or data dictionary, coming up with clinical data points that would be important for this data set to include. And the gathering of data to make it available completely open and free is rapidly progressing. And so

I hope that this spirit of collaboration and open source data will hopefully continue in the future where we see these groups recognizing now the benefits of having this available to everyone so that we sort of, in the sense of the public good, we're all working together to make this work. And so, again, I'm confident that this...

that this effort will lead to something that I think will stand as a model for data set sharing in the future.

I hope so too. It sounds like COVID-19 is in some sense a catalyst for this quote unquote image net for medicine. Could you perhaps give some motivations as to why medical data is particularly challenging compared to, let's say, open sourcing and image net of perception data to our listeners and also why this is very difficult to do rapidly, especially now?

Absolutely. Yeah. And I do like the comment you made about a medical image net, because I think that's a wonderful analogy for, for what we're trying to accomplish. And, you know, I think that before image net was a thing, um, it was difficult to, uh, at least the story goes, it was difficult to convince people how important data was, uh, in, in sort of the model and AI communities, um, towards the process of, of making useful models. And I think that that conversation has sort of been answered, um,

or that question has been answered by, by the success and the subsequent breakthroughs because of the availability of that data in medicine. It's, it's very tricky. I think that, um, you know, historically, um, obviously privacy is a huge concern and we share that concern. We certainly don't want to, uh, violate anyone's privacy, particularly as it comes to, uh,

to medicine, uh, and medical diagnoses, uh, in the U S in particular, we have a strange healthcare system with a lot of different stakeholders, some of which are not always aligned. Um, and you can imagine if your, uh, diagnosis is, uh, is somehow made available to an open source group and you're identified, uh,

maybe you won't be able to get insurance in the future. Maybe you'll have difficulties, you know, getting a job or whatever. And we, we certainly recognize those sensitivities. And that's why I think in, in, you know, again, why there's laws and other things that really clamp down on, on any sort of effort that would, that would potentially violate medical privacy. At the same time, I think that patients are increasingly comfortable with, with the concept that, um, in the right hands, uh, for, for research and education use, et cetera, uh,

they're very comfortable with making data available for advancing science and learning more about disease. I am frequently in touch with patients for different studies at Stanford and other institutions where they're freely offering up data. They are volunteering to take home wearables, for example, and make that data completely available and, of course, used for research. And so I think that there is definitely a spirit of interest from the patient side.

And I think that part of that is related to trust. We don't sell their data. We make that very clear. We don't use their data for other profit purposes beyond its primary use. And then when we do make these data sets available, they're not for commercial use. They're not meant for that purpose. And I think that those kinds of statements and stances, at least from our perspective, have really made it possible for us

that trust with our patients and others, other institutions who are interested in doing this. And so, and by the way, it is, it is quite easy, at least in the medical imaging space to make this data available and to do it safely. There are regulatory hurdles and there are certainly various nuances regarding how to de-identify data, but it is definitely doable.

and medical imaging, particularly in the radiology space, has been a strictly digital enterprise for more than a decade and a half. And because of that, we really are in a wonderful position to make large volumes of imaging data available

to a community like the computer science community in a very similar way to ImageNet. And I think that we'll see even similar breakthroughs. I mean, the work that we've seen happen in this field so far, I think has been astounding and has really moved the ball forward. But

At the same time, we know that we can do much more. And the key to doing more, just as we saw with the initial work with ImageNet, is to make more data available so that models that are developed on it can be more generalizable. We can have new and fascinating insights and potentially harmonize across large other efforts that seek to do the same thing.

I think that, you know, as we've again, as we've built trust, as we've made it clear that we're interested in research and education, sharing medical imaging data with the public good in mind, obviously in a responsible way, has really has really been something that that we found to be the best strategy going forward.

Yeah, it sounds like norms are shifting not only from physicians' or researchers' perspectives, but also patients and the public and their level of comfort with sharing data, rather, and understanding that it's becoming increasingly useful. And so I think you've touched on this a little bit, but more formally, in addition to the data repository, RSNA also released a new survey to gather radiology business leaders' thoughts and level of interest in data sharing during the pandemic and

revealed after that that it has agreed with the European Society of Medical Imaging Informatics to share that information as part of its COVID-19 AI initiative. So hopefully norms are shifting. And so moving gears a little bit,

As we see this exponential curve of growth in COVID cases, as well as, unfortunately, the death rate as well. We also see, and I saw this on Twitter somewhere, an exponential curve of growth in academic papers in this space as well. Where...

Matt, do you think recent advances can make the largest impact? A lot of people say, you know, urgent times like COVID-19 right now are the best times to apply AI where we do have limited medical bandwidth. Also, areas generally speaking with limited medical bandwidth, for example, develop

in countries without access to care? Yeah, that's a great question. And, you know, I think that we have a tendency, those of us who are kind of sequestered in our own fields to, you know, everything looks like a nail when you have this AI hammer, right? So certainly we think about, oh, well, how can we,

how can we be a part of this effort, uh, to, to help, you know, treat this disease, to help predict the next outbreak, um, to come up with the next therapy. And of course, if you ask someone in the research, they're going to say, well, I have an AI tool that can maybe do that. But in fairness, I think that the fact is, is that this, um, pandemic, um, has generated large amounts of data. Uh, that data has probably, um, at this point saved many lives. Um,

If you look at countries like Taiwan and Hong Kong, Singapore and even South Korea early on in the pandemic, because of the opportunity to have access to data with a well-reasoned plan, they were able to sort of predict outbreaks and sort of mitigate some of the some of the struggles that I think others found themselves in. And even with the pandemic.

that we've seen this pandemic cause to different healthcare systems. I think that the, again, the massive amount of data, the research that's being published, the release of information is so critically important in a disease that we've just never had

had to fight before and we aren't familiar with. I didn't learn about COVID-19 in medical school. No one did. And so we're all learning together. And I think that the, you know, one of the reasons why I think those of us in data science and AI are so excited about an opportunity to have an impact here is that we're seeing the data really can move the needle

in saving lives. And, you know, if I had to choose an area to your question where, you know, AI is going to have the biggest impact, you know, I'd love to say it was medical imaging or even clinical data prediction, but I don't actually think that's going to be where the ultimate win will be. I do feel that some big opportunities, I think, are being explored with understanding the structure of

of the COVID-19 genome and protein structure. And, and it's tempting to find drugs and, and even vaccines, uh, that can effectively treat it. I think that we, we may be able to treat these patients, uh, more effectively with data and better decisions. I think that we can diagnose and identify them with, with AI tools and things that we're developing. But, um, but I think that some of the applications where you're able to screen infinitely large numbers of potential drugs, uh,

and, and, um, and treatment sites. I think that, that is an application where, uh, the superhuman quote unquote component of, of some of the AI applications are really are going to pay off. Um, and I, and I sincerely hope that it does, um, because, you know, right now we're still fighting, uh, with, without, um, without the kind of, uh,

arsenal that we need to really treat this disease. Yeah, that's fantastic. Basically, understanding the structure of COVID-19, drug discovery, and given the fact that COVID-19 is open source, the genome is open source. I've actually had a couple of friends put out papers because they saw that open source structure, which I found very exciting. I almost thought that they...

I thought they were working with the actual virus at first. So that does point to one thing. So many of our listeners do want to help, and perhaps that is one way to help to start reading those papers and understanding what's going on and what's at the front lines in terms of academic problems within genomics and how AI could help.

But many of our listeners are also not only in AI, but in tech at large. What are ways that you think they could help in general? I know we've chatted about this a little bit before, but would be curious to hear your thoughts listed out here.

No, absolutely. I have been incredibly impressed with my connections with tech and obviously living here in Silicon Valley. I think that maybe at the end of all this, we will have an opportunity to thank everyone.

uh, our colleagues in tech and, and some of the things that I've already seen happen have been absolutely tremendous and maybe not even recognized yet for, for how much of an impact they've had. And the first thing I wanted to say is that, you know, for a large group of highly educated folks that, that work in this area in the tech industry to make the decision prior to any medical, uh, uh,

you know, mandate to shelter in place, to work from home 10 to 15 days before the county did in the Silicon Valley area, San Francisco Bay Area. That saved lives. We know that that saved lives. That's tens of thousands of people that were no longer, you know, in contact with one another. They're working from home.

All of those contact points help flatten the curve, as we say here and as we're all familiar with now. And I think that that is the first thing I point to as where our tech colleagues have really saved lives and helped all of us on the medical side.

I think that some of the other areas that I've seen that have been just absolutely fantastic, um, include, um, large scale surveillance and send in the sense of understanding where folks are searching for, you know, flu like symptoms. Some of those applications I've found to be very, very useful. Uh, there have been some groups, um, uh,

Ericsson, I believe in Slack and put together a HIPAA compliance, uh, channel that verifies that you're a clinician, uh, with, with your national credentials as well as personal information in order to get you in to a system where you can have free flowing communication, uh, with medical colleagues all across the country. Um,

that has been tremendously helpful. We are all learning on our own in, on the front lines about this disease and how best to treat our patients. What we don't want to do is continue to reinvent the wheel in multiple places. And so tech's ability to connect people in a way that's never been possible before has been, uh,

It's just been phenomenal. And we're all learning from one another, again, in ways that we weren't ever able to do before. And it's possible because, you know, tech has reached out. I think that some of the other things that we've seen is a tremendous spirit of of how can I help and providing resources, providing, you know, storage resources, providing Internet and communication services.

for frontline workers, even being able to, uh, as we've seen with some of the larger groups, um, make PPE available through large connections, help create, um, the, the, uh, the connections in order to make, uh, different resources available to, to, uh, hospitals that may not have had access to that before. Um,

there's a, there's a tremendous spirit of, of, um, of collaboration and willingness to help. And I think that we've seen that already take place for those out there who are, you know, sheltering in place that maybe they're a little bit bored, uh, and they're really looking to roll up their sleeves and get involved. I think that there are still a phenomenal number of opportunities to do so. I think that I would point them certainly to their own internal groups and, and, uh, in corporate, uh,

efforts and initiatives, simple things like obviously giving blood and donating PPE and things like that are on the table. But bringing technical and data expertise to these large amounts of data that are being publicly released, I think is an opportunity. We've seen that obviously the group of Johns Hopkins has made practically every COVID case that's publicly available open. But

But, you know, similar efforts at UCSF, similar efforts at Stanford, similar efforts around the world. All that data is possibly, you know, a source of new information that we can learn about. And I think we're seeing groups being able to very accurately predict things like when do we expect the next pandemic?

a surge of patients? When should we expect the doubling time? How do we prepare our ICUs for having enough, you know, equipment and ventilators? Um, these are all areas where, um, the data sciences crowdsourcing community have really made a huge difference sharing, just sharing packages or sharing code that you've written that may make a, make a someone's life a lot easier. Uh,

These are all things that are happening right now that are really, you know, again, making a tremendous difference for all of us. And I, you know, now I'll end with just the, you know, there's the group out there that I think works in the in the, you know, the virtual world. But then there's also the group of engineers that we've seen doing doing very, very helpful things. I mean, they 3D printed face shields for us. They have.

uh, you know, come up with new and innovative ways to ventilate patients that, you know, decades of crusty old medical research has not been even remotely looked at. And now they're getting dusted off and people are saying, Hey, why can't we do it this way? Uh, with some of our modern technologies and techniques. And it's just, it's just incredible. I haven't seen anything more inspiring than, than the way that the tech community at large has really gone hand in hand with the medical community, uh,

and really helped us do our job. Wow, that is a very humble opinion.

I'm glad we're helping in the tech community. And I hope we continue to help and also be equally, if not more, as humble. This has been extremely informative and helpful, Matt. Thank you so much for taking your time to lend your insight and expertise at the intersection of AI and medicine, especially in light of the COVID-19 pandemic. Your insight and expertise at the intersection of AI and medicine, especially in light of

the COVID-19 pandemic. So thank you so much for you can find the articles we discussed and Professor Matt Lundgren's profile as well as subscribe to our weekly newsletter with similar ones at skynettoday.com. Subscribe to us wherever you get your podcasts and don't forget to leave us a rating if you like the show or if you like Matt. Be sure to tune in next week.