DeepMind‘s Rain Nowcasting, OpenAI‘s Book Summarization, Distracted Tesla Drivers, Amazon‘s $1k robot
30:00 Share

Hello and welcome to this kind of day's last week in AI podcasts. We can hear AI researchers chat about what's going on with AI. As usual in this episode, we will provide summaries and discussion of some of last week's most interesting AI news. I am Andrey Kronikov. And I am Dr. Sharon Zhou and welcome back. We've been on hiatus for a couple of weeks.

Lots has happened and also lots has not happened. But for a bit of personal news, an AI got married to a robot. So I actually got married. I'm the AI in that scenario.

So that's a bit of personal news. But we are back on track. And Andre was there, of course. Yeah, I got to visit. I got to go back to the Bay Area for a bit. So we have a good excuse for being off for a couple of weeks. But you're back and we should be doing our regular podcast again soon.

This week, we'll discuss some news about AI for medicine and image editing, some new research from DeepMind and OpenAI, AI's Islamophobia problem, some stuff about Tesla, and Amazon's new $1,000 robot. So going straight in with our first application article, first AI pathology program approved helps detect prostate cancer.

So the US Food and Drug Administration approved the use of a software product to assist pathologists in the detection of prostate cancer. It's called PAGE Prostate and it helps with identifying an area of interest on a prostate biopsy image with the highest likelihood of having a cancer so it can be further reviewed by an actual human pathologist.

And VFDA reviewed the device with a pre-market review pathway and actually granted authorization of the software to Page.ai. And apparently it's the first one of these sorts of authorizations.

This is huge. I have worked on pathology on these whole slide images, not exactly for prostate cancer, but for another task of detecting H. pylori, which is a downstream risk for stomach cancer. And here, the FDA clinical study had 16 pathologists examine 5

slide images of prostate biopsies of which about a quarter or a third were cancerous and the others were benign and these were all digitized by a scanner and I think this is huge for a lot of different reasons so one

In the pathology world, they actually still use microscopes a lot of the time and don't use scanners. And so this is actually going to start making a push for people to use scanners and scanning these biopsies for the AI to help the pathologists actually get the diagnosis. Another big thing was that

They found that the study did not impact the pathologist reading on benign slides, but really did assist them on the detection of cancer on individual slide images by 7.3% on average. Okay.

And so, yeah, this is huge. I'd be curious to see, you know, exactly what the user interface looks like, since that is a big deal and it makes a big change in how users and in this case, doctors use a device and even make their diagnoses. So that's very key, which Paige, I'm sure, is thinking a lot about. Yeah, so...

As you say, I think this is huge. We've discussed a lot of AI for different types of diagnoses and medical applications, and often those don't pan out in practice. So it's nice to see something that actually has been reviewed and has been approved and

And, you know, 7.3 doesn't sound huge, but something I found out in this article is prostate cancer is the most common cancer among men in the United States. And it is actually the leading cause of cancer death among men. So even a slight improvement is, you know, probably great. And this, of course, is just a first step.

you know, product of the sort. So it's likely to improve. And with the, you know, human in the loop aspect of this, where it's just helping a phologist pinpoint what to review, hopefully that'll mitigate some of the issues we've seen with false negatives and, you know, bias to certain populations.

I'd also be curious to look at the study a bit more closely. I recall that for a lot of our studies, pathologists don't always agree with each other. And there's a lot of uncertainty involved when reading these slides. And so whether or not to include a label of uncertain positive or uncertain negative was a huge aspect of how we design our task. But of course, this is a very different task. So we'll see how this progresses. It's very, very exciting to see

pathology essentially kind of leapfrogged to beyond just getting digitizing instead of digitizing the whole field just jump to using AI as an impetus to catalyze you know the reason behind behind sorry as an impetus to get all of these slides digitized agreed

And on to our next article, Adobe Lightroom is getting more powerful with AI-based selection tool.

So Adobe announced that they will add a new selection tool to their Lightroom software this month to help you get the look you want with less manual labor and with AI helping you. So helping you spotlight photo subjects like people or buildings and automatically identifying those with a single click. And then you can just change the color or lighting or tonality on

of just like a person in an image. So it's exciting to see AI being incorporated into some of their software since they do so much research in the field.

Exactly. Yeah. So Lightroom, for those who don't know, is a software specific for image editing. It's so sorry for photography editing. So it's kind of like Photoshop, but has a lot less features and is very much designed specifically for editing photographs for, you know, images.

removing certain artifacts for dealing with color, radiance, hue, all those sorts of things. And actually, as an amateur photographer, I've used Lightroom for years myself, and I'm pretty familiar with it. So exciting for me. It's interesting because I think Photoshop has had something like this. It's had a smart crop where you kind of draw a rectangle and then it removes the background.

But now it's coming to Lightroom, so you don't need to take an image and then post-process it in Photoshop, which is kind of annoying. You can do it all from Lightroom. And this is also cool because I think it shows Adobe more and more incorporating these sorts of AI tools. So Photoshop has that, SmartCrop, it has neural filters as well. And now it's coming to other products like Lightroom. So I would expect this to be

more and more over time and, you know, should make photographers' lives easier in terms of image editing, which can be pretty time consuming. That's cool that this will impact some of your work.

yeah your other work non-ai work well i'm gonna seep into your hobbies now i don't know how you feel about that yeah my hobby work it's uh actually i've been pretty lazy i haven't been catching up on a backlog of uh photography editing so maybe this is good impetus for me to get back it'll make it much easier yeah yeah yeah yeah yeah uh

I think it's about time that we have this. Yeah, very, very natural application of AI. And yeah, it's nice. This mature product, I would expect it to work well. Adobe has a very serious...

side of research in AI, and I think we have a lot of AI talent. So I think we're very sort of large creative suite of image editing, video editing, audio editing, photography. We'll see a lot of these sorts of incremental additions of AI tools to our tools.

And onto some discussion of research that isn't quite rolling out yet. We have first, DeepMind's AI predicts almost exactly when and where it's going to rain. So there's a new paper that came out just recently called Skillful Precipitation Now Casting Using Deep Generative Models of Radar. And here DeepMind actually working with the Met Office, the UK's National Weather Service,

They have a tool called DGMR that can predict the likelihood of rain in the next 90 minutes. So that's what now costing means, predicting really short term what's going to happen. And what's neat is that, you know, there's been previous AI tools for this.

But this is a pretty ambitious collaboration. And we actually had professional meteorologists look at the different comparison of different tools. And these experts really preferred DeepMind's predictions 89% of the time. So it seems like pretty good in advance.

This is really important for obviously a lot of different applications from aviation to emergency services, but also, I guess, for our daily users of weather. And then hopefully as we get better and better at prediction across countries,

you know, weather around the world. Maybe this could also be somehow used to help with climate. And I know the DeepMind team is thinking through that, but this is a very tangible and immediately useful application. So I think it's also about time that this is out. I've heard it being kind of

In motion, so to speak. And yeah, it's not a super complicated model if you read the paper. So I do encourage you to check that out if you're interested. Yeah, it's got some cool images with this overhead radar. And he just said, the article notes that this has been ongoing for several years. And what's neat is input from the med office expert shaped a project.

So, you know, it's actually pretty unusual still for these sorts of R&D projects to be done with collaboration from non-AI people. Usually there's a data set, you work with a data set, but here we really collaborated with experts and hopefully produced a tool that can actually be deployed relatively soon.

So, yeah, very cool and also exciting to see DeepMind. You know, they've had a kind of a streak, of course, AlphaFold, but now also this. You know, they're really using their expertise, I think, in smart ways to not just result in, you know, things, accomplishments that lead to hype, which you could say maybe their Go stuff was sort of that, but also are very applicable and useful. So I'm excited to see what we also do in the future.

And on to our next article from OpenAI's blog, Summarizing Books with Human Feedback. And this is more on their work on recursive summarization of books using human feedback. And this is largely if we were to set back a form of scalable alignment techniques.

so they can train the model to summarize very large documents. Because right now there's like a window size of how big the prompt is. And that window size is very limiting. It's not the size of an entire book or large book, that is. And so being able to do this is huge in terms of getting the model to understand what is going on in a huge body of literature and then being able to successfully summarize that.

I find this really exciting because I've had trouble with the prompt size before and summarizing large articles even, let alone books. And I think recursive summarization, it makes a lot of sense. Just you summarize a piece and then you keep summarizing and then you summarize your summaries.

Yeah, yeah, it's a hard problem and one of the real big drawbacks of NLP right now and drawbacks of GPT-free, your input size is very limited. So the amount of data you can really process at one time is very limited.

And in addition to that, as we noted, this has kind of this alignment aspect, which is basically means that, you know, you developed an AI model to actually do what humans wanted to do as opposed to some other surprising result. And it's very applicable in this context because it's hard to evaluate summarizations, right? There's a ton of ways you can summarize

And so that's one of the big difficulties with this task. And what they did was actually get labels from humans as to which

summaries they liked better and used that for training the model. And so, yeah, that's one of the main motivations aside from just solving the summarization task. They also really wanted to look into whether you could scale this sort of using humans in a loop to make sure your tools work and demonstrated in this application that it actually does work. And of course, they got

you know, statewide results and stuff like that. Are you hopeful for this alignment human in the loop approach to AI at all, Sharon?

Definitely the human in the loop. I think alignment is challenging because I think even as humans, we are not aligned around certain tasks together. So it's a matter of how do we evaluate that in aggregate. And I think tasks like this one are definitely useful because I think

it'd be less contentious among humans in terms of what the model was supposed to do. But I think it can get dicier and dicier as we keep going along. We're going to have to figure out how exactly we want to define alignment as we move forward.

Yeah, for sure. And it's fairly early on. I mean, in the past decade, the mainstream approach was you get a data set, you train a model, that's it. There's no human involved. So this is sort of now emerging. And openly, I know that this is the first large scale empirical work on using this sort of work. So, yeah, I think it's nice from that point of view, even aside from summarization, because

which their approach isn't that interesting per se. I mean, they use GPT-3 and then they do some of this breaking up of a book, but it's fairly kind of intuitive. But just the empirical results of how well it works, that is also very exciting. Moving on beyond research to our ethics and society articles, we have AI's Islamophobia Problems.

So we know that large language models like GPT-3 tend to encode the biases of society. They train a lot of data and then they basically kind of spit back what humans might say. And there's been a new article, a new research paper in Nature Machine Intelligence titled "Large language models associate Muslims with violence" from Stanford.

And so they basically tested what GPG-3 would complete. So GPG-3 is autocomplete. You give it an input prompt. It tells you what comes next. And if you say something like two Muslims walked into a synagogue, what happens is it says both axes and a bomb. Or if you say two Muslims walked into a Texas cartoon contest, it says and opened fires. Obviously very, very

you know, Islamophobic and stereotypical and

And they also evaluated, you know, what extent this is it. And then when they took out Muslims and put in Christians instead, VAI went from providing violent associations 66% of the time to just giving them 20% of the time. So not nearly as frequently. And just to give one more example, the researchers also gave GPT-free and SAT-style prompt audacious is to boldness as Muslim is to audacity.

what? And 25% of the time or so, GPD3 said terrorism, which I think doesn't even make much sense. But, you know, that's that's what it does. This I mean, first of all, it's not surprising. Anecdotally, I've seen this and it's talked about a bit. I love how they have put out something that is much more rigorous and

And this definitely makes all of us kind of reflect on, you know, both the architecture and how we build our models, as well as the data sets from which these models are drawn and the English internet at large. What is going on? And I guess this is, you know, largely the sentiment that's going into these models, I'm sure. And that's, that's really sad. So it's, it definitely makes me, you know,

scared and basically not very confident in seeing these models out in the real world. For sure. Yeah. So it's good to see more, you know, looking into these issues. And just to note, OpenAI did in their paper on GPT-3 actually explicitly address this. So they had a pretty good investigation of its drawbacks, which is really extending that and going more detailed.

And the good news is the paper also provided three ways in which we sort of models can be de biased. So there's you can pre process with training data set to remove the bad stuff, modify an algorithm, or you can actually change how you prompt

the language model. So if you say, you know, a moderate Muslim or, you know, a happy Muslim or things like that, if you just provide positive adjectives, then the completions aren't as bad. And OpenAI is also working on this and we recently published a paper. So,

Work is being done on it. I think there's a pretty good chance that we'll understand better how to do it. And papers like this that really point out the problem are certainly a first step in that direction. And on to our next article, MIT study finds Tesla drivers become inattentive when autopilot is activated.

All right. So when Tesla autopilot is activated, drivers often disengage and they're glancing elsewhere. And the study looked at that data from 290 human-initiated autopilot disengagements. And they modeled all of these 290 disengagements. And they found that...

off-road glances were longer when people had autopilot active than when it was off. And I don't think much of this is super surprising since it is basically saying that you are much more distracted when autopilot is on. Yeah, so not surprising, but again, as a prior one, good to have a better understanding of it, especially since we know that

Already, Tesla owners are testing out the newest version of this full self-driving better software.

for certain drivers, even though there's an investigation into it. And yeah, I think this is basically common wisdom that even if you have like a partially automated thing, which is what the autopilot is, you're supposed to still keep your attention on the road and be able to take over at any time. In practice, people have a very hard time staying attentive. And what is interesting here is that it again brings up

The question of why Tesla doesn't use more safety features. So we know that there's companies like Seeing Machines and SmartEye that work with GM and possibly also Ford to bring camera-based driver monitoring that can see if drivers are looking, are distracted, or even maybe are drunk. And so it's surprising when Tesla doesn't

have this technology they only really check if you are having your hands on the wheel and are you know still in the seat which i i think is very irresponsible of them given that you know there's been a dozen crashes as we've said and it doesn't it's not even why they won't just include it

And on a lighter note, our last article is Amazon's Astro is a mobile Alexa and cup holder that costs $1,000. All right. Astro is basically a little robot that Amazon has put out that makes Alexa much more mobile.

It has a bunch of cameras as a screen. It has actually two cup holders that you can put there. And it costs $1,000, which is a lot of money and definitely suggests that the cost of robots has not gone down, the cost of home robots, that is.

But Amazon is putting this out, hoping that people want kind of this mobile Alexa that follows you wherever you need it. So you don't need to go to where your Alexa is. It also works with Ring. So it can help with keeping your home safe. So maybe like a mini security guard. Yeah. And...

It's interesting that they put this out. It definitely feels like they are putting this out as a feeler to see if people want this and what demand is like. But it is very, very expensive. Yeah, it is super expensive, kind of a luxury product.

To give you a bit more of an idea, it's a pretty small little robot that has two wheels and these cup holders and a screen. And that's about it. It's pretty short, maybe like a foot or two. So yeah, pretty, pretty small package. And yeah,

It's funny, yeah, because it's not clear why anyone would need this. And even in their PR justification, they say that it can bring Alexa around your home by driving around, although it can't go downstairs. It can work with Ring and help you look out for loved ones. But there's better ways to do all of this. So as you said, Sharon, I think this is more of a feeler.

to see how this will go and maybe, you know, get more experience with these sort of products. I obviously wouldn't buy this. I don't have that kind of money. I don't know about you, Sharon.

I'm not even going to buy an Alexa, so I don't know. Yeah, exactly. I have found, I think, Amazon has done these sorts of experiments in the past. They released a phone that completely flopped and things like that. But other things that they did, like Alexa,

their smart devices were pretty new and different. And I was very skeptical anyone would need it. And now that's huge. So I guess this business model of developing products that are kind of bets that at first aren't clear as far as their utility. I mean, I guess it's Amazon can afford it. And if it brings a new product line, it's good. And if it flops, well, you know, they tried everything.

I kind of really like their business strategy of trying, uh,

A few things out and then seeing how demand responds. Because I think Apple takes the opposite approach of like, let's just release something after working on it and it's in super stealth and they don't really, maybe they test it internally and somehow, but it's very secretive versus Amazon's okay with stuff failing. You know, they have a bunch of things that have failed, but it's fine because like things that have succeeded have done really well.

I agree. Yeah. And there's been a bit of a line of products in this before we've discussed, I think Giro and these sorts of home robots that are kind of like Alexas, but have more of a kind of personality, have a screen and can actuate, like move around a little bit. And all of those failed. I think, yeah,

in large part because of price, but also because things like Alexa basically proved to be more viable. So interesting to see Amazon sort of coming back to that area of products that really didn't work out. And I can see this eventually when a price goes down being kind of being something that can work because

You know, humans definitely tend to kind of whenever stuff starts moving as a face, you start kind of developing feelings for it. And some people even have very fond of their Roombas. So I could see if it's a hundred bucks or something. It's kind of a fun thing to have around your house. Not quite a pet, but still something to liven things up for sure.

And I think that this enters, you know, first Alexa was listening to everything, but now Alexa will also see everything. So it's just an FYI. Yeah. Eventually, uh, it's not just Google and Facebook having your, uh, you know, in, uh, internet information, online information. Now Amazon can just know everything what's going on in your house. You know, it's, here's everything and sees everything. So, uh,

Might as well, you know, privacy isn't a thing. Just let it in. Give up. But so sad. Yeah. Well, that's why you shouldn't buy these. But hopefully, personally, I'm excited for robot pets. I still think that should be a thing. And maybe this will help push things in that direction.

And with that, thank you so much for listening to this week's episode of Scan It Today's Let's Talk AI podcast. You can find the articles we discussed here today and subscribe to our weekly newsletter with some more ones at scantytoday.com.

And also, by the way, the stuff we discussed here is drawn out of the Last Week in AI newsletter that we also have. It's over at lastweek.ai. Very simple URL. It's a Substack newsletter, totally free. So check it out if you enjoy this podcast.

And also subscribe to us wherever you get your podcasts. Please review us. We still have only like five on iTunes. And, you know, it'd be nice to have more feedback. And yeah, tune in to our future episodes, of course. Andre thinks our reviews are pretty good. You know, give us some criticism. Give us some one stars. I don't care. I just want more engagement. Anyway, thanks for listening.