Our Bright Future with Robots | Daniela Rus & Gregory Mone
30:48 Share

Welcome to the Talks at Google podcast, where great minds meet. I'm Matthew, bringing you this week's episode with roboticist Daniela Russ and science writer Gregory Monet. Talks at Google brings the world's most influential thinkers, creators, makers, and doers all to one place. You can watch every episode at youtube.com slash talks at google.com.

Everywhere you look, there's talk of artificial intelligence and machine learning. Daniela and Gregory have dedicated their lives to studying both. Daniela is a world expert in robotics. She's the first female director of MIT's Computer Science and Artificial Intelligence Lab and a MacArthur Genius Fellow. Gregory has authored 19 books and is a former columnist for Popular Science magazine.

He's collaborated with Neil deGrasse Tyson, Susan Cain, Bill Nye, and Skeletor. They join Google to talk about their book, "The Heart and the Chip: Our Bright Future with Robots." The book overviews the interconnected fields of robotics, AI, and machine learning. It reframes the way we think about intelligent machines while also weighing the moral and ethical consequences of their role in society.

Here are Daniela Russ and Gregory Monet. The heart and the chip. Our bright future with robots. Welcome, Daniela and Greg. It's a pleasure to have you. Hello, Ayush. Thank you so much. Great to see you. Thanks for having us. Absolutely. Greg, this is actually your second time being on Talks at Google. Is that correct? That's correct. Yes.

So I think 14 years ago, you gave a talk about the truth about Santa, your book. And with the robotic advancements we've had over the last 14 years, how do you think Santa's operations would change if he had the robotic technology of today? After speaking with Daniela and learning everything she's up to and what's happening out there in the world today, I think it'd be a lot easier. A lot easier. No need for magic at all.

So Daniela, in the book, the book is structured in three subtopics. One is the dreams, the reality, and then the responsibilities that come with this new reality. Can you paint a picture for what the ideal day might look like 15 years from now, where robots have been seamlessly integrated into our day-to-day life?

Ayush, I really like to think about how robots can help us with physical tasks and AI can help us with cognitive tasks and what a world with robots and AI could really mean for all of us, how it could really improve our lives. Hence, the first part is called dreams.

And so let me start by observing that robots can extend our reach. They can amplify our strengths. They can refine our precision. They can save us invaluable time when they take on the physical tasks. They can enhance our ability to be precise. They can magnify our vision. And they can empower us to defy gravity, like my favorite character, Iron Man.

And this would enable us to get into environments once deemed inaccessible. There are so many possibilities. But just imagine driving home from work, knowing that your vehicle has the intelligence to help you get there safely and the smarts to give you the relaxing experience you need after a long day at work. You stop at the store to pick up some supplies for dinner.

And when you walk in the door, you hand your dinner menu to a robot at the door that connects with your house to confirm what you needed to buy. And then a robot could collect what you need and hand you a box. And when you get home, you pass the box to your kitchen robot and you let your kids help with cooking because even if they make a mess, the robotic house cleaner will help sweep up when they're done.

And so I know this sounds like one of those cartoons about the future that never quite comes to pass, but that future really isn't that far off. We can do so many aspects of this idealized view of how

your return from work to home might look like. We can also extend into other activities in your home. Just imagine waking up enabled by your personal robot and the robotic bedroom that figures out the optimal time for you to wake up and helps you organize your outfit and collects everything you need for the day. And then at work,

You'll have the opportunity to learn how to do your physical tasks. For example, surgery, if you're a surgeon, if you're a student in surgery. So you have the opportunity to learn from leading experts because you can use robotic shirts that help you move the way they

They do. And these kinds of robotic jackets can double up as tennis coaches, which is what I would like. I would really like to improve my forehand. But then they can teach you and they can help you improve so much of what you do for leisure.

And at home, you can imagine everything in your home being more adaptive, more intelligent, more customized. One of the fun things about working with Danielle on this project is she would bring up examples like this, Ayush, and then...

all of a sudden it would be, oh, and we have a project where we're working on this. So, you know, it's not just science fiction. It's not just the movies, right? They, you know, they have this, this robot bake bot that actually baked cookies and prepared everything from scratch. And, you know, she showed me videos in the papers and worked through everything that, you know, how it worked and also some of the challenges they met with about making something like this an actual reality. So

All these wild ideas are grounded in real projects. And I think that's what makes Daniela's work so much fun.

That is quite interesting. I know one of the biggest challenges with robotics is dexterity. And then we have a reasoning issue, which I think AI has helped us with a fair bit. But what are some challenges, Daniel, let's start with you. What are some challenges you're seeing that will prevent us or that could hold us back from reaching this ideal state?

Well, there are a wide range of challenges that I like to cluster in three categories. There are challenges about the hardware parts of machines. There are challenges about the software parts of machines. And there are challenges about the interaction we have with machines.

So let me talk about each one of them separately. On the hardware side, so the hardware is important because the body of the robot defines what the robot will be able to do. If you have a robot on wheels, that robot is not going to bake cookies for your climb stairs.

And so thinking about how we shape the body, how we integrate actuators and sensors in the body is very important. At the same time, most robots today are pre-programmed.

And so their brains are pretty simple. Their brains consist of mostly low-level control that helps us figure out how the robots can move and then a layer of algorithms on top of that. We need to make these brains more capable if we're going to have intelligent robots that can adapt, they can learn, they can reason about the world. And then right now, most of the interactions we have between humans and machines are

are with experts. Right now, you more or less have to be an expert in order to know what to do with a robot. I would really like to elevate the interactions so that anybody can have a robot. So you see, if we start thinking about the capabilities of the robots, there are some really fundamental questions that come up.

We've made a lot of progress in autonomy. We have autonomous vehicles that deliver medicines in hospital. We have some trials with autonomous vehicles in certain enclosed environments that help people get to their destination. We have robotic trucks that move containers in ports. So we have a lot of

products already in autonomy. We have not made the same level of advancement in manipulation.

And that is because advancing robots dexterity is about improving the ability of robots to handle objects delicately and precisely. It's about the ability to interact with the physical world. And this is hard because we do not quite have the right hardware, meaning the robot hands and the sensors.

And it's also really complex to think about replicating the human touch and the human manipulation dexterity.

Now, we are working on new designs for robot hands that are both soft and compliant, yet strong, so that these hands can lift heavy objects, they can grasp gently fragile objects, they can begin to demonstrate the kind of in-hand manipulation that we take for granted.

So we are making progress on manipulation and the solutions are at the intersection of innovation on the hardware side, innovation on the data side, and innovations on the algorithm side. Let me just say that a couple of years back, I used to think that soft robots could give us what we needed for dexterity.

And we worked a lot in soft robots. But then I realized that pure soft robotic fingers are not enough because they do not have the strengths. They cannot handle the payloads that we need to exert when we manipulate the world. And so now we are developing robots that incorporate both hard and soft elements in them in order to get the best of both worlds. We want the compliance of soft materials.

that provide higher tolerance of uncertainty when a robot is trying to grasp something or move something in its hand. And we also want the strength of hard materials that can give strength to the mechanism. So just imagine a soft robotic hand that incorporates bone-like structures within it. And this concept excites me greatly because it opens up so many possibilities.

If I may, I would talk about two more things.

In addition to challenges in dexterity, we also have challenges in perception. And perception usually involves enhancing the robot's ability to interpret and understand their surroundings accurately. This is critical for navigation. We've made a lot of progress in perception for self-driving vehicles when we developed an important sensor, the LiDAR.

Before we had the LiDAR, all the navigation algorithms used sonar. And sonar has a wide pond of uncertainty. No algorithms worked with sonar. And then all of a sudden, we had the very precise LiDAR measurements. And all of a sudden, all these algorithms that used to not work now work.

And so perception relies on important sensors, on accurate sensors, and then on the algorithms that we use or we develop to manipulate the data from these sensors. In manipulation, we have the challenge that we actually do not have the sensors that replicate the skin, the continuous touch that we experience with a human hand and with our skin.

Now, perception is very important because without perception, the robot cannot really understand the world. So it needs information to navigate spaces, to recognize objects, to interact with humans and other objects, to perform tasks accurately.

And so the advances we're pushing on the hardware side, on the algorithm side and on the data side are really helping us get to the next level. And I'm very excited because in my lab, we're developing skin-like touch sensors.

And we're also looking at machine learning and multimodal foundational models and large language models. In other words, the introduction of language as a way of augmenting perception and delivering more intelligence, more capabilities to think abstractly for our machines.

So I believe that this is getting us closer to what I like to call physical intelligence or physical AI. And this is where AI's power to understand digital text, images, and other types of online information can be used to make physical machines smarter, can be used to make them capable of adaptation and learning.

And if we achieve physical intelligence, then we will be able to help devices like robots and other mechanisms do their jobs better by using knowledge and using data to extract knowledge from it.

I think something really important you touched on was this human-robot interaction piece and how soft robotics can make robots even more, I think, safer with interactions with humans. So Greg, I know you spend a lot of time hosting workshops and teaching children.

In an environment like a school where safety is paramount, how do you think robots can help and what level of advancement do we need to see this human-robot interaction flourish?

So before I answer that, I just want to say quickly, building on what Daniela was saying, one of the other things that fascinates me about her work and that I love is this mix of, which I'm sure you picked up, this mix of working on the body and the brain, right? So advancing both the programs that are sort of directing the robot's actions and also the physical embodiment of the actual machine. And there's that. And then the other element is, you know, as...

It's may it's changed the way I experienced the world sometimes where, you know, you do something like you pick something up, right. And turn it around in your hands. And it's so automatic for us. But then when you start thinking the way, the way she does, the way roboticists do, you realize, wow, you know, my hand is this really amazing thing. And what is my brain doing to help my hand do that? Um, now switching back to, you know, your education question, um,

The other thing, my own thinking has changed in a way is, you know, I was originally introduced to Daniela's work a long time ago when I was asked to write a story about, you know, when is our robot Butler coming? Right. And I interviewed a whole bunch of different roboticists and computer scientists and, you know, eventually came to the conclusion that it was, you know, not really for a long time. And, and I think this, this idea of the robot Butler or the robot teacher is sort of, it's a,

it's almost a misdirection, you know, it's kind of a Hollywood idea where really what we should be thinking about is, is these smaller point solutions and tasks specific systems. And Danielle, correct me if I'm getting this wrong, but you know, so with education, I don't think so much about a humanoid being in the classroom as, you know, these kind of virtual tutors that can

that can work with kids and respond to them and see if they're losing focus and introduce something to keep them on task. And we've already seen some fascinating research in that area that Danielle and I picked up on and studied. And so I really see it happening in that space initially. I agree, Greg. Although I have to say that when my daughters were

young and I was trying to teach them French, I would have loved to have a robot chase them around and work on French drills with them instead of me. And I think that's totally possible right now. The other thing I would say is that I really believe that it is important for

all of us to understand something about how we make things and how we program them. And you see, those of us who know how to make things and how to program them, we have a kind of a superpower because we make real anything we imagine. And who wouldn't want to do this? And so I advocate teaching both

computational making, the physics of the world, and computational thinking or the programming of the world to children early on. In our lab, we developed a robot garden

The thesis was that if we actually create an embodiment of lots and lots of different machines that could also somehow visualize and capture the execution of algorithms, it would be easier and it would be more fun for children to learn how to program. Because basically we have these robotic flowers, which can be programmed to move and can be programmed to change color.

And so when you teach algorithms, you can kind of implement the flow of execution of the algorithm in color using the garden. So the children could really visualize what is happening, how computation proceeds. And so I think that there are really wonderful ways, innovative ways for bringing these robotic tools to the classroom. Yeah.

Go ahead, Drake. No, I was just going to say, um, computational thinking, you know, so Daniela introduced this idea to me and was, you know, we were talking about the importance of it and we ended up effectively taking a computational thinking approach to working on this book together where normally my process would be kind of a, I'd want to ingest all the important information, talk about it and then write, you know, basically go away and write a, write a book.

And Daniela, her approach is, no, no, no, we have a big project. So we're going to break it down into small, achievable tasks. And then we're going to accomplish those one at a time. And then we're going to link them together and find the connective tissue. And then we're going to have the larger task solved. And so we ended up doing it that way. And it was great. It was refreshing. It was a lot of fun. Yeah.

I think one really important point both of you touched on is this level of having a personalized approach to different things. And I know, Greg, you mentioned robot teacher. Something I noticed with LLMs, I've been using Gemini, ChatGPT for a while. I noticed that if I give it any piece of content,

I can learn it better because it can adapt the content to me. And then over time, I've realized there's certain things I tend to focus on. So for me, like I love technology. I love brain computing. I love neuroscience. So if I'm watching a lecture, there are pieces where I'm extremely focused, but then there are pieces where it's like about the math behind it. And I tend to zone out a bit. So I actually built an experiment where I was using an EEG headset.

and measuring my focus. And I found the topics which would keep me engaged. And I would then use the LLM to translate those topics into something that is more personalized for me. And I think education overall has such a large, it's going to be impacted so profoundly when we have these tools accessible to anyone because everyone learns in a different way. And if we can make it more

accessible for everyone and i know greg you do this with your writing where you make it so engaging that you know younger audiences want to want to stay focused and like pay attention um that's i think that's uh that's a very big value add we can provide it is yeah absolutely absolutely and uh i guess um what what i can add to what you said ayush is that it's important

for us to make the mass exciting also. So there are many parts of the education space that we have right now. And so figuring out how to make it interesting for people and accessible for people to access and grasp that knowledge is important. And then knowledge is changing so fast. We are inventing new things. We're generating new knowledge at home.

A pace that has been unseen and unheard of since the history of humanity. And so keeping up with all this new knowledge is challenging. Finding ways to use the support of machines to keep us abreast, to keep on top of what we are doing.

what is most important for us to know is something that the AI and the machine learning and robotic tools can help all of us with. It's interesting. I think one thing that is

useful to keep in mind is a lot of people fear this robot uprising and they're concerned about robots taking away their job. While we have AI helping people learn and upskill, for students coming into colleges,

in the next five years, Daniel and Greg, because I know both of you spend a fair bit of time with young adults. What areas do you think people should focus on going forward? Because back in the 1970s, we had the third industrial revolution. We had mechanical engineering and nuclear engineering take over. Then we had the computer become a big deal and electronics came up and then computer science came up and it became the biggest, more in-demand industry.

major going forward where we've seen most of this, you know, computer science stuff abstract the way like yesterday there was a AI engineer called Devin that was released, which can do a lot of the coding for you. So people's focus changes as time goes on. What do you think people, students especially coming into their senior year of high school should focus on to prepare for college? Ayush, let me say

That I believe everyone should know how to code and everyone should know how to make things. This should be part of literacy in the 21st century. But this doesn't mean that it's all we need to know.

I also think that it's very important to have a broad education, to know about all the fields, to know about history, geography, literature, music, art, because

These fields are really important for understanding our place on this planet, understanding our interconnections with everything and everyone else on the planet, understanding where we come from, enjoying our time, and really having the knowledge in our minds to be creative and to make connections between seemingly disparate fields that can

lead to new discoveries, to new ideas.

And it's also very important to learn some of the softer skills like communication. Communication is super important. And that means both written communication and spoken communication, but also collaborative problem solving, critical thinking. These are all very important aspects of preparing to be successful in

at this point in time in our world. Yeah, and the other piece here is this is gonna, it's gonna upend a lot, obviously, but it's also gonna create so many opportunities. So, you know, Daniela had me talk, my daughter, who's a senior in high school,

had me talk her into taking computer science this year. And she loved it and was totally interested. But it's not going to be her thing, right? Yet, I'm from a family of lawyers, and I could see her going into that field, but having an understanding of AI and machine learning and how it all works. And I mean, I know that that field in particular is trying to figure these technologies out and what they're going to do with them.

So that could be an advantage. I would like to add to what Greg said, because I really think it's important for us to understand the tools that we develop and we use. And in the future, we will use increasingly more sophisticated artificial intelligence, machine learning, robotic tools. And

understanding them is very important. In fact, this was one of the motivations for parts two of our book. We wanted to demystify how it all works so that people could understand, um, not just the mechanisms, but also, um, the,

What are these machines like? What are they capable of? If you understand the nature of, let's say, statistical thinking, then you can really see what the ceiling and limitations of these tools are. And at the same time, when...

When people tell us that they are worried about machines taking over, I think it's important to listen, understand the fears, and then figure out ways to explain how to have a different perspective. And I hope our book will play an important role in demystifying robotic machines and what they can do for us.

Yeah. That is, it is very interesting to see how while these tools come up and they become more prevalent, people tend to rely on them. And one thing I am particularly worried about is people losing their ability to think critically. I think, I think writing and Greg, obviously you're a writer, it's like, you know, you would know that when you write something and you articulate it, you'd actually, you know, you know, using the muscle that actually is required to think critically. But people are using, you know,

all these LLMs and different AI tools to abstract away that part of their life. Do you have any advice for people going into this age over the next five years, if there's something they should focus on to keep themselves mentally, cognitively fit, and then physically, obviously, people will keep up with, they can't keep up with machines, but anything, any advice you have for people, what people should do over the next five years to stay relevant?

It's important for people to continue to learn how things work. If we imagine a future where we don't know anything anymore, we just take a lot of data, we feed it into machines and we see what comes out. We run the risk of reaching a point where something breaks and nobody knows anything about how to fix it. Yet we have these tools that we depend on.

And so I think foundational knowledge is very important and will continue to be so. And Ayush, you're absolutely right. With something like writing, I look at these tools not as kind of a first draft engine because your first draft is really where you're getting the ideas straight in your head and trying to understand the concept. So if you're

putting that task off to an LLM, then you're not getting it clear and you're not really understanding what you're writing about. You're not putting your ideas out there. I think it can be valuable in terms of shaping the product and tweaking and finding better ways to say things. But ultimately, you need to make sure all that critical thinking is happening in your head and not a virtual brain.

And besides, today's tools are mostly about getting the average of what's in the ether in response to our queries. So it's not about getting your own ideas out. It doesn't let you differentiate as much unless you actually stay at it and

and keep developing that. Daniel and Greg, I would love to keep this conversation going, but unfortunately we're coming up on time. Thank you so much for coming to Talks at Google. It was my pleasure hosting you. For everyone watching, I'd highly recommend reading The Heart and the Chip, A Future with Robots. It's available anywhere you buy books and you'll learn a thing or two. So I'd highly recommend giving it a read. Thank you so much, Daniel and Greg. All right. Thank you, Ayush. Thank you.

Thanks for listening. You can watch this episode and tons of other great content at youtube.com slash talks at Google. Talk soon.