#146 Barbara Tversky: Action Shapes Thought
02:00:40 Share

I'm listening to when the boiling of the rice, I'm smelling the olive oil on the saucepan and I'm picking up the danger sign, got to lower the heat. That's only from a great deal of experience and paying attention to the smell and the sound.

of those cues, and that isn't in the recipe. The abstraction you get either in a diagram or verbal instructions isn't going to enable you to be a great cook. You can make satisfactory things, and you need to know not just adding salt, you need to know when to throw it in the trash and start all over again.

Welcome to the Knowledge Project Podcast. I'm your host, Shane Parrish. The goal of this show is to master the best of what other people have already figured out. To do that, I sit down with people at the top of their game to uncover the useful lessons you can learn and apply in life and business. If you're listening to this, you're missing out.

If you'd like special member-only episodes, access before anyone else, hand-edited transcripts, and other member-only content, you can join at fs.blog.com. Check out the show notes for a link. Today, I'm speaking with Barbara Dversky. Barbara is a professor of psychology at both Stanford and Columbia. Barbara has published more than 200 scholarly articles about memory, spatial thinking, design, and creativity.

She's the author of Mind in Motion, How Action Shapes Thought. If you're interested in cognitive psychology, how the brain works, the way that language shapes thought and thought shapes language, you're in for a real treat with this episode. We cover her nine laws of cognition, why movement, not language, is the foundation to thought, how language changes how we think,

gestures, and their relationship to language, why she dove into the work of Leonardo da Vinci, and the importance of perspective taking. It's time to listen and learn.

The IKEA Business Network is now open for small businesses and entrepreneurs. Join for free today to get access to interior design services to help you make the most of your workspace, employee well-being benefits to help you and your people grow, and amazing discounts on travel, insurance, and IKEA purchases, deliveries, and more. Take your small business to the next level when you sign up for the IKEA Business Network for free today by searching IKEA Business Network.

What got you interested in how the mind works? I think everybody is puzzled by the mind. Whether to pursue it or not is a choice, and how to pursue it is a choice.

When I got into newly birthed cognitive psychology, I had been interested anyway in the visual spatial world. We are constantly experiencing it, and so it's a large part of our lives.

And most of the theories at that point were based on what philosophers called propositions, on language-like representations, and attempting to reduce everything in the visual-spatial world to these propositional statements, which are minimal statements about truth conditions. And that struck me as odd.

Because the visual spatial world precedes language evolutionarily. It takes up far more of the cortex than language. And it has to have its own logic, not logic-based, not a logic or a structure of

based on language. If anything, I thought maybe language is based on the way we perceive and interact, perceive space and interact with things in space. So I began looking at various aspects of the visual spatial world, looking for its own structure and own logic. And that brought me back to language.

Because we use language to describe and talk about the world, and spatial language seems both primitive and primitive in the sense of its origins, both ontologically and phylogenetically. We talk about where things are, where to look, what to do. Those are all about our interactions with space, and they seem crucial in life.

So it brought me back to language, but it also got me interested in understanding how we perceive and understand space and interactions with things in it.

And if you think about actions in space, the most basic action is whether to approach something or avoid it. To go forward, we like something, we're attracted to it, we must interact with it, or we avoid it in one way or another. And that simple basic action that even one-celled creatures must take

seems inherently value-laden, emotional. So is it fair to say that space and movement have changed how we describe things? Is that fair to say? Yes. And in fact, I think it's fair to say that all our spatial representations depend on interactions. They're a coupling.

of what we do and how we perceive it. And those studies go way back 150 years when Stratton and later others put

lenses on people's eyes that move the visual system or the visual perception leftwards or rightwards. You can do it upside down. And people adapted by moving, by grasping things. And if they didn't grasp things and recalibrate...

how space was represented in the brain, then they didn't learn to behave in the new situation. And it's undoubtedly something we're doing continuously all our lives, is recalibrating our many visual spatial systems to the way they allow us.

to behave in the world. Has language changed how we think too? The language that we use to communicate is also, I would assume, somewhat the language that we use to think in. Right. And that, I think, goes deep on how...

So there's beautiful work on rodents and other primates on how space is represented, the larger space that we move in, not space of hand interaction, but the larger space we knew. And this won the Nobel Prize for the Mosiers and John O'Keefe, the combinations of their work.

So O'Keeffe with Nadel and other people had found in hippocampus of rats single cells that respond when the rat is in particular places.

And what's beautiful about these cells is the single cell brings together information about vision, smell, touch, the movement of paws on the ground, and so forth. All this multisensory information into a single cell, and that encodes a place.

They thought, in fact, they wrote a book called "The Hippocampus is a Cognitive Map." However, those place cells weren't arrayed spatially.

So the problem was still open. How do you get all these place cells related one to another? And the Mosher's working in O'Keeffe's lab found grid cells one synapse away in entorhinal cortex that arrayed the place cells relative to one another, essentially a spatial array. And later work, work that came quite a bit later, beautiful work,

showed that the single cells in hippocampus also encode people, events in time, conceptual relations, and that grid cells map them in temporal space, conceptual space, social spaces.

So that shows quite beautifully, elegantly, that the foundation for spatial thought is also the foundation for conceptual thought, for conceptual relations, because what the grid cells are doing is mapping relationships among single items.

So that is one cornerstone of the claim I've been making, that spatial thinking is the foundation of all thought. And I qualify that by saying it's just the foundation, not the entire edifice. So language, I'm going back a long way to your question, is...

Rats don't talk. They communicate, but not abstract ideas, or so we think. And even more what we would call advanced primates don't seem to...

communicate in language or gesture more abstract ideas. So chimpanzees, gorillas will point, but members of the cone species won't look at where they're pointing. And nobody so far has seen a gorilla or an orangutan or any of these other primates make a map in the sand to explain to somebody else how to get somewhere or even gesture directions.

So language does affect, in humans, does affect the way we conceive of space. We're all wandering around now with Google Maps trying to get us to all kinds of places.

And we tell people how to get to places. So that gives human beings another layer or two layers, if we include three layers, if we include maps, gestures, and language for understanding space and for understanding conceptual relations that are spatial.

So our language reflects our spatial thinking. We say somebody's on the top of a heap.

and we gesture up or we say someone's fallen into a depression and we gesture down and our body goes down to indicate up and down correspond to being happy versus being depressed. We've created a dimension. It's vertical. And so, again, the way we use space to communicate to other people, to think ourselves up,

is layered with these gestures and language and diagrams that themselves have a basis in spatial thinking.

But they give us additional tools to navigate the world, to do things in the world, like set a table or operate a bicycle, all the more manual activities that humans engage in, play a piano.

I want to get into some of the implications of that in a second. The question that came to mind is that people tend to think of themselves as a visual or verbal or mathematical thinker. Is there any evidence behind that? Or can you be good at more than one of them? Or how does that work? Yeah, I mean, the disappointing is that you can be good or bad at any of them more or less independently.

So that's discouraging in some way. You think if you're a visual person,

If you're not very good at visual, maybe you're better at verbal, and maybe you are because we do tend to compensate and use media and channels that we feel more at home in, we're more adept in, but they do seem to be pretty independent. And I should say that a lot of our spatial thinking is really not visual.

The taxi drivers in London that get you to any of these obscure streets aren't imagining a map as they're navigating. They simply have a spatial sense of where things are relative to each other that may not exist.

You might not necessarily always have a conscious experience of them. Blind people navigate quite well, many of them, and it's spatial, not visual.

And they'll navigate by other cues, acoustic cues, smells, wind, the texture of the ground. These are all kind of landmarks and cues that keep people without sight oriented. And I know many people who've been blind from birth or blind later who have amazing spatial abilities. They can be excellent mathematicians,

and get around space better than I can. So space is not just visual. And in fact, the areas of the brain that capture space are multimodal. They get input from many different modalities and output to them. Now, each modality is going to have its special characteristics, but to some extent, they're compensatory.

Let's talk about some of those modalities, I guess, in the sense of like, how does this change how we write or how we present material? And maybe we'll keep Zoom separately and some of the implications for that. But when we're writing an email or we're writing a document at work or we're giving a presentation, what are some of the impacts of this? I don't have direct information. I can describe some studies that we've done where you're learning and how you're learning and

And there we put people alone in a room. They're not talking to anyone. And they're alone in a room. And we've given them complicated material to learn.

and they're going to be tested. And these are students, so they're used to learning and being tested, and they're good at it. And the material we gave them is varied. We started with descriptions of environments, so describing where four or eight landmarks were in a gym or in a small town or a museum, and describing incidental information about the environments to make them more vivid.

And we told them that they would be tested and they could read these descriptions four times. And we filmed them as they were reading. And as they were reading, they were making gestures.

So they're not talking. The gestures are for themselves. And for the environments, they were drawing lines for the paths in the environments and points, dots for the various landmarks. And they were doing it over and over the way you might rehearse a phone number. So they were rehearsing that, checking it, and doing it in different ways. But for the environments, it's

Just about 70% of the people did this spontaneously. We never suggested that gestures would be helpful. We never gave them a hint that we were interested in their gestures. But about 70% made lines for paths and dots for places. The way you would do if you were drawing somebody a sketch map of how to get from one place to another is,

And when they did it, they remembered better. And when we took a new group of people, we've run this now four or five times, a new group of people had them sit on their hands, and that group did worse on the memory test. And some of them even said, I can't think without my hands.

So this was spontaneous. We tried it with other material, how a car brake works. It's quite complicated. And you can again see people making models of how the car brake works. And here, a smaller percentage did it, but it again helped their memory. Those who did it remembered better. We've tried it with...

time, scheduling multiple events that happen at the same time, and people made matrices, sometimes with the knuckles on their hands, sometimes on their tables, sometimes in the air. So people model them differently, but they make models. Again, it's somewhere around 60% of the people make these models spontaneously, and when they do, they remember better.

So those models are spatial motor. For the most part, people weren't looking at their hands. Some were every once in a while, but for the most part, their eyes were on the screen reading the material and their hands and their body were translating what they were reading into thought.

or that's the way we conceive of it, and when you watch it, you can't help but feel you're watching somebody think. I mean, you see them tilt their head, think, look at the screen, gesture, going back and forth in that way that we do when we're thinking. But the models were spatial motor, so they were a separate channel from language.

And they're a redundant channel. They're another way of representing the same information. And the gestures are representing. And they're representing in a more direct way than language. A line on a map is analogous to a path in the road or a line that your fingers draw. And a point is analogous to a place.

And so that that representation is more direct than saying path or road or saying place or landmark. So we think it gets to meaning more directly. And the fact that we use spatial language to describe these things like line and dot seems to again indicate that

that are thinking about it is spatial visual and based on the spatial visual that we're representing abstract ideas in these simple geometries of lines and dots and boxes.

So you see that in diagrams as well? I mean, people make, if I take the rudiments of a sketch map that it's lines and places, well, those are lines and nodes or links and nodes. And we can use links and nodes to describe decision trees or conceptual hierarchies or phylogenetic hierarchies.

office hierarchies. So we use that same way of conceptualizing, which is spatial, for describing, for representing abstract things in a diagrammatic form, but also in language. So does that have implications for how we present? Like, should we be presenting in

more graphically in terms of bullet points and maps and diagrams and relationships, explaining them in a spatial sort of way to people? I think it's a good idea, and there's more and more of it. Now that pixels don't care if they're representing words or diagrams, and we do have access to many, many, many pixels,

So that making diagrams, it's certainly easier digitally, and then it's easier in printing books before books were typeset, and that gave an advantage to language, whereas diagrammatic things had to be done differently and integrated in a different way, in a far more work-intensive way. So there's still a bias, I think, in publishing towards language,

rather than toward diagrams. The bias isn't simply the ease of printing. There are all kinds of reasons. But, I mean, language is great, so I use it quite frequently. But I do think that diagrams go a long way to conveying conceptual information. There is reams of experiments supporting that,

Now, diagrams in science, even in history for showing time and dynasties and so forth,

They can be well-crafted, just as text can be well-crafted or poorly crafted. So you have to take that into consideration. But well-crafted diagrams are superior to language for explaining many, many, many kinds of information more directly, more succinctly.

Diagrams also allow huge numbers of inferences. So, I mean, Venn diagrams for logic, you have intersecting circles, and you can make a great deal of, a huge number of inferences just from that diagram that would take many sentences.

to describe. And even those sentences you have to put into a conceptual framework. And the Venn diagrams, the intersections, give you that conceptual framework, just like a phylogenetic tree will give you those inferences directly. They're compact ways of incorporating huge amounts of information.

So certainly on the teaching side, diagrams can help enormously in almost any field. Again, they have to be well thought through, certainly in data, in bar charts and line diagrams and so forth go a long way to allowing many inferences about data and to allow the more direct way of expressing it.

But we found that producing diagrams is a good way to learn. A study with Eliza Bobik looked at junior high students learning chemical bonding, and she taught them. She was a teacher in the school. She asked many other teachers who were teaching the same sort of material to participate and

So she taught the material in the normal way. You have text and diagrams and animations and explanations, teaching chemical bonding. After a couple of days of learning, she tested the class to see how they were doing.

and then divided them into two groups. The groups were equal on that test. Half the students were asked to give a verbal explanation of chemical bonding, which is the normal way of testing. The other half were asked to give a visual explanation.

And then everybody was retested. What's interesting is that everybody does better on the second test, even though they're not learning new material. The fact that they have to create an explanation helps consolidate and make sense of

the information in their own minds, so that everybody's doing better on the second test than the first test, even though they haven't had access to study materials. But what interested and excited us was that the group that had made visual explanations did far better than the group that had done verbal explanations.

And we expected this for the structural information. We divided the information into information about the structure of the molecules and the function, the action, the causation. And we thought making a visual explanation requires you to put down the structure, but

Surely students will put more of that information into the visual explanation than the verbal, and we found that. But to our surprise, we also found better representations of the behavior, of the function, of the causality in the system, in the visual explanations, than the verbal.

So we started thinking, why does this happen? And one reason we think is that making a diagram gives you a platform for inference. You can make inferences about the behavior from the structure. So you've got the structure, you can make inferences. Another is that it gives you a check for completion.

you know all the parts are there because if they weren't, it wouldn't be coherent, it wouldn't make sense. So having the diagrammatic representation gives you a check for the coherence and the completeness of the information. One word after another doesn't do that as easily, the check for coherence and completeness.

So we thought those were the reasons that the visual explanations did so well. They're a check for coherence, a check for completeness, and a platform for inference. They also force you to abstract, to find the essentials and put those down. Now, in some sense, language can do that too.

But it apparently doesn't do it as well. That getting things in this diagrammatic form really does force you to be more abstract and

So that abstractness has been found by other people in other situations. Dan Schwartz did a number of studies, again with junior high students, and they were supposed to be diagramming something like the nitrogen cycle. And many of them put in lily pods and frogs and that sort of decorative features. They made these lovely little diagrams. You often see them in books.

But when two people were working together, those individual decorative features disappeared and the more abstract stayed in. The diagrams can get you to the essentials better than a verbal explanation.

This is so interesting. There's three rabbit holes I want to follow up with here individually before we even talk about Zoom. So the first sort of rabbit hole that I want to explore a little bit is you talk about relationships and used Venn diagrams as an example. And some of the relationships that come to mind that I think you also used vocabulary around are structure and comparisons.

What are the other type of relationships that visual tends to draw out for people in a way that's sort of almost 3D and allows us to see something that was previously invisible? Yes, structure is important for physics, for chemistry, for social relations, for corporate relations, for business. Getting the structure down and only the structure can be really important.

The relations you write compare and contrast, and we do that quite nicely with gestures. We can set up a spatial schema on the one hand, on the other hand, and then I can keep pointing left or right, and you know that I'm referring to that sort of compare and contrast. I can order things differently.

So I'm deciding which automobile to buy or which vacation to take, and I can order the criteria, the most important things at the top.

and less important things lower down. I can order things left to right in reading order, and we typically do that. We put time going in reading order, which is left to right in most Western languages, right to left in some other languages, top to bottom in more traditional versions of Chinese and Japanese.

And I don't really know how they also have horizontal ways of ordering time. So we can order things. We can order things. We can group things into these networks, categories and subcategories.

We can make dimensions, and we do that. So we can have ordinal scales, which are just ordered. We can have interval scales where the difference between items is

is meaningful, and we can certainly do that on a line or either vertical or horizontal, and then more absolute scales where there's an absolute zero, as in temperature, money, and we can make, so we can make scales. So we can diagrammatically represent many of the ways that we order the world, organize the world, and measure the world.

I think probably all of that can be done in a diagrammatic way. Certainly, math and statistics do it that way. If you look at even formulas in math, they have a diagrammatic aspect. Things like equal signs are arbitrary characters. We put

two sides that balance on either side of the equal sign, so that's a spatial array of equality. So we see those spatial visual aspects even in writing. We indent for paragraphs, we space between words.

We make more important ideas in bigger font so that we're using those features that are visual-spatial, even in print, where it's almost strictly linguistic. Of course, there are analogies in the way we speak.

We use intonation, we raise our voices, lower them, we pause in dramatic places. Those are all kind of almost spatial devices that we use to convey meaning more effectively. That's fascinating. I think as you were talking about the text, it's really interesting because we use the space to draw attention, maybe not to relationships, but

to heighten our focus on what's important or what the author thinks is important and wants to convey or communicate with bold, with sort of fonts, with spacing.

Right, and we make tables, even in text, and the tables, again, are diagrammatically displayed. Truth tables, this was an observation by Reviel Netz, who's a really brilliant, creative, classic scholar, has studied Greek mathematics, but he pointed out that truth tables in logic, which were developed by the ancient Greeks,

are tables, that things are lined up in columns and rows, and that that enables you to see the relationships much more clearly than if they were in continuous text. So does that mean that when we're writing in continuous text or we're writing a book or a document that we should use some sort of visual to summarize or reinforce what we've written in

not only allowing the reader to sort of scan and find the relationship and then reinforce that with maybe more detail and nuance in the writing, but also to allow a bit of spaced repetition between what we just told them in words and allowing them to see it visually. Oh, yes. And again, it's there anyway. Chapter headings are usually lower than

And paragraphs have some sort of parallel structure in them, or were taught to use paragraphs in that way. So the visual devices are there anyway, but using them more to advantage is probably a good idea. I've become a huge fan of graphic books.

Exactly because they integrate so beautifully text of various forms, not just words. There are all kinds of textual components to comics that aren't traditional words, but also visuals of many types, and they can be integrated seamlessly together.

I've tried to do that when I write articles, to not have table one, table two, each with a caption, but to have it integrated into the text, and editors never let me do it. It's going to happen soon because there are more and more graphic books, and I think people are seeing how beautifully they can be designed and what artistry they

both conceptual and visual, go into it. So one of my favorites is Larry Goenig, who's written cartoon guides to various forms of math, to the environment, to chemistry, to calculus, to physics, and so forth. Anything where there's an accepted body of knowledge, he'll do a graphic guide to.

And the books are enchanting. He uses diagrams in ways that textbook authors never thought of, building them up from concrete to more abstract, then using bits of the diagrams as icons throughout the book to remind you and bring you back. They're mnemonic devices that are conceptual and bring you back to

So I think more and more books will be crafted this way. I see more and more young people learning the rudiments of drawing because the drawing that you need to do for art

for explanations or even for narratives are quite different from the paintings you see on the wall or the drawings you see on the wall that are standalone. They have to be integrated and they have to, again, abstract the essentials, convey it in an appealing way.

So what graphic books do is what ordinary conversation has. And that ordinary conversation is multimodal.

We are using words, we're using gestures, we're using intonation, pauses, many, many devices beyond words in a row to convey meaning. We interrupt ourselves, we interrupt each other. And all of that multimodal conveys a message. And if you...

Look at people's spontaneous explanations in conversation and take out the visuals. So you're taking out the gestures. A lot of the meaning goes there.

If you take out the intonation, a lot of the meaning goes. If you turn it just into words, a lot of the meaning goes. And because we're so used to learning from words and explaining in words, written ones, not ones in conversation or words,

And we get quite good at that. I think we become blind to the multimodal ways that we convey information. And graphic books use that.

They can convey music, they can convey smells, and they do it all through kinds of symbols and so forth. I can't do it now, but I've collected, along with John Bresman, who was a former student who was a comics geek, just brilliant with comics, we've got a huge collection of innovative ways that graphic books can convey, meaning they're poetic,

and inspirational and multimodal in this way. Did gestures override language too? Like I'm thinking of a scenario where somebody says yes, but they're shaking their head no. Right. And my husband, who was a paratrooper and had an uncanny sense of direction,

Right and left in any language eluded him, as it does many people. Many people confuse right and left for good reason. And he would point one, he'd say, go right and point left twice.

and I knew follow the gesture. It's not, the language is not, it comes later. Gestures precede language, and they're more in tune because they're more direct.

Susan Goldenmeadow and some of her colleagues, she's done gorgeous work on gesture, especially in children, looked at children on the cusp of conceptual thinking. They're going from this sensory motor.

So there are, you get two cups, one is thin and one is fat and pour water from one to another. And children who are small think that pouring the water changes the amount in the glass.

And you ask them to explain. And children on the cusp of getting that idea will gesture something like pouring before they can articulate it. So they've got the concept of conservation, that pouring water doesn't change the quantity. They've got the concept. They can't say it in words, but they can show it in gesture.

So their understanding is ahead of their ability to articulate it, which makes sense even for us.

not just for children. The articulation is an additional layer that's really quite difficult. Do gestures also help us remember? And I'm thinking like the last time I was lost when somebody explained directions to me, I'm just remembering that two things that happened when I was lost, I turned down the volume to increase my focus sort of on what I was doing. But the second thing when somebody explained directions is I was actually like

okay, so up and right. And, you know, sort of like I was using a gesture to help me remember. No, absolutely. They help you remember. And part of it is a different code and adding codes adds redundancy and that adds retrieval ways of retrieving information. And so certainly adding another code, but again, because it,

Your gestures are directly representing going up and turning in a way that language doesn't. And I found...

getting lost in many places and asking for directions in languages that I don't really know. Watching the gestures helps. And people don't say things in words that they do in gesture, like going up or turning slightly, that a road bends. They won't say that. They might neglect saying that, but it'll be there in their gesture. But certainly mimicking gestures

And there's some evidence from the neuron mirror work that we're mimicking even without externally showing that we're mimicking.

So there's a lot of work on understanding through enacting. I mean, there was old work on language that people understood language by evoking it internally in themselves. I have a good friend who now repeats what I'm saying.

in order to understand it. Now, he doesn't do it out loud, it's very softly, and I've seen other people do that, that as they're listening, they're doing a little bit of talking, so they're externalizing that internal information. But there's beautiful brain work on watching dance,

videos of dance. And you can take experienced dancers, either say ballet or capoeira, this very acrobatic South American dance, and have them watch videos of ballet or capoeira. And in both cases, the motor areas of the brain will be active, but they'll be far more active for the dance that you can enact.

than the one you can't, again indicating that something in the body, in the brain and probably in the muscles, there is evidence from other work in the muscles.

is resonating with the action that you see and incorporating it into your body. And probably when you're watching someone giving directions, you're doing that. Certainly for direct physical activity, you know, a tennis serve or a dive that you're going to be doing,

incorporating that into the muscles and your brain. One of the things, as you were saying, that gestures sort of override language, when you're using the example of giving directions, but, and your late husband in terms of pointing one way and saying a different way and realizing that gesture was more, that was, it was more right. It was more true because it was lower language sort of in your, in your,

in your sense or your body or your brain than language. So it comes before language. And that speaks to the whole notion of body language when people are communicating, when they're saying one thing, but their physicality is saying something different. To some extent, we do learn not to express certain emotions.

because we know that people can read them, and people read them very quickly, and in many cases without awareness. So we do learn to suppress some of that, not to show that we're afraid or unhappy or upset or depressed. We learn to

to suppress that at some cost probably. And how much, how good we are at it varies with people, how much we want to suppress it. But there is an awareness, I think, on the part of most people that the body language does communicate. And sometimes we don't want to communicate certain aspects of the body language.

Some of it, you know, I become more and more aware of it as I study gesture. And, you know, someone wants a meeting to end. They stand up and that's a signal. If you're around a table, who you look at next gets the floor.

and women aren't looked at. So there are aspects like that that are social, of how we interact socially, that again have effects, the body language has effects on the conversation, on the speaker or the communicator, and on the communicatee.

So as you were saying that two things stuck out, one, you said that we don't tend to look at women. I'm interested in sort of exploring that a little bit. And the other thing is, how does Zoom change this? So what are the positive and negative aspects of communication over something like Zoom versus face to face? And how does communicating over video, which sort of changes our spatial perceptions, affect what's communicated and what's understood? Right.

Like face-to-face communication uses our whole body and even the language we use, right? We say this or that, or we look at something to draw attention to it. Yeah, no, I'm with you completely. And the research needs to be done and probably is being done. And obviously it depends on what's being communicated, right?

A big thing for me is that we're only seeing face and shoulder, so we don't have a shared workspace. And if you look at designers or planners or any kind of committee, you often have a whiteboard or something that's central on the table that everybody can point at.

and add to and subtract from, and that a great deal of the communication and the joint understanding is externalized in that way.

And we've done a small amount of work, and others have too, on collaborative design and found that when people can work over the same diagram, the communication is much more successful. People are looking at each other's hands on the diagram, not at each other's faces. The words are annotating that diagram.

They aren't independent. So people are more, the communication is better, the result is better, the product is better. People are happier with the collaboration when they can do that than when it's just over the phone and we have separate diagrams. So I think for any kind of collaborative work, and teaching is often collaborative,

Conversations certainly are, but in many cases we need that external representation of our shared understanding. And there's a huge amount of research and practice on dissolving conflict that way, that if you take conflicting parties in an organization, between organizations, put everything up on a whiteboard with a facilitator,

And then you realize the points where you misunderstood each other. You use the same words to refer to different things. You can come to a consensus much better and feel better about it than if you don't have this external space that you can all, your joint thinking space out there in the real world. So that's a real drawback of Zoom.

Now, I can talk and lecture with my PowerPoint, but I can't animate it as well as I can in a classroom where I'm standing and moving and pointing and gesturing big. I mean, you make big gestures and they look like these giant hands and they're disturbing, whereas you don't have that in real life.

So these are some of the disadvantages, not seeing the nuances of facial expression, not hearing the nuances of vocal expression. But I also found in teaching students, I have many students for whom English isn't the first language.

So it was easier. And being in a classroom with masks, I couldn't have understood them. They couldn't have understood me. So Zoom for the pandemic had that help. I could also put up transcription.

So that if I didn't understand their accent and they didn't get all my words, they at least had that other code. I mean, transcription is funny sometimes, but it is self-correcting. It's quite remarkable. So that helped.

What also seemed to help, interestingly, my classes were around 20. So everyone, if I wasn't speaking from a PowerPoint, everybody was there on the screen. We're each a box. We're equal. So it seemed to encourage more discussion from the students. Everybody could see everybody's faces, whereas in a classroom, everybody's facing me.

And I'm facing them, but they don't see each other. And I'm standing, they're sitting. So I get a primacy as a teacher that I don't have if I'm a box and a screen. So it seemed to have equalized, democratized, encouraged more students to just treat me as another box.

I think other instructors had a similar experience. If your class is 140, that doesn't work. The breakout sessions worked, again, for teaching. I don't know. In corporate situations, they might work.

But it allowed the students, in fact, it gave them more opportunities to interact with each other than being in class would. And that was enormously facilitatory, the breakout. So there were features in Zoom that I could use teaching.

Again, I can't speak for the corporate experience. I should ask my sons, both of whom had, you know, lived from one Zoom meeting for another for several years. But, yeah, it is fascinating. And I think...

The need for gestures and a shared workspace is something that I hope designers of these media know about and will make efforts to correct. There's only so much you can smash into a screen.

So then just having the smaller space is in until we can do it surround and in VR, then that will go a step further. Sounds like one of the important aspects of Zoom. I guess the two sort of things that we can take away from that are one,

Hopefully it encourages more diversity of opinions to be expressed rather than the same people expressing opinions over and over. And the second thing that I took away from that was that if we're going to communicate or facilitate over Zoom, then a shared visual space is super important, allowing people to not only focus on something, but also see and view representation.

Right. The second rabbit hole that I wanted to, I promised three rabbit holes before we get to the sort of like next big theme, but we did, we did explore zoom, which was good. You mentioned that we, when we create a diagram or a visual representation, what we're doing is we're consolid, I'm trying to use your words here. We're consolidating and we're creating an explanation. And what that drew attention to in my mind was the process by which we learn to

I call it the learning loop. And so my belief on this is that we don't learn from experience. We learn through reflection. So we have an experience. We reflect on that experience. And from that reflection, we draw abstractions or relationships. And those abstractions and relationships become reality.

an action that we take the next time we're in a certain situation. The underrated part of that is the reflection, because the reflection is the distillation of all of this information into its salient points, into the relationships that matters, the variables that matter. And maybe this sort of goes to my third point, which was a third rabbit hole, which was it forces us to abstract

I'm curious as to how you think about that reflection component in terms of not only learning, but in terms of how we encode this information in our minds. Yeah, no, I think all your observations are astute and couldn't agree more.

And there has been some research on that. So sheer rehearsal of material we're trying to learn isn't as helpful as trying to draw implications, make comparisons, do the sorts of reflections that you're referring to. And there again,

Just as you asked me, what can you put in diagrams? And I said categories and hierarchies and themes and one-to-one correspondences and much more.

orderings, that those are the sorts of reflective processes that we might undertake. There has been research showing that if we want to transfer learning, and of course, that's one of the goals of learning, is taking more than one case that are similar and

and abstracting what's similar about them is going to help you transfer. A single case isn't going to help you. Two cases are better. And two cases and a diagram indicating the relationships is even better. So that in making those abstractions that can then get applied to new situations.

It's hard to go from abstractions to instances. It's even hard to go from instances to abstractions, but probably from abstractions to instances is even harder.

And I've seen so many phenomena in cognitive psychology be shown over and over again in particular situations, because until they're in a particular situation, no one believes them. And I see them and I say, it's just the same as X. What's new here? Because I've got the abstraction. But other people need the concrete showing that particular thing in this situation.

Same in politics, same in history. We have it all over.

So going from instances to abstractions is one thing. It's not that easy. And more than one case, drawing the similarities and diagramming the abstraction helps you get to that abstraction. But getting from the abstraction to the particular is still a challenge. And I can't think of people who've worked on that, on teaching that.

I wanted to say one more thing about education. Most of our education is vicarious.

We're not encountering. You said we learn from situations. The situations for us, schooled people, is classroom in many cases. Sometimes it's actually experiences. And of course, classrooms that create experiences through laboratories or field trips or whatever are more effective because learning from experience is effective.

But so much of our learning is vicarious and through language or other math, other abstractions. So making it concrete is important. Just to riff on this for a second, experiences don't need to be a physical experience. An experience can be reading somebody else's experience. I'm experiencing it. A conversation is an experience.

You know, reading a textbook is an experience. Doing something like building a sauna or a shed or something is an experience. So these are all different types. The way that we have an experience is very different from how we reflect on it and how we abstract it. And I think that the experiences don't necessarily need to even be firsthand. I can learn from your experiences, but this gets to a different point here, which is,

When we consume information, a lot of the times we're consuming the abstraction. So these diagrams are the abstraction, they're the salient points, but they're one person's idea of what the salient points are. That's not my encoding of it, that's their encoding of it. And I think of consuming other people's abstractions as the illusion of knowledge.

I'm the line cook, not the chef, right? To use an example from a friend of mine, Tim Urban, which is, you know, when things go right, I look like a genius. I follow the recipe and everything works out. And because I've consumed the abstraction, the recipe in this example, everything turned out right. I look like I know what I'm doing. But when things go wrong, the chef who has distilled

the experience, the reflection to create the recipe that has gone through the thousands of trial and errors will taste it and be like, oh, this needs more salt, you use too high of heat. And instantly, they will know what the problem is, because they've, they've got the abstraction, but they've got the, it's almost like an earned abstraction versus a learned abstraction.

And that's really interesting because it changes when we're consuming information. We're consuming a lot of these abstractions. We assume their knowledge and we assume there are knowledge and we put them into practice. And when they don't work, we don't know why. And the way to go back up this chain is to actually decompose it and ask people, okay,

well, how did you come to these conclusions? What variables were important to you? And you need firsthand knowledge, not second, third, or fourth hand. So you want to get as close to the person who had the original experience or created the abstraction. And often when you pull back the onion here, we're two or three layers deep, and it's a very sort of weak relationship. No, no, you're absolutely right. And you think about teaching a

sport or piano or diving tennis serve any of that you can give verbal instructions but you've got to do the serve over and over and over again and the dive over and over again because translating that language that abstraction into the body into your body is

is going to be very different. And a good coach can watch you and see the tiny place where you're leaning forward too much on the dive or breaking the jump too quickly so that you're not jackknifing fast and high enough. A coach can see those minute things and tell you, and then you have to work

to incorporate it into your body. And a similar thing will happen on cooking.

I'm listening to when the boiling of the rice, I'm smelling the olive oil on the saucepan and I'm doing as it's heating and I'm picking up the danger sign, got to lower the heat. That's only from a great deal of experience and paying attention to the smell and the sound of those cues. And that isn't in the recipe. And you will go crazy if you had to pay attention to those details.

in a recipe or an instruction to dive, you'd give up. The abstraction you get either in a diagram or verbal instructions isn't going to enable you to be a champion diver or a great cook. You can make satisfactory things.

And you need to know not just adding salt, you need to know when to throw it in the trash and start all over again. And, you know, potters know that. Something doesn't rise properly and it's hopeless. Go start again. And I think any creator knows when things simply can't be corrected and when they...

when they can be corrected. And again, a good coach can tweak, can use language to tweak and abstractions in language to, how does it feel in your body? You know, your body should feel this, the muscle tension, because a lot of that isn't visible. It's, I mean, I learned a bit of yoga and that sort of thing. You have to really feel it in your body and which muscles

are working and which ones aren't to know are you doing a proper stretch and are you really stretching the muscles you need to stretch.

Weightlifting is similar. You have to get it into your body in that way. So an example I use sometimes in navigation, so I talked about sketch maps getting you to one place or another. So even though sketch maps could be analog, they aren't. They don't give the exact distance and they don't give the exact angle of turn. They're approximate.

And they're approximate in the way that verbal instructions are. You say, "Make a right, turn right." You say, "You get to this street and then turn." We don't specify either in a sketch map or verbal instructions the exact angle or the exact distance, but those are meant to be used in the environment.

So if you come to an intersection and it isn't 90 degrees, it's less or more, you're going to go the way the road goes.

So you use that information as if it were approximate, and you know it's being used in coordination with the world. Same with distance. You're not measuring the distance. It's until you get to that next intersection or highway cutoff, and you'll go to there, and you know approximately what the distance is, but you don't need that complete information because it's meant to be used in a context.

And I think most instructions have to be that way because, again, the detail would drive you crazy. And you have to assume some knowledge of a cook.

or some knowledge of a piano player, for a piano player to read the notes and translate them into actions of the hand. And again, where it goes wrong, a good teacher can supplement. Or hearing yourself, you can be your own good teacher.

You know that dive didn't work out quite right. What were you doing? And the piano, and you do it over and over again until you get the right sound. And we do that speaking. Learn a foreign language. What's the accent? And I can make words and sentences in languages that I don't know that well, and I know they sound off.

And can I get the accent right? And I go over and over. And I can even hear it in my head that I don't have it right or that I've right. There's something missing. But then you're reflecting, right? So you're creating your own abstractions, which I think is, I think one of the takeaways, if maybe you can challenge or push back if you don't agree, is that abstractions are necessary. We need to consume other people's abstractions. We would get overwhelmed in the detail of

And a different way to think about that is earned versus learned knowledge. So this is learned knowledge versus earned knowledge.

Earned knowledge involves, you know, you can't necessarily be taught all the details. You have to sort of do it or embrace, you know, dive into the details. And I think where this becomes important in life is to think of decisions, for example, and cost of failure. So when the cost of failure is low, it's okay to use learned knowledge. And when the cost of failure is high, you want to make sure that you're not using learned knowledge and you're using earned knowledge.

Or at least if you think of those things on a continuum, you want to be more towards earned knowledge, the more the higher cost of failure. But I'm just throwing that out there and sort of pontificating on the spot. But I'd love your reaction. So I wonder about that. Some of the high cost decisions we make, we can't have had experience choosing a spouse. Oh, but trust. Hold on. You have thousands of experiences with trust with people all

all over your lifetime. Sure, but it's more than trust. It's really a huge amount of trust, a lifetime of trust. It's trust about your physical, your body. It's trust about joint decision-making, how we're going to grow each of us in life. If we haven't had a family, it's how we're going to do that. You can't accumulate trust.

And enough experience is the same with getting a job. You can't try many, many different jobs because jobs are so complicated. What you do, how you relate to the people around you, how you relate to your boss, what happens when things change. And there I think the vicarious, I mean, people talk about gossip.

as being informative and hearing about troubles that other people have gotten into, either in work or love.

And trying to map that on your own life is, even babies, you know, after our first kid, a friend came over, a guy deeply ingrained in economic theory came over and wanted to tell us details of how to care for this baby in diapers. He said, you learn so much, you want to pass it on.

But so all of that, and, you know, new mothers, new parents talk endlessly about crying and feeding and diapering and so forth, things that...

that ordinary adults would find boring. So I think even on big decisions, the vicarious, you know, how to raise children, who you should get involved with, when you should leave, all of those things, we need other people's stories.

as a way of making the big decisions. As you're speaking, I'm thinking you're probably right. And there's a huge body of knowledge that comes from avoiding problems that are learned knowledge and not necessarily earned knowledge, because you don't want to, you don't have enough time in life to earn all the knowledge from all the problems. And so you have to adapt and incorporate learned knowledge. And yeah, I think you're onto something there.

I want to switch gears to part of your book. You talk about the nine laws of cognition. I think they're more like nine generalizations, but let's start with the first one. There are no benefits without cost, meaning creativity versus learning. How do they offset each other? So the big example there, I think is one big example is categorizing things.

So if we awoke anew as a newborn each day and didn't know how to categorize things into animate, inanimate, parent, friend, chair, table, these sorts of things that we do regularly.

that our brain does quite quickly. If we didn't know how to make categories very quickly and had to depend on new learning every day, we'd never advance beyond that new learning.

People wonder when they have a baby, what are they looking at? What are they thinking about? They're doing a huge amount of learning about what the world looks like and acts like and sounds like and smells like and what happens in time. These are huge things that a child needs, a baby needs to learn. Some of it, the proclivities are built in and even more

maybe some of the kinds of knowledge, but a great deal of it, words, have to be learned. And if we had to go through that learning every day anew, we'd remain helpless infants.

So we need to categorize, but any kind of categorization has error in it. We can miscategorize. You know, I've mistaken twins for their sibling and said hello and wondered why they greeted me so coldly.

I think that's an experience many people have had. The extreme leads to biases that, you know, I'm a small woman, so alone at night, I'm very alert to any kind of danger. And because I'm hyper alert to any kind of danger, I can miscategorize what might be dangerous to me. But it's something I need to survive on the streets.

So we can miscategorize, but we need to do it. And so that's one of the costs and benefits of anything in cognition. And there's a reason for every one of these biases that is...

There's an underlying reason that's deeper, more abstract or broader where these things have been useful in one way or another. But their use is highly limited and can lead you into trouble. It's when somebody pulls a hand out of a pocket and another person thinks it's a gun. So it can lead to error and sometimes catastrophic error.

You can't live with them, you can't live without them. And there are just many cognitive and social phenomena that are like that, that the efficiency of, say, enacting an action over and over and over again, that you get very good at it, but you

you can apply that action, misapply it to situations where it's not appropriate, and then it'll get you into trouble. I think that's something people often don't see. The second generalization is that action molds perception. Right. And there I gave the example of the distorting lenses.

that we don't accommodate to lenses that distort unless we're doing the actions, and the actions actually mold perception, mold our perception in that sense. They do shift the world if we're acting in a shifted world. I mean, it can happen in...

In a more general form, if we decide that somebody isn't worth interacting with because they seem like, remind us of somebody who hurt us in the past, then we don't interact with them. So we're taking that, our very action...

Not only changes it changes our perceptual world. We won't include that person and that's another kind of error that we can make from from the first law of cognition that we get Experienced at knowing what kind of people we can trust what kind of people might hurt us what kind of people to stay away from? But we can make mistakes and that's a speed versus accuracy trade-off. Yep, and

And sometimes we have the opportunity to correct that mistake. That certainly happened to me where somebody whom I dismissed and somebody else said no, and I had new encounters and found that I was wrong and could correct my impression. But if I never had that opportunity to correct, I would have been left with that misunderstanding.

impression for a long time. So our actions affect the world we experience in big ways. We move to a new city. We take a new job. All of those things. We have a baby. All of those actions change our world completely. The third generalization is that feelings come first.

So that's a work of Bam Vazian and many others after that, that he showed in what seemed like random shapes to people and got affect, liking, and memory judgments. And people had...

The more people saw a strange stimulus, the more they grew to like it. So familiarity is good, or at least a modicum of familiarity. Too much might be harmful again. But they couldn't remember them. So the systems are somewhat independent.

Our affective reactions and our memory are somewhat independent, and the affective seems to work faster. And that, again, might be tied to danger and avoiding danger, that if there are things in the world that are dangerous to us, we better know it quickly.

and act quickly before we might remember or not that we might think someone's dangerous and withdraw before we remember, no, no, that person looks dangerous but is fine. Just because there's such a high cost to the danger, responding first to the emotional is probably adaptive. And keeping the system somewhat separate

is probably adaptive in any case, the brain does it. - And the fourth, which was one of my favorite ways of putting it anyway, was that mind can override perception. - Yeah, and that's where we know. And that'll be a kind of system two as opposed to a system one that we then, the person that looked dangerous, we then, next time we encounter, next time we remember that.

as an exception. When we're, this isn't perception, this is action, when we're repressing emotions because we don't want, we think revealing them will harm the other person or harm ourselves, we're using our mind to overcome our first reaction, which is to burst into tears. And bursting into tears or screaming, neither of them is going to help us at this point.

So that repressing it is probably a good idea. Does this work in reverse too, which is like that we don't want to see information that disconfirms our existing beliefs, so our mind walks out? Yeah, so you're pointing out nicely that everything has an advantage and disadvantage, and that mind overcoming can also lead us to self-reliance.

because we're trying to fit the world into our beliefs. I should have said that. I think I got it from one of your other interviews for what it's worth. I was doing research for this. Okay.

The fifth one was cognition mirrors perception. Yeah, and all of these things other people have pointed out, they aren't my invention. Many of the biases in perception, in perception we categorize, in cognition we categorize, so that many of the phenomena that we see in perception end up being cognitive phenomena as well. So studying both...

is useful. What happens in sheer perception, what happens in cognition, the borders are murky.

So, yeah. And let's just briefly go through the other ones. I mean, we can spend a little bit more time on the ninth one because I thought that was super interesting. But six was spatial thinking is the foundation of abstract thought. So there I think I spoke at length at the beginning on that, that the brain structures, the brain strata that encodes spatial or abstract

Information called relational information or spatial information because relational information can be spatialized for time, for social relations, for abstract. And again, we've seen it in language. So it's there in brain. It's in language with somebody who's at the top of the heap. People have grown close together. And we see it in language. We see it in gesture too.

We see it in diagrams, which again are specializations of thinking. - And seven is one I think we're sort of all familiar with, but is also invisible to us, which is the mind fills with missing information. - The information we get from the world is incomplete.

I mean, the work on change blindness, where you show two photographs in rapid succession and large things are missing from one, like the engine of a jet airplane, and people don't even notice it. So it takes a long time to notice. Even when you know it's an engine, it takes you a while to notice it.

So the world that we see is in the head, not out there. We have the sensation that it's out there, but it's in the head and we're filling in. When we make errors, we know that. So my vision is going and I will fill in shapes with a uniform color that it isn't uniform at all.

When I get closer and get better acuity, I can see that it isn't. But the mind forms, the visual system makes shapes and the colors are added the way you fill in the blanks.

And so I'm aware of those errors. And again, it's one of those things we couldn't function if the mind didn't fill in. We don't know what's behind us. I mean, I do because I can see myself in that. But if I'm in my office and my books are behind me, I can reach for a book. I know where it is.

So the mind is filling in constantly for what isn't there. In understanding sentences, we can miss a lot of words and fill in. Again, making errors makes us aware of how much we're filling in. The eighth and penultimate one was when thought overflows the mind, the mind puts it out into the world.

So there we put our mind and the world in many ways, in language, in action, in gesture, in making lists. We put the things we need to bring to work or to wherever we're going by the door.

so that we remember them. David Kirsch pointed out we lay out the materials for a recipe in the order that we'll use them on the counter, so that it's convenient to enact the recipe. So there are many, many ways that we put the mind in the world. Again, I think it's probably uniquely human. I don't think that gorillas make trail markers,

for how they should go on a path. I grew up in a small town, in a town that had been occupied by Native Americans. And the park I walked through to get to school had various trees that had been shaped as trail markers. So there was a huge oak tree that was an H. And

And there was another one that was bent into the ground, and they were trail markers so that people wouldn't lose their way. So we do it. I mean, birds mark territory. This I know because I used to run into the territory of a bird, and it would attack me, which was rather disconcerting. A friend who was an atheologist told me, wear a hat. That worked.

But birds mark territory. So there are small bits of that kind of externalization of mind. Bees tell each other where to go, but they're very limited. But they're still extremely interesting that other creatures do put their minds into the world and partly to communicate. They just don't do it in this enormously plastic way.

an inventive way that we do. The ninth generalization was we organize stuff in the world the way we organize stuff in the mind. So we've talked a little bit about how we organize things in the world. We categorize, we make hierarchies of categories, we order things, we make one-to-one correspondences in place settings.

And other situations. We organize by themes. Our houses are that way. Everything for cooking is in the kitchen. Everything for cleaning is in the bathroom. So these are things from different categories, but they're organized around a common function or a common goal.

And we do that in the world. So we have buildings that have one-to-one correspondences, windows for each apartment. We draw lines, which are roads, in the world for the paths we can go on and make intersections. We even diagram the world.

So that if you look at an aerial photograph of a city, you can see there are lanes for people, lanes for buses, lanes for bicycles, where people can go, where buses can go, where you have to stop, where you can turn, where you can't turn, where you can park. The world is communicating to us, not just indirectly in these categories and hierarchies,

furniture stores are arranged by theme. So the world is communicating to us not just in categories and hierarchies and themes and orders, cues for getting into the theater, but it's also telling us how to behave. So it's controlling our actions by the way it's diagrammed in the world, turn signals and turn arrows, etc.

And if you think about the world that we grew up in and that our children are growing up in, it's so different from the world of hunter-gatherers. I mean, we've designed the whole world. You fly in an airplane and look down and there's hardly a spot humanity hasn't touched and designed.

Now, other creatures designed too, there are beaver dams and territorial rights of birds and so forth. But we've diagrammed it and we've designed it in much more detail. And we've designed almost all the inhabitable world. So how about the inverse of that? How does the physical environment influence how we think? And if the physical environment does influence how we think,

How can we design that to improve our thinking? So diagrams are one way to improve our thinking in the small way.

There's certainly one, and even written text is a way of improving our thinking, making lists. We have so many, all those ways of externalizing mind can help bring it into the mind and improve it, especially if it's well-designed. When you think about the world, you know, I can go into a supermarket in any country, buy food and put it on the counter.

And that's a communication to the person at the cash register that I want to purchase those items. I don't have to talk. It's the way the world is designed and it's universal. Now I have to go to the automatic cash registers and find the barcode and scan it. And I can do that now in any country except China because I can't pay anywhere.

because I don't have the right app on my phone. It does change our actions in many ways that makes it more efficient. I can't help but think that babies growing up in a world where their homes are organized by themes and where books are on a shelf in an order, where toys are kept in different categories and so forth,

that that can help it influence their minds. And I wish someone would study that. You know, I remember my own children, and if they're in the living room, everyone's talking and playing, and I'd pick up a child and walk toward the bedroom. They'd scream because they knew they were being taken away from the social environment and put to bed. And so they knew.

from the organization of the house, what was going to happen. If they were brought to the kitchen, they would calm down because they knew they were going to be fed. Children growing up with that kind of organized, designed world

are certainly using the way the space is designed, the way that hunters and gatherers used the natural design of the world. And people use the environment as a way of organizing their behavior, what to approach and what to avoid.

But the design world now seems to communicate so much more. Where's the door to a building? Where do you put your car, your bike, or your stroller? And where is the food? And so forth. So all of that, it seems to me, is changing or is affecting the cognition of everyone around. You know, when...

So I have had the pleasure of visiting Japan many times, and there are huge roads that intersect, and at first you sort of go along them, but they're unpleasant. And then you learn everything's done in quadrants and quadrants inside quadrants. So you start walking inside the quadrants, and it's a completely different world. And then in Japan, you see a torii gate, and you know that's going to lead to a temple.

Now, that won't happen in New York City or Paris.

So then you have to navigate differently. You spent a lot of time diving into the work of Leonardo da Vinci. I'm curious as to what you learned. That was a wonderful opportunity. An incredibly talented musician friend wrote an opera on da Vinci for the 500th year of his birth. Unfortunately, the performances, the pandemic interfered as it did with

with so much of our lives. It's Jonathan Berger, and Jonathan's at Stanford, and whenever he has a new opera, and his operas are nonfiction based on real lives and really enormously moving. There was one on hallucinations that stays with me years later, on personal experiences with hallucinations that were gripping me

And he turned them into opera. And one of the people whose experiences he turned into opera said afterward, Jonathan conveyed my anguish better than I could have. So very moving. He made an opera on da Vinci. And whenever he does an opera like that, he has a symposium where he invites people with different views on the topics to speak.

And I had the honor of being invited and used the opportunity to talk about da Vinci's sketches. And there are millions of them, not millions, but thousands of them. You can find them, thankfully, online in many cases, because in many cases they're in libraries all over the world.

And he invented uses of sketches that weren't reinvented until many years later because his sketches were unavailable. So compare and contrast, you've talked about that. He did a great many like that of heads and then skulls and then the brain structures inside. He was looking for the soul. And that was the kernel of the opera story.

was looking for the soul and looking for the soul in the brain. And thought he found it at the confluence of sensory pathways. So there are cutaways. There are compare and contrasts.

You see him, and this is a phrase I'm borrowing from a catalog on an exhibit of Raphael that's currently at the National Academy of Arts in London, where somebody pointed to Raphael's sketches as thinking out loud. And that's what da Vinci's sketches were. You can see on the same page many sketches from different perspectives. He's thinking out loud how to prepare the painting.

So he used these sketches in especially the cutaways and the compare and contrast in ways that, again, medical illustrators didn't come up with till centuries later and were truly inventive. Now, I haven't seen sketches of other creators yet.

at the same time, but da Vinci was unique in being a creator of art and a creator of architecture, a creator of implements and so forth. So he was an engineer and an artist and he created events. The talent was explosive. So, and similarly, the use of sketches was,

to inform him because they were for him, not for show. Although in many cases, they're as beautiful as the paintings because of- It was his way of understanding. Exactly. Thank you. Yeah. It's his way of making sense.

And you can certainly see that today. Many people making sketches, it's their way of making sense of things, of data. You display it in different ways to make sense of it. Yeah. And then eventually how to show it, how to use it to communicate. In your writing, you've used the term cognitive collage. What did you mean by that? So people have talked about cognitive maps, right?

And a term that came from Tolma, who showed that rats' learning routes could put those routes together into a map of where things were relative to each other. And of course, the beautiful work on place cells and grid cells has shown that conclusively, that rats have ways of organizing places in a space that's viewpoint independent.

and then navigating in it, or using that to navigate. So we studied, along with many other people, various errors that people make systematically in making judgments about spatial relations, distance and direction in particular. So people think that Philadelphia is north of Rome,

when it's actually south of Rome, and it seems to be because people line up Europe and North America, and Europe is actually farther north than North America. And anyone who goes to Europe at this time of year knows the sun sets very late.

in Paris and Rome compared to northern American cities. So people line them up and then they see that Philadelphia is a northern part of America and Rome is sort of south in Europe and use that to infer the direction between Philadelphia and Rome.

People think that an ordinary building, say Jacques' house, is closer to a landmark Eiffel Tower than Eiffel Tower to Jacques' house, and that violates any metric assumption. And it seems to me because the Eiffel Tower defines a neighborhood. You might say, I live near the Eiffel Tower. Unless somebody knew Jacques, you wouldn't say, I live near Jacques' house.

Landmarks in any city will tell you, I live near NYU.

or I live near Columbia, I can say from New York and people will know about where I'm living. And so Columbia and NYU for academics define a neighborhood. So ordinary things are drawn into the space of a neighborhood, of a landmark, and that defies any metric representation.

Eleanor Rausch has shown that in judgments of color. Magenta is closer to red than red to magenta. And again, it's similar. Magenta is a form of red. Red includes many shades of red. So she'd shown that in prototypical categories.

that things are more similar to the prototype than vice versa. My husband showed that in social political situations, people think a son is more similar to a father than a father to a son. People think or thought years ago, probably think again now, that North Korea is more similar to the PRC than the PRC to North Korea was.

So you have those asymmetries in conceptual relations as well as in spatial relations.

And there are many other errors like that. Milgram, famous for his work on punishment, spent a sabbatical in Paris and found people straighten the Seine, which makes them take longer routes in many cases because the Seine actually forms a kind of upside-down crescent in Paris if you orient the map north.

So there are many errors like that that have been documented. And if you put them all together, you don't get a coherent map. You can't. If you add to that maps that we know about environments from maps and use them to navigate or memory of maps to navigate, we use verbal instructions, go up in that street until another street, turn right.

We're mixing all that information and coming up with a judgment of how far things are, what directions things are, how long it will take me to get there, which is confounded by means of transportation. So any of those judgments, we're pulling together disparate information. How do we combine it?

And it's not going to be combined in a two-dimensional or even three-dimensional map, nor do we have them stored in our heads. We have partial information stored in our heads. And making it coherent is by no means easy.

And so I called it, I said the analogy is probably to a collage. It's multimodal. It isn't coherent in a metric sense, but maybe more beautiful than a map.

I want to explore one final topic today, which is the importance of perspective taking in terms of how we interpret things. And I know you've done some work where you got people to think creatively. And one of the ways that you got them to think creatively was to ask them to think about the different roles of people. So like, how would a doctor use this? How would a gardener use this? I wonder if you can talk to me a little bit about that. Yeah, sure. We've done a lot of work on perspective taking, starting with...

With the map situation, are we taking an overview perspective or embedded in the environment? And that is really hard. We work with Google Maps and we're seeing something. How do we reconcile them? There's been beautiful work on the brain on that and how the brain might do it by Russ Epstein at Penn and undoubtedly others. So there's perspective taking there. There's perspective taking in

Should I take your perspective or my perspective when I'm describing things? And if you're in front of me, I generally take your perspective because I understand and your cognitive load is higher.

So changing perspective is key in many, many places in our lives. We found that even in situations that are non-communicative or where you and I have the same perspective and we're looking at somebody else acting, we take the perspective of the actor instead of our own perspective. And I think it's a way of understanding their action

is taking their perspective, and we do that more quickly and automatically than taking our own perspective. So that was surprising to many people who assumed that an egocentric perspective was primary and overcoming it was difficult. But by now, that experiment has been replicated by many people in many cases.

We talk about social perspective taking and taking the perspective of others. It's probably, it seems to be different from spatial perspective taking. And there we found it useful in creativity. So there's been a lot of work on how to get people more creative. And some of the work has been done by a group advocating mind-wandering.

Just let your mind go. And we thought, thinking about creative people and how they work with their own minds, that they don't completely let their mind go. And that mind-wandering can help you get out of fixation, which is a real problem in design, that you keep designing the same way. You can't see new uses for old objects anymore.

And even experienced designers fall prey to fixation. Mathematicians, product designers, architects fall prey to that. And it's partly that first cognitive law. Those old design tricks work.

and they've worked over and over again. And learning depends on learning those old tricks, but if you want to do something differently, then you have to get out of those old tricks. So all your learning has a cost, and you want to get out of it. So MindWarner can release you from fixation, partly because it brings in new stimuli with new responses.

But it doesn't give you a productive way of coming up with new uses. So the test that's typically used in creativity and even in engineering design courses and graphic design courses is to come up with new uses for a brick.

And it's a kind of warm-up exercise, like doing scales on a piano. And it's often used in design situations. Come up with new uses. How do you do that? So we started thinking deeply about how people come up with new uses. We looked at sketches, and that is one way, active way, that designers come up with new uses is they make a sketchy sketch.

They look for one reason, they look at it, and because it's full of ambiguity, they can reinterpret it, and they see new things in it. So they're designed for one reason, and the messiness allows them to see new things. So we knew that would help, and the new things turn out to be new perspectives, new really ways of organizing the visual, whatever you're seeing visually,

So we thought maybe that would be a way of increasing finding new uses, that just letting your mind go might help release you from fixation on these various objects, bricks, umbrellas, ping pong balls, and so forth, but that they wouldn't be a productive way of coming up with new uses. And then we started thinking, what are natural experiences in life?

So we experience people, we experience places, we experience events. And all of those seem to lead to new uses. But what seemed to combine all of them are roles. So adopter is a different person or a particular person in a particular place for particular events.

A musician, same thing. A gardener, same thing. That each of these roles in life is associated with particular events, particular equipment, people, places, and forms of interaction. So roles capture a huge amount of information that we know about just from living. Children are asked from an early age, what do you want to be when you grow up?

So then their books are filled with firemen and policemen and doctors, roles they might encounter.

So we knew people knew a lot about roles. So we gave people different objects, asked them to think of new uses. They were things, again, like umbrellas and ping pong balls. We pretested them to make sure that people could think easily of new uses. And one group, we said, just let your mind go. We used the instructions from the previous work.

Another group we just told, "Think of new uses." We didn't give them any particular instructions. That's the normal task. And the third group, we said, "Think of different roles. Think of what an artist might do with it. Think of what a gardener might do with it. Think of what a policeman might." So we gave them a list of roles and said, "Think, use these. Think of new uses. Think of their roles."

And then we looked at what we got. We ran this on Mechanical Turk several times, and that gives us a broader range of humanity than college students, and we did get a broader range. And the most effective strategy was roles, by far.

In fact, the mind-wandering didn't differ from the no-instruction group. And when we asked people, how did you do it? The no-instruction group said, well, I just let my mind go. But the roles group came up with far more new uses. They came up with far more new uses that were unique or rare.

So we tabulated all this work with Julia Chow, and she was just a beautiful eye for data. So she tabulated all the uses. She looked at the number of people who came up with them. And for the uses that only one person or only three came up with, they were in the role group. The role group persisted more.

Their ideas came later, the unique ideas. And doing the task, it's almost as if you have to get rid of all those ordinary ideas before you can think of the unique ones. And then once you're thinking of the rarer or unique ones, you need a way to go back and try again and a way to go back and try again. And Juliet was able to show that at least half of

of the ideas they came up with were connected with the roles that we gave them. They also invented roles. They got the idea, think of new roles, and they invented roles.

often imaginary creatures or creatures from stories or animated cartoons or movies, and thought about how those creatures might use the new objects. So they were doubly productive. It wasn't just the roles. It was inventing roles and using roles to come up with new ideas. And that perspective-taking...

It seems to underlie a great deal of creative behavior, not just in design, but Phil Tedlock has studied superforecasters. These are people who year after year are able to predict outcomes of world events better than most people. So they have certain skills.

that other people don't have. It isn't like predicting the stock market where the good predictors one year aren't the good predictors the next year because there really aren't many skills there. But they are skilled. So what are their skills?

They are knowledge junkies. They love information. They are numerate, so they can distinguish between 10% probability and 15% probability. And they can change their predictions by a percentage point or two percentage points. So they're well calibrated in their use of probabilities.

But most of all, they seem to change perspective. They challenge their own perspective. So this is, I think, what you were saying about confirmation bias and conforming, looking only at evidence that conforms to your views. You want to look at evidence that challenges your views more.

in order to be calibrated with the world. Otherwise, you fall prey to wishful thinking and terror. So they challenge their own perspective. They also deliberately think of perspectives of other people. So if you're a liberal, how would a conservative see this? If you're Zelensky, how would Putin see it? And how would...

the Chancellor of Germany see it? How would different viewers see it? Challenge your perspective, take other perspectives. So we see that in social decision-making, in political prediction, in economic prediction. So taking other perspectives seems to be a useful way of coming up with other ideas. I sometimes worry, though,

that certain stubborn people don't take the perspective of others. Some of them are bullies. We can all think of examples. And I wonder if that doesn't allow them to get their way, the fact that they will not change perspective, nor will they see yours. And so that worries me.

But I choose to think that on the whole, perspective taking enlarges the mind and increases possibilities. If you think about a lot of the cognitive biases, they sort of come down to frame of reference.

And we only have our own frame of reference. And the example that sort of comes to mind is standing on a train and holding a ball, and the train is moving at 60 miles an hour. And relative to you, the ball is not moving because you're standing on the train. But relative to the somebody watching the train go by, the ball is moving at 60 miles an hour.

And we have so many blind spots because all we see is what we see in front of us, right? And we think that's all there is to the world. And one of the advantages to perspective taking or changing your frame of reference into a problem is that without any additional...

sort of information that you're coming across, you can put yourself in somebody else's shoes and you can change your lens into the situation. And that will reveal a little bit of your blind spots that you have from your own perspective if you let it. And by walking around a problem almost in a three-dimensional way, if you want to conceptualize it, you can sort of begin to eliminate the blind spots. And if you think of decision-making as

the source of all bad decision making is blind spots because you're only acting on things that you feel are rational. So you're blind to some piece of information or some implication or some sort of future that you can't see. And by walking around and changing your frame of reference or taking a different perspective, you can learn to reduce those blind spots quite a bit.

No, absolutely. And it's, again, the benefit of vicarious stories, of vicarious learning, is that's one way of getting different perspectives, reading the newspaper, and not just reading your news, but other people's news. Perspective-taking allowed Einstein to take the different temporal and spatial perspectives. It allowed Newton to take. So it also helped for leaps.

of advances in science and math, as well as these small sorts of things of taking a spouse's perspective on your own behavior instead of getting defensive and screaming, but just saying, "Oh," or your child's perspective. They see it that way, and of course, if they see it that way,

So it's useful on a personal level, on a national level, on a creative level. Just everywhere is a frame of reference, taking different ones and seeing. And it's not always easy.

I mean, we didn't jump to the insights that Einstein did. So it's by no means easy. But as you say, with examples like your bow on the train, it can be really insightful. You can say all of a sudden, wow.

I get it. And then how much transfer that is. We again talked about how you transfer and telling people to take different perspectives probably can help in some situations. Providing them with perspectives to take can help even more. So it's...

It's capturing a deep phenomenon, but instantiating it can still be difficult. Although I think it's midway between really abstract and really specific. Perspective-taking changing is something in the middle that people can grasp. I think that's an excellent place to sort of wrap this up. One final question. What does success mean to you?

Oh, it's so personal, isn't it? And it keeps changing. And it's dependent. Am I successful as a parent? Am I successful as an academic? Am I successful as a friend? There are different. Have I professionally achieved what I hope to achieve? But what I hope to achieve keeps changing. I certainly went beyond what I hoped for.

When I was a small child, I hoped for an interesting life. I certainly had an interesting life.

And I hope for a life that had friends and intellectual excitement. I certainly wanted to be a parent and have a family. So I hope for all those things. I think I've been very fortunate to have even more, but there's more I could have had. And time is running out. So at some point, you know, I think life cycles...

do that too. There are periods when you're hungrier and there are periods when you need to reconcile. I think being female, I look at what our opportunities are open to women now and they're incomparable. Both familial, my parents had no expectations for me. They thought being smart was dangerous. It led to insanity.

And they certainly didn't have any aspirations for their two daughters other than successful marriages. They certainly had other aspirations for their sons.

And that was there all along. And I think, again, opportunities for women. Now I look at my children, I look at my grandchildren, I see what's expected of them and what they can aspire to is if...

Climate change doesn't kill us all. That's the pessimist in you coming here. Well, you know, you look at the data. You can't be self-deluding. But, you know, I am hoping for even more enriched lives. I've been very lucky to have an extraordinarily rich life, and I can see that they're even better prepared

for a rich life than I was. Both the world is better prepared for them and they are better. They're enabled in wonderful ways. Well, thank you so much for taking the time to speak with me today, Barbara. Thank you. It's been a real pleasure. Thanks for listening and learning with us. For a complete list of episodes, show notes, transcripts, and more,

go to fs.blog slash podcast, or just Google The Knowledge Project. Until next time.