Shownotes Transcript
Hey, everyone. This is Russ Altman, your host for The Future of Everything. It's Teacher Appreciation Week, and along with schools all across the country, we here at The Future of Everything want to send out a heartfelt thank you to the teachers who make a difference every day in the lives of our children and students everywhere.
In light of this, we're rerunning an episode on the future of educational technology with the Dean of the Stanford School of Education, Dan Schwartz. Dan is exploring effective ways to use AI in the classroom to support teachers, not to replace them, and to enhance student learning. I hope you'll take a moment to tune in again and enjoy. Before we get started, please remember to follow the show in the app that you listen to. You'll be alerted to all of our episodes and it'll make sure that you never miss the future of anything.
You know, the rise of AI has been on people's minds ever since the release of ChatGPT, especially the powerful one that started to do things that were scary good. We've seen people using it in business, in sports, in entertainment, and definitely in education. When it comes to education, there are some fundamental questions, however. Are we teaching students how to use AI or are we teaching students?
How do we assess them? Can teachers grade papers with AI? Can students write papers with AI? Why is anybody doing anything? Why don't we just have the AI talk to itself all day? These are real questions that come up in AI. Fortunately, we're going to be talking to Dan Schwartz, who's a professor of education and a dean of the School of Education at Stanford University, about how AI is impacting education.
Dan, the release of ChatGPT has had an impact all over the world. People are using it in all kinds of ways. And clearly, one of the areas that AI, especially generative AI, has made impact is in education.
Students are clearly using it. Teachers are thinking about using it or using it. You're the Dean of Education at Stanford. What's your take on the situation right now for AI in education? Okay, so lots of answers to that. But, you know, the thing I've enjoyed the most is showing it to people and watching their reaction. So I'm a cognitive psychologist. I study creativity, learning, what it means to understand AI.
And you show this to people and you just see them go, oh, my Lord. And then the next thing you see is they begin to say, what's left for humans? Like, what's left? And then they sort of say, wait a minute, will there be any jobs? And then finally, they sort of say, oh, my goodness, education needs to change.
And as a dean who raises money for a school, this is the best thing to ever happen. Whether it's good or bad, it doesn't matter. Everybody realizes it's going to change stuff. And so it's really an exciting time. So that is really good news. I have to say going into this, and I have to reveal a bias.
I have often wondered if technology has any place in a classroom. And I think it's because I was injured as a youth. This is in the 1970s when some teachers tried to put a computer program in front of me. And I was a pretty motivated student. And I worked with this computer for about six minutes. And I should say, I'm not an anti-computer person. I literally spent all my time writing algorithms and doing computation work. But I just felt as a youth that
that I wanted to have a teacher in front of me, a human telling me things.
And so that is clearly not the direction. I hear you laughing. So talk to me about the appropriate way to think about computers, because I really have a big negative reaction to the idea of anything standing between me and a teacher. You must have had very good teachers. I might have. So Ross, you sound like someone who doesn't play video games. I do not play video games. So there's this world out there. Yeah.
People get to experience things they could never experience directly. And no teacher can deliver this immersive experience of you in the Amazon searching for anthropological artifacts. There's also something called social media that people use. I've heard about this. Yeah, yeah. I think we disseminate the show using it. So back in the day. Okay. So I'm a dinosaur. Back in the day, you've got the Apple II maybe. And it's about 64 bucks.
K, maybe. It's got a big floppy drive and it takes all its CPU power to draw a picture of a two plus two on the screen.
So I think things have changed a little bit. They have. But I appreciate your desire to be connected to teachers. I don't think we're replacing them. I'm not going to give you a lecture about teaching, but I will say this one sentence that was reverberating through my brain when I was getting ready for our interview, which was when I'm in a classroom, and this has been since I've been in third grade, I am watching the teacher trying to understand how they think about the information and how they struggle with it to like understand it and then try to relay it to me.
And so it is that's where I'm learning. It's not even what they're saying. It's they're painting a picture for their cognitive model of what they're talking about. And that's what I'm trying to pull out to this day. And so that's why I have such a negative reaction to anything standing between me and this other human who has a model that is more advanced than mine about the material that we're struggling with. And I just I'm trying to download that model.
Wow. You are a cognitive psychologist, Russ. I mean, like I had a buddy who sort of became a Nobel laureate and he talked about how he loved to take apart cars. And I'd say, I love to watch you take apart cars, right? Just figure out what you're doing.
No. So I think I'm going to, let's separate this. There's the part where you think the interaction with the teacher is important. I don't know that you need it eight hours a day. You know, that's an awful lot of interaction. I'm not sure I want to be with my mom and dad for eight hours a day trying to figure out their thinking. So you don't need it all the time. On the other side, you know, we can do creative things with the computers. So for example, I wrote a program where students learn by teaching a computer agent.
And so they're trying to figure out how to get the agent to think the way it should in the domain. Turns out it's highly motivating. The kids learn a lot. The problem was the technology quickly became obsolete.
Because after kids used it for a couple of days, they no longer needed it because they'd figured out sort of how to do the kind of reasoning that we wanted them to teach the agent to do for reasoning. That's exactly what I was talking about before about my relationship with my teacher. And you just flipped it, but it's the same idea, which is that there's a cognitive model that you're trying to transfer. And by doing that transfer, you introspect on it and you understand what it is that you're thinking about. I think that's right. So the concern is the computer does all the work.
Right. And so I'm just sitting there pressing a button that isn't relevant to the domain I'm trying to learn. But, you know, one of the things computers are really good at, like as good as casinos, is motivation. So some computer programs, they gamify it. I'm not sure that's a great use of it because you, you know, you try and you learn to just beat the game for the reward. Right. As opposed to learn the content. But things like having, teaching an intelligent agent how to think.
There's something called the protege effect, which is you'll try harder to learn the content to teach your agent than you will to prepare for a test. So we can make the computer pretty social. Okay. So you are clearly a technology optimist in education. And
in addition to the amazing fundraising and like there's so many questions to be answered, what I think a lot of people are worried about is are we at risk of losing a gen? We've already lost a few generations of students. Some people argue because of the pandemic and the terrible impact it had, especially on people who weren't privileged in society and in their education. Are we about to enter yet another shock to the system where because of the ease of having essays written
and grading papers that we really don't serve a generation of students well? Or do you think that's an overhyped, unlikely to happen thing? No, it's a good question. Part of this is people's view about cheating. And so it's too easy for students to do certain things. There's another response that I want to hang on to. I want to ask you, Russ. Are you using UTeach?
Yeah. Are you like putting in all sorts of rules to prevent students from cheating and you're saying, "Use it, do whatever you can. I'm going to outsmart your technique anyway." It's a little bit more on the latter. So, I teach an ethics class, which is a writing class and we allow ChatGPT because my fellow instructor and I decided
And this was the quote, we want to be part of the future, not part of the past. So we said to the students, knock yourself. Sorry, the future of everything, Russ. Thank you. Thank you. Thank you. Thanks for the plug. So we allow it. We asked them to tell us what prompt they used and to show us the initial output that they got from that prompt.
And then we, of course, have them hand in the final thing. And we instruct the TAs and ourselves when we grade that we're grading the final product with or without a declaration of whether ChatGPT is used. We do have engineers as TAs, which means that they did a careful analysis. Students who used ChatGPT, and I don't think this is a surprise, got slightly lower grades, but spend substantially less time on the assignment.
So if you're a busy student, you might say, I will make that trade-off because the grades weren't a ton worse. It was like two points out of 100, like from a 90 to an 88. And they completed it in like half the time. Do you think they learned less?
So we don't know. We don't know. And the evaluation of learning is something that I'm looking to you, Dan. How do I tell? So we do try to use it, but we are stressed out. We have seen cases where people...
Say they used ChatGPT but tried to mislead us in how they used it. They said, I only used it for copy editing, but it was clear that they did more than copy editing with it. And so at the edges, there are some challenges. But in the end, we said, motivated students who want to learn will use it as a tool and will learn. And the students who we have failed to motivate, and it is our failure, you could argue, they're just going to do whatever they do, and we're not going to be able to really –
impact that trajectory very much. Yeah, you know, you sort of see the same thing with video-based lectures. So I'm online, I've got this lecture. Do I really want to sit and listen to the whole thing? Not really. I'm going to skim forward to find the information, I skim back.
I'm probably going to end up doing the minimum amount if it's not a great lecture. Yeah. So I'm not sure this is a chat GPT phenomenon. It's just, it's sort of an enabler. I think the challenge is thinking of the right assignment. Yep. Yep. So like you can grade things on novel and appropriateness. So are they novel?
you know, if they use chat GPT like everybody else, they won't be novel. They'll all produce the same thing. It's incredibly, yes. So it is, there's the most common type of moral theory is called common morality. And it turns out that chat GPT does pretty well at that one because there's so many examples that it has seen and it's terrible at Kant. Deontology, it really can't do. Okay. So let me, let me, let me. So let me get, let me get back to your question. Yeah.
So here's what I see going on right now. There are like big industry conferences because they're producing the technology that schools can adopt. Right. And there's a lot of money there. And 20 years ago, there were zero unicorns in about, I think last year, $54 billion valuation companies in ed tech. So this is a big change.
So what are they doing? They're basically creating things to do stuff to students, right? So maybe they're marketing to the teachers, but it's, you know, it's I'll make a tutor that is more efficient at delivering information to the students. Or I will make a program that can correct their math very quickly. And so what's happening is the industry is sort of using the AI in the way that nobody else uses it.
Because everybody's got this tool, wants to create stuff, right? Like my brother, it's my birthday. What does he do? He asked ChatGPT to write me a poem about Dan Schwartz at Stanford.
What he doesn't know is that there's a lot of Dan Schwartzes, and so evidently I wear colorful ties. But this is what everybody wants to do. They want to create with it. Meanwhile, the field is trying to push towards efficiency. Can we get the kids done faster? Can we get them through the curriculum faster? Can we correct them faster? In which case, the kids are going to optimize for being really efficient.
Right. As opposed to just trying to be creative, innovative, use it for deeper kinds of things. So this is my big fear. And so and so you're watching these companies and I'm guessing that they don't always ask your opinion about what's what would you tell? So let's say a one of these unicorn billion dollar or more companies comes to you and says, we want to do this right. We want to use the best educational research to create AI that can bring education to people who might otherwise not have quality education. What would you tell them?
So this is a challenge, right? This is something we're actively trying to solve. So we've created Stanford Accelerator for Learning to kind of figure out how to do this. Because I've been in this ed tech position for quite a while and the companies come in and they say, we really want your opinion. And then they present what they're doing. And I go, have you ever thought of, and they go, wait, wait, wait, let me finish. And this goes on for 55 minutes.
where they're telling me what they want to do. And I'm trying to say, you know, if you just did this and the way it ends is I say to them, look, you, you, you, if you do these three things, I'll consider being an advisor. Right. They never come back. So the message, the message you're sending them is just not in their worldview. It's because they have a vision. Everybody wants to start their own school. Yeah. They have their vision of what it should be and, and they're urgent to get it done. And you know, it's a startup mentality.
So trying to figure out how can we educate them? I think we know a lot about how people learn that we didn't know 20 years ago when they went to school. And the AI, one of the things it can do is implement some of these theories of learning in ways that don't exist in textbooks and things like that. So that's the big hope. And the question is, how can you take advantage of industry? Education is a public good, but they still buy all their products.
And so going through those companies is one way to sort of bring a positive revolution. But again, I'm a little worried that the companies are – they're sort of optimizing for local minima. Yeah. You know, to accommodate the current schools and things like that. Should we take – so should we take solace in the teachers? So many of us are fans of teachers, grammar school teachers, middle school teachers, high school teachers. Yeah.
Many of these folks are incredibly dedicated. Will they be a final filter that looks at these educational technologies and says, absolutely not? Or, yeah, we'll use that, but we're going to use that in a way that makes sense for my way of teaching? Or are they not in a position to make those kinds of what you could call courageous decisions about kind of modifying the use of these tools to make them as good as possible on the ground?
Great expression, courageous decisions. I really like that. So it's pretty interesting. The surveys I've seen sort of over the last year, the different groups do different surveys. It's sort of, if I take the average, about 60% of K-12 teachers are using Gen AI, right? And about 30% of the kids. If I go to the college level, about 30% of the faculty are using Gen AI in teaching and about 80% of the kids are using it.
So I do think in the pre-K to 12 space, the teachers are making decisions. They do a lot of curriculum. So a great application is project-based learning. So project-based learning is a lot of fun. Kids learn a lot. They sort of develop a passion, a certain depth, as opposed to just mastering sort of the requirements. But it's really hard to manage. You know, when I was a high school teacher, I had 130 kids.
If all of them have a separate project, I have to help plan them and make them learning goal appropriate. So the Gen AI can help me do that. It can help me have the kids sort of help use it to help them design a successful project. It can help me with a dashboard that helps manage them hitting their milestones, things like that.
And there, you know, the teacher is like, I can do something I just couldn't do before. Yeah, yeah. It's different than the model where you put the kids in the back of the room who finished early and say, go use the computer. Right. But I think, you know, most schools, kids are carrying computers in classes. So it's a little different. It's more integrated than it used to be. This is The Future of Everything with Russ Altman. More with Dan Schwartz next.
Welcome back to The Future of Everything. I'm Russ Altman, and I'm speaking with Dan Schwartz, professor of education at Stanford University. In the last segment, Dan told us about AI, education, some of the promises and some of the pitfalls that he's looking at on the ground, thinking about how to educate the next generation. In this segment, I'm going to ask him about assessment, grading.
How do we do that with AI and how do we make sure it goes well? Also going to ask him about physical activity, which turns out physicalness is an important part of learning.
I want to get a little bit more detailed, Dan, in this next segment. And I want to start off with assessment, grading. I know you've thought about this a lot. People are worried that AI is going to start to be doing all the grading. Everybody knows that a high school teacher with a couple of big classes can spend their entire weekend grading essays. It is so tempting just to feed that into ChatGPT and say, hey,
How good is this essay? How should we think about, maybe worry about, but maybe just think about assessment in education in the future? Yeah, this was, remember the MOOCs? Yes. Massively online open courses. And you're hoping you have 10,000 students. And then you got to grade the papers for 10,000 students. So what do you do? You give a multiple choice test, which can be machine coded. Right.
So I think that's always there. I'm going to take it a slightly different direction, which is I'm interacting with a computer system. And while I'm interacting with it, it can be constantly assessing in real time. Right. And so there's a field that's sometimes called educational data mining or learning analytics. Right.
And there's thousands of people who are working on how do I get informative signal out of students' interactions? Like, are they trying to game the system? Are they reflecting and so forth? So this is something the computer can do pretty well, right? It can sort of track what students are doing, assess, right?
And then ideally deliver the right piece of instruction at the moment. So you could use the assessments to give people a grade, but really the more important thing is can you use the assessments to make instructional decisions? So I think this is a big area of advancement. But here's my concern. We've gotten very good at assessing things that are objectively right and wrong. Like did you remember the right word? Did you get two plus two correctly?
For most of the things we care about now, they're like strategic and heuristic, which means it's not a guaranteed right answer. And so what you really want to do is assess students' choices for what to do. So for example, creativity, it's just, for the most part, it's a large set of strategies, right? There's a bunch of strategies that help you be creative. The question is, do the students choose to do that? Or do they take the safe route? Because creativity is a risk, right? Because you're not sure.
So I think this is where the field needs to go, is being willing to say that certain kinds of choices about learning are better than others.
And it becomes more of an ethical question now. Instead of saying two plus two equals four, there's no ethics to it. Are you going to be able to convince non-educators who hold purse strings, let's call them the government, that these kinds of assessments are important and need to be included? Because my sense is that when it filters up to boards of education or elected leaders, a lot of that stuff goes out of the window and they just want to know,
How good are they at reading comprehensive and can they do enough math to be competitive with, you know, country X? Yeah. Yeah. So different assessments serve different purposes. Like the big year end tests that kids take, those aren't to inform the instruction of that child. They're not even for that teacher. They're for school districts to decide are our policies working.
And so it's really a different kind of assessment than me as a teacher trying to decide what should I give the kid next. So I think it's going to vary. You know, the tough question for me is, should you let the kid use chat GPT during the test? Yeah. Right. And we had this argument over calculators. Right. And finally, they came up with ways to ask questions where it was OK if the kids had calculators.
Because the calculator was doing the routine stuff and that's not really what you cared about. What you cared about was could the kid be innovative? Could they take another, a second approach to solve a problem? Things like that. So I teach another class where it's a programming class. The students write programs and we have switched classes.
And we've actually downgraded the value. So as you know very well, just as background, there is now an amazing, ChatGPT can also write computer code, essentially. And so a lot of coding now is kind of done for you and you don't need to do it. We are trying to make sure that they understand the algorithms that we asked them to code. And so what we're doing is we're downgrading the amount of points you get for working code. You still get some, but we're upgrading the quiz about how the algorithm works
Do you understand exactly why this happened the way it did? Why is this data structure a good choice or a bad choice? And so it's forcing us. And you could have argued that we should have done this 20 years ago in the same class, but this is making it a more
urgent issue because if we don't, people can just get an automatic piece of code. They can run it. It'll work. They have no understanding of what happened. And so it's really a positive. It's putting more of a burden on us to figure out why the heck did we have them write this code in the first place?
No, this was my point. It makes you sort of rethink what is valuable to learn. And you stop doing what was easy to grade. Right. So I have an interesting one. This is a little nerdy. Okay, I love it. I love it.
Right. And I'm sort of like, well, that's great. Let me ask you one question. And I say, I'm going to email you a question and you'll have five minutes to respond. You let me know when you're ready for it. And I ask them, this is just for you, Russ, but why is the tail of the T distribution fat at small sample sizes?
And what I get back usually is because they're small sample sizes. Right. Or because it's the T distribution. Or it's, yes, even better. And then I come back and I sort of say, well, have you ever heard of a standard error? And I begin to get at the conceptual stuff. Right. And I suspect if I gave it. So there are ways to get conceptual questions that are really important.
But, you know, being able to prompt or write R code, you know, that's a good thing. You want them to learn the skills as well. Exactly. So I don't know, you know, when the calculator showed up, there's a big debate, right? What should students learn? Can they use the calculator? The apocryphal solution was you had to learn the regular math and the calculator now. You just had to learn twice as much.
And so maybe that's what it's going to be. And that's a very likely transitional strategy. And then we'll see where we end up. Okay. In the final few minutes, this seems like it's unrelated to AI, but I bet it's not.
You've done a lot of work on physical activity and learning. You've even been on a paper recently where you talk about having a walk during a teaching session and whether you get better outcomes than if you were just standing or sitting. So tell me about that interest and tell me if it has anything to do with today's topic.
I can make the bridge. I can do it, Russ. Right. So we did some studies. I've done a lot of it. It's called embodiment. Embodiment. Yeah. I got clued into this where I was asking people about gears. And I say, you know, you have three gears in a line. And if you turn the gear on the left clockwise, what does the gear on the right do? Far right. And I'd watch them and they'd go like this with their hands. They'd model with their hands. Right.
And then I was sort of like, well, what's the basis of this? And I'd say, well, why? Why? And they say, because this one's turning that way, that one, I go, but why? And in the end, they just bought them out. They just show me their hands. They didn't say things like one molecule displaces another. Right. So that sort of clued me in. This pinky is going up and this other pinky is going down.
Yes. What don't you understand about that? Pretty much. Well, it was non-verbal. Yeah. So we went on, you know, we discovered that the basis for negative numbers, right, is actually perceptual symmetry. And we did some neuro stuff. And so the question is sort of how does this perceptual apparatus, which some people, we're just loaded with perception, right? The brain is just one giant perceiving device.
So how do you get that going? So part of the embodiment is my ability to take action, right? And so this is where we started, right? Right now, the AI feels very verbal, very abstract. Even the video generation, it's amazing, but it's pretty passive for me. So enter virtual worlds. They're still working on the form factor where I can move my hand in space.
And something will happen in the environment in response to that. You know, I think medicine is, you know, really been working on haptics so surgeons can practice. There was a great guy who made a virtual world for different heart congenital defects and you could go in and practice surgery and see what would happen to the blood flow.
So I think that embodiment where you get to bring all your senses to bear, it's not just words, but it's everything, can really do a lot for learning, for engagement, not just physical skills.
So that's a challenge to, I'm hearing a challenge to AI, which is as an educator, you know that this physicality can be a critical part of learning. And by the way, would this be a surprise? I mean, we've been on earth evolving for several hundred million years and you would be surprised if our ability to manipulate and look at three-dimensional situations wasn't critical to learning. And yet that's not what AI is doing right now. So this is a clear challenge to AI among other things.
Right. So I have a colleague, Renata Fruchter, and she teaches architecture and she has students make a blueprint for the building.
Right. And then she feeds the blueprint to a CAD system that creates the building. She then takes the building and puts it into a physics engine. It can basically render the building and make walls so you can't move through them and it has gravity and things like that. She then puts the original student who designed the building in a wheelchair and has them try to navigate through that environment. At which point they sort of understand, oh, this is why you need so much space so they can turn around, so they can navigate near the door.
I am sure that is an incredibly compelling experience that allows them to be generative about all their future designs. So yeah, this is a challenge. And part of the commingling of the AI and the virtual worlds, I think this is a big challenge. It's computationally very heavy, but it will open the door for lots of ways of teaching that you just couldn't do before.
Thanks to Dan Schwartz. That was the future of educational technology. You've been listening to the future of everything. And I'm Russ Altman. You know what? We have an archive with more than 250 back episodes of the future of everything. So you have instant access to a wide array of discussions that can keep you entertained and informed. Also remember to rate, review and follow. I care deeply about that request.
You can connect with me on many social media, usually at RB Altman or at Russ B. Altman, on Blue Sky, Mastodon, and Threads, and of course, LinkedIn. And you can also follow Stanford Engineering at Stanford Engineering or Stanford ENG.