A narrowing of AI research? with Juan Mateos-Garcia
59:18 Share

Hello and welcome to Skynet today's Let's Talk AI podcast where you can hear from AI researchers about what's actually going on with AI and what is just clickbait headlines. We release weekly AI news coverage and also occasional interviews such as today. I am Andrey Kronikov, a third year PhD student at the Stanford Vision and Learning Lab and the host of this episode.

In this interview episode, we'll get to hear from one of the authors of a recent paper, A Narrowing of AI Research, whose name is Juan Mateos Garcia. Juan Mateos Garcia is the director of data analytics at Nesta, the UK's innovation foundation. He leads a team of data scientists, developers, visualizers, and innovation experts who use new data sets, analytics methods, and visualization tools to inform innovation and AI policy.

Prior to joining Nesta, Juan worked as a researcher at SPRU or the Science Policy Research Unit at the University of Sussex and Centrum at the University of Brighton. Juan has a degree in economics from the Universidad de Salamanca and an MSc in science and technology policy from SPRU University of Sussex. So thank you so much, Juan, for joining us for this episode. Thank you so much for having me.

And let's go ahead and dive straight in. So as I said, the focus will be on the recent paper and narrowing of AI research, which I found quite interesting. Before we dive into any of the details, perhaps I can let you go ahead and give a high level summary of what is the paper about and how did you come about to do this research?

Okay, so the idea for this paper came from, actually it was inspired by some thinking I did while I was in Toronto.

for the Economics of AI conference last year. So it's a conference where lots of economists get together to talk about the economics of AI. And so it was a super exciting discussion. But one thing that I was a bit perhaps frustrated about was that when people were talking about AI and levels of AI activity and innovation in AI and all of those things, they were using very aggregate measures of what's going on with AI. They were just counting papers, counting patents,

looking at level of investment on AI in firms. And they weren't really thinking a lot about the composition of AI. You know, the fact that AI as a field has many different techniques, has many different applications. People over the years have tried to do things in different ways.

And really, there's not a lot of attention paid to that within that literature. You know, everything is aggregated into a single number, which is level of activity. And I thought that actually it's quite important that we start to decompose, I guess, this AI field a bit more because I guess we know that, you know, there's been obviously very successful techniques that have arrived in recent years, especially deep learning that have perhaps replaced other approaches.

and they have become in some ways dominant. But we also know that there are some concerns around these methods, you know, in terms of

a bunch of things like from their propensity to be a bit brittle and break down, where you expose them to, I guess, situations outside of their training set, obviously their low levels of explainability levels. So the amount of data they require to be trained and even their potential impacts on the environment.

So there's this sense that, you know, if you start to open the black box of the AI field and look at its composition, you know, we might find some interesting possibilities

evidence about how the field is evolving. And it's particularly interesting to see whether we are seeing some sort of, you know, increasing concentration of AI research in, for example, these techniques I just mentioned. And there are questions about the extent to which there might perhaps be almost like too much concentration on those techniques. I guess,

Yeah, my thinking around this is very much influenced by ideas coming from complexity economics and evolutionary economics, where actually, I guess, a very important idea within these disciplines is that, I guess, technology called progress is not deterministic.

So you can have technologies appearing and becoming dominant, not because they are better than their competitors, but simply because they arrived earlier. They were lucky at the beginning. There was this process of, I guess, positive network effects that make these technologies dominant, a bit like what happens with internet platforms. So, you know,

There's a similar question that you could ask yourself about in the context of AI. Is it the case that

we have had a technology that has arrived. Basically, it was in the right place at the right time. Deep learning, a lot of data becoming available, computational infrastructure becoming available, processing infrastructure like GPUs and things like those becoming available. It's almost like this technology winning this thing that people have been referring to as a hardware lottery and almost becoming established

even though it's got important limitations. That means that actually maybe we wouldn't want to bet a house on it. And actually, we would want to maybe, sure, like continue developing it, but we would want to also like maintain some alternatives alive because, you know, just in case it doesn't work almost, we want to have other ways to build AI systems that continue being developed so that we can, I guess, adopt them in those cases where deep learning doesn't work.

That was kind of where all of these ideas came from. I guess, obviously, it was quite nice to be thinking about this in Toronto because obviously it's one of the hotspots for AI work.

And obviously it's interesting that Canada itself as a country that has funded AI research over the years, you know, they did invest a lot in preserving diversity on AI, like in the last few decades when actually ideas from artificial neural networks, they were very interesting, but they weren't very applicable. And, you know, the CIFAR and places like that kept investing on these technologies and kept them alive.

And, you know, eventually we got to a point where these technologies could be applied and that has led to the current almost like revolution with deep learning we are seeing. And I guess what this led me to think is, okay,

So, you know, it's almost like if we have too much concentration of AI research in deep learning now, and we're only focusing on the things that are successful now and create value now, what are the ideas that might be almost like the future, like the source of the future AI revolution that might, you know,

might not have a space to breathe because we are losing that diversity. So, yeah. And obviously that has some very important policy components, right? Which is, what is the role of policymakers in, say, for example, maintaining diversity in the AI research system? What is the role of private sector companies as well?

And obviously, once you start to get into a conversation and thinking about policy and thinking about policy decisions, and very quickly, you're going to have to start to get into thinking about measurement and metrics. And I guess that's where, you know, the idea of, I guess, the method side of the paper came. In a way, the question was, how do we take this idea of...

looking at the composition of AI and looking at how AI might be becoming maybe more concentrated in a small number of techniques that have problems. How do we turn all of those ideas into something that we can measure and quantify so that we can look at this evolution and think about what the results mean for policy? I see. Thank you for that very good summary. Definitely, I found it interesting to see this new research trying to quantify diversity in AI.

Before we dive into the details of how this was done and what were the results, I'm curious to talk a bit more about the kind of background, the related work in the field. So I was browsing and I saw, you know, some of the related ideas had to do with directed technological change, rationales for preserving technological diversity,

And as you said, there's kind of a pretty direct known reasons. What if there's other techniques besides deep learning that don't have the drawbacks of deep learning? Just for the sake of our listeners, I wonder, can you make a comparison to any other technologies? I think in the paper you discuss automobiles and there's also some discussion of military funding. So, yeah, I'm curious to hear the analogy. Yeah.

Yeah, so I guess there's like, um, you're going to be historians like to look at almost like the history of technologies that were maybe an example of what I was talking about before, you know, the sense that a technology that became dominant, um, without being the best option.

Because I know that's a big challenge for, I guess, mainstream economists who like to think of markets generally producing efficient outcomes. So economic historians like to look at these technologies and be a bit like, "Hey, how can you explain this one?" You know, obviously the most famous example, paradigmatic example of this is the QWERTY keyboard.

which, you know, was initially developed in order to prevent almost like mechanical typewriters getting blocked because the arms of the typewriters would hit each other and the machine would get blocked. So the keyboard was designed to basically avoid that kind of blockage.

And then, you know, when we moved into, you know, digital typewriters and then eventually computers where the problem of like arms getting blocked wasn't like really like an issue anymore. We still continue with this keyboard. We actually, according to some, you know, like almost like benchmarking is not the optimal keyboard for typing. And we stayed with this technology just because everyone was used to it. And all of the key, like the typists were being trained with this keyboard.

it was almost like there was this inertia that meant that it was very difficult to, at any point in time, almost like coordinate the whole market of designers of computers and keyboards and typists and people who train typists to say, let's all together find out a standard and move to it. It's very difficult to do. And therefore, here we are with the QWERTY keyboard. It's a bit of, you know, kind of...

you know, folk example, but I think it captures quite well the issues here. Once this technology becomes adopted and gains momentum, it might be quite difficult to change away from it, even though everyone would be better off if we changed. Obviously, like a good example of, maybe a more topical example of this kind of situation is just with the automobile and the combustion engine. You know, at the

In the early days of the history of the automobile, people were exploring many different ways of building an automobile. And they were exploring very different kinds of designs for engines. They were exploring, obviously, the combustion engine. But there was obviously also the steam engine. They were exploring electric cars. And combustion engine became dominant.

You start to invest in it. You start to learn by doing, developing that technology. It gets better. You start to invest in things that complement that technology, not least, I guess, very well-developed pipelines to be able to access petrol that this engine requires. And then eventually you find yourself in a situation where the combustion engine is dominant and you have almost redesigned all

all of society and lots of industries have emerged around the combustion engine and this is the way things work. And, you know, maybe that's what's most efficient and no one really except kind of eccentrics will say, okay, let's continue developing electric cars or steam powered cars. Uh, but actually, you know, over time as, uh, things change and we discover more about this technology and its environmental impacts, you know, we start to think, uh,

Maybe if we look at the situation now with the benefit of hindsight, we think, okay, maybe it might have been actually better if we had chosen one of the other designs just because they didn't have the same type of environmental impacts. But, you know, now we need to almost start from scratch to build these technologies. Imagine that...

Say, in the early 20th century, someone had said, say some foundation had said, actually, I'm going to continue investing on electric cars. I'm going to continue investing on steam-powered cars and get that technology alive and being developed so that, say now, when we have found that

combustion engines have like all of these massive impacts on the environment, we would actually quite easily swap to this other technology instead of saying we need to almost like start building it from scratch, which is what we're having to do. And obviously it's going to take a while to do because we are trying to compress a lot of technological learning and lots of like changes in infrastructure, like, for example, charging points for electric cars and things like that. We're trying to compress all of that activity in a very short period of time because we want to change now.

So I guess maybe that's another example of technological, I guess, looking to technologies and homogenization and concentration of activity in a single technology, a loss of diversity that in this case we are, I guess, we are coming to regret. And I guess the question we were seeing in the paper is whether

We might find ourselves in a similar situation with AI where maybe we look into the future, I don't know, a few decades, and we think about backwards. We might say, actually, gosh, maybe we shouldn't have been so quick on betting on this technology and developing all of these massive data collection infrastructures and computational infrastructures and label datasets and almost ways of working with it.

And actually we should have maybe preserved a few other almost like alternative trajectories going because, you know, now say problems we have discovered about this technology that we can't address and these other alternatives can address. We are in a good place. They are in a good place that allow us to almost like swap or like make a change of technology without having to develop new AI systems or new ways of doing AI from scratch.

Great. Yeah, that makes a lot of sense. And I think having these examples from the other fields does motivate a bit more. I think a lot of people are now very positive on deep learning, but having comparisons from history kind of should inspire us to imagine that there might be other techniques that are being overlooked, as you say. So with that being established, maybe we can go ahead and dive in into a bit more specifics.

The high level questions that you sought to address have already been established, kind of a diversity narrowing questions, but maybe can you let us know the more specific questions, the specific kind of metrics you were looking at and what were you looking to answer? So I guess, you know, like once we start to think about diversity, you know, then the question is how do you measure diversity?

how do you measure diversity in an ecosystem? You know, an ecosystem of, say, you could think about any kind of population is going to contain almost like different types, you know, and basically diversity is almost like trying to get a sense of how many different types are there.

Is there a lot of concentration on a single type or are they more evenly spread? How different are the types from each other? So this is kind of like these are some key ideas in the literature on diversity that we were trying to think about. Okay, how do we operationalize this? How do we turn this into something that we can measure to generate these indicators? And the strategy that we decided to take is, okay,

Let's think about almost like all of AI research or, well, I mean, sort of like a relevant corpus of AI research. In this case, the corpus of research in archive. Let's think about that as the population of ideas around AI. And some of these ideas are going to be around methods. Some of these ideas are going to be around use cases. Some of these ideas are going to be around different techniques.

And some of these ideas are going to be more popular. Some of these ideas are going to be less popular. Some of these ideas are going to be more similar to each other, similar to each other. We're going to be over time, see more new ideas appearing, some ideas disappearing. So let's try to measure this. And basically the strategy we took to almost like go from the corpus of AI research into this

almost like a population of types or ideas is to use topic modeling and basically decompose this corpus into topics. And we think of the topics as almost like the themes or ideas in AI research.

And then we can look at, I guess, their distribution. Sorry, their number, like how many of these ideas are present at different moments in time. We can look at their distribution. So are there ideas that appear in all of the papers? Are there ideas that appear in very few papers? Are they more evenly spread or less evenly spread? And also consider their distance. So are there ideas that are very similar to each other? You always find them in the same papers. You always find them in the same places.

Or are there ideas which are more different to each other? Actually, they tend to almost define different fields or different areas of AI. So that's what the topic modeling gave us. And then basically, what we did was, I guess, use this kind of like operationalization of the composition of AI to, I guess, generate different metrics of diversity.

One thing that we decided to do was, you know, if you look at the literature on diversity, there's like a zillion metrics and there's like many different ways to pressurize those metrics in terms of how do you find distance between different topics? How do you find whether a topic is present in a paper? Obviously, with topic modeling, you need to accept thresholds perhaps to decide whether a topic is present in a paper.

Or you could allocate a paper to its top topics. There's different ways of doing it. And I think there's not a clear, principled way to select between all of these different parameters. So what we said actually in the spirit of diversity is actually use different metrics, let's use different operationalization of the metrics, and actually let's run them, all of them in parallel.

And actually then we can compare the results and I guess this allows us to get a sense. We are not almost like betting the house on a single metric. We're actually considering different metrics and we can compare between them. And I guess, you know, they're all kind of going in the same direction. Then this suggests that our findings are robust and this gives us confidence in their in their validity. So, yeah.

That's what we did to measure diversity. I guess something that was a key component of the paper, and I haven't really mentioned until now, is the idea of comparing the diversity of the research carried out by organizations in the private sector.

with with with research carried out by organizations in the in the public sector or academia. You know, and the reason why I was quite interested in comparing this is that, you know, I guess over time with with with our research, especially since 2012, I guess with with Alex Net and the arrival of deep learning, we have seen this

I guess, situation where the private sector and especially tech companies, I guess, mostly based in Silicon Valley and also in China, have really started to play a very active role in AI research. They have been, you know, hiring lots of researchers, hiring lots of professors from universities. They have been, I guess, increasingly contributing, like playing a leading role in setting the research agenda. And, you know, this is partly...

Because they have a lot of money, they have a lot of data, they are able to invest in the computational infrastructure required to develop these methods. And also they have the platforms that are required to monetize the application of these methods, like big internet platforms, lots of opportunities for

generating recommendations for targeting ads. So there's a sense in which these companies that have come in and become very successful in developing these methods, they might have even less incentives than the average researcher to preserve diversity because they have to, even though they invest a lot of R&D, they don't have that many reasons to invest on preserving diversity or invest in ideas which are not immediately applicable.

And, you know, some literature in economics suggests that in this kind of situation where, say, you have private sector companies racing against each other to develop technologies and apply them, you're going to have this loss of diversity. So what we decided to do is, OK, actually,

Let's look at the evolution of diversity, but also let's consider the extent to which, say, how diversity in AI research is evolving might be linked to increasing levels of participation of private sector companies in the field. And the way we operationalize this is, I mean, we built a pretty...

involved pipeline, you know, to go from papers to institutions, then from institutions to determining whether the institutions are private sector or educational. You know, for some organizations like DeepMind and OpenAI, we actually had to scrape their websites to get their archive IDs. So it was like a bit gnarly to collect all the data. But yeah, we ended up in a situation where we had

I guess the key elements of the analysis were we knew what the papers, we had identified the AI papers and I mean, I'm not even going to get into that part of the paper because that was quite involved as well. Maybe if you're interested, we can discuss later. Maybe people can have a look at the paper. We had the AI papers, we had the abstracts so we could generate the measures of diversity.

We knew when they had been published, so we could look at the evolution of this over time. We knew what papers came from private labs and the private sector, and what papers came from academia. So then we were going to be able to compare diversity of research carried out in academia and diversity of research carried out in the private sector.

Maybe one thing to flag up that's quite important is that all of the data and the code that we use for the analysis actually is all in the GitHub repo for the paper. And anyone who wants to look at the data or collect it, analyze it further, look at what we did in the paper, maybe play with some of the metrics, play with some of the parameters. It's all there for people to use. And I would love to, I guess, see the community taking that data and that analysis and taking it in new directions.

I see. Yeah, fascinating. As for readers, I want to mention, as you might imagine from hearing a description, this paper is pretty rich. It has a lot going on, a lot of data, a lot of results. So you can definitely go ahead and Google it and browse it yourself as always. Although we'll try and get through some of the highlights in this discussion.

And yeah, as you just said, I found it quite interesting. You have like five main research questions having to do with the topical composition of AI research, with participation of private sector organizations, with thematic diversity, with thematic diversity of private sector organizations, and then in which AI research topics do private sector organizations tend to specialize. So lots of interesting stuff.

Maybe to get started, we'll just go ahead and go for the punchline, kind of the main interesting bit, which is a question of, is the female diversity of AI research increasing or declining? So what was your result there? So actually it is, well,

It actually did increase quite a lot with the deep learning revolution. And you can imagine, you know, there's like this explosion of people looking at AI, you know, starting to develop the methods, finding new use cases for them, like obviously the revolution in computer vision and revolution in computer language, robotics. So I initially like there's this like increasing diversity as the field like becomes more

Almost like you have this Cambrian explosion of activity around AI and deep learning. But actually what's happened more recently is that in the last few years, you start to see this almost like stabilization. So actually, if you look at the trends for all of the metrics that we have generated and actually all of the different operationalizations, it starts to become quite flat and in some cases even decline. And I think that's really interesting because this is in a context where

the number of papers is growing so fast. It is like 60% of the papers in the corpus were published after 2018. And even with this massive growth of activity, you're seeing diversity starting to stagnate and even decline. This suggests that we might have a bit of a narrowing of AI research going on, which I guess is what gives the paper its name.

I see. Yeah. So, yeah, I found this very interesting that since 2012 at first there was this big increase in diversity you found and then basically now stabilizing and maybe starting to decrease. So it's maybe a good time to have done this research to know that's happening. Now, I think it is quite interesting to know

you were able to get the data to do this analysis. So can you speak a bit more about the corpus construction? I know it's from archive or some keywords, but maybe you can dive into a bit more of the details. Right. So basically the pipeline is, I'll describe the pipeline at a high level. So basically what we do is, so we have like a pipeline to collect data

data from archive. And I think like when while we were working in the paper, it was announced that, you know, I feel like the data, the archive dataset has now going to Kaggle or something. But we have our own pipeline to collect the data because actually what we do is we get the data from archive. We match it with this thing called Microsoft Academic Graph. It's like a scientific database that's run by Microsoft Cognitive Services.

And the nice thing about doing this is that then we get, for every researcher, we get their affiliation, which you don't get from archive. And then obviously once you have their affiliation, like the institution where they were working when they published the paper, then we can match this with this other database called Grid. There's a global research identifier. And this allows us to look at the geography. So where are these institutions based? Are they in the US? Are they in China? In what city?

And also you get like the type of institutions. So is it private sector? Is it educational or health or whatever? Then in order to identify the AI papers, this actually was quite complicated, you know, because in previous work we have just used topic modeling and because we're less interested in deep learning, it was relatively straightforward to identify AI.

Almost like what were the topics within that model that related to deep learning, because we need the terminology in this paper, because actually we're interested in the evolution of diversity in AI research over a longer timeframe. We needed to think about how do we generate a vocabulary, not just for deep learning, but also for symbolic AI, for expert systems, for statistical machine learning,

So how do we do this in a more, I guess, in a more automated way and data driven way rather than just coming up with a bunch of keywords that I guess people do, but sometimes can feel a bit arbitrary and a bit involved.

So what we did was, I guess I'll try to describe it at a high level, is we used the various archive categories that I guess seem to relate to different types of AI. So there's a category which is called AI. There's a category called neural computing. There's a category called statistical machine learning, statistics, ML. Another one is called CS learning.

And basically, we thought, okay, that's almost like a label dataset. That's like the categories within the dataset that where people doing AI and different types of AI might put their papers. Let's look for salient terms in those categories compared to the rest of the corpus. So what's like the unique terms in...

in AI, in neural computing and whatever. And then basically take those terms, expand, look for terms which are similar to them using fast text just to, I guess, not rely just on the initial seed, but make it a bit broader and increase recall. And then let's look for papers outside of these categories where those terms are also appearing. So they are almost like

that are using terminology that's used within the label dataset, but they are not in the label dataset. And actually, when we do this, we start to get that same categories that it's almost like they are not

In principle, you wouldn't expect them to be developing AI techniques, actually have a lot of AI activity. And obviously you would know this because computer vision is the key one that appears. Lots of papers using AI vocabulary,

Also from computer vision, from computer learning, lots of papers from actually CR, which is cryptography and computer security. Actually, lots of stuff around adversarial examples and things like that in those categories.

So then in a way we start to almost like snowball from this initial label data set into like the rest of the corpus and start to get a rich kind of picture of like where the AI activity is going on. And then, yeah, once we have these categories, then it's a matter of, I guess, starting to do the kind of analysis I was talking about before.

And yeah, as I said, like all of them, all of the pipeline is simply you can have a look at it in our GitHub repo. And also like all of the underlying data, including, say, papers matched to, you know, institutions, matched to geographies, matched to types of institutions. All of the papers we scraped from like all of these other websites, all of that's there for people to use. So they don't have to reinvent the wheel because it was a very

involved will to build. We wouldn't want other people to waste their time trying to reproduce this work when we can just put it out there and let others build on it. Yeah, it definitely sounds like it was a lot of work, but I think as a result, you have very intriguing data and analysis going on. Yeah. I'm curious, as far as the corpus kind of simple thing that you might answer is

Of course, we know that AI as a field has exploded a little bit since 2012. There's been a lot more interest, a lot more activity, a lot more people. Is the speed of growth of how many AI papers are being published every year, is that slowing down or are we still kind of going as fast as we have been going 2016, 2017 at this point? Yeah.

Yeah, it's growing very fast. I'd say this is speeding up the pace of production of papers. So yeah, a lot of activity. And again, that's why we find surprising that despite this proliferation of activity that we see the diversity stagnating. It's something like there's more people coming into the field, but they are concentrating in the same areas as those who are already there. So yeah, that's, I guess, one of the

I mean, I guess, through prices or, you know, areas where we wonder, you know, like what's going on here. Yeah. And then related to that question, what patterns do you find with the private sector companies? Obviously, they've had more involvement, but are they

less diverse than academia maybe or what is their diversity and yeah I'm just curious to hear whatever we made insights yeah so we basically I guess compare like their diversity in three steps so first we simply looked at the corpus of private sector research and compared with the corpus of academic research and compared their diversities according to different metrics and we found that in all metrics

Yeah, like diversity in the private sector was lower. You know, we thought, all right, but you know, maybe this is just caused by the fact that there's still more research going on in academia than in the private sector, you know, in terms of like the number of papers, because I guess academics jobs is to in some ways pump out papers in industry. Some people do it, maybe some people don't do it. So there's a smaller corpus. So maybe like the fact that it's a smaller corpus, maybe drives these results. So what we did was

Let's just take samples of the same size from the private sector corpus and from the academic corpus and take this, I don't know, like a bunch of times to calculate metrics and compare them, controlling for the size of the corpus. And still, like we found that for all metrics, for all parameters, there's more...

There's more diversity in academia and less diversity in private sector. And then we said, actually, let's look at the micro picture. And in terms of thinking of private sector, research in general and lumping together the papers by all of the private sector organizations and looking at the diversity, let's actually look at diversity at the organizational level. So what we did was take

for the last three years. For each organization, I think in the data that had more than 75 papers in AI, we basically calculated the diversity at the organizational level. And then basically we used this to create a panel

data set, we regressed diversity at the organizational level on whether the organization was a private sector company or an academic institution. We regressed it on the year. And then we had some fixed effects, almost like some dummies at the organizational level to almost be able to look at almost like

If you control for type of organization, if you control for year, also very importantly, if you control for the volume of papers being published by different organizations, can we almost like find different levels of diversity that tell us maybe something or the strategies that different organizations are following in how they do research, AI research?

And we present these results in the paper. I guess the headline is, you know, like for I think like maybe one exception out of nine with one exception, all of our models show a negative link between, I guess, this being a company and level of thematic diversity consistent with the idea that all things being equal, if you're a company, you're going to tend to be less diverse.

We find that organizations that have more papers tend to have more diversity just because they can explore more things and more ideas. And then we find that I think it's very interesting. Actually, when you look at almost like the organizational level picture, you know, it's like controlling for type of organization, controlling for size. You find some big differences in different organizations. And we find that

Actually, like say Google and Microsoft, after you control for the volume of activity that they have, they have relatively low levels of diversity. We find that open AI has very low levels of diversity. They seem to be very concentrated in a very small number of topics and areas, maybe consistent with this idea of diversity.

being very focused on pursuing one way to build AI systems based on very big models built with lots of compute. We find that organizations like DeepMind, IBM, Facebook actually seem to be more diverse. Anecdotally, maybe because they follow a bit more what DeepMind do, very often you find papers they write where they are saying, "Oh, we're going to look at symbolic logic. Here we're looking at Bayesian methods. Here we're looking at causal methods." It's almost like it feels like

They seem to be more diverse in the kind of techniques that they explore and they, and, you know, it seems to be reflected in the data. Something that's super interesting, I think, and this is one of the areas where I really want to dig further and actually connects with some new research that's been coming out by other, by other teams, uh, is the idea that actually when

When you look in the US, like actually the elite institutions like MIT, like Stanford, Carnegie Mellon, places like this, actually, when you control for their almost like the volume of activity, when you control for the fact that they are academic institutions and you control for time, actually, they have very low levels of thematic diversity.

So it's almost like they are not the places which are contributing to diversity in AI research. They seem to be part of the mainstream in terms of the techniques which are being used and very much not making the field more diverse, perhaps making it less diverse. And I think this is super interesting because I guess

There's this idea that these leading institutions, obviously, where you're based as well, and it might be interesting to actually hear your reflections on this, because they are so connected with the work that's going on in industry. And lots of people from these institutions going to industry, lots of professors have actually, in some cases, dual affiliations, industry affiliations.

and academia, because they actually very often collaborate with industry and they have access to the industry, industrial computational infrastructure and data. In some ways, these institutions are becoming quite similar to industry in what they do. And this is leading to almost like this homogenization of the field, at least in terms of the research activities which are taking place in the elite, including industry and universities. So yeah, I guess

Yeah, it's interesting when you start to learn, like dig a bit further and go beyond like this very kind of high level picture of, oh, you know, what does university do? What does industry do? And you look at individual institutions and the picture becomes more nuanced and you find some private sector institutions which are quite diverse and you find some academic institutions which are very homogeneous. Yeah, I certainly find that very interesting to hear as someone at Stanford and very

I can definitely see that making sense. There has been a lot of discussions in the field of, you know, are we just chasing state of art results? Are we just building, you know, different iterations of models about really trying to address deeper research questions? Are we kind of too addicted to metrics and evaluation? And I think a lot of people in academia

have a bit of angst about it actually. Yeah, it comes up quite a bit and people often feel that there's too much focus on metrics and too much sort of

non-interesting research which probably is reflected in the diversity right because then you know you see a paper and it's a kind of paper you've seen already like a thousand times yeah so um yeah it's something that academia i think is reckoning with in terms of like are we moving too fast is the pace such that we can't actually kind of pause and try something weird and different

And then it also relates to peer review right now, I think, where there's a lot of angst as to if you try something different and you don't evaluate on like 50 data sets and compare to all the previous work, it's harder to get published. Some people pause it and that's part of why there's less diversity. So super interesting to hear that you found that in your results and

uh, maybe a good wake up call to organizations just like Stanford. I think that is very, it's super interesting. And I feel that, you know, I guess like the story I was telling before it was like, um,

you know there's a story about like oh the private sector is coming in and you know it's making like the field more homogeneous because they are just focusing on the techniques they can apply like in their platforms and things like that i think like it's an interesting almost like flip side to that question you know to which extent that maybe like some of the osmos like scholarly racing you know the academic racing to get published the academic racing to get into the hot conferences

All of these behaviors as well, in a way, make the field more homogeneous or even spread into industry in a way that makes industry even less diverse than it could be.

I think this is like very interesting questions and I would love to look at them in follow up research where we start to look maybe more at, I guess, some of the complex dynamics of interaction between what's going on in academia, what's going on in industry. How do people move between these domains? How do they build in each other's work? Maybe using citations and things like that.

So, yeah, like a lot of... A very interesting kind of research agenda to build on. And again, that's the reason why I'm so excited to make all of the data and code available so maybe some people beyond us can pursue those ideas. I guess then there's the question of what this means for policy because I guess, you know, you could say...

Okay, so academic researchers need to fund their work in some way. You know, you have industry coming in with like megabucks, you know, a lot of funding, like lots of like opportunities for, I guess, career opportunities for researchers, a lot of data, a lot of compute, you know, like who's going to compete, like who's going to compete, try to compete against that, you know? And, you know, it's not like compete as in,

you need to match their investment because I don't think that's possible at this point. The amount of investment that you would need from the public sector would be massive budgets to be able to match what the private sector is doing. But there's this, say, funders in the US, funders in the UK, thinking almost more strategically about, okay,

What is it that we could do that is additional and is going to preserve diversity? And it's not just doing, finding more of the same work that the industry is already doing. How can we make the research that's going on in academia more distinctive and focus on maybe those, I think you said them like more weird kind of like,

like maybe like less, less mainstream ideas that we need to keep alive because they might be, you know, they might be the ones that generate the future, like the next like AI revolution. We need to keep them going because we, we don't know what they're going to be. We just need, we, we, we just know that we need to keep investing in a bunch of them. So they are there, you know, maybe for the next, to win the next hardware lottery, to win the next like application lottery. Yeah.

So, yeah, we have been talking with some like funders, with some like people in foundations, people in like, I guess, funding bodies in the UK about, okay, you know, how could they almost like start to use this thinking and maybe some of these indicators to inform what they do and start to almost like solve for preserving diversity rather than solve for

almost like getting people to write papers that are going to get lots of citations or are going to be doing the same kind of stuff that's already happening in the private sector. Makes a lot of sense. And speaking of that, one question I have is, I guess, probably finding that these elite institutions aren't very diverse was somewhat surprising. So I'm curious with respect to the results that you got once you crunched the numbers and did all this whole pipeline,

What were the main surprises for you and sort of the main interesting conclusions where maybe the narrowing is somewhat expected given deep learning, but what was surprising? I think that was the most surprising thing. It was like this idea that the leading academic institutions

seem to be following strategies which are very homogenizing, like in terms of like the like diversity and very focused on, I guess, the same techniques that are already been pursued into industry rather than doing more blue skies thinking and more blue skies work. I thought it was quite interesting that

And I feel like this kind of shows in the kind of like last bit of analysis we do in the paper, which is like this kind of like a bit of a play with, I guess, vector representations of papers that we created using BERT and actually projecting those on a 2D space and seeing who's close to each other. You know, the fact that

It sounds like when you see the diversity coming into the field and the field becoming very scattered and very kind of fragmented in a way that you might want if you want to have diversity. It's with all of the small institutions doing random stuff. It's almost like those are the ones that seem to be doing more diverse things. And there's this question for policymakers and for researchers. Is it sustainable to rely on

Some universities that maybe some people might think of as backwaters, in terms of the quality of their work or even their ability to retain top talent. Can we rely on those institutions to preserve diversity? Is that sustainable? Is that even efficient? What's the value of that diversity? How much of it is actually dead weight, like really poor quality or like...

you know, work done by people who haven't really got on with the times. So I think that's almost like something that came up from looking at that map almost like of the semantic map of the field and seeing like how you have like this cluster of like the top institutions at the center and then like this like very kind of scattered constellation of institutions around it.

So, so yeah, that was like surprising, a bit kind of concerning. I did something else that was interesting. And again, something I would like to really like pursue going forward was the fact that there seemed to be some maybe national differences in the international differences, sorry, in like the, in almost like the, the, the semantic map. So actually you see like all of these,

you know, like almost like the Western kind of institutions coming together in the same area, you know, like the ones we mentioned before and the tech companies. And then actually you see Chinese institutions and Chinese companies actually almost like in a not very far away, but it's almost like a different part of the map. And this suggests that they are focusing on different methods. They are focusing on different applications and use cases.

And maybe they are focusing again on this as almost like that different trajectory of AI, which, you know, it's interesting because it's keeping like the field more diverse, right? If they're doing different things, but it might be keeping the field diverse by looking at the technologies or almost like use cases that we must find quite worrying, you know, things like emotion recognition, pedestrian surveillance, was it like controversial technologies like facial recognition? So again, like it was interesting.

this kind of thing where I guess as a researcher I love it but it makes it quite difficult to generate a clear-cut message which is almost like looking at this map and realizing what some of the downsides of diversity might be in terms of inefficiency or the fact that preserving diversity might mean doing some pretty dodgy things or developing some very problematic variations of AI

Yeah, makes sense. And I agree with you that this conclusion about diversity was very surprising. Maybe as a whole field being more narrow makes sense, but

Saying that Stanford or Berkeley are similarly non-diverse to open AI is really eye-opening. So, yeah, I guess reflecting on that, you've mentioned a bit that maybe there should be government initiatives to preserve diversity. Do you have any sort of favorite ideas as far as policy solutions, or do you think this is an open question that requires some thinking?

So I'd say it requires thinking. I mean, we do say in the paper that because we haven't really looked at the mechanisms like underpinning diversity or the other that's at a very high level, like this idea of

the field becoming more homogeneous and private sector seems to be playing a role in that and private sector becoming more intertwined with elite institutions. But we don't really get into the nitty gritty of almost like what is causally determining changes in diversity that might almost give you a policy lever to pull. So we are a bit, I guess we get into the territory of speculation and hand waving about potential policies that could be explored to address this.

And the thing is, some of these policies are already being explored. So we just give some examples, I guess. One that I'm very interested in, I guess, also because of the place where I work, which is a very kind of mission driven organization, is to set up innovation missions, you know, say,

Let's find an area where AI is not being applied, maybe because it's not suitable for a deep learning kind of frame, or there's not enough data, or you need to have a high degree of explainability or robustness or whatever. It's actually within those fields, let's set up missions which are

to apply AI within the field in a way that encourages researchers to maybe come up with new ideas or think about almost like looking in the toolkit and taking two techniques, which maybe you would never want to use like in a larger scale recommendation engine, but you might want to use in these other fields. So start to encourage almost like more applied types of work where those techniques might shine. So I think that's interesting. In the UK, we have lots of mission work

I know that in other countries as well. So it would be great to see that really scaling up and almost like thinking, how would I say,

almost like thinking, targeting those policies are creating more diversity. We have like this other paper, like this kind of analysis of like AI and COVID-19. We published like, I guess when everyone was publishing everything about COVID-19 a few months ago. And actually we found that like most of the AI research looking at COVID-19 was computer vision to detect like COVID in like medical scans, you know, and you could see how

AI researchers were tackling the mission to, I guess, fight COVID with the data, you know, in the area where it was easier to apply the deep learning methods and computer vision rather than maybe thinking, actually, sorry, even if you've solved this, still not what's going to help us against the pandemic. You know, we need to be looking at other areas and maybe using other techniques. I think, yeah. So I think that in general, if we have mission-oriented policies to develop AI,

to tackle like various challenges and actually the idea wasn't let's use that let's see how we use deep learning to tackle this challenge and instead it was okay let's see what methods within ai not necessarily deep learning can we use to tackle this challenge i think that'd be quite interesting um i'll mention maybe another couple of them um i guess another one that's interesting and maybe again this is not like my area and it would be great to see what you think as well like but i think it's interesting is actually can you find

you know, other benchmarks, other metrics, other, I guess, ways of evaluating performance of models that create opportunities for the techniques that might perform better with those. I think that would be an interesting way of maybe like expanding the paradigm, I guess, the common task framework in a way that levels the playing field for the techniques.

And then I guess the third one is something that I think we're seeing in the US. We are seeing it in the EU as well. Also in the UK, the idea of developing infrastructure, data and computational infrastructure that researchers can use to do, I guess, state-of-the-art work without having to collaborate with industry. I think that's something that's quite useful. We have some new research coming out about flows of researchers from academia into industry.

And it looks like, you know, obviously researchers like money, like everyone else. And one of the drivers of them moving into industry is probably like financial. But we also find that AI researchers who move into industry,

We're almost like the best performers in academia and they become even better for performance in industry. So it's almost like people go to work in the industry to be able to do amazing work. And, you know, the question for policymakers and research funders and people thinking about infrastructure for public interest research is how do we almost like create infrastructure that enable those like amazing researchers who just want to find a place to do fantastic work.

How do we create infrastructure so they can do that work in the public sector and academia and not have to move into industry to do it?

Definitely. Yeah. And hopefully that is exactly what we will be tackling. I think conversations are starting, especially now in Stanford, there is a Stanford Human Centered AI Institute, which is trying to encourage more diversity, more interdisciplinary work, and also trying to tackle this question of how do we enable research in academia to sort of keep up and not fall behind and do interesting things. Yeah.

So, yeah, I think we have covered quite a bit of many interesting things from the paper. So my last question will be just, is there anything we have not touched on that you think would be interesting to our listeners to highlight that you can think of? Yeah, so my final observation is to say that this paper is just one within like, you know, almost like a new...

It's almost like a new field that seems to be like, almost like more complex analysis of AI trajectories and the role of the private sector in those trajectories. There's some really nice papers coming out now looking at this from different angles, using different data sets. I think there's a really nice MSC dissertation, I saw it came out last week, actually looking at progress in AI, almost like in terms of the metrics and starting to see some evidence of diminishing returns

I think it would be really interesting to almost like link that kind of analysis with the kind of analysis we are doing here in order to be able to almost like get a sense of the extent to which preserving diversity gives us, helps us to almost like preserve ideas that we can leverage to improve performance, like in the metrics,

So I think like, yeah, I would be very interested in, so, so yeah, I mean, all of the references are in the paper. I'm not going to go through them now because I don't think we have time, but I think there's like so much interesting work going on in this space right now. I think that, um, and I think that, you know, the, the, the frontier and where I really want to go next with this, I mean, obviously I mentioned before the idea of looking maybe at the more like almost like network dynamics of all of these, um,

So it's like, look at the impact, you know, and what is it? Almost like, how do we measure the value? Not just measure diversity, but measure the value of diversity in terms of like the problems it helps us to tackle, the innovations that it enables, the advances in the field that it drives, etc.

So I think that's something I'm very excited about looking at, like maybe using like some of the really cool data sets which are coming online, things like papers with code and stuff like that. And obviously, you know, if people in the audience are looking at those questions or are interested in those questions or have interesting data to look at those questions or, you know, are looking for data to look at them, you know, please get in touch because I would be very excited to have those conversations and I guess to take this research to the next level.

Fantastic. Yeah, so I'm hopeful that listeners got a lot of discussion. I know I did. I've learned quite a bit. Once again, the paper we've been discussing is a narrowing of AI research. You can go ahead and Google it to find the archive paper, which is open to anyone to read. And even if you're not too technical, you can get a lot out of just browsing it and seeing the work and the figures and

And as Juan mentioned, you can find also the source code and the data in the GitHub, presumably by Googling. It's all linked from inside the paper. So it all leads to the repo eventually. It's fantastic. So yes, with that, I think we're going to go ahead and conclude. Thank you once again, Juan, for joining us for this episode. And thanks so much for having me. It was really fun.

And yeah, I know I also had a lot of fun. And thank you also to our listeners for being with us for this episode of Scania Today's Let's Talk AI podcast. You can find articles on similar topics to today's and subscribe to our weekly newsletter of similar ones at scania2day.com.

Subscribe to us wherever you get your podcasts. And don't forget to leave us a rating and a review if you like the show or you have anything to let us know as far as how we can improve. And be sure to tune in to our future episodes.