Human Data is Key to AI: Alex Wang from Scale AI
30:56 Share

There will be a lot more divergence between the other lives in terms of what research directions they choose to explore, in which one of too many have breakthrough at various times. One of the home Marks of this next space is actually going to be da production. Basically no agent really works well. And turns out the internet, the pricing for model in prince fall dramatic, dramatic dramatically over two years.

If you've been listening to the asic extensive podcast for a while, you'll know we talk a lot about ai. We've covered the algorithms of power, alms and the computer required to run them. But equally important is data.

Our guest is as deep as you can get in this world of data. The fuel behind allam s. In fact, he even recently said, quote, as an industry, we can either choose data abundance or data scarcity. So what data exists today and what needs to be created, either measured or synthesized? Listening to find out, as I passed over to a sixty cy growth general partner sewing to properly introduce .

this eb sode, hey guys, i'm throwing general partner on the eight sixteen grow t welcome back to our AI revolution series where we talk to industry leaders, but how they are harney, the power of general ai. I guess this episode is Alexander wang, the founder and C. E, O of scale I, A company that has become synonymous gena I.

And the data needed to power advances in large landward models and beyond with scales work across enterprise, automotive and the public sector. Els is also building the critical infrastructure that will allow any organization to use the proprietary data to build the spoke gena I applications. For those of you who don't know alex, he is one of the most impressive ceos we've ever met.

And that saying something given a sixteen first about alex when he was twenty one and already the C. E. O of one of the fastest growing companies at its scale, what he found IT right before dropping out of MIT in two thousand and sixteen.

In this conversation with a sixteen z general partner, David George, I was discussing the three pillars of A I models, compute and data, and how creating abundant data is core to the evolution of gene. Alice also shares his learnings from the gross of scale has approached to leadership and what he thinks growth stage founder CEO tend to get wrong about hiring. Let's get starting.

As a reminder, the content here is for informational purposes only, should not be taken as legal, business tax or investment advice or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any asic sense e fund. Please note that a six senses and ezoe illiac may also maintain investments in the companies discussed in this podcast. For more details, including a link to warn investments, please see a six and see outcome slack disclosures.

Were very excited today to have alex waging the founder and C E of scale A I with this essential manner. Thanks for having me. I always love talking to you, and I always learn a ton. But maybe to start one of you to sell us a little bit about what your building in scale eye in the world diving yeah so it's ale.

We're building the data boundary for A I. So taking that back, A I boils down to three pillars. All the progress we've seen has come from compute data and algorithms and the progress among all three of these pillars, computers empowered by folks like video. The new advancements have been LED by the large labs like open eye and others, and data is fuelled by scale. And so arable is to produce the frontier data necessary to fuel frontier level advancements in partnership with all the large labs, as well as enable every enterprise and government to make use of their own proprietary data to fuel their fronta aid development.

So on this topic of frontier data practically, but how do you actually get IT?

Yeah, I think this will be one of the great human projects of our time, if that sense. And I think that the only model that we have in the world for the level intelligence that we seek to create is humanity. And so the production of frontier data looks a lot like a for marriage between human experts and humanity, with technical and algorithmic techne around the models to produce huge amounts of this kind of data.

And by the way, all the day we've produce today, the internet has looked like that too. The internet, in many ways as this, like collaboration between machines and humans to, of course, large amounts of content and data. It'll look like the internet on steroids. What happens if the internet basically, instead of just being a human entertainment device with this, like by product of data generation, what if I worked just this lower scale degeneration experiment?

So you have a very unique perspective into the state of the industry. So how do you character ze the state of models, language models, right now? And I love to sort of get into things like market structure, which is sort of.

what's the state of the industry right now? Yeah, I think we're sort of closing in at the end of maybe face two of english small development. I think face one was the early years of almost pure research.

So face one hallMarks are the original transformer paper, the original small scale experiments on GPT all the way leading up, probably until, like GPT three, was this sort of face one. All research very, very focus on to relax mosque tinkering and always make advancements. And in face two, which is sort of maybe GPT three till now, is really the sort of like initial scaling face.

So we had GPT three that were very well. And then open the eyes to start with, really scaled up these models to g pty beyond. And then many companies, google, anthropic, meta X A I. Now many, many companies have also joined on this sort of race to scale of these models to incredible capabilities. So I think for the past, let's say, three years, it's almost been more about execution than anything.

It's a lot of just engineering like how do you actually have large scale training work? Well, how do you make sure there aren't where bugs in your code? How do you set up the larger clusters? A lot of execution work to get to where we are now, where we have kind of a number of very advanced models. And then I think centering the face where the research is going to start matting a lot more like I think there be a more divergence between the little labs in terms of what research directions they choose to explore and which one ultimately have breakout through the various times. And it's sort of an exciting alternating phase between maybe just blow execution verses through a more innovation power cycle.

They've kind of gotten to a point where consider like abundant compute, but they've had enough compute that they needed in order to get to the models wherever that's not a constraint necessarily, they've kind of exhausted as much data as possible can of the frontier labs. yep. And so the next thing will be breakers on that in the advancing the ball on the data .

side yeah I think basically yeah you look at the pillars compute or of the continuing to scale up the training clusters. So I think that direction is pretty clear on the algorithms. I think there are to be a lot of innovation there.

Frankly, I think that's a lot of lives are really working hard. I think on the peer research of that and then data you can have looted to IT, we've kind of run out of all the easily accessible and easily available data out there is exact. And so love people talk about, this is the data wall of in this wall, we've leverage all the publications ailed data.

And so one of the a home Marks of this next space is actually going to be data production. And what is the method that each of these labs is going to use to actually generate the data necessary to get you to the next levels of intelligence? And how do we get towards data abundance? And I think this is going to require a number of fields of sort of advanced work and advanced study.

I think the first is really pushing on the complexity of the data. So moving towards frontier data. So a lot of H, A lot of the capabilities that we want to build the models, the biggest blocker is actually a lack of data.

So for example, agents has been the buzz d for the past two years. And basically no agent really works well in turns. Others just no agent date on the internet. There's no just pool of really valuable agent data are sitting round anywhere.

And so we have to figure how to produce really hy given example of like what would you have to produce? So we have some work coming out on this soon, which demonstrates that right now, if you look at all the french or models, they suck at composing tool. So if they have to use one tool and then a note l, let's say, they have to look something up and then write a little python script and then charge something, they use multiple holes in a row. They just suck at really, really bad, realizing multiple and row and that something is actually very natural for humans to .

do if it's not captured anywhere, right? That's the point. Take the capture of somebody going from one window to another into a different application and then feed that to the model.

So learn, react. Yeah ah so so these sort of reasoning chains through when humans are solving complex problems we naturally will use of ultra tools, will think about things or reason through what needs happen next. We will hit errors and failures and then will go back and look like we consider you know a lot of these raising changes, agents, chains are the data just doesn't exist today.

So that's an example, something that needs to be produced. But taking big step back when he had on data, first is increasing da complexity, so wards frontier data. The second is just data on the increasing the data production, capturing more of what humans actually do in the field of work.

Yeah both capture more what humans do. And I think investing into things like synthetic ata hybrid data. So utilizing the theory, having humans be part ops, you can generate much more high quality data.

We need basically just the same way, I think with chips, we talk a lot about chip foundry ies and how do we ensure that we have like enough means production chips? I mean, same thing is true for data. We need to have effective data founders and the ability to generate huge amounts data to fuel the training of these models. And then I think the last leg of the store, which is often rated as measurement of the model and ensuring that we actually have no, I think for a while, the industry is just sort of like, okay, we just add a bunch for data and we say good model is out, a bunch for data with model is. But we're going to to get pretty scientific around exactly what is the model not capable today and therefore, what are the exact kinds of data that need to be added to improve the most performance.

How much of an advantage do the big tech companies have with their CoOperation of data versus the independent lads?

Yeah, all there's a lot of regulatory issues that they have with utilizing. They are existing at purposes you know you can look through this is before all this gender vii work. But at one point, met IT is some research that utilized basic all the public insulation photos, along with their cash tags to train really good image recovery tion algorithms.

They had a lot of regular or problems with that in europe, like was IT turned out to be a huge painting. yes. So yes, so I think that that's one thing that kind of difficult to reason through, which is to what degree from a regulatory perspective, particularly europe, these companies are going able to utilize their data manages.

So I think that was kind of T V D. I think that the real way in which a lot of but large lips have just dramatic uh, advantages is just they have very profitable businesses that can provide near infinite sources of capital for these A I efforts. And I think that that's something that i'm watching pretty intently. I'm workers to see how plays out.

There's this whole question. The industry is like early over investment. And if you listen, the earnings is calls of the act companies. They are like, look, our risk is is under investing, not over investing. What do you make of that?

Yeah I mean, if you think about, let's take the incentives of any one of the C E O of the put yourself in the shoes of a or march a and to your point, if they really nail this A, I think they could generate another trillion dollars of market cap, probably very easily if they really are ahead of, and the prototyping in a good way, the trillion, those of market cap cut him no brain.

And if they don't invest the extra, whatever IT is, twenty or thirty billion of capex per year and they miss out on that. And then there's some real extension al risk, I think to for each of the large the each each form um yes, all their businesses are potentially deeply disrupted by AI technology. So the risk reward for them is very obvious so that I think the big pictures thinking and then from a more tactical level, I think all of them are going to be able to pretty easily recruit their captain investments, just by worst case, making their core businesses more efficient, effective.

So for example, like you get G, P U utilization for a facebook advertising like google, they make their advertising systems a little bit Better. They can recruit billion years just by you have Better performance performance. I can use the investment if IT drives up cycle. I mean, these are things that I think pretty clear.

Look at generally great for the industry that they are investing so much capital because they also in the business of renting this computer, out case of google, microsoft.

they are in the models are making their way, like lama, three per one is open source. And so even the literal fruits of all the investment are becoming broadly accessible. And so the surplus generated from the this is kind of insane.

It's insane. okay. So that's a great segway into market structure in the model later.

So what do you think actually happens? Are there the few players that we've all identified now, the hand fall and they all compete? Do you think it's a profitable business? What impact is open sources have on the quality of the businesses? Take us a couple years ahead and give us your forecasts?

Yes, we've seen over the past, even just like year and a half, the praising for mal inference fall dramatic, dramatic, dramatic two order matic over two years. And so is this shocking thing that IT turns out intelligence might be a commodity? But no, I mean, I think that this huge sort of lack of pricing power, let's say, on the model air certainly indicates that renting models out on their own may may not be the best long term business. I think it's likely to be able to immediately long term business.

Well, yes, IT depends on the breakthrough thing, which is the earlier point, right? To the extent that someone actually has a double breakthrough and multiple people have done breakthrough like the potentially market .

structure different. So two things, if meta continues open sourcing, that puts a pretty strong cap s to the value that you can get from the mare. And then too, if at least a handful of the labs are able to have similar performance over time, then that also dramatically changed the pressing equation.

So we think that is not one hundred percent, but chances are the pure model renting business is not the highest body business, whether are much hieratic businesses are going to be above, below. So below, I mean, in video a is are seen incredible business, but the clouds also really great businesses because turns out it's pretty hard logistically to actually set up large clusters of GPU. And so the cloud providers actually pretty good margins when they range out in .

the traditional data center business is very much a scale game, right? So they are massively benefit goal to be small er players.

Yeah exactly. So I think pixel shovels. So if you're under the model layer, I think there's great businesses there.

And if you're above them all there, if you're building applications, ChatGPT is a great business. And a lot of the apps in the started prom actually are working pretty well. I mean, of them are quite as biggest tetch pity, obviously.

But a lot of apps, if they nail the early proud market fit and of being pretty good businesses, great businesses as well because the value that they generate for customers, if they get the whole user experience correct, far exceeds the inference costs of the models. There's some cool stuff here, right, I think, and topics launch of artifacts in close. It's like the first pin drop of this major theme of all the laws are going to be pushing much deeper product immigrations to people to drive high poly businesses.

So that'll be the other story is I think we're going to see a lot of iteration at the product layer and the product level. The sort of boring chap hots is not in the end product that's actually iteration. And the product innovation on cycles very hard to predict because I mean, opening, I was surprised how popular chat P.

T. was. I don't think is like super obvious to me, anyone in the industry, Frankly, what exact products are gona be the ones that hit and what's gona provide the next legs of growth. But you have to believe that an opening iron anthropic can build great applications businesses to for them doing long term for 是 yeah then it's .

what drives competitive advantage。 I will see you have the model, a tightly integrated product on top of IT and in the good old fashion, motes from there. Yeah work flows, integrations.

all that stuff. I think you can clearly see the thank you. I mean, like both open a eye and anthropic chief product officer within and two months of each other and like .

it's a sort of a change of tune where, like I know, we're very purely focused on this and it's okay. I think there's the realization hit. So yes, I makes you've got an allocation business really interesting customers. What are you hearing from enterprises is so like how they're actually putting this into place?

I think what we've seen is there was a huge mental excitement from the enterprise. A lot of enterprises were like, shit. We have to start doing something.

We have to get ahead of this. We have to start experimenting with A I, I think that that let them to this fast. I O all food like A I stuff.

Let's go trial of IT. And some of the of things are good, some them aren't good. But I think regardless, it's been this big fancy, much fewer of the P O, S have made to production than I think the industry overall expected.

And I think all of enterprises are looking at now. And the dome's day that they thought might have happened hasn't really happened. A I has not fully terraform and transformed most of the major industries. Like it's not like totally know .

this sort of marginal stuff. It's like efficiency gains and support on some of the creative tests and things like that. Yeah exactly othe wise is pretty late.

The thing that we think a lot I was like what A I improvements or A I translation or A I effort that we're working on actually can meaningly drive the stock Price of the companies that were working. So that's what we encourage all our customers to really be thinking about because at the end of the day, the potential there there's laden potential for almost every enterprise to implement A I at a level that would meaningful ly boost their stock Price.

Mostly the former consists efficiency as well today in the form of css savings, but then also much Better customer experiences like I think and a lot of industries where there's lot more menu interaction with customers. You should be able to drive much Better customer interactions if you have more sAnitation and you really believes more automation and then those eventually would make their way to gains of market chair with his bike bandanas. So that's we're pushing our customers towards.

And I see IT some of the cees that we work with. They're all on board and they understand that it's going to be a multi year investment cycle they might not see in the next quarter. But if they actually pull through the other side, they're going to see mass of transformations.

You I think that a lot of the friends around small use cases and more marginal use cases, I think that's good. I think is exciting. I think they should be doing to me that's not what we're all here to do .

is very much like application al lers, like very much like face one right now, which is and yeah the some automation, but it's largely like chap ots. My hope as a start of investor is that over time, there's a window that opens for the startups where product innovation will help them to win and beat.

The in comments like my partner along pill is this phrase, which is is the start of going to get to dirigo tion before the incoming finds innovation. And I think there's an opportunity for IT, but it's like the tech is too early right now. yeah.

And if you would agree with that, but I think the tech is too early to imagine. But again, because it's mostly saving and the energy on the then that's not really enough to disrupt large content that is already kind of like push the way through all the cost of growing distribution.

How does anybody think is the data inside of enterprises? Like you ve said, jp morning has whatever fifteen pet by its data. But like, is that overrated? How much of IT is actually useful because today, most of that data has not given them some meaningful competitive advantage. So do you think that actually changes?

I think A I is the first time you could see that potentially change because basically, obviously, of the whole big data wave, big data boils down to Better analytics, which is helpful, like marginally helpful for business making, but not deeply.

They changed the .

product work exactly, whereas you actually can't imagine some massive translation the way the products were. So let's take get any big bank, a lot of the valuable interactions between a user and a large bank like A G P. Morgan or Morgan Stanley, what not our human driven or people driven and know they try their best to ensure that the quality experiences very high across the board.

But obviously, with any large processor is only so much you can do to sure that. But all of your prior customer interactions and all the ways in which your business has worked historically is the only available data to bible train models to do well at this party. Their task. And if you think well, like alth management.

t is very rich.

yeah, huge amounts of IT. So I think that all of the data is probably super relevant. Taxation transforming your business, but some of the day is hyper valuable.

So I think you know enter Price have a lot of trouble and chAllenge around actually utilizing any amount of data that they have, right? It's poorly organized. It's sort of all over the place.

They pay consulting firms tens of minutes of dollars, hundreds of minutes of dollars to do these data migrations. And it's even after that, no change of results and no change results. So I think storage ally very difficult place for enterprises to really drive transformation. And so in some ways, this is the race. Are they going to be able to figure out how to utilize and leverage their data faster than some startup figures out how to .

like somehow get access street of useful different product with a little bit of shifting gears, how you run your company and how culture company one of the things that you've talked about is a mistake that you ve made during the good times of twenty, twenty and twenty one around hiring. And this notion that in order to scale, you had to hire a ton. And it's something we saw with all report for the companies was like, hey, this war for talent and IT meant that we ve got to go higher, or we get to go higher, or we get to go higher. So what are the lessons that you learn through that process and then how you change how you've .

done things after that? So over the past few years, we've basically capture have count flat. I mean, we've grown IT very slightly as the business growing, but the business itself is five x or six x. You know the businesses growing dramatically and the take away from this entire processes, IT feels very logical that more people. Better results in more people equals more stuff being done. But rather paradoxically, I think if you have a very high performer team in a very high performing org, it's nearly impossible to grow IT dramatically without losing all that high performance and all of the winning culture.

Yet reducing the communication in ordination overhead actually increased productivity.

That's not only true, and I think it's actually something even deeper, which is that a very high performing team of a certain size is almost like this very intricate sculpture in this interplay between all the people in the team. And if you just add a bunch of people into that, even if the people are great, like you just screw the whole thing up. And no matter what, as you add people, you're gonna were russian.

To the mean, if you kind of observe companies that do scale how carload and pretty good to their finial salts, I think they acknowledge that gresson scaling of large sales, for example, yes, sure. You acknowledge that you're going to have that in ogresses an we just Operationalize, ed, that you're like a little bit above the mean. And if you're able to do that, then the whole equation still works financially.

And i'd say sales .

of different product yeah totally course. But our observation is just startups work because you have very high performing teams and you want to keep those high performing teams intact as long as you possibly can know. I think a common start up failure mode is that you have something that works, but everybody in the companies really junior, so then things are scaling, but all the wheels are kind of falling off. Your investors tell you age to harass some executives. You go to these searches that are somehow uniquely saw crushing every time but through of your great IT works .

half time yeah so .

you the exact searches, you bringing exact and then you give the exact a lot of rope and your exact say, hey, we need to hire massive team for us to hit our results and you're like, yeah, I mean, i'm pretty experiences, you similar experience. Let's do what you say and you let these big teams sort of be built. And the reality is, I think this almost always result in brain.

I think that this isn't to say that you can hire executives from the outside, but I you need to do when you hire e executions from the outside is they really get steped in how the company works. And before they make any major sweeping suggestions, they get into the rythm and the Operations of the company, and they understand why is the whole thing work in the first place? Why are the things that are working, working? And then they make off for suggestions. Initially, they take small steps, and you sort of like you trust and verify each of these small steps and eventually maybe they can make more sleeping suggestions. But IT should be at a point where they have a clear track .

of IT of making small steps that have been really Better. And it's interesting and very tangible is a executive and it's not the any of the yeah I think .

that this kind of exact fantasy that i've noticed, which is any either I think executive are great people. They're like turn around but there is a tendency for an executive fantasy, particularly for like silk eli companies with Young founders and what not, which is oh, i'm going to come in and i'm going to fix this whole thing. I'm going to make this professional Operation.

You're recruiting teammates at in the day. You're not recruiting like some magic one. You're recruiting a teammate who you believe over extended period of time is one of great judgment in making repeated decisions about the business.

But in this where we've made mistakes like you're not buying some magical bag of goods that is going to bring this magic formula into your business, that will always also make the whole thing work. On the flip side, there's a founder fantasy, the founder fantasy or the founder C E. O.

Fantasy, which is, oh, i'm not just higher bunch of incredible exacts show me fucking and prose and then i'm in A O stuff. I to do all the stuff I don't want to do and I may be able to sit back and wash the machine work. And that's also extremely unrealistic because the foot size is also true. The reason that you are a good Fanny o is because you make very good decisions over, over again, over example of time, and to pull yourself out of those decision making loops.

we would be kind of crazy. That's a Better we've seen a lot, which is on the heart IT. I'm going to step back a bit and then its oh IT realization that like some big decisions go wrong and wait. Is the point .

of me being here? Yeah I think IT can work if your industries is very stable potentially well.

look at any public company when they change ceos and the stock Price moves like two percent and it's like, oh, okay well so IT doesn't really matter. That is a cop but that is very different from a high grove start of that run by founder.

exactly. yeah. And I think that a lot of startups and a lot of companies are valuable because of an innovation premium.

You know person investors believe that founder that companies are going to out innovate the market. And so your job is to out innovate the market. So you Better .

be in the started yeah for sure. How about imi? So you recently rolled out this concept. I think like half of my x feed was praising you and more than half some portion of x speed was yelling at you. Talk about the concept and what are your observations of rolling IT out so far?

yes. So mi, we basically rolled out this idea of merit, excEllence and intelligence. And the basic idea is, in every role, we're going to hire the best possible person regardless of their demographics, and we're not going to do any sort of quoted based optimization of our workforce to meet certain demographic targets.

That doesn't mean we don't care about diversity. We actually care about having diverse pipelines and diverse top of function for all our rules. But at the end of the day, the best, most capable person for every jobs going to be the one that we hire is one of these things that was mildly controversial.

But I think is also if I just take a big step back as to who should companies be hiring, I think it's kind of the most people and obvious became big questions like how much social two companies have in what they do. My take is I Operate in a very competitive industry scales. Role is to help fuel article.

Intelligence is very important technology. We need incredible smart people to be able to do this, and we need the best people to be able accomplish this. I think that this is something that, you know, I think most people at scale would say I was of like implicitly true or sort of IT wasn't like a departure from how many of us thought of what we do at scale. But IT was really valuable for us to qualify IT because that gives everybody confidence that even if this is how we Operate today, companies change over time. We're not going to change this quality.

Well, this has been awesome, going to close with an optimistic question and forecast, which is what is your sort of own view of our definition of agi? And what is your expected time line to winging reach that?

Yeah I like definition this that sort of like, let's say, eighty plus percent of jobs that people can do purely a computers of digital focus jobs, A I can accomplish those jobs. It's not like imminent. It's not like immediately on the horizon. So on the order of four plus years, but you can see the glimmer and depending on the i'll read, make innovative cycle to talk about before could .

not make that much sugar. No, it's also very exciting. Well, I was thanks for being here going to share with you was always learned the time, really appreciated .

yeah thanks .

for any all right. That is over today. If you did make IT as far, first of all, thank you. We put a lot of thought into each of these episodes.

Ether it's guests, the calendar touches cycles, where are amazing at of your Tommy until the music is just right. So if you like what we put together, consider dropping is online at break this pocket or com, flash a extensive and let us know what your favorite pit is. It'll make my day and i'm sure tomes to catch you on the flip side.