Money Talks: Can Math Really Crack the Stock Market? (Encore)
51:45 Share

Apple Card is the perfect card for your holiday shopping. You can apply on your iPhone in minutes and start using it right away. You'll earn up to 3% daily cash back on every purchase, including products at Apple like a new iPhone 16 or Apple Watch Ultra. Start holiday shopping for your friends and family today with Apple Card.

subject to credit approval. Apple Card issued by Goldman Sachs Bank USA, Salt Lake City branch, terms and more at applecard.com. Happy holidays, folks. It's Felix here. We at Slate Money are taking a little bit of a break. So we're rerunning the fabulous Mary Charles interview from a few months ago. Mary Charles is kind of the best financial journalist in audio journalism, if not in journalism

Generally, she wrote a fantastic piece for Bloomberg about valuations in the stock market. She is very funny. And if you want to know what that has to do with spherical cows, listen on. Hello. Hello.

Welcome to Money Talks from Sleep Money. I'm Felix Salmon of Axios, and this is a special one because I get to nerd out with the most awesome and fabulous person in the world, Miss Mary Childs. Hi. Hi.

Hi, Mary. Hi, I'm really honored to be here. And thank you for that intro. It's mutual. Introduce yourself, Mary. Who are you? I'm Mary Childs. I'm a co-host of Planet Money. I'm the author of the book, The Bond King, which Felix can admire in my background. About Bill Gross, no less. Yes, Bill Gross and the bond market. And then I also just wrote this article for Bloomberg Markets Magazine that was pretty wonky, pretty in the weeds, and a bat signal for Felix Salmon. It's so good. You're...

is an amazing article and it opened my eyes to something that I guess in principle I knew, but in practice I'd never thought about, which is that we like to think of stock prices as being these objectively true things. And we can just look them up on the internet or look them up on the consolidated ticker or something. And it is a fact. And it's,

Then when you kind of like start clawing underneath the surface, you realize that with a little bit with individual stocks, but certainly when it comes into sort of aggregates of stocks, there's just a lot of very interesting fuzziness in here. And not everything you think is true is actually true. And you have this amazing story. And it took you were telling me it took you like 11 months. Yeah, never work on a story is is the lesson.

Why don't we start at the beginning? Where did you find this story? Who are you talking to? And what were you thinking?

Oh, so many thoughts. So this all began because I went to a business law conference. And I'll just tell you, when I heard that I was going to a business law conference, I was going to give a talk. I was still talking about my book and doing like, this is what Bill Gross did in the market, telling people how great my book is. But I sort of misunderstood what an academic conference was. I didn't know about academic conferences, embarrassingly.

so I heard conference. So I like put on my stage makeup, you know, I like get ready to like do a normal, like a conference conference. And I show up and it's like a small room. It's like a seminar and it's all just professors. And I'm like, Oh my God, I'm wearing 200% too much makeup for this. Like what have I done? But I sat through the whole thing and they were like, you don't have to be like, this isn't,

interesting for you, right? Like you can go and I was like, I'm having the best time. Like they were just talking about their papers presenting. And then, you know, in an academic conference setting like this, you present your work and the next person comments on it. And so it's, it's really interesting. It's how they like revise when, when they're in kind of in the process of polishing and pushing forward their own research. And, um,

It was so fun. But just someone made an offhand remark about this paper that had been out there about how Fama and French were, you know, that their numbers were like being juiced or something. And I was like, I blacked out. Now, I want to just say like...

That's a subjective interpretation. Before we move forward, I just want to put a little disclaimer on that. But I literally blacked out. I was like, sorry, what? So Eugene Fama and Kenneth French are two of the biggest names in empirical finance and just enormously well-respected, the fathers of this entire world that we study of looking at asset prices and understanding them and trying to articulate the relationship between risk and reward and all of these things. And to say

allege that is huge. And I like start Googling around. I find the paper. I reach out immediately to as many people as I can. And I'm like, what's the deal? And I get, it turned out that like the entire business law community was like so inflamed about this research paper called noisy factors that had come out that basically discovered that the, the

asset prices, the data set that Kenneth French and Gene Fama provide for free to everyone for use in research and whatever else, that those numbers were changing. And there was, depending on who you ask, inadequate clarity or disclosure around that. And the thing that was so funny to me was that there's this cultural schism here too, many in fact, between practitioners and

And within academia, business law and empirical finance or academic finance. And I was just fascinated. I was just going through a Disneyland moat tour. I'm in the little...

bucket seat and I'm like watching all these things unfold in front of me and like all these different characters that I've like never really looked at close up or telling me about their worlds and and basically like this is a huge huge deal if you care and this is a total nothing burger to so many people in asset pricing those people I don't even know they're not for you I don't even know her like seriously I mean it's a funny reaction to get yeah I need to pick up on this because I

When you first said I was at a business law conference, I'm like, wait, what law? And then you said, well, there's this whole difference between the way that the business lawyers look at this and the way that the empirical finance people look at it. And what we are talking about, to be clear, is just like,

a historical data set of stock market returns. And the allegation was that, you know, that if you looked at value stocks and how much they returned in 1977, then a few years ago, Farmer and French would have said they returned this much. And then now they're saying that they returned that much. And in fact, those returns seem to have been going up over time. We will get to that in a minute. But before we get to that, what is the connection between this and

law? Like why are the lawyers into this? Yes. And this is, you've actually identified what I think is my favorite part because the Fama French data, the Fama French factors, as they're called, are used so prevalent. They're so prevalent throughout our society in ways that we often don't see. And in particular in like securities litigation. So this becomes extremely important. These numbers matter so much in the business law community, because if you're

Here's an example. There's a company, a manager makes a mistake. They do something stupid and the stock goes down. And the shareholders are all like, you did a very stupid thing. The stock went down. You owe us money for your stupid mistake. How do you know the counterfactual of how bad that mistake was? You make up the counterfactual by using the Fama French factors. So they basically provide this alternate universe version. And that's how you calculate damages, settlements.

So basically, if I'm the CEO of a value stock, and then I embezzle a bunch of money and the stock goes down, as our mutual friend Matt Levine would say, everything is securities fraud, then I get sued for securities fraud. And the damage is the difference between where the share price is now in the full knowledge of my embezzlement versus where it would have been

Had I not embezzled. And the way we determine that is we say, well, this is what value stocks like yours did. And we use the farmer and French factors. Why we don't just use any other index of like my comparable peers. I don't know. We use the farmer and French factors because they're farmer and French because farmer and French, they have like Nobel prizes and a million citations and they are the gold standard and no one can argue with farmer and French. And,

Then someone argued with Farmer in France. Well, it's funny because...

They are the gold standard in large part because they've been providing this data set for free online for ages. So the fact that they, it's like this, like Eugene Fama called it a good deed that they did, that they like put out this data set. And indeed it caused so many people to be able to look more closely at the relationship between risk and reward and yada, yada, and do all this great research and push the ball forward. And now we know a little bit more about the world or we think we do. But at the same time, like,

That also magnified their influence. The fact that they shared that data freely and openly like that made it the benchmark, made them be these central parties, and made the data all that more important. And so tell me who was on which side of the cleavage, the business lawyers versus the empirical finance people. Which ones cared about this and which ones didn't? So business law is this...

I mean, I think of it as this world that demands precision, right? Like you need to know within some degree of, you know, reasonable extrapolation that this is how much you, the CEO who embezzled the funds, this is what you cost me. Because we need to dole out the damages. Like we have to act on that. And so we need some way of estimating.

And in the empirical asset pricing world, I think there's more of an acceptance of like, we don't know anything. What are you talking about? The idea of precision is a lot. Yeah, the empiricists...

in a world of statistics and the lawyers live in a world of facts. And if you live in a world of statistics, then everything is fuzzy. And if you live in a world of facts, you're like, but I need the fact. And you're like, but this is an impossible counterfactual. Like, of course it's not a fact, but the lawyers are like, you need to give me the fact. Exactly. You hit the nail on the head. And it made me so happy to cut. And like,

This is also because the asset pricing people have engaged with the data set and know that it changes. You know, they're like, oh, yeah, that changed when I was trying to do my PhD paper. And I was like, it's whatever. Didn't affect my outcome. So I just moved forward. Like everyone seemed to have some degree of familiarity with this. And the researchers who wrote the paper that caused this whole furor, they were shocked.

kind of shocked that other people noticed the numbers changing and then didn't do anything about it and just like went on their merry way but like you know you can't solve everything but some people were just like yeah that's numbers that's science we'll be right back after this

Apple Card is the perfect card for your holiday shopping. You can apply on your iPhone in minutes and start using it right away. You'll earn up to 3% daily cash back on every purchase, including products at Apple like a new iPhone 16 or Apple Watch Ultra. Start holiday shopping for your friends and family today with Apple Card.

Subject to credit approval. Apple Card issued by Goldman Sachs Bank USA, Salt Lake City Branch. Terms and more at applecard.com.

So your dollar goes a long way.

Visit Progressive.com to see if you could save on car insurance. Progressive Casualty Insurance Company and Affiliates. Potential savings will vary. Not available in all states or situations. This podcast is brought to you by Progressive Insurance.

You chose to hit play on this podcast today? Smart choice. Progressive loves to help people make smart choices, and that's why they offer a tool called AutoQuote Explorer that allows you to compare your Progressive car insurance quote with rates from other companies, so you can save time on the research and could enjoy savings when you choose the best rate for you. Give it a try after this episode at Progressive.com, Progressive casualty insurance company and affiliates. Not available in all states and situations, prices vary based on how you buy.

So there were two things that really jumped out at me from your article. The first one was the chart. And there is just the most amazing chart, which is basically the historical performance of value stocks over the past however many decades. Well, you describe it.

So yeah, the three factors that we're looking at in this data set are the market, which everyone agrees exists, and then value, which is kind of you should buy cheap stocks, which seems to make sense, and size, which is small cap investing, basically. And in this chart that you're focusing on, we basically look at a portfolio of $10,000 invested in 1926, which bets against growth stocks, so betting on the value stocks that are cheaper for whatever reason. So from 1926 to 2005,

you would have gained an average of 0.41% per month. And that is using the 2005 vintage of the data. So that $10,000 portfolio would be now $250,000 in 2005. So great. We love it. But if you checked the exact same figures, the exact same data in 2022, so it's now a different vintage of the data set, the numbers have changed. The value, that 0.41 becomes 0.4

Is that big? I don't know. That sounds little, right? That's not that much. But that would add up to a difference. That would give you $400,000 instead of $250,000 over the same period. So that becomes the power of compounding. It's always beautiful to behold. But the numbers sound really little on an average monthly basis.

but they become enormous and it matters. Yeah. So according to what farmer and French that what their best guess or their best calculation is,

of the return to $10,000 invested in value stocks in 1926, if you calculate that in 2005, when they did these sums in 2005, they came up with an answer of $250,000. And then they went back and did the same sums in 2022, and they got an answer of $400,000. Right, exactly. And obviously, in some kind of platonic way,

you know, ideal world, that would never happen. There were a bunch of stocks, they had an actual return. Like, this is a known fact of how much they return. So what happened? There are a number of things that happened. And this is because when you construct any data set, you're making 1000 tiny arbitrary choices is what basically everyone how everyone phrased it to me.

And you're saying, okay, when I look at the data for this company, does this count as value? Does it count as growth? Like, does this company, is this company big or little? Or what are the, what's the profit ratio of blah, blah, blah. Like what's the ratio of its assets to its liabilities versus the, and those, you know, you can draw kind of arbitrary lines. Like I might come up with a different threshold than you might come up with as to what constitutes value or what's

the most fundamental and completely inarguable reason to change data is that the underlying data changed. So they draw from these two data sources, CRISP and Compustat, and those data providers themselves changed data. So they'll say, oh, we went back and looked and there was a mistake. And in 1950, we actually had this cell double counted. Or, oh, the number of shares outstanding, it changed. We got better information for whatever reason. We found this

treasure trove of information from ancient New York Times archives, and we reconciled it differently, and this is what happened. And all of these things will change the outcomes, will change your outcomes as the person using a data set that's like three steps removed from that little tiny change. And again, it sounds little, but then these things add up.

But the weird thing and the slightly suspicious thing is that they all seem to add up in the same direction. Yeah. Yeah, it's weird. Yeah. Like if it was just random noise of some companies dropping out, some companies coming back in, some companies having a share count which was too high, some which was too low, then all of those like individual, you know,

double counting sell errors and all the rest of it would basically cancel each other out. And the end result of 250,000 would barely move because you have the law of large numbers here. When we're talking about the universe of all value stocks, that's a big universe. And changing one or two or 20 individual stock returns shouldn't have that much of an effect on the whole thing.

corpus. So the first reason why this smelled a bit fishy and the reason why you blacked out when you went to this business law conference is because all of these numbers were consistently changed to improve the returns to this factor, which farmer and French were very, very famous for saying outperforms the market.

And we need to probably just rewind very briefly here to talk about the whole, you know, the big concept, which late money listeners will be familiar with, of like,

passive investment and invest in the index and invest in the market and efficient markets and you can't outperform the market and this kind of thing. And Pharma and French are very much in that mold. They believe in the efficient markets hypothesis. They are that mold. Yeah, Pharma invented it. They more or less invented it. And yet, that's

There are these two factors, which you mentioned, value and size. And they're basically saying, yeah, markets are efficient, but they're not perfectly efficient. And there are these two things you can do, which is go overweight value and overweight small caps. And over the long term, and even over the medium term, those two factors will outperform the market. And you can make yourself an extra little bit of cash by investing in those little subsets of the market rather than the market as a whole.

Yeah, there are like settings to the efficiency of the market. And you can believe in a perfectly efficient and like medium sized efficiency. Like there are like different, you get to kind of pick where you sit on that spectrum. But also I think that they think that it fits within that, that you're getting appropriately, if not perfectly rewarded for the risk that you're taking, that you're actually over allocating on a risk because smaller companies are more volatile or because we don't really understand why the value factor exists, but it does. But there is some like, it can be squared with the, it should be noted, unprovable efficiency.

But this is the value factor, which is the big one, right? Because I feel like my entire career, I started as a financial journalist writing about bonds. And bonds are easy to understand. And I never understood stocks. And I would just be like, stocks, la, la, la, la, la. I can't hear you. Go away. Hard same. And then eventually, if you spend enough time as a financial journalist, someone forces you to write about the stock market. So true. Yeah.

And one of the intuitions about the stock market is that probably the way to make money is buy low, sell high. Like that's often a good idea. You should do that if you can. Yeah, you should do that if you can. And so like, and so if you want to make money, like look for things that are cheap and then buy those. And then when it comes time to sell, buy the ones which have gone up a lot. Something like that.

Yeah, and that's the intuition behind the idea that value stocks are good. And this is also completely turbocharged. I don't think people would have this same intuition if it wasn't for the existence of Warren Buffett. Like Warren Buffett is the sort of elephant in the room here. And he is...

He became the richest man in the world by basically doing this and identifying undervalued stocks and holding them for the long term and making incredible returns by doing so. And so everyone's like, well, if Warren Buffett can do it, and if Farmer and Friendship prove that this outperforms, then I can do it. Then let me at it. Which brings the second number which jumped out at me from your article. Well, the first number that jumped out at me and the other thing being the chart. Yeah.

$677 billion. I knew it. I knew it. I knew you were going to say that. So what is that? So that is assets under management at Dimensional Fund Advisors. Dimensional is founded by David Booth of the Booth School. You might know that name and the Chicago Business School. I should also say that Booth is a former student of Eugene Fama's and that Ken and Jean are consultants and stakeholders in Dimensional. And Dimensional invests around...

basically factors. And dimensional is, it has always presented itself as a passive investment shot, but in stark contrast to the rest of the passive investment world, which is just like, we will sell you a S&P 500 ETF and charge you four basis points for it. They charge an arm and a leg in terms of fees. Now, I mean,

Yeah, well, I mean, certainly by current standards, but even when they launched, their fees were high and they can charge what, like 1%, something like that for the privilege of investing your money. And when you're charging 1% of $677 billion, that's a lot of money, which is one of the reasons why David Booth can afford to have the University of Chicago Business School named after himself. And is one of the reasons why...

you know, friendly Ken and Jean are also extraordinarily wealthy. Yes, it's worked out well for them. As Jean always jokes, a lot of people have used my ideas to make money. And this is the only person who offered to pay me for it. That's a joke that I've seen a couple times when people ask him about this. And dimensional, I should say, like always is presented and presents itself as like extremely academic. We just go where the research goes. And like, we work really hard for this like kind of

truly almost like university level, academic level integrity towards the numbers and where the numbers tell us to go. And that's something that they have videos on their website where Ken is explaining exactly that. And I think they've been this kind of

enormous player in this space for a while and only started getting, I think, a couple, like there have been more competitors popping up in recent years. And values had a bit of a hard time in recent years, too, which has kind of created a complicated backdrop for this particular research article. Doesn't your heart just bleed for the $677 billion fund management behemoth?

Yeah. So dimensional does a couple of things. And one of the things it does is it basically goes up to baby boomers and says, you are going to fuck up. If you try to invest in a passive investment and just put it all in ETFs, you're just, you just don't have the testicular fortitude to do that. And so we are going to sit there and sit on your money and force you to do this value investment thing. And whenever you tell us to buy this or sell that, we will ignore you. And then you will thank us in the long term and, and for taking your call and,

talking you off the ledge we will charge you lots of money and for employing these great academics whose minds we we mine for you know their next big insight but yeah and then the other thing which

Like people kind of appreciate that. But on the other hand, they also aren't willing to pay that much for it. The thing they're paying for is also the thing that Dimensional says over and over again, which is effectively we will outperform the market. We will do better than your S&P 500 index fund because we have Farmer and French on our board and because we know these factors and we know how to invest in things that outperform. And so that's what we're going to do and we're going to outperform. And they have a whole bunch of

Their own data about their own funds having outperformed the market. And everyone goes, oh, well, that's great. If I give you my $10 million nest egg, then you'll make it even bigger than if I put it in the market. So I'll do that. And so this whole edifice is predicated on effectively, like if you strip away all of the verbiage, it is predicated on this idea that

especially value stocks outperform. Yeah. Yeah. I mean, there are so many ways to interpret the findings of the noisy factors paper and indeed all of the reporting that Justine and I did over the past year. There's the like uncharitable version that's very suspicious and like,

There's the extremely charitable version that there's nothing at all going on here and that data updates all the time. And why would you even think about this? I'm somewhere in between personally. Like, I don't I don't think that this is nothing. I think that there are technological changes and generational changes and like views towards transparency and like.

whether or not you share your code, like all of that stuff is very relevant here where research practices have changed a ton. And like, just because they're not sharing their code, there's this implication that like, because Fama and French won't share their code and like, didn't respond to comments and didn't like respond to the paper for ages, that there was something definitely nefarious, like it left people to speculate. And I think that like,

I don't know. They're like godfathers of the industry. Like people throw rocks at their work all the time. So to some degree, you're kind of like, I wouldn't respond either. Like this looks annoying. This is like very obnoxious to have to manage. Like, you know, there's people talk about the publication bias that like, oh, you just want to make a splash with your paper. Why would they like dignify that with a response? But wait, that is a real thing. Like, let's be clear about that. Like,

I, too, I'm like you, I'm in the middle. And I, my views of pharma and French and indeed dimensional are not uncharitable. I totally believe that they believe that what they have done is found a way to outperform the market. And you link to a wonderful, wonderful paper by Drew Gelman about the garden of forking parts. And

Drew is one of these godfathers of empirical statistics. If you want to understand anything about empirical statistics, you phone up Andrew Gelman and he will explain everything to you very patiently. And he was the guy who really popularized this whole concept of p-hacking, which is basically where you do an experiment and you get a bunch of data. And then in that data, something interesting pops out. And you're like, I have found something interesting. And unless you...

We're looking for that. The chances are that it's actually nothing because if you get enough data, there's going to be something statistically speaking, something is going to pop out. And so you have to be very careful not to do that. And this name p-hacking kind of entered the lexicon. And he explains in this paper that he kind of regrets ever using that word because it does imply that people are doing something consciously nefarious and

And the thing that the point that he makes over and over again in this paper is that you don't need any conscious bias. You don't need to think you're doing it. You don't need to do it on purpose in order for this outcome to be the case. There is this garden of forking parts. There's a.

million different outcomes that you can have. And if you have a million different researchers all looking at a million different factors, then, you know, and a million different like fund management companies getting founded to take advantage of these million different factors, then statistically speaking, you know, one or two of them are going to take off and become $677 billion behemoths.

And you get that with noise. You don't need any signal there in order to get that outcome. I think that's completely right. And, you know, there's another paper that came out in the recent years that we've been talking about all this. And it's this guy, Matthias Hassler, who was totally independent from the group, the Noisy Factors group, who are kind of the kind of central characters in the story that

that we wrote that look at those arbitrary decisions right at the beginning in the sample period that Fama and French originally considered. And he's like, what if they had made different choices? And indeed those different choices that he like kind of randomly, you know, picks other thresholds, other ways to construct value. And he's like, yeah, you would have also gotten completely different outcomes, which had the same effect of making value, the value factor look less. And it's just like,

Again, he's not ascribing any negative intent. He does not think that Fama and French are p-hacking or trying to inflate the value of whatever, whatever. He's just like, you can do this without trying. You can just think that there's something interesting and be interested in it. And almost by virtue of you being interested, this is me talking a little bit, not him, you imbue it with that power, right? Your interest creates the thing to some degree. And there's this whole cottage industry of like,

I don't want to say fake factors, but some of them are just patently fake. Like the factors that are robust and that show up in the data all the time, there are only so many of those. And even those are kind of fragile. We'll take a quick break and be back in a second.

Hey, I'm Ryan Reynolds. Recently, I asked Mint Mobile's legal team if big wireless companies are allowed to raise prices due to inflation. They said yes. And then when I asked if raising prices technically violates those onerous two-year contracts, they said, what the f*** are you talking about, you insane Hollywood a**hole?

So to recap, we're cutting the price of Mint Unlimited from $30 a month to just $15 a month. Give it a try at mintmobile.com slash switch. $45 upfront for three months plus taxes and fees. Promote it for new customers for a limited time. Unlimited more than 40 gigabytes per month. Slows. Full terms at mintmobile.com.

This episode is brought to you by GlobalX. Since 2008, GlobalX ETFs has been committed to empowering investors with unexplored intelligence solutions. GlobalX specializes in exchange-traded funds that offer exposure to the artificial intelligence ecosystem, including themes like data centers, robotics, semiconductors, and cloud computing. To learn more about GlobalX's entire suite of ETFs, from covered calls, fixed income, emerging markets, and more, visit GlobalXETFs.com.

It's literally the job of empirical finance academics to dive into the data and try and find something interesting in it. If they didn't find anything interesting in it, they would be out of a job. And specifically, when we say interesting, the brass ring, the most interesting thing and the most profitable thing you can possibly find is

is a particular factor in stocks that consistently outperforms. And if you spend your entire career looking for that and you become very successful, like, yeah, like it, there seems to be a decent probability that maybe you just... You're going to find something. Yeah. Yeah. And like...

There are a couple of things here. Like I've met over the course of reporting a lot of asset pricing people that are, you know, academics or that, but they'll also consult or like people that are practitioners that like really believe in this stuff and love the research. And it is really, it's both like, I don't know how they live with the uncertainty. I really don't because I,

I really like getting to the exact perfect word in my stories. I really like understanding something with some degree of completeness. I really like precision, even as I know it's a lie and truth doesn't exist, yada, yada. But living in the world of empirical asset pricing, they just accept that there's been a lot of research over the last 50 years into why some things perform better than others. And I think that's a really good point.

And we kind of just still don't know. It's imperfect. Humans make no sense. We act bananas all the time and we change. And the structural factors of the market change. Meaning not factors in the common French sense, but the other sense. But yeah, I think it's this beautiful...

Like I would find it excruciating to live in this space and be studying these things and devoting my entire mind towards trying to discern where I'm falling on that spectrum of robust versus fragile. But like they, they think it's this, like a lot of them just talking to them, it was like this beautiful quest to understand something more precisely and more fully and to like just inch forward. And I, I really admire that. It's like a very Moby Dick type thing.

feeling to me. Yeah, you have to kind of suspend disbelief to be in that world. I remember recently seeing the results of a poll of mathematicians. And they asked a large group of mathematicians, including a lot of mathematicians on some massive mathematics subreddit. This is a big, statistically significant poll. They're like, are numbers real? Are they actual real things? Yeah.

And 86% of them said yes. And only 14% said no. And like, of course they're not real. Like numbers, numbers, they're adjectives. That's what they are. Like three sheep is just like an adjective describing the sheep. And...

And the idea that there are like these objects that exist out there in the universe somehow, like obviously if you, if you ask a mathematician, like, so number, so real, like point to it. How much does it weigh? What color is it? You know, like they're not real, but like 86% of mathematicians will tell you that numbers are real because you kind of, when you're doing math, when you're in the math, when you're

manipulating equations, you have to believe that the numbers are real because otherwise none of it means anything. Right. Nothing. Well, and that's the problem is that nothing means anything fundamentally. So that I think that's the core of this article. It's a deeply existential article.

Yeah, I got... I mean, that's... I kept having this problem. I would be interviewing people and I would go full galaxy brain and just be like, so we're talking about... And they would be like, I don't know how to help you. Like, you're in space. Like, this is too far for me. But it is... I mean, that's what it is. Like, you have to either be able to...

accept that these things are never going to be perfect. There's a line in this article that's from Andrew Lowe, a really fantastic researcher. And he says, you know, he was talking to me about how we don't have anything as perfect as physical sciences. Like things don't replicate. Like we tried one experiment, we find, you know, this research shows X. And then the next day we tried the research again in the exact same way. And it shows Y, what gives, like nothing works. And it's really frustrating, but that's life. We have nothing so perfect as the physical sciences.

And I said that to my friend who used to be a reporter, a science reporter. And she was like, oh, don't tell him that the physical sciences entirely crumble when you look closely. There's no truth there either. No, this is totally something that Andrew Gelman has written about a lot. And has been written about a lot by many other people. It was like the replication crisis in science broadly. If you can take...

throw a dart at the pile of science papers it doesn't matter what science it could be physics it could be chemistry it could be biology it doesn't need to be you know economics or finance it could be poor behavioral economics bless its heart yeah like behavioral economics gets all the bad rap because but like you know you can get like nuclear physics and throw a paper and try and replicate it and there's a very very very high probability that you will not be able to do so

Yeah, you can really, you can make things evaporate. The idea is that like if you fiddle with the parameters, you can ruin your findings. And if you fiddle with the parameters, you can create findings. Without ascribing malign intent, the fact is that if you see someone who's managed to amass $677 billion, then it seems...

reasonable to assume that they got lucky somewhere along the way. I mean, all success is to some degree lucky. So yes. Yeah. Yeah. Yes. And when you try and like isolate,

what are the elements of luck that went into that $677 billion? Strip out the factors, say. And you just happen to define the value factor in that amazingly lucky way that it not only outperformed when you found it, but it then kept on getting revised in ways that made it look even better and better. And of course...

the better it looks in retrospect, the more money you can attract today. You can go back and say, well, if you'd invested this much on this date, it would be now worth $400,000. And then everyone would be like, great, take my money. And so all of this sort of retroactive fiddling around with what the returns to the value factor were turns into real extra dollars, four dimensional today.

Yes. Yes. And it provides the justification for basically their fund's existence, like that you can look at how great this would have done on a historical basis. And we are the keepers of this, you know, this data set and these great minds and these great strategies, and we know what we're doing. And that I just sort of accidentally said something that I think was a pretty...

Material finding of the noisy factors paper and material to me. Okay. Like it to other people were like, I literally don't care about this. I assumed this 25 years ago, but the, the data set that is published on Kenneth's website, Kenneth French's website is, you know, at, you know, dartmouth.tuck.edu or whatever is actually constructed and maintained by dimensional. So it, it,

It almost doesn't matter. But if you're a researcher downloading this data from a tuck.edu, you know, that's the business school at Dartmouth. That's where Kent French is. If you're downloading what you think is a not-for-profit data set, like you're downloading it from a not-for-profit institution, but it's actually a for-profit provider or like construct, like it's kept actually at a for-profit institution, then

I don't know. That seems weird. And that just wasn't clearly disclosed on the website. That was like in the source code on the website. Although, you know, if you try and look at what the S&P 500 did, that is kept by Standard & Poor's, which is a for-profit company. Like all of this stuff comes from for-profit companies.

Yeah. And universities as a, as not-for-profit institutions anyway. But like, it is like what, that was one of the things where, again, you talk to asset pricers and they're like, you think Kenneth French is sitting there like typing these data updates every month? Like, what are you? No. And I'm like, okay, that is reasonable. But like a lot of people just didn't sit there and think like, who is maintaining this and how, and like, what are the implications of that? And maybe, I mean, it's fair that the implications could be not that much, but it's just, it's just one of those things that like, huh.

And then like going from these like galaxy brain, like, oh my God, what if there is nothing to existence, whatever to like, the wonderful other part of the story is then you just go to the complete opposite end of the spectrum to like sheer academic pettiness and the way that farmer and French eventually respond, but like without actually footnoting or mentioning the paper they're responding to anywhere in their own paper. Yeah.

Yeah, it's only in email that they acknowledge noisy factors. So they write this response, this like full 18-page, you know, SSRN paper, you know, distributed on the website that, you know, everybody distributes their economic papers on. And they spend all 18 pages explaining how they construct...

And they make no mention of why they are saying any of this. And like, it's been 30 years. Like, if you don't know what's been going on, you would be like, whoa, okay, thank you for letting me know how you responded to a FASB 109 changes. Like, I don't understand.

And the thing that, you know, the context is that Fama sent out an email, Eugene Fama sent out an email right before it hit SSRN to the people thanked in the noisy factors paper being like, you are thanked and noisy. And there were some other people that are just like important that he cc'd, but he just clearly was so annoyed that.

by the whole thing. And, you know, I, he says basically that, you know, this data, if you have any experience in empirical asset pricing, you know, that data are bonkers. Like, you know, that this stuff is impossible to pin down, use at your own risk. Like that's the caveat. But he added that it really illustrates the old saying, no good deed goes unpunished. Okay. So Pharma and French, when they finally released this, would you call it a rebuttal? I don't know.

A response, but without acknowledging the question. And they end it with this amazing paragraph. They say, a final warning is in order. The details of factor construction are arguable and there is no magic. I like how this is memorized. No matter how constructed, leave holes in the explanation. Parameter instability and statistical estimation error combine to

to imply that asset pricing models are unreliable. And then they end this by saying the appropriate caveat is use at your own risk. Like they are just in this paper on page 11 of this paper, they are just coming out with like,

a huge amount of like CYA kind of like, I cannot be held responsible for any of this. This is all highly empirical and full of noise and holes and use as your own risk. And don't, whatever you do, don't try and build a $677 billion fucking asset management behemoth out of it because this is all just statistics. We're just playing here. Yeah. And you're like, you can't have it both ways. I know, but

all models are wrong, but some are useful. Like they just live in this world where you, the best thing you have is a spherical cow and like, you just move forward with it. And like, does it give milk or no? Like, I don't know. I don't know. I think like, it's funny you read that as CYA. Cause I read that as just like middle fingers mostly. Oh, you think it was just middle fingers. Like you guys don't even understand how empirical economics works. I got,

that read of like you children, like, like, like,

welcome to the big game. Like, yeah, numbers can change. I don't know what to tell you. Like that, that was the tone that I heard. Cause he's, I mean, he's made no, like, it's not new that Eugene Fama is like, this is a model. It's not reality. Like he says that all the time. So there is to some degree, like in his career, he's, he's not out here being like, these numbers are in stone. God gave them to me. Here are the numbers. Like that's never been his stance, but at the same time, the tone of all of this was like, and I think the noisy factors crew, like,

When they read this note, this 18-page note that was clearly in response to their paper without actually naming them, they were like, first of all, it would have been nice to name us. That's...

common good practice in the world we live in, in academia. But also like you're answering how, but not why. Like you have told us about the methodological changes, that the ways that you measure this, the response that you came up with to accounting rule changes in the whatever decade, like, okay, helpful. Thank you. Thank you for doing that.

but why? And like, to the point of the garden of forking paths, like, okay, you responded to these two FASB changes, but what about all the bajillion other changes in the world that you didn't risk, like that you didn't tweak the methodology for like, why? Like where are, so to them, there's still something missing. And I don't know that they're ever going to get further information out of Ken and Jean, but. Did you, did you ask, did you ask that question to Ken and Jean?

Why are they? I mean, I ran all of this by them, obviously, as is journalistic practice. And Jean was like, I don't know what to tell you. Like, these are numbers. Like, I think there's also like a difference in how people approach code, like not just from a transparency perspective, but like, I think.

Gene's generation doesn't think it's useful to release the code because it's like, you can't read my chicken scratch. But also, yeah, Gene Farmer, I'm just going to come out and say Gene Farmer has never learned R. I feel like it's like,

I understand on some parallel level where like if someone was like, let me see your organizational strategy for how you categorized your interviews in the Bond King, I would be like, oh, you don't want to look at that. Like that is my disgusting, dirty brain. Like I haven't cleaned that room. Like don't look like they're there. If I needed to, I could.

you know, substantiate everything. But like, I don't want you to see how I've been keeping that information because it's just embarrassing. So it's stuff like that where like, I get it. But the call to action, the noisy factors, like the way to cure all of this is just to release the code. And I think that's just not, that's just not something that is going to happen. And plus, as you were saying, like garden of walking paths, it wouldn't,

actually cure all of it. It still doesn't answer the big question of like, why did you incorporate that FASB change, but not this FASB change? Yeah. Which probably, honestly, much more than the code explains that move from 250,000 to 400,000. I'm sure like, if you look at the thousands of different regulatory changes and the, the many, many, you know, more than two FASB changes in the past 30 years, um,

I'm sure it's justifiable. Like I'm sure there are completely reasonable explanations for these two and not the others. Right. But we don't have those explanations. But there are equally plausible explanations for taking those five or this, these seven, or, and there's an overlap here. And that's like, yeah, like you can create a plausible explanation for anything. And, and yeah, we're just in this sort of contingent Borgesian, you know, world where like,

This is the only world we've got. And, and farmers like, this is the only world we've got. And like, there are a thousand other worlds in there where dimensional does not have $677 billion. And the value factor is not a thing. And that's okay.

And then it gets very, as you say, Douglas Adams and improbability theorem. Right, right. So it's like these millennials are demanding that these godfathers of industry engage with them and answer these kind of harsh questions. The implications are the subtext is there. People could read the subtext. And the

The godfathers of industry were like, I owe you nothing. Like I provided this data set. Get off of me. Like, who are you? So it is really, I mean, it's technological. It's generational. It's like goes to the meaning of life. It's perfect.

It's my favorite story ever. It's such a great story. Mary Childs, thanks so much for writing the story. Thanks so much for reporting the story. And more than anything else, thanks so much for coming on Money Talks. This has been absolutely amazing. You're a superstar. This has been so fun. What a delight. Thank you for having me. Thank you very much to Jared Downing and to Shana Roth for producing. And let us know what you thought on SlateMoneyAtSlate.com.

Hi, I'm Melanie, a tech lead at Capital One where we use serverless at scale. Recently, my team used serverless to build an API that lets you generate a pass you can add to your digital wallet for a faster check-in at Capital One airport lounges. And we made it reusable so other Capital One dev teams can also implement digital pass services.

Our serverless first approach allows engineers like me to focus on the fun part, building impactful solutions for our business and our customers. That's technology at Capital One. Learn more at CapitalOne.com slash tech slash serverless. Hey, Slate listener. This is Mary Harris from over at What Next, Slate's daily news podcast. I'm here to remind you that at Slate, we are here to help you make sense of what comes next.

With Slate+, you'll get unlimited access to the tools you need to navigate this moment. That includes exclusive episodes of Amicus, which unpacks the legal battles shaping our future, and Political Gab Fest, where the biggest questions about our political landscape are tackled with depth and a sense of humor. Plus, you'll get ad-free listening across all your favorite Slate podcasts, including my show, What Next?, where I connect the dots on the day's headlines. It's really easy to join, so

Subscribe to Slate Plus directly in the Apple Podcasts app or on Spotify. You can also visit slate.com slash podcast plus to get access wherever you listen. We've been here before. We can make sense of what comes next together.