Taped October 6, 2014
Table of Contents
I: Origins of the Experimental Method 00:15 – 25:59
II: Experiments for Public Policy 25:59 – 46:31
III: Applied Predictive Technologies 46:31 – 58:13
IV: Experimentation and Political Life 58:13 – 1:19:51
I: Origins of the Experimental Method (00:15 – 25:59 )
KRISTOL: Hi, I’m Bill Kristol. Welcome back to CONVERSATIONS. My guest today is Jim Manzi, founder and chairman of Applied Predictive Technologies, and not only a successful businessman, but a very interesting thinker about business and about politics and government. So, welcome, Jim.
MANZI: Glad to be here. Thanks for having me.
KRISTOL: In your excellent book that came out a couple of years ago, Uncontrolled: the Surprising Power of Trial and Error for Business, Politics, and Society. Hope I got that right.
MANZI: You got that right. You’re talking to the guy who named a company Applied Predictive Technologies, so titles aren’t my strong suit.
KRISTOL: Right. Well, Uncontrolled, search for that on Amazon or any other place you want to buy books. The book is full of interesting things, and we’ll talk, of course, about its implications for public policy and government. But in the book, you also kind of – you go back and in a way unveil – maybe isn’t quite the right work – but expose the prehistory, intellectual prehistory of this, of the experimental method. Bacon is really central to your argument.
MANZI: That’s correct, and, you know, I think it didn’t probably do a lot for sales and the popularity of the book, but I actually thought it was very important to really go all the way back to the nature of our understanding of causality to explain why experiments are not this kind of nice, interesting thing to have when you come to public policy but central to developing reliable predictive rules. And I do think that Bacon is –
KRISTOL: Francis Bacon, I should have said, early 17th century.
MANZI: That’s right. So an English statesman and philosopher simultaneously who wrote a book in, I think, 1620 called Novum Organum or New Method. And while you can always play the game of “he was influenced by him and he was influenced by him”, this really is a watershed moment in my view in understanding and inventing the scientific method. And there are an incredible array of insights in that book but one of the most fundamental is his argument that we have a tendency to jump to conclusions, that we see patterns in nature and believe we’ve found what we would call today causal rules, but we’re kidding ourselves.
And it’s only possible, I think, to read Bacon and understand him as reacting to the ultimately Aristotelian tradition that had been developed under Aquinas and the Scholastics and so on, which attempted to, in overly simplified terms, deductively reason. And what I’ll get to at the end of this is it’s striking if you go back and read Aquinas and Aristotle. First of all, they’re all geniuses. And their arguments and the method of debate to me is strikingly similar to macroeconomics today in so many ways. And what Bacon basically says is look you need to run – he didn’t use this term but – you need to run experiments, you need to test these beliefs. And this is central to understanding, to developing correct understanding.
And there’s this lineage that I go through that proceeds from Bacon in rough form in which Bacon says, “Well, we can carefully and inductively build up knowledge from the lowest level – this chemical when put, this chemical in a test tube turns red – ultimately to these high-level predictive rules that we want for the ultimate purpose of being able to control nature.” It’s striking that –
KRISTOL: The conquest of nature and the relief of man’s estate.
MANZI: Exactly right. And he says this several different places, and one of the translations was written in the Latin. In the book, he says it’s literally to increase the power and riches of man. It was very direct. And so I think that a deep issue is for Baconian science, the ultimate purpose of science is not actually to discover truth, it’s better engineering. And he’s also very clear about scientists will not be motivated if they thought that. And so sort of the noble lie of scientists in his view is we are discovering truth. You could argue it’s a leap of faith –
KRISTOL: He doesn’t fully give up on that Aristotelian notion of understanding the natures of things –
MANZI: Well, that’s exactly correct. In fact, he’s arguing –
KRISTOL: But his own thinking undercuts that no? I mean –
MANZI: Well, in fact, what he says is – I believe very clearly – his argument is not that it’s wrong, that the Aristotelian tradition is wrong, it’s just that it’s impractical. What he’s saying is the idea under Aristotle of understanding the essences of objects – what he, Bacon, calls “the mode of principal of being” – where from, instead needs to be, in his words, the mode of – in translation – where to, which is “I don’t really care about this idea of ultimate cause and essence; what I care about is when I mix chemical A and chemical B, what’s going to happen.” And that to me is at the deepest level of the unlock, which is stop worrying about truth with a capital T and just figure out engineering rules that work better.
He’s also clear that if all you’re doing are these – he calls them “experiments with fruit,” these very, very micro experiments – and you’re not attempting to develop more general laws and develop experiments which test these more general laws, you’re not actually going to make fast progress.
But the purpose of those general laws remains better engineering. So in kind of looking at just the mountaintops of very deep, you know, metaphors, sort of thoughts, you know, that is the crucial transition that creates modern science or earlier proto-scientific traditions. And I think over time over the next several hundred years, you have the development of a philosophy of science which, again in super high-level thumbnail terms, is you have Bacon saying, “Look we can use induction and create progress,” and it almost immediately – he was reacting to things that were already happening, he was describing things that were already happening and predicting where it would go. But almost immediately you have the incredible peak of Newtonian physics, which grants incredible prestige to this idea and practical success.
And you have Hume, a British philosopher who comes along and says, “Ah, but you know there’s this problem with induction, which is if I observe that when I take some action or some set of circumstances arise, the following outcome always occurs. I don’t, I cannot know that that will hold in the future.” And in a very famous paragraph, he basically says, “There’s stuff that looks like bread, and it’s brown, and it’s shaped like a loaf and every time I eat it, I seem to be nourished but that doesn’t mean that tomorrow I absolutely know it’s true that in fact if I eat this bready loaf thing I’m going to actually be nourished.” And this is the fundamental problem of induction.
KRISTOL: Though Hume does not draw as a consequence from that, I don’t think, that you should go back in any way to an older Aristotelian science.
MANZI: No, in fact, he makes fun of himself, actually in this and before someone else can do it for him. And the way he puts it is, “I understand that you should not stop eating bread.” He doesn’t say those words but he says words to that effect. But isn’t it interesting that we can’t really know this is true?
And I think that when we start to get to social policy and social science, we move from a world of “Yeah, if I let go of this cup right now, it’s almost certainly going to fall. I don’t know that absolutely. It’s possible it might not,” to a world where that problem in a complicated way becomes central to evaluating social policies and making predictions about social policies.
KRISTOL: But it does seem like Hume almost radicalizes Bacon, that along the same lines, that in a sense, the experimental method becomes even more important, ultimately. No? I mean –
MANZI: I think that’s correct. And in fact, what you then see is again in this thumbnail sort of nickel tour of the history of the philosophy of science, what you then see is Popper come along. Karl Popper, early 20th century, he’s the founder almost of the modern philosophy of science. And what he says is, “You know, I can never prove that a statement like ‘If I let go, a dropped cups fall,’ I can never prove for sure it’s correct. The only thing an experiment can ever do is falsify it because if I ever let go of the cup and it doesn’t fall, I now know that rule isn’t true.”
And he develops this complicated – and once you hear it, completely intuitive although I never thought of it till I read it in his book – this very powerful idea that what I try to do is develop more and more difficult-to-believe rules that somehow can never, no one can figure out an experiment to falsify them but we have always got to keep open the possibility in our minds that someone in the future might figure out some experiment that could falsify them.
And then sort of the termination of our little tour here. You get to Thomas Kuhn who in the mid-20th century says, “You know, that’s not really the way science goes day-to-day.” What really happens is people in a particular scientific discipline accept a certain set of rules as being for all practical purposes true, they accept rules for what apparatus is used to test theories, they develop all kinds of what we would think of almost as craft knowledge.
The way I’ve often explained this to people is think about a group of carpenters, which is what my grandfather was, right. They all have a set of rules of thumb for how they build houses and how they judge what’s a good house and a bad house and what tools you’re supposed to use, etc. And it works building houses and Kuhn calls this “worker-bee science” or “everyday science,” this is what scientists do every day.
And this all sort of sounds like this is directly contrary to the way science really works. You’re supposed to just hold everything open as a possibility and test theories. But a good example of this was when I was an undergraduate, I did a research project and we were looking at photographic plates taken from the Palomar Observatory in 1900 and 1980, and there are these white dots representing stars as far as we could tell on these plates. And if you look at the two white dots in 1900 and in 1980, they had separated a degree on these plates, right. And given strongly held beliefs about how far away these were, that degree of angular separation over 80 years implied they were moving faster than the speed of light. And the paper written about this was called “Apparent Super-Luminal Motion in Compact Radio Sources.” And I’m simplifying all this a little bit. But essentially that worked.
And to me the interesting thing is the word apparent, right, because it can’t be right, we are rejecting data, right, we’re rejecting the observation because we would have to reject, especially, relativity theory. And, you know, if you think about it, that you could say we’re just being close-minded, like you’ve got evidence, you know, Einstein was wrong. But the problem is what are you going to do if you reject Einstein’s theory? Do you go back to Newtonian mechanics, which had many, many problems or what in Kuhn’s terms, anomalies of its own? No. Or are you going to develop another theory, which is better than Einstein’s theory – and we thought we were smart guys but we didn’t think that. And so you start to say, “Well, are we right about the distance of these objects, look at the chemical process of the plates, was there some problem?” And eventually you figure out a way to align, understand the observation in a way consistent with relativity theory, which a guy at Berkeley published a paper, it was true. And that process of resolving anomalies to a paradigm is how he describes most science happening day by day, which is true. And that every once in a while, there’s a scientific revolution where you’re forced to a new paradigm.
And so I actually think as we talk about how to use what in medicine is called evidence-based medicine or in policy is now called evidence-based policy, it’s important to have an understanding of that background in order to interpret a lot of debates. And I think at the same time that was happening, you know, we’re talking about hundreds of years now, what was happening is the experimental method on engineering basis – how do you really do experiments? – was being applied in a broader range of contexts and in more, ever more complicated areas.
Biology is phenomenologically just more complicated than physics in a lot of ways and so applying the experimental method in biology required the invention of whole new methods and approaches.
KRISTOL: And was it biology that some of the key breakthroughs were made over the last – I mean, in terms of experimental method do you think and legitimatized it sort of?
MANZI: That’s right. So, you know, the classic storybook example of a physics experiment is Aristotle has this theory that heavy objects ought to fall faster than light ones. And by legend, Galileo goes up on the Tower of Pisa and he takes a heavy cannon ball and a light cannon ball and lets go of them and they hit the ground at the same time. And I remember learning about this in high school, I think it was the coolest thing I ever heard. And then –
KRISTOL: I remember learning about it and thinking that’s probably not correct. It seems like the heavy one should go down faster.
MANZI: Well, it’s interesting you say that. I remember that night deciding this story couldn’t be true and the reason it couldn’t, I thought it couldn’t be true was Aristotle wrote, what he wrote in – I don’t know, like 350 BC and this is like 1600, that’s 2,000 years – like, it just seemed impossible to me that over 2,000 years, all these people all over the world, no one had ever thought like, “Hey, Giacomo, let’s go take a big rock and a little rock and drop them and see what happens.”
And it turns out that what Galileo did was much more interesting than that and a lot more complicated. And the problem was, if you think about it, if the distance, if the difference in the rate at which they’re going to fall is not gigantic, you have to measure it pretty precisely. And, like, how do I know I let go of them at exactly the same time and the ground has to be level, etc.? And so the most accurate clock he had was a water clock, which wasn’t that fast, it wasn’t that good, wasn’t that accurate. And so he had to slow the balls down and when you go back to his notebooks, you can see what he did, which is he rolled balls down an inclined plane. And by changing the angle, he could slow them down and therefore be able to measure these distances with his water clock. And in fact, he couldn’t just roll them down a plane, he had a cart and he had workman carve grooves into it in the shape of the ball and then he took parchment paper and he saturated it in oil and lined the grooves with it. And the point of all of that was to minimize a confounding factor, which is friction.
And what you see over and over again is the essence of an experiment is to – a controlled experiment – is to hold all possible causes of some outcome constant, change only one and then see if the outcome changes. And in physics, again in this relatively simplified version, you physically hold all other causes constant.
And so when people started trying to apply this in therapeutic biology, you know in medicine theories, you can go back, and research of the book, you can find examples of this in China, in the Middle East a thousand years ago; the Biblical book of Daniel has a primitive version of it. Conventionally in the West, we give credit to James Lind who was a surgeon on the HMS Salisbury who did an experiment in which he determined that the best treatment for scurvy, which was a terrible problem for British sailors at the time, was citrus fruit. And this is, hence the origin of the term limey for British sailors.
And when you read, you go back and you read his experiment, it’s very funny because there were a total of 12 guys and he put them, he tried 6 different treatments, so you have like 2 people, right, n equals 2 in each treatment. And what he did was basically what Galileo tried to do, which is hold all other factors constant. He kept them in the same part of the ship, he gave them each the same diet, other than whatever thing he was varying. And then there was the fairly awful sounding treatments, actually for most of them, other than citrus fruit. It was turpentine and all kinds of stuff. And what –
KRISTOL: Was there even a particular theory of why citrus fruit might work, or was it just kind of –
MANZI: Not that I know of. I’ve wondered the same thing.
KRISTOL: Around and maybe there was some local, I wonder if there was some local tradition somewhere in the West Indies that, you know, it was good for you. I mean, you do wonder.
MANZI: I’ve wondered the same thing, and I don’t know. It may be that that’s in the historical record, and I never found it. But I wondered about that. You know they –
KRISTOL: But anyway it didn’t depend on that, he just had that at hand and it was –
MANZI: That’s correct. And you know it is – it’s a mistake that people often make to assume people were dumb in 1747. Of course, they were very smart, and I’m sure he thought a lot about this but I didn’t find it anywhere in the writing.
And, you know, the attempt then, that’s 1747, the attempt to apply this method accelerates in Europe in the 19th century and they made some real advances. Like you can see places where you see discovery and proof of efficacy of treatments this way. The problem is because biology is a more complicated environment, it would often become a problem of saying, “Well, here are the people who got the treatment, here are the people who didn’t or they get an alternative treatment, are they really the same?” They did it in hospitals trying to control the environment as much as they could. But you get in these debates about were the people who got this treatment actually somewhat different, in some hidden way that’s not obvious to us, in this other folks?
And around 1900, several researchers independently hit upon the idea of, well, we randomly assign people to the treatment group versus the test group versus the control group or treatment A versus treatment B, then as long as the samples are big enough, subject only to sampling error, we know they must be the same. So if you think about this, say, I’m testing some drug in 1910. And it turns out that it’s a hypertension drug, hypothetically. And let’s say that there’s a genetic predisposition that 10 percent of the population has to hypertension, I would never have thought to hold that constant between the groups because I didn’t know about DNA yet. Right, it wouldn’t even occur to me that such a thing could be true. If I’ve randomly assigned a thousand people to get the treatment and a thousand people to be the control group, then I’m going to have about 100 people in each group that have this – have this genetic variance, right. So that will be true for everything. I will hold everything constant. It’s a fundamental change in how you do these experiments.
By the 1930s, there was a pertussis, which is whooping cough, vaccine trial in the United States where you really achieve modern levels of what we mean by randomized clinical trial; it’s double-blind, you have randomization, you have a whole lot of other features. And that process has revolutionized modern medicine. So there have been several hundred thousand trials that have been executed to test therapeutics and it’s fundamental to approving any drug or some therapeutic and has created enormous change. But the key, you think about Bacon, his key unlock is controlled experiments. Then the key unlock to move this to biology is randomized assignments in order to hold the features constant.
KRISTOL: Now people have argued to me – and I don’t know much about this – that maybe we’ve gone too far in this kind of controlled direction, though, in the sense that the FDA is often criticized, for example, for not approving drugs until they really can prove efficacy and safety, I think those are the name of the legal standards. And that in fact, if you insist on sort of high-level proof, you end up doing, spending a fortune on these controlled experiments and they’re never quite controlled enough and, you know, and then someone challenges one of them and so there’s another delay. And in a way, if you were more experimental and less scientific, you’d actually have faster progress and faster outcomes. That in a way, the bureaucracy can turn what you want, I think, which is controlled experiments into a kind of barrier to progress. Is that, is there some truth to that?
MANZI: Well, I think it’s – there’s, I think likely as non-expert, there’s actually some truth to it, although it gets complicated because you use the term experimental, you want to be experimental, and what is an experiment, what does it mean to have control? And generally whenever I’ve looked at the way the clinical trials experts have thought about a lot of these issues – not dumb, right, you know, very, very smart in how they thought about them. I think that you could argue that first of all, there may be kinds of diseases for which the treatment that the population has the disease is small enough that you can’t justify a full-scale clinical trial, and should there be some other procedure there? Because to your point, absence of proof is not proof of absence, right.
Second, I think that you get to a big issue, which we’ll talk about in applying this to social policy of what is called generalization from an experiment, right. So, when I dropped this – when Galileo hypothetically dropped these balls and found that they fell at different rates, he didn’t have to test that in London and in the Andes Mountains and he didn’t have to test on Tuesdays and Thursdays. You, physicists assume that laws of nature – and it’s a funny thing to think about because we assume without thinking about it, laws of nature are uniform across time and space, right.
When you run a pertussis vaccine trial, it’s a reasonable engineering assumption to say that “Well, you know, if I inject this into the bloodstream of people in Norfolk, Virginia, and it prevents pertussis, that’s probably going to work in urban Houston and rural Saskatchewan and suburban Ohio.” But we’re getting to the point now with our knowledge of the human genome and the kinds of diseases we’re trying to treat that this assumption called “uniform biological response” is for many things tolerable engineering approximation but now if you accept the concept of personalized medicine, there are occasions where, in fact, we need to be able to think about differential responses. And I think that’s something that FDA is wrestling with and likely to be a place where there’s opportunity to do the kind of thing you described.
KRISTOL: Yeah, I guess the case I’m thinking about, I have very limited knowledge of it, but I think I’ve got this right at a very high level is the AIDS epidemic. And the FDA is going about its usual business of not approving drugs until they can prove that they work and also don’t hurt people. And that requires a lot of people, a lot of subjects, it requires not giving the drugs to half the people who were meanwhile dying of this disease for which there’s no cure. And I don’t know much of a palliative or delay.
And the AIDS activists don’t like this, quite understandably, and others don’t like them either, don’t like it either, think it’s kind of crazy to be insisting on scientific method when people are dying, why don’t they just try everything, you know, it can’t be worse? And in fact, I believe this is true, that that lobby, if that’s the right word for it, but that the sense got strong enough in the body politic that they basically just overrode the normal FDA protocols for dealing with AIDS. And I think if you step back and say over the last 30 years what disease that looked untreatable has had the quickest success in being treated now really is, I mean, HIV infection is manageable apparently, it’s probably that one.
So that’s a bit of a – and that was just because in a funny way, the FDA just threw the gates open and said, “Okay, you know, do what you want.” And then it quickly became clear what worked and what didn’t. Maybe I’ve got this history wrong a little. I’m sure I have it wrong some but so that would be kind of the argument that goes against, I don’t know, too much scientific method or too much –
MANZI: Right. So I think that, you know, there is a tension. I talk about this in the book. Between the idea of expert scientific knowledge and political freedom because at a certain point, you say well, if we know that you shouldn’t do this, it’s a simple jump, although not necessarily a justified one, to say we should forbid you from doing it. So I think it’s important to segregate two issues in the process you’re describing.
One is what evidence should a rational observer demand as proof of causality in this case for a treatment creating some improvement in disease state. And the second is should you be legally forbidden from using a thing, which is not shown to be effective or in fact potentially shown to be negative? So I think that in general, there is very good evidence that we should be extremely reluctant to forbid people from doing things, even if we believe there’s scientific evidence that says it’s bad for them.
KRISTOL: I suppose especially in cases where the alternative –
MANZI: It’s death, right.
KRISTOL: It’s one thing if someone is hurting himself, someone is healthy and then some quack comes along and says you should take, I don’t know, you know, 19 aspirins a day or something and obviously it makes sense for the government to say, well, wait, no, that’s not good for you. It’s another thing if you’re dying of AIDS and you know.
MANZI: I agree. That’s certainly a factor that matters a lot in anyone’s intuitive reaction to the second issue. I think in terms of the first issue, it’s extremely dangerous to believe you found causality in most medical treatments. So I think that you can find evidence of, in ancient Egypt, of something which approximates modern jaw surgery. And surgery made much more rapid progress than therapeutics like pharmaceuticals and stuff for a long time, and it’s because, look, we have all these complicated assignments about random assignment, but basically you can just see before and after that the causal arrow was so clear, that you, it’s more like physics, designed like physics, its like a daily observation that this is going to drop when I let go of it. You can see that if I didn’t do this jaw surgery, you were going to die, and now your jaw works.
So it is the case that there are therapeutic treatments that have evidence, whose effects are so incredibly obvious they can withstand all kinds of imprecision in how we test them. In fact, normal day-to-day observation of cause and effect works. Unfortunately, there aren’t many of those available to us. And so in general, it’s a very good idea for almost any therapeutic to actually go through the painful process that we go through. We test things like we test a drug. Although when we think about what we’ve done, what businesses have figured out is how can I drive the cost per test and time per test much, much, much lower. That’s not necessarily going to work for testing drugs but in a lot of contexts, we can use technology and other things to make tests so much cheaper and so much faster that it’s worth doing them.
II: Experiments for Public Policy (25:59 – 46:31)
KRISTOL: And business has an incentive to do that, which, I guess, gets us to the transition to government and about policy, which your book addresses in good measure and made the argument, I’ll simplify some, that these methods have worked in science and medicine and business could be applied in a much more – more than they are to public policy and social policy, for which government is in business, so there are some problems with that. So, explain the argument and then can argue about it or not argue but discuss the limitations, which you know well also. And so the ability perhaps to apply this to social policy.
MANZI: Yeah in fact, a huge fraction of the book is describing what the limitations are. So I guess I’d start by describing the thesis of that book with respect to the subject as the following, I mean several parts.
First, that non-experimental social science is not able to create many useful, reliable, and non-trivial predictive rules for the effect of our social interventions, various social programs. Second is we can get better at doing that by executing experiments. The third part of it is we should go do that, but we shouldn’t kid ourselves that we’re going to have anything like the experiment or revolution we saw in physics or even biology, that these improvements are going to be useful and should be done but we are not suddenly going to be enabled to scientifically manage political issues.
And therefore, in general, improvement through trial-and-error learning, which is informal of the kind we were just talking about, rather than structured experiments should actually be more central. This is the fundamental libertarian idea.
And, last, the boundaries to that, you know, why don’t we just, why doesn’t the conclusion of this “just let it rip and let people do whatever they want” – are settled by the need to have coherence and strategy for some unit of society like the United States. So, it actually has several, you know, parts.
KRISTOL: That’s great, yeah. So, the first thesis is, I think, would be familiar to people who have read Hayek or who have read decades or years from The Public Interest or other journals like that, the kind of fundamental insight that the unintended consequences are often more consequential than the intended consequences and the real limits of our ability to plan and the limits of expertise, right. That you basically –
MANZI: That’s right. So I mean the – fully, yeah. So you, know, I think that the reason I wrote the – the motivation for writing the book was, for anyone who’s ever done a startup, when you do it, you go down this dark, dark tunnel where you don’t really observe much about what’s going on in the world. And I did that.
And I got to a point where I had sold part of the company and was kind of coming back up for air a little bit. And the United States was in the midst of the worst economic crisis since the Great Depression. And I remember looking, first time I’m watching TV in years and seeing these baritone-voiced, very intelligent, bearded economist confidently saying, “Well, if we execute the following program, this is going to solve the problem in this way or these will be the effects of this amount of stimulus spending, etc.” And I just looked at the TV and thought you can’t possibly know that. I mean, like I just spent 10 years figuring out how many Snickers bars ought to go on a shelf at the local convenience store, predicting what effect that was going to have was really hard and I don’t believe you really know this. And that was kind of what unwound, you know, to the book.
And I think that if you take the example which everyone is presumably familiar with of what was going on in 2000, late 2008 and early 2009, the huge debate was should we spend a lot of money on – should we execute a stimulus program, which has been a topic of intense academic work for many, many decades. And you have in January of 2009 a number of people with Nobel Prizes in economics saying we should do this, in fact, we need to spend a lot more. And you had another group of people with a Nobel Prize in economics saying this is a terrible idea. And this would be like the night before the Apollo moon launch you have about half the guys with Nobel prizes in physics saying like we can get to the moon but we need more fuel, and about half saying, you know we’re never going to make it to the moon. And then, with apologies to my Austrian friends, a group of guys saying like there’s no moon.
You know, it was just there were frontier debates in all areas of science but there – it seems to me that there’s not the interior of sufficiently, for the moment, agreed upon principles that allow us to develop engineering knowledge which is reliable and not obvious. And so yeah, I mean I have, I wouldn’t exactly describe it as radicalized but I have a very strong point of view about this subject. And I think I’m actually not a voice in the wilderness on this. I think there are within social science communities now there is an experimental revolution that’s happening, there is enormous growth in share of social science using experiments to try and adjudicate debates.
KRISTOL: I think among the – in the conservative social policy world, I think there would be, I think your breakthrough maybe was not so much confirming the skepticism about all these experts and so forth, which was pretty widespread and maybe not politically efficacious but then taking the next step or two that you then took, as you say, which was to say that doesn’t mean know nothing, that doesn’t mean we need to do nothing.
MANZI: Right. I think that’s right, and I think that experiments are a powerful tool that ought to be used to help ameliorate this. And I think that the way – the metaphor I have for this is the scientific or experimental revolution is like this huge wave and it’s moving up a hill of complexity and as this wave comes in, you know, physics is at the lowest level and it completely, totally transforms what physics is.
And then as you move into biology, to the point you made, it’s not exactly like physics, we don’t have that kind of understanding. Experiments are still very powerful and really help us but we’re now dealing with statistical analysis of randomized science trials. And then I think as you get further up this hill of complexity and you get to social policy, the problem becomes physicists can reliably assume that physical laws are universal. Biologists can make this tolerable engineering approximation that you know biological response is uniform so I can generalize experiments but I still have to deal with randomization and so on.
I think when you get to social policies, the problem becomes one of how do I generalize results from an experiment and it’s much more complicated. So in the example I gave you, you can run a vaccine trial here and assume it’s going to work in all these environments. If I ran a literacy program and proved in a randomized control trial it improved third grade reading in Norfolk, Virginia, I can’t reliably assume that’s going to work in this district in urban Houston and also in rural Saskatchewan or also in a high-income Ohio suburb. And what that calls for in my view is not just testing things with experiments but many, many, many experiments to find the conditions under which some program might or might not work.
KRISTOL: I suppose that makes sense. It seems to me to make sense for discretionary programs or HeadStart or whatever. You can structure one, two teachers in a classroom with 8 kids here, and one teacher with an education degree there, and this material there and this material here.
But in a nation devoted to equality under the law, the bigger programs, you can’t – well, maybe I’m wrong but it seems to me at first blush, at least, that it would be hard to do it. I mean you can’t very well, we’d like to know whether student loans are effective in helping kids or Pell grants. I’m just making up this example. But Pell grants are helpful in helping poor kids get to college and also whether it just lets colleges boost their tuition and whether they actually graduate and all these things. You can’t very well, I don’t think, say, well, Pell grants are available to young people whose names begin with A through K and there’s no $4,000 for the ones in the L through Z. I mean, that’s not appropriate in a country where, in some way, programs are equally available to everyone who’s similarly situated, which is kind of the heart of the rule of law.
So you can’t have sentences, two years for robbery here and six years for robbery there and we’re going to do a controlled experiment to see what the recidivism rates are. So that’s a pretty big problem, though.
MANZI: Yeah, so, let me in fact leap ahead before I come back to this.
KRISTOL: Maybe I misstated –
MANZI: No. You haven’t misstated at all. I mean, here are some problems with thinking that you’re – to go back to what I was saying earlier – you’re going to now somehow have the capacity to scientifically manage policy. One is there are many questions you can’t test. And in fact, many of the most important questions you can’t conceivably test. Another is like we haven’t repealed human nature so it’s not like the presence of this information, all the problems of interest-group politics go away and somehow we are governed by enlightened Platonic leaders, right. So yes that is absolutely the case.
KRISTOL: Also on that one, if I could just – I mean not repealing human nature – I think it would also apply to the fact that people can adjust their behavior in response to programs or laws in a way that, you know, scurvy doesn’t decide that it’s going to become impervious to citrus fruits, you know, but people can decide to change their behavior once they see a certain set of incentives.
MANZI: Exactly correct, which is it’s a subset of what I’ll call the problem of generalization. So, in business, there’s this great expression, which is as soon as you find a causal relationship – you know, by running a buy-one-get-one sale instead of a 10 percent off sale, I’m actually making more money – the expression is “ice-cream melts.” So as soon as you find the causal relationship, it’s changing on you immediately because competitors are changing, consumers are changing, the world is changing.
KRISTOL: They get to expect buy-one-get-one free, so then they won’t –
MANZI: That becomes a huge problem. They’re on the drug, and then how do you? So there are a myriad of problems I could go through.
Actually, interestingly the specific examples you gave, there are good randomized experiments. One well-known one is called the HOPE trial in which different treatments were randomly applied to people who failed drug tests who were on probation. And there have been not exactly on Pell grants but there have been a lot of experiments where there are waivers given to states to randomly assign some people into some welfare treatments, for example, and not others. So, it’s not as black-and-white as you described but there definitely are huge questions that you can apply this to.
You know, what’s interesting is where you can – or an interesting observation I think that’s relevant for this is when I look at – I attempted to catalog every randomized control trial done for social programs ever in the history of social, for 2008, in the developed world in the topic – on the topics of social welfare policy, education, criminology, political science, and economics. And I didn’t get all of them but I think I got most of them. And one observation is there have only been a few thousand, which is kind of mind-blowing since a business will run, an individual large business might run a few thousand trials a year. There have been hundreds of thousands of medical trials.
But another striking observation is if you ask the question, okay, when we’ve developed some social program or theory and we’ve subjected it to a test in the cases we have, what percentage of the time does it actually demonstrate improvement in the outcome we’re trying to change? So of criminology programs, what percent create a measurable – it’s called statistically significant – reduction in crime rate for the test group versus the control group and so on? The answer is in less than 10 percent of cases did they do that, which is not even getting to was that enough of a reduction to pay for the program, is it a positive cost benefit?
That is not totally dissimilar to the results you see in business where something like 80 or 90 percent of tests or programs don’t work. It’s not that dissimilar to what you see in drug trials. If you say of all the drug candidates that go into phase one trials, how many complete what are called phase three trials, which are the randomized control trials? Not that far off. In other words, Bacon is correct – we’re almost always wrong.
Right, and so we didn’t test randomly chosen social programs, right? Like these had theory behind them, and people believed them, and you have these advocates standing up and saying, “Look, let me show you the guy this really worked for.” And to go back to your AIDS example, like, “You moron, this is helping people who otherwise. . . .” Yeah, well, you know 95 percent of the time, you’re wrong. And like two and a half percent of the time, you randomly should succeed in the trial, right.
So when somebody says this is a non-testable subject, I understand and respect that and I believe that. When they say, therefore, believe my theory, I don’t think that’s the conclusion. I think our Bayesian prior whenever anyone recommends some program, even if it’s very well thought through is what I hear in my head is there’s a 95 percent chance you’re wrong.
KRISTOL: And if it’s easier for the private sector, and I think this would be true for business, would also presumably have voluntary organizations and so forth, and it’s easier for them to experiment to this end, sort of to learn from reactions and government for various reasons of interest groups and it’s complex to change laws and get waivers – but even that’s not so easy and there are equal protection issues.
That would I suppose be generally building a bias towards limited government and towards letting, either letting non-governmental organizations do more rather than less or structuring government programs so that they’re I suppose limited in their intervention and perhaps allow people then in the private sector to do more rather than less. Is that right? I mean –
MANZI: Very strongly. And if you think about it, my view calls for embedding experimental capability and to government units. But recognize this is going to be a marginal benefit. In absolute terms, in an economy of whatever it is, $6 or $18 trillion now and 320 million people, it’s worth doing and it will create real benefit.
But in general, it leads you to, in my view, first, disbelieved claims of expertise on this subject, to cast a very skeptical eye towards, not just in the traditional Republican talking-point sense, you know, people who are professors, but in general, assertions of knowledge about effects of changes to government programs. But also if you think about it, if I’m skeptical about your expert knowledge but I’m also skeptical about my ability to build alternative experimentally derived knowledge, what am I left with?
By process of elimination, I’m left with some kind of trial, unstructured trial and error. One of the things I try and emphasize in the book is this is what you do when you’re out of options, right. I mean, it’s not like the market process or the democratic process or even in certain aspects, science. It’s not like you do this because you want to, it’s you do it because you recognize how ignorant you are. And if you think about the thesis I laid out in the book, this is sort of the second last point, which is in my view, it leads you strongly to a Hayekian somewhat libertarian-ish point of view. And then you get to then what are the limits to that? But, yes, I think it does.
KRISTOL: So, structured trial and error where possible, but unstructured trial and error is still useful, that’s how people –
MANZI: That’s right. It’s the base. If you think of that, exactly. And really structured experiments are really just a thin overlay on that on these topics where we can build true experimentally verified expert knowledge.
KRISTOL: It’s so interesting to me because I was in government a little bit years ago and I’ve watched since and people come in and say we have to apply business principles to government. And the Pentagon has always – they want to apply business principles at the Pentagon. But it strikes me when I watch them try to apply business principles, they’re not doing it in the way you want, it’s the opposite, actually.
So, it is in a way, your earlier consulting, almost like a parody, though a cartoon version of the consulting firm, not a very good consulting firm where business principles means we know how to procure weapons officially or set up chow halls on bases. So we can get the military, can get out of that business, we’re going to get these businessmen in to do it. And then there’s a study 5 or 10 years later, it turns out it’s costing more or it’s having these unanticipated side-effects or contractors cost twice as much as soldiers. So it’s a business principle and you have to pay them double to go into a war zone, obviously.
So this great business principle you had is now, you know, causing, you know, much greater expenditures on running the chow halls in Iraq than if you just had the old-fashioned way of – the old-fashioned, inefficient way of having some private, you know, ladling out the stuff from behind the counter. I don’t know. I mean, just it is striking how I think when people say business principles in government, they don’t mean what you mean.
MANZI: I think that’s right and I think right in two different ways. One is that if you think about one implication of what I described of unstructured trial and error of letting people do things is on the one hand, it needs you to devalue or, in my view, to appropriately – to have an appropriate degree of skepticism about external experts’ ability to do analysis and draw useful, reliable, non-obvious conclusions about effects and advantages.
But what it does lead you to more greatly value is the tacit knowledge of operational experts because the vehicle by which that trial and error is happening is not all of us as atomic individuals, it’s this – you know I’ll make up an example I don’t know anything about, so it’s totally hypothetical. Like, actually, you know the guys in the Marine Corps know a lot about landing on a beach and so maybe you ought to not have a bunch of guys do studies to explain to them how they ought to land on a beach better. Like you ought to allow them to exercise expert knowledge because the relative value of that tacit localized expertise is higher because you’re appropriately being skeptical about this high level of expertise.
The second thing is –
KRISTOL: I suppose your method would then be to look at all the landings on beaches and see if there are some things you can – that aren’t simply random, you know.
MANZI: Well, that’s right. So if you then looked at – so, we’re now going to go sort of down the rat hole a little bit – but if you think about, look at what a rational expert would do is say well, I’m going to look at all the beach landings that these guys did and I’m going to analyze, I’m going to describe, I’m going to get data about all the landings. How deep the water was, was it high tide or low tide, was it day or night? And I’m going to find the pattern.
You know, actually what you guys ought to do is – I’m making this up entirely, right – you ought to open the front door in the water instead of on the beach when it’s nighttime and the water depth is less than four feet. And what I’m saying is that ability to see those patterns and find insights like that that are useful and reliable and not obvious is – we’ve got to be very skeptical about that. And when the guys goes – look, whatever, I don’t want to argue professor, but that’s a really bad idea, you ought to – we ought to put more weight.
KRISTOL: If you’re landing in the Atlantic as opposed to the Mediterranean where as that turns out, that difference will swamp all the other.
MANZI: Exactly, exactly. And so I think that’s one implication of this, right. And I think the other one, reacting to your point about sort of “applying business principles” is it is true then the absence of constant, the greatest – it’s not, I don’t think it’s an accurate definition actually, but it’s a great, it evokes the correct thing, I think – the definition of bureaucracy I’ve ever heard is a large organizational unit that does not believe it’s in competition, right. And so in the absence of the constant very politically incorrect brutal pressure of capital markets, businesses tend to do the same thing exactly, which is they become inwardly focused.
And this guy I know who is one of the greatest consultants I ever knew and a lot of consultants are kind of full of it, whatever, but he’s a great consultant. And there were these waves of methodologies that roll through businesses. And they kind of go by different names – CISCO Process Control and 6 Sigma and you know. Newt Gingrich was talking about one of them and so on. And what happens is they all are basically the same idea, which is smart people look at what’s going on, figure out what people are doing and say, that’s kind of, that doesn’t really add towards the goal, why don’t we change that, etc. And then they become these systematized methods that can be reproduced. And ultimately they become sort of self-parody to your point.
And the expression he had was – I was asking him, he went to do some stuff at this company and we were going to go and do some work there and I said, “So tell me about what’s going on there.” He goes, “Jim, you know, it’s like The Planet of the Apes but the monkeys have taken over for the humans.” And so that’s really what happens in these situations. And so I think that unless you have competitive pressure that is felt by the people in the organization, it’s very difficult to have anything of a parody of business practices.
III: Applied Predictive Technologies (46:31 – 58:13)
KRISTOL: So, tell us about Applied Predictive Technologies. How did you start it and what is its distinctive business – what’s the theory behind the firm?
MANZI: Well, APT is a software company that provides tools that lets – that let typically very large marketing companies know the cause and effect relationship between business programs and financial and other outcomes. What if I offer this new product, what will sales really be, what if I change price, what will really happen, etc?
And I think the distinctive thesis of the company is that the most reliable way to know the answer of that question is to run controlled experiments to test the theory in a small scale way before rolling it out, which is a kind of common-sense idea but it turns out it’s trickier than it might seem to do that reliably in a broad variety of contexts.
KRISTOL: And what would be, give an example or two of such a controlled experiment. How do you do that? I’m a bank, or I’m a firm that produces A, B, or C, how do I, what are you going to show me about this product or service?
MANZI: So a classic example of this is Subway, the sandwich shop, came – some franchisees of Subway came up with the idea that we could sell large sandwiches at a low price, the so-called, now called the $5 foot-long. It was a very controversial idea actually at the time.
And some franchisees tried it out informally, and it appeared contrary to intuition to work pretty well economically. And so what the company was then able to do was structure an experiment in which certain local market areas were given this promotional special and others weren’t; and by carefully comparing what happened to sales and profits in the markets that got it versus those that did not get it, they were able to determine actually it was very profitable and the kinds of conditions under which it worked better or worse. And then we were able to run a series of experiments to further refine it, you know to determine should I give it TV support and if so, how much, does that – is the incremental economic – economics of that attractive or not and so on? So that’s a kind of very classic, basic example.
KRISTOL: And is the point to come up with sort of one solution then, one answer, or in fact is it more, just listening to you, it sounds like it must be more, well, what if some areas of the country like $5 foot-long subs and others for various bizarre cultural and historical reasons don’t – I mean, you could discover that I suppose and tell Subway, tell your franchisees in Dallas to do this and in Wichita not to do this?
MANZI: Yes. And once again, it’s typically, and part of the business theory of the company is we’re providing tools and they’re doing that themselves. We believe –
KRISTOL: So they get to decide, obviously.
MANZI: That’s right and that will become important when we talk about this if we do in terms of potential application in public policy, that embedding this capability inside an operational organization is in my view much more effective than an external entity telling – giving them advice about what to do. And, yes, very typically, you will discover that a given business idea works in some context and not others, which will also become relevant when we think about this in public policy terms.
And there’s a classic debate then, of course, which is often encapsulated by the – you know, Henry Ford saying you can have any car – you can have this car in any color you want as long as it’s black. And General Motors under Alfred Sloan developing the concept of a car for every person and purpose. And the tradeoff is typically you will have greater versus lesser acceptance in some context; sometimes that will be geography, sometimes it might be time of day, sometimes it might be time of year, etc., versus others. But the cost of providing a differentiated offer are obviously typically higher than a uniform offer. And then there’s a business tradeoff decision to get made.
KRISTOL: And is your general sense that businesses, though, or at least was it the sense before you started, that they were being too uniform and not experimental enough? I mean, was the sort of, what was the insight, what was wrong, what was the flaw in the – in business planning that in a way your firm solved or began to solve?
MANZI: Well, I think that it certainly was my view that companies were not as aggressive as they could have been in using experiments to resolve debates about causality, about the true effectiveness of business programs or ideas.
And it wasn’t because they weren’t trying to do that. I mean, the first – I’m sure the first grain merchant in ancient Mesopotamia who had two shops tried something here and not there. I mean, the concept is not new. It turns out to apply it in other than very niche contexts required a bunch of analytical and mathematical advances that became much more plausible in the current era of very cheap computation and very cheap data storage and transmission.
And so really the unlock was applying information technology to resolve analytical problems to make experimentation more practical in a broad array of contexts. It’s one of almost an infinite number of instances of how Moore’s Law and related increases in information technology productivity are driving change in the economy.
KRISTOL: So that would have been hard to do this before the computer and IT? The Internet revolution.
MANZI: And in fact when I started the company in 1999, we were probably still shooting a little bit ahead of the duck. It was still very difficult to do it practically, it really was several – it took us several years to figure out how to do it but also for the underlying technology to get cheap enough.
KRISTOL: And my sense is – you once told me the story of how you actually thought to found the company. I think people would be interested in that. I’d be interested in hearing it again.
And left and went to work for a little group that had spun out of the Boston Consulting Group, which is a strategy consulting firm. And this little spin-off firm was focused on the idea of using data and fairly intensive analytics, which at the time, this is 1987, which was a pretty innovative idea, to do really high-level strategy corporate work. And we very quickly started getting deep into using mathematical models to understand how to answer the kinds of questions I was referencing before, we’re now answering experimentally. Would the following major investment in redesign of the bank branches actually pay out or not, would we create enough incremental customer volume to pay for this or not.
And I remember well standing in a conference room talking to a guy who was a partner when I was a very young consultant and laying out with great pride this kind of very complicated program of sophisticated conjoined analysis and cost analysis and all kinds of other things to answer this question for this large bank: would you actually get enough incremental business at the bank to justify this major rehab of the branches and change in employee behavior and so on? And he kind of politely listened and then at the end, said, “Okay, but why wouldn’t you just try it at a few branches, you know and see if it works or not?” And I kept trying to, you know, respond to him, and I’d get halfway through the sentence in which I was explaining why that was naïve and not nearly as smart as what I was describing and then I’d kind of stop and say, “No, no, that’s not true.” Finally, I kind of gave up and said, “You know you’re right.” And that conversation needled at me for years and years and years.
And as we were doing a lot of this analytical work over a long period of time, you start to realize when you’re actually the person down building the econometric models that deep down in the model, there’s some parameter that you have to set – is it .2 or is it .4, and ultimately you’re going to have a judgment, you have some analysis around that but you obviously have a judgment and if it’s .2, the answer is the program is great. And if it’s .4, the answer is it’s terrible. And it makes you in some ways skeptical and cynical about the ability of that kind of modeling to work correctly.
What we would think of in a public policy context as econometric modeling, macroeconomics and so. And if you confront that issue seriously, I think one of the paths you eventually go is – you take is to try and apply experiments to try and answer that question. And that ultimately was the intellectual genesis of the company. It turns out the guy, the partner who told me you should try things was Rich Fairbanks and he and another guy who also worked there named Nigel Morris started a company called CapitalOne, the huge credit card company, really, literally as a platform for large-scale randomized experimentation. That really was the concept of what CapitalOne was, which of course is now a company worth tens of billions of dollars, so, you know, like he won the argument.
KRISTOL: And so was it generally the case then and I guess is it still the case now that most of business consulting and strategizing, both I suppose, internal to companies and when they hire an external consultant, is it a little more along the lines of what you were trying to do, which is do all this analysis and then we’ll come up with the answer, as opposed to – I guess one could describe this contrast in a lot of different ways – as opposed to an experimental bottom-up study where you don’t really presume you know much and you just – not just – but you let the market, I suppose, tell you what works in different circumstances?
MANZI: Yes, it is disproportionately that way. I think there is an experimental revolution in business that’s happening. I think we’re riding a wave. I think that is changing. I think the – if you take, in some crude terms, the proportion of business decisions weighted by shareholder value that ought to be made in experimental versus non-experimental terms, the information technology revolution has created the possibility to make a higher proportion of those decisions through experimentation and that change is occurring as we and others invent the technologies to enable that to happen.
I think of, you know, the kind of model – I keep calling it model building or analytics, etc. – I think of that as theory building and experimentation as testing theories. And I think it’s like respiration, it’s like the scientific method. You know, you inhale, then you exhale, then you inhale, then you exhale. You build a theory, you run a test, you build a theory, you run a test. So I see them as goods that support one another, rather than being necessarily in direct competition, although it is the case that the opportunity to make more decisions through testing is one that’s through experimentation it is one that’s being exploited.
KRISTOL: Yeah, I guess that one can see sort of how they fit with each other but isn’t there – they also seem to me to represent somehow, you know, contrast that in kinds of scientific method. I’m not sure if that’s the right –
MANZI: Yeah. No, it’s true, I think that’s right. I mean, I think my view is the insight of Francis Bacon, 400 years ago is well, if we take what is really a proto-scientific method and add experiments that test theories, that will create a revolution in human understanding. And I think he was correct, and I think that was one of the most important insights, maybe, in human history.
So, yes, it is definitely the case that there is a kind of mindset of someone who’s an experimentalist versus a theorist and both are necessary and one was underrepresented, I think in all kinds of business and in my view, social science.
And in a lot of ways, personally I have kind of taken the journey from Plato to Aristotle, right. From theory to, from ideal to practice, from theory to experiment, etc., and believe that I undervalued it, which is not to say there is no value for theory but I think I undervalued the importance of experimentation. I think that that has generally been true in business analytics and strategy. I think it’s been true in social science. I think it’s been true in a lot of areas.
IV: Experimentation and Political Life (58:13 – 1:19:51 )
KRISTOL: You mentioned the limitations ultimately of this method. One that occurs to me I remember from studying many years ago Aristotle’s Politics with Harry Mansfield. There’s this character, Hippodamus, who’s sort of the city planner, I guess, either real or made up by Aristotle in ancient times. And he wants to do everything scientifically.
And one of the points Aristotle makes is kind of a Hayekian point in a way about the limits of this kind of rational mathematical planning but also the point that ultimately the rule of law depends on habit, and also virtues depend on habit, good character depends on habit. Which means not in a way subjecting everything to constant experiments.
You don’t really want a society – I always think of something arbitrary like speed limits as an example, like it’s obviously crazy at some scientific experiment point of view to have every speed limit be 55 miles an hour within a 30-mile area on highways when then it goes to 65. But obviously you could, the state of Virginia could hire your company, and you could probably show that the optimal speed limit for both speed and safety is 58 miles in this zone near Fredericksburg and 67 miles in this zone near Roanoke. And why not just have these limits change every mile or two depending on what’s rational? But there is a case for just having 55 mile an hour speed limits in highways and, you know, 35 miles an hour or whatever in residential areas because people remember, get used to it. And there’s a case for green lights being a certain, you know, I guess, having a certain length in Washington, DC, so you sort of get habituated to it as opposed to perfectly calibrated to the traffic. So what’s the answer to the case for sort of habit and simple rules and laws as opposed to constant experimentation?
MANZI: Well, I mean, I think when I laid out the thesis of the book as I see it, this is getting to the last point of it, which is even more aggressively, I’d say there is a need – if I were an alien observing the earth like a Petri dish, I could just say, great, just let trial and error rip and we are going to discover over time some societies will develop more simple and some less simple rules and we’ll see at a complicated level which set of rules and institutions ends up winning.
The problem, of course, is we identify with my family and my country and my company and so on and we care about it, we have commitments. We care about whether they succeed or not. And I think that if you look at any institution which over a long period of time is highly successful, you know the United States, the Berlin Philharmonic Orchestra, IBM, etc., there is a non-rational degree of identification with the enterprise that people have. And constant change undermines that, variability undermines that, a set of rules which don’t comport in some very rough way with my view of the good life undermine that. And so in order for the entity to survive, it has to have some degree of coherence.
So in a very micro way, that probably means not it’s 58 miles an hour here and 56 miles an hour there. In a macro way, it means well, one thought-experiment I go through is suppose I live in San Francisco and Alabama passes a rule that says we are going to put kids in orphanages who don’t have parents and I stipulate that no one from Alabama is ever going to move to where I am and bother me and I’m never going to move there. Okay, suppose I now have corporal punishment for kids in orphanages. Okay, suppose I have the death penalty for kids in orphanages. At a certain point, I just recoil from it because I don’t want to be in political association with people who do that. And you can imagine the reverse thought experiments and so on.
And so I think that there is an incredibly strong case for not expecting most normal human beings to live out at the entrepreneurial frontier where life is constant ferment and change and experimentation or random change, etc. I mean Marx is actually brilliant in writing about this, is capitalism creates this kind of constant change which is disruptive to people, and in fact, I think that the basic underlying tension for a lot of American domestic political economy for 30 or 40 years has been how do we manage the tension between the need to have that kind of change variation, experimentation, structure, unstructured in one hand and the need to have social cohesion on the other.
KRISTOL: Yeah and I think even, it’s maybe not just social cohesion but even from sort of the experimental – I don’t know – profit optimizing, let’s say efficiency optimizing point of view of a social policy maker. Even there, there’s something to be said for just, I mean, stability has a value of its own in maximizing efficiency.
I mean, I just saw the Berlin Philharmonic last week, as it happens, very nice. I hadn’t seen them in decades. And it struck me watching them that – I was thinking about this conversation a little bit that obviously they have very intense competitions to be the first oboist or something there. But I have the sense they don’t, I don’t think, throw it up every year. So, now they do, so in a way but you wouldn’t, it would be kind of crazy if this guy is the best in the world and someone else is the second best in the world by, you know, one-tenth of a percent and then the next year, the second place guy replaces the first guy. I mean the whole place – thing would kind of fall apart. Now, on the other hand, you don’t want this guy to be there for 30 years and being, you know, the 300th best oboist in the world and not having any turnover.
But every institution thinks about this question, of course. But they don’t lead – it seems to me, some conservatives, some entrepreneurial types, some business types talk about the world as if there should be this endless, you know, creative-destruction competition every year, and not just at the level of the Berlin Philharmonic competing with the New York Philharmonic but also at the level of sort of every seat in the Berlin Philharmonic. At some point that becomes counterproductive, I guess.
MANZI: Yeah, I think that’s right. So I mean, if you think about the examples you talked about, I would analogize the should the speed limit be this here and that there to the thing I was talking about earlier in our conversation about you can have any color you want as long as it’s black versus, you know, lots of kinds of cars.
The tradeoff being on one hand, I would maximize revenue by having a unique car per person, I’d also make costs very high. And there is some tradeoff. And I think, and just practical engineering you’re pointing to, there are some of these tradeoffs that apply to government for sure, independent of all the political issues. I think also as relevant to any organization, including a business, this question of do I sit there with every employee and each, you know, six months, put the stack of resumes in front of the guy and I’ve seen people do this. These are people who want your job, you know, why should you keep it?
Depending on the kind of company you’re running and so on, that’s not always the best way to motivate people. In fact, there are companies that, you know, go more – you know, and the world is shades of gray, there are companies that are appropriately more like that and less like that. And I think that is again there is at a higher level is evolutionary competition would mean different kinds of companies say in an industry that compete one way versus another. And you know hypothetically the New York Philharmonic might be much more aggressive about you try out every year and the Berlin Philharmonic not, and over time, one of those models might win out over the other. So I do think –
KRISTOL: Or one of those models might be better at some things than the other.
MANZI: That’s what I’m – exactly.
KRISTOL: So the Berlin Philharmonic – I mean this is all made up, I have no idea if this is true – but the Berlin Philharmonic would do a better job of playing certain pieces that they play every year and the New York Philharmonic would be more adaptable to some new piece of music.
MANZI: That’s right. In fact, I think in general – and I go into this in a very kind of, try to do it in a structured way in the book. I think of business, the economy, and more broadly, these kinds of competitions, as being more like evolutionary competition than the way antitrust theorists think about it. Like, you know, every company is a monopoly in its zone of competence, almost by definition. And so the question is how big is the niche you serve this way versus that way, and can you find a niche like you’re saying.
You know, it’s certain, again now totally making this up, it turns Mozart, you play Mozart better this way and you play Beethoven better that way. So, yeah, I think there are those, there are these kind of efficiency – to misuse the term but I think you get what I’m saying – there’s efficiency maximizing recognition of various kinds of costs implied in trying to optimize everything. So I think that is definitely the case.
I think beyond that, however, there is or at least in a very generalized sense would have fallen under that heading but I think different in kind and practice. The issue is we don’t want to live our lives in an environment where everything I do and everything being done to me is constantly being managed against some output objective like that. You know we need to – we need to identify in an irrational way with institutions. And, actually, institutions that succeed over time know that. And that’s central to what they are.
KRISTOL: Which means that some higher level of rationality, it’s not an irrational way, right, if it ultimately makes you happier.
MANZI: That’s exactly right. And so then there’s – I believe then there’s the question of – and I do talk about this in the book – which is whether it is true that this is mere epiphenomenon of biochemical evolution or whether there really is some transcendent, whether it’s true or not, the subjective feeling needs to be what I described. And you know I think that – again to take an example I don’t know well – the degree to which that kind of loyalty and identification matters is probably much higher for the United States Marine Corps than for the University of Cambridge. But it actually matters for both of them and other things matter more versus less.
But I think this is the thing that doctrinaire, one of the things the doctrinaire of libertarians miss, which is we exist in a social context in this way and the vehicles by which this kind of competition happens which aren’t the individual entirely but groups, succeed by having this kind of identification. I also think that you – it leads you to have a point of view, which is I call the paradox of libertarianism, right, which is you could think of – take the case of like prostitute – legalized prostitution, right, should prostitution be legal? So the canonical libertarian party position is, of course, it should be legal, it’s a voluntary exchange between adults. A point of view which says I am, I have this libertarian-ish belief because I think I’m ignorant and I don’t really know how to adjudicate between the two arguments which say on the one hand, it’s going to break up families and it’s terrible, on the other end which says actually it reduces these other problems and that net-net that’s a good thing. And I certainly don’t know what for everybody for all time, for all of human history and all places.
So, it leads me to want to let local entities try different things and let people and entities be free to do things. But because if you combine this idea of “I identify with some place which has some rules or some entities that has some rules,” it leads you to let entities that have some degree of, of power like towns or states be able to put in place course of laws that in fact violate a theoretical libertarian point of view that says free exchange for an individual should be allowed because you’re not sure whether those rules are good or bad or might be good for some people and not others.
And I think it’s when you think about the limitations to this, I mean, these are the sort of fundamental limitations are driven ultimately from this kind of human psychology and need to identify and competition that actually happens at a level above people and happens in organizational units of various kinds.
KRISTOL: So your libertarianism leads to a little more of an emphasis on federalism, decentralization and so forth and to a sort of dogmatic, nationally – applied libertarianism.
MANZI: That’s right and you know I tried to be studious in the language and be honest about my point of views. But I call these – I call this distinction, you know, liberty as goal and liberty as means. And I think Hayek is and Adam Smith and that whole tradition is the liberty as means tradition and that’s what I believe strongly because I believe I and we are so ignorant.
KRISTOL: On that note, thank you, Jim, for being with us for a very stimulating discussion and thank you for joining us for CONVERSATIONS.
MANZI: Thanks very much.
[END]
