Science Doesn't Trust Your Rationality

Followup toThe Dilemma: Science or Bayes?

Scott Aaronson suggests that Many-Worlds and libertarianism are similar in that they are both cases of bullet-swallowing, rather than bullet-dodging:

Libertarianism and MWI are both are grand philosophical theories that start from premises that almost all educated people accept (quantum mechanics in the one case, Econ 101 in the other), and claim to reach conclusions that most educated people reject, or are at least puzzled by (the existence of parallel universes / the desirability of eliminating fire departments).

Now there's an analogy that would never have occurred to me.

I've previously argued that Science rejects Many-Worlds but Bayes accepts it.  (Here, "Science" is capitalized because we are talking about the idealized form of Science, not just the actual social process of science.)

It furthermore seems to me that there is a deep analogy between (small-'l') libertarianism and Science:

  1. Both are based on a pragmatic distrust of reasonable-sounding arguments.
  2. Both try to build systems that are more trustworthy than the people in them.
  3. Both accept that people are flawed, and try to harness their flaws to power the system.

The core argument for libertarianism is historically motivated distrust of lovely theories of "How much better society would be, if we just made a rule that said XYZ."  If that sort of trick actually worked, then more regulations would correlate to higher economic growth as society moved from local to global optima.  But when some person or interest group gets enough power to start doing everything they think is a good idea, history says that what actually happens is Revolutionary France or Soviet Russia.

The plans that in lovely theory should have made everyone happy ever after, don't have the results predicted by reasonable-sounding arguments.  And power corrupts, and attracts the corrupt.

So you regulate as little as possible, because you can't trust the lovely theories and you can't trust the people who implement them.

You don't shake your finger at people for being selfish.  You try to build an efficient system of production out of selfish participants, by requiring transactions to be voluntary.  So people are forced to play positive-sum games, because that's how they get the other party to sign the contract.  With violence restrained and contracts enforced, individual selfishness can power a globally productive system.

Of course none of this works quite so well in practice as in theory, and I'm not going to go into market failures, commons problems, etc.  The core argument for libertarianism is not that libertarianism would work in a perfect world, but that it degrades gracefully into real life.  Or rather, degrades less awkwardly than any other known economic principle.  (People who see Libertarianism as the perfect solution for perfect people, strike me as kinda missing the point of the "pragmatic distrust" thing.)

Science first came to know itself as a rebellion against trusting the word of Aristotle. If the people of that revolution had merely said, "Let us trust ourselves, not Aristotle!" they would have flashed and faded like the French Revolution.

But the Scientific Revolution lasted because—like the American Revolution—the architects propounded a stranger philosophy:  "Let us trust no one!  Not even ourselves!"

In the beginning came the idea that we can't just toss out Aristotle's armchair reasoning and replace it with different armchair reasoning.  We need to talk to Nature, and actually listen to what It says in reply.  This, itself, was a stroke of genius.

But then came the challenge of implementation. People are stubborn, and may not want to accept the verdict of experiment.  Shall we shake a disapproving finger at them, and say "Naughty"?

No; we assume and accept that each individual scientist may be crazily attached to their personal theories.  Nor do we assume that anyone can be trained out of this tendency—we don't try to choose Eminent Judges who are supposed to be impartial.

Instead, we try to harness the individual scientist's stubborn desire to prove their personal theory, by saying:  "Make a new experimental prediction, and do the experiment.  If you're right, and the experiment is replicated, you win."  So long as scientists believe this is true, they have a motive to do experiments that can falsify their own theories.  Only by accepting the possibility of defeat is it possible to win.  And any great claim will require replication; this gives scientists a motive to be honest, on pain of great embarrassment.

And so the stubbornness of individual scientists is harnessed to produce a steady stream of knowledge at the group level.  The System is somewhat more trustworthy than its parts.

Libertarianism secretly relies on most individuals being prosocial enough to tip at a restaurant they won't ever visit again.  An economy of genuinely selfish human-level agents would implode.  Similarly, Science relies on most scientists not committing sins so egregious that they can't rationalize them away.

To the extent that scientists believe they can promote their theories by playing academic politics—or game the statistical methods to potentially win without a chance of losing—or to the extent that nobody bothers to replicate claims—science degrades in effectiveness.  But it degrades gracefully, as such things go.

The part where the successful predictions belong to the theory and theorists who originally made them, and cannot just be stolen by a theory that comes along later—without a novel experimental prediction—is an important feature of this social process.

The final upshot is that Science is not easily reconciled with probability theory.  If you do a probability-theoretic calculation correctly, you're going to get the rational answer.  Science doesn't trust your rationality, and it doesn't rely on your ability to use probability theory as the arbiter of truth.  It wants you to set up a definitive experiment.

Regarding Science as a mere approximation to some probability-theoretic ideal of rationality... would certainly seem to be rational.  There seems to be an extremely reasonable-sounding argument that Bayes's Theorem is the hidden structure that explains why Science works.  But to subordinate Science to the grand schema of Bayesianism, and let Bayesianism come in and override Science's verdict when that seems appropriate, is not a trivial step!

Science is built around the assumption that you're too stupid and self-deceiving to just use Solomonoff induction.  After all, if it was that simple, we wouldn't need a social process of science... right?

So, are you going to believe in faster-than-light quantum "collapse" fairies after all?  Or do you think you're smarter than that?

 

Part of The Quantum Physics Sequence

Next post: "When Science Can't Help"

Previous post: "The Dilemma: Science or Bayes?"

Comments

sorted by
magical algorithm
Highlighting new comments since Today at 2:26 PM
Select new highlight date
All comments loaded

Science first came to know itself as a rebellion against trusting the word of Aristotle.

Digressing, this is PC history.

Science first came to be as Roger Bacon writing up the scientific method. His approach was to not trust anyone, but to trust Aristotle more than most. Unsurprisingly, he was put in solitary confinement on bread and water. The Church then issued a list of forbidden thoughts, with Aristotle prominently on the list. That science started with a revolt against Aristotle is a whitewash of the conflict between the theocratic state and Science. Science, science in the sense of the scientific method, not science in the sense of a state anointed priesthood ceremonially wearing labcoats as white robes, is inherently revolutionary, a defiance of authority, but it was not the authority of Aristotle that they were revolting against. Rather, all beliefs were subject to empirical scrutiny, including the beliefs of the authorities of Roger Bacon's day, which was revolt against present authority, not Aristotle.

We do not know what the charges were against Roger Bacon (most likely the nominal charges were irrelevant, and the real charge was having a bad attitude), but it was more likely he was imprisoned for respecting Aristotle, than disrespecting him.

Indeed, Aristotle was in many ways the first Empricist, and fell into/ out of fashion several times in the history of the Church

Libertarianism secretly relies on most individuals being prosocial enough to tip at a restaurant they won't ever visit again. An economy of genuinely selfish human-level agents would implode.

In other words, libertarianism could only ever work with real people. It would never work with the fictional creatures that people both for and against libertarianism philosophize themselves into imagining.

Gray Area: Eliezer, why are you concerned with untestable questions?

Questions you can easily test experimentally are hard for Science to get wrong.

There are numerous questions that are hard to test experimentally right this minute but are extremely important because of their future consequences. I bet you can think of one or two.

I chose quantum physics as my point of departure because the case is mathematically clear-cut.

Incidentally, it looks to me like you should be able to test macroscopic decoherence. Eventually. You just need nanotechnological precision, very low temperatures, and perhaps a clear area of interstellar (intergalactic?) space.

Where are the witty critical posters? I'm surprised to be the first to observe that this post (favorably comparing Science with Libertarianism) reads kind of like a self-parody of the OvercomingBias blog. Is one libertarian if one holds up each claim of libertarianism and says "Well, that's an empirical question. Let's look at the data". Because that's the scientific, empirical approach, it seems to me. I think libertarianism starts to look sill when viewed in that light. To be fair, so do the claims of any political party that size or larger, of which I'm aware.

Hi,

I've been reading LW sequences sine a few months, and I find them very interesting, but I think you made a mistake in mixing politics (libertarianism, french/american revolutions, ...) into this post.

I won't go into explaining why I think economical libertarianism is deeply flawed and not similar at all to the process of Science (for once, I don't think it degrades well at all), but above that, by calling into very complicated and very debated concepts, you're just making following your core reasoning harder to follow.

I also think you make some factual errors : saying the "american revolution" is a success but the french one a collapse is a great mistake. Most of the progress of the French Revolution lasted for very long and still last. The whole concept of "Human Rights", both the "first generation" rights, like the freedom of speech, and the "second generation" rights like universal access to education, come mostly from the French Revolution (the Declaration of Humans and Citizen Rights, to my knowledge, predates the 1st Amemendement). Most countries of Europe and South America use a civil code derived from it. The abolition of slavery by the French Revolution in 1793 was temporarily undone afterwards by Napoleon, but it was a firm stone on which the abolitionist have built afterwards.

And some very fundamental measures of the French Revolution, that were totally opposite to economical libertarianism, like the "taxation du prix du pain" (state-fixed price of bread to block speculation on breads and flour (the "Accapareurs")) lasted for almost two centuries, protecting France from famine, and making the "french baguette" a world-renewed food (because, to the contrary of what economical libertarianism predicts, the fixed price of bread leaded to a massive development of the bread industry in France, making bread the fundamental food, and forcing the bakery to compete on quality since they couldn't compete on price). That's just a few examples among many. Wiping the jump forward in humanism that represented the French Revolution and its continuing consequences nowadays in a few words as you did is, in my opinion, just not true.

Anyway, thanks for those very interesting posts.

I also think you make some factual errors : saying the "american revolution" is a success but the french one a collapse is a great mistake. Most of the progress of the French Revolution lasted for very long and still last.

With all due respect, your account of the French Revolution is just cartoonishly biased. The "progress of the French Revolution" included, among other things:

  • The introduction of total war fought with mass conscript armies, for which all the resources of the nation are requisitioned, in place of the 18th century limited and professional warfare regulated by strict codes and financed mostly from monarchs' private purses.

  • This invention leading to two decades of Europe-wide mass slaughter and destruction that left an unknown number of millions of people dead. It also left the recurring idea of spreading the national glory and ideology (as opposed to mere interests of rulers, which may be vicious but are at least limited and sane) by war and conquest.

  • Overall, the nationalist ideology born in the Revolution and the Revolutionary Wars, both in France and elsewhere as a reaction to it, had subsequent historical consequences for which "cataclysmic" would be an understatement. Subsequent European revolutions inspired by the French one, even if initially non-violent, would usually lead straight to bloody ethnic conflicts.

  • These "rights" introduced by the Revolution included the "right" to the imposition of a rigid centralized government and elimination of all local historical customs and institutions in the name of national homogenization. Those who resisted were dealt with by methods that Europe wouldn't see again until the Nazis. This is analogous to the bloody total wars between nations engendered by the nationalist ideology, only in this case the violence is directed towards those elements within the nation who refuse to fall in line.

The legacy and inspiration of all these innovations around the world has indeed lasted until the present day, but I don't think it's something to be happy and proud about -- certainly not a "jump forward in humanism" by any stretch of imagination.

As for the events specific to the Revolution itself, the picture is perhaps even more gruesome -- from mobs dismembering their victims and parading their heads on pikes (which revolutionary propagandists proudly bragged about) to the mechanized mass executions with the guillotines.

Even the abolition of slavery was done in a way that caused a race war of enormous brutality in Haiti (the main center of French slaveholding), which was concluded by an all-out extermination of the losing side down to the last man, woman, and child. And in any case, the effective abolition of worldwide slave trade (and subsequently slavery) was due to the ideological and political developments in Britain and America, and their subsequent political and military actions. The French Revolution had little or nothing to do with it.

You claim to be critiquing "economical libertarianism", but in fact you are critiquing microeconomics. For instance, you critique the familiar critique of price-fixing by presenting a purported counterexample. But the idea that price-fixing has certain predictable perverse consequences comes, not from libertarians, but from standard microeconomics, since it's a simple deduction from basic theory of supply and demand.

Libertarians do, to be sure, make heavy use of microeconomic theory, but this does not warrant calling microeconomics "economical libertarianism", any more than the use of a bicycle by Mao to commute to work would warrant calling bicycles "transportational communism".

So, to reinterpret your post, taking you to be attacking microeconomics, you are saying that the science of microeconomics is not in fact a science, since it is immune to empirical refutation, such as by the purported success of price-fixing.

The whole concept of "Human Rights", both the "first generation" rights, like the freedom of speech, and the "second generation" rights like universal access to education, come mostly from the French Revolution (the Declaration of Humans and Citizen Rights, to my knowledge, predates the 1st Amemendement).

If you think legal documents are important, this is backwards. Both were written in August 1789 and probably had little influence on each other. But both were largely based on George Mason's 1776 Virginia Declaration of Rights. Mason went on to write the US list, after refusing to sign the Constitution in 1787 and demanding a national list. In particular, his 1776 document gave freedom of press. I don't know how they expanded from press to speech; maybe one of the 1789 documents copied the other in this expansion, but Milton bundled the two in the 1644 Areopagitica as did many later people. As to the Enlightenment concept of human rights, it is pretty clear in Jefferson's Declaration of Independence. But it owes a lot to pre-Enlightenment English documents, particularly the Petition of Right of 1628.

But why should we trace these concepts to their endorsement by governments? The documents did not create the ideas, but adopted them from a long train of authors, particularly the French Enlightenment. The documents are important as signposts in the triumph of the Enlightenment, but they are very crude measures because talk is cheap.

How did France manage to have subsidized and/or price controlled bread without degrading the quality?

In a free market, price, weight, and quality would all fluctuate, so the system would be flexible at those points. Legislating two of those variables to be fixed would seem to leave the third responsible for reflecting economic changes, despite diminishing returns on changing it when the other two haven't been adjusted at all.

Avoiding degrading quality wouldn't be hard, one would set the price higher than the free market would have borne, so bakers have to compete for what inelastic demand there is by improving quality alone. The cost to society is the absence of cheaper, lower quality bread, which would be a reasonable policy choice.

If the legislated price were too low, bakers would compete on low manufacturing costs, and quality should spiral downward.

It seems controllable whether the cost of regulation is quality or affordability.

I hope someone shows up with knowledge of the actual history.

I'm assuming it wasn't a price floor-- that would make bread less affordable. I believe that subsidy to manufacturers + price controls leads to decline in quality because the incentives become doing just enough to meet the regulations and competing to get permissions and subsidies from the government.

It's possible that France exists to annoy libertarians. Essence of Style: How the French Invented High Fashion, Fine Food, Chic Cafes, Style, Sophistication, and Glamour (a book I've heard an interview about but not read) tells the story of Louis XIV making France into a world center of style. He picked winners, or created them. He promoted companies which continued to make high quality goods for centuries.

By libertarian standards, this should be just about impossible. Or maybe it's like winning the lottery-- you might succeed, but the odds are so low that it's a very bad strategy.... and, of course, the French aristocracy's spendthrift ways and insulation did lead to the French Revolution, and a utilitarian might say that any amount of delightful food and fashion just isn't worth it.

Still, picking winners on that scale is amazing. Does it take the unlikely combination of really smart person at the top chasing their own fascinations without those fascinations being too crazy rather than trying to guess what other people want?

It's telling that the Quatorze winner-picking you single out are in "high fashion, fine food, chic cafes, style, sophistication, and glamour." It's notoriously difficult to find objective measures of quality in those sectors.

17th century England was far more "libertarian" than 17th century France. And more prosperous too, right?

An economy of genuinely selfish human-level agents would implode.

Wouldn't the implosion leave all the selfish agents worse off? If they were even rudimentarily rational, wouldn't they then act in a way to prevent that inward collapse?

A lot of people have a problem with Kolmogorov complexity and Solomonoff induction being "ideals". Sure, you can't build a working perfect compressor in order to compute the Kolmogorov complexity of a binary string. The best you can do is to approximate it. Furthermore, the ways in which your compressor fails to achieve the perfect compression of Kolmogorov complexity are weaknesses of your compressor that a more powerful compressor could overcome... and so on and so on. It's only in the limit that you get a completely general compressor that can't be beaten, but by that point your compressor is requiring infinite amounts of computation in order to work perfectly. The solution is not to redefine the "perfect compressor", indeed that wouldn't work because anything less than Kolmogorov complexity can be beaten by some computable compressor. Instead, accept that the ideal can only be approximated in reality and that the better we can approximate it the better we are doing. The same goes for Solomonoff induction.

Eli: In your previous post you write about Copenhagen vs. MWI as if we have to decide on one of them. However, that's a somewhat un-Bayesian thing to do! A strict Solomonoff-Bayesian would simply accept that there is a posterior distribution over a space of infinitely many theories and interpretations. When this strict Bayesian goes to make a prediction about the outcome of an experiment he will take all of these interpretations into account according to their posterior probabilities - including interpretations far more insane than anything you have described.

Has anyone tried to actually DO Solomonoff induction against the real world? If I understand, it's incomputable, and even the idea of encoding reality as a program... well, it would be a very big program. So except as a pointer to and clarification of Occam's razor, does it have a real world use?

Generalizing: is it actually possible to use pure Bayesianism in any non-contrived, non-trivial context? And if purity can't be attained, is there an optimal impure approximation?

Libertarianism secretly relies on most individuals being prosocial enough to tip at a restaurant they won't ever visit again.

I'm puzzled that you gave that specific example, given that it's obviously wrong. Most countries do not have a culture of tipping, and their economies don't implode. They just have less headaches at bill time. And in many cases (a long way from libertarianism) their wait staff get paid a living wage.

I'm also not sure what it means for libertarianism to rely on something, since libertarianism is not an actual functioning thing in existence. But if it did exist and function, it would not rely on tipping.

Echoing Hal Finney...

It seems like you (Robin Hanson) are arguing that Libertarianism (small or big L) is some kind of alternative to rule making, or as I would say it 'believing in your theory'. But my impression -- not extravagantly well informed theoretically, but fairly informed by looking at actual, self styled libertarians -- is that Libertarianism is precisely an anti-theory theory. Terrified of the failures of other rules/models/belief systems, they create a new rule which says that all rules are wrong. The obvious tail chasing is well, what about your rule? They don't like that game.

Unfortunately, in practice it turns out that trying to decide ahead of time what rules are going to be valid and what rules aren't, is just as hard a problem, if not harder, than just deciding rules on a case by case basis. So the decision to once and for all make up a minimal rule set, and then disallow all future data and all future questions of rule making, turns out to fail just as badly, if not worse, than making bad rules to begin with. Thus for example, we see free market fundamentalists trying to prove that what has really gone wrong in the american housing market is too much government intervention, even though anybody willing to allow new data (new relative to their decision about how the world works that is) recognizes that the last 7-20 years (depending on how hard core you are) have been all about 'letting the market decide'.

In other words, libertarianism is just as much a religion as communism, and appeals to exactly the same psychology -- the need for easy answers, and the urge to push others around when your easy answers don't work. One of it's biggest problems is that it does not handle incremental acceptance well -- you have to drink the kool aid, or stay out of the party, no two ways about it. In the real world this requirement effectively kills the theory.

the fact that I have a visceral agreement with Scott Aaronson's claim that there is something similar about science and libertarianism says something about how down on science I am these days. The fact that I agree for exactly opposite reasons, says something about the relationship between outputs (objects) and processes (rules). Less opaquely: I agree there is something similar about Libertarianism and Science (big S) -- and it is not good at all if you are a fan of science (little s)

Incidentally, it looks to me like you should be able to test macroscopic decoherence. Eventually. You just need nanotechnological precision, very low temperatures, and perhaps a clear area of interstellar (intergalactic?) space.

Short of that, building a scalable quantum computer would be another (possibly easier!) way to experiment with macroscopic coherence. The difference is that with quantum computing, you wouldn't even try to isolate a quantum system perfectly from its environment. Instead you'd use really clever error-correction to encode quantum information in nonlocal degrees of freedom, in such a way that it can survive the decoherence of (say) any 1% of the qubits.

"Or do you think you're smarter than that?"

No?