So the second factor is that most people are rational enough for their own purposes. Oh, they go on wild flights of fancy when discussing politics or religion or philosophy, but when it comes to business they suddenly become cold and calculating. This relates to Robin Hanson on Near and Far modes of thinking. Near Mode thinking is actually pretty good at a lot of things, and Near Mode thinking is the thinking whose accuracy gives us practical benefits.

Seems to me that most of us make predictably dumb decisions in quite a variety of contexts, and that by becoming extra bonus sane (more sane/rational than your average “intelligent science-literate person without formal rationalist training”), we really should be able to do better.

Some examples of the “predictably dumb decisions” that an art of rationality should let us improve on:

• Dale Carnegie says (correctly, AFAIK) that most of us try to persuade others by explaining the benefits from our point of view (“I want you to play basketball with me because I don’t have enough people to play basketball with”), even though it works better to explain the benefits from their points of view. Matches my experiences, and matches also many/most of the local craigslist ads. The gains if we notice and change this one would be significant.
• Lots of people decide to take a job “to make more money”, but don’t bother to actually research the odds of getting that job, and the average payoff from that job (the latter, at least, is easy to look up on the internet) before spending literally years training for the job. Even in cases like med school. Again, significant payoff here, and in this case fairly minimal willpower requirements.
• Lots of us tend to mostly stick to our own opinions in conversations, even in cases where our impressions are no better data than our interlocutor’s impressions, and where the correct opinion can actually impact the goodness of our lives (e.g., which course to take on a work project whose outcome matters; which driving route is faster; which carwash to try) (these latter decisions are small, but small decisions add up).
• Similarly, lots of us decide we’re “good at X and bad at Y”, or that we’re “the sort of people who do A in such-and-such a specific manner”, and quit learning in a particular domain, quit updating our skill-sets, keep suboptimal beliefs or practices glued to our identities instead of looking around to see how others do things and what methods might achieve greater success. Lots of us spend far more of our thinking time noting all the reasons why we’re best off doing what we’re doing than we do looking for new ways to do things, even when such looking has tended to give us useful improvements.
• Lots of people run more risk of death by car than they would upon consideration choose, e.g. by driving too close to the car in front of them (the half-second earlier that you get home isn’t worth it) or by driving while tired. At the same time, lots of people refrain from enjoyable activities such as walking around at night or swimming off the coast of Florida despite the occasional sharks, in cases where the activities in fact pose nearly negligible danger, but the dangers in question are vivid and easy to over-estimate.

Imagine a world where the only way to become really rich is to win the lottery (and everybody is either risk averse or at least risk neutral). With an expected return of less than $1 per $1 spent on tickets, rational people don't buy lottery tickets. Only irrational people do that. As a result, all the really rich people in this world must be irrational.

In other words, it is possible to have situations where being rational increases your expected performance, but at the same time reduces your changes of being a super achiever. Thus, the claim that "rationalists should win" is not necessarily true, even in theory, if "winning" is taken to mean being among the top performers. A more accurate statement would be, "In a world with both rational and irrational agents, the rational agents should perform better on average than the population average."

I'm not sure if it was your intent to point this out by contrast, but I would like to point out that a reasonable art of "kicking" would not rely on you making conscious decisions, let alone explicitly rational ones. Rather, it would rely on you ensuring that your subconscious has been freed from sources of bias ahead of time, and is therefore able to safely leap to conclusions in its usual fashion. An art that requires you to think at the time things are actually happening is not much of an art.

Case in point: when reading "Stuck In The Middle With Bruce", I became aware of a subconsciously self-sabotaging behavior I'd done recently. So I "kicked" it out by crosslinking the behavior with its goal-satisfaction state. It would be crazy to wait until the next occasion for that behavior to strike, and then try to reason my way around it, when I can just fix the bloody thing in the first place. (Interestingly, I mentioned the story to my wife, and described how it related to my own behavior... and she thought of a different sort of self-sabotage she was doing, and applied the same mindhack. So, as of now, I'd say that story was one of the top 5 most valuable things I've gotten from LW.)

Now, in the case of extinguishing a behavior, there's no way you can absolutely prove you've fixed something permanently; the best you can do is show that the thought process that you use to produce an autonomous response before applying a technique, no longer produces the same response afterward. Also, sometimes you catch a break: you find yourself in a situation, expecting yourself to do the same old stupid thing you've been doing before, and then you find you don't need to, or notice a few seconds later that you already did something completely different, and a much better choice.

Truth is, our brains really aren't that bad at making decisions, once you take out the "priority overrides" that mess things up.

Anyway, I'm rambling a bit now. The point is, "kicking" is generally not something you do at the time -- you do it in advance of the next time....

Because your brain is faster than you are.

And this is why I am not so impressed by Eliezer's claim that an x-rationality instructor should be successful in their non-rationality life. Yes, there probably are some x-rationalists who will also be successful people. But again, correlation 0.1. Stop saying only practically successful people could be good x-rationality teachers! Stop saying we need to start having huge real-life victories or our art is useless! Stop calling x-rationality the Art of Winning! Stop saying I must be engaged in some sort of weird signalling effort for saying I'm here because I like mental clarity instead of because I want to be the next Bill Gates! It trivializes the very virtues that brought most of us to Overcoming Bias, and replaces them with what sounds a lot like a pitch for some weird self-help cult...

I think the truth is non-symmetrical: rationalism is the art of not failing, of not being stupid. I agree with you that "rationalists should win big" is not true in the sense Eliezer claims. However, rationalists should be generally above average by virtue of never failing big, never losing too much, e.g. not buying every vitamin at the health food store, not in cults, not bemoaning ancient relationships, etc.

And for this post, I use "benefits" or "practical benefits" to mean anything not relating to philosophy, truth, winning debates, or a sense of personal satisfaction from understanding things better. Money, status, popularity, and scientific discovery all count.

In my life, I've used rationality to tackle some pretty tough practical problems. The type of rationality I have been successful with hasn't been the debiasing program of Overcoming Bias, yet I have been employing scientific thinking, induction, and heuristic to certain problems in ways that are atypical for the category of people you are calling normal rationalists. I don't know whether to call this "x-rationality" or not, partly because I'm not sure the boundaries between rationality and x-rationality are always obvious, but it's certainly more advanced rationality than what people usually apply in the domains below.

On a general level, I've been studying how to get good (or at least, dramatically better) at things. Here are some areas where I've been successful using rationality:

• Recovering from social anxiety disorder and depression
• Social skills
• Fashion sense
• Popularity / social status in peer group
• Dating

I'm not using success necessarily to mean mastery, but around 1-2 standard deviations of improvement from where I started.

I do find it interesting that many people are not achieving practical benefits from their studies of more advanced rationalities. I agree with you that akrasia is a large factor in why they do not get significant practical benefits out of rationality. I am going to hypothesize an additional factor:

The practical benefits of x-rationality are constrained because students of x-rationality (such as the Overcoming Bias / Less Wrong) schools of thought focus on critical rationality, yet critical rationality is only good for solving certain types of problems.

In my post on heuristic, I drew a distinction between what I'm calling "critical rationality" (consisting of logic, skepticism, and bias-reduction) and "creative rationality" (consisting of heuristic and inference). Critical rationality concerns itself with idea validation, while creative rationality concerns itself with idea creation (specifically, of ideas that map onto the territory).

Critical rationality is necessary to avoid many mistakes in life (e.g. spending all your money on lottery tickets, high-interest credit card debt, Scientology), yet perhaps it runs into diminishing returns for success in most people's lives. For developing new ideas and skills that would lead people to success above a mundane level, critical rationality is necessary but not sufficient, and creative rationality is also required.

If in 1660 you'd asked the first members of the Royal Society to list the ways in which natural philosophy had tangibly improved their lives, you probably wouldn't have gotten a very impressive list.

Looking over history, you would not have found any tendency for successful people to have made a formal study of natural philosophy.

An understanding of 'x-rationality' has helped me find the world a little less depressing and a little less frustrating. Previously when observing world events, politics and some behaviours in social interactions that seemed incomprehensible without assuming depressing levels of stupidity, incompetence or malice I despaired at the state of humanity. An appreciation of human biases and evolutionary psychology (some of which stems from an interest in both going back well before I ever started reading OB) gives me a framework in which to understand events in the world which I find a lot more productive and optimistic.

An example from politics: it is hard to make any rational sense of drug prohibition when looking at the evidence of the costs and benefits. This would tend to lead to an inevitable conclusion that politicians and the voting public are either irredeemably stupid or actively seeking negative outcomes. Understanding how institutional incentives to maintain the status quo, confirmation bias and signaling effects (politicians and voters needing to be 'seen to care' and/or 'seen to disapprove') can lead to basically intelligent and well meaning people maintaining catastrophically wrong beliefs at worst allows for accepting the status quo without assuming the worst about one's fellow man and at best maps out plausible paths for achieving political change by recognizing the true nature of the obstacles.

An example from social interactions: I suffered a fair amount of personal emotional stress reconciling what I had been led to believe 'ought' to work when interacting with others and the apparently much less pleasant realities of what seemed to be successful in reality. The only conclusion I could draw was that everyone deliberately lied about the way human interactions worked for their own mysterious and possibly malicious reasons. Coming to an understanding of evolutionary psychology and signaling explanations for many common patterns of human behaviour allows me to reconcile 'doing what works' with a belief that most people are not consciously misleading or malicious most of the time. Many people don't appear to be aware of the contradictions inherent in social interactions but as someone who saw them but could not explain them without assuming the worst, discovering explanations that did not require imputing conscious malice to others allowed for a much more positive outlook on the world.

I could give a number of examples of how 'regular' rationality rigorously applied to areas of life where it is often absent have also directly helped me in my life but they seem slightly off topic for this thread.

Am I the only one who is isn't entirely positive towards the heavy use of language identifying the LW community as "rationalists", including terms like "rationalist training" etc.? (Though he is by far the heaviest user of this kind of language, I'm not really talking about Eliezer here, his language use is whole topic on its own - I'm restricting this particular concern to other people, to the general LW non-Eliezer jargon). Is strongly self-identifying as a "rationalist" really such a good thing? Does it really help you solve problems? (I second the questions raised by Yvain). Though perhaps small, isn't there still a risk that the focus becomes too much on "being a rationalist" instead of on actually solving problems?

Of course, this is a blog about rationality and not about specific problems, so this kind of language is not suprising and sometimes might even be necessary. I'm just a bit hesitant towards it when the community hasn't actually shown that it's better at solving problems than people who don't self-identify as rationalists and haven't had "rationalist training", or shown that the techniques fostered here have such a high cross-domain applicability as seems to be assumed. Maybe after it has been shown that "rationalists" do better than other people, people who just solve problems, I would feel better about this kind of jargon.

Sometimes, people do worse when they try to be rational because they have a poor model of rationality.

One error I commonly see is the belief that rationality means using logic, and that logic means not believing things unless they are proven. So someone tries to be "rational" by demanding proof of X before changing their behavior, even in a case where neither priors nor utilities favor not X. The untrained person may be doing something as naive as argument-counting (how many arguments in favor of X vs. not X), and is still likely to come out ahead of the person who requires proof.

A related error is using Boolean models where they are inappropriate. The most common error of this type is believing that a phenomenon, or a class of phenomena, can have only one explanation.

I’m partly echoing badger here, but it’s worth distinguishing between three possible claims:
(1) An “art of rationality” that we do not yet have, but that we could plausibly develop with experimentation, measurements, community, etc., can help people.
(2) The “art of rationality” that one can obtain by reading OB/LW and trying to really apply its contents to one’s life, can help people.
(3) The “art of rationality” that one is likely to accidentally obtain by reading articles about it, e.g. on OB/LW, and seeing what happens to rubs off, can help people.

There are also different notions of “help people” that are worth distinguishing. I’ll share my anticipations for each separately. Yvain or others, tell me where your anticipations match or differ.

Regarding claim (3):
My impression is that even the art of rationality one obtains by reading articles about it for entertainment, does have some positive effects on the accuracy of peoples’ beliefs. A couple people reported leaving their religions. Many of us have probably discarded random political or other opinions that we had due to social signaling or happenstance. Yvain and others report “clarity-of-mind benefits”. I’d give reasonable odds that there’s somewhat more benefit than this -- some unreliable improvement in peoples’ occasional, major, practical decisions, e.g. about which career track to pursue, and some unreliable improvement in peoples’ ability to see past their own rationalizations in interpersonal conflicts -- but (at least with hindsight bias?) probably no improvements in practical skills large enough to show up on Vladimir Nesov’s poll. Does anyone’s anticipations differ, here?

Regarding claim (2):
I’d a priori expect better effects from attempts to really practice rationality, and to integrate its thinking skills into one’s bones, than from enjoying chatting about rationality from time to time. A community that reads articles about skateboarding, and discusses skateboarding, will probably still fall over when they try to skateboard twenty feet unless they’ve also actually spent time on skateboards.

As to the empirical data: who here has in fact practiced (2) (e.g., has tried to integrate x-rationality into their actual practical decision-making, as in Yvain’s experiment/technique, or has used x-rationality to make major life decisions, or has spent time listing out their strengths and weaknesses as a rationalist with specific thinking habits that they really work to integrate in different weeks, or etc.)? This is a real question; I’d love data. Eliezer is an obvious example; Yvain cites the impressiveness of Eliezer’s 2001 writings as counter-evidence (and it is some counter-evidence), but: (1) Eliezer, in 2001, had already spent a lot of time learning rationality (though without the heuristics and biases literature); and (2) Eliezer was at that time busy with a course of action that, as he now understands things, would have tended to destroy the world rather than to save it. Due to insufficient rationality, apparently.

I’ve practiced a fair amount of (2), but much less than I could imagine some practicing; and, as I noted in the comment Yvain cited, it seems to have done me some good. Broadly similar results for the handful of others I know who try to get rationality into their bones. Less impressive than I’d like, but I tend to interpret this a a sign we should spend more time on skateboards, and I anticipate that we’ll see more real improvement as we do.

The most important actual helps involve that topic we’re not supposed to discuss here until May, but I’d say we were able to choose a much higher-impact way to help the world than people without x-rationality standardly choose, and that we’re able to actually think usefully about a subject where most conversations degenerate into storytelling, availability heuristics, attaching overmuch weight to specific conjunctions, etc. Which, if there’s any non-negligible chance we’re right, is immensely practical. But we’re also somewhat better at strategicness about actually exercising, about using social interaction patterns that work better than the ones we were accidentally using previously (though far from as well as the ones the best people use), about choosing college or career tracks that have better expected results, etc.

Folks with more data here (positive or negative), please share.

Regarding claim (1):
I guess I wouldn’t be surprised by anything from “massive practical help, at least from particular skilled/lucky dojos that get on good tracks” to “not much help at all”. But if we do get “not much help at all”, I’ll feel like there was a thing we could have done, and we didn’t manage to do it. There are loads of ridiculously stupid kinds of decision-making that most people do, and it would be strange if there were no way we could get visible practical benefit from improving on that. Details in later comments.

Michael Vassar:

nerds, scientists, skeptics and the like who like to describe their membership in terms of rationality are [not] noticibly better than average at behavioral rationality, as opposed to epistemic rationality where they are obviously better than average but still just hideously bad.

Simply applying "ordinary rationality" to behavior is extreme. People don't use reason to decide if fashion is important, they just copy. Eliezer's Secret Identities post seems to make a very similar point, which seemed to largely match this post. One point was to get rationality advice from people who actually found it useful, rather than ordinary nerds who fetishize it.

By "decision", I don't mean the decision to get up in the morning, I mean the sort that's made on a conscious level and requires at least a few seconds' serious thought.

Consider yourself lucky if that doesn't describe getting up in the morning for you.

Anyway, not that this counts at all (availability bias), but I made a rational decision a couple of days ago to get some sleep instead of working later into the night on homework. I did exactly that.

In fact, I just made a rational decision-- just now-- to quit reading the article I was reading, work on homework for a few minutes and then go to bed. I haven't gotten to bed yet. Otherwise, that's going well.

...but you will disagree with me. And we are both aspiring rationalists, and therefore we resolve disagreements by experiments. I propose one.

I'm surprised you expected most of your readers to disagree. I think it's pretty clear that the techniques we work on here aren't making us much more successful than most people.

Humans aren't naturally well equipped to be extreme rationalists. The techniques themselves may be correct, but that doesn't mean we can realistically expect many people to apply them. To use the rationality-as-martial art metaphor, if you taught Shaolin kung fu to a population of fifty year old couch potatoes, they would not be able to perform most of the techniques correctly, and you should not expect to hear many true accounts of them winning fights with their skills.

Perhaps with enough work we could refine the art of human instrumental rationality into something much better than what we've got, maybe achieve a .3 correlation with success rather than a .1, but while a fighting style developed explicitly for 50 year old couch potatoes might give your class better results than other styles, you can only expect so much out of it.

This experiment seems easy to rig4; merely doing it should increase your level of conscious rational decisions quite a bit. And yet I have been trying it for the past few days, and the results have not been pretty. .... [O]ne way to fail your Art is to expect more of it than it can deliver.... Perhaps there are developments of the Art of Rationality or its associated Arts that can turn us into a Kellhus or a Galt, but they will not be reached by trying to overcome biases really really hard.

To make a somewhat uncharitable paraphrase: you read many articles on rationality, did not actually use them to change the way you make decisions, and found that the rationality hasn’t changed the results of your decisions much. You conclude, not that you aren’t practicing rationality, but that rationality can’t deliver practical goods at all, at least not as taught.

I agree we need practices for better incorporating OB/LW/new techniques of rationality into our actual practice of inference and decision-making. But it seems like the “and I’m not actually using this stuff much” result of your experiment should prevent “it hasn’t made my life much better” from telling you all that much about whether the OB/LW inference or decision-making techniques, if one does practice them, could make one’s life better.

Here's one example of a change I've made recently, which I think qualifies as x-rationality. When I need to make a decision that depends on a particular piece of data, I now commit to a decision threshold before I look at the data. (I feel like I took this strategy from a LW article, but I don't remember where now.)

For example, I recently had to decide whether it would be worth the potential savings in time and money to commute by motorcycle instead of by car. I set a threshold for what I considered an appropriate level of risk beforehand, and then looked up the accident statistics. The actual risk turned out to be several times larger than that.

Had I looked at the data first, I would have been tempted to find an excuse to go with my gut anyway, which simply says that motorcycles are cool. (I'm a 23-year-old guy, after all.) A high percentage of motorcyclists experience a serious or even fatal accident, so there's a decent chance that x-rationality saved me from that.

I have yet to hear what anyone even means by "rationalism" or "rationalist," let alone "x-rationality." People often refer to the "techniques" or "Art of rationality" (a particularly irksome phrase), though as best I can tell, these consist of Bayes theorem and a half-dozen or so logical fallacies that were likely known since the time of Aristotle. Now, I've had an intuitive handle on Bayes theorem since learning of it in high school pre-calc, and spotting a logical fallacy isn't particularly tough for anyone accustomed to close reading of philosophy or doing science (or who's studied for the LSAT). So apart from simply calling oneself a "rationalist" and feeling really good about being a part of some "rationalist community" (much like Dennett's tone-deaf coining of the term "brights" to describe atheists), is there actually anything to this?

Would Newton have gone even further if he'd known Bayes theory? Probably it would've been like telling the world pool champion to try using more calculus in his shots: not a pretty sight.

An interesting choice of example, given that Bayesian probability theory as we know it (inverse inference) was more or less invented by Laplace and used to address specific astronomical controversies surrounding the introduction of Newton's Laws, having to do with combining multiple uncertain observations.

Only a handful responded

I am reserving my judgment for a couple of years. See how I'm doing then.