A lot of dojos preserve to some degree the social standards of Eastern countries where the sensei's sensei came from. And in Eastern countries, it's much less acceptable to try to question your teacher, or change things, or rock the boat, or show any form of weakness. I taught school in Japan for a while, and the first thing I learned was that naively asking "Any questions?" or "Any opinions on this?" or even "Anyone not understand?" was a waste of time.

Western cultures are a lot better at this, but not ideal. There's still pressure not to be the one person who asks all the questions all the time, and there's pressure not to say anything controversial out of the blue because you lose more status if you're wrong than you gain if you're right. I think part of the problem is that there really are dumb or egotistical people who, if given the chance will protest that they know a much better way to do everything and will waste the time of everyone else, and our society's decided to .make a devil's bargain to keep them under control.

The best solution to this is to found a new culture, live isolated from the rest of the world for a century developing different cultural norms, and then start the rationality dojo there. Of possible second-best solutions:

• My Favorite Liar. Tell people that you're going to make X deliberately incorrect statements every training session and they've got to catch them.

• Clickers. One of my lecturers uses these devices sort of like remote controls. You can input information into them and it gets sent wirelessly and anonymously to the lecturer's laptop. The theory is that if he says "Raise your hand if you don't understand this" or even "...if you disagree with this", no one will, but if he says "Enter whether or not you understand this into your clicker" he may get three or four "don't understand" responses. Anonymous suggestion boxes are a low-tech form of the same principle.

• I always found the concept of Crocker's Rules very interesting. I also remember hearing of a community (wish I could remember which) in which it was absolutely forbidden to give negative feedback under certain circumstances, and the odd social dynamics that created. In a dojo-like setting, there might be situations when either of these two rules could be ritually enacted - for example, a special Crocker Hat, such that anyone wearing that hat was known to be under Crocker's Rules, and a special No Negative Feedback Hat (but with a flashier name, like White Crane Hat of Social Invincibility), which someone could wear when questioning the master or something and be absolutely immune to any criticism.

http://www.youtube.com/watch?v=gEDaCIDvj6I

" Kiai Master vs MMA "

By far my biggest problem with the way you discusses rationality is the way that you draw on the tropes of Eastern martial arts instruction, and it's because of exactly this sort of thing - those tropes are appropriate for one who wants to be considered a guru, which is the opposite of your stated aims. It's something I have to warn people about if I'm recommending something you've written.

The best study I know of that addresses rationality in pro sports is Moneyball, by Michael Lewis. It tells the story of how Billy Beane followed the approaches recommended by people who studied the voluminous statistics on Baseball and pointed to non-standard evaluations of what talents and strategies made a difference in getting to the post-season. It's relevant for two reasons.

1) It talks about the psychology of players and coaches who found reasons to stick with the tried-and-true, even when non-standard approaches had some evidence in their favor.

2) it talks about the process of re-analyzing the statistics to figure out what aspects of the game matter. Part of this is deciding what the goal is, and part is figuring out what helps you reach the goal. In the case of baseball, Beane agreed with the those who argued that getting to the post-season cost-effectively was the goal. That means figuring out how to win more games over a season, which is more straightforward than figuring out how to win individual games. Cost-effectiveness translates to recruiting players whose value is higher than what other teams are willing to pay. Many unconventional styles of play turned out to be valuable, which led to a team that looked bizarre by accepted standards, but who won consistently but unspectacularly.

edited to use proper LW linking

Actually, the martial arts world has recently benefited from a big dose of reality in the form of mixed-martial-arts tournaments. Throwing together fighters from different styles demonstrated that some were overwhelmingly superior to others and unleashed a rapid evolution of technique that blended together the clearly superior methods. http://en.wikipedia.org/wiki/Mixed_martial_arts#Evolution_of_fighters

I don't know much about American professional sports--even less about pro sports in other countries--for that matter, I don't know much about martial arts. But as far as I do know, pro sports have none of these problems. Athletes do all sorts of outrageous things; coaches, athletes, and strategies are chosen on merit; absurdly detailed statistics are collected. Baseball players admire Babe Ruth but they don't idolize him. The analogy between pro sports and martial arts isn't perfect, but neither is the analogy between martial arts and rationality.

So, what do pro sports have to "keep them honest", that martial arts don't?

• Teams of athletes compete in tournaments that directly demonstrate their skills at their sport. In theory, the sport of martial artists is hand-to-hand combat, but martial arts tournaments never allow eye-gouging, biting, and so on. The further the distance between the tournament rules and reality, the less useful the tournament will be. Fortunately, I don't think there's a rationalist equivalent of eye-gouging, so setting up tournament rules should be relatively easy.

• An athlete or coach who gives up a pet technique for one that works better will be rewarded with status and money. The culture of pro sports permits athletes to train in entirely different ways from one season to the next, and coaches to change their playbooks whenever they like. Martial arts schools are stagnant by comparison. The money in pro sports comes from fans (directly through sales or indirectly through advertising) and it would take a lot of effort to raise awareness for rationality. But if rationality masters were really so awesome they'd have no trouble getting the money, right?

• Pro sports aren't considered a "way of life" the way martial arts are. Athletes move from one team to another and it's not a big deal, but if Bruce Lee had given up Jeet Kune Do during his life, and taken up Shotokan Karate instead, the martial arts world would still be talking about it. It would be like the Pope converting to Wicca. Readers of OB will probably agree with me that rationality should be a way of life; but I hope they'll also agree that no particular school of rationality should be.

Comment edited for suitable URL tags.

This is an aside, but related: there is an awesome website dedicated to investigating and uncovering fraud and vice within the Martial Arts world: Bullshido. Participants post about questionable doings (such as masters who claim to teach or use ki) and visit those schools to report on the fraud. It also has areas for regular Martial Arts chat, but this sub-forum is the one that primarily focuses on investigations.

This, then, is an incomplete outline to an answer to your question: due diligence and active pursuit of those who fraudulently represent "arts of rationality." If there were a series of Dojos of the Bayesian Sect, those dojos would be responsible for exposing the Thousand Schools of False Ways. It would ever be an ongoing battle. Just as the members of Bullshido are constantly encountering and exposing "McDojos" that claim to teach practical self-defense techniques that would, in reality, probably just get you killed or kids' grappling schools taught by instructors with sex assault convictions.

I believe the video you were looking for is here:

[censored the link on account of comment below]

http://www. yachigusaryu. com/blog/2007/02/no-touch-knockout-fraud-exposed.html

In defense of martial arts I want to note that in their case the mentioned key observations for epistemic viciousness are not independend and partially unavoidable. Especially the "deference to famous historical masters" point is no more than a logical conclusion of the "we live data poverty" point: Until less than a hundred years ago people would risk their health (even life) in fights to establish the superiority of the brand of martial artists they espoused. Not to mention life circumstances in general, which more often than not included more or less regular violent conflicts, which usually meat hand-to-hand combat (as opposed to guns). These people did certainly not live in data poverty regarding the efficacy of their techiques, hence it is fair to assume they were (at least on average) better fighters than today's martial artists.

What can be done about it? We can fight.

The Master can argue for Creationism, and try to defeat the pupil's refutation of it. We can argue for or against One-boxing on Newcomb's problem. Or pretend to be the AI arguing that the Gatekeeper should free it. The Master is only Master for as long as s/he is undefeated.

What successful(as winning in the UFC) martial arts have in common: realistic sparring. BJJ(Brazilian Jiu-Jitsu), Boxing and Muay-Thai all have this(of course with certain restrictions) and they have shown to be effective.

Edit: translating to the rationality dojo, we need some method of empirically testing the techniques. One way would be to do some exercises after the theory was learned like asking some question or doing some test and then seeing who gets the answer right. Another would be to ask for real-life experiences like: what irrational believe or habit did you manage to get rid of in the last month/week? Where did you improve something?

What can be done about it?

I'll suggest it first so nobody else has to lose karma over it: don't start a rationality dojo.

Is this what you're looking for?

I was disappointed in my dojos because I went there to learn self defense and psychological survival but only learned about punches and kicks and heard promises of eventually knowing enough to "win" in a fight.

One of the ways they measure how much "better" you are is by having you punch or kick easily breakable wooden boards. Three or more is impressive but broken boards neither prepares you for "winning" a fight or knowing the self-defense techniques involved in preventing or de-escalating a potential fight. Yet, it feels pretty darn good to break those boards.

Martial Arts exists as one extremely unlikely and limited-use scenario of self defense. Against most of your peers, some MA knowledge will help you not get your ass kicked. Against people who employ violence for a living, MA skills are more likely to make you either a corpse or involved in a legal dispute and likely sporting a wonderful case of PTSD (whether you win or lose the fight, you still lose).

Self defense isn't about finding the "best" martial art. If rationality is going to have true value, teaching it isn't mainly going to be for some lower utility such as fighting the theists.

One of my probability books was about the usage of the term probability - what it meant. (Ian Hacking?) Once upon a time, probable meant being attested to by some authority. That was the measure of truth - what Authority had to say.

"The Truth" is judged differently by different people. For some, The Truth is what an Authority says, where Authority is identified by signaling. In the absence of data, how else are you to judge truth claims? So competing Authorities compete through signaling louder, more congruently, more forcefully, which often amplifies the nuttiness, with increasing claims of certainty and disrespect for other theories.

For a lot of problems like this, I take the mind as a bunch of competing pattern matching algorithms, and note that it doesn't appear that your algorithms are weighted the same way mine are. Most people have their Authority algorithms weighted much higher than I do, and much higher than most people here do as well.