Followup toEvery Cause Wants To Be A Cult

    A novice rationalist studying under the master Ougi was rebuked by a friend who said, "You spend all this time listening to your master, and talking of 'rational' this and 'rational' that—you have fallen into a cult!"
    The novice was deeply disturbed; he heard the words, "You have fallen into a cult!" resounding in his ears as he lay in bed that night, and even in his dreams.
    The next day, the novice approached Ougi and related the events, and said, "Master, I am constantly consumed by worry that this is all really a cult, and that your teachings are only dogma."
    Ougi replied, "If you find a hammer lying in the road and sell it, you may ask a low price or a high one.  But if you keep the hammer and use it to drive nails, who can doubt its worth?"
    The novice said, "See, now that's just the sort of thing I worry about—your mysterious Zen replies."
    Ougi said, "Fine, then, I will speak more plainly, and lay out perfectly reasonable arguments which demonstrate that you have not fallen into a cult.  But first you have to wear this silly hat."

    Ougi gave the novice a huge brown ten-gallon cowboy hat.
    "Er, master..." said the novice.
    "When I have explained everything to you," said Ougi, "you will see why this was necessary.  Or otherwise, you can continue to lie awake nights, wondering whether this is a cult."
    The novice put on the cowboy hat.
    Ougi said, "How long will you repeat my words and ignore the meaning?  Disordered thoughts begin as feelings of attachment to preferred conclusions.  You are too anxious about your self-image as a rationalist.  You came to me to seek reassurance.  If you had been truly curious, not knowing one way or the other, you would have thought of ways to resolve your doubts.  Because you needed to resolve your cognitive dissonance, you were willing to put on a silly hat.  If I had been an evil man, I could have made you pay a hundred silver coins.  When you concentrate on a real-world question, the worth or worthlessness of your understanding will soon become apparent.  You are like a swordsman who keeps glancing away to see if anyone might be laughing at him—"
    "All right," said the novice.
    "You asked for the long version," said Ougi.
    This novice later succeeded Ougi and became known as Ni no Tachi. Ever after, he would not allow his students to cite his words in their debates, saying, "Use the techniques and do not mention them."

 

    A novice rationalist approached the master Ougi and said, "Master, I worry that our rationality dojo is... well... a little cultish."
    "That is a grave concern," said Ougi.
    The novice waited a time, but Ougi said nothing more.
    So the novice spoke up again:  "I mean, I'm sorry, but having to wear these robes, and the hood—it just seems like we're the bloody Freemasons or something."
    "Ah," said Ougi, "the robes and trappings."
    "Well, yes the robes and trappings," said the novice.  "It just seems terribly irrational."
    "I will address all your concerns," said the master, "but first you must put on this silly hat."  And Ougi drew out a wizard's hat, embroidered with crescents and stars.
    The novice took the hat, looked at it, and then burst out in frustration:  "How can this possibly help?"
    "Since you are so concerned about the interactions of clothing with probability theory," Ougi said, "it should not surprise you that you must wear a special hat to understand."
    When the novice attained the rank of grad student, he took the name Bouzo and would only discuss rationality while wearing a clown suit.

 

Part of the Death Spirals and the Cult Attractor subsequence of How To Actually Change Your Mind

Next post: "Asch's Conformity Experiment"

Previous post: "Guardians of Ayn Rand"

Comments

sorted by
magical algorithm
Highlighting new comments since Today at 2:11 PM
Select new highlight date
All comments loaded

Well, it's clear that a lot of people found the koans confusing. Silly me, I realized I forgot to include Mumon's commentaries!

Mumon's commentary on the first koan:

Because Ni no Tachi hated rich men, Ougi gave up his wealth of silence. When Ni no Tachi became a rich man, He loaned money to his students.

Mumon's commentary on the second koan:

A flower is neither bread nor water, Why not replace it with a weed? Bouzo changed the shape of Ougi's garden, But did he really understand?

From the Discworld novel Thief of Time:

In the Second Scroll of Wen the Eternally Surprised a story is written concerning one day when the apprentice Clodpool, in a rebellious mood, approached Wen and spake thusly:
"Master, what is the difference between a humanistic, monastic system of belief in which wisdom is sought by means of an apparently nonsensical system of questions and answers, and a lot of mystic gibberish made up on the spur of the moment?"
Wen considered this for some time, and at last said: "A fish!"
And Clodpool went away, satisfied.

Student: What is truth? And what is God?

Teacher: You don't really want an answer to that question.

Student: Yes, I do. Please.

Teacher (after thinking for a moment): If I take a lamp and shine it toward a wall, a bright spot will appear on the wall. The lamp is our search for truth, for understanding. Too often, we assume that the light on the wall is God. But the light is not the goal of the search. It is the result of the search. The more intense the search, the brighter the light on the wall. The brighter the light on the wall, the greater the sense of revelation upon seeing it.

[All the students are looking on with blank faces]

Teacher: Similarly, someone who does not search, who does not bring a lantern with him...sees nothing. What we perceive as God is the byproduct of our search for God. It may simply be an appreciation of the light, pure and unblemished, not understanding that it comes from us. Sometimes we stand in front of the light and assume that we are the center of the universe. God looks astonishingly like we do. Or we turn to look at our shadow and assume that all is darkness. If we allow ourselves to get in the way, we defeat the purpose, which is to use the light of our search to illuminate the wall in all its beauty... and all its flaws... and in so doing better understand the world around us.

Student: Ah...yes...but... What is truth? And what is God?

Teacher (rolls eyes): Truth is... a river.

[amazed looks and murmurs from the students]

Student: And god?

Teacher: God is the mouth of the river.

[pleased expressions and excited exclamations among the students, Teacher facepalms]

Excellent quote, but you should really attribute it.

It's from Babylon 5, season 5, episode 14, "Meditations on the Abyss".

The Teacher is G'kar, the student is some random Narn.

If you can't figure out how to use the hammer, but you nonetheless convince younger students of its importance, aren't you likely to be in a cult? Or more likely, if you can see that many of the other teachers are teaching material they don't understand?

Sounds like either a cult or a college.

Chris, for a while I was a member of what most people would consider a "cult", and from my experience "cults" usually teach people useful things. So it is a mistake to assure yourself you are not in a cult because you see that you are learning useful things.

That was cute. I'm not sure I understand it, though.

I think I'll side with the novices against Ougi here. The novices deserve a clearer answer than "think of ways to resolve your doubts" and "all will be clear when you try to use this stuff." Cults usually tell people many things that are actually useful, and confronting leaders seems a reasonable way to resolve doubts. As I said before, the word "cult" is a bit too easy a word to throw around - I'd prefer a clearer description of what it means exactly and how to recognize one.

http://en.wikipedia.org/wiki/Cult

A few things to look for, gleaned from the Wikipedia article:

In Cults, coercive Mind Control is said to take these forms:

- People are put in physically or emotionally distressing situations;
- Their problems are reduced to one simple explanation, which is repeatedly emphasized;
- They receive what seems to be unconditional love, acceptance, and attention from a charismatic leader or group;
- They get a new identity based on the group;
- They are subject to entrapment (isolation from friends, relatives and the mainstream culture)

Cults also seem to involve the adulation of charismatic leaders, who become corrupted by the power the situation brings them. The most common complaints against cults involve sexual abuse of members. Cults have also been known to cause harm through getting members to forego medical care.

I think this is ok for a place to start.

Okay, so taking a look at this checklist for LessWrong, we get...

People are put in physically or emotionally distressing situations;

Nope. (Well, not intentionally, anyway... does the basilisk count? People did have nightmares from that...)

Their problems are reduced to one simple explanation, which is repeatedly emphasized;

...Sort of? I mean, "rationality" is trumpeted as the Winning Way, but I'd say that's justified in that it is the Winning Way. Besides, the category of "rationality" is rather broad, which might mean it doesn't satisfy the definition of "one simple explanation"... eh. Tentatively going with "yes" for now.

They receive what seems to be unconditional love, acceptance, and attention from a charismatic leader or group;

Uh, yeah, not even close.

They get a new identity based on the group;

Pretty clear that this one gets a "yes". People self-identify as "LWers" all the time.

They are subject to entrapment (isolation from friends, relatives and the mainstream culture)

Nope.

So, LessWrong meets two of the five criteria, and does not satisfy the remaining three. Does that make it a cult, or not?

(Unlurking and creating an account for use from this point onwards; どうかお手柔らかにお願いします。)

Something I found curious in the reading of the comments for this article is the perception that Bouzo took away the conclusion that clothing was in fact important for probability.

Airing my initial impression for possible contrast (/as an indication of my uncertainty): When I read the last sentence, I imagined an unwritten 'And in that moment the novice was enlightened', mirroring the structure of certain koans I once glanced through.

My interpretation is/was that those words of Ougi's were what caused the novice to realise his error (in focusing on the clothing rather than the teachings when considering likelihood of cultishness), the absurdity of a hat worn affecting one's understanding revealing the absurdity the clothing worn by those in the dojo inherently for their rationality (though arguments could be made about indirect advantages and/or disadvantages in both directions?).

From that, the clown suit could be taken as a result of him being humbled by this lesson, it making such a deep impression on him that, taking it to heart, he established something to remind him (and others?) of it as part of his daily behaviour.

(This would be that, rather than thinking a clown suit really could make him more rational, him wearing a clown suit when discussing rationality to dramatically demonstrate the lack of importance of what he wore for what he said. I'm also reminded of the concept of abnormality being going to school in a clown suit rather than in all black, though this is arguably not directly relevant.)

I am as yet unfamiliar with the architecture of this location, and so do not know if anyone will automatically know about it nor in fact if it will ever be read, but it nevertheless is a significant weight off my mind to speak about my impressions alongside those of others. I thank you all.

I am as yet unfamiliar with the architecture of this location, and so do not know if anyone will automatically know about it nor in fact if it will ever be read, but it nevertheless is a significant weight off my mind to speak about my impressions alongside those of others. I thank you all.

The "Recent Comments" section on the right hand side displays the five most recent comments. So even comments on old posts have a chance to catch the eye one who is browsing. Some even subscribe to all the comments using RSS,

A few of you touched on the point I got out of this, but no one explained it very well. In the first koan, Ougi says two things. The clearer one is tangential to rationality, but important for self-doubting cultists. "You are like a swordsman who keeps glancing away to see if anyone might be laughing at him".

The more important point was that the teachings are valuable if they are useful. (This is applicable to the sword fighter because allowing yourself to be distracted is an immediate danger.)

The importance of the parable about hammers doesn't relate to prices, but to usefulness. "Use the hammer to drive nails" in a discussion about rationality is metaphoric for using the techniques to make better decisions. If Ougi's teachings help you make better decisions in your life, then they are valuable. If they merely bind you more tightly to Ougi, then you are a cultist.

Bouzo didn't learn anything that helped him make decisions, he was merely cowed into following Ougi more closely. Ni no Tachi learned to "concentrate on a real-world question", so "the worth ... of his understanding [became] apparent."

Ni no Tachi figured out how to use the hammer, but Bouzo only sold them without understanding their value.

"Since you are so concerned about the interactions of clothing with probability theory," Ougi said, "it should not surprise you that you must wear a special hat to understand."

But isn't this almost the exact opposite of what the student was saying? Questioning the robes indicates to me that the student felt there was not any interaction between learning probability theory and clothing, and that therefore it served some other purpose, presumably differentiating between an in group and an out group.

Or am I just nuts for trying to argue with you about the internal thoughts of your own fictional characters?

"Why not?"

Because it confuses them into thinking that maybe the clothes have something to do with the meat of the instruction, and it takes longer for them to learn what they're being taught. It obscures the the real teaching, hides it among irrelevant rituals.

For anyone reading this at a future point there's more recent discussion from a sequence rerun here

A koan: If you have ice cream, I will give it to you. If you have no ice cream, I will take it away from you. It is an ice cream koan.

Caledonian, did you get your's from Stargate?

"I THINK that's what you're saying..."

I think it is saying that if you want to know if an idea is true or not, compare it to reality. Clothes are an irrelevancy. "If it can drive nails, who can doubt it's worth?"

Tiiba, do you mean to imply that if your group is cultish, the group I belong to is safe by the tu quoque principle? ;)

In all seriousness, I think the second point is a little more complicated than you described. Specifically, I think Mr. Yudkowsky is trying to point out that Bouzo had plenty of evidence more closely entangled with cult/non-cult status than whether or not they had uniforms. He had experienced Ougi's teachings directly. In the absence of that evidence, of course, the uniforms would be more important evidence.

But Mr. Yudkowski, why shouldn't we be concerned with such apparently superficial signs of cultishness? Shouldn't we -- especially the less experienced rationalists among us -- be on the lookout for objective clues to bias? After all, as has been pointed out already on Overcoming Bias, situations often look different from the inside, in ways that generate inaccuracies. (People tend to underestimate how biased their beliefs are. It's easier to show that someone's wearing uniform clothes than to show that they're wearing irrationally uniform beliefs.)

Snort. Got here while trying to figure out what, if any answer (the comic provides none) there is to "What kind of ice cream do you put on a Koan?" And here I find this... lol

Seriously, though, like most arguments presented by monk style philosophers, the answer given is flawed. Which is more valuable of the following?

1. A pure gold hammer with a mess of rhinestones in the handle. Estimated material cost - $200.

2. A real hammer, with a gold filigree handle and an idiotic mess of bangles hanging off of it, jewels strung onto them. Estimated material cost - $200.

3. A real hammer, simple black handle, estimated store cost - $10.

Well, if the point of the hammer is to build a house, you have a problem, because both 2 and 3 are options. Number 2 is silly as hell, regardless of whether you glued all the junk on it yourself, or you claim to have found it in the lost tomb of Jesus. It may still hammer nails, but no one in their right mind would use it, except out of necessity of there being no other hammers available. Option #1 is completely useless, gold being too soft to hammer much of anything, and again, it doesn't become a more valuable hammer if you claim that "it" was found in the lost tomb of the great Illuminati or some similar gibberish. A cult is a cult because it either contains nothing but gold hammers, or it insists that everything must be done using hammers that have lots of idiotic junk glued onto them, which don't help make the hammer any more useful. Someone intent on building a house can as easily use the second option as the third, but the moment you start thinking that you "have to" have all the silly BS glued to the hammer for it to work, it becomes a cult. Witness pretty much all religions, which nearly universally insist that its impossible to invent a hammer, without the guidance of high order of hammer makers, as to what sort of bangles and filigree one has to add to it before the magic spirits will make it work right. ;) Some are though tend to be satisfied with just the hammer, but insist the handle must be red, or something, to make it lucky (which might be the approach of the few relatively "rational" religions in the world), of which most are not.

In general, the more rational your starting framework, the least silly the necessary robes/clown suits are, but even the most rational groups, if they don't take a critical look at certain assumptions, can get cultish. The key difference being if they opt to make those choices for distinction, or because they have actually come to believe that some detail of their world won't work without the silly hat.

Oh, and anyone know the answer to the original question that got me here? lol

What's wrong with being in a cult, as long as you're reasonably self-aware about it? Cults are fun! ;)

Incidentally, Wikipedia lists itself in its list of alleged cults.

I had a friend in college who was a philosophy major; he'd been raised a fundamentalist Christian, and turned from that into some sort of chaotic evil deist. I used to enjoy arguing philosophy and ethics with him. But the further he got into his study, the more his arguments turned into half-understood quotations of Wittgenstein and Kant, and his debating technique turned into sophistries and trying to name and call out others' logical fallacies, sometimes correctly and sometimes not -- the same techniques he grew up with, only without having to wake up early on Sunday. I don't argue with him anymore...

"If I had a hammer that seemed to me to work really well, but no one was willing to pay me the going rate for hammers of that quality"

.. then by definition you have mis-estimated the going rate.

Did I mention I hate Zen-style teaching?

I presume you feel the same way about the Socratic method?

If clothes are unrelated to probability theory, why do the students have to wear robes?

Because we've read too many comics, manga, and sci-fi/fantasy books that romanticize a particular aesthetic. Robes are cool, silly. So is writing ideas in Japanese, makes the wisdom sound wiser, but what's especially important is that it's fun. Who wants to be part of a boring revolution?

Who wants to be part of a boring revolution?

(raises hand)

I only got into the rationalist game because I'm fond of hats. Oh, and Robin's advice on gift-giving.

I'll try to decipher the message.

If I am concerned that the group I belong to is becoming cultish, the thing to do is to ask what is a cult and what is not a cult, and see if the definition applies to your group. The second koan reminds us not to use extraneous details like uniforms. More non-cultists than cultists use uniforms. In general P(Category|Feature) != P(Feature|Category).

I THINK that's what you're saying...

Maybe... but here's the thing: Uniforms actually ARE a very cultish thing. They are one of the quantitative traits that can add up into driving you into the cult attractor. The proper rationalist response is actually "I will not wear the hat, because I don't want to and you've not given me a reason to."

The only sort of "uniform" that is rationally justifiable is something like body armor, or a hazmat suit, or a labcoat; sure, it's all the same, because it SERVES A PURPOSE---there's a reason soldiers wear Kevlar instead of tissue paper. But if you can't actually justify the uniform (like a nurse hat, or epaulettes, or the Pope's miter), then it really IS a bad sign that you are slipping into irrationality.

The line isn't nearly as crisp as you make it sound.

For example, is a nurse's uniform as "rationally justifiable" as a hazmat suit? No. But it does serve a useful purpose for nurses, in that it frequently makes patients more likely to treat them as authority figures.

Now, you might ask why patients do that, but in some sense that doesn't matter. Even if patients are irrational to do that, it is still pragmatically useful for nurses to wear the uniform if it reliably obtains that benefit.

But in fact it isn't a senseless thing for patients to do, either, in that wearing a nurse's uniform is a more costly signal if I'm not a nurse than simply saying "I'm a nurse" (since other nurses might see me wearing the uniform and punish me), and therefore more reliable than simply saying that.

More generally: uniforms are one way humans signal a certain kind of social status, and status signaling is a valuable function.

Why are almost all fire trucks red? They would work just as well if they were blue and yellow polka dots. But they are uniform because they are recognizable.The same with the blue-white-red lights on a police car and the sirens.

A nurse's uniform tells you that this is probably a nurse, even in contexts where the scrubs are not useful. A monk's or priest's robes tell you that this is a religious person who might give you religious advice. The act of picking a uniform for a group lets you begin to associate some properties of that group with the people in it, at a glance.

Maybe... but here's the thing: Uniforms actually ARE a very cultish thing.

I think Tiiba2 perfectly addressed this with

In general P(Category|Feature) != P(Feature|Category).

It helps to have read An Intuitive Explanation of Bayes' Theorem, but the short of it is this: The majority are not in cults. The minority are in cults. The minority of non-cults may wear uniforms, and the majority of cults may wear uniforms. The majority of the minority is not necessarily greater than the minority of the majority. So taking uniforms to mean cults is kind of intuitive, but not necessarily true.

If, for instance, your stamp collecting club decides they should all wear matching t-shirts at their meetings, it doesn't mean they've crossed the line into becoming a cult, just that they want to wear matching t-shirts.

Did I mention I hate Zen-style teaching?

Hate hate hate.

Unless you're going to tell me Godel, Escher, Bach consisted of Zen-style teaching, in which case I'll bring it down to two hates. (But it sure is pretty.)

Also, college: hate hate hate.

I don't think Hanson is saying economics is a cult, he is referring to either this or this.

The student asked, "Where is the true master of the Way?" The master replied, "Where is a true student of the Way?"

I would greatly appreciate it if you or anyone else would explain it.

You cannot understand the moon by deconstructing the finger.

Well, this is a bit better than normal.

If you can immediately recognize the candlelight as fire, the meal was cooked a long time ago.