I read recently an article on charitable giving which mentioned how people split up their money among many different charities to, as they put it, "maximize the effect", even though someone with this goal should donate everything to the single highest-utility charity. And this seems a bit like the example you cited where, if blue cards came up randomly 75% of the time and red cards came up 25% of the time, people would bet on blue 75% of the time even though the optimal strategy is blue 100%. All this seems to come from concepts like "Don't put all your eggs in one basket", which is a good general rule for things like investing but can easily break down.

I find myself having to fight this rule for a lot of things, and one of them is beliefs. If all of my opinions are Eliezer-ish, I feel like I'm "putting all my eggs in one basket", and I need to "diversify".You use book recommendations as a reductio, but I remember reading about half the books on your recommended reading list, thinking "Does reading everything off of one guy's reading list make me a follower?" and then thinking "Eh, as soon as he stops recommending such good books, I'll stop reading them."

The other thing is the Outside View summed up by the proverb "If two people think alike, one of them isn't thinking." In the majority of cases I observe where a person conforms to all of the beliefs held by a charismatic leader of a cohesive in-group, and keeps praising that leader's incredible insight, that person is a sheeple and that leader has a cult (see: religion, Objectivism, various political movements). I respect the Outside View enough that I have trouble replacing it with the Inside View that although I agree with Eliezer about nearly everything and am willing to say arbitrarily good things about him, I'm certainly not a cultist because I'm coming to my opinions based on Independent Logic and Reason. I don't know any way of solving this problem except the hard way.

"note: Hofstadter does not have a cult"

I tried to start a Hofstadter cult once. The first commandment was "Thou shalt follow the first commandment." The second commandment was "Thou shalt follow only those even-numbered commandments that do not exhort thee to follow themselves." I forget the other eight. Needless to say it didn't catch on.

PG runs a discussion site. He's using it as a sort of wide-flung net to catch worthy candidates for the "inner circle" - startup founders who get into his YC program - and is quite open about it (e.g. he explicitly says that YC submissions will among other things be judged on how well their authors are known as HC commenters and how worthy their comments have been judged to be). Why is it surprising that this creates a cult atmosphere of sorts?

Before Hacker News, PG was already famous in the relevant community for his essays, which are often credited, among other things, for the modern revival of interest in Lisp (this is probably an exaggeration). Nobody called him a cult leader back then.

Joel Spolsky is a famous blogger in the programming/CS/IT niche; he has an active discussion forum on his site. Lots of people respect him, lots of other people look down on his posts. Nobody calls him a cult leader.

RMS doesn't even have a discussion forum, and doesn't write a blog. He browses the web through an email-mediated wget; that's not even Web 1.0, it's Web -0.5 or something. He's widely considered to be a cult leader.

I'd guess that to make people think you're behaving like a cult leader, you need some or all of the following:

1. An ideological commitment that is seen as overriding most other priorities. Something that no matter what other things you're talking about, much of the time you're still really talking about that. Something that, from the perspective of someone not as committed as you are, you won't shut up about.

   Paul Graham won't shut up about startups and how they're the natural way of existing for a talented programmer or entrepreneur. Stallman won't shut up about free software and how you're ethically bound to call your OS GNU/Linux. You won't shut up about Topics that Won't Be Named and a few other things.

2. Actually being a leader or being thought a leader; having a real or widely imagined amount of influence. PG determines who gets into the very prestigious - in the relevant community - YC program. RMS controls GNU and has huge mindshare among free software enthusiasts. Within the admittedly smaller community at OB, you're seen as the most active blogger/proprietor, and the one most involved in its community formation. Unlike Hanson, who's opinionated but detached, you're opinionated and very attached. After lurking at OB for a year or so, I couldn't possibly tell who among the commenters are Hanson's friends, colleagues or fierce antagonists.

3. You need to be seen as molding the community, or your audience, to your liking - either by filtering the undesirables, or boosting the voices of the desirable. In other words, you need to be seen as growing "the cult", sometimes with active choices, sometimes simply by choosing the rhetoric or the content that'll repel the unfaithful.

   PG acts, actively and passively, to limit the total audience of even the outwardly inclusive HN. The theme of keeping HN 'small' so it doesn't deteriorate to the level of Reddit is reiterated by PG and widely shared by the audience. RMS is famous for his attempts to enforce ideological purity. You're explicitly engaged in conscious community-building, which you sometimes describe as leading to a new generation of rationalists which will embrace the Topics that Shall Not be Named. That is, you can be seen (I'm not saying that must necessarily be the case) as not merely hoping to draw an audience of people interested in rational thinking, but actually filtering that audience to a subset that substantially shares your commitment to the Cause.

4. This is an anti-property of being considered a cult leader: actively inviting and nurturing disagreement with yourself. In a blog format, that can work by explicitly encouraging dissent through various stylistic and content-based clues, by being especially mindful of dissenting voices in comments, etc. PG, as far as I could notice, never discouraged criticism and handled it superbly, so he possesses this anti-property (and is consequently much less of a cult leader than he could be otherwise). I hesitate to say I've never seen RMS change his opinion as a result of an argument - I guess this happened a few times on very technical issues - but it's a rare exception. You, while not discouraging criticism at all, are prone to ignore criticism (not mere trolling, but serious criticism) in comments and talk over it with people who mostly agree with you; you're also prone to present criticism against you as a result of a trendy choice to stand up to a perceived cult leader (this is a dangerous stance for oneself to adopt, even when true).

So... just for the record... this post got up to #1 on HN, and then HN crashed, and is, so far as I can tell, still down a couple of hours later.

When you consider that the Less Wrong site format was inspired by HN, that LW is based on Reddit source code, and that Reddit is a Y Combinator company, I guess that writing about Paul Graham and then getting voted up on Hacker News exceeded the maximum recursion depth of the blogosphere.

Picture of Eliezer in monk's robes (That is you, right?), stories about freemason-esque rituals, specific vocabulary with terms like, "the Bayesian conspiracy".

It's all tongue in cheek, and I enjoy it. But if you're trying to not look like a cult, then you're doing it wrong.

It seems to me that only a few groups get the label "cultish", so its not like people put the label on any group with an apparent leader. Such selective labels probably contain a lot of info, so it seems worth figuring out just what that info is. It is not wise to just find one group that gets the label which you think is fine, and then decide to ignore the label.

The straightforward approach would be to collect a dataset of groups, described by various characteristics, including how often folks call them "cultish." Then one would be in a position to figure out what this label actually indicates about a group.

This whole concept is confusing to me. I enjoy Eliezer's writing because it makes sense and is useful so it becomes part of my identity. I haven't found as many of his newer posts to be useful so a lower number of them are drafted into my identity. My 'self' is largely a collection of ideas and thoughts transmitted to me from other people and I don't find anything wrong with this. I do hope to produce useful knowledge myself but for right now I am educating myself to that point.

If I find a useful tool lying on the ground then I pick it up and use it, I do not try to recreate the tool from scratch in order to make it 'mine', which I feel is a meaningless concept. As long as my beliefs and skills pay for themselves in terms of useful benefits to my life I don't see the point in throwing them away because they came from someone else. I don't care who I am and I am not attached to any specific view of my self other than to try to pick the most effective tools to accomplish some core goals and values.

IAWYC but I think you forgot to include something about jealousy in your analysis, even if few people would admit it's part of it.

I think it's very possible to greatly admire someone and at the same time feel some form of jealousy that inhibits the clear expression of that admiration. By saying that someone else is better (much better) than you are - especially at something that you value - you are in effect admitting to a lower status.

So all the forced disagreements and claims of independence are in effect just trying to signal that your status is high and you're not submissive, or something like that.

I find myself moved to break possibly the greatest taboo amongst our kind, but if this act of status suicide moves just one reader to action, the sacrifice is worth it.

OK, here goes...

"Gödel, Escher, Bach" by Douglas R. Hofstadter is the most awesome book that I have ever read.

Me too!

When you read someones writings or follow the things they do but don't actually KNOW them, it's very easy to get sucked into a sort of 'larger-than-life' belief about them.

Because they're famous (and they must be famous because you've heard of them), they're obviously different and special and above regular, normal people. I've found it takes conscious effort to remember that no how famous or smart or talented they are, in the end they're just some guy or girl, with the same flaws as everyone else.

And when you think someone's larger-than-life, it's easy to praise highly, because you're not thinking of them as a normal person, you're thinking of them as ABOVE normal people. That they are special. In light of this, it's easy to see why praise for someone or something, no matter what it is, can be seen as cultish., and how you can fall into the trap of believing praise for anything is cultish.

For what it's worth I don't think you've deliberately set out to become a "cult leader" -- you seem like a sincere person who just happens to be going about life in a rather nonstandard fashion. You've got some issues with unacknowledged privilege and such, and I've gotten impressions from you of an undercritical attractance to power and people who have power, but that's hardly unique.

I think mostly it's that you confuse people via sending off a lot of signals they don't expect -- like they think you must have some weird ulterior motive for not having gone to college, and instead of seeing public discussion of your own intellect as merely the result of somewhat atypical social skills, it's seen as inexcusable arrogance.

That said, because of my own negative experience(s) with people who've seemed, shall I say, rather "sparkly" at first, but who HAVE turned out to be seeking puppy-dog supplicants (or worse), I tend to be very very cautious these days when I encounter someone who seems to attract a fan club.

With you I've gone back and forth in my head many times as to whether you are what you first struck me as (a sincere, if a bit arrogant, highly ambitious guy) or something more sinister. It's been difficult to tell as you're sort of surrounded by this buzzing cloud of subcultural interference, but at this point I've sort of determined that if there's anything sinister there it's not a special sort above and beyond what you'd find in any given random middle/upper class American geek.

I think you get called out as a symbol of "smartypants white boys obsessed with trying to save the world from their basements" because you've ended up more visible than most. But, no, that doesn't make you a cult leader, it just makes you someone who would (like many of us living in wealthy, industrialized nations) benefit from making a greater effort to understand the effects of power and privilege.

IAWY, and I actually already replied to your question about this in a comment, but:

One of the prime issues for me as a rationalist trying to learn about marketing (especially direct/internet marketing) was having to get over the fear of being a "dupe" pulled into a "scam" and "cult" situation. Essentially, if you have learned that some group you scorn (e.g. "suckers" or "fools" or whatever you call them) exhibit joining behavior, then you will compulsively avoid that behavior yourself.

I got over it, of course, but you have to actually be self-aware enough to realize that you chose this attitude/behavior for yourself... although it usually happens at a young enough age and under stressful enough conditions that you weren't thinking very clearly at the time.

But once you've examined the actual evidence used, it's possible to let go of the judgments involved, and then the feelings go away.

In other words, persons who have this issue (like me, before) have had one or more negative social experiences linking these behaviors to a disidentified group -- a group the person views negatively and doesn't want to be a part of. It's a powerfully irrational, compulsive motivator.

Hell, I had the same issue about exercise: I didn't want to be one of those shallow, vain jerks that likes to exercise!

It doesn't matter what the group is or what the behavior is, your brain picks up on the behaviors and attributes that signal participation in the groups you're around. And if you've decided you don't like the group... well, there you go.

Getting rid of it, however, is a matter of consciously re-evaluating the evidence and dropping your grudge against the target group...

Which means you're never going to talk people out of it directly -- trying to do so just makes people raise shields... "you're trying to get me to be one of THEM, aren't you?"

People are full of irrational compulsions that follow this pattern... and you're probably not immune, as pointed out by your Tolerating Tolerance article. Most likely, the reason you have to fight yourself to tolerate the tolerators of fools, is because of your own intolerance regarding fools. If you let go of your emotional grudge(s) against fools (whatever those grudges might be), you'd find it to be a lot less of a problem.

(Then, you'd also be in a lot better position to ask others to give up their grudges... including the ones you wrote this post about.)

I agree with your conclusion, and I love your library allegory. It's pretty clear that America fears strong emotions in general, and also that "our type" learns cached patterns of ritually approved of nonconformity.

That said, some may be balking, not at admiring someone hugely, but at forming nearly their entire manner of evaluating ideas from a single person, without independent sources of evidence that can label that person "trustworthy". Anne Corwin reports fearing networks of abstractions that distance people from their own concrete experiences and root-level knowledge. Carl's recent post is in part an attempt to say that many of the ideas you've expressed are supported by other, labeled-as-trustworthy sources, and folks who believe those ideas needn't feel as though they're putting all their error-catching eggs in one basket.

Other people may just fear changing their beliefs and actions, especially changing those beliefs and actions in a manner affected by an unknown person's intent. When I ran into some pamphlets on critical thinking seven years ago, and I heard arguments that I should (gasp) only believe claims supported by evidence... the prospect of only believing what was supported by evidence felt like a frightening loss of control. "It's a cult! It could take over my mind!" may just express fear at shifting which mental system (the "your choice" system, or this scary foreign "rationality/Eliezer" system) will control one's future beliefs and actions.

These fears may also apply to peoples' relationships to Paul Graham, I'm not sure.

Figured, since this was linked to again, that I might as well say some of what I think on this:

My reaction is more, well, a couple of things, but part of it could be described like this: Yes, I do indeed admire you and think you're cool... and my natural instinctive reaction to you is kinda, well, fanboyish, I guess. Hence I try to moderate that... TO AVOID BEING ANNOYING... that is, to avoid, say, annoying you, for instance.

If you have 'teachings' rather than suggestions or opinions, and you can't support those claims in a systematic and explicit way, then it doesn't much matter whether you intended to propagate a cult - that's precisely what you're doing.

Alright! a few points that I can sort of disagree on or feel were omitted in the essay. I'm being skeptical, not a cultist at all! .

My fears aren't really that you're trying to foster a cult, or that it's cultish to agree with you. I got worried when you said that you wanted more people to vocalize their agreement with you and actually work towards having a unified rationalist front. For some reason, I had this mental picture of you as a supervillain declaring your intention to take over the world. So I reflected that I was doing things, somewhat unconventional things (which I focus on more) because of your advice, but hey, it's good advise and I should probably take it (btw it's good to hear that cryonics is less expensive than I thought it was, sorry for making your life difficult by propagating false information). I mean, I followed similar patterns when I decided to learn lisp as a first programming language.

I think I'm worried because you're charismatic, and that makes you much more persuasive than an ineloquent and unimpressive philosopher/AI Hacker. Combined with the fact that I get really happy and a little self righteous when there's an eloquent speaker who makes a really persuasive argument for something I agree with, makes reading you, and other charismatic people in the atheist/revolutionary/technophile cluster, a rather deep experience with uncomfortable parallels to religion.

I've thought it over though, and this particular pattern probably won't cause too many problems, the reason is that Eliezer Yudkowsky isn't the only eloquent speaker in the world. I'm betting on something similar to the "three stooges syndrome" where I get shaped by too many intelligent and charismatic people to be influenced in to making large mistakes, because they'll probably call each other out on the more contentious claims and my bullshit detector will be reactivated.

I'm not even sure if I even agree with you more than average, but it does feel better to agree with you than usual, so that might be the source of worry, in your trip to the library of convenient rhetorical metaphors, it might be that the reason people are so anxious to say that they're not copying you is because your deep, piercing stare and badass coffee metaphors

So other than that you've totally persuaded my fear of your ability to totally persuade me away. Well it'll probably gone in a week after my subconscious stops thinking you're a super-villain.

"But upon reflection, I strongly suspect that I would feel no barrier to praising Gödel, Escher, Bach even if I weren't doing anything much interesting with my life."

You don't feel yourself to be in status competition with Hofstadter do you? Or E.T. Jaynes, for that matter. Think about effusively praising Nick Bostrom as the last best hope for the survival of humane values, instead.

"I'm hoping in particular that someone used to feel this way - shutting down an impulse to praise someone else highly, or feeling that it was cultish to praise someone else highly - and then had some kind of epiphany after which it felt, not allowed, but rather, perfectly normal."

I've always disliked generic praise, of the form 'X is great,' but find it easy enough to talk about objective standing within a population (e.g. 'in the Y percentile for math capabilities'), including the population of people I know (top Z% of my close acquaintances for social skills). The first leaves a bad taste of uncriticality, but I'm fine with saying that someone is probably even one in a million or one in a hundred million with respect to some formula aggregating across several features. I can't recall an epiphany in which I radically changed in this regard.

IMO being accused of wanting to be a cult leader is a pure double bind. You either say "yes, I do" and then you're a cult leader, or you say "what? that's crazy because of X, Y, Z..." and then people point at your protestations as evidence that their arguments have some minimal credibility (I am sure someone will do this to EY at some point). It is, prima facie, evident to me that talking to people on the internet about rationality is a poor method of getting acolytes (and even if it were a good one for some people, the Objectivists already picked up those guys) so if you are, in fact, a prospective cult leader, Eliezer, it seems to me you've had a serious failure of rationality and no-one should join. Everyone else, please note that this is exactly what I would say if I were trying to recruit you into my own, competing cult. Including the previous sentence.

In answer to the end of your post, even though this site is very young, I can already think of occasions where I've hesitated to speak up on your behalf because I anticipated being branded a sycophant. I've now decided that I would rather let otherwise interesting people exclude themselves from LW by adopting these "it's a cult!" beliefs than attempt to persuade them to stay or join in those circumstances.

If there's an aspect of the whole thing that annoys me, it's that it's hard to get that innocence back, once you even start thinking about whether you're independent of someone.

Cross-referencing my comment on a different post for a related idea:

Your brain remembers which "simple" predictor best described your decision [. . .]

Your brain learns to predict other peoples' judgments by learning which systems of predictive categories other people count as "natural". If you have to predict other peoples' judgments a lot, your brain starts to count their predictive categories as "natural". The effect can be viral [. . .] and it can change how you think about yourself.

I'm afraid to read GEB now. It's been built up so high the only possible reactions I could possibly have are "as good as everybody else thinks it is", or "didn't live up to expectations", with the latter being far more likely.

Ok, I'm coming out and will admit that I admire you, Eliezer very highly. I think you are the one who taught me the most about rationality and what intelligence is all about.

Now, I admit that in my past I have fallen into the "adore the guru" trap so I still have this fear in my head and am cautious to not do the same mistake again. The cult-threads here are helping me to evaluate my position carefully.

But I like what you wrote about that innocence of being able to experience real admiration and excitement. I think if you let your critical thinking stifle that you are losing some of the important driving forces in your life. We should be happy if we experience those strong positive emotions that pull us in the right direction, if they are justified by reality:

http://www.overcomingbias.com/2007/04/feeling_rationa.html