Followup to: Excuse Me, Would You Like to Take a Survey?, Return of the Survey

Thank you to everyone who took the Less Wrong survey. I've calculated some results out on SPSS, and I've uploaded the data for anyone who wants it. I removed twelve people who wanted to remain private, removed a few people's karma upon request, and re-sorted the results so you can't figure out that the first person on the spreadsheet was the first person to post "I took it" on the comments thread and so on. Warning: you will probably not get exactly the same results as me, because a lot of people gave poor, barely comprehensible write in answers, which I tried to round off to the nearest bin.

Download the spreadsheet (right now it's in .xls format)

I am not a statistician, although I occasionally have to use statistics for various things, and I will gladly accept corrections for anything I've done wrong. Any Bayesian purists may wish to avert their eyes, as the whole analysis is frequentist. What can I say? I get SPSS software and training free and I don't like rejecting free stuff. The write-up below is missing answers to a few questions that I couldn't figure out how to analyze properly; anyone who cares about them enough can look at the raw data and try it themselves. Results under the cut.

Out of 166 respondees:

160 (96.4%) were male, 5 (3%) were female, and one chose not to reveal their gender.

The mean age was 27.16, the median was 25, and the SD was 7.68. The youngest person was 16, and the oldest was 60. Quartiles were <22, 22-25, 25-30, and >30.

Of the 158 of us who disclosed our race, 148 were white (93.6%), 6 were Asian, 1 was Black, 2 were Hispanic, and one cast a write-in vote for Middle Eastern. Judging by the number who put "Hinduism" as their family religion, most of those Asians seem to be Indians.

Of the 165 of us who gave readable relationship information, 55 (33.3%) are single and looking, 40 (24.2%) are single but not looking, 40 (24.2%) are in a relationship, 29 (17.6%) are married, and 1 is divorced.

Only 138 gave readable political information (those of you who refused to identify with any party and instead sent me manifestos, thank you for enlightening me, but I was unfortunately unable to do statistics on them). We have 62 (45%) libertarians, 53 (38.4%) liberals, 17 (12.3%) socialists, 6 (4.3%) conservatives, and not one person willing to own up to being a commie.

Of the 164 people who gave readable religious information, 134 (81.7%) were atheists and not spiritual; 5 other atheists described themselves as "spiritual". Counting deists and pantheists, we had 11 believers in a supreme being (6.7%), of whom 2 were deist/pantheist, 2 were lukewarm theists, and 6 were committed theists. 14 of us (8.5%) were agnostic.

53 of us were raised in families of "about average religiousity" (31.9%). 24 (14.5%) were from extremely religious families, 45 (27.1%) from nonreligious families, and 9 (5.4%) from explicitly atheist families. 30 (18.1%) were from families less religious than average. The remainder wrote in some hard to categorize responses, like an atheist father and religious mother, or vice versa.

Of the 106 of us who listed our family's religious background, 92 (87%) were Christian. Of the Christians, 29 (31.5% of Christians) described their backgrounds as Catholic, 30 (32.6% of Christians) described it as Protestant, and the rest gave various hard-to-classify denominations or simply described themselves as "Christian". There were also 9 Jews, 3 Hindus, 1 Muslim, and one New Ager.

I didn't run the "how much of Overcoming Bias have you read" question so well, and people ended up responding things like "Oh, most of it", which are again hard to average. After interpreting things extremely liberally and unscientifically ("most" was estimated as 75%, "a bit" was estimated at 25%, et cetera) I got that the average LWer has read about half of OB, with a slight tendency to read more of Eliezer's posts than Robin's.

Average time in the OB/LW community was 13.6 ± 9.2 months. Average time spent on the site per day was 30.7 ± 30.4 minutes.

IQs (warning: self-reported numbers for notoriously hard-to-measure statistic) ranged from 120 to 180. The mean was 145.88, median was 141.50, and SD was 14.02. Quartiles were <133, 133-141.5, 141.5-155, and >155.

77 people were willing to go out on a limb and guess whether their IQ would be above the median or not. The mean confidence level was 54.4, and the median confidence level was 55 - which shows a remarkable lack of self-promoting bias. The quartiles were <40, 40-55, 55-70, >70. There was a .453 correlation between this number and actual IQ. This number was significant at the <.001 level.

Probability of Many Worlds being more or less correct (given as mean, median, SD; all probabilities in percentage format): 55.65, 65, 32.9.

Probability of aliens in the observable Universe: 70.3, 90, 35.7.

Probability of aliens in our galaxy: 40.9, 35, 38.5. Notice the huge standard deviations here; the alien questions were remarkable both for the high number of people who put answers above 99.9, and the high number of people who put answers below 0.1. My guess: people who read about The Great Filter versus those who didn't.

Probability of some revealed religion being true: 3.8, 0, 12.6.

Probability of some Creator God: 4.2, 0, 14.6.

Probability of something supernatural existing: 4.1, 0, 12.8.

Probability of an average person cryonically frozen today being successfully revived: 22.3, 10, 26.2.

Probability of anti-agathic drugs allowing the current generation to live beyond 1000: 29.2, 20, 30.8.

Probability that we live in a simulation: 16.9, 5, 23.7.

Probability of anthropic global warming: 69.4, 80, 27.8.

Probability that we make it to 2100 without a catastrophe killing >90% of us: 73.1, 80, 24.6.

When asked to determine a year in which the Singularity might take place, the mean guess was 9,899 AD, but this is only because one person insisted on putting 100,000 AD. The median might be a better measure in this case; it was mid-2067.

Thomas Edison patented the lightbulb in 1880. I've never before been a firm believer in the wisdom of crowds, but it really came through in this case. Even though this was clearly not an easy question and many people got really far-off answers, the mean was 1879.3 and the median was 1880. The standard deviation was 36.1. Person who put "2172", you probably thought you were screwing up the results, but in fact you managed to counterbalance the other person who put "1700", allowing the mean to revert back to within one year of the correct value :P

The average person was 26.77% sure they got within 5 years of the correct answer on the lightbulb question. 30% of people did get within 5 years. I'm not sure how much to trust the result, because several people put the exact correct year down and gave it 100% confidence. Either they were really paying attention in history class, or they checked Wikipedia. There was a high correlation between high levels of confidence on the question and actually getting the question right, significant at the <.001 level.

I ran some correlations between different things, but they're nothing very interesting. I'm listing the ones that are significant at the <.05 level, but keep in mind that since I just tried correlating everything with everything else, there are a couple hundred correlations and it's absolutely plausible that many things would achieve that significance level by pure chance.

How long you've been in the community obviously correlates very closely with how much of Robin and Eliezer's posts you've read (and both correlate with each other).

People who have read more of Robin and Eliezer's posts have higher karma. People who spend more time per day on Less Wrong have higher karma (with very strong significance, at the <.001 level.)

People who have been in the community a long time and read many of EY and RH's posts are more likely to believe in Many Worlds and Cryonics, two unusual topics that were addressed particularly well on Overcoming Bias. That suggests if you're a new person who doesn't currently believe in those two ideas, and they're important to you, you might want to go back and find the OB sequences about them (here's Many Worlds, and here's some cryonics). There were no similar effects on things like belief in God or belief in aliens.

Older people were less likely to spend a lot of time on the site, less likely to believe in Many Worlds, less likely to believe in global warming, and more likely to believe in aliens.

Everything in the God/revealed religion/supernatural cluster correlated pretty well with each other. Belief in cryonics correlated pretty well with belief in anti-agathics.

Here is an anomalous finding I didn't expect: the higher a probability you assign to the truth of revealed religion, the less confident you are that your IQ is above average (even though no correlation between this religious belief and IQ was actually found). Significance is at the .025 level. I have two theories on this: first, that we've been telling religious people they're stupid for so long that it's finally starting to sink in :) Second, that most people here are not religious, and so the people who put a "high" probability for revealed religion may be assigning it 5% or 10%, not because they believe it but because they're just underconfident people who maybe overadjust for their biases a little much. This same underconfidence leads them to underestimate the possibility that their IQ is above average.

The higher probability you assign to the existence of aliens in the universe, the more likely you are to think we'll survive until 2100 (p=.002). There is no similar correlation for aliens in the galaxy. I credit the Great Filter article for this one too - if no other species exist, it could mean something killed them off.

And, uh, the higher probability you assign to the existence of aliens in the galaxy (but not in the universe) the more likely you are (at a .05 sig) to think global warming is man-made. I have no explanation for this one. Probably one of those coincidences.

Moving on - of the 102 people who cared about the ending to 3 Worlds Collide, 68 (66.6%) prefered to see the humans blow up Huygens, while 34 (33.3%) thought we'd be better off cooperating with the aliens and eating delicious babies.

Of the 114 people who had opinions about the Singularity, 85 (74.6%) go with Eliezer's version, and 29 (25.4%) go with Robin's.

If you're playing another Less Wronger in the Prisoner's Dilemma, you should know that of the 133 who provided valid information for this question, 96 (72.2%) would cooperate and 37 (27.8%) would defect. The numbers switch when one player becomes an evil paper-clip loving robot; out of 126 willing to play the "true" Prisoner's Dilemma, only 42% cooperate and 58% defect.

Of the 124 of us willing to play the Counterfactual Mugging, 53 (42.7%) would give Omega the money, and 71 (57.3%) would laugh in his face.

Of the 146 of us who had an opinion on aid to Africa, 24 (16.4%) thought it was almost always a good thing, 42 (27.8%) thought it was almost always a bad thing, and 80 (54.8%) took a middle-of-the-road approach and said it could be good, but only in a few cases where it was done right.

Of 128 of us who wanted to talk about our moral theories, 94 (73.4%) were consequentialists, about evenly split between garden-variety or Eliezer-variety (many complained they didn't know what Eliezer's interpretation was, or what the generic interpretation was, or that all they knew was that they were consequentialists). 15 (9%) said with more or fewer disclaimers that they were basically deontologists, and 5 (3.9%) wrote-in virtue ethics, and objected to their beliefs being left out (sorry!). 14 people (10.9%) didn't believe in morality.

Despite the seemingly overwhelming support for cryonics any time someone mentions it, only three of us are actually signed up! Of the 161 of us who admitted we weren't, 11 (6.8%) just never thought about it, 99 (59.6%) are still considering it, and 51 (31.7%) have decided against it.

Comments

sorted by
magical algorithm
Highlighting new comments since Today at 4:59 PM
Select new highlight date
All comments loaded

Awesome work.

One thing that disappointed, but didn't really surprise me, was the lack of diversity in the community

"160 (96.4%) were male, 5 (3%) were female, and one chose not to reveal their gender.

The mean age was 27.16, the median was 25, and the SD was 7.68. The youngest person was 16, and the oldest was 60. Quartiles were 30.

Of the 158 of us who disclosed our race, 148 were white (93.6%), 6 were Asian, 1 was Black, 2 were Hispanic, and one cast a write-in vote for Middle Eastern. Judging by the number who put "Hinduism" as their family religion, most of those Asians seem to be Indians."

The thing that particularly worries me is our low age. Now it's to be expected as internet communities are a young person's game but I'd be more comfortable with an average age closer to 30.

Combine that with the fact that most of us seem to be in Computers or Engineering (I'd really like to know what those "Other Hard Sciences" were) I do worry about our rationality as a group. One thing I've noticed with junk science is that Engineers and to a lesser extent Computer Scientists seem to be overrepresented. I'm not sure of all the reasons for this, I suspect that part of the problem is that we regularly work with designed systems that have a master plan that can be derived from a small amount of evidence. The problem being if you take that tendency to problem spaces that aren't designed you have a tendency to go flying off in the wrong direction.

I'm worried that we could start turning into an echo-chamber where a localized consensus masks a growing dissonance with the outside world. The Shangri-la diet sounds interesting (I'm even giving it a try) but it also sounds a bit like pseudo-science. There could be a completely different mechanism at work, it could even be the good old fashioned placebo effect. I worry that we'll develop a tendency to believe our rationality is strong enough to wade outside of our fields of expertise, the halls of kookdom are filled with brilliant scientists who wandered into a neighbouring discipline and I worry we could risk the same fate.

I'm not saying Less Wrong is a doomed cause or anything, the topics we explore (oh that crazy old Omega!) we seem to do fairly well on and I've picked up many useful lessons and insights. I just worry since we all want to apply our rationality and find answers, but regardless of how rational you are you can't unravel the secrets of the universe just from analysing a piece of cake.

ps Oh yeah, how many of us 83.4% Libertarians/Liberals were very torn because while we really liked the free-market and social liberty ideals of libertarians there were just too many crackpots over there so we considered giving up some economic freedom for the mainstream democrats.

I suspect that part of the problem is that we regularly work with designed systems that have a master plan that can be derived from a small amount of evidence.

I've been playing alot of portal and half life 2 lately. (first person shooters with heavy puzzle elements) and I wonder about how the level design is affecting my thought process.

I'm often in a room with a prominent exit and it is clear that that is the exit I'm supposed to take. When the way I came in is blocked I know that there is some other way to get out. When my computer controlled squad mates parrot 'which way do we go?' I think to myself 'What do you mean? It's obvious the level designer wants us to go this way."

I wonder if this will affect how I deal with real world puzzles where there are many paths that don't lead to defined goals, but also don't lead to a clear dead end.

You could play procedural games like nethack or Dwarf Fortress, which have no railroading, and some things you encounter just can't be solved.

Those kind of games aren't as popular as "mainstream" ones like Portal, but they may better reflect the real world.

I'm surprised to see how close I was to the mean in so many cases. I expected on several questions that I would be, if not an outlier, then outside the middle quartiles. I was wrong in most cases. Clearly the OB/LW brainwashing process has been more successful than I realised... :P

Seriously, very interesting results. I'm a bit dismayed by the 3% female figure -- I knew I was in a minority, but I didn't realise it was that tiny. I wish I could articulate some suggestions for getting hold of more female readers/commenters. I can sort of see intuitively how this place could seem like not the most attractive one to some women, but I don't have any ideas for sorting that out. Largely I guess it may just be a self-perpetuating thing. Perhaps the first step ought to be just getting some of the current female readers/commenters to make (more/some) top-level posts too. I wish I felt brave and knowledgeable and intelligent enough to attempt one touching on some aspect or other of feminism.

I'm perplexed by the person who believes that the most severe existential risk is an asteroid strike, yet gave only 70% odds of surviving the century.

Hopefully it's not because they're an astronomer...

Maybe a human-caused asteroid strike?

Or maybe this person can think of many different extinction scenarios which are individually very improbable, but add up to a 30% chance?

The main thing I take away from this survey is that many of us still think they can assign 99%+ probabilities based on no good info (the aliens and existential risk questions stand out in particular). Maybe the LW community needs to focus more on the basics?

I voted for a high probability of life occurring at least once elsewhere in the universe due to a combination of the very large size of the universe and the generalized Copernican principle.

Two things surprise me.

First while 73.4% of responders are consequentialists and only 9% deontologists, at the same time 45% of responders are libertarians. While labels like that are vague, libertarianism is in most versions highly deontologist ideology, and cares about processes and not results as such.

The other thing was 33.3% of "single and looking" (plus 24.2% of "single and not looking" consists of some mix of "single and not interested" and "single, tried but given up"). There are some well known seduction techniques based on adjusting for biases people have because we're evolutionarily adapted to a different environment. I'd guess most responders compartmentalize their rationality and do not apply rational thinking to their personal lifes. More compartment-busting posts please?

While labels like that are vague, libertarianism is in most versions highly deontologist ideology, and cares about processes and not results as such.

The libertarian writing I've seen is primarily consequentialist: arguments, mostly by economists, that governments produce worse results than people directing their own efforts for their own reasons. So I see no contradiction in the survey responses.

If, in the places I don't read, which are surely more numerous than those I do, libertarianism is mainly promoted by arguments that government is morally wrong, then I can see why the Libertarian party has never been more than a splinter movement.

The argument I've seen is mostly something like:

  • People have rights to absolute personal liberty in economic matters (pure deontology).
  • In the fairytale case of perfectly competitive markets with no externalities, no transaction costs, no information asymmetry, no cost of entry, and so on and so on, government intervention is inefficient.
  • Somehow based on the fairytale case, following libertarian process is bound to produce the best possible results, without any need for serious empirical evidence that this link is true.
  • In cases where following the rules doesn't seem to produce best results, we're supposed to follow the rules anyway, as breaking them would most likely result in something even worse, even if we don't immediately see it.

This sort of post-hoc consequentalization of essentially deontological ethics is extremely common. I don't know that many deontologies that have balls to avoid this trick, and assert that they don't care about consequences.

Ask a typical Christian, or other theist, and just like a libertarian, they will tell you something like that:

  • sinning is wrong (deontological basis)
  • sinning results in bad consequences (unsupported assertion)
  • even when sinning seems to result in right consequences, they're bad anyway we just don't see it (post-hoc consequentialization)

On the other hand the modern European politics (there's not terribly much difference between "left" and "right") mixes market-based, government-based, and other solutions, based on what is estimated to work best, not on any big ideology, which it lost long time ago, even though it clings to all kinds of labels like "social democratic" or "christian democratic" etc.

I don't know where you've been finding this argument but it's hardly representative of a good argument for libertarianism. I grew up in Europe (well, the UK, which is kind of Europe) with Labour voting parents and grandparents with fairly socialist views and considered myself a socialist into my early 20s. Weak arguments like these wouldn't have been enough to convert me to a generally libertarian worldview.

I had a similar caricature of the views of supporters of the free market (back when I didn't even know the term libertarian) but learning more about economics and being confronted with evidence of better outcomes in freer economies, together with learning that few serious economists (or libertarians) believe in perfectly efficient markets and learning about Public Choice Theory were key in changing my political views.

Key to the economic arguments for libertarianism is the idea that incentives matter and that the incentives facing actors in a free market tend to be far less perverse than those facing politicians or employees of state run monopolies.

The moral arguments stem largely from a view that personal freedom is a high moral value and that the evidentiary bar should be set very high for any demonstration of harm to justify restriction of individual freedoms. That tendency seems to be correlated with certain personality types according to some research and the crossover between libertarians and progressives/liberals on social issues seems to be as much a factor of personal values as of consequentialist reasoning.

And being fairly familiar with UK politics (less so with European politics in other countries) the idea that European politics pick policies based on 'what is estimated to work best' strikes me as pretty laughable.

First while 73.4% of responders are consequentialists and only 9% deontologists, at the same time 45% of responders are libertarians. While labels like that are vague, libertarianism is in most versions highly deontologist ideology, and cares about processes and not results as such.

The most straightforward types of both libertarianism and utilitarianism take the form of systems that can be logically built from a base of few, powerful, elegant axiomatic principles. This type of system appeals deeply to mathematicians and engineers, hence both the large intersection and the high representation here.

Person who put "2172", you probably thought you were screwing up the results, but in fact you managed to counterbalance the other person who put "1700", allowing the mean to revert back to within one year of the correct value :P

Not to worry - I am a believer in the wisdom of crowds, so I knew full well that I wasn't going to be screwing up anything. That response was pure noise.

I just don't like guessing, and so I put "0%" for my confidence on that question, so that one of my answers was definitely wrong and the other was definitely right.

I believe in the wisdom of crowds, but I also think that your actions were screwing up the results.

If you weren't going to take a question seriously, I wish you wouldn't have answered it at all.

ADDED: I decided not to downvote you because I don't want to discourage being honest/forthcoming.

Why the heck is the average stated probability of a creator god greater than the average stated probability of something supernatural existing? Or did a team of scientists in a parallel universe count as a creator god?

I think the supernatural question was phrased as something supernatural happening within the universe. So in a deist perspective, if God created the universe and then went away, that would qualify for creator god but not supernatural.

IQs (warning: self-reported numbers for notoriously hard-to-measure statistic)

Yeah, I'm extremely skeptical of the IQ data. Assuming a standard mean=100 SD=15 test (although at least one respondent says he took a test with SD=24), our reputed median is above the 0.003th percentile. I don't think any public blog is that elite.

ERRATUM: Oh, dear. I meant 99.7th percentile.

I'm skeptical of the IQ data because of the number of IQs above 140. Most IQ tests don't measure well above IQ 140, and so even if we have that many truly exceptional people, I would not expect it to show up in their measured IQs.

If we were to assume a test with a standard deviation of 24, a median of 141.5 would be just below the 96th percentile. That still seems too high for the median user, but it's almost plausible -- much more so than 99.7th percentile.

It's also quite likely that LW readers with abnormally high IQs (relative to LW) are (A) much more likely to have been tested and to know (and remember) the result, and (B) include the score on the survey.

It doesn't strike me as all that implausible, given how many other indicators of quirkiness we have as a group (e.g., the 95-97% male, the 12% with PhDs (and 23% of members over 35 with PhDs), the portion with advanced math/compsci skill, etc.).