They would study logic, probability theory, argument, scientific method, and other core tools of truth-seeking. They would inquire into epistemology, the study of knowing. They would study artificial intelligence to learn the algorithms, the math, the laws of how an ideal agent would acquire true beliefs. They would study modern psychology and neuroscience to learn how their brain acquires beliefs, and how those processes depart from ideal truth-seeking processes. And they would study how to minimize their thinking errors.

Not necessarily. Hindsight bias is likely at work here. You know that studying these fields helped you to acquire better beliefs, and so you conclude that this consequence should be obvious. But unless a curious but untrained reasoner somehow finds out that studying these fields will help them, we shouldn't expect them to study them. Why on earth would someone try to read The Logic of Science if they didn't already know that it would improve their reasoning skills?

There are a lot more genuinely curious people out there than there are rationalists. But unless those curious people happen to meet a LWer, or stumble across a link to this site, their chances of learning the benefits of studying these subjects are not great. There are a few books that might get them started (Robyn Dawes is coming to mind), but how likely is it that they're going to stumble across one of those books, especially if they aren't explicitly interested in the field of human thought already (like EY?).

I would bet that there are a lot of genuinely curious people out there who have realized that thinking is a skill. But if you were to ask them for the best way they knew of to improve that skill, would say something along the lines of "sudoku puzzles". And that's pretty sad.

I'd handle shame-flavored incentives with tongs. It's plausible that I have an unusual degree of sensitivity on the subject, but I'm making progress on a very bad case of self-hatred and akrasia, and "is my curiosity good enough?" strikes me as a sort of self-alienation which takes focus away from paying attention to whatever you might be curious about.

"What might I be missing about this?", "How can I increase my enthusiasm for learning?", "How can I spend less time on errors while still taking on difficult projects?" seem much safer. "What am I doing to improve my life? Is it having the desired effect?" should probably be on the list.

I'm having difficulty knowing what level of rationalist this is aimed at. Are the people you talk to every week students of rationality, or 'normal' people?

This post applies to both, I imagine. But because you talk about "people" instead of explicitly talking about people like me, it's easy to see this post as not being aimed at me. (Maybe it's not).

What I mean is: It's easy to praise oneself and one's peers by talking about people of a lower class. When I was young, it was 'dumb people', when I was a bit more sophisticated it was 'theists', when I was an Objectivist, it was 'non-Objectivists', and now that I'm a rationalist the temptation is to criticize those who "know almost nothing of logic, probability theory, argument, scientific method, epistemology, artificial intelligence, human cognitive science, or debiasing techniques." So this post, because it isn't clearly directed at people who have worked hard to do better in the ways prescribed by the Sequences, causes my semiconscious mind to ask: "is this a beginning level post, or something I should actually pay attention to?" Are you telling me to do better, or criticizing outsiders in order to promote group bonding?

Of course my rider knows I should pay attention; I must always work harder to cultivate the virtues. And I don't actually expect that you're just trying to promote group bonding. But what I really want - and what this post may not have been designed for - is a honest criticism of people you see who are a lot like me in that they have access to all the same correct memes as I have, and they have exerted effort to improve in the appropriate areas.

Also, they would seek to personally become an immortal super-intelligence, since many truths simply can't be learned by an unenhanced human, and certainly not within a human lifetime.

(Which is why the Yudkowsky-Armstrong Fun-Theoretic Utopia leaves me cold. Would any curious person not choose to become superintelligent and "have direct philosophical conversations with the Machines" if the only alternative is to essentially play the post-Singularity equivalent of the World of Warcraft?)

Dare I say it? Few people look like they really want true beliefs.

I think otherwise - most people want to have true beliefs. However, they have rather limited trust in the powers of their own logic, as the experience of school has taught them that they are often wrong. They don't have the numerical skills to embark on anything more numerically ambitious than what money requires. They expect to be wrong often, and rarely use formal reason as such. But they still want to have true beliefs, and rely mostly on intuition and experience to decide on that.

For most people, most beliefs are socially acquired - people acquire their beliefs from the people around them, and they tend to acquire large blocks of belief together. One shouldn't underestimate the sheer amount of work needed to do anything different.

Most people never create a new idea (in the sense you're talking about) in their entire lives - they have experiences, yes, and they change beliefs based on experience. But they do not regard themselves as having the basic equipment to generate ideas, or to be sophisticated in judging between them.

In the end I've come to the view that none of us can change this (well, not anytime soon at any rate). Human beings think in groups - and the best most of us can do to help others think better is to do it for them, and talk about it sometimes. Obviously there is a group of people who can do more than that, but they are a minority.

The other comment I have on your post is that all of the above is actually just one idea. Which is that the basis of all knowledge about the world is inductive reasoning, and, in principle, all such reasoning should be based on the sound use of statistics. There are many mathematical ways of messing up such use, and our intuitive reasoning also messes up these stats too. If you really need the right answer, you will need to learn enough to get your statistics right, and compensate for the shortcomings of your wetware. And that's the whole idea in a nutshell.

An example. Did you know that brakes are the most dangerous piece of equipment on your car? In a staggeringly large number of accidents, the driver was using the brake at the time of the collision. Surely we could make driving safer by removing the brakes, then? Of course the thesis is ludicrous, but how many of us are confident that we wouldn't make a similar statistical mistake in a different context? But once you embark on the journey of trying to fix such problems in your own thinking, the road leads all the way to the place your post describes.

Most people think the journey is beyond them, and leave it to people like you and Eliezer to make the journey for them, and report back your findings. And unfortunately I don't think they're wrong about that.

This is a bit like a member of the Kalenjin tribe irritatedly asking me as I gasp and wheeze and double over:

"Why can't you just, like, run faster?"

But it's also really well-written and concise and valuably motivating (by way of shaming).

You're doing that thing where you write like Yudkowsky again. It's kind of hot.

FWIW I linked to this though my Twitter feed and got a very negative reaction from my friends, though the reasons they give for disliking it are pretty varied; they said things like

• it's so arrogant!
• It's absolutist - not everyone will have time to study these things
• but what is "truth" really?
• what about my-favourite-discipline? (Physics in one case)

One person has promised to write up what they felt in a blog post, which I look forward to reading.

They would study artificial intelligence to learn the algorithms, the math, the laws of how an ideal agent would acquire true beliefs.

Really? The others make sense, but it's not clear this will be useful to a human trying to learn things themselves. If I want to notice patterns, "plug all of your information into a matrix and perform eigenvector decompositions" is probably not going to get me very far.

I think you are confusing between wanting to know, and being good at it.

Imagine someone in the stone age, would you say none was genuinely curious because they didn't know about all those fields which weren't invented yet ?

Then, what about someone living in our world, but not knowing about Bayesian reasoning, AI, ... ? How can he know that those fields are fundamental to learn, to satisfy their curiosity on another field, before at least learning the basis of them ? When you don't know about Bayes' theorem, but you are curious (you really want to know the truth) about, say, ancient Rome history or about if there ever was life on Mars, what would drive you to learn probability theory ? How can you know you must learn it to learn other thing, when you don't know much about that other thing ?

Sure, if you are curious, you'll want to learn in all fields. But since we have a limited amount of time, you can't except someone to learn Bayesian reasoning, even if he's really curious, unless there is some kind of trigger that makes him realize how useful it would be to be efficient in being curious.

Genuinely wanting something and being good at doing it are not directly linked. You can't say someone isn't really wanting something just because he pursues it in an efficient way.

So are those who claim to seek truth, but really don't, merely akratic or are they hypocritical?

Do "curious" people want to learn the (already discovered) truth or to discover heretofore unknown truths? You seem to confound the two. Data on the statistical correlation between these distinct motives would be interesting, but I doubt most scientists are primarily concerned with personally accumulating true beliefs. Preparing to make contribution to human knowledge probably looks a lot different from preparing to absorb the greatest mass of truths. It probably also looks different from preparing to function rationally as far as go quotidian beliefs.

I'm not sure there's an overarching "curiosity" that people have or don't have: I'm very curious about whether a specific kind of database will perform adequately in certain circumstances (long story) but I'm only mildly curious about how to identify which French painter during the 19th century painted which picture. Some art experts, I'm sure, have cultivated the skill to guess within seconds which painter it is for every picture. I wouldn't mind having that skill -- it sounds like a fun skill to have -- but it seems like it would be more resources than it's worth. OTOH, I really want my probability estimations re: the database to reflect reality. Do I need to use AI theory? Doubtful. Probably a little bit of statistics, and even that fairly mild, but I do have to think a lot about how to use my knowledge of databases to design experiments to find the truth out. I'm not sure if that would look "curious" to the lay person (and, of course, there's also a factor of "signaling curiosity" -- I want to make sure that everyone with a stake in the process sees that I've done the due diligence), but nonetheless, I'm truly curious about this (and yes, it could go both ways...I think this is the most important part of curiosity vs. fake curiosity).

When I was genuinely curious about the US immigration laws applied to me (and again, it could have gone both ways -- before running any experiments, I made sure to visualize both options, and realizing I can live with both) I just called an immigration lawyer (and, for a latter question, the paralegal who was involved with my visa). In that case I needed very little knowledge from LW -- I didn't apply my knowledge about Bayes, or about heuristics and biases, and just went and asked a professional (of course, in some cases, like wanting to know if a stock will go up, asking a professional is disasterous, but with immigration law, lawyers can estimate probabilities fairly accurately even if they lack formal rationality training).

Those were the two examples of real curiosity from my life that I could think of, that looked nothing like the description here of "real curiosity"....