The Neglected Virtue of Scholarship

Eliezer Yudkowsky identifies scholarship as one of the Twelve Virtues of Rationality:

Study many sciences and absorb their power as your own. Each field that you consume makes you larger... It is especially important to eat math and science which impinges upon rationality: Evolutionary psychology, heuristics and biases, social psychology, probability theory, decision theory. But these cannot be the only fields you study...

I think he's right, and I think scholarship doesn't get enough praise - even on Less Wrong, where it is regularly encouraged.

First, consider the evangelical atheist community to which I belong. There is a tendency for lay atheists to write "refutations" of theism without first doing a modicum of research on the current state of the arguments. This can get atheists into trouble when they go toe-to-toe with a theist who did do his homework. I'll share two examples:

  • In a debate with theist Bill Craig, agnostic Bart Ehrman paraphrased David Hume's argument that we can't demonstrate the occurrence of a miracle in the past. Craig responded with a PowerPoint slide showing Bayes' Theorem, and explained that Ehrman was only considering prior probabilities, when of course he needed to consider the relevant conditional probabilities as well. Ehrman failed to respond to this, and looked as though he had never seen Bayes' Theorem before. Had Ehrman practiced the virtue of scholarship on this issue, he might have noticed that much of the scholarly work on Hume's argument in the past two decades has involved Bayes' Theorem. He might also have discovered that the correct response to Craig's use of Bayes' Theorem can be found in pages 298-341 of J.H. Sobel’s Logic and Theism.

  • In another debate with Bill Craig, atheist Christopher Hitchens gave this objection: "Who designed the Designer? Don’t you run the risk… of asking 'Well, where does that come from? And where does that come from?' and running into an infinite regress?" But this is an elementary misunderstanding in philosophy of science. Why? Because every successful scientific explanation faces the exact same problem. It’s called the “why regress” because no matter what explanation is given of something, you can always still ask “Why?” Craig pointed this out and handily won that part of the debate. Had Hitchens had a passing understanding of science or explanation, he could have avoided looking foolish, and also spent more time on substantive objections to theism. (One can give a "Who made God?" objection to theism that has some meat, but that's not the one Hitchens gave. Hitchens' objection concerned an infinite regress of explanations, which is just as much a feature of science as it is of theism.)

The lesson I take from these and a hundred other examples is to employ the rationality virtue of scholarship. Stand on the shoulders of giants. We don't each need to cut our own path into a subject right from the point of near-total ignorance. That's silly. Just catch the bus on the road of knowledge paved by hundreds of diligent workers before you, and get off somewhere near where the road finally fades into fresh jungle. Study enough to have a view of the current state of the debate so you don't waste your time on paths that have already dead-ended, or on arguments that have already been refuted. Catch up before you speak up.

This is why, in more than 1000 posts on my own blog, I've said almost nothing that is original. Most of my posts instead summarize what other experts have said, in an effort to bring myself and my readers up to the level of the current debate on a subject before we try to make new contributions to it.

The Less Wrong community is a particularly smart and well-read bunch, but of course it doesn't always embrace the virtue of scholarship.

Consider the field of formal epistemology, an entire branch of philosophy devoted to (1) mathematically formalizing concepts related to induction, belief, choice, and action, and (2) arguing about the foundations of probability, statistics, game theory, decision theory, and algorithmic learning theory. These are central discussion topics at Less Wrong, and yet my own experience suggests that most Less Wrong readers have never heard of the entire field, let alone read any works by formal epistemologists, such as In Defense of Objective Bayesianism by Jon Williamson or Bayesian Epistemology by Luc Bovens and Stephan Hartmann.

Or, consider a recent post by Yudkowsky: Working hurts less than procrastinating, we fear the twinge of starting. The post attempts to make progress against procrastination by practicing single-subject phenomenology, rather than by first catching up with a quick summary of scientific research on procrastination. The post's approach to the problem looks inefficient to me. It's not standing on the shoulders of giants.

This post probably looks harsher than I mean it to be. After all, Less Wrong is pretty damn good at scholarship compared to most communities. But I think it could be better.

Here's my suggestion. Every time you're tempted to tackle a serious question in a subject on which you're not already an expert, ask yourself: "Whose giant shoulders can I stand on, here?"

Usually, you can answer the question by doing the following:

  1. Read the Wikipedia article on the subject, and glance over the references.
  2. Read the article on the subject in a field-specific encyclopedia. For example if you're probing a philosophical concept, find the relevant essay(s) in The Routledge Encyclopedia of Philosophy or the Internet Encyclopedia of Philosophy or the Stanford Encyclopedia of Philosophy. Often, the encyclopedia you want is at your local library or can be browsed at Google Books.
  3. Read or skim-read an entry-level university textbook on the subject.

There are so many resources for learning available today, the virtue of scholarship has never in human history been so easy to practice.

Comments

sorted by
magical algorithm
Highlighting new comments since Today at 8:45 PM
Select new highlight date
All comments loaded

There are so many resources for learning available today, the virtue of scholarship has never in human history been so easy to practice.

Indeed.

I followed the links to In Defense of Objective Bayesianism by Jon Williamson and Bayesian Epistemology by Luc Bovens and Stephan Hartmann. They were expensive and unreviewed and my book reading heuristics generally require three independent suggestions before I start taking a book seriously.

A cheaper trick was to search the Stanford Encyclopedia of Philosophy for Bovens, Hartmann, and Williamson which lead to a nest of articles, some of which mentioned several of them. I listed and prioritized them using ad hoc scoring (points for mentioning each person and a good title). Hartmann jumped out because he had wider ranging interests and was tapped to co-author the encyclopedia article "Models In Science". To reduce the trivial inconvenience of starting to read, I reproduce the suggested reading list with my ad hoc numerical priorities right here:

I see that you have some experience applying the virtue of scholarship... :)

In general I'm very sympathetic to this point of view, and there are some good examples in your post.

One bad example, in my opinion, is Eliezer's recent procrastination post vs. the survey of "scientific research on procrastination." I read the chapter, and it appears to mostly cite studies that involved casual surveys, subjective description, and fuzzy labeling. Although there are many valid scientific endeavors that involve nothing but categorization (it is interesting to know how many species of tree frog there are and what they look and sound like even if we do not make any predictions beyond what is observed), categorization should at least be rigorous enough that we can specify what we expect to see with a modicum of precision.

When a biologist says that frogus neonblueicus has neon blue spots and chirps at 500 Hz, she will give you enough information that you can go to Costa Rica and check for yourself whether you have found one of the rare neonblueicus specimens. Although there will be some controversies around the edges, your identification of any particular frog will not correlate with your political biases or personal problems, and repeated observation of the same frog population by a few different researchers will tend to decrease error.

When a psychologist says that procrastinators can be divided into "relaxed" types and "tense-afraid" types, the "science" being done is not merely descriptive, but also horrifyingly vague. What does it mean for a human to be "tense-afraid" when "procrastinating"? The three paragraphs or so of context on the topic give you enough of an idea of what the researcher is saying to conjure up a mental image, but not nearly enough to carve thing-space at the joints.

In my experience, this is a very serious problem in social and human sciences -- there are whole subfields where the authors do not know how little they know, and proceed to wax eloquently about all of the empty concepts they have coined. There are other subfields where the researchers suspect that they might not have done very good research, and they cover their tracks with advanced statistics and jargon. After you dig through a few of these booby-trapped caves of wonder, you start to lose, if not respect for scholarship, at least some of the urge to do the moderately hard work of digesting literature reviews yourself on a regular basis. It is dangerous to assume that casually studying the leading textbook in a soft field will usually make you smarter.

I very much agree with your final sentence.

Do you think Eliezer's post is more precise and useful than the controlled experiments published in peer-reviewed journals described in the book I linked to? I find that most writing on psychology is necessarily pretty soft, because the the phenomena it is trying to describe are vastly more complicated than those of the hard sciences.

Now, that link is a must-read. I got through the whole first chapter before I could look away, and I'll be going back for the rest.

I have nothing against psychology or psychologists or social science in general -- AP Psych was my second favorite class in high school, my mom has a master's degree in it, my bachelor's degree is in political science, etc. It's noble, hard work, and we even have a little bit of knowledge to show for it.

As for the "controlled experiments" described in the book you linked to, I'm afraid I missed them, for which I apologize. I only saw descriptive papers. Maybe a page reference when you get a chance? Or just link directly to one or two of the studies or the abstracts?

It is dangerous to assume that casually studying the leading textbook in a soft field will usually make you smarter.

However, enough rationality training will have alarm bells ringing when reading soft textbooks and studies. That in itself - "this field is overpopulated with concepts and undermeasured" - is marginally more useful than knowing nothing about the field.

If you haven't already, you should try reading postmodern philosophy. An uninterrupted wall of alarm bells. :)

I was a philosophy student for my brief attempt at tertiary education - I know what you mean. Our lecturer would describe the text as 'dense' - more aptly, I thought, the author is dense.

An anecdote from that class: after a lecture on Wittgenstein, a student asked the lecturer if the rest of the semester's lectures were to be canceled.

Though I agree with you strongly, I think we should throw the easy objection to this out there: high-quality, thorough scholarship takes a lot of time. Even for people who are dedicated to self-improvement, knowledge and truth-seeking (which I speculate this community has many of), for some subjects, getting to the "state of the art"/minimum level of knowledge required to speak intelligently, avoid "solved problems", and not run into "already well refuted ideas" is a very expensive process. So much so that some might argue that communities like this wouldn't even exist (or would be even smaller than they are) if we all attempted to get to that minimum level in the voluminous, ever-growing list of subjects that one could know about.

This is a roundabout way of saying that our knowledge-consumption abilities are far too slow. We can and should attempt to be widely, broadly read knowledge-generalists and stand on the shoulders of giants; climbing even one, though, can take a dauntingly long time.

We need Matrix-style insta-learning. Badly.

getting to the "state of the art"/minimum level of knowledge required to speak intelligently, avoid "solved problems", and not run into "already well refuted ideas" is a very expensive process.

So is spending time and effort on solved problems and already well refuted ideas.

In my experience, Ph.D. dissertations can be a wonderful resource for getting an overview of a particular academic topic. This is because the typical -- and expected -- pattern for a dissertation is to first survey the existing literature before diving into one's own research. This both shows that the doctoral candidate has done his/her homework, and, just as importantly, brings his/her committee members up to speed on the necessary background. For example, a lot of my early education in Bayesian methods came from reading the doctoral dissertations of Wray Buntine, David J. C. MacKay, and Radford Neal on applications of Bayesian methods to machine learning. Michael Kearns' dissertation helped me learn about computational learning theory. A philosophy dissertation helped me learn about temporal logic.

Of course, this requires that you already have some background in some related discipline. My background was in computer science when I read the above-mentioned dissertations, along with a pretty good foundation in mathematics.

I think that most people just don't believe that philosophy has any value. I used to believe that it didn't, gradually concluded that it did, but then gradually concluded that yes, 99.9% of it really is worthless such that even reading contemporary famous people or summaries of their arguments (though not discussing such arguments with your epistemic peers who are familiar with them, and not reading pre-WWII philosophers) really is a waste of time.

I agree that 99.9% of philosophy is very close to worthless. Its signal-to-noise ratio is much lower than in the sciences or in mathematics.

This brings to mind Eliezer's comment that "...if there’s any centralized repository of reductionist-grade naturalistic cognitive philosophy, I’ve never heard mention of it."

But reductionist-grade naturalistic cognitive philosophy is probably an even larger sub-field of philosophy than the formal epistemology I mentioned above. Names that come immediately to mind are: John Bickle, Pat & Paul Churchland, Paul Thagard, Tim Schroeder, William Calvin, Georg Northoff, Thomas Metzinger.

There's some good philosophy out there. Unfortunately, you normally only encounter it after you've spent quite a while studying bad philosophy. Most people are introduced to philosophy through Plato, Aristotle, Aquinas, Kant, and Hegel, and might never suspect a neurophilosopher like John Bickle exists.

Which reminds me of the old Bertrand Russell line:

Hitherto the people attracted to philosophy have been mostly those who loved the big generalizations, which were all wrong, so that few people with exact minds have taken up the subject.

I don't know what percentage of writing that gets called "philosophy" is worthwhile but it isn't that hard to narrow your reading material down to relevant and worthwhile texts. It's really weird to see comments like this here because so much of what I've found on Less Wrong are ideas I've seen previously in philosophy I've read. Moreover, a large fraction of my karma I got just by repeating or synthesizing things I learned doing philosophy- and I'm not the only one whose gotten karma this way.

I find it particularly perplexing that you think it's a good idea to only read pre-WWII philosophers as their ideas are almost always better said by contemporary authors. One of my major problems with the discipline is that it is mostly taught by doing history of philosophy- forcing students to struggle with the prose of a Plato translation and distilling the philosophy from the mysticism instead of just reading Bertrand Russell on universals.

In another debate with Bill Craig, atheist Christopher Hitchens gave this objection: "Who designed the Designer? Don’t you run the risk… of asking 'Well, where does that come from? And where does that come from?' and running into an infinite regress?" But this is an elementary misunderstanding in philosophy of science. Why? Because every successful scientific explanation faces the exact same problem. It’s called the “why regress” because no matter what explanation is given of something, you can always still ask “Why?”

IMO, it is perfectly reasonable to object with: "Who designed the Designer?".

The logic being objected to is: it takes a big complex thing to create another big complex thing. Observing that Darwinian evolution makes big complex things from scratch is the counter-example. The intuition that a complex thing (humans) requires another complex thing to create it (god) is wrong - and it does tend to lead towards an escalator of ever-more-complex creators.

Simplicity creating complexity needs to happen somewhere, to avoid an infinite regress - and if such a principle has to be invoked somewhere, then before the very first god is conjoured seems like a good place.

Checking with the "common sense atheism" link quite a few people are saying similar things in the comments.

timtyler,

Hitchens did not mention complexity or simplicity as you propose. And he did not mention evolution as you propose. If you read the Hitchens quote, you will see he only gave the why-regress objection, which is just as valid against any scientific hypothesis as it is against a theistic one.

There are ways to make the "Who designed the Designer?" objection stick, but Hitchens did not use one of them. If you read the Hitchens quote, you'll see that he explicitly gave the why-regress objection that could be just as accurately be given to any scientific hypothesis ever proposed.

Here, let's play Quick Word Substitution. Let's say a physicist gives a brilliant demonstration of why his theory of quarks does a great job explaining a wide variety of observed subatomic phenomena. Now, Hitchens objects:

"But what explains the quarks? Don’t you run the risk… of asking 'Well, where does that come from? And where does that come from?' and running into an infinite regress?"

Hitchens explicitly gave the why-regress objection that is just as potent against scientific explanations as it is against theistic explanations.

More specifically it is completely rational to use that argument against theists, because one of their arguments for god is that the world is too complex not to have been designed; so in that circumstance you are just pointing out that their claim is just pushing the complexity back one step. If the world is so complex that it needs a designer, then so is god.

LessWrong often makes pretty impressive progress in its discussions; I would be thrilled to see that progress made beginning at the edge of a field.

I sincerely doubt that the discussions which began on the leading edge would return anywhere near the same amount of progress as those which start in the scholarly middle. After all, those problems are on the edge because they're difficult to solve given the intellectual tools we have today. Though Less Wrong is often insightful, I suspect it's the result not of discovering genuinely new tools, but of applying known tools in ways most readers haven't seen them used. For Less Wrong to make progress with a problem that a lot of smart people have been thinking about in detail for a long time either requires that the entire field is so confused that no one has been able to think as clearly about it as we can (probably hubristic), or that we have developed genuinely new intellectual techniques that no one has tried yet.

Voted up, but I think that there are LOTS of fields that are that confused. Possibly every field without regular empirical tests (e.g. chemistry, engineering, computer science, applied physics, boxing) is that confused.

i ghost write papers for lazy rich undergrads at prestigious institutions and my experience has been that the soft sciences are a muddle of garbage with obscenely little worth given the billions of dollars poured into them that could be saving lives.