I kept having nightmares for about a decade after completing my last university course, so I find it hard to understand anyone wanting to go back to school. Ok, now that I've gotten that off my chest...

When I visited SIAI a few months ago, I participated in drafting a list of study topics for visiting fellows. I think it's now being used by SIAI internally, but perhaps eventually a version will be produced for public consumption. For now I'll just try to answer 2 and 4.

What learning actually facilitates achieving something useful or otherwise fulfilling one's CEV?

It seems to me that an important but easily overlooked step to fulfilling one's EV is to figure out what it is. Philosophers haven't found the answer yet, but studying philosophy at least gives you some idea of what kinds of answers people have already considered and found unsatisfactory. I would prefer to do this by reading/skimming books, but if you must take a course...

Which course do you just happen to like?

My favorite university courses were:

• C++
• theory of computation
• game theory (graduate level)
• industrial organization (graduate level)

My favorite self-taught topics are:

• micro- and macro-economics
• cryptography
• algorithmic information theory
• decision theory
• philosophy of science/mind/mathematics/etc.

By "favorite" I mean courses/topics that I had the most fun learning, but these also turned out to be quite useful for my purposes. I might just have been lucky, but you should perhaps consider the possibility that your "impulsive unconscious preference satisfaction" knows what it's doing, and put some thought into what courses would be fun for you.

My guess is that for instrumental rationality the optimal courses are dance, yoga, and above all acting. Just don't try to use them as credentials.

First, you should take the minimum number of classes necessary for graduation because school is so much less efficient than other learning methods. (For example, you can watch videolectures from the best professors in the world for free at your convenience. Then download the lectures + VLC, change the playback speed, and learn everything 50% faster.)

As long as you are taking classes, you might as well try to learn things that are a good fit for them.

• Public speaking and acting are good fits for classroom learning.
• Chemistry classes and some others give you a chance to use materials that would be hard to find on your own.
• IMO classes involving computers are an awful fit for classroom learning--it's always easier for me to figure out stuff myself. This includes programming. Only take these if you're going for a compsci degree (but, it is a pretty good degree to have as far as I can tell).
• Take classes by professors that are well-reviewed (see ratemyprofessors.com), especially those with a reputation for high levels of class participation or interesting personalities.
• Also, it's worth choosing classes based on the classmates you expect to have. So consider taking classes full of smart people like physics and game theory.

As for subject matter, I think I became smarter learning physics and economics basics because I became more comfortable applying math and thinking about things in simple, non-intuitive ways. Economics is particularly great for rationality--microeconomics is essentially an extended description of what the world would be like if we were all rational. It's great to have a cheat sheet. Programming was definitely my biggest ever mindfuck, but as I mentioned it's best learned outside school.

I just graduated from undergrad in mathematics, so perhaps I have less perspective, or perhaps I have a "fresher" perspective! I don't know.

A few classes that I enjoyed without expecting it:

-a class called "Feminism and Science." I would be very surprised if there were classes in feminist science studies at your school, but they have a perspective on rationality and science studies that is unique and valuable.

--relatedly, I wish that I had taken courses in feminism. It wasn't until the last year that I realized how much of feminism deals with things like resolving hidden inferences (first link NSFW!)

--also, science studies classes will almost certainly benefit from having someone from LessWrong in them. So will feminism classes!

-"Politics and Religion," a class about the stale religious metaphors that get used in modern politics. Again you may not have a perfect analog, but a cursory class or two in politics or religion could give a lot of insight about how other people operate, and also expose them to how you operate, if you care about other people's rationality as well. (whereas math classes will be much more homogenous.)

-a topical course from the linguistics department. Linguistics is very, very interesting. When I say a "topical course" I mean I took a course for non-majors which was more of a class in "why people study linguistics" and less in "how people study linguistics." I learned a lot about what makes questions of linguistics important in questions of rationality (again, see hidden inferences above!)

-language classes. I took Japanese, and it was enjoyable, stretched my mind a bit (for reasons detailed in the above class!) and kept my work ethic going. Also let me interact with people from various backgrounds, instead of only math majors.

-Playwriting. I actually expected to enjoy this. Whether it's good for rationality... well there are some applications of behavioral psych, and some ability to learn about how much of the theory of writing actually has a foundation.

classes I wish I had taken but didn't:

-any psychology classes, especially evolutionary psych!

-more linguistics

-more than one computer science course (though I wouldn't want to major in it)

-evolution and ecology

-science fiction-themed literature classes

-I mentioned feminism and science studies earlier

-lots of different languages. Having a designated place and time to speak different languages (at least in my experience) makes it a lot easier to learn, and college is a great opportunity for that that won't come back.

-bioethics, legal studies

-game theory

classes I didn't enjoy as much as I expected:

-real analysis. But that's because I like algebra.

-economics. I once had a TA tell me, when I asked about a question on a test, "well I was grading that problem, and I thought what you had was okay, but the answer key said it was (c) so I marked it off."(sic). I never took another econ class again. Not sure if that would be a problem in other places.

-history. Too much reading and not enough real knowledge.

Repeating stuff I think is important

I think that both feminist studies and linguistics have a lot more potential for carving reality at the joints than, say, mathematical physics. Of course, the background that mathematical physicists have is better for actively doing this, and you might have to fight some cultural battles in feminist studies classes. But rationalists and feminists have a lot in common and I think more crossover is important there.

I can't really speak about other specialisations, but mathematical physics seems to me not too beneficial for general thinking skills. I know at least two mathematical physicists who think that the main task in the job is to suppress your intuitions (which isn't itself bad) at any cost (this is worse). So any heuristic argument they dismiss until you have find the correct way to prove it. The obsession with proofs makes mathematical physics (and mathematics in general) in my eyes closer to what is here called Traditional rationality than to Bayesian approach, and a little bit legalistic. Subjects like theoretical physics rely more on intuitions, but still firmly based in reality (well, string theory not so much, but still), and so closer to everyday thinking.

If the school has some "big names" who teach about their area of specialty, consider taking their classes. One of the very important researchers in my field was a professor at my university while I was there, but I had no idea at that time what he did, so I missed out.

1 . Really there are two things you need to get good at. First, there's "rationality" itself, for which you should learn the math and learn about cognitive bias etc. Second, there's the content of your beliefs; in other words, learning everything there is to know. So my answer for #1 is: find a big gap in your knowledge, and fill it.

2 . This is a very complicated question. In the big picture sense, the things that would be of most benefit to you would be things that have a huge impact on the future, like topics relevant to FAI or Transhumanism. You could work on improving the human lifespan and quality of life, or create something that will do that for you.

3 . If you've already done an undergraduate degree in AI, computing, or a field like that, you really should be able to just jump into a master's degree program in fields like Mathematics. Another bachelor's degree in a particular field doesn't make a lot of sense, partially because the main point of a bachelor's degree is to make you a well-rounded person and give you access to a wide variety of perspectives, and you've presumably already done that.

That said, some fields do expect you to have done a lot of specific work during your undergrad, so if you wanted to (say) go into a biotech-related field, you might need to do a bachelor's in a related field first.

4 . Computer science is always fun. Mostly, it's a long stream of interesting toy problems. Ditto for philosophy, but you really need to be at the right school for that to work. (Your generalizations aside, I must agree that philosophers who have done some programming seem superior, though I'm not a good choice for an unbiased opinion).