Teachable Rationality Skills

Recent brainstorming sessions at SIAI (with participants including Anna, Carl, Jasen, Divia, Will, Amy Willey, and Andrew Critch) have started to produce lists of rationality skills that we could potentially try to teach (at Rationality Boot Camp, at Less Wrong meetups, or similar venues).  We've also been trying to break those skills down to the 5-second level (step 2) and come up with ideas for exercises that might teach them (step 3) although we haven't actually composed those exercises yet (step 4, where the actual work takes place).

The bulk of this post will mainly go into the comments, which I'll try to keep to the following format:  A top-level comment is a major or minor skill to teach; upvote this comment if you think this skill should get priority in teaching.  Sub-level comments describe 5-second subskills that go into this skill, and then third-level comments are ideas for exercises which could potentially train that 5-second skill.  If anyone actually went to the work of composing a specific exercise people could run through, that would go to the fourth-level of commenting, I guess.  For some major practicable arts with a known standard learning format like "Improv" or "Acting", I'll put the exercise at the top and guesses at which skills it might teach below.  (And any plain old replies can go at any level.)

I probably won't be able to get to all of what we brainstormed today, so here's a PNG of the Freemind map that I generated during our session.

Comments

sorted by
magical algorithm
Highlighting new comments since Today at 8:38 PM
Select new highlight date
All comments loaded

Skill: Being strategic. I.e., life-consequentialism, where you actually do things based on their expected future consequences, as opposed to drifting into a PhD program because your friends are doing it.

Exercise materials: We either need to develop efficient probes for getting people to list out major life choices that they could actually remake (are under serious reconsideration) or we need to develop hypothetical life stories and policy decisions to use in exercises.

Subskill: Avoid pitfalls of verbal strategic reasoning.

  • List consequences without shifting from intuitive-sum to verbal-justification mode.
  • Don't exacerbate scope insensitivity or attending to rare events.

There are studies showing that people who consider their decisions more make worse decisions. As I understand it, the main explanation for this is that people shift from an intuitive sum of costs and benefits, to seeking verbally justifiable decisions, which in turn might lead them to one-reason-decisionmaking, ignoring some of their costs and benefits which are important to them but seem less "sensible". I also suspect it may exacerbate other biases like scope insensitivity or rare events - thinking about cases which are rare or short in duration.

The classic case being "Let's get a bigger house, further away from work, so it has an extra bedroom in case Grandma comes over", which she does once a year, but the 20 minutes of extra commute time happen every day and are not acclimated-to.

Subskill: Maximize on big things, satisfice on small things.

Subskill: Unbundling; optimize separate things separately.

Example 1: Optimize fuzzies and utilons separately.

Example 2: Optimize grades and learning separately (instead of just optimizing grades, or haphazardly optimizing both at the same time).

5SL: Notice when you're optimizing two things at once. (Maybe because you (a) have a sense of awkwardness or of not getting enough done, and when you list out the desirable consequences, there's more than one?) Then, find two different things you can do which optimizes each one individually and without worrying about the other one.

Subskill: Notice foreign goals. Are your parents making you do it? Are you doing it because you read it in a book? Did you just drift into doing that?

Converse subskill: Notice feeling of actually caring about something. Notice whether that caring is giving energy to what you're doing. Notice its absence.

Subskill: Detach from sunk costs.

Material for exercises: Requires a case where the person has sunk costs and the option of continuing or not continuing. Might want to choose cases slightly less fraught than a marriage or a PhD program - you want to work up to those gradually.

Exercise A: For a case where sunk costs exist, imagine that you are a new person who was just now teleported into this person's life. Variant: Imagine you were just teleported into this person's life, and everyone else knows this, and they all expect you to execute sudden changes of course. See what this changes about your thinking, regardless of whether it changes your decision.

Exercise B: For whatever sunk costs you're in the middle of expending, look at the same scenario and rephrase it as a sunk benefit, the purchased option to finish a task more quickly than before. E.g., if you already paid $100 on a $150 item, change from "I paid $100" to "I now have the option of purchasing this item for $50". Again, the stated objective of the exercise will be, "Notice the difference in your thinking", not, "try to change your decision".

Exercise C: As above, but imagine that you bought the option for a penny on eBay. E.g. if you're one year into a four-year PhD program, imagine that you paid a penny on eBay to purchase an option to get a PhD in three years rather than the usual four years. Would you exercise that option if you paid a penny for it?

That last exercise seems to run afoul of some value-related heuristics. Price is so common a proxy for utility that imagining that you paid a penny for the option on eBay might irrationally devalue whatever you're looking at.

Of course, looking at the contrast might still give you some useful insights -- provided you can untangle them.

Subskill: Use fungibility. There are different ways to achieve many goals. E.g. instead of wasting an evening with a relative in a way you resent, send them a postcard.

Subskill: Don't express emotions as policy decisions. (Anna.) Find some other outlet for the emotion besides the policy decision, i.e., screaming. (Eliezer.)

Keep in mind that the current psychology literature indicates that cathartic release is actually undesirable-- while catharsis might improve affect in the short term, this is largely because it represents giving in to one's anger, and in the long run it actually makes you more likely to experience anger in the future.

I therefore advise that, in the event that one needs to make a particularly important decision and is currently angry, cathartic screaming may be effective, but its use should be restricted only for cases where it is strictly necessary.

(and this is coming from the ex-leader of a cathartic screaming club :P)

Or, more generally, what you do reinforces your mental state at the time. Acting in anger (even just a cathartic release) reinforces your tendency toward (the likelihood of future) anger; the same with acting from fear and most other emotions. Also, whatever the cause of your procrastination, acting on it, letting yourself get away with procrastinating, increases your problems with it in the future. As a poster I made for my study wall many years ago said: "Commit through Action: Do It".

Subskill: Chain from feelings of angst and frustration into saying, "I need to be strategic!" and the other skills listed.

Rationality skill: Recognize rationality skill in others.

There is a strong tendency for people to use the heuristic, "P(Person X is right) = # of times person X has been observed to be right / # of statements person X has made", or worse, "(# statements by X - # of times X has admitted to making a mistake) / # statements by X".

This encourages people who want to be respected to do 4 bad things:

  • Make many pronouncements on things that are obviously right, perhaps in a manner suggesting they are controversial claims.
  • Avoid saying anything unless they are certain they are correct.
  • Avoid saying anything concrete enough to possibly be proven wrong.
  • Never, ever admit to having made a mistake.

Unfortunately, these methods are extremely effective.

Skill: Anti-rationalization (1): Prevent your mind from selectively searching for support of only one side of an argument.

Subskill: Analyze the underlying reasons why you're trying to rationalize for or against something - why a conclusion feels required, or disallowed.

Important subskill: Notice when a candidate "the reason I'm trying to rationalize something" is a poor guess or itself a rationalization - when it's a guess that sounds plausible about someone like you in your position, but doesn't seem to ring true.

Subskill: Notice the process of selectively searching for support, and halt it. Chains into "Ask whether, not why", or into trying to unbind the need to rationalize.

Level 1, notice the active process after performing a conscious check whether it's running; Level 2, detect it perceptually and automatically; Level 3, never run this process.

Exercise material: List of suggested conclusions to rationalize. I'm not quite sure what the prerequisites here would be; one of my own childhood epiphanies was finding that I could argue reasons why there ought to be a teapot in the asteroid belt, and thinking to myself, "Well, I better not do that, then." I suspect the primary desideratum would be more that the topic offers plenty of opportunity to come up with clever rationalizations, rather than that the topic offers motivation to come up with clever rationalizations.

Exercise A: Pick a conclusion from the list. Come up with a clever argument why it is true. Notice what it feels like to do this. Then don't do it.

(This is distinct from noticing the feeling of being tempted to rationalize, a separate subskill.)

Exercise B: In the middle of coming up with clever arguments why something is true, stop and chain into the Litany of Tarski, or some other remedy. Variant: Say out loud "Ew!" or "Oops!" during the stop part.

Exercise C: For a conclusion on the list, argue that it is true. Then "Ask whether, not why" - try to figure out whether it is true. Notice the difference between these two processes. Requires a question whose answer is less than immediately obvious, but which someone can find info about by searching their memory and/or the Internet.

Hmmm this just made me think that debate clubs deliberately teach the opposite of this skill.

Skill: negotiation - deliberately reaching a mutually beneficial trade or agreement, even in situations of slight power imbalance. Important for rationalists who aim at earning money as an instrumental goal.

(At the 5-second level a key component of this is learning to say "no", being able to overcome one's agreeableness as the default decision.)

(For some reason negotiation in situations of extreme power imbalance seems like it should have a different name, and I don't know what that should be.)

Because normal comments will inevitably get mixed up with those that satisfy the requested sort of post, I recommend you bold the words "Skill", "Subskill" and "Exercise".

Skill: Maintaining "contextual pointers" with your knowledge, both for the sake of evidential sourcing and contextual usage.

The idea here is twofold.

First, many intellectual conversations between people who lack this skill are are random walks through topic space. Each tangential leap is potentially useful for mining relevant evidence from each person's mind... but in conversations exploring an important central thesis it is good to treat the tangents as objects pushed onto a stack in the course of linear conversation. When you go to far afield with a tangent you need the presence of mind to notice that D was inspired by C which was inspired by B, which was profoundly relevant to the pragmatically important question of A. Stepping back to C or B or even A is frequently called for in important conversations but requires contextual mindfulness.

The second use of context is evidential. Being mindful of where evidence comes from (if it can be managed) helps keep track of the value and meaning of evidence. Confabulation induced or permitted by source amnesia, isn't necessarily a bad thing in terms of "usage optimized concepts" but in terms of epistemic hygiene its a killer. It is especially pernicious in modern media environments full of advertising, fiction, and bullshit.

I bring these two sorts of context up because they both involve "context pointers" and together the two kinds of context pointers enable clear "tree formatted thinking" where reasons radiate from a root node, and leaf nodes carry citations to allow validation.

Skill: Productivity/Munchkinism. Not giving up in pursuing a difficult worthwhile goal, always acting on the best available plan, including the search for a better one. Ability to convert additional time and resources into additional (if slow) optimization, as opposed to wasting them.

Subskill: Creativity, ability to generate new prototype ideas and refine them, modifying and combining known things to gradually reach into new territory.

Subskill: Scholarship, ability to collect ideas and skills from elsewhere, for future use as building blocks for your problem.

Subskill: Optimization of own activity. Creatively looking for better ways of doing things instead of only following a fixed routine. Improving effectiveness of the routine past the point of satisficing a "reasonable" level (where that's consequentialistically helpful).

Skill: An accurate and functional awareness of time.

By this I mean to include noticing that pre-established dates and times have arrived, predicting the amount of time things will take you or other people, not carrying through with schedules or plans inflexibly when real surprises should cause updates, understanding how to effectively communicate about time with people about time management in ways that respect the listener's time management skills, and so on.

some of these skills can be explained to a much younger audience (catch them young), in the form of stories, one such story made up for my daughter,

               As the Sun rose, the whole forest was filled with loud but sweet chatter of the squirrels chirping, as all the little ones were busy getting ready for school. Squeaky, the youngest among them, a very energetic and clever guy, was also gentle and kind, hence was adored by all the squirrels in the forest.

Squeaky enjoyed school very much and was always the first to get there. He loved what he learned each day and seeing his enthusiasm his parents gifted some chalk for him to practice his lessons at home. Squeaky was very happy and immediately set about scribbling on the trunk of the tree where his home was. That day in school, he was taught the number 'one', and his home tree was filled with that number. But soon there was a problem, “How could a little tree trunk be enough to hold all my knowledge?” wondered Squeaky. He thought about this and then decided to borrow the other trees in the forest for this purpose.

The next day he learned about 'two', and painted the whole forest, all the trees, including his neighbours with that number. When the other squirrels climbed their respective trees to get to their home, they were fully covered with chalk dust. And by the time they reached their destination, they looked like Eskimos. Thanks to Squeaky. Since, he was an adorable child, no one complained.

Every day the forest bore witness to Squeaky's expanding knowledge, one day it was 'three' and the next day it would be filled with 'four'. As Squeaky's knowledge was near 'three hundred and twenty six', the whole colony shuddered to think of the days ahead. They did not want in the future to start as a squirrel at the base of their tree and end up at the top as an Eskimo.

So, they sought the help of an wise old Owl, who thought about it for a moment and said, "Each one of you should gift different coloured chalks to Squeaky and before he could use it, ask him to come and see me". They all were perplexed, "Is this some kind of ZEN thing, the solution to a problem is more of the same problem?" they whispered amongst themselves, but nevertheless decided to implement the Owl's idea. "It would at-least be a change from the usual white", bought a chorus laughter from the crowd. "See you tomorrow covered in red" giggled one. "Hope some one gives him blue, I would look good in blue" wondered another.

The next day to Squeaky's surprise, all his neighbours gave him chalks of different colours, for his use. But they said there was a catch; he should first go and see the wise old Owl and follow his instructions if he wants to use them.

Squeaky went to the Owl and introduced himself. The Owl said “ I want you to write about the things you do-not-know, so that I can teach them to you”. This was music to his ears. Now he had a mentor. He thanked the Owl and promised to start from tomorrow.

The next morning when Squeaky was about to begin, he thought about the Owl's instruction and as the chalk touched the trunk, he was filled with a strange feeling. He had never felt this way before and an uneasy calm settled on him. His mind was blank. He wondered, 'What do I do-not-know?'. He was rather clueless.

Squeaky sat down to think, could not think of things he did not know and whatever he thought, he knew. Slowly, the seconds ticked to minutes, and minutes to hours, hours to days, but still there was no progress. All the squirrels heaved a sigh of relief but at the same time felt sorry, as it was rather a mean trick to play on the little fella and in the mean time the forest came back to its original colour.

(As Squeaky is not very good at knowing what he does not know, and you children being so much smarter, can you think about what you do-not-know. Can you?)

more of my effort to teach rationality skills to my daughter can be found at this address, http://bpsundar.weebly.com

Skill: Distill yourself to simple terms. Be funny and interesting.

Exercise:

  • Have list of topics. Select one using a random number generator. Or use random ANKI cards. Whatever.

  • Immediately after selecting, turn on your digital camera/computer camera and record.

  • You can aim for lots of different goals: tell an interesting story, give a short two-sentence answer, etc.

This is great for preparing for job interviews also.

Skill: Mindfulness / not doing things on autopilot.

An underlying capacity that seems worth exercises to train explicitly, if we can figure out how to do that.