Another question (pre-mortem):

"It's now the future, and your current plan has failed. Why?"

"What is the next thing that physically needs to happen for this to move forward?"

Next actions are hard to get right. If something was stuck on my to-do list for a while, usually the next action wasn't quite immediate enough. Sometimes the answer is just 'schedule 20 minutes at a specific point in the future to think about how to move forward.'

(Of course, sometimes the problem is that I don't really want to do the list item. Then there's a different set of questions to ask.)

"What hidden obstacle could be causing my failures?"

My mental shorthand for this is the following experience: I try to pull open the silverware drawer. It jams at an inch open. I push it shut and try again, same result. I pull harder, it opens a tiny bit more before stopping.

Reflection: Some physical object is getting in the way of the motion. Something could be on the drawer track, but more likely it is inside the drawer. It is a rigid object, because I always stop at the same place, although slightly squashable because I was able to yank and pull a little harder. It is probably striking the inner wall of the cabinet in which the drawer is mounted. It is on an angle because I can't see it when I look through the inch gap. There is a fork or knife angled up and poking against the inner wall. Digging around with my finger quickly finds a fork.

Since then, I've brought up this question by asking myself "what is the fork in the drawer"?

For example, my linear algebra students generally seem smart and attentive, but they become confused whenever I do a detailed computation with inner products. After some thought about which computations confuse them, hypothesize that whoever taught them basic matrix manipulations didn't teach the "transpose" operator, and particularly didn't teach the rule (AB)^T = B^T A^T. Fixed very quickly. (Of course, I also try to encourage them to ask questions about what confuses them, but I think that it is impossible to ever get a class comfortable enough questioning you to not need to think on your own about what is the underlying difficulty causing confusion.)

"Are we in a waiting loop?"

I've gotten into these with my wife, where I will be doing something thinking I am waiting for her, and she will be doing something thinking she is waiting for me. Asking this question periodically can help end deadlocks.

Given Superationality, TDT, and the weight-loss post;

"Do I want to be the kind of person who makes this particular choice in this kind of situation?"

This pertains to decision theory as well as something you do often (adapted from weight-loss post: "Do I want to be the sort of person who eats a pack of doughnuts every time I pass a doughnut shop, or do I want to be thin?).

Awakening the Giant Within had a nice set of questions you could use if you were in what felt like a terrible situation, in order to reorient yourself to make the best of it:

• What is great about this problem?
• What is not perfect yet?
• What am I willing to do to make it the way I want it?
• What am I willing to no longer do in order to make it the way I want it?
• How can I enjoy the process while I do what is necessary to make it the way I want it?

The author gives an example of a situation where he was exhausted after traveling for four months, and had countless of urgent memos and phone calls that he had to answer as soon as possible. Noticing that he was asking questions like "How come I have no time? Why don't they leave me alone?" that were just demotivating and depressing him further, he instead asked himself the above questions.

What is great about this problem?

My first response, like so many other times, was "Absolutely nothing!" But I thought about it for a moment and realized that just eight years ago I would have given anything to have twenty business associates and friends who wanted to visit with me, much less 100 people of such national impact and caliber that this list of friends and business associates represented. As I realized this, it broke my pattern, and I began to feel grateful that there were so many people whom I respect and love who wanted to spend time with me.

What is not perfect yet?

My schedule obviously needed more than a little fine-tuning. I felt like I had no time to myself, and that my life was out of balance. Note the presupposition of this question: asking "What is not perfect yet?" clearly implies that things will be perfect. This question not only gives you new answers, but reassures you simultaneously.

What am I willing to do to make it the way I want it?

I decided then that I was willing to organize my life and my schedule so that they were more balanced, and I was willing to take control and learn to say no to certain things. I also realized that I needed to hire a new CEO for one of my companies, someone who could handle some of my workload. This would give me more special time at home and with my family.

What am I willing to no longer do in order to make it the way I want it?

I knew that I could no longer whine and complain about how unfair it was or feel abused when people were really trying to support me.

How can I enjoy the process while I do what is necessary to make it the way I want it?

When I asked this last, most important question, I looked around for a way to make it fun. I thought, "How can I enjoy making 100 calls?" Sitting there at my desk did not turn up the mental and emotional juice. Then I got an idea: I'd not been in my Jacuzzi in six months. I quickly slipped on my swim trunks, grabbed my portable computer and speaker phone, and headed for the Jacuzzi. I set up shop out in my back yard, and started making the calls. I called a few of my business associates in New York and teased them, saying, "Really, it's that cold? Hmmm. Well, it's really tough out here in California, you know. I'm sitting here in my Jacuzzi!" We all had fun with it and I managed to turn the whole "chore" into a game. (But I was so wrinkled that I looked around 400 years old by the time I got to the bottom of my list!)

That Jacuzzi was always there in my back yard, but you'll notice that it took the right question to uncover it as a resource. By having the list of these five questions in front of you on a regular basis, you have a pattern of how to deal with problems that will instantly change your focus and give you access to the resources you need.

Related post on Less Wrong: Your Inner Google:

I just heard a comment by Braddock of Lovesystems that was brilliant: All that your brain does when you ask it a question is hit "search" and return the first hit it finds. So be careful how you phrase your question.

Say you just arrived at work, and realized you once again left your security pass at home. You ask yourself, "Why do I keep forgetting my security pass?" [...]

But you are not rational, and your brain is lazy; and as soon as you phrase your question and pass it on to your subconscious, your brain just Googles itself with a query like

why people forget things

looks at the first few hits it comes across, maybe finds their most-general unifier, checks that it's a syntactically valid answer to the question, and responds with,

"Because you are a moron."

Your inner Google has provided a plausible answer to the question, and it sits back, satisfied that it's done its job.

If you instead ask your brain something more specific, such as, "What can I do to help me remember my security pass tomorrow?", thus requiring its answer to refer to you and actions to remember things and tomorrow, your brain may come up with something useful, such as, "Set up a reminder now that will notify you tomorrow morning by cell phone to bring your security pass."

When prolonging a decision until you know more:

Would having this information actually influence my choice, or is it just an excuse to put off a painful decision?

Do I really want to do this thing/buy this thing/eat this thing? Or is my impulse to do/buy/eat it just a cached thought, a habit from other times I have enjoyed it, even though now I do not really feel like it?

"What skill am I actually training?"

Teachers can also ask themselves, "what skill am I actually teaching?"

"What is the part of me that is preventing me from moving forward worried about?"

Be careful not to be antagonistic about the answer. The goal is to make that part of you less worried, thus making you more productive overall, not just on your blocked task. The roadblock is telling you something that you haven't yet explicitly acknowledged, so acknowledge it, thank it, incorporate it into your thinking, and resolve it.

Example: "I'm not smart enough to solve this math problem." Worry: "I would need to learn a textbook's worth of math right now in order to solve it. I must go learn it now." Resolution: "It's fine that I don't have the ability to solve the problem, and learning the math in 5 minutes is impossible and not necessary to satisfy my goals here. Trying to solve it the best I can will help me learn the math for future problems."

You're probably learning (reinforcing) more than one skill at a time, in different proportions. Sometimes you unavoidably reinforce a bad thing but it's still worthwhile in order to learn a good thing at the same time. It's very good to be aware of all that you are learning - so I suggest formulating your question in the plural: "what skills am I training, at what tradeoffs?"

For example:

• You're going to make some mistakes as you learn a new piece of music; as you practice more, you stop making mistakes. There are better and worse tradeoffs in terms of initial playing speed and mistakes made.
• While students learn to take notes (irrelevant) and parrot the teacher's answers (mostly harmful), they also learn actual math, and this can be a great tradeoff.

"Which assumptions am I relying on without being conscious of it?"

To answer this question it's often useful to invert it: what would have to be true (or not true) to invalidate my conclusion?

• What model of the world/way of thinking/worldview generated the things this person is saying?

• What game am I playing? What are the rules (or, more relevantly, what aren't) and how do I win?

• What is the goal of this conversation? (Note: you must be careful to answer with the actual goal, not the goal you think you should have or the goal you would answer with in public. For instance, if your answer is just "truth seeking" you are wrong with very high probability.)

Some variation of “What is the other person’s actual objective?” Or “Why did they do that?” or “What are they actually asking me?”

I started this habit in chess where it’s always useful to ask ‘why did my opponent make their last move?’ (and then see if there are answers past the obvious one). But I’ve also found it useful in other areas. Several times at work I've gone through iterations of something with someone because I answered exactly what they said instead of what they actually wanted. I now try to stop and ask them what their actual purpose is and it often saves me a bit of work.

"What am I believing that I have not noticed I am believing?"

To be asked whenever one notices oneself doing or feeling something unwelcome, and followed up with the usual "what is true about this and how do I know it?"

Am I doing something right now which I know to be stupid?

You teach students math by putting them in a classroom and having them take notes while a lecturer talks about math. What skill are you actually training? How to take notes.

Also 'how to follow mathematical reasoning,' which can be even more insidious: they feel like they understood all the lectures but then see the exam and have no idea where to start.

In a different scenario they learn to solve problems by doing textbook exercises, spend four years acing courses, then get to grad school and realize they have no idea how to go about research.

How would I update my probabilities if I saw the opposite piece of evidence? What I’m trying to get at here is that “A” and “not A” can’t really be evidences for the same thing. And often it’s more obvious which way “not A” is pointing. A couple of examples:

I saw someone suggesting that maybe a certain Mr. Far Wright was secretly gay because, when the subject was broached, he had publicly expressed his dislike of homosexuality. There was even a wiki page (that I now can’t find) laying out the “law” that the more a person sounds like they hate gays the more likely they are to be gay. At first this sounded appealing*, but then I applied the “not A” test: “if Mr. Far Wright’s sexual orientation is unknown and I heard him publicly declare that he loved homosexual behavior, how would I update the probability that he is gay?” In that case, it seems clear that I’d update it towards him being gay. Therefore, it doesn’t really make sense that when Mr. Wright does that opposite—publicly declaring that he hates homosexual behavior—I also update my probability that he is gay.

Or another recent example I had from talking with someone about Mormonism. Someone said that not having the golden plates available for inspection wasn’t really evidence against Joseph Smith’s story because there were several good reasons why they weren’t available. I was about to concede when I realized that a world where the golden plates were observable would be strong evidence for Joseph Smith’s story so a world where they aren’t has to be at least weak evidence against his story. If A moves the probability quite a bit one way, not A has to at least minimally move the probability the other way.

*Sometimes, if all I can observe, is a denial, it is evidence that the person is guilty. For example, if I walked through the door and the first thing I heard was my toddler denying to my wife that he took the candy, it increases my probability that he did take candy. But too my wife—who already has the evidence that led her to make the accusation—a denial is evidence against him taking the candy (it increases the relative odds that his brother did it instead).

Did I keep all of my reasoning here correct? If not, there might be a better way to express the idea with a Bayesian network.

I'll share the important questions from a game I play:

"If I act now, will I make gains? If not, can I just slowly grow and wait for an opening, or do I have to do something dramatic to even have a chance?"

"I've just finished something - what does that give me an advantage on doing right now?"

"What are the people around me better than me at, so that I can just follow their lead?"

"Are my feelings getting in the way right now?"

What is my goal here?/What counts as winning in this situation?

When thinking about explanations for my feelings or beliefs:

"Is that a cause, or a justification?"

It's usually important to consider both kinds of explanations. Mistaking one for the other is an especially nasty trap -- rationalization on one hand, miscalibration on the other.

Examples:

Say I'm explaining to someone why I always take a particular bike route to work -- "it's less stressful to cross the major streets this way." OK, that's a justification. What might be a cause? Well, it could be that my brain evaluated alternative routes, considered all arguments, and came to the conclusion that this was the best route. On the other hand, maybe when I ask myself this question, I remember that when I started biking, this was the only feasible route due to construction. So maybe there's a causal factor here unrelated to the optimality of the route -- I haven't changed routes out of habit. That's a red flag that I might be rationalizing, so I start more seriously considering alternate routes...

Or say I'm anxious about a talk I'm preparing. I ask myself why I'm anxious, and my brain comes back with "every time you think about it, you vividly recall how someone asked a question you couldn't answer at your last talk." That sounds more like a cause -- and on reflection, it doesn't justify being so anxious. (That realization doesn't immediately cure the anxiety, of course, but it gives me a lever to recalibrate it. I'll also put more thought into what questions might come up this time.)

List of questions for approaching and contextualizing problems I bookmarked a while ago:

21 Solution-Focused Techniques

Some of them seem to be for a setting where you're troubleshooting someone else's problems, but many seem like they'd also be useful as self-calibration questions.

"Is this a science problem, or an engineering problem?"

Reading the lengthy discussion of this distinction in Eric Drexler's Radical Abundance made problems in several projects I've worked on recently seem much more solvable.

From Drexler's blog:

Science and engineering are inseparable domains of thought and action, linked by a shared language of mass and energy, molecules and thermodynamics, physical systems and physical law. This shared language makes communication deceptively easy — easy, because scientists and engineers can see every detail in the same way; deceptive, because they see these details in different contexts, forming different patterns and presenting different problems. In a fundamental sense, science and engineering are antiparallel, facing in opposite directions. The resulting gaps in understanding can open a chasm wide enough to trip a manager, or to swallow a project. [Emphasis added]

[...]

While science aims (ideally) to produce exact descriptions of all parameters of all members of a general class of physical systems, engineering aims to manufacture instances of a single kind of system, making choices to ensure that its functional parameters will equal or exceed those specified by a design description.

Likewise, while science aims to formulate a single theory that exactly fits all parameters of every description, engineering aims to design at least one description of a system having functional parameters that equal or exceed those required by one of a potential multiplicity of system concepts.

In this connection, is a proliferation of possible ways of satisfying a constraint good, or bad? In science, finding more possibilities creates greater uncertainty; in engineering, finding more possibilities provides greater freedom of design. This is a basic question with opposite answers — and there are many more.

Science and engineering share a language of physical systems and physical law, but they ask different questions, seek different knowledge, and serve different ends. The ramifications range from different views of the non-linear system dynamics to differences in working relationships and institutions. The consequences are pervasive and deep, familiar and surprising, and extend far beyond what I have sketched here.

[...]

"What is the experience of the other people I'm interacting with?"

I have sometimes found empathy to be a complicated concept, but this question really cuts to the heart of it and causes your brain to model the situation from the other person's perspective and use that in your decision-making.

Intriguingly, this seems to even apply to interactions with my future selves. If I don't ask this question or one like it, I'm likely to write a massive todo list for myself—probably a completely impossible list, and delegate it to my tomorrow!self with very little thought. Then when that self encounters the list, it's so overwhelming and inconsiderate that I find it hard to deal with at all. If instead, I stop to wonder what the experience will be of my future self encountering the tasks I've delegated for it, I realize that it makes sense to prioritize much of that up front, and to frame the delegation process with more context around why my past self wanted the thing done. This is basically the difference between saying "Future self, do this" and "Future self, at the moment it seems to make sense to me that you do this stuff, for reasons X, Y, Z." or some similarly empathetic request... which is much less likely to produce e.g. reactance.

I'm trying to figure out how to actually get around to installing these into my brain in some sort of useful manner instead of just reading through them and then forgetting all about them.

So now I got a meta-question for each of the questions, "In which situations would I like this question to automatically pop into my head?"

"What am I not noticing?"