Dissolving the Question

Reductionism 101

Followup toHow an Algorithm Feels From the Inside, Feel the Meaning, Replace the Symbol with the Substance

"If a tree falls in the forest, but no one hears it, does it make a sound?"

I didn't answer that question.  I didn't pick a position, "Yes!" or "No!", and defend it.  Instead I went off and deconstructed the human algorithm for processing words, even going so far as to sketch an illustration of a neural network.  At the end, I hope, there was no question left—not even the feeling of a question.

Many philosophers—particularly amateur philosophers, and ancient philosophers—share a dangerous instinct:  If you give them a question, they try to answer it.

Like, say, "Do we have free will?"

The dangerous instinct of philosophy is to marshal the arguments in favor, and marshal the arguments against, and weigh them up, and publish them in a prestigious journal of philosophy, and so finally conclude:  "Yes, we must have free will," or "No, we cannot possibly have free will."

Some philosophers are wise enough to recall the warning that most philosophical disputes are really disputes over the meaning of a word, or confusions generated by using different meanings for the same word in different places.  So they try to define very precisely what they mean by "free will", and then ask again, "Do we have free will?  Yes or no?"

A philosopher wiser yet, may suspect that the confusion about "free will" shows the notion itself is flawed.  So they pursue the Traditional Rationalist course:  They argue that "free will" is inherently self-contradictory, or meaningless because it has no testable consequences.  And then they publish these devastating observations in a prestigious philosophy journal.

But proving that you are confused may not make you feel any less confused.  Proving that a question is meaningless may not help you any more than answering it.

The philosopher's instinct is to find the most defensible position, publish it, and move on.  But the "naive" view, the instinctive view, is a fact about human psychology.  You can prove that free will is impossible until the Sun goes cold, but this leaves an unexplained fact of cognitive science:  If free will doesn't exist, what goes on inside the head of a human being who thinks it does?  This is not a rhetorical question!

It is a fact about human psychology that people think they have free will.  Finding a more defensible philosophical position doesn't change, or explain, that psychological fact.  Philosophy may lead you to reject the concept, but rejecting a concept is not the same as understanding the cognitive algorithms behind it.

You could look at the Standard Dispute over "If a tree falls in the forest, and no one hears it, does it make a sound?", and you could do the Traditional Rationalist thing:  Observe that the two don't disagree on any point of anticipated experience, and triumphantly declare the argument pointless.  That happens to be correct in this particular case; but, as a question of cognitive science, why did the arguers make that mistake in the first place?

The key idea of the heuristics and biases program is that the mistakes we make, often reveal far more about our underlying cognitive algorithms than our correct answers.  So (I asked myself, once upon a time) what kind of mind design corresponds to the mistake of arguing about trees falling in deserted forests?

The cognitive algorithms we use, are the way the world feels.  And these cognitive algorithms may not have a one-to-one correspondence with reality—not even macroscopic reality, to say nothing of the true quarks.  There can be things in the mind that cut skew to the world.

For example, there can be a dangling unit in the center of a neural network, which does not correspond to any real thing, or any real property of any real thing, existent anywhere in the real world.  This dangling unit is often useful as a shortcut in computation, which is why we have them.  (Metaphorically speaking.  Human neurobiology is surely far more complex.)

This dangling unit feels like an unresolved question, even after every answerable query is answered.  No matter how much anyone proves to you that no difference of anticipated experience depends on the question, you're left wondering:  "But does the falling tree really make a sound, or not?"

But once you understand in detail how your brain generates the feeling of the question—once you realize that your feeling of an unanswered question, corresponds to an illusory central unit wanting to know whether it should fire, even after all the edge units are clamped at known values—or better yet, you understand the technical workings of Naive Bayesthen you're done.  Then there's no lingering feeling of confusion, no vague sense of dissatisfaction.

If there is any lingering feeling of a remaining unanswered question, or of having been fast-talked into something, then this is a sign that you have not dissolved the question.  A vague dissatisfaction should be as much warning as a shout.  Really dissolving the question doesn't leave anything behind.

A triumphant thundering refutation of free will, an absolutely unarguable proof that free will cannot exist, feels very satisfying—a grand cheer for the home team.    And so you may not notice that—as a point of cognitive science—you do not have a full and satisfactory descriptive explanation of how each intuitive sensation arises, point by point.

You may not even want to admit your ignorance, of this point of cognitive science, because that would feel like a score against Your Team.  In the midst of smashing all foolish beliefs of free will, it would seem like a concession to the opposing side to concede that you've left anything unexplained.

And so, perhaps, you'll come up with a just-so evolutionary-psychological argument that hunter-gatherers who believed in free will, were more likely to take a positive outlook on life, and so outreproduce other hunter-gatherers—to give one example of a completely bogus explanation.  If you say this, you are arguing that the brain generates an illusion of free will—but you are not explaining how.  You are trying to dismiss the opposition by deconstructing its motives—but in the story you tell, the illusion of free will is a brute fact.  You have not taken the illusion apart to see the wheels and gears.

Imagine that in the Standard Dispute about a tree falling in a deserted forest, you first prove that no difference of anticipation exists, and then go on to hypothesize, "But perhaps people who said that arguments were meaningless were viewed as having conceded, and so lost social status, so now we have an instinct to argue about the meanings of words."  That's arguing that or explaining why a confusion exists.  Now look at the neural network structure in Feel the Meaning.  That's explaining how, disassembling the confusion into smaller pieces which are not themselves confusing.  See the difference?

Coming up with good hypotheses about cognitive algorithms (or even hypotheses that hold together for half a second) is a good deal harder than just refuting a philosophical confusion.  Indeed, it is an entirely different art.  Bear this in mind, and you should feel less embarrassed to say, "I know that what you say can't possibly be true, and I can prove it.  But I cannot write out a flowchart which shows how your brain makes the mistake, so I'm not done yet, and will continue investigating."

I say all this, because it sometimes seems to me that at least 20% of the real-world effectiveness of a skilled rationalist comes from not stopping too early.  If you keep asking questions, you'll get to your destination eventually.  If you decide too early that you've found an answer, you won't.

The challenge, above all, is to notice when you are confused—even if it just feels like a little tiny bit of confusion—and even if there's someone standing across from you, insisting that humans have free will, and smirking at you, and the fact that you don't know exactly how the cognitive algorithms work, has nothing to do with the searing folly of their position...

But when you can lay out the cognitive algorithm in sufficient detail that you can walk through the thought process, step by step, and describe how each intuitive perception arises—decompose the confusion into smaller pieces not themselves confusing—then you're done.

So be warned that you may believe you're done, when all you have is a mere triumphant refutation of a mistake.

But when you're really done, you'll know you're done.   Dissolving the question is an unmistakable feeling—once you experience it, and, having experienced it, resolve not to be fooled again.  Those who dream do not know they dream, but when you wake you know you are awake.

Which is to say:  When you're done, you'll know you're done, but unfortunately the reverse implication does not hold.

So here's your homework problem:  What kind of cognitive algorithm, as felt from the inside, would generate the observed debate about "free will"?

Your assignment is not to argue about whether people have free will, or not.

Your assignment is not to argue that free will is compatible with determinism, or not.

Your assignment is not to argue that the question is ill-posed, or that the concept is self-contradictory, or that it has no testable consequences.

You are not asked to invent an evolutionary explanation of how people who believed in free will would have reproduced; nor an account of how the concept of free will seems suspiciously congruent with bias X.  Such are mere attempts to explain why people believe in "free will", not explain how.

Your homework assignment is to write a stack trace of the internal algorithms of the human mind as they produce the intuitions that power the whole damn philosophical argument.

This is one of the first real challenges I tried as an aspiring rationalist, once upon a time.  One of the easier conundrums, relatively speaking.  May it serve you likewise.

 

Part of the sequence Reductionism

Next post: "Wrong Questions"

Comments

sorted by
magical algorithm
Highlighting new comments since Today at 7:04 AM
Select new highlight date
All comments loaded
"What kind of cognitive algorithm, as felt from the inside, would generate the observed debate about 'free will'?"

I would say: people have mechanisms for causally modeling the outside world, and for choosing a course of action based on its imagined consequences, but we don't have a mechanism for causally modeling the mechanism within us that makes the choice, so it seems as if our own choices aren't subject to causality (and are thus "freely willed").

However, this is likely to be wrong or incomplete, firstly because it is merely a rephrasing of what I understand to be the standard philosophical answer, and secondly because I'm not sure that I feel done.

When you're done, you'll know you're done, but unfortunately the reverse implication does not hold.

So when you have the impression you are done, you are not necessarily done because some have this impression without really being done. But then when you are really done, you won't actually know you are done, because you will realize that this impression of being done can be misleading.

"Many philosophers - particularly amateur philosophers, and ancient philosophers - share a dangerous instinct: If you give them a question, they try to answer it."

This line goes in that book you're going to write.

Eliezer, you seem to be saying that the impression you get when you are really done feels different from the impression you get when you ordinarily seem to be done. But then it should be possible to tell when you just seem to be done, as this impression is different.

Yes, exactly; it feels different and you can tell the difference - but first you have to have experienced both states, and then you have to consciously distinguish the difference and stay on your guard. Like, someone who understands even classical mechanics on a mathematical level should not be fooled into believing that they understand string theory, if they are at all on their guard against false understanding; but someone who's never understood any physics at all can easily be fooled into thinking they understand string theory.

'Free will' is the halting point in the recursion of mental self-modeling.

Our minds model minds, and may model those minds' models of minds, but cannot model an unlimited sequence of models of minds. At some point it must end on a model that does not attempt to model itself; a model that just acts without explanation. No matter how many resources we commit to ever-deeper models of models, we always end with a black box. So our intuition assumes the black box to be a fundamental feature of our minds, and not merely our failure to model them perfectly.

This explains why we rarely assume animals to share the same feature of free will, as we do not generally treat their minds as containing deep models of others' minds. And, if we are particularly egocentric, we may not consider other human beings to share the same feature of free will, as we likewise assume their cognition to be fully comprehensible within our own.

...d-do I get the prize?

...d-do I get the prize?

You have, in the local currency.

So, you are saying that free will is an illusion due to our limited predictive power?

Some rough notes on free will, before I read the "spoiler" posts or the other attempted solutions posted as comments here.

(Advice for anyone attempting reductions/dissolutions of free will or anything else: actually write notes, make them detailed when you can (and notice when you can't), and note when you're leaving some subproblem unsolved for the time being. Often you will notice that you are confused in all kinds of ways that you wouldn't have noticed if you had kept all of it in your head. (And if you're going to try a problem and then read a solution, this is a good way of avoiding hindsight bias.))

What kind of algorithm feels like free will from the inside?

Some ingredients:

  • Local preferences:

    The algorithm doesn't necessarily need to be an optimization process with a consistent, persistent utility function, but when the algorithm runs, there needs to be some locally-usable preference function over outcomes, since this is a decision algorithm.

  • Counterfactual simulation:

    When you feel that you "could" make one of several (mutually exclusive) "choices", that doesn't mean that all of them are actually possible (for most senses of "possible" that we use outside the context of being confused about free will); you're going to end up doing at most one of them. But it occurs to you to imagine doing any of them, because you don't yet know what you'll decide (and you don't know what you'll decide because this imagining is part of the algorithm that generates the decision). So you look at the choices you think you might make, and you imagine yourself making each of those choices. You then evaluate each imagined outcome according to some criterion (specifying which, I think, is far outside the scope of this problem), and the algorithm then returns the choice corresponding to the imagined outcome that maximizes that criterion.

    (Imagining a maybe-impossible world — one where you make a specific decision which may not be the one you will actually make — consists of imagining a world to which all of your prior beliefs about the real world apply, plus an extra assumption about what decision you end up making. If we want to go a bit deeper: suppose you're considering options A, B, and C, and you're imagining what happens if you pick B. Then you imagine a world which is identical to (how you imagine) the real world, except with a different agent substituted for you, identical to you except that their decision algorithm has a special case for this particular situation consisting of "return B".)

    (I realize that I have not unpacked this so-called "imagining" at all. This is beyond my current understanding, and is not specific to the free will issue.)

Why does that feel non-deterministic?

Because we don't have any way of knowing the outcome for sure other than just following the algorithm to the conclusion. Due to the mind projection fallacy, our lack of knowledge of our deterministic decisions feels like those decisions actually not being deterministically implied yet.

...Let me phrase that better: The fact that we don't know what the algorithm will output until we follow it to its normal conclusion, feels like the algorithm not having a definite output until it reaches its conclusion. Since our beliefs about reality just feel like reality, our blank or hazy or changing map of the future feels like a blank or hazy or changing future; as is pointed out in "Timeless Causality", changing our extrapolation of the future feels like changing the future. When you don't know what decision you'll make, that feels like the future itself is undecided. And the fact that we can imagine multiple futures until it's not imaginary or the future anymore, feels like there are multiple possible futures until we pick one to go with.

Why does the idea of determinism feel non-free?

Well, there's the whole metaphor of "laws", to begin with. When we hear about fundamental physical laws, our intuition doesn't go straight to "This is the fundamental framework in which everything in the universe happens (including everything about me)". "Laws" sound like constraints imposed on us. It makes us imagine some causal force acting on us and restricting us from the outside; something that acts independently of and sometimes against mental causes, rather than what you see when you look at mental causes under a microscope (so to speak).

That also seems to explain why people think that physical determinism would preclude moral responsibility. When someone first tells you that everything about you is reducible to lawful physics, it can intuitively sound like being told that you're under the Imperius curse or that you're a puppet and some some demon called "Physics" is pulling the strings. If your intuition says that determinism means people are puppets, then surely it's easy to think that implies people cannot be held responsible for their actions, clearly Physics must get the credit or the blame.

(In one sense, yes, physics must get the credit or blame — but only the region of physics that we call "you" for short.)

And there's the fact that, if it's explained poorly, the idea of physical determinism can sound about the same as the idea of fate. (Or even if it is explained well, but you pattern-match it as "fate" from the beginning and let that contaminate your understanding of the rest of the explanation.) Of course, the ideas couldn't be more different, fate is the idea that your choices don't matter because the outcome will be the same no matter what; and this (rightly) sounds non-free, because it implies that this algorithm you're running doesn't ultimately have any influence on the future. Physical determinism, on the other hand, says quite the opposite, that the future is causally downstream from your actions, which are causally downstream from the algorithm you're running; but given sufficiently confusing/confused descriptions of determinism (like "everything is predetermined"), it is possible to mistake them for each other.

Why does the idea of predictability feel non-free?

The previous bit, on physical determinism feeling non-free, isn't the whole story. Even when the idea of "lawfulness" isn't invoked, people still think as though being theoretically predictable is a limitation on free will. They still wonder things like "If God is omniscient, then he must know every decision I will make, so how can I have free will?" (And atheists say things like this a lot to argue that an omniscient god is impossible because then we couldn't have free will (particularly as an argument against religious traditions that argue (badly) against the problem of evil by saying that God gave us free will). I'm not sure if this is because it's a soldier on their side or if they just don't know reductionism. Probably some of both.) This probably goes back to the bit about the mind projection fallacy; if you don't know what you're going to do, that feels like reality itself being indeterminate, and if you're told that reality itself is not indeterminate — that the territory isn't blank where your map is blank — then, if you haven't learned to strictly distinguish between the map and the territory, you'll say "But I can see plainly that the territory is blank at that point!", and you'll dismiss the idea that your decisions could theoretically be predictable.

(Tangential to the actual reduction, but: this seems like it could be covered by a principle analogous to the Generalized Anti-Zombie Principle. If the thing you think "free will" refers to is something that you'd suddenly have less of if I built a machine that could exactly predict your decisions (even if I just looked at its output and didn't tell you it even existed), then it's clearly not the thing that causes you to think you have "free will".)

Why do we "explain" free will in terms of mysterious substances placed in a separate realm declared unknowable by fiat?

I don't have the cognitive science to answer that, and I'll consider it outside the scope of the free will problem in particular, because that's something we seem to do with everything (as in MAMQ), not just free will.

Most of the proposed models in this thread seem reasonable.

I would write down all the odd things people say about free will, pick the simplest model that explained 90% of it, and then see if I could make novel and accurate predictions based on the model. But, I'm too lazy to do that. So I'll just guess.

Evolution hardwired our cognition to contain two mutually-exclusive categories, call them "actions" and "events."

"Actions" match: [rational, has no understandable prior cause]. "Rational" means they are often influenced by reward and punishment. Synonyms for 'has no understandable prior cause' include 'free will', 'caused by elan vitale' and 'unpredictable, at least by the prediction process we use for things-in-general like rocks'.

"Events" match: [not rational, always directly caused by some previous and intuitively comprehendable physical event or action]. If you throw a rock up, it will come back down, no matter how much you threaten or plead with it.

We are born to axiomatically believe actions we take of this innate 'free will' category have no physical cause. In this model, symptoms might include:

  • believing there is an interesting category called 'free will'

  • believing that arguing whether humans either belong to, or don't belong to, this 'free will' category, is an interesting question

  • believing that if we don't have 'free will', it's wrong to punish people

  • believing that if we don't have 'free will', we are marionettes, zombies, or in some other way 'subhuman'.

  • believing that if we don't understand what causes a thunderstorm or a crop failure or an eclipse, it is the will of a rational agent who can be appeased through the appropriate sacrifices

  • believing that if our actions are caused by God's will, fate, spiritual possession, an ancient prophesy, Newtonian dynamics, or some other simple and easily-understandable cause, we do not have 'free will'. However, if our actions are caused by an immaterial soul, spooky quantum mechanics, or anything else that 'lives in another dimension beyond the grasp of intuitive reason', then we retain 'free will'.

I'm not particularly confident my model is correct, the human capacity to spot patterns where there are none works against me here.

A difference of predictions between Maksym's proposed answer and mine occurs to me. If the sense of free will comes from not being able to model one's own decision process, rather than from taking the intentional stance towards people but not other things, then I would think that each individual would tend to think that she has free will, but other people don't. Since this is not the default view, my answer must be wrong or very incomplete.

Here's my attempt (I haven't read the comments above in detail, as I don't want the answer spoiled in case I'm wrong).

For whatever reason, it is apparent that the conscious part of our brain is not fully aware of everything that our brain does. Now let's imagine our brain executing some algorithm, and see what it looks like from the perspective of our consciousness. At any given stage in the algorithm, we might have multiple possible branches, and need to continue to execute the algorithm along one of those possible branches. To determine which branch to follow, we need to do some computation. But that computation isn't done on a conscious level (or rather, sometimes it is, but the fastest computations are done on a subconscious level). However, the computation is done in parallel, so our consciousness "sees" all of the possible next steps, and then feels as if it is choosing one of them. In reality, that "choice" occurs when all of the subconscious processes terminate and we pick the choice with the highest score.

I think I'll give this a try. Let's start with what a simple non-introspective mind might do:

Init (probably recomputed sometimes, but cached most of the time): I1. Draws a border around itself, separating itself from the "outside world" in its world model. In humans and similarly embodied intelligences you could get away with defining the own body as "inside", if internal muscle control works completely without inspection.

Whenever deciding on what to output: A1. Generates a list of all possible next actions of itself, as determined in I1. For human-like embodieds, could be a list of available body movements. A2. Computes a probability distribution about the likely future states of the world resulting from each action, consulting the internal world-model for prediction. Resolution, temporal range and world-model building are beyond the scope of this answer. A3. Assigns utilities to each considered future state. A4. Assigns preferences to the probability distribution of futures. This could e.g. use Expected Utility or some satisficing algorithm. A5. Chooses the possible next action with the most-prefered distribution of futures. Tie breaking is implementation defined. A6. Execute that action.

As part of A2, the reactions of other intelligent entites are modeled as part of the general world model. This kind of mind architecture does not model itself in the present; that'd lead to infinite recursion: "I'm thinking about myself thinking about myself thinking about ...". It also wouldn't achieve anything, since the mind as instantiated necessarily has a higher resolution than any model of itself stored inside itself. It will, however, model past (for lost data) or future versions of itself.

The important point here is that this mind doesn't model itself while computing the next action. In the extreme case it needn't have facilities for introspection at all. Humans obviously have some such facilities. Either inbetween deciding on output, inbetween the individual steps A1-A6, completely in parallel, or some combination of those, humans spend cputime to analyze the algorithm they're executing, to determine systematic errors or possibilities for optimization. I'll call the neural module / program that does this the introspector. When introspecting a thread which generates motor output (A1-A6 above), one (very effective) assumption of the introspected algorithm will always turn out to be "My own next action is to-be-determined. It'd be ineffective for me to run a model of myself to determine it.". For a mind that doesn't intuitively understand the self-reference and infinite-recursion parts, this turns into "My own actions can't be modeled in advance. I have free will.".

In the cases where the introspected thread is running another instance of the introspector, the introspector still isn't attached to its own thread; doing that would lead to infinite recursion. Each introspector will work similarly to the motor-output-choosing algorithm described above, except that the generated output will be in the form of new mental heuristics. Therefore, the same "It'd be ineffective to run a model of myself to determine my next action." assumption in the algorithm can be observed, and "Free will." is still the likely conclusion of a mind that doesn't understand the rationale behind the design.

I have no idea why or how someone first thought up this question. People ask each other silly questions all the time, and I don't think very much effort has gone into discovering how people invent them.

However, note that most of the silly questions people ask have either quietly gone away, or have been printed in children's books to quiet their curiosity. This type of question- along with many additional errors in rationality- seems to attract people. It gets asked over and over again, from generation unto generation, without any obvious, conclusive results.

The answer to most questions is either obvious, or obviously discoverable- some easy examples are "Does 2 + 2 = 4?", or "Is there a tiger behind the bush?". This question, however, creates a category error in the human linguistic system, by forcibly prying apart the concepts of "sound" and "mental experience of sound". Few people will independently discover that a miscategorization error has occurred; at first, it just seems confusing. And so people start coming up with incorrect explanations, they confuse a debate about the definition of the word "sound" with a debate about some external fact (most questions are about external facts, so this occurs by default), they start dividing into "yes" and "no" tribes, etc.

At this point, the viral meme-spreading process begins. An ordinary question ("Is the sky green?") makes reference to concepts we are already familiar with, and interrelates them using standard methodology. A nonsensical question either makes reference to nonexistent concepts ("Are rynithers a type of plawistre?"), or uses existing concepts in ways that are obviously incorrect ("Is up circular?"). Our mind can deal with these kinds of questions fairly effectively. However, notice the form of a question asked by the tribal chief/teacher/professor/boss: things like "Does electromagnetism affect objects with no net charge?". Even at large inferential distances, the audience will probably pick up on some of the concepts. Most laymen have heard of "electromagnetism" before, and they have a vague idea of what a "charge" is. But they lack the underlying complexity- the stuff beneath the token "electromagnetism"- needed to give a correct answer.

From the inside, this sounds pretty much like the makes-a-sound question: familiar concepts ("tree", "falling", "sound") are mixed together in ways which aren't obviously nonsense, but don't have a clearly defined answer. The brain assumes that it must lack the necessary "underlying knowledge" to get past the confusion, and goes on a quest to discover the nonexistent "knowledge". At the same time, the question conveys an impression of intelligence, and so the new convert tells it to all of his friends and co-workers in an attempt to sound smarter. Many moons ago, this exact question even appeared in a cartoon I saw, as some sort of attempt to get kids to "think critically" or whatever the buzzword was.

Eliezer, you wrote:

But when you're really done, you'll know you're done. Dissolving the question is an unmistakable feeling...

I'm not so sure. There have been a number of mysteries throughout history that were resolved by science, but people didn't immediately feel as if the scientific explanation really resolved the question, even though it does to us now -- like the explanation of light as being electromagnetic waves.

I frequently find it tricky to determine whether a feeling of dissatisfaction indicates that I haven't gotten to the root of a problem, or whether it indicates that I just need time to become comfortable with the explanation. For instance, it feels to me like my moral intuitions are objectively correct rules about how people should and shouldn't behave. Yet my reason tells me that they are simply emotional reactions built into my brain by some combination of biology and conditioning. I've gotten somewhat more used to that fact over time, but it certainly didn't feel at first like it successfully explained why I feel that X is "wrong" or Y is "right."

Eliezer, also, the bet your proposed would only be enforced in situations where I am not dreaming, so it would really be a bet conditional on not dreaming, which defeats the purpose.

Only in humans does it make predictive sense to talk about intent, capability, and inclination, and the wide gap between these kinds of perceived "properties" of fellow socially interacting humans, and the generally much simpler properties seen in inanimate objects and animals, leads the brain to allocate them to widely separated groups of buckets. It is this perceived separation in mental thing-space that leads to the the a free-will boundary being drawn around the cluster of socially interacting humans.

careful there. animistic beliefs are quite widespread in tribal societies, so the notion that the brain allocates two entirely distinct clusters to humans and animals vs. inanimate objects is quite suspect.

I think a brain architecture/algorithm that would debate about free will would have been adapted for large amounts of social interaction in its daily life. This interaction would use markedly different skills (eg language) from those of more mundane activities. More importantly it would require a different level of modeling to achieve any kind of good results. One brain would have to contain models for complicated human social, kin and friendly relationships, as well as models for individuals' personalities.

At the center of the mesh of social interactions would be the tightest wad connections. That would be the brain/person, interacting with and modeling all the other members of their tribe/society. However, their brain cannot model itself modeling others with perfect fidelity, and so many simplifications are made even there. These simplifications pile on top of the perceptual differences that a human sees between (itself, other humans) and (everything else). A whole different mental vocabulary arises between descriptions/models of fellow humans and descriptions/models of everything else. Only in humans does it make predictive sense to talk about intent, capability, and inclination, and the wide gap between these kinds of perceived "properties" of fellow socially interacting humans, and the generally much simpler properties seen in inanimate objects and animals, leads the brain to allocate them to widely separated groups of buckets. It is this perceived separation in mental thing-space that leads to the the a free-will boundary being drawn around the cluster of socially interacting humans. When this boundary is objected to, people go their natural arguing ways.

This is just a first attempt, so I think I may have fallen for some of the traps specifically proscribed against in the post. I hope others will attempt to put up their own explanations and maybe even poke some holes in mine :)

I'll definitely pay attention to further comments on this homework assignment.

HOMEWORK REPORT

With some trepidation! I'm intensely aware I don't know enough.

"Why do I believe I have free will? It's the simplest explanation!" (Nothing in neurobiology is simple. I replace Occam's Razor with a metaphysical growth restriction: Root causes should not be increased without dire necessity).

OK, that was flip. To be more serious:

Considering just one side of the debate, I ask: "What cognitive architecture would give me an experience of uncaused, doing-whatever-I-want, free-as-a-bird Capricious Action that is so strong that I just can't experience (be present to) being a fairly deterministic machine?"

Cutting it down to a bare minimum: I imagine that I have a Decision Module (DM) that receives input from sensory-processing modules and suggested-action modules at its "boundary", so those inputs are distinguishable from the neuron-firings inside the boundary: the ones that make up the DM itself. IMO, there is no way for those internal neuron firings to be presented to the input ports. I guess that there is no provision for the DM to sense anything about its own machinery.

By dubious analogy, a Turing machine looks at its own tapes, it doesn't look at the action table that determines its next action, nor can it modify that table.

To a first approximation, no matter what notion of cause and effect I get, I just can't see any cause for my own decisions. Even if somebody asks, "Why did you stay and fight?", I'm just stuck with "It seemed like a good idea at the time!"

And these days, it seems to me that culture, the environment a child grows up within, is just full of the accouterments of free will: make the right choice, reward & punishment, shame, blame, accountability, "Why did you write on the wall? How could you be so STUPID!!?!!", "God won't tempt you beyond your ability to resist." etc.

Being a machine, I'm not well equipped to overcome all that on the strength of mere evidence and reason.

Now I'll start reading The Solution, and see if I was in the right ball park, or even the right continent.

Thanks for listening.

Eliezer, you seem to be saying that the impression you get when you are really done feels different from the impression you get when you ordinarily seem to be done. But then it should be possible to tell when you just seem to be done, as this impression is different. I can imagine that sometimes our brains just fail to make use of this distinction, but it is quite another to claim that we could not tell when we just seem to be done, no matter how hard we tried.

Robin: So when you have the impression you are done, you are not necessarily done because some have this impression without really being done. But then when you are really done, you won't actually know you are done, because you will realize that this impression of being done can be misleading.

You'd think it would work that way, but it doesn't. Are you awake or asleep right now? When you're asleep and dreaming, you don't know you're dreaming, so how do you know you're awake?

If you claim you don't know you're awake, there's a series of bets I'd like to make with you...

As usual, this is better settled by experiment than by "I just know". My favourite method is holding my nose and seeing if I can still breathe through it. Every time I've tried this while dreaming, I've still been able to breathe, and, unsurprisingly, so far I've never been able to while awake. So if I try that, then whichever way it goes, it's pretty strong evidence. There — now it's science and there's no need to assume "I feel that I know I'm awake" implies "I'm awake".

Of course, if you're the sort of person who never thinks to question your wakefulness while dreaming, then the fact that you've thought of the question at all is good evidence that you're awake. But you need a better experiment than that if you also want to be able to get the right answer while you actually are dreaming.

[Apologies if replying to super-old comments is frowned upon. I'm reading the whole blog from the beginning and occasionally finding that I have things to say.]

Apologies if replying to super-old comments is frowned upon. I'm reading the whole blog from the beginning and occasionally finding that I have things to say.

I have been reading LW since the beginning and have not seen anyone object to replies to super-old comments (and there were 18-month-old comments on LW when LW began because all Eliezer's Overcoming-Bias posts were moved to LW).

Moreover, a lot of reader will see a reply to a super-old comment. We know that because people have made comments in really old comment sections to the effect of "if you see this, please reply so we can get a sense of how many people see comments like this".

Moreover, discouraging replies to super-old comments discourages reading of super-old comments. But reading super-old comments improves the "coherence" of the community by increasing the expected amount of knowledge an arbitrary pair of LW participants has in common. (And the super-old stuff is really good.)

So, reply away, I say.

My favourite method is holding my nose and seeing if I can still breathe through it.

That's awesome.

I never devised anything as cool as that, but I did discover a pretty reliable heuristic: If I ever find myself with any genuine doubt about whether this is a dream, then it definitely is a dream.

Or in other words not feeling like you "just know" you're awake is very strong evidence that you're not.

It's funny that the working reality tests for dreaming are pretty stupid and decidedly non-philosophical. For instance, the virtual reality the brain sets up for dreams apparently isn't good enough to do text or numbers properly, so when you are dreaming you're unable to read the same text twice and see it saying the same thing, and digital clocks never work right. (There's an interesting parallel here to the fact that written language is a pretty new thing in evolutionary scale and people probably don't have that much evolved cognitive capacity to deal with it.)

There's a whole bunch of these: http://en.wikibooks.org/wiki/Lucid_Dreaming/Induction_Techniques#Reality_checks

This reminds me of a horrible nightmare I had back in High School. It was the purest dream I had ever had: the world consisted of me, a sheet of paper, and a mathematical problem. Every time I got to the bottom of the problem, before starting to solve it, I went back up to make sure I had gotten it right... only to find it had CHANGED. That again, and again, and again, until I woke up, a knot in my stomach and covered in sweat.

To realize that this dream has an explanation based on neural architecture rather than on some fear of exams is giving me a weird, tingly satisfaction...

Isn't fear of exams also due to neural architecture?

That's awesome.

It is indeed. I can't take credit for it, though; don't remember where I learned it, but it was from some preexisting lucid dreaming literature. I think it's an underappreciated technique. They usually recommend things like seeing if light switches work normally, or looking at text and seeing if it changes as you're looking at it, but this is something that you can do immediately, with no external props, and it seems to be quite reliable.

I never devised anything as cool as that, but I did discover a pretty reliable heuristic: If I ever find myself with any genuine doubt about whether this is a dream, then it definitely is a dream.

Or in other words not feeling like you "just know" you're awake is very strong evidence that you're not.

That's similar to what I originally did, but it doesn't always work — false awakenings (when you dream that you're waking up in bed and everything's normal) are especially challenging to it. In those cases I usually feel pretty confident I'm awake. Still, that heuristic probably works well for most dreams that are actually dreamlike.

My favourite method is holding my nose and seeing if I can still breathe through it.

"Do you think that's air you're breathing now?"

  • Morpheus

A warning to those who would dissolve all their questions:

Why does anything at all exist? Why does this possibility exist? Why do things have causes? Why does a certain cause have its particular effect?

I find that for questions like these, it is better to ask "how" than to ask "why". When you replace "why" with "how", the questions

Why does anything exist? We observe that everything seems to obey simple mathematical rules. Do these rules become what we observe as reality, and if so, how?

Why does this possibility exist? How is it that we observe only this possibility?

Why do thing have causes? How does causality work?

Why does a certain cause have its particular effect? How does causality work in this particular case?

"Free will" is a black box containing our decision making algorithm.

What kind of mind would invent "free will"? The same mind that would neatly wrap up any other open ended question into a single label, be it "élan vital" or "philogeston". Our minds are fantastic at dreaming up explanations for things, and if they are not easily empirically testable at the time, then such explanations tend to stick. Without falsifying evidence, our pet theories tend to remain, and confirmation bias slowly hardens them into what feels like brute facts.

It's appealing because it ties up (or at least hides) loose ends. If we play taboo on free will, we might get something like the concept that people can narrow a number of possible futures into one future that is optimal. With this definition, free will would indeed exist. If, however, free will was postulated in such a say as to include some fantastical element, or another black box, Occam's razor may strike it down. Alternatively, it may be superficially appealing enough to stick, so long as we don't think about it too thoroughly. For example, the idea that humans are in control of their actions feels like an explanation, but contains control as a nested black box.

But what does this process actually feel like when we make such a mistake? Well, it's based on implicit assumptions, so nothing feels amiss. You don't realize that you are making an implicit assumption. All the loose ends look like they are tucked away, at least at a glance. but if you take a closer look, say by repeatedly asking "why?", then you start to feel less confident. This is a sign of trouble, but you should make sure aren't just asking "does 1 plus 1 really equal 2" in a pretentious tone of voice. If repeated self inquiry seems to be creating a rabbit hole of nested black boxes, then you should go back to the highest level box, and try a different form of inquiry. Ask yourself if there's anything about the nested black boxes that feel wrong. Use all your tools as a rationalist to inquire into this, and hopefully find a path of inquiry besides infinite regression. For a thorough analysis, ask yourself whether the nested boxes make observable predictions, and how those predictions might differ from reality.

With our example of playing taboo on free will to get the idea that humans are in control of their actions, we might intuit that "control" is just an inherently complex concept. Although usually applying agency during an explanation is a good way of sicking Occam's razor on it, perhaps this is an exception. We are discussing agency itself, after all. But what does it mean to have agency? What are the observable differences in the world? When we hold these sorts of questions in our minds, and again try to play taboo, we are more likely to get something like the concept that people can narrow a number of possible futures into one future that is optimal. That's a much different answer, because a computer program could also down-select from a number of different options, given some criteria. This answer also doesn't leave loose ends, and doesn't leave that nagging feeling of doubt that comes from having left something unexplained. It turns out that all our sense of confusion was contained within the mysticism implied by using the phrase “free will”. It may be easy to forget about that doubt when only taking a broad view, but as soon as you zoom in on the problem it will become detectible. We live in a messy world, and so frequently have to say say "good enough" and leave unexplained doubts due to time constraints, but when we decide that something is important and pursue the little doubting feeling to it's conclusion, it can be incredibly satisfying. You've just answered one of the mysteries of the universe, after all.

So that's what it feels like to make and then correct a mistake, but why and how did we make the mistake in the first place? Well, our minds are naturally wired around concepts of agency. This is an observable fact in others, and as in personal examples; it really does feel like the dice are out to get us, or that "the system" must be consciously malicious rather than merely incompetent. It is even more natural to endow ourselves with that same vague agency we give inanimate objects and bureaucratic systems. It's only been in the past couple hundred years that humanity has been able to group everything under the same laws of physics. Before that, the stars obeyed their own special rules, and living things were unexplainable mysteries running on "élan vital" instead of something more akin to a combustion reaction. Unless we specifically look close enough at a belief to notice that requires the world to operate under a different set of physical principles, we will default to what is most natural to believe.

As for why the concept of free will should exist in the first place, it is because it is the most natural explanation. It is a fault in out minds that the most natural is not also the simplest, but it is also a useful feature. This form of vague, associative reasoning let's us jump to reasonably accurate conclusions quickly. The difference between the two breakdowns of free will I gave is that one only uses known, well understood phenomena, and the other revolves around making the concept of agency unexplainable. One might also entangle the question with the concept of self and our individual sense of identity. By rolling a bunch of concepts up into one, there are less easily recognizable loose ends. The problem is that tools like associative reasoning and vague definitions aren't enough to actually arrive at a satisfying answer. It ties up enough loose ends that we declare that we've solved it, and move on, ignoring the incompleteness.

Note that this in itself isn't a completely 100% exhaustive expiation of why we naturally want to believe in free will. Where does this concept come from? How do we form it in the first place? To fully answer these, we'd have to also examine the concept if our personal sense of identity, since that is the thing that gets conflated with the “free will” concept to form the more vague and fluffy version. If someone thinks computers don't or can't have free will but humans can, this is likely what they mean by “free will”. Our sense of identity is a large issue, and well enough outside the scope of the question that I think it isn't necessary for this explanation. I'll leave that particular novel length post for someone else.

Those who dream do not know they dream, but when you wake you know you are awake.

I actually use this fact to enable lucid dreaming. When I'm dreaming, I ask myself, "am I dreaming?" And then I answer yes, without any further consideration, as I've realized that the answer is always yes. Because when I'm awake, I don't ask that question, because there's never any doubt to begin with. So when I'm dreaming and I find myself unsure of whether or not I'm dreaming, I therefore know that I'm dreaming, simply because the doubt and confusion exists. It's a method that's a lot simpler (and more accurate) than trying to analyze the contents of the dream to see if it seems real.

I've been going through the sequences, and this is probably the post I disagree with most.

Philosophy may lead you to reject the concept, but rejecting a concept is not the same as understanding the cognitive algorithms behind it.

More importantly, rejecting a concept doesn't solve the problem the concept is used for. The question to ask isn't what the precise definition of free will is, or whether the concept is coherent. Ask instead "What problems am I trying to solve with this concept?"

Because we do use the concept to solve problems. People take actions, and those actions have effects on us. When do I retaliate against a harmful action, and when do I reward a beneficial action? How do I update my evaluation of the actor, in terms of the likelihood that they will repeat this kind of beneficial or harmful behavior?

It's nonsense to say that free will doesn't exist - because the problem the concept is used to solve does exist, and if you say free will doesn't exist, you'll still end up creating a concept just like it to solve this problem. And for the most part, people are already effectively using the concept to reward and punish appropriately. They are effectively solving a problem. By and large, they use the concept of free will to both accurately predict future behavior (epsitemic rationality) and effectively take action (instrumental rationality). Isn't that what rationality is all about?

Now both sides haven't effectively made that concept coherent with their knowledge of physics, one side thinking the concept is nonsense, and the other side thinking they're not bound by the laws of physics. Both sides are making a mistake. The concept isn't nonsense, because it solves real problems, and we are bound by the laws of physics.

See compatibilism for details. Problem solved. A Rationalist solves problems.