After being told whether they are deciders or not, 9 people will correctly infer the outcome of the coin flip, and 1 person will have been misled and will guess incorrectly. So far so good. The problem is that there is a 50% chance that the one person who is wrong is going to be put in charge of the decision. So even though I have a 90% chance of guessing the state of the coin, the structure of the game prevents me from ever having more than a 50% chance of the better payoff.

eta: Since I know my attempt to choose the better payoff will be thwarted 50% of the time, the statement "saying 'yea' gives 0.9*1000 + 0.1*100 = 910 expected donation" isn't true.

I tell you that you're a decider [... so] the coin is 90% likely to have come up tails.

Yes, but

So saying "yea" gives 0.9 1000 + 0.1 100 = 910 expected donation.

... is wrong: you only get 1000 if everybody else chose "yea". The calculation of expected utility when tails come up has to be more complex than that.

Let's take a detour through a simpler coordination problem: There are 10 deciders with no means of communication. I announce I will give $1000 to a Good and Worth Charity for each decider that chose "yea", except if they all chose yea, in which case I will give nothing. Here the optimal strategy is to choose "yea" with a certain probability p, which I don't have time to calculate right now.

But anyway, in coordination problems in general, the strategy is to find what the group strategy should be, and execute that. Calculation of expected utility should not be done on the individual level (because there's no reliable way to account for the decisions of others if they depend on yours), but on the group level.

It is an anthropic problem. Agents who don't get to make decisions by definition don't really exist in the ontology of decision theory. As a decision theoretic agent being told you are not the decider is equivalent to dying.

I claim that the first is correct.

Reasoning: the Bayesian update is correct, but the computation of expected benefit is incomplete. Among all universes, deciders are "group" deciders nine times as often as they are "individual" deciders. Thus, while being a decider indicates you are more likely to be in a tails-universe, the decision of a group decider is 1/9th as important as the decision of an individual decider.

That is to say, your update should shift probability weight toward you being a group decider, but you should recognize that changing your mind is a mildly good idea 9/10 of the time and a very bad idea 1/10 of the time, and that these should balance to you NOT changing your mind. Since we know that half the time the decision is made by an individual, their decision to not change their mind must be as important as all the decisions of the collective the other half the time.

I can't formalize my response, so here's an intuition dump:

It seemed to me that a crucial aspect of the 1/3 solution to the sleeping beauty problem was that for a given credence, any payoffs based on hypothetical decisions involving said credence scaled linearly with the number of instances making the decision. In terms of utility, the "correct" probability for sleeping beauty would be 1/3 if her decisions were rewarded independently, 1/2 if her (presumably deterministic) decisions were rewarded in aggregate.

The 1/2 situation is mirrored here: There are only two potential decisions (ruling out inconsistent responses), each applicable with probability 0.5, each resulting in a single payoff. Since "your" decision is constrained to match the others', "you" are effectively the entire group. Therefore, the fact that "you" are a decider is not informative.

Or from another angle: Updating on the fact that you (the individual) are a decider makes the implicit assumption that your decision process is meaningfully distinct from the others', but this assumption violates the constraints of the problem.

I remain thoroughly confused by updating. It seems to assume some kind of absolute independence of subsequent decisions based on the update in question.

Your second option still implicitly assumes that you're the only decider. In fact each of the possible deciders in each branch of the simulation would be making an evaluation of expected payoff -- and there are nine times as many in the "tails" branches.

There are twenty branches of the simulation, ten with nine deciders and ten with one decider. In the one-decider branches, the result of saying "yea" is a guaranteed $100; in the nine-decider branches, it's $1000 in the single case where everyone agrees, $700 in the single case where everyone disagrees, and $0 otherwise. The chance of defection depends on the strength of everyone's precommitments, and the problem quickly goes recursive if you start taking expected behavior into account, but in the simplified case where everyone has a well-defined probability of defection the expected payoffs are much higher if your precommitments are strong.

If we're assuming that all of the deciders are perfectly correlated, or (equivalently?) that for any good argument for whatever decision you end up making, all the other deciders will think of the same argument, then I'm just going to pretend we're talking about copies of the same person, which, as I've argued, seems to require the same kind of reasoning anyway, and makes it a little bit simpler to talk about than if we have to speak as though that everyone is a different person but will reliably make the same decision.

Anyway:

Something is being double-counted here. Or there's some kind of sleight-of-hand that vaguely reminds me of this problem, where it appears that something is being misplaced but you're actually just being misdirected by the phrasing of the problem. (Not that I accuse anyone of intentionally doing that in any of the versions of this problem.) I can't quite pin it down, but it seems like whatever it is that would (under any circumstance) lead you to assign a .9 decision-theoretic weighting to the tails-world is already accounted for by the fact that there are 9 of you (i.e. 9 who've been told that they're deciders) in that world. I'm not sure how to formally express what I'm getting at, but I think this is moving in the right direction. Imagine a tree of agents existing as a result of the coin flip; the heads branch contains one decider and nine non-deciders; the tails branch contains nine deciders and one non-decider. And each decider needs to have its own judgment of decision-theoretic weighting... but that varies depending on what kind of decision it is. If each one assigns .9 weight to the possibility that it is in the tails branch, then that would be relevant if every agent's decision were to be counted individually (say, if each one had to guess either heads or tails, and would get $1 if correct; they'd do better guessing tails than by flipping a coin to decide), but in this case the decision is collective and only counted once — so there's no reason to count the multiple copies of you as being relevant to the decision in the first place. It's like if in tails-world you run a (constant) program nine times and do something based on the output, and in heads-world you run it once and do something else based on the output. The structure of the problem doesn't actually imply that the algorithm needs to know how many times it's being executed. I think that's the misdirection.

(Edit: Sorry, that was sort of a ramble/stream-of-consciousness. The part from "If each one assigns..." onward is the part I currently consider relevant and correct.)

The first is correct. If you expect all 10 participants to act the same you should not distinguish between the cases when you yourself are the sole decider and when one of the others is the sole decider. Your being you should have no special relevance. Since you are a pre-existing human with a defined identity this is highly counterintuitive, but this problem really is no different from this one: An urn with 10 balls in different colors, someone tosses a coin and draws 1 ball if it comes up head and 9 balls if it comes up tails, and in either case calls out the color of one ball. Suppose that color is red, what is the probability the coin came up tails?

If the coin came up heads there is a 1/10 chance of you being drawn, if it came up tails there is a 9/10 chance of you being drawn, and a 1/9 chance of you being you. Since in your intuition you are always you this probably seems nonsensical. But if you distinguish between yourself and the others a priori you can't really treat the payoff like that.

(No points for saying that UDT or reflective consistency forces the first solution. If that's your answer, you must also find the error in the second one.)

Under the same rules, does it make sense to ask what is the error in refusing to pay in a Counterfactual Mugging? It seems like you are asking for an error in applying a decision theory, when really the decision theory fails on the problem.

So let’s modify the problem somewhat. Instead of each person being given the “decider” or “non-decider” hat, we give the "deciders" rocks. You (an outside observer) make the decision.

Version 1: You get to open a door and see whether the person behind the door has a rock or not. Winning strategy: After you open a door (say, door A) make a decision. If A has a rock then say “yes”. Expected payoff 0.9 1000 + 0.1 100 = 910 > 700. If A has no rock, say “no”. Expected payoff: 700 > 0.9 100 + 0.1 1000 = 190.

Version 2: The host (we’ll call him Monty) randomly picks a door with a rock behind it. Winning strategy: Monty has provided no additional information by picking a door: We knew that there was a door with a rock behind it. Even if we predicted door A in advance and Monty verified that A had a rock behind it, it is no more likely that heads was chosen: The probability of Monty picking door A given heads is 0.9 1/9 = 0.1 whereas the probability given tails is 0.1 1. Hence, say “no”. Expected Payoff: 700 > 0.5 100 + 0.5 1000 = 550.

Now let us modify version 2: During your sleep, you are wheeled into a room containing a rock. That room has a label inside, identifying which door it is behind. Clearly, this is no different than version 2 and the original strategy still stands.

From there it’s a small (logical) jump to your consciousness being put into one of the rock-holding bodies behind the door, which is equivalent to our original case. (Modulo the bit about multiple people making decisions, if we want we can clone you consciousness if necessary and put it into all rock possessing bodies. In either case, the fact that you wind up next to a rock provides no additional information.)

This question is actually unnecessarily complex. To make this easier, we could introduce the following game: We flip a coin where the probability of heads is one in a million. If heads, we give everyone on Earth a rock, if tails we give one person a rock. If the rock holder(s) guesses how the coin landed, Earth wins, otherwise Earth loses. A priori, we very much want everyone to guess tails. A person holding the rock would be very much inclined to say heads, but he’d be wrong. He fails to realize that he is in an equivalence class with everyone else on the planet, and the fact that the person holding the rock is himself carries no information content for the game. (Now, if we could break the equivalence class before the game was played by giving full authority to a specific individual A, and having him say “heads” iff he gets a rock, then we would decrease our chance of losing from 10^-6 to (1-10^-6) * 10^-9.)

The devil is in the assumption that everyone else will do the same thing as you for presumably the same reasons. "Nay" is basically a better strategy in general, though it's not always right. The 90% odds of tails are correctly calculated.

• If you are highly confident that everyone else will say "yea", it is indeed correct to say "yea"
• If you're highly confident that everyone will say the same thing but you're not sure what it is (an unlikely but interesting case), then make a guess (if it's 50-50 then "yea" is better - the cutoff in fact is just a bit above 45% chance of all-yea over all-nay)
• If you don't really know what the others will say, and they're all acting independently, then odds are they've blown it by giving different answers whatever you do. In that case, 1-decider case dominates your decision, for essentially different reasons (so "nay" is better).
• The most difficult case is the more "natural" assumption that the other people are basically idential copies of you. In this case, what you have is a stochastic version of Newcomb's Paradox. Actually it's more like Newcomb's Mugging. Once you know you are a decider, saying "yea" will genuinely give you a better average payout. But, when you're not a decider, you'll see more money donated if you are the sort of person who would say "nay" when you are a decider. It's obvious why that's true if you make the "identical copies" assumption explicitly, because your identical copy will say "nay" and cause a larger donation. So, you expect to make more money saying "yea" once you know you're a decider, but you make more money overall if you're a "nay"-sayer. The trick is that most of the extra money is made in the cases where you're not a decider.

So, nay is definately the better precommitment, and TDT would presumably tell you to say nay. But the argument that, as an individual, your expected returns are better saying "yea" once you know you are a decider is still perfectly valid.

Each decider will be asked to say "yea" or "nay". If the coin came up tails and all nine deciders say "yea", I donate $1000 to VillageReach. If the coin came up heads and the sole decider says "yea", I donate only $100. If all deciders say "nay", I donate $700 regardless of the result of the coin toss. If the deciders disagree, I don't donate anything.

Suppose that instead of donating directly (presumably for tax reasons), you instead divide the contribution up among the deciders, and then let them pass it on. As I figure it, that simple change in the rules eliminates the paradox.

I'm not sure what this means, but I'm going to blame the paradox on the tax-code. :)

Un-thought-out-idea: We've seen in the Dutch-book thread that probability and decision theory are inter-reliant so maybe

Classical Bayes is to Naive Decision Theory

as

Bayes-we-need-to-do-anthropics is to UDT

Actually now that I've said that it doesn't seem to ground-breaking. Meh.

I would decide "nay". Very crudely speaking a fundamental change in viewpoints is involved. If I update on the new information regarding heads vs tales I must also adjust my view of what I care about. It is hard to describe in detail without writing an essay describing what probability means (which is something Wei has done and I would have to extend to allow for the way to describe the decision correctly if updating is, in fact, allowed.).