The reason why Bob should be much more skeptical when Alice says "I just got HHHHHHHHHHHHHHHHHHHH" than when she says "I just got HTHHTHHTTHTTHTHHHH" is that there are specific other highish-probability hypotheses that explain Alice's first claim, and there aren't for her second. (Unless, e.g., it turns out that Alice had previously made a bet with someone else that she would get HTHHTHHTTHTTHTHHHH, at which point we should suddenly get more skeptical again.)

Bob's perfectly within his rights to be skeptical, of course, and if the number of coin flips is large enough then even a perfectly honest Alice is quite likely to have made at least one error. But he isn't entitled to say, e.g., that Pr(Alice actually got HTHHTHHTTHTTHTHHHH | Alice said she got HTHHTHHTTHTTHTHHHH) = Pr(Alice actually got HTHHTHHTTHTTHTHHHH) = 2^-20 because Alice's testimony provides non-negligible evidence, because empirically when people report things they have no particular reason to get wrong they're quite often right.

(But, again: if Bob learns that Alice had a specific reason to want it thought she got that exact sequence of flips, he should get more skeptical again.)

So, now suppose Alice says "I just won the lottery" and Amanda says "I just saw a ghost". What should Bob's probability estimates be in the two cases?

Empirically, so far as I can tell, a good fraction of people who claim to have won the lottery actually did so. Of course people sometimes lie, but you have to weigh "most people don't win the lottery on any given occasion" against "most people don't falsely claim to have won the lottery on any given occasion". I guess Bob's posterior Pr(Alice won the lottery) should be somewhere in the vicinity of 1/2. Enough to be decently convinced by a modest amount of further evidence, unless some other hypothesis -- e.g., Alice is trying to scam him somehow, or she's being seriously hoaxed -- gets enough evidence to be taken seriously (e.g., Alice, having allegedly won the lottery, asks Bob for a loan to be repaid with exorbitant interest).

On the other hand, there are lots and lots of tales of ghosts and (at best) very few well verified ones. It looks as if many people who claim to have seen ghosts probably haven't. Further, there are reasons to think it very unlikely that there are ghosts at all (e.g., it seems clear that human thinking is done by human brains, and by definition a ghost's brain is no longer functioning) and those reasons seem quite robust -- they aren't, e.g., dependent on details of our current theories of quantum physics or evolutionary biology. So we should set Pr(ghosts are real) extremely small, and Pr(Amanda reports a ghost | Amanda hasn't really seen a ghost) not terribly small, which means Pr(Amanda has seen a ghost | Amanda reports a ghost) is still small.

Bob's last comparison (claims of seeing ghosts, against actual wins of big lottery prizes) is of course nonsensical, and as long as one of it's of the form "more claims of ghosts than X" it actually goes the wrong way for his purposes. What he wants is more actual sightings of ghosts and fewer claims of ghosts.

I don't see the paradox. P(Alice saw this sequence) is low, and P(Alice presented this sequence) is low, but P(Alice saw this sequence | Alice presented this sequence) is high, so Bob has no reason to be incredulous.

My response is here, a post on my blog from last August.

Basically when Bob sees Alice present the particular sequence, he is seeing something extremely improbable, namely that she would present that individual sequence. So he is seeing extremely improbable evidence which strongly favors the hypothesis that something extremely improbable occurred. He should update on that evidence by concluding that it probably did occur.

Regarding the lottery issue, we have the same situation. If you play the lottery, see the numbers announced, and go, "I just won the lottery!" you are indeed probably wrong. Look again. In most cases you will see that the numbers don't quite match. In the few cases where they do match, you are seeing extremely improbable evidence that you won the lottery, namely that your numbers match after repeated comparisons.

I would personally argue that, even given any particular non-fatal objection to the core of this article, there is something interesting to be found here, if one is charitable. I recommend Chapter 2, Section 4 of Nick Bostrom's Anthropic Bias: Observation Selection Effects in Science and Philosophy, and the citations therein, for further reading. There also might be more recent work on this problem that I'm unaware of. We might refer to this as defining the distinction between surprising and unsurprising improbable events.

It also seems noteworthy that user:cousin_it has done precisely what Bostrom does in his book: sidesteps the issue by focusing on determining the conditional probability implicit in the problem. (In Bostrom's case, however, it is P(There exists an ensemble of universes | An observer observes a fine-tuned universe)). Perhaps this concentration on conditional probabilities is a reduction of the problem, but it does not seem to cause the confusion to evaporate in a puff of smoke, as we should probably expect.

It is true that sometimes humans systematically happen upon wrong-headed ideas, but it may also be the case that user:CronoDAS and others have converged upon a substantial problem.

I have seen this argument on LessWrong before, and don't think the other explanations are as clear as they can be. They are correct though, so my apologies if this just clutters up the thread.

The Bayesian way of looking at this is clear: the prior probability of any particular sequence is 1/2^[large number]. Alice sees this sequence and reports it to Bob. Presumably Alice intends on telling Bob the truth about what she saw, so let's say that there's a 90% chance that she will not make a mistake during the reporting. The other 10% will cover all cases ranging from misremembering/misreading a flip to outright lying. The point is that if Alice is lying, this 10% has to be divided up between the other 2^[large number]-1 other possible sequences - if Alice is going to lie, any particular sequence is very unlikely to be presented by her as the true sequence, since there are a lot of ways for her to lie. So, assuming that Alice was intending to speak the truth, her giving that sequence is very strong (in my example 9*(2^[large number]-1):1) evidence that that particular sequence was indeed the true one over any specific other sequence - 'coincidentally' precisely strong enough to turn the posterior belief of Bob that that sequence is correct to 90%.

A fun side remark is that the above also clearly shows why Bob should be more skeptical when Alice presents sequences like HHHHHHHHHH or HTHTHTHTHTHT - if Alice were planning on lying these are exactly the sequences that she might pick with a greater than uniform probabilty out of all the sequences that were not thrown, and therefore each possible actual sequence contributes a higher-than-average amount of probability that Alice would present one of these special sequences, so the fact that Alice informs Bob of such a sequence is weaker evidence for this particular sequence over any other one than it would be in the regular case, and Bob ends up with a lower posterior that the sequence is actually correct.

Nothing is wrong with this picture -- it's just Bob trolling Alice :-)

Suppose Alice and Bob are the same person. Alice tosses a coin a large number of times and records the results.

Should she disbelieve what she reads?