Overcoming these biases is very easy if you have an explicit theory which you use for moral reasoning, where results can be proved or disproved. Then you will always give the same answer, regardless of the presentation of details your moral theory doesn't care about.

Mathematicians aren't biased by being told "I colored 200 of 600 balls black" vs. "I colored all but 400 of 600 balls black", because the question "how to color the most balls" has a correct answer in the model used. This is true even if the model is unique to the mathematician answering the question: the most important thing is consistency.

If a moral theory can't prove the correctness of an answer to a very simple problem - a choice between just two alternatives, trading off in clearly morally significant issues (lives), without any complications (e.g. the different people who may die don't have any distinguishing features) - then it probably doesn't give clear answers to most other problems too, so what use is it?

I would assume that detecting the danger of the framing bias, such as "200 of 600 people will be saved" vs "400 of 600 people will die" is elementary enough and so is something an aspired moral philosopher ought to learn to recognize and avoid before she can be allowed to practice in the field. Otherwise all their research is very much suspect.

Anyone wants to organize an experiment?

Heh

Nitpick: Sarunas mentioned it first

So what you're saying is that 60% of the reduction in magical thinking that scientists show compared to the general population is at the 3 second level?

That... seems pretty impressive to me, but I'm not sure what I would have expected it to be.

Remember that you need to put a > in front of each paragraph to do a blockquote in comments.

Where did that assumption come from?

This assumption comes from expecting an expert to know the basics of their field.

If you ask physics professors questions that go counter to human intuition I wouldn't be to sure that they get them right either.

A trained physicist's intuition is rather different from "human intuition" on physics problems, so that's unlikely.

It might be worth saying explicitly what these three (equivalent) axioms say.

• Axiom of choice: if you have a set A of nonempty sets, then there's a function that maps each element a of A to an element of a. (I.e., a way of choosing one element f(a) from each set a in A.)
• Well-ordering principle: every set can be well-ordered: that is, you can put a (total) ordering on it with the property that there are no infinite descending sequences. E.g., < is a well-ordering on the positive integers but not on all the integers, but you can replace it with an ordering where 0 < -1 < 1 < -2 < 2 < -3 < 3 < -4 < 4 < ... which is a well-ordering. The well-ordering principle implies, e.g., that there's a well-ordering on the real numbers, or the set of sets of real numbers.
• Zorn's lemma: if you have a partially ordered set, and every subset of it on which the partial order is actually total has an upper bound, then the whole thing has a maximal element.

The best way to explain what Zorn's lemma is saying is to give an example, so let me show that Zorn's lemma implies the ("obviously false") well-ordering principle. Let A be any set. We'll try to find a well-ordering of it. Let O be the set of well-orderings of subsets of A. Given two of these -- say, o1 and o2 -- say that o1 <= o2 if o2 is an "extension" of o1 -- that is, o2 is a well-ordering of a superset of whatever o1 is a well-ordering of, and o1 and o2 agree where both are defined. Now, this satisfies the condition in Zorn's lemma: if you have a subset of O on which <= is a total order, this means that for any two things in the subset one is an extension of the other, and then the union of all of them is an upper bound. So if Zorn's lemma is true then O has a maximal element, i.e. a well-ordering of some subset of A that extends every possible well-ordering of any subset of A. Call this W. Now W must actually be defined on the whole of A, because for every element a of A there's a "trivial" well-ordering of {a}, and W must extend this, which requires a to be in W's domain.

(A few bits of terminology that I didn't digress to define above. A total ordering on a set is a relation < for which if a<b and b<c then a<c, and for which exactly one of a<b, b<a, a=b holds for any a,b. OR a relation <= for which if a<=b and b<=c then a<=c, and for which for any a,b either a<=b or b<=a, and for which a<=b and b<=a imply a=b. A partial ordering is similar except that you're allowed to have pairs for which a<b and b<a (OR: a<=b and b<=a) both fail. We can translate between the "<" versions and the "<=" versions: "<" means "<= but not =", or "<=" means "< or =". Given a partial ordering, an upper bound for a set A is an element b for which a<=b for every a in A. A maximal element in a partially ordered set is an element of the set that's an upper bound for the whole set.)