The Ten Commandments of Rationality

(Disclaimer/TL;DR: This article, much like Camelot, is a silly place/post. Nonetheless I think it presents a pretty solid list of 10 rationality lessons to take away from Less Wrong which must not be forgotten upon pain of eternal damnation/irrationality.)


In a realm not far from here, somewhere within a bustling metropolis, there lies an old and dusty book. It is placed in a most conspicuous location; in the middle of a busy street where countless citizens walk by it every day. Yet none pick it up, for it is placed on a pedestal just high enough that it cannot be reached or seen easily, and the slight inconvenience of standing on one’s toes to reach for it is sufficient to deter most. Yet if a traveller were sufficiently aware to look up and see the book,  and curious enough to reach for it, and willing to suffer the slight discomfort of having to touch its muddy cover to open and read its ancient pages, that one would find within a wealth of wisdom and rationality that would transform the reader’s life forever. For this is the most holy Book of Bayes, and its first and last pages both read thusly:

 

The Ten Commandments of Rationality

 

1)  Thou shalt never conflate the truth or falsehood of a proposition with any other characteristic, be it the consequences of the proposition if it be true, or the consequences of believing it for thyself personally, or the pleasing or unpleasant aesthetics of the belief itself. Furthermore, thou shalt never let thy feelings regarding the matter overrule what thy critical faculties tells thee, or in any other way act as if reality might adjust itself in accordance with thine own wishes.

2)     Thou shalt not accept any imperfect situation if it may be optimized, nor shalt thou abstain from improving upon a situation by imagining ever better options without acting on any of them, nor must thee allow thyself to be paralyzed with fear or apathy or indecision when any action is still superior to doing nothing at all. Thus let it be said: Thou shalt not allow thyself to be beaten by a random number generator.

3)     Thou shalt not declare any matter to be unscientific, or inherently irrational, or a false question, or with any other excuse wilfully close thine own eyes and expel all curiosity regarding the matter before thou hast even asked thyself whether the question is worth answering. To transgress thusly is to forfeit any chance to update thy own beliefs on a matter that is truly unusual to thee.  

4)    Thou shalt not hold goals or beliefs which conflict with each other, in such a manner as to violate most divine transitivity, and thereby set thyself up for most ignominious defeat, and rest easy in knowing this fact. Rather shalt thou engage in mindfulness and self-reflection, and in doing so find thy own true priorities, and solve any inconsistencies in a utility maximising manner so that thou may not fall prey to the wrath of the most holy Dutch Book, which is merciless but just.

5)     Thou shalt never engage in defeatism, nor wallow in ennui or existential angst, or in any other way declare that thy efforts are pointless and that exerting thyself is entirely without merit. For just as it is true that matters may never get to the point where they cannot possibly get any worse, so is it true that no situation is impossible to improve upon. 

6)    Thou shalt never judge a real or proposed action by any metric other than this: The expected consequences of the action, both direct and indirect, be they subtle or blatant, taking into account all relevant information available at the time of deciding and no more or less than this.

7)     Thou shalt never sit back on thy lazy laurels and wait for rationality to come to thee, nor shalt thou declare that thy beliefs must be correct as all others have failed to convince thee of the contrary: The cultivation of thy rationality and the falsification of thine beliefs is thine own most sacred task, which is eternal and never finished, and to leave it to others is to invite doom upon the validity of thine own beliefs and actions, for in this case others will never serve thee as well as thou might serve thyself.

8)    Thou shalt never let argumentation stand in the way of knowledge, nor let knowledge stand in the way of wisdom, nor let wisdom stand in the way of victory, no matter how wise or clever it makes thee feel. Also shalt thou never conflate exceptions for rules or rules for exceptions when arguing any issue, nor bring up minutiae as if they were crucial issues, nor allow oneself to be swept away in arguing for the sake of argumentation, nor act to score cheap and yea also easy points, nor present thy learnings in a needlessly ambiguous manner such as this if it can be helped, or in any other way allow oneself to lose sight of thine most sacred goal, which is victory.

9)     Thou shalt never assign a probability exactly equal to 0 or 1 to any proposition, nor declare to the skies that thy certainty regarding any matter is absolute, nor any derivation of such, for to do so is to declare thyself infallible and is placing thyself above thine most holy lord, Bayes.

10)  Thou shalt never curse thy rationality, and wish for ye immediate satisfaction over thy eventual victory, all for the sake of base emotion, which is transient whereas victory is transcendent. Let it be known that it is an unspoken truth amongst rationalists -indeed it is the first and most elementary rule of rationality and yet oft forgotten by those practiced in the art- that base impulse and most holy reason are as a general rule incompatible, as there cannot be two skies.

 

Such are the Ten Commandments of Rationality. And Lo! If one abides by these rules, then let it be said that they act virtuously, and the heavens shall reward them with the splendour of higher expected utility relative to the counterfactual wherein they did not act virtuously. But to those who do not act virtuously, but rather act with irrationality in their minds and biases in their thinking, and who in doing so break any of the Commandments of Rationality, to them let it be said that they have transgressed against thy lord Bayes, and they shall be smitten by the twin gods of Cause and yea also Effect as surely as if they had smitten themselves. For let it be said: The gods of causality may be blind, but their aim doth be excellent regardless.

 

(All silliness aside, what do you all think? Is this a good list of 10 things to take away from Less Wrong? Do you have a better list? Are posts like these a waste of time? Or, Bayes forbid, did I get my thees and thous wrong somewhere? Let me know in the comments.)

Comments

sorted by
magical algorithm
Highlighting new comments since Today at 9:24 AM
Select new highlight date
Rendering 50/73 comments  show more

Here, let me take a stab:

1) Don't confuse beliefs and values.

2) Be agenty.

3) Never leave information on the table.

4) Strive for consistent beliefs.

5) There is actually a Christian formulation of this one: "thou shalt not blaspheme against the Holy Spirit" (Acquinas interpretation). Judaism and Catholicism (perhaps Sufism also, but I am not very familiar with the Sufi tradition so will not comment) have many elements of "proto-rationality", for a number of reasons, one of them being that at one point studying religion was "academia" -- where smart people went.

6) Use CDT :).

7) Having accurate beliefs and completed goals takes work. Remember to work for what you want.

8) Argue collaboratively.

9) Never be certain of anything.

10) Remember to integrate your utility with respect to time.


You know, the difference between people like Dennett or Dawkings and the LW crowd, is that while all are atheists, Dennett and Dawkings genuinely do not miss God or religion. I get the feeling you guys do, with your commandments, and virtues, and solstices, and wedding ceremonies.


I disagree with 4). I think our cognitive architecture is not consistent, and I think wishing it were so is not really very productive. "Man, to thyself be true."

Thank you for that brevity. It makes clear, what can then also be seen in the original, a striking omission: any injunction to pursue the truth, to make one's beliefs correspond with reality. Which highlights the problem with (4): updating towards consistency -- also called decompartmentalising -- while neglecting to update towards reality is a short road to crackpottery.

Twelve Virtues would be a great first chapter in the book. Ten Commandments would be a great last chapter.

The former is about the mindset one needs to start reasoning about rationality; the latter is a list of conclusions to remember.

Ilya's right, it's too long. For example, in Exodus it is written, "Thou shalt not commit adultery." The author doesn't bother defining adultery because his audience shares enough history and culture to know pretty what what's meant. I suspect that you're trying to defend yourself against corner cases and nitpicking. That's a reasonable thing to want (especially in this crowd!) but that's what commentaries are for.

Come to think of it, that division between commandment and commentary might be useful to you. For example, I would rewrite your ninth commandment as, "Thou shalt not assign probability of 0 or 1", and appeal to Rebe Yudkowsky's writings for questions like "but what about epsilon?"

I like that it's self-contained, not a maze of hyperlinks. Could be a bit shorter, though.

Perhaps each point could start with the essence (in bold) and follow with an explanation, like this:

Thou shalt not assign a probability exactly equal to 0 or 1.

Declaring to the skies that thy certainty regarding any matter is absolute is declaring thyself infallible and placing thyself above thine most holy lord, Bayes.

Actually, the Old Testament has three versions of the commandments, each one of different length (Exodus 20, Deuteronomy 5, and the third one I forgot. Fun fact: I learned that at literature lessons in high school, not at any kind of religious lessons). The shorter commandments are the same, but the longer ones differ - maybe it was too difficult even for ancient izraelites to remember them exactly?

Let's try to make some other points shorter.

Number 10. Thou shalt meekly accept battles lost in pursuit of wars won

Number 7. Thou shalt not cease falsificating thine beliefs

I'm only familiar with the two versions of the commandments given in Exodus and Deuteronomy: I specified Exodus specifically to clarify that distinction, then wound up using an example that's the same in both of them. Oh well. I've never heard of a third, though; can you remember any other context?

I'd expect there to be exactly two versions, for the same reason that there are two creation stories in Genesis: the early books of the Bible are the first written form of a faith with two competing (though closely related!) oral traditions.

Anyway, now that I've thought about it more I think this concept would work better as a riff on the book of Proverbs.

Thou shalt never engage in solipsism or defeatism, nor wallow in ennui or existential angst, or in any other way declare that thy efforts are pointless and that exerting thyself is entirely without merit. For just as it is true that matters may never get to the point where they cannot possibly get any worse, so is it true that no situation is impossible to improve upon. Verily, the most blessed of silver linings is the fact that the inherent incertitude of one’s own beliefs also implies that there is never cause for complete hopelessness and despair.

Absolute-certainty/universal applicability red flag raised.

Silver-lining claim red flag raised.

And by far, most importantly: map-territory conflation red flag raised.

Some possible situations truly can't be improved upon. The fact that you must always be uncertain about whether you are really in one is no help. Just a guarantee that in such a situation a rationalist will always have a little bit of false hope.

Upvoted anyway, most of these are good.

Okay, I acknowledge that "no situation is impossible to improve upon" is not strictly speaking true for literally every conceivable situation, but if ever there is a time where it's acceptable to leave out the ol' BOCTAOE for the sake of prose, I'd say a post including this many thees and thous would be it.

I don't think I conflated map and territory though. The statement "There is never cause for complete hopelessness and despair" is a policy recommendation (read it as: "complete hopelessness and despair is never useful"), not a statement about the territory.

I think "Grant me the serenity to accept the things I cannot change, the courage to change the things I can, and wisdom to know the difference." is much better than wanting to optimize everything.

1) Thou shalt never conflate the truth or falsehood of a proposition with any other characteristic, be it the consequences of the proposition if it be true, or the consequences of believing it for thyself personally, or the pleasing or unpleasant aesthetics of the belief itself. Furthermore, thou shalt never let thy feelings regarding the matter overrule what thy critical faculties tells thee, or in any other way act as if reality might adjust itself in accordance with thine own wishes.

The map is not the territory. Rationality is about making effective decisions.

If you as an American ask me whether I come from Berlin, I'm going to say "Yes." I have been born in Berlin. If someone from Berlin asks me I could say: "No. I have been born in Spandau." Spandau is a district of Berlin and there a complex history. Both answer are true because it depends on the context in which the question is asked.

When doing biological modeling there often a tradeoff between complexity of the model and accuracy. Which model you want to use depends on the purpose. If you want to model a whole brain you are going to use a less complex model of a neuron than when you want to model 100 neurons and how those neurons interact with each other.

Beauty is a guiding principle in theoretical physics.

Feeling are a valuable source of information. Shutting down any source of information is no good idea.

6) Thou shalt never judge a real or proposed action by any metric other than this: The expected consequences of the action, both direct and indirect, be they subtle or blatant, taking into account all relevant information available at the time of deciding and no more or less than this.

Basically you are saying that Eliezer is wrong with Timeless decision theory.

The map is not the territory. Rationality is about making effective decisions.

I profess I entirely fail to see how your post refutes the quoted paragraph. Yes, using models is useful, but that is in no way the same as falling prey to wishful thinking. I keep trying to re-read that paragraph to see how it might be interpreted in a way that makes your reply seem natural, but my best guess is that you might have read "Do not let feelings overrule critical thinking or in any other way engage in wishful thinking" as "ignore your feelings". And I still don't see how saying models are useful flows from there.

Basically you are saying that Eliezer is wrong with Timeless decision theory.

As far as I know, that sequence is meant to detail ways in which your actions might have indirect/timeless/acausal consequences, and therefore supplements rather than contradicts consequentialism. If I'm wrong, please explain how and why.