Guardians of the Gene Pool

25 points

expand_less expand_more

Eliezer_Yudkowsky

Dec 16, 2007

Followup to: Guardians of the Truth

Like any educated denizen of the 21st century, you may have heard of World War II. You may remember that Hitler and the Nazis planned to carry forward a romanticized process of evolution, to breed a new master race, supermen, stronger and smarter than anything that had existed before.

Actually this is a common misconception. Hitler believed that the Aryan superman had previously existed—the Nordic stereotype, the blond blue-eyed beast of prey—but had been polluted by mingling with impure races. There had been a racial Fall from Grace.

It says something about the degree to which the concept of progress permeates Western civilization, that the one is told about Nazi eugenics and hears "They tried to breed a superhuman." You, dear reader—if you failed hard enough to endorse coercive eugenics, you would try to create a superhuman. Because you locate your ideals in your future, not in your past. Because you are creative. The thought of breeding back to some Nordic archetype from a thousand years earlier would not even occur to you as a possibility—what, just the Vikings? That's all? If you failed hard enough to kill, you would damn well try to reach heights never before reached, or what a waste it would all be, eh? Well, that's one reason you're not a Nazi, dear reader.

It says something about how difficult it is for the relatively healthy to envision themselves in the shoes of the relatively sick, that we are told of the Nazis, and distort the tale to make them defective transhumanists.

It's the Communists who were the defective transhumanists. "New Soviet Man" and all that. The Nazis were quite definitely the bioconservatives of the tale.

Part of the Death Spirals and the Cult Attractor subsequence of How To Actually Change Your Mind

Next post: "Guardians of Ayn Rand"

Previous post: "Guardians of the Truth"

25 points

expand_less expand_more

Eliezer_Yudkowsky

Comments

sorted by

Highlighting new comments since Today at 2:13 PM

All comments loaded

5 points

expand_less expand_more

'Progress' is the accumulation of changes towards a pre-defined goal.

Just a few lifespans ago, 'progress' consisted of spreading settlers into sparsely-populated land, killing or driving away the aborigines, draining the wetlands, slaughtering the predators, and converting the ecology into farmland. It was using antibiotics widely and prophylactically, replacing ancient crop strains with monocultures, and designing our communities around the automobile.

'Progress' is the hobgoblin trotted out by everyone who thinks they know what the future should be. Anyone foolish enough to name their political goals 'progressive' ought to be excluded from the political process.

5 points

expand_less expand_more

burger_flipper2

4 points

expand_less expand_more

Relatively new to the forum and just watched the 2 1/2 hour Yudkowsky video on Google. Excellent talk that really helped frame some of the posts here for me, though the audience questions were generally a distraction.

My biggest disappointment was the one question that popped up in my mind while watching and was actually posed wasn't answered because it would take about 5 minutes. The man who asked was told to pose it again at the end of the talk, but did not.

This was the question about the friendly AI: "Why are you assuming it knows the outcome of its modifications?"

Any pointer to the answer would be much appreciated.

4 points

expand_less expand_more

3 points

expand_less expand_more

The Soviets weren't what I'd call transhumanists, because their New Man wasn't a definable goal or factual trend, he was a utopian catch-all of projected virtue. A transhumanist will be able to break down his goals ("uploading") into subgoals ("AI and brain scans") and roughly sketch a research path ("symbolic AI") that would either approach the goal, or fail in an informative way ("combinatorial explosion"). The Soviets could do no such thing, because NSM was nothing definable. He would certainly pop up as a consequence of the experience of enough Marxism. Timescale couldn't be defined. Success could not be predicted until it was encountered. Keep plugging on and have faith.

I call that religion. It isn't set in the real future. It's set in the same never-never land that contains the Second Coming. Importantly, it doesn't lead to a future-oriented culture, which is more precisely a realist goal oriented culture. Nobody works towards NSM (except in propaganda posters) and so he never gets any closer.

3 points

expand_less expand_more

2 points

expand_less expand_more

A minor semantic point: wouldn't advocating a return to the ancient Nordic race make them racial reactionaries rather than racial conservatives?

Taking the British National Party as an example of a racial conservative group, we see that they endeavour to PRESERVE the white race. They believe the master race (or, in this case, the race that somehow deserves ownership of the UK) is extant, and must be protected. On the other hand, the Nazi wished to RESTORE a racial standard that they believed had been long buried.

2 points

expand_less expand_more

2 points

expand_less expand_more

I missed this the first time through:

I asked Mark Plus for ownership of the word and was granted it

I... wow. I don't quite know how to respond to a person who makes a statement such as this.

2 points

expand_less expand_more

2 points

expand_less expand_more

Caledonian a Singularitarian? I doubt he knows what the word means. I don't recall him on the Singularity Institute donors list, or any of the mailing lists or websites.

It's wonderful using words in arguments when you get to redefine them. Who's the source for the alternate definition, the one that replaced "one who believes the concept of a Singularity" and the one more complex than "activist for the Singularity"? Hmmm...

I also love that you equate "working toward bringing about the Singularity" with donating money to your Institute or being on a mailing list.

Seekers of truth do not attempt to hardwire goals and evaluations means into entities they create, whether deistic or merely offspring. Only Guardians value their beliefs so much that they attempt to transmit them as arbitrary, received 'wisdom'.

2 points

expand_less expand_more

2 points

expand_less expand_more

"The moment anyone makes a biased argument because of their attachment to an Idea, they become a Guardian."

I think it's more important what happens when the bias is discovered. Does the group in question reward it or try to eliminate it? For example there is corruption in democracies as well as less free forms of government, what makes the difference is what happens when it is discovered.

2 points

expand_less expand_more

2 points

expand_less expand_more

A Truth-Guardian is someone who 'guards' an Idea by zapping (in its myriad forms) rather than through rational argument.

Are you willing to tell me that you've never met a Singularitarian who has attacked an opponent's authority (zap), or denigrated another's work (zap), or sought to work on their Idea's strong points to the neglect of its weak points (subtle zap), or acted in an elitist manner in order to confer perceived authority on themself (smug zap), or presented new data in such a way as to strengthen their previous predictions (super Bottom Line zap!)? Have you, Eliezer, never ever guarded your view of the future rather than argued dispassionately, even against a plainly wrong argument?

If you say no, I wholly withdraw my (well-meant) comment. Caledonian can be my second example. :p

The moment anyone makes a biased argument because of their attachment to an Idea, they become a Guardian. Singularitarians are people, and they take criticism, and defend their beliefs, requisitely passionately. Apologies if it seemed as though I was singling anybody out for specific criticism of bias - not my intention. For the record, I'm a firm believer. :)

2 points

expand_less expand_more

0 points

expand_less expand_more

A Truth-Guardian is someone who 'guards' an Idea by zapping (in its myriad forms) rather than through rational argument.

How does this idea relate to "Well tended gardens die by pacifism?"

0 points

expand_less expand_more

3 points

expand_less expand_more

Is that an actual question, or an oblique way of suggesting that the thesis of "Well tended gardens die by pacifism" is promoting a form of Truth-Guardianism, and therefore contradicts the thesis of "Guardians of the Truth", and therefore perhaps both theses are flawed?

If it's the latter: yes, yes, very clever.

Assuming charitably that it's the former, my two cents about how they relate:

WTGDBF predicts that where local community norms N1 differ from global norms N2 there's a tendency for N2 to displace N1 whenever the local community interacts with the larger world, and suggests that if I consider N1 superior to N2 I have a moral responsibility to counteract this tendency, which sometimes requires violating N2.
GOTT suggests that certain norms which involve punishing attempts to challenge or question certain ideas regardless of how novel, well-formed, or carefully reasoned those challenges/questions are, are bad for communities that embrace them, despite being well-protected from outside norms.
Combining the two suggests that when I choose to defend my local community norms against corruption by outside norms, I also have a moral responsibility to be right about the superiority of my community's norms.

3 points

expand_less expand_more

1 points

expand_less expand_more

There is some difference between group ideas and group norms, although sometimes these two overlap. There is also a difference between challenging group ideas, and breaking group norms.

An example of a group idea: "It is reasonable to give million dollars to an organization that will freeze your head when you die, because someone might scan your brain and make a machine simulation of you, and it will be really you."

An example of a group norm: "We should refrain from political examples, personal attacks, irrational arguments, etc."

An example of challenging a group idea: "I think the machine simulation is not really you. Even if it is 'alive', it is a new life form; and your old self is dead."

An example of breaking group norms: "This is so stupid!!! I guess you have also voted for [political party]!"

Sometimes these two things can be confused. For example it can be a group norm to never challenge group ideas (or to limit challenging them to ways that have no chance to succeed). This should not happen. On the other hand, it is also very frequent to obviously break group norms and then complain about group's intolerance to challenging its ideas -- this is a typical pattern for many internet trolls, and the community should be able to recognize it.

An example: "Cryonics does not work, f*** you!" "Downvoted for swearing." "You just downvote me because I disagree with you, f*** you!"

1 points

expand_less expand_more

0 points

expand_less expand_more

Yes, agreed with all of this. Though as you suggest, the two can overlap. "Give million dollars to an organization that will freeze your head when you die" can become a group norm, and "refrain from political examples, personal attacks, irrational arguments, etc." can be a group idea. And as you say, it is common for one to be confused for the other, sometimes deliberately for rhetorical effect.

0 points

expand_less expand_more

0 points

expand_less expand_more

Is that an actual question

Yes.

or an oblique way of suggesting that the thesis of "Well tended gardens die by pacifism" is promoting a form of Truth-Guardianism, and therefore contradicts the thesis of "Guardians of the Truth", and therefore perhaps both theses are flawed?

Apparent contradictions are often interesting areas of inquiry. Since I had to join my girlfriend for Phở, I only had time to post the one sentence.

It doesn't mean that both theses are flawed. They could be opposing forces. The apparent contradiction might indicate a point where optimization is challenging. This could explain why groups seem doomed to fall to one pathology or another. There's probably positive feedback in either direction, making groups dynamically unstable on this "axis" -- whatever that might be. Maybe this is to be explained by our group cohesion mechanisms being designed to help us survive when the next tribe over decides to attack, and why things too easily devolve into might makes right?

Combining the two suggests that when I choose to defend my local community norms against corruption by outside norms, I also have a moral responsibility to be right about the superiority of my community's norms.

The last phrase makes me cautious. I think one has a moral responsibility to respect the truth by seeking the truth. If we look at ideologies, how well do they deal with the notion of superiority? How many past notions of superiority seem barbaric? Is there a way of transcending or sidestepping this notion of superiority altogether?

0 points

expand_less expand_more

1 points

expand_less expand_more

Agreed that apparent contradictions are often interesting areas of inquiry.

The only ways I know of to sidestep having to decide which norms best align with my values is to adopt values such that either no community's norms are superior to any others', or such that whatever norms happen to emerge victorious from the interaction of social groups are superior to all the norms they displace. Neither of those tempt me at all, though I know people who endorse both.

If I reject both of those options, I'm left with the possibility that two communities C1 and C2 might exist such that C1's norms are superior to C2's, but the interaction of C1 and C2 results in C1's norms being displaced by C2's.

I don't see a fourth option. Do you?

For example... you say I have a moral responsibility to seek truth, which suggests that if I'm in a community whose values oppose truthseeking in certain areas, I have a moral responsibility to violate my community's norms. No?

1 points

expand_less expand_more

0 points

expand_less expand_more

This has interesting parallels to the Friendly AI problem. For example, one could posit that material wealth might somehow be a suitable arbiter, but I can imagine plenty of situations where C2 displaces C1 (Corporate lobbying?) followed by global ecological catastrophes. Here, dollars take the place of smiley faces strewn across the solar system. Maybe the problem of a sustainably benevolent truth-seeking group is somehow the same problem as FAI on some level?

0 points

expand_less expand_more

0 points

expand_less expand_more

Yes! The problem of Friendly Corporate Behavior is an urgent and unsolved one. (Indeed, corporations have many of the attributes of artificial intelligences, though of course not all.)

The sustainably benevolent moral group is not Friendly AI; it is Friendly NI (natural intelligence). The two problems are probably closely related, but I can see a few important differences: NIs had to evolve, so they're going to start out optimized for reproduction. AIs are designed, so they're optimized for whatever you optimize them for.

0 points

expand_less expand_more

0 points

expand_less expand_more

AIs are designed, so they're optimized for whatever you optimize them for.

My prediction: The ones optimized for reproduction are the ones that will be around in the long term.

0 points

expand_less expand_more

2 points

expand_less expand_more

Not necessarily, because there's no law saying that AIs have to die. This changes the evolutionary calculus significantly; you don't need to reproduce if you can just keep existing and expand your power over the cosmos.

But you're right, insofar as AIs that rapidly self-destruct and never reproduce are not going to stick around long. (I think this is actually a tautology, but it's a tautology with the character of a mathematical theorem---definitely true, but not obvious or trivial.)

It's also worth considering that there are different constraints between NIs and AIs though. NIs have to change gradually, piece by piece, gene by gene. AIs can be radically overhauled in a single generation. This gives them access to places on the fitness landscape that we could never reach---even places that are in fact evolutionarily stable once you get there.

2 points

expand_less expand_more

0 points

expand_less expand_more

Not necessarily, because there's no law saying that AIs have to die. This changes the evolutionary calculus significantly; you don't need to reproduce if you can just keep existing and expand your power over the cosmos.

As wedrifid pointed out, that depends on what one can do about the lightspeed limit. And thermodynamics. I don't think not dying of old age changes evolution that much. Humans are prone to geriatric diseases because evolution can't do much for us past the reproductive years. Beings without a lifespan won't face that.

I highly doubt that no AI won't ever destroy another, though.

It's also worth considering that there are different constraints between NIs and AIs though. NIs have to change gradually, piece by piece, gene by gene. AIs can be radically overhauled in a single generation. This gives them access to places on the fitness landscape that we could never reach---even places that are in fact evolutionarily stable once you get there.

That just means that they'll evolve without the constraints of genetics, much as designs and memes do.

I think it's a mistake to treat superhuman AI as magic. In some contexts it will seem magic, but not all. Human habitations viewed from 10,000 meters look like growths of lichen. In some contexts, some dogs are "smarter" than some people. Human intelligence gives us a tremendous advantage over all other life on Earth, but it is not magic. Superhuman intelligence is not magic. It's just intelligence.

0 points

expand_less expand_more

0 points

expand_less expand_more

you don't need to reproduce if you can just keep existing and expand your power over the cosmos.

Apart from the practical lightspeed limitations. You do need to reproduce or in some other way split yourself into space-separated parts if you wish to expand your power over a sufficient distance.

0 points

expand_less expand_more

0 points

expand_less expand_more

One of our mind children might read this someday and think, "Distance? What a quaint idea!"

0 points

expand_less expand_more

0 points

expand_less expand_more

Solving Friendliness involves capturing desirable ethical guidelines in a robust and sustainable way, so I'd expect the relationship between Friendliness and sustainably benevolent truth-seeking to depend a lot on the relationship between ethics and truth-seeking. I'd agree that they are thematically related, but very much non-identical.

0 points

expand_less expand_more

2 points

expand_less expand_more

"Sometimes it will produce people who guard a specific view of the future."

Anyone read Joseph's post (just above) and immediately think 'Singularitarians!'?

Certainly the majority of (though not all) people I've met who assign that word to themselves are closer to being Guardians than Seekers. Fairly natural in that all causes want to be cults, but still likely to be harmful to the cause.

2 points

expand_less expand_more

Joseph_Hertzlinger

2 points

expand_less expand_more

The example of Communism shows that being future-oriented will not always eliminate the "Guardians of Truth" syndrome. Sometimes it will produce people who guard a specific view of the future.

2 points

expand_less expand_more

1 points

expand_less expand_more

This is somewhat true. (It gets even stranger when you find out that they were also trying to similar things with animals, trying to somehow breed dogs back to the first dog ancestor.) However, it's worth noting that Nazis directly tapped into the common "eugenics" mode of thought in our society, and eugenicists in general were trying to "breed better humans" (by doing things like encouraging the forced sterilization of the insane and the physically disabled, ect).

Of course, it's still a fundamental fail of an idea all the way around. Sure, you could do artificial selection on humans even without understanding genetics, the same way we did with dogs, but for that to work you'd have to have absolute and total control over the reproduction of entire large human sub-populations for dozens of generations, nothing short of that would work. And there's no way a tyrannical govenrment that absurd manages to stay in power for that long. On the scale of the kinds of stuff eugenicists were actually doing, it simply couldn't have a significant effect on the human gene pool in any plausible time-frame.

If you really wanted to try to breed better humans, and you weren't all powerful, probably your best best would be to try to ingrain a powerful and ubiquitous "sexy people are smart" meme into the culture and then keep it alive. If you were able to do that and maintain the meme for 1000 years or so, then sexual selection might start to have an effect to increase the average intelligence of the human race.

1 points

expand_less expand_more

1 points

expand_less expand_more

try to ingrain a powerful and ubiquitous "sexy people are smart" meme into the culture

I've been trying to do that for years. It doesn't seem to be working, as I can't even get laid myself, let alone smart people in general. ;-)

1 points

expand_less expand_more

0 points

expand_less expand_more

Actually, looking at the last couple of decades, I would say that he "smart people are sexy" meme really started to take off around the time that a number of computers science geeks suddenly became some of the richest people in the world.

0 points

expand_less expand_more

1 points

expand_less expand_more

Caledonian, shouldn't you check up on who currently owns the word 'reasoning' before stating that?

I guess there must be some sort of register somewhere...

1 points

expand_less expand_more

1 points

expand_less expand_more

Caledonian,

Yes, it has occurred to 'them'. I hope you haven't read http://www.singinst.org/AIRisk.pdf, since if you have, you haven't grasped the challenge. The crux isn't excluding branches of possible action by an AI, it's ensuring those avenues aren't attractive options for any reason.

1 points

expand_less expand_more

1 points

expand_less expand_more

A fair point, Eliezer. I'd agree that if it weren't for dis/confirmation biases, nothing would ever get done. If Einstein, when questioned about what he would have done if his special theory was disproved, had said 'meh, I can take it or leave it,' he probably wouldn't have had the drive to discover it in the first place. Attachment to your Big Idea is often what drives us.

That said, I don't see that a Big Idea About The Future is so different from a Big Idea About The Past in terms of value for humanity. Both can be open or closed, pacifistic or violent, inclusive or exclusive. It's what you do with it that counts! The question of whether the Singularity as currently defined has positive utility for the human race is not a given, neither will it be unanimous.

I've tried and tried, but I can't think of any other Big Ideas that have stemmed from people looking at where science and technology are going, and extrapolating them to a future point. Perhaps someone who's less hungover can think of one. Office Christmas do last night, still coming around.

Caledonian - I'd say that one of the key concepts in my current understanding of the Singularity is that it's the polar opposite of a hard-wired goal. Surely the very idea is that we don't know what happens inside/beyond a singularity, hence the name?

1 points

expand_less expand_more

1 points

expand_less expand_more

Values (that is, goals of optimizers) are vastly meaningful; they affect the future shape of the universe.

1 points

expand_less expand_more

1 points

expand_less expand_more

Seekers of truth do not attempt to hardwire goals and evaluations means into entities they create, whether deistic or merely offspring. Only Guardians value their beliefs so much that they attempt to transmit them as arbitrary, received 'wisdom'.

Values are not "beliefs", "true", or "false". (What about this is so hard to understand?)

1 points

expand_less expand_more

1 points

expand_less expand_more

Ben, of course no one is 100% un-Guardian-like, but you seemed to be claiming Singularitarians were unusually Guardian-like.

1 points

expand_less expand_more

Richard_Hollerith2

1 points

expand_less expand_more

I have not noticed that, Ben.

Not all of us who believe physics-since-1600 and biology-since-1860 have seen unequivocal progress believe there has been unequivocal progress in popular political or moral opinion. The civil-rights movement of the 1950s and 1960s for example clearly represents an increase in the consistency of the application of the ideal of equality, but it constitutes unequivocal progress only if you believe that the spread of the ideal of equality constitutes unequivocal progress.

1 points

expand_less expand_more

1 points

expand_less expand_more

It is often forgotten in the early days of proto-Nazi racial theory the Prussians were said to be the Master Race because they were a combination of German and Slav! Their combination was supposed to be just right from the perspective of Prussians, reminding me of Charles Murray's "Who wants to be an elephant?". Nietzsche also proposed breeding ubermenschen by giving Prussian officers jewish brides (haven't read him myself, just heard he said this in BG&E).

1 points

expand_less expand_more

FeepingCreature

0 points

expand_less expand_more

Beyond Good and Evil, Aphorism 251. Nietzsche is not entirely serious there.

For the record, Nietzsche's concept of the Overman is primarily a spiritual genesis of a post-human creature that would have the strength to freely chose their own values in pursuit of a higher goal (If I understand it correctly). It has little to do with eugenics.

0 points

expand_less expand_more

0 points

expand_less expand_more

When you're determining the value structure of a mind, "ensuring those avenues aren't attractive options for any reason" IS excluding them from the set of possible courses of action. The key phrase there is for any reason.

As for the rest of your argument, reasoning is precisely what the normal human does NOT do, and it's hilarious that you think logical arguments are what keeps most people from theft.

0 points

expand_less expand_more

0 points

expand_less expand_more

Sure - it's the difference between not stealing because you think you'll get caught and go to prison, and not stealing because you think theft is irrational/immoral/wrong/you name it. The first is sociopathy, the second is what we'd term normal human reasoning. Can I assume you believe there is no such thing as a Friendly AI?

0 points

expand_less expand_more

0 points

expand_less expand_more

caledonian said: "Perhaps the possibility that a consequence of an entity being utterly good might be its being utterly unsafe has never occurred to them."

This describes monotheism rather well. It has occured to me.

0 points

expand_less expand_more