Does everyone else here think that putting aside your little quirky interests to do big important things is a good idea? It seems to me that people who choose that way typically don't end up doing much, even when they're strongly motivated, while people who follow their interests tend to become more awesome over time. Though I know Anna is going to frown on me for advocating this path...

Replace FAI with Rapture and LW with Born Again, and you can publish this "very personal account" full of non-sequiturs on a more mainstream site.

The problem with this argument is that you've spent so much emotional effort arguing why the world is screwed without FAI, that you've neglected to hold the claim "The FAI effort currently being conducted by SIAI is likely to succeed in saving the world" to the standards of evidence you would otherwise demand.

Consider the following exercise in leaving a line of retreat: suppose Omega told you that SIAI's FAI project was going to fail, what would you do?

There's a lot to process here, but: I hear you. As you investigate your path, just remember that a) paths that involve doing what you love should be favored as you decide what to do with yourself, because depression and boredom do not a productive person make, and b) if you can make a powerful impact in gaming, you can still translate that impact into an impact on FAI by converting your success and labor into dollars. I expect these are clear to you, but bear mentioning explicitly.

These decisions are hard but important. Those who take their goals seriously must choose their paths carefully. Remember that the community is here for you, so you aren't alone.

I finally realized that when people were talking about donating to SIAI during the rationality minicamp, most of us (certainly myself) were thinking of may be tens of thousands of dollars a year. I now understand that's silly. If our goal is truly to make the most money for SIAI, then the goal should be measured in billions.

Eliezer has said that he doesn't know how to usefully spend more than 10 million dollars...

I wish you well, but be wary. I would guess that many of us on this site had dreams of saving the world when younger, and there is no doubt that FAI appeals to that emotion. If the claims on the SI are true, then donating to them will mean you contributed to saving the world. Be wary of the emotions associated with that impulse. Its very easy for the brain to pick out a train of thoughts and ignore all others- those doubts you admit to may not be entirely unreasonable. Before making drastic changes to your lifestyle, give it a while. Listen to skeptical voices. Read the best arguments as to why donating to SI may not be a good idea (there are some on this very site).

If you are convinced after some time to think that helping the SI is all you want to do with life, then, as Villiam suggests, do something you love to promote it. Donate what you can spare to SI, and keep on doing what makes you happy, because I doubt you will be more productive doing something that makes you miserable. So make those rational board games, but make some populist ones too, because while the former may convert, the latter might generate more income to allow you to pay someone else to convert people.

Oh, wow. I was reading your description of your experiences in this, and I was like, "Oh, wow, this is like a step-by-step example of brainwashing. Yup, there's the defreezing, the change while unfrozen, and the resolidification."

I've spent the past 10 years making games and getting better at it. And just recently I've realized how really really good I actually am at it.

Good enough to make billions and/or impact/recruit many (future) academics? Then do it! Use your superpowers to do it better than before.

And if you are not good enough, then what else will you do? Will you be good enough in that other thing? You should not replace one thing for another just for the sake of replacing, but because it increases your utility function. You should be able to do more in the new area, or the new area should be so significant that even if you do less, the overall result is better.

I have an idea, though I am not sure if it is good and if you will like it. From the reviews it seems to me that you are a great storyteller (except the part of writing dialogs), but your weak point is game mechanics. And if you made a game, you are obviously good at programming. So I would suggest to focus on the mechanical part and, for a moment, to forget about stories. People as SIAI are preparing a rationality curicullum; they try to make exercises that will help people improve some of their skills. I don't know how far they are, but if they already have something... is there a chance you could make a program (not a game, yet) that students could use? Focus on the function, not form; do useful software, not a game. So, step 1, you do something immediately useful for increasing sanity baseline. Step 2, after the rationality curicullum is ready and you have made dozen programs, then think about the game where you could reuse these exercises as parts of the game mechanic. And invite other people to help you (you already know you need someone to help with dialogs; and you probably also need game testers). My point is, first do something useful for teaching rationality, and only then make it a game. Thus the game will not only be about rationality, but it will actually teach some parts of rationality -- and that will be also a great recruiting tool, because if someone liked doing it in the game, then LW will be like an improved version of the game, except that it is also real.

Here's what I was thinking as I read this: Maybe you need to reassess cost/benefits. Apply the Dark Arts to games and out-Zynga Zynga. Highly addictive games with in-game purchases designed using everything we know about the psychology of addiction, reward, etc. Create negative utility for a small group of people, yes, but syphon off their money to fund FAI.

I think if I really, truly believed FAI was the only and right option I'd probably do a lot of bad stuff.

In re FAI vs. snoozing: What I'd hope from an FAI is that it would know how much rest I needed. Assuming that you don't need that snoozing time at all strikes me as a cultural assumption that theories (in this case, possibly about willpower, productivity, and virtue) should always trump instincts.

A little about hunter-gatherer sleep. What I've read elsewhere is that with an average of 12 hours of darkness and an average need for 8 hours of sleep, hunter-gathers would not only have different circadian rhythms (teenagers tend to run late, old people tend to run early), but a common pattern was to spend some hours in the middle of the night for talk, sex, and/or contemplation. To put it mildly, this pattern in not available for the vast majority of modern people, and we don't know what if anything this is costing.

I think of FAI as being like gorillas trying to invent a human-- a human which will be safe for gorillas, but I may be unduly pessimistic.

I'm inclined to think that raising the sanity waterline is more valuable than you do for such a long range project-- FAI is so dependent on a small number of people, and I think it will continue to be so. Improved general conditions means that the odds of someone who would be really valuable not having their life screwed up early are improved.

On the other hand, this is a "by feel" argument, and I'm not sure what I might be missing.

A question that I'm really curious about: Has anyone (SIAI?) created a roadmap to FAI? Luke talks about granularizing all the time. Has it been done for FAI? Something like: build a self-sustaining community of intelligent rational people, have them work on problems X, Y, Z. Put those solutions together with black magic. FAI.

Sometimes, it feels like part of me would take over the world just to get people to pay attention to the danger of UFAI and the importance of Friendliness. Figuratively speaking. And part of me wants to just give up and get the most fun out of my life until death, accepting our inevitable destruction because I can't do anything about it.

So far, seems like the latter part is winning.

I recently had a very similar realization and accompanying shift of efforts. It's good to know others like you have as well.

A couple of principles I'm making sure to follow (which may be obvious, but I think are worth pointing out):

1. Happier people are more productive, so it is important to apply a rational effort toward being happy (e.g. by reading and applying the principles in "The How of Happiness"). This is entirely aside from valuing happiness in itself. The point is that I am more likely to make FAI happen if I make myself happier, as a matter of human psychology. If the reverse were true, and happiness made me less effective, I would apply a rational effort to make myself less happy instead.

2. In line with #1, be aware of the risks of stress, boredom, and burnout. If I hate a certain task, then, even though in most cases it may be the best choice for working toward FAI, it may not be in my case. At the same time, be aware of when this is just an excuse, and when it's possible to change so as to actually enjoy things I otherwise wouldn't have.

I think when LWers say "raise the sanity waterline," there are two ideas being presented. One is to make everyone a little bit more sane. That's nice, but overall probably not very beneficial to FAI cause. Another is to make certain key people a bit more sane, hopefully sane enough to realize that FAI is a big deal, and sane enough to do some meaningful progress on it.

There's another possible scenario: The AI Singularity isn't far, but it is not very near, either. AGI is a generation or more beyond our current understanding of minds, and FAI is a generation or more beyond our current understanding of values. We're making progress; and current efforts are on the critical path to success — but that success may not come during our lifetimes.

Since this is a possible scenario, it's worth having insurance against it. And that means making sure that the next generation are competent to carry on the effort, and themselves survive to make it.

Cultivating a culture of rationality, awareness of existential risks, etc. is surely valuable for that purpose, too.

As a relatively new member of this site, I'm having trouble grasping this particular reasoning and motivation for participating in FAI. I've browsed Eleizer's various writings on the subject of FAI itself, so I have a vague understanding of why FAI is important, and such a vague understanding is enough for me to conclude that FAI is one, if not the most, important topic that currently needs to be discussed. This belief may not be entirely my own and is perhaps largely influenced by the amount of comments and posts in support of FAI, in conjunction with my lack of knowledge in the area.

With this lack of understanding, I think it is clear /why/ I haven't given up my life to support FAI. But it seems to me that many others on this site know much, much more about the subject, and they still have not given up their lives for FAI.

So my brain has made an equivalence between supporting FAI and other acts of extreme charity. I think highly of those who work for years in impoverished countries battling local calamities, but I don't find myself very motivated to participate. From my observations, I think this is because I have never heard of anyone with the goal of saving the world actually making significant progress in that direction. However, I have heard of many people who have made the world a better place while never exhibiting such lofty motivations.

I guess this is similar to cousin_it's response in that it seems strange to me to pursue something because it is a "big important problem". But I am also worried about the following line of reasoning:

Motivation to participate in FAI => motivation to do charitable work => I should be motivated to do all sorts of charitable work.

This seems like it would become reality only if my interests were aligned with the charitable work. In the OP's reasoning, is the motivation to save the world enough to align interest with work? To me, it seems analogous to the effect of a sugar high on your energy level.

If I did find myself working with FAI, it would probably be because I found that these were interesting problems to solve, and not because I wanted to save the world.