Utility functions are really bad match for human preferences, and one of the major premises we accept is wrong.

They may be a bad descriptive match. But in prescriptive terms, how do you "help" someone without a utility function?

You want a neuron dump? I don't have a utility function, I embody one, and I don't have read access to my coding.

I've put a bit of thought into this over the years, and don't have a believable theory yet. I have learned quite a bit from the excercise, though.

1) I have many utility functions. Different parts of my identity or different frames of thought engage different preference orders, and there is no consistent winner. I bite this bullet: personal identity is a lie - I am a collective of many distinct algorithms. I also accept that Arrow’s impossibility theorem applies to my own decisions.

2) There are at least three dimensions (time, intensity, and risk) to my utility curves. None of these are anywhere near linear - the time element seems to be hyperbolic in terms of remembered happiness for past events, and while I try to keep it sane for future events, that's not my natural state, and I can't do it for all my pieces with equal effectiveness.

3) They change over time (which is different than the time element within the preference space). Things I prefer now, I will not necessarily prefer later. The meta-utility of balancing this possibly-anticipated change against the timeframe of the expected reward is very high, and I can sometimes even manage it.

Here's one data point. Some guidelines have been helpful for me when thinking about my utility curve over dollars. This has been helpful to me in business and medical decisions. It would also work, I think, for things that you can treat as equivalent to money (e.g. willingness-to-pay or willingness-to-be-paid).

1. Over a small range, I am approximately risk neutral. For example, a 50-50 shot at $1 is worth just about $0.50, since the range we are talking about is only between $0 and $1. One way to think about this is that, over a small enough range, there isn't much practical difference between a curve and a straight line approximating that curve. Over the range -$10K and +$20K I am risk neutral.

2. Over a larger range, my utility curve is approximately exponential. For me, between -$200K and +$400K, my utility curve is fairly close to u(x) = 1 - exp (-x/400K). The reason is that, for me, changing my wealth by a relatively small amount won't radically change my risk preference, and that implies an exponential curve. Give me $1M and my risk preferences might change, but within the above range, I pretty much would make the same decisions.

3. Outside that range, it gets more complicated than I think I should go into here. In short, I am close to logarithmic for gains and exponential for losses, with many caveats and concerns (e.g. avoiding the zero illusion. My utility curve should not have any sort of "inflection point" around my current wealth; there's nothing special about that particular wealth level).

(1) and (2) can be summarized with one number, my risk tolerance of $400K. One way to assess this for yourself is to ask "Would I like a deal with a 50-50 shot at winning $X versus losing $X/2?" The X that makes you indifferent between having the deal and not having the deal is approximately your risk tolerance. I recommend acting risk neutral for deals between $X/20 and minus $X/40, and use an exponential utility function between $X and minus $X/2. If the numbers get too large, thinking about them in dollars per year instead of total dollars sometimes helps. For example, $400K seems large, but $20K per year forever may be easier to think about.

Long, incomplete answer, but I hope it helps.

thinking "If I had X, would I take Y instead", and "If I had Y, would I take X instead" very often resulted in a pair of "No"s

It's a well-known result that losing something produces roughly twice the disutility that gaining the same thing would produce in utility. (I.e., we "irrationally" prefer what we already have.)

I feel some people here are trying to define their utility functions via linear combinations of sub-functions which only depend on small parts of the world state.

Example: If I own X, that'll give me a utility of 5, if I own Y that'll give me a utility of 3, if I own Z, that'll give me a utility of 1.

Problem: Choose any two of {X, Y, Z}

Apparent Solution: {X, Y} for a total utility of 8.

But human utility functions are not a linear combination of such sub-functions, but functions from global World states into the real numbers. Think about the above example with X = Car, Y = Bike, Z = Car keys.

It seems obvious now, but the interdependencies are much more complicated. Like money utility being dependent on the market situation, food utility being dependent on the stuff you ate recently, material (as in building) utility being dependent on available tools and vice versa, internet availability utility being dependent on available computer, power, and time.

This leads me to two possible conclusions

A third possibility: Humans aren't in general capable of accurately reflecting on their preferences.

Utility functions are really bad match for human preferences, and one of the major premises we accept is wrong.

If utility functions are a bad match for human preferences, that would seem to imply that humans simply tend not to have very consistent preferences. What major premise does this invalidate?

This is a good exercise, I'll see what I can do for MY utility function.

First of all, a utility function is a function

f: X --> R

Where X is some set. What should that set be? Certainly it shouldn't be the set of states of the universe, because then you can't say that you enjoy certain processes (such as bringing up a child, as opposed to the child just appearing). Perhaps the set of possible histories of the universe is a better candidate. Even if we identify histories that are microscopically different but macroscopically identical, and apply some crude time horizon, we still have a GARGANTUAN set, probably of the order of 10^(10^10) elements.

We thus need a way of compressing the amount of numerical assignment we have to do. We can do this by saying things like

I assign utility $k to having a loving relationship with a partner with qualities a, b, c ... irrespective of almost any other factors

Moving away from abstract mathematical considerations to more pragmatic ones, the main tradeoff that I can't decide upon is how to trade personal gains such as status, wealth, nice social circle and partner against saving the world.

I realize that my utility function is inscrutable and I trust the unconscious par of me to make accurate judgments of what I want. When I've determined what I want, I use the conscious part of me to determine how I'll achieve it.

Interesting exercise. After trying for a while I completely failed; I ended up with terms that are completely vague (e.g. "comfort"), and actually didn't even begin to scratch the surface of a real (hypothesized) utility function. If it exists it is either extremely complicated (too complicated to write down perhaps) or needs "scientific" breakthroughs to uncover its simple form.

The result was also laughably self-serving, more like "here's roughly what I'd like the result to be" than an accurate depiction of what I do.

The real heresy is that this result does not particularly frighten or upset me. I probably can't be a "rationalist" when my utility function doesn't place much weight on understanding my utility function.

Can you write your own utility fuinction or adopt the one you think you should have? Is that sort of wholesale tampering wise?

What counts as a "successful" utility function?

In general terms there are two, conflicting, ways to come up with utility functions, and these seem to imply different metrics of success.

1. The first assumes that "utility" corresponds to something real in the world, such as some sort of emotional or cognitive state. On this view, the goal, when specifying your utility function, is to get numbers that reflect this reality as closely as possible. You say "I think x will give me 2 emotilons", and "I think y will give me 3 emotilons"; you test this by giving yourself x, and y; and success is if the results seem to match up.

2. The second assumes that we already have a set of preferences, and "utility" is just a number we use to represent these, such that xPy <=> u(x)>u(y), where xPy means "x is preferred to y". (More generally, when x and y may be gambles, we want: xPy <=> E[u(x)]>E[u(y)]).

It's less clear what the point of specifying a utility function is supposed to be in the second case. Once you have preferences, specifying the utility function has no additional information content: it's just a way of representing them with a real number. I guess "success" in this case simply consists in coming up with a utility function at all: if your preferences are inconsistent (e.g. incomplete, intransitive, ...) then you won't be able to do it, so being able to do it is a good sign.

Much of the discussion about utility functions on this site seems to me to conflate these two distinct senses of "utility", with the result that it's often difficult to tell what people really mean.

So, we're just listing how much we'd buy things for? I don't see why it's supposed to be hard.

I guess it gets a bit complicated when you consider combinations of things, rather than just their marginal value. For example, once I have a computer with an internet connection, I care for little else. Still, I just have to figure out what would be about neutral, and decide how much I'd pay an hour (or need to be payed an hour) to go from that to something else.

Playing a vaguely interesting game on the computer = 0.

Doing something interesting = 1-3.

Talking to a cute girl = 5.

Talking to a cute girl I know = 8.

Talking to the girl I really like = 50.

Thinking about a girl I really like if I talked to her within the last couple of days, or probably will within a couple of days = 4.

Having hugged the girl I really like within the last two hours = 50.

Hugging a cute girl I know = 50. Note that this one only lasts for about a second, so it's only about a 7000th as good as the last one.

Hugging a cute girl I don't know = 20.

Hugging anyone else except my brother = 10.

Homework = -2, unless it's interesting.

Eating while hungry = 2.

Asleep = ??? I have no idea how to figure that one out.

I didn't use dollar value because I'm too cheap to actually spend money. Knowing how much I can help people for the same cost will do that to you. Check out the Disease Control Priorities Project (http://tinyurl.com/y9wpk5e). There's one for $3 a QALY. Even the hugging the girl I like I only estimate at 0.02 QALYs.

Using that estimate, one unit is about twice my average happiness. More accurate than I'd expect.