One of the interesting things about the law is that for core concepts like murder, the rules are well defined and fairly unambiguous,

It a quite controversial discussion about whether or not abortion is murder. I would guess that the current US supreme court would rule it murder or manslaughter to hit a woman who's pregnant in the 8th month against her own preference strong enough to kill her unborn child.

The same goes for the actions of soliders. Is George Bush a murderer because he started an aggressive war against Iraq in which a lot of people died?

If I sequence the DNA of a Neanderthal and then let one be born via a human mother, do I engage in murder when I kill the individual?

Florida's stand your ground law?

Is it murder/manslaughter to do cryonics on a terminally ill person before their brain ceases to produce signals that are visible on an EEG?

Is it murder to defreeze a person from cryonics?

Is the person who doesn't push the fat man murdering the 5 on the train track?

I don't see much FAI-use for mapping human values because I expect to need to solve the value-loading problem via indirect normativity rather than direct specification (see Bostrom 2014).

Also, there is quite a lot of effort going into this already: World Values Survey, moral foundations theory, etc.

"Mapping human values" strikes me as being just as productive an activity for AGI development as mapping constellations is for astronomy: it is superficial and the really important stuff is much deeper. Sample questions: Where do values come from? How does one simulate a value for an optimizer? Is value even the right level of thinking or is it a byproduct of something more fundamental?

I dont understand why we don’t try to prototype a high level ontology of core values

Philosophers and theologians have been doing this for ages -- at least since the time of the Ancient Greeks.

A bit of familiarity with history will illuminate... issues with this approach.

Are you kidding me? I'm staring right now, beside me, at a textbook chapter filled with catalogings of human values, with a list of ten that seem universal, with theories on how to classify values, all with citations of dozens of studies: Chapter 7, Values, of Chris Peterson's A Primer In Positive Psychology.

LessWrong is so insular sometimes. Like lionhearted's post Flashes of Nondecisionmaking yesterday---as if neither he nor most of the commenters had heard that we are, indeed, driven much by habit (e.g. The Power of Habit; Self-Directed Behavior; The Procrastionation Equation; ALL GOOD SELF HELP EVER), and that the folk conception of free will might be wrong (which has been long established; argued e.g. in Sam Harris's Free Will).

Mapping human values is even more difficult than mapping human everyday concepts as e.g. Cyc did/tried. Put vagueness into exact symbolic form. And with vagueness I don't mean 'I don't care' but 'related in a varying way'. Varying with respect to other relations (recursively) and varying with individual differences.

If we really tried to map human values symbolically we'd have to map each individuals values symbolically too and then symbolically aggregate that.

I don't think that we can do that. An AGI could but that is too late.

What we can do is map human values vaguely. We could e.g. train large deep neuronal nets to learn and approximate these concepts from whatever evidence we feed it. And then look at the inferred structure whether it is sufficiently close to what we want. That way we do not have to do the mapping ourselves; only the checking.

There are ambiguous edge cases for murder too: self-defense, abortion, war, hunting, animal experiments, euthanesia...

Legal systems are maps of human values.

...for core concepts like murder, the rules are well defined and fairly unambiguous, whereas more trivial things (in terms of risk to humans) like tax laws, parking laws are the bits that have a lot of complexity to them.

I suspect there's a lot of hidden complexity here. The malice requirement for murder, for example, strikes me as the sort of thing that would be hard to get an algorithm to recognize; similar problems might arise in mapping out the boundary between premeditated and non-premeditated murder (in jurisdictions where it's significant), figuring out culpability in cases of murder by indirect means, determining whether a self-defense claim is justified, etc.

Tax law (e.g.) has more surface complexity, but it also looks more mechanistic to me. I don't think this has to do with risk so much as with its distance from domains we're cognitively optimized for.

A typical (which may not be the most common) related local belief is that it's both possible (practically speaking) and necessary to build an AGI which we know to be more reliable than human brains.

If one posits this, then comparing the initial AGI's results to the output of our own brains and rejecting the AGI if it fails to match is obviously silly. It's kind of like the role of Linnean taxonomy in evaluating DNA-based graphs of species relationships... where they conflict, we simply note that the Linnean taxonomy was wrong and move on.

That's not to say a taxonomy of human value is useless... it might be good for something, just as Linnean taxonomy might have once been. It might document value drift over time, for example, or value variation in different communities.

But given local assumptions, it doesn't do much towards AGI, so it's not too surprising that there's not much effort going in those directions here.