We can "influence" them only insofar we can "influence" what we want or believe: to a very low degree.

It seems instrumental rationality is an even worse tool to classify "irrational" emotions. Instrumental rationality is about actions, or intentions and desires, but emotions are neither of those. We can decide what to do, but we can't decide what emotions to have.

This plan seems to be roughly the same as Yudkowsky's plan.

Assuming that users can figure out intended goals for the AGI that are valuable and pivotal, the identification problem for describing what constitutes a safe performance of that Task, might be simpler than giving the AGI a complete description of normativity in general. [...] Relative to the problem of building a Sovereign, trying to build a Task AGI instead might step down the problem from “impossibly difficult” to “insanely difficult”, while still maintaining enough power in the AI to perform pivotal acts.

This sounds really intriguing. I would like someone who is familiar with natural abstraction research to comment on this paper.

For Jan Leike to leave OpenAI I assume there must be something bad happening internally and/or he got a very good job offer elsewhere.

This post sounds intriguing, but is largely incomprehensible to me due to not sufficiently explaining the background theories.

Gemini also supported audio natively.

Thinking about what's happened with the geometric expectation, I'm wondering how I should view the input utilities. Specifically, the geometric expectation is very sensitive to points assigned zero-utility by any part of the voting measure.

This comment may be relevant here.

For a countable set, a uniform probability distribution is also possible by replacing the axiom of countable additivity with finite additivity. See here. It would mean each element in the countable set has probability 0.

This makes sense from the concept of potential infinity: Take a finite set of size $n$ with uniform probability distribution. As $n$ approaches infinity, the probability of each element approaches 0. Under potential infinity, a countable set is just the infinite limit of a growing finite set, so each element must be assigned zero probability. This means it almost surely doesn't happen, not that it is impossible.

The standard example is an infinite lottery. Insofar such a lottery seems possible in principle, a uniform probability distribution on countable sets must be admitted.

The video linked above also discusses other approaches. The topic has applications in cosmology.