Intertheoretic utility comparison: examples

Stuart_Armstrong

AI ALIGNMENT FORUM
AF

6 Intertheoretic utility comparison: examples

by Stuart_Armstrong

17th Jul 2019

2 min read

6

Personal Blog

Intertheoretic utility comparison: examples

1Davidmanheim

New Comment

1 comment, sorted by

top scoring

Click to highlight new comments since: Today at 9:47 PM

[-]Davidmanheim6y10

This is very interesting - I hadn't thought about utility aggregation for a single agent before, but it seems clearly important now that it has been pointed out.

I'm thinking about this in the context of both the human brain as an amalgamation of sub-agents, and organizations as an amalgamation of individuals. Note that we can treat organizations as rationally maximizing some utility function in the same way we can treat individuals as doing so - but I think that for many or most voting or decision structures, we should be able to rule out the claim that they are following any weighted combination of normalized utilities of the agents involved in the system using any intertheoretic comparison. This seems like a useful result if we can prove it. (Alternatively, it may be that certain decision rules map to specific intertheoretic comparison rules, which would be even more interesting.)

Moderation Log

The methods

All methods presented here obey the axioms of Relevant data, Continuity, Individual normalisation, and Symmetry. Later, we'll see which ones follow Utility reflection, Cloning indifference, Weak irrelevance, and Strong irrelevance.

Max, min, mean

The maximum of a utility function $u$ is ${max}_{s \in S} u (s)$ , while the minimum is ${min}_{s \in S} u (s)$ . The mean of $u$ $\sum_{s \in S} u (s) / | | S | |$ .

The max-min normalisation of $[u]$ is the $u \in [u]$ such that the maximum of $u$ is $1$ and the minimum is $0$ .
The max-mean normalisation of $[u]$ is the $u \in [u]$ such that the maximum of $u$ is $1$ and the mean is $0$ .

The max-mean normalisation has an interesting feature: it's precisely the amount of utility that an agent completely ignorant of its own utility, would pay to discover that utility (as a otherwise the agent would employ a random, 'mean', strategy).

For completeness, there is also:

The mean-min normalisation of $[u]$ is the $u \in [u]$ such that the mean of $u$ is $1$ and the minimum is $0$ .

Controlling the spread

The last two methods find ways of controlling the spread of possible utilities. For any utility $u$ , define the mean difference: $\sum_{s, s^{'} \in S} | u (s) - u (s^{'}) |$ . And define the variance: $\sum_{s \in S} (u (s) - μ)^{2}$ , where $μ$ is the mean defined previously.

These lead naturally to:

The mean difference normalisation of $[u]$ is the $u \in [u]$ such that $u$ has a mean difference of $1$ .
The variance normalisation of $[u]$ is the $u \in [u]$ such that $u$ has a variance of $1$ .

Properties

The different normalisation methods obey the following axioms:

Property	Max-min	Max-mean	Mean-min	Mean difference	Variance
Utility reflection	YES	NO	NO	YES	YES
Cloning indifference	YES	NO	NO	NO	NO
Weak Irrelevance	YES	YES	YES	NO	YES
Strong Irrelevance	YES	YES	YES	NO	NO

As can be seen, max-min normalisation, despite its crudeness, is the only one that obeys all the properties. If we have a measure on $S$ , then ignoring the cloning axiom becomes more reasonable. Strong irrelevance can in fact be seen as an anti-variance; it's because of its second order aspect that it fails this.