Linda Linsefors's Shortform

Linda Linsefors

This is a special post for quick takes by Linda Linsefors. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

What is alignment? (operationalisation)

Toy model: Each agent has a utility function they want to maximise. The input to the utility function is a list of values describing the state of the world. Different agents can have different input vectors. Assume that every utility function monotonically increases, decreases or stays constant for changes in each impute variable (I did say it was a toy model!). An agent is said to value something if the utility function increases with increasing quantity of that thing. Note that if an agents utility function decreases with increasing quantity of a thing, then the agent values the negative of that thing.

In this toy model agent A is aligned with agent B if and only if A values everything B values.

Q: However does this operationalisation match my intuitive understanding of alignment?
A: Good but not perfect.

This definition of alignment is transitive, but not symmetric. This matches the properties I think a definition of alignment should have.

How about if A values a lot of things that B doesn't care about, and only cares very little about the things A cares about? That would count as aligned in this operationalisation but not necessarily match my intuitive understanding of alignment.

What is alignment? (operationalisation second try)

Agent A is aligned with agent B, if and only if, when we give more power (influence, compute, improved intelligence, etc.) to A, then things get better according to B’s values, and this relation holds for arbitrary increases of power.

This operationalisation points to exactly what we want, but is also not very helpful.

1Linda Linsefors4y

Yes, I like this one. We don't want the AI to find a way to give it self utility while making things worse for us. And if we are trying to make things better for us, we don't want the AI to resist us. Do you want to find out what these inequalities implies about the utility functions? Can you find examples where your condition is true for non-identical functions?

1Mateusz Bagiński4y

I don't have a specific example right now but some things that come to mind: * Both utility functions ultimately depend in some way on a subset of background conditions, i.e. the world state * The world state influences the utility functions through latent variables in the agents' world models, to which they are inputs. * UA changes only when MA (A's world model) changes which is ultimately caused by new observations, i.e. changes in the world state (let's assume that both A and B perceive the world quite accurately). * If whenever UA changes UB doesn't decrease, then whatever change in the world increased UA, B at least doesn't care. This is problematic when A and B need the same scarce resources (instrumental convergence etc). It could be satisfied if they were both satisficers or bounded agents inhabiting significantly disjoint niches. * A robust solution seems seems to be to make (super accurately modeled) UB a major input to UA.

Lets say that

U_A = 3x + y

Then (I think) for your inequality to hold, it must be that

U_B = f(3x+y), where f' >= 0

If U_B care about x and y in any other proportion, then B can make trade-offs between x and y which makes things better for B, but worse for A.

This will be true (in theory) even if both A and B are satisfisers. You can see this by assuming replacing y and x with sigmoids of some other variables.

Linda Linsefors's Shortform

2

Linda Linsefors's Shortform

2