User Comment Replies — AI Alignment Forum

You might be interested in some of my open drafts about optimization;

One distinction that I pretty strongly hold as carving nature at its joint is (what I call) optimization vs agents. Optimization has no concept of a utility function, and it just about the state going up an ordering. Agents are the thing that has a utility function, which they need for picking actions with probabilistic outcomes.

1Matt MacDermott7mo

Nice, I'd read the first but didn't realise there were more. I'll digest later. I think agents vs optimisation is definitely reality-carving, but not sure I see the point about utility functions and preference orderings. I assume the idea is that an optimisation process just moves the world towards states, but an agent tries to move the world towards certain states i.e. chooses actions based on how much they move the world towards certain states, so it make sense to quantify how much of a weighting each state gets in its decision-making. But it's not obvious to me that there's not a meaningful way to assign weightings to states for an optimisation process too - for example if a ball rolling down a hill gets stuck in the large hole twice as often as it gets stuck in the medium hole and ten times as often as the small hole, maybe it makes sense to quantify this with something like a utility function. Although defining a utility function based on the typical behaviour of the system and then trying to measure its optimisation power against it gets a bit circular. Anyway, the dynamical systems approach seems good. Have you stopped working on it?

Optimization Concepts in the Game of Life

Alex_Altair1y2

I feel very on-board with this research aesthetic.

Here are just some nit-picks/notational confusions I had while reading this;

The sequence $p, s t e p (p), s t e p (s t e p (p)), \dots$ , i.e., $n \mapsto {s t e p}^{n} (p)$ , is the computation seeded at $p$ (or a “trajectory” in dynamical systems terminology).
...
A property $P$ is achieved by a computation s if there exists some number of steps $n$ such that $s (n) \in P$ ...

It took me a second to figure out what $s (n)$ referred to, partly because the first s was not rendered in LaTeX, partly because it was n... (read more)

1Victoria Krakovna1y

Thanks Alex for the detailed feedback! I have updated the post to fix these errors. Curious if you have high-level thoughts about the post and whether these definitions have been useful in your work.

Your posts should be on arXiv

Alex_Altair1y39

I would especially especially love it if it popped out a .tex file that I could edit, since I'm very likely to be using different language on LW than I would in a fancy academic paper.

2David Manheim1y

Seconding the .tex export, since it's much more useful than just getting a pdf!

AI ALIGNMENT FORUM
AF

All of Alex_Altair's Comments + Replies