Jan Kulveit

Posts

Sorted by New

Comments

Risks from Learned Optimization: Introduction

I don't see why portion of a system turning into an agent would be "very unlikely". In a different perspective, if the system lives in something like an evolutionary landscape, there can be various basins of attraction which lead to sub-agent emergence, not just mesa-optimisation.