Sequence Summary. This sequence documents research by SatisfIA, an ongoing project on non-maximizing, aspiration-based designs for AI agents that fulfill goals specified by constraints ("aspirations") rather than maximizing an objective function. We aim to contribute to AI safety by exploring design approaches and their software implementations that we believe might be promising but neglected or novel. Our approach is roughly related to but largely complementary to concepts like quantilization and satisficing (sometimes called "soft-optimization"), Decision Transformers, and Active Inference.

This post describes the purpose of the sequence, motivates the research, describes the project status, our working hypotheses and theoretical framework, and has a short glossary of terms. It does not contain results and can safely be skipped if you want to get directly into the actual research.

Epistemic status: We're still in the exploratory phase, and while the project has yielded some preliminary insights, we don't have any clear conclusions at this point. Our team holds a wide variety of opinions about the discoveries. Nothing we say is set in stone.

Purpose of the sequence

Inform: We aim to share our current ideas, thoughts, disagreements, open questions, and any results we have achieved thus far. By openly discussing the complexities and challenges we face, we seek to provide a transparent view of our project's progression and the types of questions we're exploring.
Receive Feedback: We invite feedback on our approaches, hypotheses, and findings. Constructive criticism, alternative perspectives, and further suggestions are all welcome.
Attract Collaborators: Through this sequence, we hope to resonate with other researchers and practitioners who our exploration appeals to and who are motivated by similar questions. Our goal is to expand our team with individuals who can contribute their unique expertise and insights.

Motivation

We share a general concern regarding the trajectory of Artificial General Intelligence (AGI) development, particularly the risks associated with creating AGI agents designed to maximize objective functions. We have two main concerns:

(I) AGI development might be inevitable

(We assume this concern needs no further justification)

(II) It might be impossible to implement an objective function the maximization of which would be safe

The conventional view ...

Posts

Wikitag Contributions

Comments