This challenge from 2018 basically asked about building a data set for training AI on human values (loosely construed so as to allow many approaches) and many of the submissions proposed ways to do it. You might find some interesting ideas there.

Caveat, I won the challenge by saying I didn't such an approach would work.

Databases of human behaviour and preferences?

by Stuart Armstrong 1 min read21st Apr 20205 comments

6


Hey there, lesserwrongers!

Starting on some of the computer science/neuroscience of my ideas for deducing human preference.

To do this, it would be useful to have datasets of human behaviour in relatively restricted situations. Possibly datasets of people playing simple games, or solving certain puzzles, or responding to messages, or something similar?

The question is intentionally vague, so that readers can come up with suggestions. What is needed is that the dataset be largish (more than a thousand humans at least), and be of real humans making non-trivial decisions in not-too-complicated circumstances.

Any suggestions?