AI ALIGNMENT FORUM
AF

1700
Wikitags

Ontology

Edited by Zach Stein-Perlman last updated 14th Aug 2023

Ontology is a branch of philosophy. It is concerned with questions including how can objects be grouped into categories?

An ontology is an answer to that question. An ontology is a collection of sets of objects (or: a collection of sets of points in thingspace).[1] An agent's ontology determines the abstractions it makes.

For example, consider Alice and Bob, two normal English-fluent humans. "Chairs"_Alice is in Alice's ontology; it is (or points to) a set of (possible-)objects (namely what she considers chairs) that she bundles together. "Chairs"_Bob is in Bob's ontology, and it is a very similar set of objects (what he considers chairs). This overlap makes it easy for Alice to communicate with Bob and predict how he will make sense of the world.

(Also necessary for easy-communication-and-prediction is that their ontologies are pretty sparse, rather than full of astronomically many overlapping sets. So if they each saw a few chairs they would make very similar abstractions, namely to "chairs"_Alice and "chairs"_Bob.)

Why care? Most humans seem to have similar ontologies, but AI systems might have very different ontologies, which could cause surprising behavior. E.g. the panda-gibbon thing. Roughly, if the shared-human-ontology isn't natural (i.e. learned by default) and moreover is hard to teach an AI, then that AI won't think in terms of the same concepts as humans, which might be bad. See Ontology identification problem. Also misgeneralization of concepts or goals.

  1. ^

    "Objects" means possible objects, not objects that really exist.

    An ontology can also include an account of other kinds-of-stuff, like properties, relations, events, substances, and states-of-affairs.

Subscribe
Discussion
1
Subscribe
Discussion
1
Posts tagged Ontology
19The Thingness of Things
TsviBT
3y
18
59Humans provide an untapped wealth of evidence about alignment
TurnTrout, Quintin Pope
3y
42
50Natural Latents: Latent Variables Stable Across Ontologies
johnswentworth, David Lorell
1mo
0
293C's: A Recipe For Mathing Concepts
johnswentworth, David Lorell
1y
0
30Test Cases for Impact Regularisation Methods
DanielFilan
7y
4
38Implications of automated ontology identification
Alex Flint, adamShimi, Robert Miles
4y
7
21Simulacra are Things
janus
3y
7
26General alignment properties
TurnTrout
3y
2
20How toy models of ontology changes can be misleading
Stuart_Armstrong
2y
0
18How are you dealing with ontology identification?
Erik Jenner
3y
1
18Good ontologies induce commutative diagrams
Erik Jenner
3y
0
6A sketch of a value-learning sovereign
jessicata
10y
15
Add Posts