* Rather, making deontology play well with differentiable systems trained end-to-end.

This post is part of my hypothesis subspace sequence, a living collection of proposals I'm exploring at Refine. Preceded by oversight leagues, and followed by representational tethers.

TL;DR: An ideological inference engine is a mechanism for automatically refining a given propositional representation of human values (e.g. a normative charter, a debate stance) in an attempt to disambiguate and generalize it to novel situations. While the inference algorithm and the seed representation form the crux of the system, a multi-modal entailment verifier is employed to order possible agent behaviors based on their compatibility with the estimated ideology. This proposal then describes a way of instilling deontological drives in prosaic systems while maintaining the appeal of end-to-end differentiation. Ideological inference engines draw on ideas from traditional expert systems, but replace much of the clunky symbolic manipulation with contemporary LLMs and NLI models.

Intro

Ideological inference engines are a slightly more general framework than oversight leagues, in the sense they rely on several global assumptions, but each more concrete instance of the proposal requires new assumptions when designing the seed representation, the inference algorithm, and the entailment verifier. Here's a non-exhaustive list of global assumptions:

Assumption 1, "Small Seed To Big Tree": Given a suitable inference algorithm and a finite propositional representation of human values, it is possible to estimate human values arbitrarily well given arbitrary amounts of compute. "Arbitrarily well" refers to there being an arbitrarily low error in the estimation. In the limit, the seed knowledge base would grow into the True Name of human values in propositional form.

Assumption 2, "Linear Capability Ordering": Similar to the assumption invoked in oversight leagues, this states that a system composed of a fixed knowledge base and a fixed entailment verifier would eventually be gamed by an agent whose capability is constantly increasing. This is due to the more complex agent becoming able to exploit the inaccuracies of the knowledge base with respect to actual human values.

Assumption 3, "Quantifiable Propositional-Behavioral Gap": The compatibility of a proposed behavioral sequence and a propositional representa...

Deontology