1. Introduction

This post outlines two models from the social epistemology of science explaining the emergence of the a particular openness norm within the sciences, and then looks at how such models can be utilised to understand research groups trying to develop AGI. In the rest of the introduction, I will try to provide some motivation for this post. Sections 2 & 3 will briefly outline the two models I'm looking at. Section 4 more directly tries to interpret such models in the context of AGI development. Section 5 concludes.

The social epistemology of science is an interdisciplinary subfield at the intersection of philosophy and economics, which utilises formal models to understand the incentive structure of science. Here, I focus on two models from this area which try to explain the emergence of one particular openness norm: the so-called ‘communist norm’ in scientific research. The communist norm is a norm to share all ‘substantive findings’ with the scientific community. The existence of this norm seems to be taken for granted in this literature, although the best piece of evidence I can find for its existence comes from Louis et. al (2001), who find, in a sample of nearly 2,000 geneticists, that 91% agree that one should share all of one's relevant data. I nevertheless take it for granted in this post.

I wanted to see whether understanding the emergence of the communist norm in science could be important for understanding the development of AGI. In many ways, one might think the incentive structures around the development of AGI (will, or does) parallel the incentive structures of academic science. Thus, one might think that looking at the incentive structures behind scientific research are a good starting point for looking at the incentive structures surrounding the development of AGI.

As the communist norm emerged in science, one can imagine the emergence of a similar ‘communist norm’ across research groups involved in AGI development, where research groups share all relevant research with one another. In the context of AGI, one might worry that sharing all research would speed up development alongside safety, potentially leading to a suboptimal scenario where capabilities improve faster than safety. If the incentive structures surrounding the development of AGI are appropriately similar to the i...

Posts

Wikitag Contributions

Comments