This is a brief summary post of a recent paper titled "The Concept of Agent in Biology: Motivations and Meanings" by Samir Okasha (2023).
In both Biology and AI Alignment, the concept of agency comes up a lot and appears relevant to properly understanding the respecting "territory" - while also being used in confusing, and sometimes confused, ways.
Many more person-hours have been invested in progress in Biology compared to AI alignment, so it seems apt to keep an eyes out for any insights (as well as mistakes) from the discourse in Biology that might help improve the quality of discourse around the notion of 'agency' in Alignment.
I decided to write this summary post in order to make the information in the paper more accessible to the Alignment discourse. The paper itself is already a succinct (5 page!) summary of decades of discussions about the appropriate use of the notion of 'agency' in Philosophy of Biology. I can recommend to read it fully, though I have tried to capture all relevant ideas/concepts in this post.
Okasha himself is a well-respected Philosopher of Biology who has a range of substantive prior work on this and related topics (e.g. Agents and Goals in Evolution (2018), Evolution and the Levels of Selection (2006)).
"Biological agency" is a topic that has recently been gaining attention within Philosophy of Biology. In this paper, Okasha sets out to discuss "what exactly we mean by the term 'agent'?".
To bring some clarity to the discourse, he:
Motivation 1 ('organisms-as-agents thesis') "says that organisms really do exhibit some or even all of the attributes of agency". Motivation 2 ('organism-as-agents heuristic') "says that it can be heuristically useful to treat organisms as if they were agents for certain intellectual purposes".
This part is not longer exclusively pulling from the paper.
Okasha illustrate that there are two very distinct motivations for which philosophers of science (and others) may invoke the notion of agency. It seems useful to see just how different the motivations are, and that they imply different things about what looks like an appropriate concept/definition of agency. For example, for the 'organisms-as-agents thesis' (motivation 1), the 'intelligent agents' concepts seems most relevant. For the 'organisms-as-agents heuristic', the concepts of 'rational' or 'intentional agency' are more fitting.
As such, in Alignment discourse too, it can be useful to make explicit the motivation for invoking the notion of agency - be that if you're using the notion in your own thinking/research, or if you're engaging with someone else's thinking/research.
What does it mean that there are different, partially overlapping and partially conflicting, notions of agency? Should we expect to eventually converge to 'the one true notion of agency'?
With respect to the last question, I don't know whether we eventually will. I think it's a valid quest to try to deconfuse our understanding of agency in the same way that we deconfused, say, our understanding of temperature some hundred years ago.
For now, however, it's often useful to adopt a pragmatic perspective. Different notions of agency (such as the ones listed above) pick out different phenomena/structures in the world. They may pick out broader/narrower versions of the same phenomena; they may pick out the same phenomena highlighting different aspects of it; or they may pick out (partially or entirely) different phenomena. For practical purposes, the main thing that matters is that a) we are precise about what phenomena/structures we mean to "pull out" and b) that any notion of agency we stipulate is coherent and well-defined (even sometimes we have to start with a work-in-progress definition).
The extent to which the four notions of agency discussed above vary highlights the need to be explicit about what notion you have in mind if you invoke the term. We should also remember that these different notions tend to come from different traditions of thought, and as such, what is considered the 'default interpretation' of the term 'agency' - and with it an entire set of assumptions and connotations that is invoked when using the term - will vary depending on what your intellectual upbringing was like.
Finally, we can ask ourselves to what extent the notions of agency that Okasha has picks out in the paper overlap with the notion(s) of agency we often care about in Alignment. While there definitely is important overlap, there might be some aspects we care about that aren't (fully/appropriately) captured in any of the four concepts. As I don't want to explode the scope of this post, let me just leave it at this: it's not clear whether any of the four concepts captures (or is properly aimed at capturing) some notion of "long range consequentialism". (Nor is it entirely clear, for that matter, whether they should, i.e. whether "long range consequentialism" points at anything real and coherent).
Okasha remarks that, for the 'organisms-as-agents thesis' (motivation 1), while the 'agency as intelligent behavior' (concept 2) notion seems most apt among the four, it fails (on its own) to "precisely capture organismic distinctiveness". Okasha goes on speculating: "The underlying reason, I suggest, is that none of the four concepts captures the idea of autonomy, which intuitively is what distinguishes organisms from other flexible goal-directed systems in nature. This conclusion fits with the position of Moreno and Mossio (2015), who argue that agency is one central component of organismic autonomy rather than the whole story."
Zooming out, this is a good example of how a given notion of agency may or may not capture all that one intended for it to capture. In this particular example, sometimes when we talk about agency, we intend to capture some notion of 'autonomy'. In the context of the paper, 'autonomy' was meant to point at what we intuitively mean when we want to say that an organism is 'autonomous' in a way that cells or organism making up that organism aren't. This notion of autonomy is not (at least not straightforwardly) captured by any of the four notions of agency captured here. (Worth noting that this is a fairly weak-sauce and largely inadequate 'definition' of autonomy that I've provided here, but for the sake of the point I'm trying to make, it shall suffice.)
It's interesting that both motivations appear to be about modelling organisms as agents, as opposed to any other level of organisation. This feels like it misses some of the most interesting insights we might get from biological agency, namely those around agents at different levels of organisation interacting - e.g: ants and ant colonies, cancer cells and multicellular organisms, or individual organisms and selection pressures (which could be treated as as-if agents at evolutionary timescales).
Okasha's paper is addressing emerging discussions in biology that are talking about organisms-as-agents in particular, otherwise being called the Return of the Organism turn in philosophy of biology.In the paper, he adds "Various concepts have been offered as ways of fleshing out this idea of organismic autonomy, including goal-directedness, functional organization, emergence, self-maintenance, and individuality. Agency is another possible candidate for the job."This seems like a reasonable stance so far as I can tell, since organisms seem to have some structural integrity -- in what can make delineated cartesian boundaries well-defined.For collectives, a similar discussion may surface additional upsides and downsides to agency concepts, that may not apply at organism levels.