Can you say something about your motivation to work on this? Why not leave it to future AI and/or humanity to figure out? Or what are the most important questions in this area to answer now?
Doesn't this depend on what we value?
In particular, you appear to assume that we care about events outside of our lightcone in roughly the way we care about events in our near future. I'm guessing a good deal of skepticism of ECL is a result of people not caring much about distant events.
Yeah, you're right that we assume that you care about what's going on outside the lightcone! If that's not the case (or only a little bit the case), that would limit the action-relevance of ECL.
(That said, there might be some weird simulations-shenanigans or cooperating with future earth-AI that would still make you care about ECL to some extent although my best guess is that they shouldn't move you too much. This is not really my focus though and I haven't properly thought through ECL for people with indexical values.)
The model does not capture the fact that the total value you can provide to the commons likely scales with the diversity (and by proxy, fraction) of agents that have different values. In some models, this effect is strong enough to flip whether a larger fraction of agents with your values favors cooperating or defecting.
I'm curious to hear more about this, could you explain what these other models are?
I find myself arriving at a similar conclusion, but via a different path.
I notice that citizens often vote in the hopes that others will also vote and thus as a group will yield benefit. the do this even when they know their vote alone will likely make no difference, and their voting does not cause others to vote.
So why do they do this? My thought is that we are creatures that have evolved instincts that are adaptive for causally-interacting, social creatures. In a similar way I expect other intelligence may have evolved in causally interacting social contexts and thus have developed similar instincts. So this is why I expect distant aliens may behave in this way.
This conclusion is similar to yours, but I think the reasoning chain is a bit different:
(1) non-self-benefiting cooperation is evolutionarily preferred for "multi-turn" causally-interacting social agents.
(2) Thus such social agents (even distant alien ones) may evolve such behavior and apply it instinctively.
(3) As a result we (and they) find ourselves/themselves applying such in cooperative behavior in contexts that are known to ourselves/themselves to be provably a-causal.
Interestingly, I can imagine such agents using your argument as their post-hoc explanation of their own behavior even if the actual reason is rooted in their evolutionary history.
How does this argument fit into or with your framework?
Interesting. The main thing that pops out for me is that it feels like your story is descriptive while we try to be normative? I.e. it's not clear to me from what you say whether you would recommend to humans to act in this cooperative way towards distant aliens, but you seem to expect that they will do/are doing so. Meanwhile, I would claim that we should act cooperatively in this way but make no claims about whether humans actually do so.
Does that seem right to you or am I misunderstanding your point?
Chi, I think that is correct.
My arguments attempts to provide a descriptive explanation of why all evolved intelligence do have a tendency towards ECL, but it provide no basis to argue such intelligence should have such a tendency in a normative sense.
Still somehow as an individual (with such tendencies), I find the idea that other distant intelligence will also have a tendency towards ECL does provide some personal motivation. I don't feel like such a "sucker" if I spend energy on an activity like this, since I know others will to, and it is only "fair" that I contribute my share.
Notice, I still have a suspicion that this way of thinking in myself is a product of my descriptive explanation. But that does not diminish the personal motivation is provides me.
In this end, this is still not really a normative explanation. At best is could be a MOTIVATING explanation, for the normative behavior you are hoping for.
~
For me, however, the main reason I like such a descriptive explanation is that it feels like it could one day be proved true. We could potentially verify that ECL follows from evolution as a statement about the inherent and objective nature of the universe. Such objective statements are of great interest to me, as they feel like I am understanding a part of reality itself.
Interesting topic!
Summary
Evidential cooperation in large worlds (ECL) is a proposed way of reaping gains—that is, getting more of what we value instantiated—through cooperating with agents across the universe/multiverse. Such cooperation does not involve physical, or causal, interaction. ECL is potentially a crucial consideration, because we may be able to do more good this way compared to the “standard” (i.e., causal) way of optimizing for our values.
The core idea of ECL can be summarized as:
Introduction
The observable universe is large. Nonetheless, the full extent of the universe is likely much larger, perhaps infinitely so. This means that most of what’s out there is not causally connected to us. Even if we set out now from planet Earth, traveling at the speed of light, we would never reach most locations in the universe. One might assume that this means most of the universe is not our concern. In this post, we explain why all of the universe—and all of the multiverse, if it exists—may in fact concern us if we take something called evidential cooperation in large worlds (ECL) into account.[1] Given how high the stakes are, on account of how much universe/multiverse might be out there beyond our causal reach, ECL is potentially very important. In our view, ECL is a crucial consideration for the effective altruist project of doing the most good.
In the next section of this post, we explain the theory underlying ECL. Building on that, we outline why we might be able to do ECL ourselves and how it allows us to do more good. We conclude by giving some information on how you can get involved. We will also publish an FAQ in the near future, which will address some possible objections to ECL.
The twin prisoners’ dilemma
Exact copies
Suppose you are in a prisoner’s dilemma with an exact copy of yourself:
This situation can be represented with the following payoff matrix:
Looking at the matrix, you can see that regardless of whether your copy cooperates or defects, you are better off if you defect. “Defect” is the strictly dominant strategy. Therefore, under standard notions of rational decision making, you should defect. In particular, causal decision theory—read in the standard way—says to defect (Lewis, 1979).
However, the other player is an exact copy of yourself. In other words, you know that they will undergo the same decision-making process as you, and arrive at the same decision as you. If you cooperate, they cooperate. If you defect, they defect. You can see this even more clearly when considering two identical deterministic computer algorithms playing the prisoner’s dilemma against each other. You know that you both cooperating gives you a better payoff than you both defecting. Some theories, among them evidential decision theory and functional decision theory, therefore recommend cooperating in the twin prisoners’ dilemma. Throughout this post, we assume the perspective that cooperating against copies is rational, although you can find many alternative perspectives in the academic literature.
General case
The argument for cooperation is most intuitive if you’re dealing with an exact copy of yourself. What if we instead consider:
The first example, the near-copy who just has different-colored shoes, seems unlikely to follow an importantly different decision process from you regarding the twin prisoners’ dilemma. After all, they are identical to you except for one superficial aspect—the color of their shoes. Hence, while it’s not guaranteed, since you and they are not fully identical, you can expect, with high confidence, for them to cooperate if you cooperate and defect if you defect. Throughout this post, we assume the perspective that this gives you reason to cooperate.
We use the umbrella term acausal influence to refer to that which makes a near-copy more likely to cooperate with you in a prisoner’s dilemma when you cooperate.[2],[3],[4] Acausal influence is a property of particular decisions rather than agents.
As we move down through the list of examples above, it stands to reason that the relationship between your and the other agent’s decisions becomes less strong, because you and they become different in increasingly important ways. Your acausal influence over them is weaker.
Note that there are reasons to think that you can only leverage a small fraction of your total acausal influence on others. For more details, see appendix.
Why is this type of acausal cooperation important in practice? Introducing ECL
Our situation as a generalized twin prisoners’ dilemma-type situation
Evidential cooperation in large worlds (ECL) is the idea that it is possible for agents across the universe/multiverse to gain from acausal cooperation with each other, in a manner similar to that discussed in the previous section. We argue below that there are likely extremely many agents over whom we have some degree of acausal influence. Putting this together with the previous discussion, we arrive at the following key claim:
We plausibly want to adopt the following policy: Whenever we can benefit the values of relevant other agents in the multiverse—to a first approximation, agents that also engage in ECL[5]—more efficiently than we can benefit our own values, we should do so.[6] This is because everyone adopting this policy would make us better off. Hence, if we acausally influence many agents to adopt this policy and enough of these agents would otherwise do something worse from our perspective, then we are better off.
The main practical implication: it may be important to build AGI systems to be cooperative towards other value systems (insofar as we build AGI at all),[7] whether they are aligned or misaligned. Another way of framing this is that we want future earth-originating AI systems to be good citizens of acausal society. Doing so plausibly benefits many agents with values different from ours. So, even if we can’t tell exactly who else is engaging in ECL, or what their individual values are, this intervention has a good chance of benefiting them. Other agents engaging in acausal cooperation might do the same, or take another action that they expect to be broadly good for values different to theirs, and so some of these agents will aid our values.
A toy model to quantitatively illustrate when cooperation is beneficial:
Note: If we assume that all value systems have an equal number of agents endorsing it, and that all value systems are all neutral for each other (i.e., orthogonal; neither agreeing nor being in opposition), then every agent faces the same situation as you.
If you defect, you increase the expected money going towards your values by:
If you cooperate, you increase the expected money going towards your values by:
Therefore, in this example, you should cooperate (for instance, by producing the jointly most efficient mix of utopia and paperclips). Note that almost all of the payoff you get from cooperating comes from acausally influencing agents with values different from yours towards cooperating. And most of the payoff from defecting comes from acausally influencing agents with your values towards defecting. However, the result is sensitive to the values you choose for the parameters. In this model, the most important factors, when it comes to deciding whether you should defect (i.e., causally optimize for our own values) or cooperate, are:
If you found out that all your acausal influences are in fact smaller (or larger) than you thought, this does not change the relative value of cooperation versus defection. What matters is the ratio of your acausal influence on agents with your values versus agents with different values.
The model presented here has some important limitations[12] and the conclusions vary slightly if you choose a different setup to model acausal cooperation. For example, depending on the model, a higher fraction of agents with your values either increases or decreases the relative value of cooperation. The general thrust stays the same across models, though: The more you acausally influence agents with different values from you, the more action-relevant ECL is.
Overall, this leaves us with the following conditions under which ECL is action-relevant (i.e., recommends us to cooperate when we would otherwise defect):
In the remainder of this post, we argue for each of these conditions. We also gradually relax the following assumptions from the above toy model:
Some of the discussion in the appendix also relaxes the assumption that there are two distinct categories: “agents with our values” and “agents with different values.”
Many agents
In our example above, we had acausal influence over 999,999 agents. This section is concerned with the question: Are there, in fact, many agents out there to cooperate with?
Our universe (or multiverse) appears to be very large, perhaps infinitely so, which means that it likely contains very many—perhaps infinitely many—agents with whom we could potentially acausally cooperate. There are three main ways in which physicists believe the universe/multiverse could be large:[15]
DALL·E 2 illustration of the multiverse.
Overall, we believe there is strong evidence for a large world, and thus many agents.[22] Additionally, even if one believes that the world likely does not contain enough agents to justify ECL, we have more (acausal) influence in the worlds that do contain many agents, and so our expected impact is higher if we focus on (the credence that we have in) those worlds.[23] Note that only one of the three ways we’ve presented need be true for the many agents premise to hold.[24] For more detail on the universe’s size, see our appendix.
Acausal influence on agents with other values
The toy example above assumes that we can acausally change the probability of agents with other values taking a decision that benefits us by 0.001. This section is concerned with the question: If we act cooperatively towards other agents, are they actually more likely to reciprocate?
We have already argued the basic case for considering acausal influence in our “Twin prisoners’ dilemma” section. The basic argument for thinking we can acausally influence agents with other values to benefit us is simple: We are faced with the decision of either trying to benefit other agents, or causally optimizing for our own values. Meanwhile, other agents face the same decision, and make decisions similarly to us. Because of this, our decision acausally influences them towards doing the same thing as us (i.e., benefiting other values, or causally optimizing for one’s own). Hence, if we try to benefit them, they will try to benefit us.
This argument has a few gaps:
To sketch an argument for why other agents might make decisions about ECL in a similar way to us: We think it’s likely that agents reason in ways that are somewhat convergent. Through a realist lens—i.e., assuming there is a unique way of reasoning that is objectively rational—one would expect reasoning to converge to what is “correct”. From an antirealist perspective, we expect some convergence to what “wins”—i.e., successful agents tend to reason in similarly effective ways. We expect reasoning that outputs “cooperate” in the twin prisoners’ dilemma, and which generalizes from there, to occur at some frequency. We further expect that some agents who reason in this way will think about the implications and discover something in the ballpark of ECL. This leads us to believe that we are able to acausally influence some agents’ ECL-inspired actions through our own ECL-inspired actions. Note that this line of reasoning is contested. A later section will touch on the potential fragility of acausal influence. (For adjacent work, see Finnveden, 2023.)
To address the latter two concerns (firstly, that different agents face different situations with no “cooperate” label; secondly, that there’s no guarantee other agents will benefit us specifically), we will look at a toy example in which agents face different decisions from each other, and cannot mutually benefit each other. We have three agents, A, B, and C:
The agents have common knowledge of the situation and of having very similar decision-making processes to each other. (For instance, maybe they each know that all three of them are copies of GPT-N that received a nearly identical prompt, with the only difference in the prompt being the part where they are told which of the three agents they are.)
What should agent A do to maximize her payoff in this situation? We propose that she should reason as follows: “Instead of framing my decision as picking between doing nothing and clapping my hands, I can frame my decision as keeping money or giving money. I would do the same reframing if I were in the other agents’ position, so they are probably also framing their decision as keeping or giving money. I now find myself in the same position as agent C:
The first thing we can see is that even in a situation where we cannot mutually affect each other’s payoffs, we can still acausally influence another agent towards trying to benefit us by trying to benefit some third agent ourselves. There can be cyclic arrangements like this with many more than three agents—the logic remains the same. We do not need to worry about finding the exact agent who can benefit us: If we generally benefit agents that are like us, we are more likely to be benefitted in turn. (For more on this, see Oesterheld, 2017, section 2.9.)
Secondly, acausal influence is not limited to situations where we have the exact same set of actions available as another agent. As in the three-agent example, we can apply the concept of reframing our decision to something more abstract such that many other agents can be said to face the same abstract decision as us. In particular, we can think of ourselves as facing one big abstract decision: Do we want to commit ourselves to acting cooperatively towards other agents who do ECL? If other agents frame things similarly, we and they find ourselves in similar situations. And if we both make such a commitment, it does not matter that the specifics of our downstream decisions look different. Therefore, even if the actions available to us humans are very different from the actions available to those we want to acausally cooperate with, we still have reason to believe that deciding to act cooperatively towards them will acausally influence them to act in kind. (For more on this, see Oesterheld, 2017, section 2.8.2. Oesterheld argues that agents who acausally cooperate should optimize for a compromise utility function. See also Finnveden, 2023.)[25]
Net gains from cooperation
The toy example assumes that we gain more if everyone cooperates than if everyone defects. This section is concerned with the following question: If we and the other agents cooperate, do we actually gain in expectation?
To answer this, we need to assess three important ingredients. Firstly, how much do we gain from one agent with other values benefitting us compared to our causal opportunity cost when we benefit them? This is the size of the potential gains from cooperation. In the initial prisoner’s dilemma we presented, the gains from cooperation would be represented in the payoff matrix. There are a few overlapping reasons for potentially high gains:[26] diminishing marginal returns within but not across light cones, comparative advantage, and combinability. These reasons mirror the reasons for why causal cooperation among humans is beneficial; we will not expand on them here. See the appendix for more.
Secondly, competence at cooperating. That is, how successful do we expect agents to be at benefiting each other? It is no use influencing others towards aiding you if they are incompetent at doing so. An important obstacle here is knowing what other agents value.[27] To figure out how good the average agent is at knowing how to benefit others, we can think about how good we are at this, and extrapolate. While it seems near-impossible for current humans to reason out the exact values of some alien civilization, we can make fairly confident broad guesses such as “agents who think about ECL tend to value having more rather than fewer resources, so they would prefer more opportunities for trade.” On this account, some ECL researchers strongly favor making future AI systems more cooperative. We think that much better educated guesses could be made in the future, after:
Overall, we expect that agents will be fairly competent at cooperating.
The third ingredient: the right kind of acausal influence. For ECL to recommend cooperation, it must be the case that our cooperation acausally influences agents to benefit us more than it influences agents to take actions that are worse for us. By and large, the acausal costs (benefits) of our actions will come from us acausally influencing agents with our (different) values to cooperate instead of defect. Note that we have already established that the absolute level of acausal influence we have over other agents does not matter if the population of agents is sufficiently large, so we can exclusively focus on the ratio of our respective acausal influences.
The size of this ratio is highly contested. We don’t take a stance in this document on what range of ratios is plausible. The personal view of one of the authors, Chi, is that we have enough acausal influence over agents with different values that the ratio is above the critical ratio. (Below which doing ECL would not be worth it.) This seems like a worthwhile topic of investigation and a large source of uncertainty about whether ECL is action-relevant. For more details, see the appendix.
Putting it all together
We have discussed the idea of acausal influence, and from there, how we might want to engage in multiverse-wide cooperation with aliens and distant AGIs. To conclude with something poetic, we excerpt from Richard Ngo’s recent reflection:
How to get involved
Short-term research fellowships
There are a few different research programs that might be open towards ECL-inspired work, either conceptual or technical. Historically, these have been:
We expect at least some of these programs to continue being open towards this kind of work, and for perhaps others to join the list.
Long-term employment
We don’t know of any organizations specifically focused on ECL. The only organizations we know of that have carried out some ECL research are the Center on Long-Term Risk (CLR), Open Philanthropy’s worldview investigations team, and a couple of individuals at FOCAL at Carnegie Mellon University. These organizations might be open to hiring more people to work on ECL in the future, although we are very unsure about how likely that is. The best bet might be CLR. Additionally, MIRI has done some decision theory research that is relevant to ECL, as have some academic institutions.
Reach out to Chi
I, Chi Nguyen, might be looking for ECL collaborators in the future. (Mostly for ML or conceptual research, but perhaps also for other projects, e.g., posts like this one or running events.) Get in touch with me (linhchismail [] gmail [] com) if you are potentially interested. I might also be able to connect you to others in ECL, recommend programs, or find other ways of helping you. If you do reach out, it would be helpful if your email includes one or two concrete questions, curiosities, objections, research angles, or details on the kinds of work or other opportunities you’d be open to. I probably won’t have much more useful to say than what is written here if you just say you are interested in ECL and wonder how to get involved. (Although maybe this will change at some point.)
Funding
In terms of funding, we encourage those interested in doing ECL research and/or upskilling to apply to the Long-Term Future Fund, the CLR Fund, Lightspeed Grants (if they run another round), or Manifund Regrants.
How to skill up to work on ECL
We expect it would not take long for someone with good personal fit to become a leading expert in ECL or one of its subtopics. Our best guess is that one can reach the frontier of ECL knowledge with around three months of skilling up half-time, starting from knowing ~nothing about ECL or decision theory. We list recommended readings below.
Recommended readings
Our number 1 recommendation is to read “Multiverse-wide cooperation via correlated decision making – Summary” (Oesterheld, 2017). This paper is ECL’s seminal work. Bear in mind that the “Interventions” section is now a little outdated. It's quite long but very good and covers lots of different topics. Many of the core concepts are already covered here, so reading it should take you less time if you’ve already read this post.
For shorter introductory-level material on ECL specifically, we recommend:
For deep dives into different aspects of ECL theory, we recommend:
For work on the potential implications of ECL, we recommend:
For other comments on ECL, we recommend:
For more on the decision theory underlying ECL (all from the perspective of proponents of such decision theories), we recommend:
We’d like to be able to point to some critiques, as well, but unfortunately, there aren’t any public critiques of ECL, as far as we can tell. This does not mean there aren’t valid potential critiques. There is, however, a rich academic literature on decision theory, which includes arguments in favor of causal over non-causal decision theories.
For an alternative (albeit heavily overlapping) reading list by the Center on Long-Term Risk, see here.
Acknowledgements
We are grateful to Daniel Kokotajlo, Lukas Finnveden, Kit Harris, Euan McLean, Emery Cooper, Anthony DiGiovanni, Caspar Oesterheld, Vivek Hebbar, Tristan Cook, Sylvester Kollin, Jesse Clifton, and Nicolas Mace for helpful feedback on earlier versions of this post and other contributions. This does not imply endorsement of all, or any, claims in the post.
To finish off, a photo of our future intergalactic snack shop:
ECL was introduced in Caspar Oesterheld’s seminal paper “Multiverse-wide Cooperation via Correlated Decision Making”. (Note: “multiverse-wide cooperation” was the old name for ECL.)
We define acausal influence as a property of decision-making, rather than a property of agents.
i) Different people refer to non-causal relationships using different terms that describe partially different and partially overlapping concepts. Examples include “logical influence,” “logical correlation,” “correlation,” “quasi-causation,” “metacausation,” and “entanglement.” We use “acausal influence” as an umbrella term.
ii) Some readers may wonder how, if at all, acausal influence and ECL relate to acausal trade. We address this in our upcoming post: “ECL FAQ”.
iii) For more arguments in favor of cooperation in the twin prisoners’ dilemma and other thought experiments, see, for example, Almond (2010a), Almond (2010b), and Ahmed (2014).
For example, evidential decision theory would say that conditioning on your own choice changes the probability that the other person will cooperate. The size of this change is the size of your acausal influence. In this definition, acausal influence is a property of particular decisions rather than agents, and not a binary, but comes in gradations. Whether you should cooperate with someone depends on how strongly you think your decision acausally influences theirs.
For now, we are assuming that everyone who is doing ECL is cooperating with everyone else who is doing ECL. The real picture is likely quite a lot more complicated but unfortunately out of scope for this document. A second toy model below hints at some of the considerations at play.
Notably, this policy would not involve causally optimizing for purely our own values.
For an attempt at quantifying this claim, see “How ECL changes the value of interventions that broadly benefit distant civilizations” (Finnveden, 2023).
Note that “high acausal influence” might trade off with “high potential gains from cooperation.” It seems plausible, for example, that misaligned artificial superintelligences have the most power to benefit us, but we can barely acausally influence them. Meanwhile, we might be able to acausally influence our close copies in other Everett branches by a lot, but they cannot benefit us as much because they have less power than an AGI and because their default action would already be quite beneficial to us (because they have similar values).
At smaller population levels, the direct gain you obtain by defecting (in the above example, $1) makes up a significant portion of your total gain ($21 in the defection case, in the above example). In these cases, your absolute acausal influence matters because it is pitched against your influence on yourself (which is exactly 1 since you perfectly influence your own action.)
In some other models you might get a non-linear scaling on the total acausal effects of cooperating.
We address Pascal’s mugging-type concerns about having very small acausal influence over a large number of agents in our FAQ. The tl;dr is that this situation is structurally quite different from a traditional Pascal’s mugging.
Most importantly:
- The model does not capture the fact that the total value you can provide to the commons likely scales with the diversity (and by proxy, fraction) of agents that have different values. In some models, this effect is strong enough to flip whether a larger fraction of agents with your values favors cooperating or defecting.
- The model does not capture that your correlation with other agents only holds if their situation is similar enough to yours. For example, if there is a group of agents who are a very small fraction of the total population, mutual cooperation is not beneficial to them and they are better off always defecting. That is because, in our model, how much a value system benefits from others cooperating scales with how many agents have that value system. However, our model cannot capture this dynamic and always assumes that us cooperating increases the chance that others also cooperate.
- In this model, once there are sufficiently many agents with your values, cooperating also becomes the causally best action. That is because you split $3 across all agents when you cooperate: Hence, in the extreme where all agents have your values, you gain $3 for you value system by cooperating and only $1 by defecting. This is a good model for some situations and not for others.
We are assuming that you cannot take any action that directly increases the payoff of agents with your values that is better than what you would do in the absence of these agents existing. In other words, if there are many agents we have acausal influence over but they all have the exact same values as us, then we should just act as we would if they did not exist.
Agents with other values also have to think they benefit more from trying to acausally influence us compared to causally optimizing for their own values. Otherwise, there is no reason for them to participate in this cooperation.
The ways we list correspond to three of the four levels in Tegmark (2009)’s multiverse hierarchy. They are also three of Greene (2011)’s nine types of multiverses: quilted, inflationary, and quantum, respectively. There are further theories, such as cyclic cosmology—another of Greene’s nine types, which also indicate that the universe/multiverse could be infinite (or at least very large), but we limit our discussion in the main text to theories that we consider to be most likely based on our understanding of what most cosmologists believe.
The two priors in footnote 6 of Vardanyan, Trotta and Silk (2011) lead to 98.6% and 46.2% posterior probabilities, respectively, of the universe being exactly flat (and therefore infinite, with infinitely many aliens).
(Note: I—Will—think that these Vardanyan et al. results are contentious. (I’m still quoting them, though, because there appears to be a dearth of papers on the topic of assigning credences to the possible shapes and sizes of our universe.) To the best of my knowledge, based on conversation, cosmologists can be roughly divided into two clusters: those who don’t subscribe to eternal inflation, and those who do. Of those who don’t, many (most?) seem to have a uniform prior over the cosmic density parameter. Under this prior, the probability that the universe is exactly flat is zero.* Meanwhile, those who do subscribe to eternal inflation believe that our universe is negatively curved and therefore infinite, on account of details in the equations that describe how bubble/pocket universes arise (Guth, 2013). (However, this second camp also believes that there are infinitely many universes, so believing that our universe is negatively curved and therefore infinite, and believing that we live in an infinite multiverse, are not independent beliefs.)
*Nonetheless, the contention with Vardanyan et al. in this case doesn’t have much effect on the likelihood that the universe is spatially infinite. The universe is measured to be almost exactly flat: to a first approximation, under a uniform prior and ignoring anthropic considerations, this gives us 50% probability that the universe is positively curved and therefore finite, and 50% probability that the universe is negatively curved and therefore infinite.)
Reasoning about the frequency of civilizations is difficult. Attempts include the Drake equation (and the Drake equation reworked with distributions rather than point estimates—Sandberg, Drexler and Ord, 2018) and Robin Hanson’s grabby aliens model.
In fact, the authors of this post disagree about how likely a finite, purely spatio-temporal universe would be to contain enough agents to justify ECL.
Everett branches and the eternal multiverse are speculatively united under the cosmological interpretation of quantum mechanics (Aguirre & Tegmark, 2011). This position does face opposition, for example, from Siegel (2023). (Note: Aguirre and Tegmark use their own terminology, using “Universe” with a capital U to refer to what most other people refer to as the multiverse.)
One of the authors, Will, endorses the viewpoint Sean Carroll puts forward in his 2019 book Something Deeply Hidden that “interpretation” is a misnomer. Copenhagen, and many-worlds, and hidden variables are not different interpretations of the same theory: they describe reality working in different ways; they are different theories. The other side of the argument would say that what makes a theory is whether it makes testable predictions. I, and Carroll and some others, I believe, would rebut, saying that what matters is what is real, and that on-paper reasoning as well as experimental verification should influence what one believes is real. In the quantum case specifically, I would also say that hidden variables has arguably been eliminated by Bell’s theorem experiments (e.g., Brubaker, 2021).
There is debate about whether, as the number of branches grows, the total amount of “stuff”—for want of a better word—grows with the number of branches, or whether it remains fixed, with each branch becoming an ever thinner slice of the total. This difference might be important in some ways. Nonetheless, this is not a relevant difference for ECL. Whether the total amount of stuff increases or not, the share of branches that we inhabit decreases in the same way as splittings occur; there is no difference in how we relate to the agents in other branches insofar as ECL is concerned. For more on the branching debate, see David Wallace’s 80,000 Hours podcast episode and/or mentions of “reality fluid” on LessWrong.
Even if this were not the case, we would be happy to wager on the possibility of extremely many agents in an expected value-style manner since the probability for extremely many agents surely is not Pascalian.
Unless you believe it is extremely unlikely that there are many agents. However, given modern cosmology, such a belief seems unjustified.
In addition to cosmology-based reasoning and evidence for believing there might be very many agents in our world, there is also Nick Bostrom’s presumptuous philosopher argument (pp. 9-10): given the fact that you exist, you should, all else equal, favor hypotheses which say that the world contains many observers (i.e., agents) over hypotheses which say there are few observers. However, the presumptuous philosopher relies on the self indicating assumption (SIA) for making anthropic updates, which is controversial and has problems when combined with evidential decision theory. The presumptuous philosopher was also originally designed as a counterargument to SIA (in the way that its conclusion, driven by SIA, arguably proves too much).
In principle, there is not even anything that limits our acausal influence to other agents’ actions. Having acausal influence simply means, “If I take action x, something else (whether that’s an action or other event) is made more likely even though I am not causally affecting it.” Although for our purposes, we usually only care about the ways in which our actions acausally influence other agents’ actions.
Assuming agents care about what happens in other light cones at all.
You might think it is of no concern to us whether we fail at predicting other agents’ values: Even if we fail to benefit other values, as long as they successfully benefit ours, we are happy. However, if we predictably fail at benefiting others, they have no reason to want to cooperate with us! In fact, if we employ a lopsided policy of “only cooperate if they know how to benefit us,” they are more likely to do the same. From the perspective of not yet knowing who will successfully figure out who can successfully benefit whom, it is better to employ the strategy “cooperate if agents on average know enough about what is beneficial to the other to make cooperation worthwhile for the other,” to make it more likely that everyone else adopts this strategy.
Oesterheld (2017, section 3.4) and Ngo (2023) discuss the latter point. In short, these agents’ values are very likely correlated with genetic fitness and beyond-the-light-cone thinking. Therefore, agents that engage in ECL seem more likely to care about happiness and suffering than about highly Earth-specific issues such as the value of the traditional family. So, we might want to weigh light cone-agnostic values more heavily than we otherwise would.
These claims about values are weakened by considering misaligned AIs. We expect the distribution of misaligned AI values, relative to the distribution of biological civilizations’ values or aligned AI values, to be closer to uniform over all values. For more on this, see Aldred (2023). (I—Will Aldred—do not think this consideration is original to me, but I could not find it written up elsewhere.)