Epistemic Status: Cautiously optimistic. Much of this work is in crafting and advancing terminology in ways that will hopefully be intuitive and useful. I’m not too attached to the specifics but hope this could be useful for future work in the area.

Introduction

Strong epistemics or “good judgment” clearly seems valuable, so it’s interesting that it gets rather little Effective Altruist attention as a serious contender for funding and talent. I think this might be a mistake.

This isn’t to say that epistemics haven’t been discussed. Leaders and community members on LessWrong and the EA Forum have written extensively on epistemic rationality, “good judgment”, decision making, and so on. These communities seem to have a particular interest in “good epistemics.”

But for all the blog posts on the topic, there is less in terms of long term and full-time efforts. We don’t have teams outlining lists of possible large scale epistemic interventions and estimating their cost-effectiveness, like an epistemics version of the Happier Lives Institute. We don’t have a Global Priorities Institute equivalent trying to formalize and advance the ideas from The Sequences. We have very little work outline what optimistic epistemic scenarios we could hope for 10 to 200 years out from now.

I intend to personally spend a significant amount of time on these issues going forward. I have two main goals. One is to better outline what I think work in this area could look like and how valuable it might be to pursue. The second is to go about doing work in this area in ways that both test the area and hopefully help layout groundwork that makes it easier for more people to join in.

One possible reason for a lack of effort in the space is that the current naming and organization is a bit of a mess. We have a bundle of related terms without clear delineations. I imagine that if I asked different people how they would differentiate “epistemics”, “epistemic rationality”, “epistemology”, “decision making”, “good judgment”, “rationality”, “good thinking”, and the many subcategories of these things, I’d get many conflicting and confused answers. So some of my goal is to try to highlight some clusters particularly worth paying attention to and formalize what they mean in a way that would be useful to make decisions going forward.

I’ll begin by introducing two (hopefully) self-evident ideas. “Epistemic Progress” and “Effective Epistemics.” You can think of “Epistemic Progress” as the “epistemics” subset of “Progress Studies”, and “Effective Epistemics” as the epistemic version of “Effective Altruism.” I don’t mean this as an authoritative cornerstone, but rather as pragmatic intuitions to get us through the next few posts. These names are chosen mainly because I think they would be the most obvious to the audience I expect to be reading this.

Effective Epistemics

“Effective Epistemics” is essentially “whatever seems to work at making individuals or groups of people more correct about things for pragmatic purposes.” It’s a bit higher level than Value of information. This is not focussed on whether something is theoretically true or with precise definitions of formal knowledge. It’s rather about which kinds of practices seem to make humans and machines smarter at coming to the truth in ways we can verify. If wearing purple hats leads to improvement, that would be part of effective epistemics.

There’s a multitude of things that could help or hinder epistemics. Intelligence, personal nutrition, room lighting, culture, economic incentives, mathematical knowledge, access to expertise, add-ons to Trello. If “Effective Epistemics” were an academic discipline, it wouldn’t attempt to engineer advanced epistemic setups, but rather it would survey the space of near and possible options to provide orderings. Think “cause prioritization.”

Effective Altruism typically focuses on maximizing the potential of large monetary donations and personal careers. I’d imagine Effective Epistemics would focus more on maximizing the impact of smaller amounts of effort. For example, perhaps it would be identified that if a group of forecasters all spent 30 hours studying information theory, they could do a 2% better job in their future work. My guess is that epistemic intervention estimations would be more challenging than human welfare cost-effectiveness calculations, so things would probably begin on a more coarse level. Think longtermist prioritization (vague and messy), not global welfare prioritization (detailed estimates of lives saved per dollar).

Perhaps the most important goal for “Effective Epistemics” is to reorient readers to what we care about when we say epistemics. I’m quite paranoid about people defining epistemics too narrowly and ignoring interventions that might be wildly successful, but strange.

This paranoia largely comes from the writings of Peter Drucker on having correct goals, in order to actually optimize for the right things. For example, a school “optimizing education for getting people jobs” might begin with High School students at one point when those are the highest impact. But if things change and they recognize there are new opportunities to educate adults, maybe they should jump to prioritize night school. Perhaps with online education they should close down their physical building and become an online-only nonprofit focussed on international students without great local schools. It can be very easy to fall into the pattern of trying to “be a better conventional High School than the other conventional High Schools, on the conventional measures”, even if what one really cares about upon reflection is the maximization of value from education.

Epistemic Progress

“Epistemic Progress” points to substantial changes in epistemic abilities. Progress Studies is an interesting new effort to study the long term progress of humanity. So far it seems to have a strong emphasis on scientific and engineering efforts, which makes a lot of sense as these are very easy to measure over time. There have been a few interesting posts on epistemics but these are a minority. This post on Science in Ancient China seems particularly relevant.

Historic epistemic changes are challenging to define and measure, but they are still possible to study. It seems clear in retrospect that the Renaissance and Enlightenment presented significant gains, and the Internet led to a complex mesh of benefits and losses. One should eventually create indices on “epistemic abilities” and track these over time and between communities.

One aspect I’d like to smuggle into “Epistemic Progress” is a focus on progress going forward, or perhaps “epistemic futurism”. Epistemic abilities might change dramatically in the future, and it would be interesting to map how that could happen. Epistemic Progress could refer to both minor and major progress, both seem important.

Why not focus on [insert similar term] instead?

I’m not totally sure that “epistemics” is the right frame for my focus, as opposed to the more generic “rationality”, or the more specific “institutional decision making.” As said earlier, there are several overlapping terms floating around. There are tradeoffs for each.

First, I think it doesn’t really matter. What matters is that we have some common terminology with decent enough definitions, and use that to produce research findings. Many of the research findings should be the same whether one calls the subject “epistemics”, “epistemic rationality”, “intellectual development”, or so on. If in the future a more popular group comes out with a different focus, hopefully, they should make use of the work produced from this line of reasoning. The important thing is really that this work gets done, not what we decide to call it.

As to why it’s my selected choice of the various options, I have a few reasons. “Epistemics” is an expression with rather positive connotations. Hopefully the choice of “epistemics” vs. “group thinking” would tilt research to favor actors that are well-calibrated instead of just being intelligent. An individual or group with great decision making or reasoning abilities, but several substantial epistemic problems, could do correspondingly severe amounts of damage. A group with great epistemics could also be destructive, but a large class of failures (intense overconfidence) may be excluded.

I prefer “epistemics” to “decision making” because it gets more to the heart of things. I’ve found when thinking through the use of Guesstimate that often by the time you’re making an explicit decision, it’s too late. Decisions are downstream of general beliefs. For example, someone might make a decision to buy a house in order to shorten their commute, but wouldn’t have questioned whether the worldview that produced their lifestyle was itself dramatically suboptimal. Perhaps their fundamental beliefs should have been continuously questioned, leading them to forgo their position and become a Buddhist monk.

I’ve been thinking about using some specific modifier word to differentiate “epistemics” as I refer to it from other conceptions. I’m trying to go with the colloquial definition that has emerged within the Rationality and Effective Altruist circles, but it should be noted that this definition holds different connotations to other uses of the term. For this essay, two new terms feel like enough. I’m going to reflect on this for future parts. If you have ideas or preferences, please post them in the comments.

Next Steps

Now that we have the key terms, we can start to get into specifics. I currently have a rough outline to formally write a few more posts in this sequence. The broader goal is to help secure a foundation and some motivation for further work in this space. If you have thoughts or feedback, please reach out or post in the comments.