This first book is about epistemology, the study of how we come to know the world.

A healthy epistemology is not only about avoiding sins like self-deception and obfuscation. It is also about living out virtues like curiosity and empiricism, and taking active steps to dissolve confusion and see the world more clearly over time.

We do not understand how many parts of reality work. We also have many motives to not look directly at it or...

A mind must be created already in motion.
There is no argument so compelling
that it will give dynamics to a static thing.
There is no computer program so persuasive
that you can run it on a rock.

This second book is about agency, the ability to take action in the world and control the future.

There’s something very strange about being human. We do not have a single simple goal, but rather a weird amalgam of desires and wants that...

"Indeed. Moving from bad equilibria to better equilibria is the whole point of having a civilization in the first place."

Coordination is the ability of multiple agents to work together.

This is the difficult question of how to work with other agents in the world, and how to have healthy reasoning and decision-making processes within a group.

If many agents in a group are working on different projects, how do we decide on resource allocation between those projects? If...

"A burning itch to know is higher than a solemn vow to pursue truth."

This fourth book is about curiosity, which is the desire to understand how some corner of the world works. 

Curiosity is diving into Wikipedia. It’s running a survey to get data from your friends. It’s dropping balls from different heights and measuring how long they take to fall. Empiricism, scholarship, googling, introspection, data-gathering, science: all these are examples of the curious impulse in...

This fifth book is about alignment, the problem of aligning the thoughts and goals of artificial intelligences with those of humans.

This relates to the art of human rationality in two ways.

First, the laws of reasoning and decision-making apply similarly to all agents, whether they are human or artificial. Many of the deepest insights about rationality at LessWrong have come directly from users’ attempts to grapple with the problem of understanding and aligning artificial intelligences.

Second, the...

The first book is about trust, the belief in something in the absence of understanding.

The Royal Society was founded in 1660, with the motto Nullius in Verba, "On No One's Word". The promise of science was that it would be a method for discovering truth without needing to trust anyone: because experiments can and would be replicated, incorrect theories and incorrect information would be !ltered out. By exploring systems that could be understood in full,...

The second book is about modularity. Well designed or evolved structures are often not just made of parts, but made of parts with simple interfaces. These interfaces allow the parts to be reused in alternative contexts, and thus recombined in different ways.

One of the most important benefits is inspection and debugging. Modular systems are easier to understand, because their components are self-contained. However, this can also be a curse; when components have boundaries that are...

The third book is about incentives, which are the patterns of what is rewarded and what is punished.

The dynamics of what is incentivized and disincentivized in a group is a primary force in determining how that group behaves. When those incentives are aligned with what the individuals actually want, the output can be far greater than the sum of the parts; when they are misaligned with what the individuals want, the results can be far...

The fourth book is about failure. It’s what happens when a system behaves differently from how we expected it to, with adverse consequences for those who were relying on the success of that system. Failure is often as much about misunderstanding how a system works, as it is about the lack of effort or plan to bring the system into a successful configuration.

Almost nothing humans build ever works on the first try. Failure is part...

Reality is that which doesn't go away when you stop believing it. Reason is the process by which we get our beliefs to more accurately reflect reality. 

This collection of essays explores the relationship between the two.

Imagine an advanced society, with complex structures more intricate and intelligent than anything that exists today – a society which nevertheless lacks any being that is conscious, or whose welfare has moral significance. In a sense, this would be an uninhabited society. It would be a society of economic miracles and technological awesomeness, with nobody there to benefit. A Disneyland with no children.

– Nick Bostrom