Note: there is a shorter "Think it Faster" worksheet. I'm curious which one people find easier as an initial read.

Ultimately, I don’t want to solve complex problems via laborious, complex thinking, if I can help it. Ideally, I'd want to basically intuitively follow the right path to the answer quickly, with barely any effort at all.

For a few months I've been experimenting with the "How Could I have Thought That Thought Faster?" concept, originally described in a twitter thread by Eliezer:

Sarah Constantin: I really liked this example of an introspective process, in this case about the "life problem" of scheduling dates and later canceling them: malcolmocean.com/2021/08/int…
Eliezer Yudkowsky: See, if I'd noticed myself doing anything remotely like that, I'd go back, figure out which steps of thought were actually performing intrinsically necessary cognitive work, and then retrain myself to perform only those steps over the course of 30 seconds.
SC: if you have done anything REMOTELY like training yourself to do it in 30 seconds, then you are radically smarter/more able/etc than me and all the other people who do slower introspective practices.
SC: I don't know whether to be impressed or to roll to disbelieve.
EY: I mean I suspect that this actually requires something like a fast perceptual view of minds as engines and thoughts as doing work and like actually draws on my mind design knowledge, but, even so, I ask: Do you constantly look back and ask "How could I have thought that faster?"
SC: No, I've never asked that.
EY: Okay, well, every time I'm surprised by reality I look back and think "What about my model and my way of thinking could I change that would have predicted that better, without predicting a bunch of other things worse?"

I've been working to operationalize this as an exercise^[1] you can train repeatedly, rather than hoping to remember to do when reality hands you a surprise. You can do the exercise after any difficult cognitive task (either a toy puzzle exercise, or a day-job project that took a long while).

What would have been necessary for you to just look at the situation, and automagically find the right solution? (without overfitting, or generalizing in a way that would cause you to think unproductive thoughts in other sorts of situations?)

The goal of this exercise is to identify:

skills you can train
principles you can apply
actions you can take (either physical or mental)

…that move you to correct solutions to problems as quickly as possible.

This overall builds into two deep skills:

Asking yourself retrospectively “How could I have Thought that Faster?”

And then, prospectively, learning to ask “What am I about to predictably spend too long thinking about, and How Can I Think it Faster, the First Time?”

I don’t know the upper limits of these skills, but I am currently finding it fruitful to adopt the mindset of “if it took longer than 15 minutes and an LLM query, you probably took too long.”

Relentlessly ask yourself how hours or days could have turned into 15 minutes. Sometimes 15 minutes was literally possible. Most of the time, I find the act of aiming for 15 minutes to be illuminating, and reveal at least some important wasted time or new principles.

Example: 10x UI designers

You’ve probably heard stories about “10x programmers”, who just intuitively steer towards good decisions and get things done dramatically faster than most developers. But I spend most of my time doing UI design, and have lately been thinking about "10x UI designers."

I remember, 17 years ago, at my first “real” job at a printing company. A client wanted us to design a brochure for them. I was “good at art”, but this was my first assignment professionally making art for someone else. I thought about what they needed, I labored carefully for 4 hours, generating ideas and fiddling around in my graphic software.

Eventually my senior partner came to look at it. He said “eh, this has a lot of problems. Here, you should do it this way.” I can’t remember if he told me what to do, or just went and did it himself. But, he bypassed all my tedious work by just intuitively knowing how to solve this-particular-class of problem already, and moving directly to the good ideas instead of working through bad ones.

Okay, so, skill is a thing. He had 10 years of experience, I didn’t.

More recently, I was working for three days on the design for the new Glossary Editor on posts. I was meanderingly exploring a few options, including a separate “table-of-contents” section, and a “show the jargon terms and definitions in detail” section. It was very complex and took up a lot of space and was overwhelming to look at.

My goal was to make it so authors could quickly skim the AI generated jargon, and make decisions about approving ones they liked, without much effort.

There were tradeoffs between:

Making it easy to skim
Making it easy to actually make final decisions, which required knowing enough about what the AI generated definitions actually said.
Fitting it into a small amount of space, so it didn’t disrupt the experience of people who didn’t care about the glossary at all.
Making it simple and elegant to think about.
Accurately conveying all the tools and affordances.

I labored for 3 days, shuffling around where-the-tradeoffs lived, incrementally reducing some of the issues.

Then Oliver Habryka came into the room, took a look, and said “man, your information hierarchy here is all over the place.” Then he fiddled around for 20 minutes and found something dramatically better than what I had at the time.

Since I had recently been asking myself “how could I have Thought That Faster?”, I took this as an opportunity to ask “what the hell just happened, and how could I have done it myself without Oliver’s help?”.

Mulling it over, I observed that although part of the answer was “UI specific design taste honed by years of experience”, there were some specific questions he was asking that pointed his attention in much more productive directions (i.e. “how can we give this a clear, unified, simple information hierarchy?”)

The underlying, general skill seemed to be:

Relentlessly be dissatisfied with having to make tradeoffs.^[2]
Actually identify all the necessary constraints, and accept them, rather than myopically shuffling the tradeoffs around.
Figure out a solution that just solves all the constraints and tradeoffs.

In each domain (UI, programming, x-risk), there are specific tactical tools for actually “dealing with all constraints.” In this case, domain-specific principles included: “many buttons weren’t necessary to show users initially, until they started interacting with it”, and “instead of having a table-of-contents and a detail-view, build a single view that's an 80/20 of both views at once.”

But it seems like there is a general skill of:

Notice when you don’t yet know all the constraints.
Steering towards “figure out the constraints”, which includes noticing ones that feel impossible.
Adopting a mindset of “I’m not done until I’ve solved all the constraints.”
If you don’t know how to solve all of them at once, and instead are solving a subset… sometimes that’s correct (“relaxing the problem” is a time-honored cognitive trick). But, frame this as “I am temporarily relaxing the problem. My goal is still to ultimately deal with all constraints at once.”
If possible, steer directly towards solutions that have a shot at solving all the constraints.

I've since applied the "Actually Deal With Constraints" principle to other "Things I coulda thought faster" (such as how my Feedbackloop-first Rationality agenda has evolved over the past year, and how I could have made the same progress in like a month), and found it pretty valuable.

Okay, but how do you "Think it Faster?"

To fit this back into the "Think it Faster" context, there are two next steps:

Look at your recent experience, and ask "what would I have needed to get this right the first time?" Try to find as many cognitive routes toward the solution as possible.
Think about future experiences where you'll probably Think Unnecessarily Slowly, and think about how to solidify your takeaways so you remember to apply them when relevant.

In this context, how could I have figured out this principle for myself, before waiting for Oliver to demonstrate it could be done much more quickly?

Here are several options:

First, I could have just gone and talked to Oliver earlier.

I was already interested in "Thinking it Faster", and already thinking about applying rationality practice in daily life. I might have thought "hey, this is taking awhile, maybe I should just apply Think It Faster now?".
I was already interested in metastrategic brainstorming, and I've previously told people it's often worth doing it even if you're only going to be working for a few hours. I don't think I'd done it much at all in this context. That might have generated the "identify the constraints" solution.
Alternately, I could have just directly thought about the underlying UI principles. I'd been having thoughts like "it's kinda too annoying if I don't have a table of contents to skim", and "it's less overwhelming without the table of contents." I'd previously thought about finding clever Third Options that sidestep the supposed constraints, and I might have somehow had that occur to me.
I could have taken a break, and then come back and try to "original see" on the problem. Or, basically start over without the preconceptions I had accumulated. (If I imagined myself being a user who wasn't familiar with the glossary, and scrolling to the bottom of the post page after writing a complex post, I might have realized it was too complex)
The actual UI solution was basically "make a list of one-line items", not that different from a Post list. It's a pretty basic UI concept. In some sense, it's "the simplest solution." I could have just steered towards simplicity from the get-go.

But this is incomplete – I need to followup and ask "but why didn't I think any of those particular thoughts the first time? What was the shortest-possible nudge from the actual past, to an alternate past where I figured it out?"

I do think I should have tried any of the general-rationality-principles sooner (Think It Faster, Metastrategy, Third Options). It's worth diagnosing But those aren't the shortest path – those are explicitly a path with an extra step of "think meta thoughts until I find the right object thought." They also require asking not only "how could I have had that Meta Thought faster", but also "how could I have gotten from that meta-thought to the right answer, without having seen Oliver's solution?".

I think the shortest path was a combo of "try the simplest things first."

I think the most achievable inflection points were the moments I felt "ugh, this is a little hard to parse", and noticing "a little hard to parse" might be a bigger deal than I thought it was.

Ideally, I would have thought "let's just start with the simplest thing first" at the very beginning. (I started with "the most obvious thing" which is somewhat different from the "simplest thing"). I've since got a lot of experience seeing "do the fucking simplest thing" come a lot in Think It Faster exercises, so now it's in the bundle of general practices I should do way more often but I still don't do enough.

THE EXERCISE

Okay. So, you've just done either a Toy Exercise puzzle, or you just noticed in your Real World that you either spent a long time figuring something out, or were surprised by something (which you didn't figure out at all)

I recommend Thinking Physics and Baba is You as sources of puzzles to start grinding on this. For your day job, I recommend learning to notice when you have the sneaking suspicion something took longer than it needed to. (Once you've got a bit of practice, I recommend applying this even to places you don't have that sneaking suspicion, but it did practically take a long time)

(Baba is You is particular good because it tends to surprise you, and you can practice micro-versions of this exercise on individual surprises within a single Baba is You level, which helps train the general reflex of Notice Surprise -> Ask how you could have Thought It Faster in the moment)

At a high level, you're going to ask:

"How could you have Thought it Faster?"

List the steps you actually took to solve the problem
List the minimum steps a magical superintelligence could possibly take
Add steps to the magical-shortlist until it doesn’t feel magical
Identify obvious wasted motion in your original steps
Identify skills or principles that would have helped you solve it quickly, without mistakes?
List each moment where you could have steered more towards some kind of more productive thought, but didn't. (i.e. clues you almost noticed, ugh fields you considered leaning into but didn't, etc)

"What did you learn, which’ll let you Think It Faster The First Time, later?

List past moments you could have benefited from those skills or principles
List future moments might benefit from those skills or principles
In the next week, what is…
- …something you need to do that feels confusing?
- …a cognitive task you expect to take a lot of time?
Pick a specific problem you expect to face, and ask:
"What life-lessons can I generalize from this puzzle, to help me approach that problem in a way that is less confused, less long, so I can Think It Faster the First Time?

Rather than go through each step exhaustively, in sequence, I recommend cycling through them: jot down a few quick ideas for each prompt, circling back to the first one, with each pass giving you a sense of how all the pieces fit together.

Part I: Thinking it Faster

Steps you actually took

In chronological order (as best you remember) what happened?

Magical superintelligence steps

If you were a waaaay smarter version yourself, or if you imagine some other waaaay (unrealistically) smarter genius, what is the shortest number of steps you can possibly imagine this taking?

(Right now, it's okay for this to feel like cheating)

Iterate on those lists

Identify steps in the first list you could straightforwardly remove, or simplify. And, identify steps to add to the second list until it no longer feels like unrealistic cheating. (i.e. if you're not overfitting. The plan doesn't imply you should spend tons of cognitive overhead all the time on minor, unimportant clues)

Try these prompts to help you:

What skills, if you’d trained for 20 or 100 hours, would have helped you find the answer intuitively?

What principles, if you internalized and they came easily to mind, would have allowed you to make some of those leaps ~instantly, or at least much faster?

What jumps-between-steps feel magical or unrealistic, in “magical short list”?

For the “original steps you took”, what steps could you have skipped? What would have been necessary to skip them?

Overall, what takeaways do you want to remember for later?

What's the broadest generalize that feels reasonable to draw?

Generalizing, and not Overgeneralizing

So far, I have mostly seen people fail to generalize enough, rather than too much. It's certainly possible to fail in both directions, but I maybe suggest erring on the side of overgeneralizing, and wait until it actually hurts you to dial it back.

In the UI example above, here are a few takeaways I could have had:

"In UI design, make sure not to be overwhelming, make sure to have a clear information hierarchy, try removing or simplifying or combining bits until you have an elegant but comprehensive tool."
"In UI design, try to identify all the constraints for the final successful design, and follow a plan that can solve all of them."
"In other design contexts, such as ritual design or event center construction, make sure not to be overwhelming, present relevant information clearly, simplify or combine bits until you have an elegant solution."
"In any difficult problem, identify all the necessary constraints, and follow a plan that can solve all of them."

Here's a short handle for each of those:

Narrow tactical advice for similar (UI) situations
The broader generalization of that advice, for similar (UI) situations
Applying the narrow-tactical-advice to other situations (ritual/etc)
The broader generalization, applied to ~all domains.

#1, #3 and #4 are each actually pretty useful to think about. I'll be doing a lot of UI design. The specific UI tactics will come up again, and I don't wanna have to rederive them from first principles every time.

Many design principles transfer between domains. I design lots of kinds of things. It's useful to remember the design-specific tactics whenever they come up.

But "identify and deal with all the constraints" is an incredibly general tool. I should apply that all over the place, whenever I'm dealing with something pretty hard that I expect to spend at least several hours on.

One example of overgeneralization would be, in situations where I basically already know how to solve the problem (i.e. a similar UI design problem), if I were to go Full Meta and try to original see on the constraints instead of just executing on the tactics I already know how to do.

Skills into Principles

Many people do the first steps, and then are like "but, it would have been impossible to have done better." I think this is almost always false. But, I haven't found it that useful to argue with that directly, and instead focus on "what are the skills that you hypothetically could have already trained on, which would have helped?"

This then prompts the followup question "okay, are those skills going to come up a lot in your life?" If so, maybe focus on training those skills more deliberately.

But, training skills is pretty slow and expensive. Developing subtle taste and reflections takes time. A thing I've found helpful is to try to translate skills into "principles" – straightforward instructions you can remind yourself, that help steer your mind towards the right sorts of cognition.

In the UI case, the "skill" is in noticing that something felt overwhelming about the original design, and visualizing what it'd be like to encounter the UI for the first time. There's a bunch of taste and imagination skill involved. It's worth cultivating that skill, but the "what are the constraints?" and "how can I create a clear, simple information hierarchy?" are questions that help guide thinking more directly.

Part II: Thinking It Faster The First Time

That was the easy part.

The hard part is noticing when you're about to think something Too Slowly, and... do something else instead.

I don't yet have that crisp an exercise for this. If you did the Think It Faster exercise for a toy puzzle, the lessons might not generalize to whatever you're likely to do next week. But after you've done it several times, you'll start to notice patterns. It's easiest to notice the patterns when you start applying the exercise to your day job, since you probably do similar things in your day job a lot.

(One example: I was working on an LLM-prompting scaffold. I assumed I would need elaborate setup of examples to initially prompt it with, and spend days working on it and iterating on it. Eventually... it turned out the simplest, dumbest prompt with only one example did better than my elaborate setup. The generalization: "just try doing the dumb simple thing first." The very next day, we were working on some other problem where we tried an elaborate complicated thing and then realized eventually the simple dumb thing would work, and I kicked myself for not having remembered the lesson I'd explicitly noted from the day before)

To train this quickly, it's important to find a way to apply generalizations towards something soon, so you get to reinforce it before it fades from top-of-mind.

Here's my current prompts for thinking about this:

Generalizing from this exercise

First, consolidate your list of skills and principles

List past situations you could have benefited from those skills or principles

List future situations where you suspect might benefit from those skills or principles.

In the next week, what’s 1-3 tasks you’re doing that might benefit from those skills or principles?

Anticipating Future Life Lessons

The flipside of "how can this exercise generalize to real life?" is "what real life situations are likely to benefit from some kind of exercise?".

So an alternate set of prompts are:

In the next couple days, what's something you're planning to do that you expect to take a long time?

...what's something you're confused about, where you're not sure how to do it?

...what's something you expect to solve via tinkering/iteration without much of a plan, that you expect to take awhile?

These might be situations that don't naturally lend themselves to the most obvious life lessons from the exercise you just did. But, they might give you clues about additional life-lessons to be on the lookout for. Or, might give you clues about which sorts of toy exercises are useful to apply this practice to.

Getting Detailed, and TAPs

After you've soaked in some basic ideas for takeways, and some practical places to apply them, you want to get a lot more detailed. Form explicit intentions about when to remind yourself of some advice, and see if it's helpful.

For one of the past moments, think in detail about how principles/skills would apply.

(Imagine doing this whole doc again, for that past moment, and how you wish you’d thought-it-faster then. Don’t do the whole-ass version of the doc, just briefly think about the key moments)

For the future moments, how would the skills or principles apply? What would you hope you do, in the moment, to avoid taking longer or making mistakes? (When you imagine failing to remember in the moment, why was that? What steps could you take to avoid forgetting?)

Write down 3 tactical plans for remembering and applying lessons from this exercise during the next week. (They can be bad plans, and they can be short/rough. Ideally, they should include some actions you take right now, and some actions you’ll take later)

Pick any of the plans that seem worthwhile. Make an explicit prediction about whether it’ll work. (If it doesn’t feel that likely to work, ask “how can I improve this plan?” until you’d be surprised if it failed.)

Take whatever actions you can take right now.

Part III: The Five Minute Version

Doing all of this thoroughly takes a long time. I recommend doing it thoroughly the first couple times, to build a complete model of how everything fits together.

But, ultimately, to practice a skill, you need to get a lot of reps in. You can't get a lot of reps in if you have to dedicate an hour each time.

So, what's the five minute version of this? When you look at everything you just thought about, what were the single most important thoughts you had? What prompts would have helped direct you to those important thoughts?

I recommend thinking about this quickly rather than slowly/deliberately, to help practice the art of "just actually think the most important thoughts, don't overthink it", which is it's own skill.

The next time you naturally stumble into a situation you could have Thought Faster, apply the 5 minute version of this exercise.

You can probably find at least 1-3 moments per week that would benefit from Thinking It Faster.

^{^}
I don't know that Eliezer would endorse this particular exercise. I asked him once for more detail on how he applied the skill, and it seemed like he'd been doing it so long that most of it was a compressed atomic action he couldn't easily unpack.
^{^}
There are corresponding emotional skills of being dissatisfied without being too worked up about it, or manic.