There's a class of prophecy that runs: "In the Future, machines will do all the work. Everything will be automated. Even labor of the sort we now consider 'intellectual', like engineering, will be done by machines. We can sit back and own the capital. You'll never have to lift a finger, ever again."

But then won't people be bored?

No; they can play computer games—not like our games, of course, but much more advanced and entertaining.

Yet wait! If you buy a modern computer game, you'll find that it contains some tasks that are—there's no kind word for this—effortful. (I would even say "difficult", with the understanding that we're talking about something that takes 10 minutes, not 10 years.)

So in the future, we'll have programs that help you play the game—taking over if you get stuck on the game, or just bored; or so that you can play games that would otherwise be too advanced for you.

But isn't there some wasted effort, here? Why have one programmer working to make the game harder, and another programmer to working to make the game easier? Why not just make the game easier to start with? Since you play the game to get gold and experience points, making the game easier will let you get more gold per unit time: the game will become more fun.

So this is the ultimate end of the prophecy of technological progress—just staring at a screen that says "YOU WIN", forever.

And maybe we'll build a robot that does that, too.

Then what?

The world of machines that do all the work—well, I don't want to say it's "analogous to the Christian Heaven" because it isn't supernatural; it's something that could in principle be realized. Religious analogies are far too easily tossed around as accusations... But, without implying any other similarities, I'll say that it seems analogous in the sense that eternal laziness "sounds like good news" to your present self who still has to work.

And as for playing games, as a substitute—what is a computer game except synthetic work? Isn't there a wasted step here? (And computer games in their present form, considered as work, have various aspects that reduce stress and increase engagement; but they also carry costs in the form of artificiality and isolation.)

I sometimes think that futuristic ideals phrased in terms of "getting rid of work" would be better reformulated as "removing low-quality work to make way for high-quality work".

There's a broad class of goals that aren't suitable as the long-term meaning of life, because you can actually achieve them, and then you're done.

To look at it another way, if we're looking for a suitable long-run meaning of life, we should look for goals that are good to pursue and not just good to satisfy.

Or to phrase that somewhat less paradoxically: We should look for valuations that are over 4D states, rather than 3D states. Valuable ongoing processes, rather than "make the universe have property P and then you're done".

Timothy Ferris is again worth quoting: To find happiness, "the question you should be asking isn't 'What do I want?' or 'What are my goals?' but 'What would excite me?'"

You might say that for a long-run meaning of life, we need games that are fun to play and not just to win.

Mind you—sometimes you do want to win. There are legitimate goals where winning is everything. If you're talking, say, about curing cancer, then the suffering experienced by even a single cancer patient outweighs any fun that you might have in solving their problems. If you work at creating a cancer cure for twenty years through your own efforts, learning new knowledge and new skill, making friends and allies—and then some alien superintelligence offers you a cancer cure on a silver platter for thirty bucks—then you shut up and take it.

But "curing cancer" is a problem of the 3D-predicate sort: you want the no-cancer predicate to go from False in the present to True in the future. The importance of this destination far outweighs the journey; you don't want to go there, you just want to be there. There are many legitimate goals of this sort, but they are not suitable as long-run fun. "Cure cancer!" is a worthwhile activity for us to pursue here and now, but it is not a plausible future goal of galactic civilizations.

Why should this "valuable ongoing process" be a process of trying to do things—why not a process of passive experiencing, like the Buddhist Heaven?

I confess I'm not entirely sure how to set up a "passively experiencing" mind. The human brain was designed to perform various sorts of internal work that add up to an active intelligence; even if you lie down on your bed and exert no particular effort to think, the thoughts that go on through your mind are activities of brain areas that are designed to, you know, solve problems.

How much of the human brain could you eliminate, apart from the pleasure centers, and still keep the subjective experience of pleasure?

I'm not going to touch that one. I'll stick with the much simpler answer of "I wouldn't actually prefer to be a passive experiencer." If I wanted Nirvana, I might try to figure out how to achieve that impossibility. But once you strip away Buddha telling me that Nirvana is the end-all of existence, Nirvana seems rather more like "sounds like good news in the moment of first being told" or "ideological belief in desire" rather than, y'know, something I'd actually want.

The reason I have a mind at all, is that natural selection built me to do things—to solve certain kinds of problems.

"Because it's human nature" is not an explicit justification for anything. There is human nature, which is what we are; and there is humane nature, which is what, being human, we wish we were.

But I don't want to change my nature toward a more passive object—which is a justification. A happy blob is not what, being human, I wish to become.

I earlier argued that many values require both subjective happiness and the external objects of that happiness. That you can legitimately have a utility function that says, "It matters to me whether or not the person I love is a real human being or just a highly realistic nonsentient chatbot, even if I don't know, because that-which-I-value is not my own state of mind, but the external reality." So that you need both the experience of love, and the real lover.

You can similarly have valuable activities that require both real challenge and real effort.

Racing along a track, it matters that the other racers are real, and that you have a real chance to win or lose. (We're not talking about physical determinism here, but whether some external optimization process explicitly chose for you to win the race.)

And it matters that you're racing with your own skill at running and your own willpower, not just pressing a button that says "Win". (Though, since you never designed your own leg muscles, you are racing using strength that isn't yours. A race between robot cars is a purer contest of their designers. There is plenty of room to improve on the human condition.)

And it matters that you, a sentient being, are experiencing it. (Rather than some nonsentient process carrying out a skeleton imitation of the race, trillions of times per second.)

There must be the true effort, the true victory, and the true experience—the journey, the destination and the traveler.