Continuation of: Is Morality Preference?

(Disclaimer: Neither Subhan nor Obert represent my own position on morality; rather they represent different sides of the questions I hope to answer.)

Subhan: "What is this 'morality' stuff, if it is not a preference within you?"

Obert: "I know that my mere wants, don't change what is right; but I don't claim to have absolute knowledge of what is right—"

Subhan: "You're not escaping that easily! How does a universe in which murder is wrong, differ from a universe in which murder is right? How can you detect the difference experimentally? If the answer to that is 'No', then how does any human being come to know that murder is wrong?"

Obert: "Am I allowed to say 'I don't know'?"

Subhan: "No. You believe now that murder is wrong. You must believe you already have evidence and you should be able to present it now."

Obert: "That's too strict! It's like saying to a hunter-gatherer, 'Why is the sky blue?' and expecting an immediate answer."

Subhan: "No, it's like saying to a hunter-gatherer: Why do you believe the sky is blue?"

Obert: "Because it seems blue, just as murder seems wrong. Just don't ask me what the sky is, or how I can see it."

Subhan: "But—aren't we discussing the nature of morality?"

Obert: "That, I confess, is not one of my strong points. I specialize in plain old morality. And as a matter of morality, I know that I can't make murder right just by wanting to kill someone."

Subhan: "But if you wanted to kill someone, you would say, 'I know murdering this guy is right, and I couldn't make it wrong just by not wanting to do it.'"

Obert: "Then, if I said that, I would be wrong. That's common moral sense, right?"

Subhan: "Argh! It's difficult to even argue with you, since you won't tell me exactly what you think morality is made of, or where you're getting all these amazing moral truths—"

Obert: "Well, I do regret having to frustrate you. But it's more important that I act morally, than that I come up with amazing new theories of the nature of morality. I don't claim that my strong point is in explaining the fundamental nature of morality. Rather, my strong point is coming up with theories of morality that give normal moral answers to questions like, 'If you feel like killing someone, does that make it right to do so?' The common-sense answer is 'No' and I really see no reason to adopt a theory that makes the answer 'Yes'. Adding up to moral normality—that is my theory's strong point."

Subhan: "Okay... look. You say that, if you believed it was right to murder someone, you would be wrong."

Obert: "Yes, of course! And just to cut off any quibbles, we'll specify that we're not talking about going back in time and shooting Stalin, but rather, stalking some innocent bystander through a dark alley and slitting their throat for no other reason but my own enjoyment. That's wrong."

Subhan: "And anyone who says murder is right, is mistaken."

Obert: "Yes."

Subhan: "Suppose there's an alien species somewhere in the vastness of the multiverse, who evolved from carnivores. In fact, through most of their evolutionary history, they were cannibals. They've evolved different emotions from us, and they have no concept that murder is wrong—"

Obert: "Why doesn't their society fall apart in an orgy of mutual killing?"

Subhan: "That doesn't matter for our purposes of theoretical metaethical investigation. But since you ask, we'll suppose that the Space Cannibals have a strong sense of honor—they won't kill someone they promise not to kill; they have a very strong idea that violating an oath is wrong. Their society holds together on that basis, and on the basis of vengeance contracts with private assassination companies. But so far as the actual killing is concerned, the aliens just think it's fun. When someone gets executed for, say, driving through a traffic light, there's a bidding war for the rights to personally tear out the offender's throat."

Obert: "Okay... where is this going?"

Subhan: "I'm proposing that the Space Cannibals not only have no sense that murder is wrong—indeed, they have a positive sense that killing is an important part of life—but moreover, there's no path of arguments you could use to persuade a Space Cannibal of your view that murder is wrong. There's no fact the aliens can learn, and no chain of reasoning they can discover, which will ever cause them to conclude that murder is a moral wrong. Nor is there any way to persuade them that they should modify themselves to perceive things differently."

Obert: "I'm not sure I believe that's possible—"

Subhan: "Then you believe in universally compelling arguments processed by a ghost in the machine. For every possible mind whose utility function assigns terminal value +1, mind design space contains an equal and opposite mind whose utility function assigns terminal value—1. A mind is a physical device and you can't have a little blue woman pop out of nowhere and make it say 1 when the physics calls for it to say 0."

Obert: "Suppose I were to concede this. Then?"

Subhan: "Then it's possible to have an alien species that believes murder is not wrong, and moreover, will continue to believe this given knowledge of every possible fact and every possible argument. Can you say these aliens are mistaken?"

Obert: "Maybe it's the right thing to do in their very different, alien world—"

Subhan: "And then they land on Earth and start slitting human throats, laughing all the while, because they don't believe it's wrong. Are they mistaken?"

Obert: "Yes."

Subhan: "Where exactly is the mistake? In which step of reasoning?"

Obert: "I don't know exactly. My guess is that they've got a bad axiom."

Subhan: "Dammit! Okay, look. Is it possible that—by analogy with the Space Cannibals—there are true moral facts of which the human species is not only presently unaware, but incapable of perceiving in principle? Could we have been born defective—incapable even of being compelled by the arguments that would lead us to the light? Moreover, born without any desire to modify ourselves to be capable of understanding such arguments? Could we be irrevocably mistaken about morality—just like you say the Space Cannibals are?"

Obert: "I... guess so..."

Subhan: "You guess so? Surely this is an inevitable consequence of believing that morality is a given, independent of anyone's preferences! Now, is it possible that we, not the Space Cannibals, are the ones who are irrevocably mistaken in believing that murder is wrong?"

Obert: "That doesn't seem likely."

Subhan: "I'm not asking you if it's likely, I'm asking you if it's logically possible! If it's not possible, then you have just confessed that human morality is ultimately determined by our human constitutions. And if it is possible, then what distinguishes this scenario of 'humanity is irrevocably mistaken about morality', from finding a stone tablet on which is written the phrase 'Thou Shalt Murder' without any known justification attached? How is a given morality any different from an unjustified stone tablet?"

Obert: "Slow down. Why does this argument show that morality is determined by our own constitutions?"

Subhan: "Once upon a time, theologians tried to say that God was the foundation of morality. And even since the time of the ancient Greeks, philosophers were sophisticated enough to go on and ask the next question—'Why follow God's commands?' Does God have knowledge of morality, so that we should follow Its orders as good advice? But then what is this morality, outside God, of which God has knowledge? Do God's commands determine morality? But then why, morally, should one follow God's orders?"

Obert: "Yes, this demolishes attempts to answer questions about the nature of morality just by saying 'God!', unless you answer the obvious further questions. But so what?"

Subhan: "And furthermore, let us castigate those who made the argument originally, for the sin of trying to cast off responsibility—trying to wave a scripture and say, 'I'm just following God's orders!' Even if God had told them to do a thing, it would still have been their own decision to follow God's orders."

Obert: "I agree—as a matter of morality, there is no evading of moral responsibility. Even if your parents, or your government, or some kind of hypothetical superintelligence, tells you to do something, you are responsible for your decision in doing it."

Subhan: "But you see, this also demolishes the idea of any morality that is outside, beyond, or above human preference. Just substitute 'morality' for 'God' in the argument!"

Obert: "What?"

Subhan: "John McCarthy said: 'You say you couldn't live if you thought the world had no purpose. You're saying that you can't form purposes of your own-that you need someone to tell you what to do. The average child has more gumption than that.' For every kind of stone tablet that you might imagine anywhere, in the trends of the universe or in the structure of logic, you are still left with the question: 'And why obey this morality?' It would be your decision to follow this trend of the universe, or obey this structure of logic. Your decision—and your preference."

Obert: "That doesn't follow! Just because it is my decision to be moral—and even because there are drives in me that lead me to make that decision—it doesn't follow that the morality I follow consists merely of my preferences. If someone gives me a pill that makes me prefer to not be moral, to commit murder, then this just alters my preference—but not the morality; murder is still wrong. That's common moral sense—"

Subhan: "I beat my head against my keyboard! What about scientific common sense? If morality is this mysterious given thing, from beyond space and time—and I don't even see why we should follow it, in that case—but in any case, if morality exists independently of human nature, then isn't it a remarkable coincidence that, say, love is good?"

Obert: "Coincidence? How so?"

Subhan: "Just where on Earth do you think the emotion of love comes from? If the ancient Greeks had ever thought of the theory of natural selection, they could have looked at the human institution of sexual romance, or parental love for that matter, and deduced in one flash that human beings had evolved—or at least derived tremendous Bayesian evidence for human evolution. Parental bonds and sexual romance clearly display the signature of evolutionary psychology—they're archetypal cases, in fact, so obvious we usually don't even see it."

Obert: "But love isn't just about reproduction—"

Subhan: "Of course not; individual organisms are adaptation-executers, not fitness-maximizers. But for something independent of humans, morality looks remarkably like godshatter of natural selection. Indeed, it is far too much coincidence for me to credit. Is happiness morally preferable to pain? What a coincidence! And if you claim that there is any emotion, any instinctive preference, any complex brain circuitry in humanity which was created by some external morality thingy and not natural selection, then you are infringing upon science and you will surely be torn to shreds—science has never needed to postulate anything but evolution to explain any feature of human psychology—"

Obert: "I'm not saying that humans got here by anything except evolution."

Subhan: "Then why does morality look so amazingly like a product of an evolved psychology?"

Obert: "I don't claim perfect access to moral truth; maybe, being human, I've made certain mistakes about morality—"

Subhan: "Say that—forsake love and life and happiness, and follow some useless damn trend of the universe or whatever—and you will lose every scrap of the moral normality that you once touted as your strong point. And I will be right here, asking, 'Why even bother?' It would be a pitiful mind indeed that demanded authoritative answers so strongly, that it would forsake all good things to have some authority beyond itself to follow."

Obert: "All right... then maybe the reason morality seems to bear certain similarities to our human constitutions, is that we could only perceive morality at all, if we happened, by luck, to evolve in consonance with it."

Subhan: "Horsemanure."

Obert: "Fine... you're right, that wasn't very plausible. Look, I admit you've driven me into quite a corner here. But even if there were nothing more to morality than preference, I would still prefer to act as morality were real. I mean, if it's all just preference, that way is as good as anything else—"

Subhan: "Now you're just trying to avoid facing reality! Like someone who says, 'If there is no Heaven or Hell, then I may as well still act as if God's going to punish me for sinning.'"

Obert: "That may be a good metaphor, in fact. Consider two theists, in the process of becoming atheists. One says, 'There is no Heaven or Hell, so I may as well cheat and steal, if I can get away without being caught, since there's no God to watch me.' And the other says, 'Even though there's no God, I intend to pretend that God is watching me, so that I can go on being a moral person.' Now they are both mistaken, but the first is straying much further from the path."

Subhan: "And what is the second one's flaw? Failure to accept personal responsibility!"

Obert: "Well, and I admit I find that a more compelling argument than anything else you have said. Probably because it is a moral argument, and it has always been morality, not metaethics, with which I claimed to be concerned. But even so, after our whole conversation, I still maintain that wanting to murder someone does not make murder right. Everything that you have said about preference is interesting, but it is ultimately about preference—about minds and what they are designed to desire—and not about this other thing that humans sometimes talk about, 'morality'. I can just ask Moore's Open Question: Why should I care about human preferences? What makes following human preferences right? By changing a mind, you can change what it prefers; you can even change what it believes to be right; but you cannot change what is right. Anything you talk about, that can be changed in this way, is not 'right-ness'."

Subhan: "So you take refuge in arguing from definitions?"

Obert: "You know, when I reflect on this whole argument, it seems to me that your position has the definite advantage when it comes to arguments about ontology and reality and all that stuff—"

Subhan: "'All that stuff'? What else is there, besides reality?"

Obert: "Okay, the morality-as-preference viewpoint is a lot easier to shoehorn into a universe of quarks. But I still think the morality-as-given viewpoint has the advantage when it comes to, you know, the actual morality part of it—giving answers that are good in the sense of being morally good, not in the sense of being a good reductionist. Because, you know, there are such things as moral errors, there is moral progress, and you really shouldn't go around thinking that murder would be right if you wanted it to be right."

Subhan: "That sounds to me like the logical fallacy of appealing to consequences."

Obert: "Oh? Well, it sounds to me like an incomplete reduction—one that doesn't quite add up to normality."

Part of The Metaethics Sequence

Next post: "Where Recursive Justification Hits Bottom"

Previous post: "Is Morality Preference?"