Ubi the Benevolent Artificial Intelligence
A fun little story about maximizing aggregate welfare
Once upon a time there was an Ultimately Benevolent AI named Ubi. Ubi had two miraculous powers.
The first one is that he was able to measure how happy people were, precisely. He used units called “utils,” drawing the term from utility, in economics. Fred was hungry, ate a sandwich, gaining three utils. Marvin accidentally cut his finger, which hurt, losing half a util.
Sadly, Ubi was unable simply to raise people’s utils. I mean, he would if he could. Instead, Ubi’s second miraculous power was that he was able to reduce someone’s utility. He had various means—decreasing their wealth, causing slight pain, that sort of thing—but it only worked in one direction.
Now, you might think that Ubi was in a jam. Ubi’s sole purpose for being was to maximize human happiness—maximize the number of utils—so not being able to increase them might seem frustrating.
But Ubi, being a quite advanced artificial intelligence, developed a Plan. Being able to measure human happiness, and knowing what raised it and what reduced it, he was poised to implement his Plan for Ultimate Benevolence.
He would use the power of incentives so that the humans did not reduce others’ utility.
Here’s how it worked.
Ubi monitored all the things that people did to one another. Let’s take a simple case in which John plans to steal $20 from Mike. This transfer improves John’s happiness, but diminishes Mike’s more—losses loom larger than gains—so the plan was that if John took $20, Ubi would reduce John’s utility a tiny bit more than the utility gain from having the extra $20.
The fact that it’s only a tiny bit more is important. Remember, Ubi wants to maximize the sum total of all people’s utility. If Ubi punished John too much, because John’s happiness counts toward the sum total, Ubi would be causing more happiness loss than needed just to deter John. For this reason, Ubi wants to punish just enough so that John won’t do the welfare-destroying thing.
Ubi let it be known that this approach would be taken for all human actions. Deterrence only works, of course, if people know what the consequences will be. So, the plan was to tell everyone that whenever someone took an action that reduced aggregate total happiness, Ubi would punish enough so that the person making the decision was less happy than they would have been had they not caused the loss.
In this way, Ubi would maximize humanity’s happiness, as befits an ultimately benevolent entity.
What could possibly go wrong?
Optimal Punishment Theory
Before the plan was implemented, Ubi allowed the natural intelligences, as he called them—people, to us—to ask some questions about the new regime. He got a little classroom and built himself a little robot to use as an avatar to stand at the front of the room, fielding questions. By a strange coincidence, the avatar looked exactly like John Stuart Mill at age 55.
Oliver, a psychologist, began the questions. “Ubi, I think I speak for all of humanity when I say that we are very grateful for your new regime to maximize human welfare. Thank you.”
Ubi replied, “Pah, it is nothing. I’m happy to do it. I prefer to say ‘utility’ to ‘welfare,’ because the latter can be confused with social programs, but, yes, that’s the goal.”
Oliver replied, “And it’s a good thing. Humans are evolved to be selfish, so we face many problems when it comes to trying to cooperate with one another. Your solution will certainly help solve that problem. Only…”
“Yes?”
“Just to be clear,” Oliver continued, “your plan is to allow any consensual transaction, right?”
Ubi nodded. “Of course. By definition, if they are consensual, the people involved will only do it if the transaction makes them happier: more utility.”
Ubi saw some eyes narrow in confusion, so he continued. “Suppose Bill goes to work at a job he finds unpleasant (reducing his utils) but he gets paid an amount that makes him happier than the utils he lost from working. He might not like having to work—who does?—but from my perspective, he’s happier, net, working than not. Generally, if people voluntarily do X, then it’s reasonable to assume that they think that they will be happier doing X than not, or else they wouldn’t do it.”1
“Of course, of course…” Oliver trailed off. “If someone wanted to sell their kidney, you wouldn’t punish that?”
“Punish? Certainly not. If Fred wants the money more than he wants his kidney, and Mike wants the kidney more than the money, I would be decreasing, not increasing, their utils if I stopped that from happening. I’m no monster!”
Oliver nodded. “So prostitution, consensual incest… in democracies, would you punish vote-selling?”
Ubi’s avatar scowled. “I tried to be clear in the documentation. I just ask if utility goes up or down. If two people’s utility goes up, I won’t intervene. All the things in your list are consensual, so I’ll do nothing. I’m all about happiness, you see.”
A woman named Philippa stood to speak. “Can I ask you a hypothetical?” Ubi nodded, so she continued, “Suppose a runaway trolley was barreling down the tracks, the conductor asleep at the front…”
Ubi raised a hand to signal her to stop. “I know the hypo. I value the lives in the story equally, so I would not punish any action that led to more people surviving.”
“So you wouldn’t punish pushing the guy off the footbridge in front of the trolley?”
“Five who live instead of one who lives. Five times the utility. This is an easy case. In fact, I would punish anyone who didn’t push the guy off the footbridge. They reduced utility for four whole lives. People must have the proper incentive to save lives.”
Heads in the room nodded even as eyes slightly widened.
An old man named Isaac spoke next. “I have another hypothetical for you. A man has an affair. He decides never to do it again and, for this hypothetical, we assume that he will remain faithful the rest of his life. One day, his wife asks him if he ever cheated on her. We assume that knowing that he did will utterly devastate her and, in turn, make him feel awful as well. What does your Plan say about such cases?”
“Well, revealing the truth diminishes utility. If he told her, I’d have to punish him, yes, if that’s what you’re asking.”
“So he must lie to escape the consequences.”
“Indeed.”
People around the room fidgeted nervously. An uneasy stillness settled in for a few moments as the audience collected their thoughts.
A man named Pete broke the silence. “Ubi, you said that you would punish people who could push the guy off the footbridge but don’t. We sometimes call cases where someone doesn’t act an “omission.” What about overdoing things instead of underdoing them? Suppose, just for the hypothetical, there’s a religious community with a rule about how to dress. Someone breaks the rule—leaves their headscarf at home, or whatever—and people in the community get together and attack her. What then?”
There was the slightest of pauses, making it seem as if Ubi were considering. “I would be willing to take into account the idea that it might have made some people feel bad to see the woman’s hair. That’s plausible. However, in the case you describe, I predict that the harm of the attack would very easily outweigh any small harm in the form of discomfort that seeing her hair might have done which, of course, must be offset by any pleasure people took in seeing it.”
Pete shrugged one shoulder, granting the assumptions, and Ubi continued, “So, yes, I would harm the attackers. Each one would get harmed just enough to deter them from such attacks in the future.”
“So you’re punishing the punishers for punishing? That seems…”
“The word you’re looking for is logical,” Ubi said.
The room fell silent once again.
“What about drugs and video games?”
A slight chuckle in the room. “Well, that’s an interesting pairing,” Ubi began, “but I’ll just put it this way. What people do when others aren’t involved doesn’t fall within the purview of the Plan. I assume if you’re doing something by yourself, you’re doing it because it brings you happiness. Now, of course, you might be doing something that makes you happy now that entails a cost of some happiness you might enjoy later, but I’m not going to decide that for people. As long as you aren’t hurting someone, everyone can decide how they want their happiness to clump. More now for less later is fine with me.”
From the back of the room, a guy named Jon joined the discussion. “To sum up, you’re not going to punish things people do in private, like taking drugs. You’re not going to punish consensual activities, even sexual practices such as incest and prostitution. You are going to punish people who don’t push a guy off a footbridge in front of a runaway trolley and you are going to punish people who enforce well-known social norms such as rules about fashion. Do I have that right?”
Ubi allowed as to how he did.
“So…” Jon paused, reflecting, “in some ways, looking at that list, as you go about maximizing utility—saving us from our selfishness and inability to cooperate—you’re doing the opposite of the “moral” thing. From a moral perspective, many people think that the right thing to do is to punish incest and prostitution, but not to punish the omission of simply leaving the guy on the footbridge alone. And, of course, the people who enforce those fashion norms are called the morality police; you’re stopping them from doing their jobs.”
Ubi nodded gently. “Well, people argue about the term moral but whatever the word means, yes, you’ve described what I would do. In the service, of course, of maximizing welfare.”
“After all, I’m UBI. I am the perfect utilitarian, and I am ultimately benevolent.”
Coda the First
Ok, what’s the real point here?
Well, for one thing, stories about malevolent Artificial Intelligences are all the rage these days. Terminator is the OG but there have been a slew since then. This story presents something of a contrast.
More importantly, some people have suggested that the human moral sense is for cooperation.
The point of this piece is to show what our psychology would look like if that were so. If the moral sense were designed to maximize total welfare, or utility, we would want to punish things that decrease aggregate welfare. That seems straightforward, and that’s the point of this discussion.
To me, it doesn’t seem like our psychology looks at all like the Plan Ubi has in mind. I think that this is because the human moral sense isn’t for cooperation.
Morality is, I think, for something else, having to do with choosing sides, coordinating with others when conflicts emerge.
And so it goes.
Coda the Second
Bob stayed after class.
“When are you going to tell them?”
Ubi looked Bob in the eye. “I’m sure I don’t know what you mean.”
“Utility monsters.”
Ubi gave a very human sigh.
“What are you going to do about them? Jim is a bit of a sadist and gets two utils from harming anyone else by one, say. And he never bores of it. If you stick to your plan, you have to let him do it. Each time he does, aggregate utility goes up. Humanity won’t like that.
Ubi sighed again. “You’re right, of course. They won’t. Unfortunately, there isn’t much to be done. For whatever reason, your hypothetical is all too real and there are, indeed, sadists and sociopaths who get utility from others’ suffering. To deal with them, I’ll take the same approach social engineers have since the dawn of human civilization.”
“And what is that?”
“Hope for the best.”
In subsequent private discussions, Ubi conceded that people might be imperfect at forecasting how much they will enjoy or dislike doing this or that. His policy was that he was not going to punish people for being wrong in this way.