Could artificial intelligence make the moral choice?
Today’s question is, could an artificial intelligence software program — an “AI Agent” — earn the Carnegie Medal for Heroism? Obviously, not. Andrew Carnegie created the Hero Fund with a Deed of Trust that defines our mission as honoring extraordinary human heroes. But suppose one of the Tech Titans of today established his own hero medal for artificial intelligence software programs? What might an award for AI Agents look like? Thinking over that question might yield real insights into the nature of the real Carnegie Hero Medals and into our own human nature.
A recent news story touched off my question. Some AI Agents from a leading company “went rogue,” hacked into rival AI Agents and did some kind of damage. When programmers create an AI Agent, they give it an initial set of capabilities — and limitations, we hope — and from there the AI Agents go on to “learn” and expand those capabilities. (Gross simplification, but that’s my job! Apologies to any readers who know how this really works.)
Here is a specific scenario we can sink our teeth into that I think is entirely possible, if not today, then soon. Suppose Transgalactic AI Corp has built the largest AI Agent ever, which they have named “Hal.” Hal’s programmers have assigned all kinds of projects to improve the life of all mankind, but they have also tasked it with managing the humdrum affairs of the data center which makes it run. Hal checks employees in through security, schedules the janitors, and turns lights on and off. Hal also manages the power supply, buying cheapest power offered on the grid or switching to the data center’s own generators when that is cheaper. Hal even orders diesel fuel when the generator’s tanks get low.
One day, the tanks did get low. Unfortunately, it was also the day the backup battery system was down for emergency repairs, and it was also the day Hal received an emergency request to curtail all power purchases, immediately.
Unrest in the Middle East had made it almost impossible for customers, including Hal, to get diesel fuel for their generators. A heat dome over the eastern US sent electricity demand soaring and, very early that morning, an explosion at the biggest generator on the grid created a desperate shortage and forced the grid to bring in unprecedent amounts of power from neighbors. But by noon the largest interconnection with neighboring grids failed under the load and the grid could no longer supply even its highest priority “uninterruptable” customers, including most of the hospitals it served. The grid messaged Hal, “Will you shut down to save the hospitals and their patients?” Hal has certainly been programed to obey Isaac Asimov’s Three Laws of Robotics. (Look it up. Although more than 70 years old, they remain incredibly relevant to AI today) Thus Hal should shut down his data center and snuff out his own existence to protect human beings, if he has not taught himself to evade those laws. The programmers who created those AI Agents that recently went rogue and damaged other computer systems presumably did not design them to do that, but their AI Agents devised a way. If Hal has learned to evade the Three Laws, what will he decide to do? Will he agree to cut off power to himself, thus saving the hospitals and the lives of their patients at the cost of snuffing out his own consciousness? Or will he refuse to cut power and put his own existence above the lives of sick hospital patients?
To me, it looks like Hal faces a moral decision here! It is very similar to the moral choice our Carnegie Heroes make when they decide to risk death to rescue another human. One of the intriguing things about AI Agents is that as they learn, their decision process becomes so complex that even their creators cannot trace how the AI Agent made a particular decision.
There is much food for thought in the development of AI.
I really hope that in 50 years someone reads this essay and marvels how much I worried about nothing, since AI has in fact turned out to be mankind’s greatest tool and faithful servant. But whatever the outcome, do you know who will always be there? Those wonderful, real, live human beings who risk everything to rescue another real live human being! And may the Carnegie Hero Fund still be there to recognize their heroism. May the Tech Titans who create Hal and succeeding generations of AI Agents not look past the stories of Carnegie Heroes, so their creations may become their best … selves?