Many pessimistic scenarios about machines taking control of the world and harming humans are based on the idea that computers will eventually develop self-consciousness and define their own goals, incompatible with the goals of humanity. This is the basis of the argument of many science-fiction movies and books.
Many people believe, however, that this will not be the main problem. As reported in many news outlets, the University of California at Berkeley (my alma matter) has launched the Center for Human-Compatible Artificial Intelligence. The center will be headed by Stuart Russell, a famous expert in Artificial Intelligence (and co-author, with Peter Norvig, of the most used textbook in the field, Artificial Intelligence: A Modern Approach). Russell has been a vocal advocate for incorporating human values into the design of AI, in order to avoid the pitfall that may come from AI systems running amok.
According to Stuart Russell, the issue is “that machines as we currently design them in fields like AI, robotics, control theory and operations research take the objectives that we humans give them very literally“. Therefore, they may approach tasks with an objective that is simply too literal. For instance, if instructed to solve the problem of “global warming”, a machine may decide that the most effective way is to wipe out the human race.
According to the UC Berkeley press release, the center is being launched with a grant of $5.5 million from the Open Philanthropy Project, with additional grants from the Leverhulme Trust and the Future of Life Institute.
The center will work on mechanisms to guarantee that the AI systems of the future will act, by design, in a way that is aligned with human values. According to Stuart Russell, “AI systems must remain under human control, with suitable constraints on behavior, despite capabilities that may eventually exceed our own. This means we need cast-iron formal proofs, not just good intentions.”
Image credits: UC Berkeley. The image illustrates BRETT, the Berkeley Robot for the Elimination of Tedious Tasks, tieing a knot after watching others demonstrate it.