Can AI develop a sense of right and wrong?

Can artificial intelligence learn the moral values of human societies? Can an AI system make decisions in situations where it must weigh and balance between damage and benefits to different people or groups of people? Can AI develop a sense of right and wrong? In short, will artificial intelligence have a conscience?

This question might sound irrelevant when considering today’s AI systems, which are only capable of accomplishing very narrow tasks. But as science continues to break new grounds, artificial intelligence is gradually finding its way into broader domains. We’re already seeing AI algorithms applied to areas where the boundaries of good and bad decisions are not clearly defined, such as criminal justice and job application processing.

In the future, we expect AI to care for the elderly, teach our children, and perform many other tasks that require moral human judgement. And then, the question of conscience and conscientiousness in AI will become even more critical.

With these questions in mind, I went in search of a book (or books) that explained how humans develop conscience and give an idea of whether what we know about the brain provides a roadmap for conscientious AI.

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!

A friend suggested Conscience: The Origins of Moral Intuitionby Dr. Patricia Churchland, neuroscientist, philosopher, and professor emerita at the University of California, San Diego. Dr. Churchland’s book, and a conversation I had with her after reading Conscience, taught me a lot about the extent and limits of brain science. Conscience shows us how far we’ve come to understand the relation between the brain’s physical structure and workings and the moral sense in humans. But it also shows us how much more we must go to truly understand how humans make moral decisions.

It is a very accessible read for anyone who is interested in exploring the biological background of human conscience and reflect on the intersection of AI and conscience.

Here’s a very quick rundown of what Conscience tells us about the development of moral intuition in the human brain. With the mind being the main blueprint for AI, better knowledge of conscience can tell us a lot about what it would take for AI to learn the moral norms of human societies.

The learning system

“Conscience is an individual’s judgment about what is normally right or wrong, typically, but not always, reflecting some standard of a group to which the individual feels attached,” Churchland writes in her book.

But how did humans develop the ability to understand to adopt these rights and wrongs? To answer that question, Dr. Churchland takes us back through time, when our first warm-blooded ancestors made their apparition.

Birds and mammals are endotherms: their bodies have mechanisms to preserve their heat. In contrast, in reptiles, fish, and insects, cold-blooded organisms, the body adapts to the temperature of the environment.

The great benefit of endothermy is the capability to gather food at night and to survive colder climates. The tradeoff: endothermic bodies need a lot more food to survive. This requirement led to a series of evolutionary steps in the brains of warm-blooded creatures that made them smarter. Most notable among them is the development of the cortex in the mammalian brain.

The cortex can integrate diverse signals and pull out abstract representation of events and things that are relevant to survival and reproduction. The cortex learns, integrates, revises, recalls, and keeps on learning.

The cortex allows mammals to be much more flexible to changes in weather and landscape, as opposed to insects and fish, who are very dependent on stability in their environmental conditions.

But again, learning capabilities come with a tradeoff: mammals are born helpless and vulnerable. Unlike snakes, turtles, and insects, which hit the ground running and are fully functional when they break their eggshells, mammals need time to learn and develop their survival skills.

And this is why they depend on each other for survival.

The development of social behavior

The brains of all living beings have a reward and punishment system that makes sure they do things that support their survival and the survival of their genes. The brains of mammals repurposed this function to adapt for sociality.

“In the evolution of the mammalian brain, feelings of pleasure and pain supporting self-survival were supplemented and repurposed to motivate affiliative behavior,” Churchland writes. “Self-love extended into a related but new sphere: other-love.”

The main beneficiary of this change are the offspring. Evolution has triggered changes in the circuitry of the brains of mammals to reward care for babies. Mothers, and in some species both parents, go to great lengths to protect and feed their offspring, often at a great disadvantage to themselves.

In Conscience, Churchland describes experiments on the biochemical reactions of the brains of different mammals reward social behavior, including care for offspring.

“Mammalian sociality is qualitatively different from that seen in other social animals that lack a cortex, such as bees, termites, and fish,” Churchland writes. “It is more flexible, less reflexive, and more sensitive to contingencies in the environment and thus sensitive to evidence. It is sensitive to long-term as well as short-term considerations. The social brain of mammals enables them to navigate the social world, for knowing what others intend or expect.”

Human social behavior

The brains of humans have the largest and most complex cortex in mammals. The brain of homo sapiens, our species, is three times as large as that of chimpanzees, with whom we shared a common ancestor 5-8 million years ago.

The larger brain naturally makes us much smarter but also has higher energy requirements. So how did we come to pay the calorie bill? “Learning to cook food over fire was quite likely the crucial behavioral change that allowed hominin brains to expand well beyond chimpanzee brains, and to expand rather quickly in evolutionary time,” Churchland writes.

With the body’s energy needs supplied, hominins eventually became able to do more complex things, including the development of richer social behaviors and structures.

So the complex behavior we see in our species today, including the adherence to moral norms and rules, started off as a struggle for survival and the need to meet energy constraints.

“Energy constrains might not be stylish and philosophical, but they are as real as rain,” Churchland writes in Conscience.

Our genetic evolution favored social behavior. Moral norms emerged as practical solutions to our needs. And we humans, like every other living being, are subject to the laws of evolution, which Churchland describes as “a blind process that, without any goal, fiddles around with the structure already in place.” The structure of our brain is the result of countless experiments and adjustments.

“Between them, the circuitry supporting sociality and self-care, and the circuitry for internalizing social norms, create what we call conscience,” Churchland writes. “In this sense your conscience is a brain construct, whereby your instincts for caring, for self and others, are channeled into specific behaviors through development, imitation, and learning.”

This is a very sensitive topic and complicated, and despite all the advances in brain science, many of the mysteries of the human mind and behavior remain unlocked.

“The dominant role of energy requirements in the ancient origin of human morality does not mean that decency and honesty must be cheapened. Nor does it mean that they are not real. These virtues remain entirely admirable and worthy to us social humans, regardless of their humble origins. They are an essential part of what makes us the humans we are,” Churchland writes.

Artificial intelligence and conscience

In Conscience, Churchland discusses many other topics, including the role of reinforcement learning in the development of social behavior and the human cortex’s far-reaching capacity to learn by experience, to reflect on counterfactual situations, develop models of the world, draw analogies from similar patterns and much more.

Basically, we use the same reward system that allowed our ancestors to survive, and draw on the complexity of our layered cortex to make very complicated decisions in social settings.

“Moral norms emerge in the context of social tension, and they are anchored by the biological substrate. Learning social practices relies on the brain’s system of positive and negative reward, but also on the brain’s capacity for problem solving,” Churchland writes.

After reading Conscience, I had many questions in mind about the role of conscience in AI. Would conscience be an inevitable byproduct of human-level AI? If energy and physical constraints pushed us to develop social norms and conscientious behavior, would there be a similar requirement for AI? Does physical experience and sensory input from the world play a crucial role in the development of intelligence?

Fortunately, I had the chance to discuss these topics with Dr. Churchland after reading Conscience.

Is physical experience a requirement for the development of conscience in AI?

What is evident from Dr. Churchland’s book (and other research on biological neural networks), physical experience and constraints play an important role in the development of intelligence, and by extension conscience, in humans and animals.

But today, when we speak of artificial intelligence, we mostly talk about software architectures such as artificial neural networks. Today’s AI is mostly disembodied lines of code that run on computers and servers and process data obtained by other means. Will physical experience and constraints be a requirement for the development of truly intelligent AI that can also appreciate and adhere to the moral rules and norms of human society?

“It’s hard to know how flexible behavior can be when the anatomy of the machine is very different from the anatomy of the brain,” Dr. Churchland said in our conversation. “In the case of biological systems, the reward system, the system for reinforcement learning is absolutely crucial. Feelings of positive and negative reward are essential for organisms to learn about the environment. That may not be true in the case of artificial neural networks. We just don’t know.”

She also pointed out that we still don’t know how brains think. “In the event that we were to understand that, we might not need to replicate absolutely every feature of the biological brain in the artificial brain in order to get some of the same behavior,” she added.

Churchland reminded that while initially, the AI community largely dismissed neural networks, they eventually turned out to be quite effective when their computational requirements were met. And while current neural networks have limited intelligence in comparison to the human brain, we might be in for surprises in the future.

“One of the things we do know at this stage is that mammals with cortex and with reward system and subcortical structures can learn things and generalize without a huge amount of data,” she said. “At the moment, an artificial neural network might be very good at classifying faces by hopeless at classifying mammals. That could just be a numbers problem.

“If you’re an engineer and you’re trying to get some effect, try all kinds of things. Maybe you do have to have something like emotions and maybe you can build that into your artificial neural network.”

Do we need to replicate the subtle physical differences of the brain in AI?

One of my takeaways from Conscience was that humans generally align themselves with the social norms of their society, they also challenge them at times. And the unique physical structure of each human brain, the genes we inherit from our parents and the later experiences that we acquire through our lives make for the subtle differences that allow us to come up with new norms and ideas and sometimes defy what was previously established as rule and law.

But one of the much-touted features of AI is its uniform reproducibility. When you create an AI algorithm, you can replicate it countless times and deploy it in as many devices and machines as you want. They will all be identical to the last parametric values of their neural networks. Now, the question is, when all AIs are equal, will they remain static in their social behavior and lack the subtle differences that drive the dynamics of social and behavioral progress in human societies?

“Until we have a much richer understanding of how biological brains work, it’s really hard to answer that question,” Churchland said. “We know that in order to get a complicated result out of a neural network, the network doesn’t have to have wet stuff, it doesn’t have to have mitochondria and ribosomes and proteins and membranes. How much else does it not have to have? We don’t know.

“Without data, you’re just another person with an opinion, and I have no data that would tell me that you’ve got to mimic certain specific circuitry in the reinforcement learning system in order to have an intelligent network.

“Engineers will try and see what works.”

We have yet to learn much about human conscience, and even more about if and how it would apply to highly intelligent machines. “We do not know precisely what the brain does as it learns to balance in a headstand. But over time, we get the hang of it,” Churchland writes in Conscience. “To an even greater degree, we do not know what the brain does as it learns to find balance in a socially complicated world.”

But as we continue to observe and learn the secrets of the brain, hopefully we will be better equipped to create AI that serves the good of all humanity.

This article was originally published by Ben Dickson on TechTalks, a publication that examines trends in technology, how they affect the way we live and do business, and the problems they solve. But we also discuss the evil side of technology, the darker implications of new tech and what we need to look out for. You can read the original article here.