All you need to know about symbolic artificial intelligence

Today, artificial intelligence is mostly about artificial neural networks and deep learning. But this is not how it always was. In fact, for most of its six-decade history, the field was dominated by symbolic artificial intelligence, also known as “classical AI,” “rule-based AI,” and “good old-fashioned AI.”

Symbolic AI involves the explicit embedding of human knowledge and behavior rules into computer programs. The practice showed a lot of promise in the early decades of AI research. But in recent years, as neural networks, also known as connectionist AI, gained traction, symbolic AI has fallen by the wayside.

The role of symbols in artificial intelligence

Symbols are things we use to represent other things. Symbols play a vital role in the human thought and reasoning process. If I tell you that I saw a cat up in a tree, your mind will quickly conjure an image.

We use symbols all the time to define things (cat, car, airplane, etc.) and people (teacher, police, salesperson). Symbols can represent abstract concepts (bank transaction) or things that don’t physically exist (web page, blog post, etc.). They can also describe actions (running) or states (inactive). Symbols can be organized into hierarchies (a car is made of doors, windows, tires, seats, etc.). They can also be used to describe other symbols (a cat with fluffy ears, a red carpet, etc.).

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!

Being able to communicate in symbols is one of the main things that make us intelligent. Therefore, symbols have also played a crucial role in the creation of artificial intelligence.

The early pioneers of AI believed that “every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.” Therefore, symbolic AI took center stage and became the focus of research projects. Scientists developed tools to define and manipulate symbols.

Many of the concepts and tools you find in computer science are the results of these efforts. Symbolic AI programs are based on creating explicit structures and behavior rules.

An example of symbolic AI tools is object-oriented programming. OOP languages allow you to define classes, specify their properties, and organize them in hierarchies. You can create instances of these classes (called objects) and manipulate their properties. Class instances can also perform actions, also known as functions, methods, or procedures. Each method executes a series of rule-based instructions that might read and change the properties of the current and other objects.

Using OOP, you can create extensive and complex symbolic AI programs that perform various tasks.

The benefits and limits of symbolic AI

Symbolic artificial intelligence showed early progress at the dawn of AI and computing. You can easily visualize the logic of rule-based programs, communicate them, and troubleshoot them.

Symbolic artificial intelligence is very convenient for settings where the rules are very clear cut,  and you can easily obtain input and transform it into symbols. In fact, rule-based systems still account for most computer programs today, including those used to create deep learning applications.

But symbolic AI starts to break when you must deal with the messiness of the world. For instance, consider computer vision, the science of enabling computers to make sense of the content of images and video. Say you have a picture of your cat and want to create a program that can detect images that contain your cat. You create a rule-based program that takes new images as inputs, compares the pixels to the original cat image, and responds by saying whether your cat is in those images.

This will only work as you provide an exact copy of the original image to your program. A slightly different picture of your cat will yield a negative answer. For instance, if you take a picture of your cat from a somewhat different angle, the program will fail.

One solution is to take pictures of your cat from different angles and create new rules for your application to compare each input against all those images. Even if you take a million pictures of your cat, you still won’t account for every possible case. A change in the lighting conditions or the background of the image will change the pixel value and cause the program to fail. You’ll need millions of other pictures and rules for those.

And what if you wanted to create a program that could detect any cat? How many rules would you need to create for that?

The cat example might sound silly, but these are the kinds of problems that symbolic AI programs have always struggled with. You can’t define rules for the messy data that exists in the real world. For instance, how can you define the rules for a self-driving car to detect all the different pedestrians it might face?

Also, some tasks can’t be translated to direct rules, including speech recognition and natural language processing.

There have been several efforts to create complicated symbolic AI systems that encompass the multitudes of rules of certain domains. Called expert systems, these symbolic AI models use hardcoded knowledge and rules to tackle complicated tasks such as medical diagnosis. But they require a huge amount of effort by domain experts and software engineers and only work in very narrow use cases. As soon as you generalize the problem, there will be an explosion of new rules to add (remember the cat detection problem?), which will require more human labor. As some AI scientists point out, symbolic AI systems don’t scale.

Neural networks vs symbolic AI

Neural networks are almost as old as symbolic AI, but they were largely dismissed because they were inefficient and required compute resources that weren’t available at the time. In the past decade, thanks to the large availability of data and processing power, deep learning has gained popularity and has pushed past symbolic AI systems.

The advantage of neural networks is that they can deal with messy and unstructured data. Take the cat detector example. Instead of manually laboring through the rules of detecting cat pixels, you can train a deep learning algorithm on many pictures of cats. The neural network then develops a statistical model for cat images. When you provide it with a new image, it will return the probability that it contains a cat.

Deep learning and neural networks excel at exactly the tasks that symbolic AI struggles with. They have created a revolution in computer vision applications such as facial recognition and cancer detection. Deep learning has also driven advances in language-related tasks.

Deep neural networks are also very suitable for reinforcement learning, AI models that develop their behavior through numerous trial and error. This is the kind of AI that masters complicated games such as Go, StarCraft, and Dota.

But the benefits of deep learning and neural networks are not without tradeoffs. Deep learning has several deep challenges and disadvantages in comparison to symbolic AI. Notably, deep learning algorithms are opaque, and figuring out how they work perplexes even their creators. And it’s very hard to communicate and troubleshoot their inner-workings.

Neural networks are also very data-hungry. And unlike symbolic AI, neural networks have no notion of symbols and hierarchical representation of knowledge. This limitation makes it very hard to apply neural networks to tasks that require logic and reasoning, such as science and high-school math.

The current state of symbolic AI

Some believe that symbolic AI is dead. But this assumption couldn’t be farther from the truth. In fact, rule-based AI systems are still very important in today’s applications. Many leading scientists believe that symbolic reasoning will continue to remain a very important component of artificial intelligence.

There are now several efforts to combine neural networks and symbolic AI. One such project is the Neuro-Symbolic Concept Learner (NSCL), a hybrid AI system developed by the MIT-IBM Watson AI Lab. NSCL uses both rule-based programs and neural networks to solve visual question-answering problems. As opposed to pure neural network–based models, the hybrid AI can learn new tasks with less data and is explainable. And unlike symbolic-only models, NSCL doesn’t struggle to analyze the content of images.

Maybe in the future, we’ll invent AI technologies that can both reason and learn. But for the moment, symbolic AI is the leading method to deal with problems that require logical thinking and knowledge representation.

This article was originally published by Ben Dickson on TechTalks, a publication that examines trends in technology, how they affect the way we live and do business, and the problems they solve. But we also discuss the evil side of technology, the darker implications of new tech and what we need to look out for. You can read the original article here.