What LLMs Still Can't Do

Large Language Models do not possess common sense in the way that people do. These models generate text based on patterns they have learned from large amounts of text data, but they do not have true understanding, consciousness, or the ability to reason in the same way that people do.

LLMs lack genuine world knowledge and can generate incorrect or nonsensical information, especially if the input data they are provided with is incomplete or ambiguous. They do not have the ability to draw on personal experiences or make judgments based on human common sense.

While LLMs can generate text that appears to be coherent and contextually relevant, up to today they are essentially just sophisticated text generators that operate based on statistical patterns in the training data. They do not have a deep understanding of the concepts and common sense that humans have developed through experience and learning.

What is common sense?

Common sense is a broad term that encompasses a wide range of knowledge and abilities, including:

• Understanding the physical world and how it works
• Understanding human relationships and social norms
• Reasoning about cause and effect
• Making inferences based on incomplete information
• Applying knowledge to new situations

Common sense is a very powerful mechanism that helps us understand the world and make decisions. Beware though that powerful, in this context, does not imply error-free. It was once common sense that the earth was flat. The power of common sense comes from how much it affects us daily.

Hubert L. Dreyfus’ critique on AI

This article (including its title) is based on the book “What Computers Still Can't Do” by Hubert L. Dreyfus. In this book, published three decades ago, Dreyfus argued that AI models, particularly those developed up until the time of his writing, were not capable of having common sense for several reasons.

Here are some reasons that I can recall:

Lack of embodied experience

Dreyfus emphasized the importance of human embodiment and sensory experience in developing common sense. Human beings acquire common sense through their interaction with the physical world, and computers and AI models lack this kind of embodied experience. They do not perceive the world as humans do through senses like vision, touch, and hearing, which limits their ability to understand the context and nuances of common sense situations.

Contextual understanding

Common sense often relies on an understanding of context, which is challenging for AI models to grasp fully. Dreyfus argued that AI models of his time struggled with context-dependent reasoning, and they couldn't effectively apply knowledge across different situations.

The symbol-grounding problem

Dreyfus discussed the "symbol-grounding problem," which is the challenge of connecting symbols or representations in a computer to real-world objects and concepts. He believed that AI models of his era relied heavily on symbolic processing but lacked the ability to ground those symbols in the real world, hindering their common sense capabilities. In essence, Hubert Dreyfus argued that this "symbolic information-processing" (SIP) model that AI models were based on was fundamentally flawed. This model assumes that the mind can be represented as a system of symbols that are manipulated according to precise rules. This works well for solving puzzles or playing chess but not when it comes to understanding how humans think and reason in the real world.

Lack of human intuition

Dreyfus argued that common sense often involves intuitive judgments and reasoning that are difficult to formalize in algorithms. AI models at the time did not possess the kind of intuitive understanding that humans have, making them ill-equipped to handle many common sense tasks. Humans have a great deal of tacit knowledge, which is knowledge that we cannot explicitly articulate. This tacit knowledge is essential for common sense reasoning.

Wrapping Up

It's important to note that today’s LLMs are at an early stage of development. They represent a different paradigm of AI compared to the AI systems that Dreyfus critiqued. LLMs rely on large-scale machine learning and neural networks. Dreyfus primarily addressed AI limitations as of the early 1990s where these approaches relied heavily on symbolic processing, expert systems, and rule-based reasoning. His critique may not directly apply to modern LLMs but many of his arguments still resonate with discussions about AI today. AI has made significant advances since then, but the challenges of imbuing machines with true common sense remain a fundamental problem in the field. Researchers continue to work on developing AI systems that can handle more complex and context-rich common-sense reasoning, but it remains an ongoing challenge.