Authors:
(1) Raphaël Millière, Department of Philosophy, Macquarie University (raphael.milliere@mq.edu.eu);
(2) Cameron Buckner, Department of Philosophy, University of Houston (cjbuckner@uh.edu). Table of Links Abstract and 1 Introduction 2. A primer on LLMs 2.1. Historical foundations 2.2. Transformer-based LLMs 3. Interface with classic philosophical issues 3.1. Compositionality 3.2. Nativism and language acquisition 3.3. Language understanding and grounding 3.4. World models 3.5. Transmission of cultural knowledge and linguistic scaffolding 4. Conclusion, Glossary, and References 3.5. Transmission of cultural knowledge and linguistic scaffolding Another interesting question is whether LLMs might engage in cultural acquisition and play a role in the transmission of knowledge. Prominent theorists have suggested that the key to human intelligence lies in a unique set of predispositions for cultural learning (Tomasello 2009). While other primates may share some of these dispositions, these theorists argue that humans are uniquely equipped to cooperate with one another to acquire and transmit knowledge from one generation to the next. Tomasello has explained the uniquely human capacity for cultural learning in terms of a “ratchet effect,” a metaphor to the ratcheting wrench which clicks into place to hold its position each time it is further turned in the desired direction. Chimpanzees and other animals, Tomasello argues, can learn in many of the same ways that humans do, and even acquire regional differences in their problem-solving strategies, such as different troops using different tool-making techniques to fish for termites. However, he claims that only humans can pick up right where the previous generation left off and continue making new progress on linguistic, scientific, and sociological knowledge. This constant ratcheting is what allows a steady progression of human knowledge accumulation and discovery, compared to the relatively stagnant cultural evolution of chimpanzees and other animals. Given that deep learning systems already exceed human performance in several task domains, it is interesting to ask whether LLMs might be able to emulate many of these components of cultural learning to pass on their discoveries to human theoreticians. For instance, humans are already reverse-engineering the strategies of AlphaZero to produce mini-revolutions in the explicit theory of Go and chess (Schut et al. 2023). Similarly, latent knowledge in specialized domains such as materials science can be extracted even from a simple word embedding model Tshitoyan et al. (2019). In these instances, it is primarily humans who are synthesizing and passing on culturally-transmissible knowledge by interpreting the model’s outputs and internal activations. This human-led interpretation and transmission underscore a crucial aspect of cultural ratcheting: the ability to not only generate novel solutions but to also understand and communicate the underlying principles of these solutions, thereby enabling cumulative knowledge growth. Could LLMs ever explain their strategies to humans in a theoretically-mediated way that participates in and enhances human cultural learning? This question is directly related to whether LLMs can genuinely generalize to out-of-distribution (OOD) data. As discussed in section 3.1, there is converging evidence that Transformer-based models may generalize compositionally under some train-test distribution shifts.[14] But the present issue intersects with a different kind of generalization – the ability to solve genuinely novel tasks. To borrow from Chollet (2019)’s taxonomy, we can distinguish between local task generalization, which involves handling new data within a familiar distribution for known range of tasks; broad task generalization, which involves handling new data under modest distribution shift for a wide range of tasks and environments; and extreme task generalization, which required handling new data for entirely novel tasks that represent a significant departure from any previous data distributions. Current LLMs seem able to master a wide variety of tasks that are reflected in their current training sets; as such, they exhibit at least local task generalization, if not broad task generalization. However, like chimpanzees that learn from observing their troop mates, they often seem to have a hard time pushing beyond the range of tasks well-represented in their training data McCoy et al. (2023). Furthermore, the ratcheting effect crucially involves stable cultural transmission in addition to innovation. Can LLMs, like humans, not only generate novel solutions but also “lock in” these innovations by recognizing and articulating how they have advanced beyond previous solutions? Such a capability would involve more than just the generation of novel responses; it necessitates an understanding of the novelty of the solution and its implications, akin to human scientists who not only discover but also theorize, contextualize, and communicate their findings. The challenge for LLMs, therefore, lies not merely in generating novel solutions to problems but also in developing an ability to reflect on and communicate the nature of their innovations in a manner that contributes to the cumulative process of cultural learning. This ability would likely require some of the more advanced communicative intentions and world models (such as causal models) discussed in previous sections. While LLMs show promise in various forms of task generalization, their participation in the ratcheting process of cultural learning thus appears contingent on further advancements in these areas, which might lie beyond the reach of current architectures. This paper is available on arxiv under CC BY 4.0 DEED license. [14] For a systematic discussion of different aspects of generalization research in NLP, including different types of distribution shift, see Hupkes et al. (2023). Authors: (1) Raphaël Millière, Department of Philosophy, Macquarie University (raphael.milliere@mq.edu.eu); (2) Cameron Buckner, Department of Philosophy, University of Houston (cjbuckner@uh.edu). Authors: Authors: (1) Raphaël Millière, Department of Philosophy, Macquarie University (raphael.milliere@mq.edu.eu); (2) Cameron Buckner, Department of Philosophy, University of Houston (cjbuckner@uh.edu). Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2. A primer on LLMs 2.1. Historical foundations 2.1. Historical foundations 2.2. Transformer-based LLMs 2.2. Transformer-based LLMs 3. Interface with classic philosophical issues 3. Interface with classic philosophical issues 3.1. Compositionality 3.1. Compositionality 3.2. Nativism and language acquisition 3.2. Nativism and language acquisition 3.3. Language understanding and grounding 3.3. Language understanding and grounding 3.4. World models 3.4. World models 3.5. Transmission of cultural knowledge and linguistic scaffolding 3.5. Transmission of cultural knowledge and linguistic scaffolding 4. Conclusion, Glossary, and References 4. Conclusion, Glossary, and References 3.5. Transmission of cultural knowledge and linguistic scaffolding Another interesting question is whether LLMs might engage in cultural acquisition and play a role in the transmission of knowledge. Prominent theorists have suggested that the key to human intelligence lies in a unique set of predispositions for cultural learning (Tomasello 2009). While other primates may share some of these dispositions, these theorists argue that humans are uniquely equipped to cooperate with one another to acquire and transmit knowledge from one generation to the next. Tomasello has explained the uniquely human capacity for cultural learning in terms of a “ratchet effect,” a metaphor to the ratcheting wrench which clicks into place to hold its position each time it is further turned in the desired direction. Chimpanzees and other animals, Tomasello argues, can learn in many of the same ways that humans do, and even acquire regional differences in their problem-solving strategies, such as different troops using different tool-making techniques to fish for termites. However, he claims that only humans can pick up right where the previous generation left off and continue making new progress on linguistic, scientific, and sociological knowledge. This constant ratcheting is what allows a steady progression of human knowledge accumulation and discovery, compared to the relatively stagnant cultural evolution of chimpanzees and other animals. Given that deep learning systems already exceed human performance in several task domains, it is interesting to ask whether LLMs might be able to emulate many of these components of cultural learning to pass on their discoveries to human theoreticians. For instance, humans are already reverse-engineering the strategies of AlphaZero to produce mini-revolutions in the explicit theory of Go and chess (Schut et al. 2023). Similarly, latent knowledge in specialized domains such as materials science can be extracted even from a simple word embedding model Tshitoyan et al. (2019). In these instances, it is primarily humans who are synthesizing and passing on culturally-transmissible knowledge by interpreting the model’s outputs and internal activations. This human-led interpretation and transmission underscore a crucial aspect of cultural ratcheting: the ability to not only generate novel solutions but to also understand and communicate the underlying principles of these solutions, thereby enabling cumulative knowledge growth. Could LLMs ever explain their strategies to humans in a theoretically-mediated way that participates in and enhances human cultural learning? This question is directly related to whether LLMs can genuinely generalize to out-of-distribution (OOD) data. As discussed in section 3.1, there is converging evidence that Transformer-based models may generalize compositionally under some train-test distribution shifts.[14] But the present issue intersects with a different kind of generalization – the ability to solve genuinely novel tasks. To borrow from Chollet (2019)’s taxonomy, we can distinguish between local task generalization, which involves handling new data within a familiar distribution for known range of tasks; broad task generalization, which involves handling new data under modest distribution shift for a wide range of tasks and environments; and extreme task generalization, which required handling new data for entirely novel tasks that represent a significant departure from any previous data distributions. Current LLMs seem able to master a wide variety of tasks that are reflected in their current training sets; as such, they exhibit at least local task generalization, if not broad task generalization. However, like chimpanzees that learn from observing their troop mates, they often seem to have a hard time pushing beyond the range of tasks well-represented in their training data McCoy et al. (2023). Furthermore, the ratcheting effect crucially involves stable cultural transmission in addition to innovation. Can LLMs, like humans, not only generate novel solutions but also “lock in” these innovations by recognizing and articulating how they have advanced beyond previous solutions? Such a capability would involve more than just the generation of novel responses; it necessitates an understanding of the novelty of the solution and its implications, akin to human scientists who not only discover but also theorize, contextualize, and communicate their findings. The challenge for LLMs, therefore, lies not merely in generating novel solutions to problems but also in developing an ability to reflect on and communicate the nature of their innovations in a manner that contributes to the cumulative process of cultural learning. This ability would likely require some of the more advanced communicative intentions and world models (such as causal models) discussed in previous sections. While LLMs show promise in various forms of task generalization, their participation in the ratcheting process of cultural learning thus appears contingent on further advancements in these areas, which might lie beyond the reach of current architectures. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv [14] For a systematic discussion of different aspects of generalization research in NLP, including different types of distribution shift, see Hupkes et al. (2023).

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Abstract

effect

reflect

Language Models and the Illusion of Understanding

Do Large Language Models Have an Internal Understanding of the World?

I Think, Therefore I am Philosophical (dot tech)

Too Long; Didn't Read

Make resilience your competitive advantage

Transmission of cultural knowledge and linguistic scaffolding

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Language Models and the Illusion of Understanding

Philosophers Challenge the Idea That AI Lacks True Understanding

The Shift from Symbolic AI to Deep Learning in Natural Language Processing

How LLMs Learn from Context Without Traditional Memory

How Philosophers and Scientists View Cognitive AI

Language Models and the Illusion of Understanding

Philosophers Challenge the Idea That AI Lacks True Understanding

The Shift from Symbolic AI to Deep Learning in Natural Language Processing

How LLMs Learn from Context Without Traditional Memory

How Philosophers and Scientists View Cognitive AI

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps