For a long time, IT specialists worked without a care in the world. They developed, built, and deployed software smoothly. Then the era of isolation hit, and suddenly, they got bored (of course, this is a playful take on the actual events). IT folks wanted to create something that could handle their work while they stayed home: answer routine questions, generate cool avatars, and analyze vast amounts of data in minutes. They dreamed of traveling to a fantastic place, and so, you guessed it, they revolutionized AI. AI is now functioning, providing answers, and improving lives. As skilled an assistant as it is, AI is truly effective only when used in the right context. We’re witnessing rapid progress in AI applications, from image and video generation to stock market forecasting and cryptocurrency analysis. Yet, AI may offer information we don’t ask for or provide blatantly false answers. Its behavior is very much like that of household cats — you know, the kind that sits quietly and then suddenly pounces on you? Our cats, as well as AI, enjoy being unpredictable: You give them the same food (or data) — sometimes they eat, sometimes they ignore it.
You train them to respond, but they only occasionally react when you call them.
The bigger and wilder the cat or the larger the AI model, the harder it is to predict its behavior.
In the morning, cats might be calm; by evening, they turn hyperactive (just like dynamic data).
Cats might be friendly (deterministic) but can scratch you without warning (stochastic). You might wonder what determinism and stochasticity mean — let’s find out. Determinism and Stochasticity A deterministic system always produces the same result given the same input — think idempotency if you're a DevOps engineer. A real-world example would be your cat that eats the same amount of food you put in its bowl every time — this is determinism. But when the cat sniffs and only eats half, it’s no longer deterministic. A stochastic process includes an element of randomness: with the same input, the result can vary. For example, a machine learning model often uses stochastic algorithms, like Stochastic Gradient Descent (SGD), which trains the model by picking random chunks of data rather than the entire dataset. These definitions don’t fully explain why our AIs sometimes hallucinate and behave chaotically. There are other contributing factors, including the following: Determinism
Stochasticity
Rounding errors and floating-point arithmetic
Multithreading and parallel computations
Continuously updating data
Chaos and the “butterfly effect” If we look a little closer, we'll see other mechanisms that influence the unpredictable behavior of AI models. A Glimpse of Neural Networks You probably know that the AIs everyone uses rely on various neural network algorithms. Here are some types of neural networks: Fully Connected Neural Networks (FCNN): A classic architecture where each neuron connects to every neuron in the next layer. Convolutional Neural Networks (CNNs): These networks use convolutions or filters that highlight image features like edges, textures, and shapes. Recurrent Neural Networks (RNNs): These networks have feedback loops that allow them to remember previous steps (namely, they remember sequences). Long Short-Term Memory (LSTM): An enhanced version of RNNs with mechanisms for selectively forgetting and remembering important data. Transformers: The most powerful class for text processing. They use multi-head attention, allowing them to consider the entire context simultaneously. Generative Adversarial Networks (GANs): They consist of two networks, one of which generates data and the other evaluates its quality. Their competition leads to better results. Autoencoders: Networks designed to compress (encode) information and then reconstruct (decode) it. Graph Neural Networks (GNNs): They work with graphs (nodes and edges) rather than regular data. We need all that context to understand why the most common model, ChatGPT, often hallucinates. How AI Hallucinations Happen ChatGPT runs on the Transformer architecture, first introduced in the 2017 paper, “Attention Is All You Need.”  This is the very mechanism that revolutionized text processing. Transformers operate on the self-attention mechanism, which allows them to consider the global context rather than just the nearest words like older recurrent neural networks (LSTM and GRU) do. The model belongs to the GPT (Generative Pre-Trained Transformer) series, which means: Pre-trained: It was initially trained on enormous amounts of text (books, articles, websites, and code).
Generative: Its task is to generate text, not just classify or extract facts. ChatGPT’s answers result from a stochastic process rather than a rigid rule. It doesn’t memorize or reproduce texts but generates responses using a probabilistic model. Word Prediction as a Probabilistic Process When ChatGPT responds, it doesn’t choose the single correct word but computes a probability distribution. P(wi|w1, w2, ..., wi-1), where: “wi” — the next word in the sentence w1, w2, ..., wi-1 — the previous words P(wi|w1, ..., wi-1) — the probability that “wi” will be the next word For example, if you ask, “What day is it today?” ChatGPT might have different probabilities: “Monday” — P=0.7
“Wednesday” — P=0.2
“42” — P=0.0001 It will mostly often choose the word with the highest probability, but due to generation temperature (a parameter that controls randomness), it might sometimes choose a less likely option based on context. Context Influence and Information Forgetting ChatGPT works with a limited context window, meaning it only "remembers" the last NN tokens. For GPT-4, the context window is about 128k tokens (around 300 pages of text). If important information is outside this context, it may: Forget details (context clipping effect)
Make-up information (stochastic process) Yet, ChatGPT can often correct its answer after you ask if it's sure. However, ChatGPT can often correct its answer if you ask whether it’s sure. AI Sometimes Corrects Itself, But Why? When you ask ChatGPT, “Are you sure?” it reanalyzes its answer using a new context where doubt is present. This results in: Recalculating answer probabilities.
Choosing a more plausible option if one exists. This process can be explained by Bayesian probability. P(A|B) = P(B|A)P(A) / P(B), where: P(A|B) — the probability that answer A is correct, considering your follow-up question B.


P(B|A) — the probability that you would have asked if ChatGPT was initially right.


P(A) — the initial probability of ChatGPT's answer.


P(B) — the overall probability that you would ask. Too much information for you? Brain overheating? Imagine that AIs also get overwhelmed by large amounts of information. Errors Due to Overfitting and Noisy Data Massive amounts of text data flow into ChatGPT’s training, including noise or contradictory information, such as: Some sources say the Earth is round, while others claim it’s flat. AI can’t always determine which information is true when it appears with varying probabilities. These are examples of model hallucinations, which occur because ChatGPT’s weights are trained on probabilistic word associations rather than strict logic. The Bottom Line Here is what we can learn from this. ChatGPT hallucinates since it: Predicts probabilistically, not deterministically.


Has a limited memory (context window).


Recalculates probabilities when questioned.


Has training data that includes noise and contradictions. It’s that straightforward. Hope you didn’t get tired. If you did, that’s a good sign because it means you’re thinking critically, which is exactly what we should do when working with AI. For a long time, IT specialists worked without a care in the world. They developed, built, and deployed software smoothly. Then the era of isolation hit, and suddenly, they got bored (of course, this is a playful take on the actual events). IT folks wanted to create something that could handle their work while they stayed home: answer routine questions, generate cool avatars, and analyze vast amounts of data in minutes. They dreamed of traveling to a fantastic place, and so, you guessed it, they revolutionized AI. AI is now functioning, providing answers, and improving lives. As skilled an assistant as it is, AI is truly effective only when used in the right context. We’re witnessing rapid progress in AI applications, from image and video generation to stock market forecasting and cryptocurrency analysis. Yet, AI may offer information we don’t ask for or provide blatantly false answers. Its behavior is very much like that of household cats — you know, the kind that sits quietly and then suddenly pounces on you? AI may offer information we don’t ask for Our cats, as well as AI, enjoy being unpredictable: You give them the same food (or data) — sometimes they eat, sometimes they ignore it. You train them to respond, but they only occasionally react when you call them. The bigger and wilder the cat or the larger the AI model, the harder it is to predict its behavior. In the morning, cats might be calm; by evening, they turn hyperactive (just like dynamic data). Cats might be friendly (deterministic) but can scratch you without warning (stochastic). You give them the same food (or data) — sometimes they eat, sometimes they ignore it. You train them to respond, but they only occasionally react when you call them. The bigger and wilder the cat or the larger the AI model, the harder it is to predict its behavior. In the morning, cats might be calm; by evening, they turn hyperactive (just like dynamic data). Cats might be friendly (deterministic) but can scratch you without warning (stochastic). You might wonder what determinism and stochasticity mean — let’s find out. Determinism and Stochasticity Determinism and Stochasticity A deterministic system always produces the same result given the same input — think idempotency if you're a DevOps engineer . A real-world example would be your cat that eats the same amount of food you put in its bowl every time — this is determinism . But when the cat sniffs and only eats half, it’s no longer deterministic. idempotency DevOps engineer determinism A stochastic process includes an element of randomness: with the same input, the result can vary. For example, a machine learning model often uses stochastic algorithms, like Stochastic Gradient Descent (SGD) , which trains the model by picking random chunks of data rather than the entire dataset. stochastic Stochastic Gradient Descent (SGD) These definitions don’t fully explain why our AIs sometimes hallucinate and behave chaotically. There are other contributing factors, including the following: Determinism Stochasticity Rounding errors and floating-point arithmetic Multithreading and parallel computations Continuously updating data Chaos and the “butterfly effect” Determinism Stochasticity Rounding errors and floating-point arithmetic Multithreading and parallel computations Continuously updating data Chaos and the “butterfly effect” If we look a little closer, we'll see other mechanisms that influence the unpredictable behavior of AI models. A Glimpse of Neural Networks A Glimpse of Neural Networks You probably know that the AIs everyone uses rely on various neural network algorithms. Here are some types of neural networks: AIs everyone uses Fully Connected Neural Networks (FCNN): A classic architecture where each neuron connects to every neuron in the next layer. Fully Connected Neural Networks (FCNN): A classic architecture where each neuron connects to every neuron in the next layer. Fully Connected Neural Networks (FCNN): Convolutional Neural Networks (CNNs): These networks use convolutions or filters that highlight image features like edges, textures, and shapes. Convolutional Neural Networks (CNNs): These networks use convolutions or filters that highlight image features like edges, textures, and shapes. Convolutional Neural Networks (CNNs): Recurrent Neural Networks (RNNs): These networks have feedback loops that allow them to remember previous steps (namely, they remember sequences). Recurrent Neural Networks (RNNs ): These networks have feedback loops that allow them to remember previous steps (namely, they remember sequences). Recurrent Neural Networks (RNNs Long Short-Term Memory (LSTM): An enhanced version of RNNs with mechanisms for selectively forgetting and remembering important data. Long Short-Term Memory (LSTM): An enhanced version of RNNs with mechanisms for selectively forgetting and remembering important data. Long Short-Term Memory (LSTM): Transformers: The most powerful class for text processing. They use multi-head attention, allowing them to consider the entire context simultaneously. Transformers : The most powerful class for text processing. They use multi-head attention, allowing them to consider the entire context simultaneously. Transformers Generative Adversarial Networks (GANs): They consist of two networks, one of which generates data and the other evaluates its quality. Their competition leads to better results. Generative Adversarial Networks (GANs): They consist of two networks, one of which generates data and the other evaluates its quality. Their competition leads to better results. Generative Adversarial Networks (GANs): Autoencoders: Networks designed to compress (encode) information and then reconstruct (decode) it. Autoencoders : Networks designed to compress (encode) information and then reconstruct (decode) it. Autoencoders Graph Neural Networks (GNNs): They work with graphs (nodes and edges) rather than regular data. Graph Neural Networks (GNNs): They work with graphs (nodes and edges) rather than regular data. Graph Neural Networks (GNNs): We need all that context to understand why the most common model, ChatGPT, often hallucinates. How AI Hallucinations Happen How AI Hallucinations Happen ChatGPT runs on the Transformer architecture, first introduced in the 2017 paper, “Attention Is All You Need.” This is the very mechanism that revolutionized text processing. Transformers operate on the self-attention mechanism, which allows them to consider the global context rather than just the nearest words like older recurrent neural networks (LSTM and GRU) do. The model belongs to the GPT (Generative Pre-Trained Transformer) series, which means: Transformer the 2017 paper, “Attention Is All You Need.” Pre-trained: It was initially trained on enormous amounts of text (books, articles, websites, and code). Generative: Its task is to generate text, not just classify or extract facts. Pre-trained: It was initially trained on enormous amounts of text (books, articles, websites, and code). Pre-trained: Generative: Its task is to generate text, not just classify or extract facts. Generative: ChatGPT’s answers result from a stochastic process rather than a rigid rule. It doesn’t memorize or reproduce texts but generates responses using a probabilistic model. Word Prediction as a Probabilistic Process Word Prediction as a Probabilistic Process When ChatGPT responds, it doesn’t choose the single correct word but computes a probability distribution. P(wi|w1, w2, ..., wi-1), where: P(wi|w1, w2, ..., wi-1), “wi” — the next word in the sentence “wi” — the next word in the sentence w1, w2, ..., wi-1 — the previous words P(wi|w1, ..., wi-1) — the probability that “wi” will be the next word P(wi|w1, ..., wi-1) — the probability that “wi” will be the next word For example, if you ask, “What day is it today?” ChatGPT might have different probabilities: “Monday” — P=0.7 “Wednesday” — P=0.2 “42” — P=0.0001 “Monday” — P=0.7 “Wednesday” — P=0.2 “42” — P=0.0001 It will mostly often choose the word with the highest probability, but due to generation temperature (a parameter that controls randomness), it might sometimes choose a less likely option based on context. Context Influence and Information Forgetting Context Influence and Information Forgetting ChatGPT works with a limited context window, meaning it only "remembers" the last NN tokens. For GPT-4, the context window is about 128k tokens (around 300 pages of text). If important information is outside this context, it may: Forget details (context clipping effect) Make-up information (stochastic process) Forget details (context clipping effect) Make-up information (stochastic process) Yet, ChatGPT can often correct its answer after you ask if it's sure. However, ChatGPT can often correct its answer if you ask whether it’s sure. AI Sometimes Corrects Itself, But Why? AI Sometimes Corrects Itself, But Why? When you ask ChatGPT, “Are you sure?” it reanalyzes its answer using a new context where doubt is present. This results in: Recalculating answer probabilities. Choosing a more plausible option if one exists. Recalculating answer probabilities. Choosing a more plausible option if one exists. This process can be explained by Bayesian probability. P(A|B) = P(B|A)P(A) / P(B), where: P(A|B) = P(B|A)P(A) / P(B), P(A|B) — the probability that answer A is correct, considering your follow-up question B. P(B|A) — the probability that you would have asked if ChatGPT was initially right. P(A) — the initial probability of ChatGPT's answer. P(B) — the overall probability that you would ask. P(A|B) — the probability that answer A is correct, considering your follow-up question B. P(A|B) — the probability that answer A is correct, considering your follow-up question B. P(B|A) — the probability that you would have asked if ChatGPT was initially right. P(B|A) — the probability that you would have asked if ChatGPT was initially right. P(A) — the initial probability of ChatGPT's answer. P(A) — the initial probability of ChatGPT's answer. P(B) — the overall probability that you would ask. P(B) — the overall probability that you would ask. Too much information for you? Brain overheating? Imagine that AIs also get overwhelmed by large amounts of information. Errors Due to Overfitting and Noisy Data Errors Due to Overfitting and Noisy Data Massive amounts of text data flow into ChatGPT’s training, including noise or contradictory information, such as: Some sources say the Earth is round, while others claim it’s flat. Some sources say the Earth is round, while others claim it’s flat. AI can’t always determine which information is true when it appears with varying probabilities. AI can’t always determine which information is true when it appears with varying probabilities. These are examples of model hallucinations, which occur because ChatGPT’s weights are trained on probabilistic word associations rather than strict logic. The Bottom Line The Bottom Line Here is what we can learn from this. ChatGPT hallucinates since it: Predicts probabilistically, not deterministically. Has a limited memory (context window). Recalculates probabilities when questioned. Has training data that includes noise and contradictions. Predicts probabilistically, not deterministically. Predicts probabilistically, not deterministically. Has a limited memory (context window). Has a limited memory (context window). Recalculates probabilities when questioned. Recalculates probabilities when questioned. Has training data that includes noise and contradictions. Has training data that includes noise and contradictions. It’s that straightforward. Hope you didn’t get tired. If you did, that’s a good sign because it means you’re thinking critically, which is exactly what we should do when working with AI.

This story will praise and/or roast a product, company, service, game, or anything else people like to review on the Internet.

The is an opinion piece based on the author’s POV and does not necessarily reflect the views of HackerNoon.

AI’s Non-Determinism, Hallucinations, And... Cats?

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

AI Might Soon Watch Your Login Attempts the Blade Runner Way

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

AI Might Soon Watch Your Login Attempts the Blade Runner Way

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps