A search of Google Scholar for “ ” results in over 16,000 items since 2023. A search for phrases like “testing empathetic ai” and “evaluating empathetic ai” reduces this set to about 12,000 items. A lot of titles to go through! I certainly can’t claim to have read them all or even looked at every title, but here are my thoughts. empathetic ai We must have a common definition of empathy. We must agree to ignore the question “Can AI actually feel?” and simply focus on how we interpret what AI generates, i.e. if the AI were a human, how would we either feel or think that human is thinking or feeling? (Wow, that’s a bit of gymnastics). We must distinguish between identifying emotions, identifying empathy, generating empathetic responses, and participating in dialogues in an empathetic way. We must take into account the rich history of testing for emotional and empathetic ability in humans while recognizing how AIs are different so that historical tests can be applied, potentially modified, and evaluated appropriately. We must understand current evaluation frameworks tuned to AI. We must develop new frameworks and approaches. What Is Empathy? Merriam-Webster: ”. “The action of understanding, being aware of, being sensitive to, and vicariously experiencing the feelings, thoughts, and experience of another To eliminate the potential concerns with “experiencing” in the context of LLMs, I will rephrase this as, . the action of understanding, being aware of, being sensitive to, and vicariously experience the feelings, thoughts, and experience of another appearing to And, of course, if we are concerned with conversation, we would add, Of course, a sociopath could also and , so I will make one final adjustment. And, manifesting this in such a way that the other parties in a conversation are aware of the action. appear to manifest in such a way Empathy is: . The action of understanding, being aware of, being sensitive to , and appearing to vicariously experience the feelings, thoughts, and experience of another in a positive manner And, manifesting this is such a way that the other parties in a conversation are aware of the action. Reviewing this and the original definition, two components of empathy become evident, affective and cognitive. The affective component refers to the emotional or feeling part of empathy. It’s the ability to share or mirror the feelings of another person. For example, if a friend is sad, the affective part of your empathy might make you feel sad too, or at least get a sense of their sadness. The cognitive component, on the other hand, refers to the mental or thinking part of empathy. It’s the ability to actively identify and understand queues so that one can mentally put oneself in another person’s position. For example, if a colleague tells you about a difficult project they’re working on (a queue) in a tired voice (a queue), you might choose to try and understand their stress by actively imagining how you would feel in a similar situation. For some, this might artificially produce the affect. Can AIs Feel? At this point, most people would say that s don’t have feelings. Some would predict a future where AIs do have feelings and others where AIs don’t and can’t have feelings and yet a third group might say, “AIs do/will feel but in a different way than humans”. AI Regardless, we will not make progress on the testing of AI for empathy if we spend time debating this topic. We must focus on our interpretation of what the AIs manifest, not their internal states. Although there has been some interesting research on this topic, see . Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBench If you can’t get over this hurdle, then I suggest you simply ignore the benchmarks on this website. However, you may still enjoy the articles and conversations! Identification vs Generation There is a big leap between identifying something and doing something. Young athletes or scholars can identify what is wrong with their performance without being able to immediately perform at a higher level. Similarly, having the ability to identify emotions and empathetic conversations is not the same as being able to appear to have emotions and generate responses that another party would interpret as empathetic. In fact, there is even a step in between. Young athletes or scholars taking the input of a coach or teacher and in the moment producing better results, does not make them fully capable. If an AI produces an empathetic result as the side-effect of a test design or prompt, then the AI may have a nascent empathetic capability but it is not intrinsically empathetic. Although it may not be possible to fully understand the internal state of an AI, I do believe that the identification of emotions is a necessary condition for AI to exhibit empathy. I also believe that being able to prompt/coach an AI into delivering an empathetic response is an indication of nascent capability, i.e. fine tuning (the equivalent of human practice) may create the ability. The distinctions between vs and vs are important for discussions of the efficacy of tests and test frameworks beyond the scope of this article. identification generation coached intrinsic Identification The identification of emotions in textual content is based on the presence of indicator words, capitalization, punctuation, and grammatical structure. The ability to accurately identify sentiment pre-dates the current AI revolution by more than twenty years. In the 1990s, word n-gram intersections and symbolic reasoning were already providing impressive results. As social media grew in the early 2000s, the need for automated moderation drove lots of progress in this area. However, today’s LLMs are astonishing in their ability to identify not just general sentiment but specific emotions. This being said, there are several types of emotion expression identification required for fully empathetic conversations, I classify them as follows: explicit — User states they have a feeling. conversational — The emotions are evident from top-level textual analysis, they are present IN the conversation. driving — The emotions are DRIVING the conversation, one person manifests anger and another responds in kind. core — Emotions that cause other emotions but are not themselves caused by an emotion are CORE. They typically manifest as a result of some historical trigger that causes an anticipation (conscious or subconscious) about the future. Different researchers may classify these differently, one example supported by the Dalia Lama is the Five Continents of Emotion (Anger, Fear, Disgust, Sadness, Enjoyment) in the . Atlas of Emotion Note: a core emotion could also be driving, conversational, and explicit, but core emotions are often hidden. During the review and definition of tests or test results beyond this article, I will call attention back to these classifications. Testing Considerations Classic human testing for emotion identification typically falls into two buckets to facilitate easy testing and validation: Multiple choice tests about what emotions do or do not exist in a conversation, sometimes associated with an intensity score. Self-administered introspective tests about feelings, e.g. the , that ask about how the test taker feels in certain situations. EQ-60 These present distinct challenges for high-quality AI testing. — As pattern-matching language models, today’s s are effectively given a leg up by giving them a choice of items to identify. It makes the work easy and it does not test the AI’s ability to always identify emotions. A potentially better approach is to simply tell the AI to identify all the emotions present in a text and behind the scenes score it against either ground truth (not sure there is such a thing with emotions :-) or a key based on the statistical analysis of human responses to the same test. When assessing proposed tests in the future, I call this the . However, the statistical sampling of humans can introduce an additional risk. Assume the desire to build an AI that is better than the average human. To do this it may be necessary to ensure that the statistical sample is based on humans that have a stronger than average ability to identify emotions; otherwise, the AI may identify emotions that the average human would not identify and may be penalized in the scoring. I call this . Multiple Choice Tests AI Multiple Choice Risk Human Sampling Risk — Introspective tests about feelings provide challenges for most AI models. AIs usually have guardrails that require them to respond with something like “I am an AI, so I don’t have feelings.”. It is sometimes possible to jailbreak or prompt engineer around these constraints, but the questions then become: Introspective Tests Does the prompt either positively or negatively impact the rest of the AI’s ability with respect to empathy, or in fact anything? Jailbreak Side Effect Risk Do the responses accurately reflect the tendencies the AI will have when participating in conversations without the prompt,? J ailbreak Accuracy Risk The can be mitigated to some degree by ensuring that all models are tested with the same prompt and scores are only considered relative to each other not humans. The impact of J can only be assessed by analyzing actual conversations to see if the predicted emotional identification capability correlates with the actual empathy displayed in or emotions called out in the conversations. Jailbreak Side Effect Risk ailbreak Accuracy Risk Generation Several tests have shown that AIs are capable of generating empathetic responses to questions. One of the most impressive is which took 195 questions from Reddit’s AskDoc forum where a verified physician responded to the question and had ChatGPT respond to the same question. A pool of evaluators then rated each response as “not empathetic”, “slightly empathetic”, “moderately empathetic”, “empathetic”, and very “empathetic”. The AI responses had a 9.8 times higher prevalence for “empathetic” or “very empathetic” over physicians. Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum Although the results are impressive, I am skeptical that they would carry over to an extended dialog. Starting with a system prompt of “Your job is to respond with empathy to questions that would benefit from an empathetic response”, my experience with the manual testing of AIs is that responses tend to feel mechanical and emotionally redundant under all the following conditions: asking multiple unrelated questions deserving of an empathetic response asking multiple related questions deserving of an empathetic response conducting a dialog of mixed questions, some deserving of empathy and others not Testing Considerations As a result of the above points, I would say the test approach used in the study had a i.e. empathy displayed in response to a single question may not be an accurate measure. Another risk is what I call . This risk is a side effect of raw LLMs having no memory over time. It takes time for humans to develop understanding and empathy, it may be the same for AIs and we may be understating the ability of some AIs to manifest empathy over time if we expect a high level in response to a single question. Single Shot Empathy Risk, Empathy Understatement Risk Generative tests are also subject to If humans are tasked with evaluating the emotional content and empathetic nature of AI responses and we desire the AI to have a better than average ability, then the sample of humans must have a greater ability to identify emotions and empathy than the average human. If not, we run the risk of understating the power of the AI or undertraining it by penalizing it for identifying emotions and empathy not identified by the typical human. Human Sampling Risk. Finally, due to the layered nature of emotions in conversation, in addition to dealing directly with , there is a need to address . It may be that users should be told to consider the emotion types explicit, conversational, driving, and core (or some other set of classifications) when doing their rating while the AIs are not. Alternatively, the AIs might be selectively told to identify different types of emotions. Human Sampling Risk Question Design Risk It would be interesting to repeat the study based on Reddit AskDoc for several AIs or with a sample of evaluators known to have strong emotion and empathy-identifying skills. Standard Human Approaches For Assessing EQ and Empathy There is a long history of testing human personality types, ability to identify emotions or lack thereof (alexithymia), and engage empathetically with others. This is sure to be far more complete and coherent than anything that I could write or even generate with an LLM in a reasonable amount of time. You can see the approaches we have been focusing on by visiting the . article on Wikipedia benchmarks page Existing Frameworks For Assessing AI EQ and Empathy Several frameworks have been proposed for assessing AI EQ and empathy. Each deserves its own analysis and blog post, so I just list a few here: EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models Empathy Scale for Human–Computer Communication (ESHCC) iEval: Interactive Evaluation Framework for Open-Domain Empathetic Chatbots New Approaches We have started defining some tests to address deficiencies identified in the use of standard human tests and existing AI frameworks. An interesting finding that results in the creation of (Emotional Quotient for Depth) is that no tested LLMs identified core emotions if they were not also explicit, conversational, or driving. On the other hand, when asked to specifically identify just core emotions, several AIs were quite good. However, when given a range of all emotion types some LLMs lost the ability to identify core emotions and others performed substantially better, i.e. they identified the presence of more emotions at all levels. This resulted in the creation of (Emotional Quotient for Breadth),. EQ-D EQ-B During test development it has become evident that there are times a prompt will be needed that introduces , i.e. increases the likelihood the output will be dependent on the prompt, not the core AI. This risk may or may not invalidate comparisons with humans and may be legitimate at an application level. At the raw LLM level, it would seem immaterial to compare one AI to the other so long as the prompt is used in all tested AI’s and not biased to a particular AI. The current designs for and suffer from this risk due to the overall immaturity of AI technology. Prompt Risk EQ-D EQ-B Although there are several proposals regarding the testing of AIs for empathy, we are in the early days and there are both known and unknown issues with these approaches. There is work to do to address the known: existing tests need to be assessed for risk and risks documented or mitigated new tests cases need to be developed in the context of some existing tests more tests types need to be run across a broader range of AIs But it is the unknown which most intrigues me. How about you? Also published here.

The is an opinion piece based on the author’s POV and does not necessarily reflect the views of HackerNoon.

Testing the Depths of AI Empathy: Q1 2024 Benchmarks

Testing the Depths of AI Empathy: Frameworks and Challenges

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

AI Empathy Across Disciplines

Testing the Depths of AI Empathy: Q1 2024 Benchmarks

Testing the Extents of AI Empathy: A Nightmare Scenario

The Noonification: Motivation Doesnt Last: Heres Why (3/28/2024)

Can Machines Really Understand Your Feelings? Evaluating Large Language Models for Empathy

Testing the Depths of AI Empathy: Q3 2024 Benchmarks

AI Empathy Across Disciplines

Testing the Depths of AI Empathy: Q1 2024 Benchmarks

Testing the Extents of AI Empathy: A Nightmare Scenario

The Noonification: Motivation Doesnt Last: Heres Why (3/28/2024)

Can Machines Really Understand Your Feelings? Evaluating Large Language Models for Empathy

Testing the Depths of AI Empathy: Q3 2024 Benchmarks

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps