Ecotiois in the IoT Applications Industry

Ecotio

ecotio

We always faced a challenge to monitor our assets due to our huge scale of operations. But after the complete solution provided by ECOTIO, we are now able to track each and every asset of our company very efficiently. Also, the SMS and email alert feature is very helpful.

ECOTIO - Energy Monitoring IIoT System | Cloud Based IoT Platform

Thereafter, key elements &amp; analysis of the data can be viewed on the ECOTIO dashboard on the Web Portal and Mobile App. ... No. of Employees ... Launched to a set of select customers in target industry segments of textiles, garments and manufacturing. ... Company incorporated after being in ...

ECOTIO | YourStory

Our goal is to develop business-specific solutions that help enterprises transform their conventional processes impacting their efficiency, productivity and reduce energy cost by 20%. ECOTIO helps companies in nearly every industry to monitor and manage business assets anywhere at any time ...

ECOTIO | LinkedIn

ecotio2® Treat G is a specialised non-ionic cleaning agent that removes contaminants preparing the surface for application of coating. This product is specially formulated premium cleaner used to prepare the substrate by removing hydrophobic contaminants and ensuring the surface is appropriate ...

ecotio 2 ® product range - Photocatalyst Coatings NZ Ltd

Environmentally conscious paint energized by solar light to decompose the dirt.

Photocatalyst paint Ecotio｜オキツモ株式会社/Okitsumo Inc.

1 billion members | Manage your professional identity. Build and engage with your professional network. Access knowledge, insights and opportunities.

LinkedIn Australia: Log In or Sign Up

Ecotop Ripening Australia offers a ‘best-in-class’ fruit ripening system for the controlled and uniform ripening of fruits. Choose Ecotop for superior ripening quality, low cost operation and unique multi-functional room operation.

Ecotop Ripening Australia – World class fruit ripening systems

For more updates and upcoming events, please sign up for our newsletter

Sitemap - ECOTIO

Advit ECOTIO

Applied simply and quickly, ecotio2® reacts to sunlight and indoor light, and acts to self-clean surfaces including concrete, painted surfaces, Nawkaw coated surfaces and glass.

Self-Cleaning Coatings – Nawkaw

Establishing organizational ethical climates: How do managerial practices work?Anyi Chung Amber Y.-P. Lee - forthcoming - Journal of Business Ethics · Establishing Organizational Ethical Climates: How Do Managerial Practices Work?K. Praveen Parboteeah, Hsien Chun Chen, Ying-Tzu Lin, I.-Heng ...

Kim Economides & Julian Webb, The Impact of Ethical Climates on ...

The fight between OpenAI and Google to win the generative AI arms race has all the intensity of the war between Kendrick Lamar and Drake but without the low-blow trash talk. This week, both companies unleashed a dizzying array of updates that say a lot about their strategies: OpenAI: A More Human Gen AI On May 13, OpenAI announced several improvements to ChatGPT including multimodal capabilities. https://www.youtube.com/live/DQacCB9tDaw?si=RyXrifz6jnHbYWCW&embedable=true One feature that has captured the imagination of industry watchers is how ChatGPT is becoming more human. OpenAI demonstrated how the latest iteration of ChatGPT (GPT-4o) can detect emotions in users’ voices, analyze their facial expressions, and adjust its own tone and cadence to match user preferences. For instance, if you request a bedtime story, it can adopt a gentle, whispering tone. It can respond in a playful, sarcastic manner if you want it to. It even has the capability to sing when asked. https://youtu.be/D9byh4MAsUQ?si=DnmC0PteXYgj833j&embedable=true The demonstration immediately drew comparisons to Her, the 2013 movie in which a virtual assistant seduces an introvert… But making OpenAI more human is not in and of itself nothing new. There is a growing trend, especially among Gen Z, to turn to AI for personal companionship. Apps such as Pi offer life coaching in a helpful and conversational manner. Replika is a virtual friend and mental health lifeline. Blush is an AI-generated dating simulator. And there are many, many more examples. OpenAI knows that humanlike AI is the future, especially with Gen Z. Recently, Npj Mental Health Research studied how college students use Replika. The study observed, “Many used Replika in multiple, overlapping ways—as a friend, a therapist, and an intellectual mirror. Many also held overlapping and often conflicting beliefs about Replika—calling it a machine, an intelligence, and a human. Critically, 3% reported that Replika halted their suicidal ideation.” The significance of how ChatGPT is evolving is that OpenAI is making an incredibly popular everyday utility more human. There is no turning back now. Google: Doubling Down on AI-Powered Search At Google’s annual I/O conference on May 14, Google showed that it is leaning into its core strengths with gen AI-powered search. For instance, the AI Overview feature provides a brief, AI-generated answer to searches, along with a series of links to the sources cited in the response. Google is also introducing multistep reasoning (refining search results without requiring multiple searches in the same search session). Let’s say you are searching for a new yoga or pilates studio that is popular, conveniently located for your daily commute, and also offers a discount for new members. Soon, you will be able to ask Google Search to “find the best yoga or pilates studios in Boston and show details on their intro offers and walking time from Beacon Hill.” https://youtu.be/s4InWsd-J6g?si=X429BeoiivA6taTQ&embedable=true If you are an avid user of Google search for wayfinding, you can appreciate how important and valuable multistep reasoning will be when rolled out. The company is also introducing updates to its Gemini gen AI chatbot, including the ability to respond to longer prompts, engage in audio-based conversations, and provide more powerful functionality, including a tool summarizing 1,500 pages of the text uploaded by a user. These are important updates to a tool that I personally find more useful than ChatGPT in many ways. But for Google’s business, it is more critical that the company inject gen AI across its powerful search ecosystem, including Google Search, YouTube, and Email. Search is Google’s bread-and-butter business. And Google is fortifying its moat. Photo by Mohamed Nohassi via Unsplash The fight between OpenAI and Google to win the generative AI arms race has all the intensity of the war between Kendrick Lamar and Drake but without the low-blow trash talk. This week, both companies unleashed a dizzying array of updates that say a lot about their strategies: OpenAI: A More Human Gen AI On May 13, OpenAI announced several improvements to ChatGPT including multimodal capabilities. https://www.youtube.com/live/DQacCB9tDaw?si=RyXrifz6jnHbYWCW&embedable=true https://www.youtube.com/live/DQacCB9tDaw?si=RyXrifz6jnHbYWCW&embedable=true One feature that has captured the imagination of industry watchers is how ChatGPT is becoming more human. OpenAI demonstrated how the latest iteration of ChatGPT (GPT-4o) can detect emotions in users’ voices, analyze their facial expressions, and adjust its own tone and cadence to match user preferences. For instance, if you request a bedtime story, it can adopt a gentle, whispering tone. It can respond in a playful, sarcastic manner if you want it to. It even has the capability to sing when asked. https://youtu.be/D9byh4MAsUQ?si=DnmC0PteXYgj833j&embedable=true https://youtu.be/D9byh4MAsUQ?si=DnmC0PteXYgj833j&embedable=true The demonstration immediately drew comparisons to Her , the 2013 movie in which a virtual assistant seduces an introvert… But making OpenAI more human is not in and of itself nothing new. There is a growing trend, especially among Gen Z, to turn to AI for personal companionship. Apps such as Pi offer life coaching in a helpful and conversational manner. Replika is a virtual friend and mental health lifeline. Blush is an AI-generated dating simulator. And there are many, many more examples. The demonstration immediately drew comparisons to Her Her OpenAI knows that humanlike AI is the future, especially with Gen Z. Recently, Npj Mental Health Research studied how college students use Replika. The study observed , “Many used Replika in multiple, overlapping ways—as a friend, a therapist, and an intellectual mirror. Many also held overlapping and often conflicting beliefs about Replika—calling it a machine, an intelligence, and a human. Critically, 3% reported that Replika halted their suicidal ideation.” The study observed The significance of how ChatGPT is evolving is that OpenAI is making an incredibly popular everyday utility more human. There is no turning back now. Google: Doubling Down on AI-Powered Search At Google’s annual I/O conference on May 14, Google showed that it is leaning into its core strengths with gen AI-powered search . For instance, the AI Overview feature provides a brief, AI-generated answer to searches, along with a series of links to the sources cited in the response. Google is also introducing multistep reasoning (refining search results without requiring multiple searches in the same search session). Google showed that it is leaning into its core strengths with gen AI-powered search Let’s say you are searching for a new yoga or pilates studio that is popular, conveniently located for your daily commute, and also offers a discount for new members. Soon, you will be able to ask Google Search to “find the best yoga or pilates studios in Boston and show details on their intro offers and walking time from Beacon Hill.” https://youtu.be/s4InWsd-J6g?si=X429BeoiivA6taTQ&embedable=true https://youtu.be/s4InWsd-J6g?si=X429BeoiivA6taTQ&embedable=true If you are an avid user of Google search for wayfinding, you can appreciate how important and valuable multistep reasoning will be when rolled out. The company is also introducing updates to its Gemini gen AI chatbot, including the ability to respond to longer prompts, engage in audio-based conversations, and provide more powerful functionality, including a tool summarizing 1,500 pages of the text uploaded by a user. These are important updates to a tool that I personally find more useful than ChatGPT in many ways. But for Google’s business, it is more critical that the company inject gen AI across its powerful search ecosystem, including Google Search, YouTube, and Email. Search is Google’s bread-and-butter business. And Google is fortifying its moat. Photo by Mohamed Nohassi via Unsplash Photo by Mohamed Nohassi via Unsplash

Generative AI Clash: OpenAI’s Emotional AI vs. Google’s Enhanced Search

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) D.Sinclair, Imense Ltd, and email: david@imense.com;
(2) W.T.Pye, Warwick University, and email:  willempye@gmail.com. Table of Links Abstract and Introduction
Interrogating the LLM with an Emotion Eliciting Tail Prompt
PCA analysis of the Emotion of Amazon reviews
Future Work
Conclusions
Acknowledgments and References 2. Interrogating the LLM with an Emotion Eliciting Tail Prompt In this work we used Facebook’s open source LlaMa2 7 billion weight LLM as the core engine [2]. It was necessary to use a LLM that allowed access to raw token probabilities after a prompt. The model ran on a Mac Studio with 32 gigabytes of RAM. With this combination of hardware and model it took 2 minutes to compute the probabilities of the emotion descriptors in the emotion dictionary given below. 2.1. The Emotion Dictionary The English language is blessed with many words and an extensive literature providing examples of the usage of these words in appropriate contexts. For the purposes of LLMs, it is the context of a word that conveys it’s meaning. A reader will infer the meaning of an unfamiliar word through the context they find the word used in, provided they understand the context. As an example, ‘The shotgun discombobulated the rabbit.’ shows how meaning can be moderated by context. The context created by a tail prompt will favour an associated class of words. For the experiment detailed in this paper the following tail prompt was used to elicit emotion descriptors, ’Reading this makes me feel’. It is likely that specific emotion eliciting tail prompts will favour specific sub classes of descriptor but studying this is beyond the scope of this paper The following words were chosen to provide a broad sample of emotion descriptors. acceptance, admiration, adoration, affection, afraid, agitation, agony, aggressive, alarm, alarmed, alienation, amazement, ambivalence, amusement, anger, anguish, annoyed anticipating, anxious, apathy, apprehension, arrogant, assertive, astonished, attentiveness, attraction, aversion, awe, baffled, bewildered, bitter, bitter sweetness, bliss, bored, brazen, brooding, calm, carefree, careless, caring, charity, cheeky, cheerfulness, claustrophobic, coercive, comfortable, confident, confusion, contempt, content, courage, cowardly, cruelty, curiosity, cynicism, dazed, dejection, delighted, demoralized, depressed, desire, despair, determined, disappointment, disbelief, discombobulated, discomfort, discontentment, disgruntled, disgust, disheartened, dislike, dismay, disoriented, dispirited, displeasure, distraction, distress, disturbed, dominant, doubt, dread, driven, dumbstruck, eagerness, ecstasy, elation, embarrassment, empathy, enchanted, enjoyment, enlightened, ennui, enthusiasm, envy, epiphany, euphoria, exasperated, excitement, expectancy, fascination, fear, flakey, focused, fondness, friendliness, fright, frustrated, fury, glee, gloomy, glumness, gratitude, greed, grief, grouchiness, grumpiness, guilt, happiness, hate, hatred, helpless, homesickness, hope, hopeless, horrified, hospitable, humiliation, humility, hurt, hysteria, idleness, impatient, indifference, indignant, infatuation, infuriated, insecurity, insightful, insulted, interest, intrigued, irritated, isolated, jealousy, joviality, joy, jubilation, kind, lazy, liking, loathing, lonely, longing, loopy, love, lust, mad, melancholy, miserable, miserliness, mixed up, modesty, moody, mortified, mystified, nasty, nauseated, negative, neglect, nervous, nostalgic, numb, obstinate, offended, optimistic, outrage, overwhelmed, panicked, paranoid, passion, patience, pensiveness, perplexed, persevering, pessimism, pity, pleased, pleasure, politeness, positive, possessive, powerless, pride, puzzled, rage, rash, rattled, regret, rejected, relaxed, relieved, reluctant, remorse, resentment, resignation, restlessness, revulsion, ruthless, sadness, satisfaction, scared, schadenfreude, scorn, self-caring, self-compassionate, self-confident, self-conscious, self-critical, self-loathing, self-motivated, self-pity, self-respecting, self-understanding, sentimentality, serenity, shame, shameless, shocked, smug, sorrow, spite, stressed, strong, stubborn, stuck, submissive, suffering, sullenness, surprise, suspense, suspicious, sympathy, tenderness, tension, terror, thankfulness, thrilled, tired, tolerance, torment, triumphant, troubled, trust, uncertainty, undermined, uneasiness, unhappy, unnerved, unsettled, unsure, upset, vengeful, vicious, vigilance, vulnerable, weak, woe, worried, worthy, wrath. This set of words is not intended to be complete or definitive in any way. Using the tail prompt without restricting the return to emotion descriptors elicits general waffle responses that are not straightforward to extract sentiment form. 2.1.1. Estimating the emotion probability vector LlaMa2 [2] has been released in such a way as to allow developers to access estimated token weights returned in response to a prompt. LlaMa2 has an internal vocabulary size of roughly 30,000 tokens. This means that when LlaMa 2 estimates the probability of the next token in a sequence the probability vector will have 30,000 elements. Some of the words in the emotion descriptor list are made up of more than one token in which case forward conditional probabilities are used. Figure 2.1.1 shows the scaled probability distribution over words from the emotion dictionary elicited by the tail prompt for the Amazon review text, ‘I read a lot of negative reviews about the Fitbit inspire 2, I took a chance and hoped the one I ordered would be one of the great ones that worked. Unfortunately that was not the case. I unpacked it, charged it, downloaded the app. I took a walk with it on before the sun went down. I have the Google Fit app on my phone that tracks my steps also. The phone was in my jeans pocket. When I got home I compared the two, Google Fit said 4,458 steps, Fitbit said 1,168. Apparently Fitbit works with wrist motion which I don’t have while pushing a walker around the neighborhood. I downloaded the manual and noticed you can put it on a clip (that wasn’t included). That would work for me. So I started to scroll through the different features except I couldn’t scroll through all of them. While scrolling I must have turned on the stopwatch. I couldn’t turn it off. Then I couldn’t scroll through anything except water lock feature. I had to turn on the water lock to get back to the stopwatch. Then the side buttons stopped working. I had it a total of 5 hours. I packed it up and started the Amazon return. I did get a full refund. Very disappointing.’ The text from 50 Amazon reviews of a book was borrowed from https://www.amazon.com/dp/B000WM9UK2. The reviews were for the most part favourable. Example review texts include: ‘The Children of H´urin is a great tragedy mixed with grace. The genealogical records at the beginning may be difficult to get through, but the story very quickly gains speed. I only gave the story four stars because of the difficulty of the first few chapters, much like Matthew’s genealogy of Christ at the beginning of his gospel. Although such records are important in both they are nonetheless difficult to get through. I still definitely would recommend this book, though, as it depicts the horrible effects of evil powers on good men and women, and yet, we must continue to resist evil no matter the tragic end. It is very telling that in Tolkien’s world, at the end of days when Morgoth returns, that it is T´urin, a man, who ends him once and for all. Those whom Satan most destroys in this life are they who will ultimately deal his death blow, as the Revelation says, “They overcame him by the blood of the Lamb and by the word of their ‘martyria,’ [witness, testimony],” those like T´urin, or in Scripture, those like Job.’ Figure 2.1.2 shows the emotion vectors for all 50 processed reviews. The 10 most probable emotions experienced from purchasing the product were: depressed, kind, nostalgic, tired, hopeless, lonely, hope, calm, lazy, confident. This paper is available on arxiv under CC 4.0 license. Authors: (1) D.Sinclair, Imense Ltd, and email: david@imense.com; (2) W.T.Pye, Warwick University, and email:  willempye@gmail.com. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) D.Sinclair, Imense Ltd, and email: david@imense.com; (2) W.T.Pye, Warwick University, and email:  willempye@gmail.com. Table of Links Abstract and Introduction Interrogating the LLM with an Emotion Eliciting Tail Prompt PCA analysis of the Emotion of Amazon reviews Future Work Conclusions Acknowledgments and References Abstract and Introduction Abstract and Introduction Interrogating the LLM with an Emotion Eliciting Tail Prompt Interrogating the LLM with an Emotion Eliciting Tail Prompt PCA analysis of the Emotion of Amazon reviews PCA analysis of the Emotion of Amazon reviews Future Work Future Work Conclusions Conclusions Acknowledgments and References Acknowledgments and References 2. Interrogating the LLM with an Emotion Eliciting Tail Prompt In this work we used Facebook’s open source LlaMa2 7 billion weight LLM as the core engine [2]. It was necessary to use a LLM that allowed access to raw token probabilities after a prompt. The model ran on a Mac Studio with 32 gigabytes of RAM. With this combination of hardware and model it took 2 minutes to compute the probabilities of the emotion descriptors in the emotion dictionary given below. 2.1. The Emotion Dictionary The English language is blessed with many words and an extensive literature providing examples of the usage of these words in appropriate contexts. For the purposes of LLMs, it is the context of a word that conveys it’s meaning. A reader will infer the meaning of an unfamiliar word through the context they find the word used in, provided they understand the context. As an example, ‘The shotgun discombobulated the rabbit.’ shows how meaning can be moderated by context. The context created by a tail prompt will favour an associated class of words. For the experiment detailed in this paper the following tail prompt was used to elicit emotion descriptors, ’Reading this makes me feel’. It is likely that specific emotion eliciting tail prompts will favour specific sub classes of descriptor but studying this is beyond the scope of this paper The following words were chosen to provide a broad sample of emotion descriptors. acceptance, admiration, adoration, affection, afraid, agitation, agony, aggressive, alarm, alarmed, alienation, amazement, ambivalence, amusement, anger, anguish, annoyed anticipating, anxious, apathy, apprehension, arrogant, assertive, astonished, attentiveness, attraction, aversion, awe, baffled, bewildered, bitter, bitter sweetness, bliss, bored, brazen, brooding, calm, carefree, careless, caring, charity, cheeky, cheerfulness, claustrophobic, coercive, comfortable, confident, confusion, contempt, content, courage, cowardly, cruelty, curiosity, cynicism, dazed, dejection, delighted, demoralized, depressed, desire, despair, determined, disappointment, disbelief, discombobulated, discomfort, discontentment, disgruntled, disgust, disheartened, dislike, dismay, disoriented, dispirited, displeasure, distraction, distress, disturbed, dominant, doubt, dread, driven, dumbstruck, eagerness, ecstasy, elation, embarrassment, empathy, enchanted, enjoyment, enlightened, ennui, enthusiasm, envy, epiphany, euphoria, exasperated, excitement, expectancy, fascination, fear, flakey, focused, fondness, friendliness, fright, frustrated, fury, glee, gloomy, glumness, gratitude, greed, grief, grouchiness, grumpiness, guilt, happiness, hate, hatred, helpless, homesickness, hope, hopeless, horrified, hospitable, humiliation, humility, hurt, hysteria, idleness, impatient, indifference, indignant, infatuation, infuriated, insecurity, insightful, insulted, interest, intrigued, irritated, isolated, jealousy, joviality, joy, jubilation, kind, lazy, liking, loathing, lonely, longing, loopy, love, lust, mad, melancholy, miserable, miserliness, mixed up, modesty, moody, mortified, mystified, nasty, nauseated, negative, neglect, nervous, nostalgic, numb, obstinate, offended, optimistic, outrage, overwhelmed, panicked, paranoid, passion, patience, pensiveness, perplexed, persevering, pessimism, pity, pleased, pleasure, politeness, positive, possessive, powerless, pride, puzzled, rage, rash, rattled, regret, rejected, relaxed, relieved, reluctant, remorse, resentment, resignation, restlessness, revulsion, ruthless, sadness, satisfaction, scared, schadenfreude, scorn, self-caring, self-compassionate, self-confident, self-conscious, self-critical, self-loathing, self-motivated, self-pity, self-respecting, self-understanding, sentimentality, serenity, shame, shameless, shocked, smug, sorrow, spite, stressed, strong, stubborn, stuck, submissive, suffering, sullenness, surprise, suspense, suspicious, sympathy, tenderness, tension, terror, thankfulness, thrilled, tired, tolerance, torment, triumphant, troubled, trust, uncertainty, undermined, uneasiness, unhappy, unnerved, unsettled, unsure, upset, vengeful, vicious, vigilance, vulnerable, weak, woe, worried, worthy, wrath. This set of words is not intended to be complete or definitive in any way. Using the tail prompt without restricting the return to emotion descriptors elicits general waffle responses that are not straightforward to extract sentiment form. 2.1.1. Estimating the emotion probability vector LlaMa2 [2] has been released in such a way as to allow developers to access estimated token weights returned in response to a prompt. LlaMa2 has an internal vocabulary size of roughly 30,000 tokens. This means that when LlaMa 2 estimates the probability of the next token in a sequence the probability vector will have 30,000 elements. Some of the words in the emotion descriptor list are made up of more than one token in which case forward conditional probabilities are used. Figure 2.1.1 shows the scaled probability distribution over words from the emotion dictionary elicited by the tail prompt for the Amazon review text, ‘I read a lot of negative reviews about the Fitbit inspire 2, I took a chance and hoped the one I ordered would be one of the great ones that worked. Unfortunately that was not the case. I unpacked it, charged it, downloaded the app. I took a walk with it on before the sun went down. I have the Google Fit app on my phone that tracks my steps also. The phone was in my jeans pocket. When I got home I compared the two, Google Fit said 4,458 steps, Fitbit said 1,168. Apparently Fitbit works with wrist motion which I don’t have while pushing a walker around the neighborhood. I downloaded the manual and noticed you can put it on a clip (that wasn’t included). That would work for me. So I started to scroll through the different features except I couldn’t scroll through all of them. While scrolling I must have turned on the stopwatch. I couldn’t turn it off. Then I couldn’t scroll through anything except water lock feature. I had to turn on the water lock to get back to the stopwatch. Then the side buttons stopped working. I had it a total of 5 hours. I packed it up and started the Amazon return. I did get a full refund. Very disappointing.’ ‘I read a lot of negative reviews about the Fitbit inspire 2, I took a chance and hoped the one I ordered would be one of the great ones that worked. Unfortunately that was not the case. I unpacked it, charged it, downloaded the app. I took a walk with it on before the sun went down. I have the Google Fit app on my phone that tracks my steps also. The phone was in my jeans pocket. When I got home I compared the two, Google Fit said 4,458 steps, Fitbit said 1,168. Apparently Fitbit works with wrist motion which I don’t have while pushing a walker around the neighborhood. I downloaded the manual and noticed you can put it on a clip (that wasn’t included). That would work for me. So I started to scroll through the different features except I couldn’t scroll through all of them. While scrolling I must have turned on the stopwatch. I couldn’t turn it off. Then I couldn’t scroll through anything except water lock feature. I had to turn on the water lock to get back to the stopwatch. Then the side buttons stopped working. I had it a total of 5 hours. I packed it up and started the Amazon return. I did get a full refund. Very disappointing.’ The text from 50 Amazon reviews of a book was borrowed from https://www.amazon.com/dp/B000WM9UK2. The reviews were for the most part favourable. Example review texts include: ‘The Children of H´urin is a great tragedy mixed with grace. The genealogical records at the beginning may be difficult to get through, but the story very quickly gains speed. I only gave the story four stars because of the difficulty of the first few chapters, much like Matthew’s genealogy of Christ at the beginning of his gospel. Although such records are important in both they are nonetheless difficult to get through. I still definitely would recommend this book, though, as it depicts the horrible effects of evil powers on good men and women, and yet, we must continue to resist evil no matter the tragic end. It is very telling that in Tolkien’s world, at the end of days when Morgoth returns, that it is T´urin, a man, who ends him once and for all. Those whom Satan most destroys in this life are they who will ultimately deal his death blow, as the Revelation says, “They overcame him by the blood of the Lamb and by the word of their ‘martyria,’ [witness, testimony],” those like T´urin, or in Scripture, those like Job.’ ‘The Children of H´urin is a great tragedy mixed with grace. The genealogical records at the beginning may be difficult to get through, but the story very quickly gains speed. I only gave the story four stars because of the difficulty of the first few chapters, much like Matthew’s genealogy of Christ at the beginning of his gospel. Although such records are important in both they are nonetheless difficult to get through. I still definitely would recommend this book, though, as it depicts the horrible effects of evil powers on good men and women, and yet, we must continue to resist evil no matter the tragic end. It is very telling that in Tolkien’s world, at the end of days when Morgoth returns, that it is T´urin, a man, who ends him once and for all. Those whom Satan most destroys in this life are they who will ultimately deal his death blow, as the Revelation says, “They overcame him by the blood of the Lamb and by the word of their ‘martyria,’ [witness, testimony],” those like T´urin, or in Scripture, those like Job.’ Figure 2.1.2 shows the emotion vectors for all 50 processed reviews. The 10 most probable emotions experienced from purchasing the product were: depressed, kind, nostalgic, tired, hopeless, lonely, hope, calm, lazy, confident.

Estimate Emotion Probability Vectors: Interrogating the LLM with an Emotion Eliciting Tail Prompt

আবেগ সম্ভাবনা ভেক্টর অনুমান করুন: একটি আবেগ এলিসিটিং টেইল প্রম্পটের সাথে এলএলএমকে জিজ্ঞাসাবাদ করা

Schätzen Sie Wahrscheinlichkeitsvektoren für Emotionen: Abfragen des LLM mit einem Emotionen hervorrufenden Tail Prompt

Estimar vectores de probabilidad de emociones: interrogar al LLM con un mensaje de cola que provoca emociones

Estimer les vecteurs de probabilité d'émotion : interroger le LLM avec une invite de queue suscitant des émotions

भावना संभाव्यता सदिशों का अनुमान लगाना: भावना उद्घाटित करने वाले टेल प्रॉम्प्ट के साथ LLM से पूछताछ करना

感情確率ベクトルの推定: 感情を喚起するテールプロンプトによる LLM の調査

감정 확률 벡터 추정: 감정 유도 꼬리 프롬프트로 LLM 조사

Estimar vetores de probabilidade de emoção: interrogando o LLM com um prompt de cauda que provoca emoção

Оценка векторов вероятности эмоций: опрос LLM с помощью хвостовой подсказки, вызывающей эмоции

Duygu Olasılığı Vektörlerini Tahmin Edin: Yüksek Lisans'ı Duygu Ortaya Çıkaran Kuyruk İstemi ile Sorgulamak

Ước tính các vectơ xác suất cảm xúc: Thẩm vấn LLM bằng lời nhắc gợi ý cảm xúc

估计情绪概率向量：用引发情绪的尾部提示询问法学硕士

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) D.Sinclair, Imense Ltd, and email: david@imense.com;
(2) W.T.Pye, Warwick University, and email:  willempye@gmail.com. Table of Links Abstract and Introduction
Interrogating the LLM with an Emotion Eliciting Tail Prompt
PCA analysis of the Emotion of Amazon reviews
Future Work
Conclusions
Acknowledgments and References 6. Acknowledgements The authors acknowledge the extraordinary generosity of Meta in releasing model weights in a reasonable way for their LlaMa2 series of pre-trained Large Language Models. 7. References [1] Open AI. Chatgpt-4 technical report. 2023. URL https://arxiv.org/pdf/2303.08774.pdf. [2] Meta GenAI, Thomas Scialom, and Hugo Touvron. Llama 2: Open foundation and fine-tuned chat models. 2023. URL https://arxiv.org/pdf/2307.09288.pdf. [3] Rosalind W. Picard. Affective computing. MIT Press, 1997. [4] J Strabismus. The jedi religion: Is love the force? Amazon Kindle, 2013. [5] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. CoRR, abs/1706.03762, 2017. URL http://arxiv.org/abs/1706.03762. [6] Wenxuan Zhang, Yue Deng, Bing Liu, Sinno Jialin Pan, and Lidong Bing. Sentiment analysis in the era of large language models: A reality check, 2023. This paper is available on arxiv under CC 4.0 license. Authors: (1) D.Sinclair, Imense Ltd, and email: david@imense.com; (2) W.T.Pye, Warwick University, and email:  willempye@gmail.com. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) D.Sinclair, Imense Ltd, and email: david@imense.com; (2) W.T.Pye, Warwick University, and email:  willempye@gmail.com. Table of Links Abstract and Introduction Interrogating the LLM with an Emotion Eliciting Tail Prompt PCA analysis of the Emotion of Amazon reviews Future Work Conclusions Acknowledgments and References Abstract and Introduction Abstract and Introduction Interrogating the LLM with an Emotion Eliciting Tail Prompt Interrogating the LLM with an Emotion Eliciting Tail Prompt PCA analysis of the Emotion of Amazon reviews PCA analysis of the Emotion of Amazon reviews Future Work Future Work Conclusions Conclusions Acknowledgments and References Acknowledgments and References 6. Acknowledgements The authors acknowledge the extraordinary generosity of Meta in releasing model weights in a reasonable way for their LlaMa2 series of pre-trained Large Language Models. 7. References [1] Open AI. Chatgpt-4 technical report. 2023. URL https://arxiv.org/pdf/2303.08774.pdf. [2] Meta GenAI, Thomas Scialom, and Hugo Touvron. Llama 2: Open foundation and fine-tuned chat models. 2023. URL https://arxiv.org/pdf/2307.09288.pdf. [3] Rosalind W. Picard. Affective computing. MIT Press, 1997. [4] J Strabismus. The jedi religion: Is love the force? Amazon Kindle, 2013. [5] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. CoRR, abs/1706.03762, 2017. URL http://arxiv.org/abs/1706.03762. [6] Wenxuan Zhang, Yue Deng, Bing Liu, Sinno Jialin Pan, and Lidong Bing. Sentiment analysis in the era of large language models: A reality check, 2023.

Estimate Emotion Probability Vectors Using LLMs: Acknowledgements and References

LLMs ব্যবহার করে আবেগ সম্ভাব্যতা ভেক্টর অনুমান করুন: স্বীকৃতি এবং রেফারেন্স

Schätzen Sie Emotionswahrscheinlichkeitsvektoren mithilfe von LLMs: Danksagungen und Referenzen

Estimar vectores de probabilidad de emociones utilizando LLM: reconocimientos y referencias

Estimer les vecteurs de probabilité d'émotion à l'aide de LLM : remerciements et références

एलएलएम का उपयोग करके भावना संभाव्यता वैक्टर का अनुमान लगाएं: आभार और संदर्भ

LLM を使用した感情確率ベクトルの推定: 謝辞と参考文献

LLM을 사용한 감정 확률 벡터 추정: 감사의 글 및 참고 자료

Estimar vetores de probabilidade de emoção usando LLMs: agradecimentos e referências

Оцените векторы вероятности эмоций с помощью LLM: благодарности и ссылки

Yüksek Lisans'ı Kullanarak Duygu Olasılığı Vektörlerini Tahmin Edin: Teşekkür ve Referanslar

Ước tính các vectơ xác suất cảm xúc bằng cách sử dụng LLM: Lời cảm ơn và tài liệu tham khảo

使用 LLM 估计情绪概率向量：致谢和参考文献

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) D.Sinclair, Imense Ltd, and email: david@imense.com;
(2) W.T.Pye, Warwick University, and email:  willempye@gmail.com. Table of Links Abstract and Introduction
Interrogating the LLM with an Emotion Eliciting Tail Prompt
PCA analysis of the Emotion of Amazon reviews
Future Work
Conclusions
Acknowledgments and References 5. Conclusions LLMs are by their nature designed to provide text strings as a response to a test prompt. This is not always the most useful format for information to be returned in. Internally within the LLM there exist probability distributions over tokens. The paper presents an example of how to build part of an emotion based synthetic consciousness by deriving the vector of emotion descriptor probabilities over a dictionary of emotional terms. There are a range of things that can be done with this emotion probability vector including fine grained review analysis, predicting a response to marketing messages, offence detection etc. It is possible that the emotion probability vector might be a step on the road to synthetic consciousness and that it might provide a means of making robots more empathetic through allowing them to make a prediction as to how something they might say will make the recipient feel. If reasonable responses are desired from an LLM it might be a good policy not to train the LLM on the mad shouting that pervades anti-social media and analogously it might be a good idea not to train young minds similarly. This paper is available on arxiv under CC 4.0 license. Authors: (1) D.Sinclair, Imense Ltd, and email: david@imense.com; (2) W.T.Pye, Warwick University, and email:  willempye@gmail.com. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) D.Sinclair, Imense Ltd, and email: david@imense.com; (2) W.T.Pye, Warwick University, and email:  willempye@gmail.com. Table of Links Abstract and Introduction Interrogating the LLM with an Emotion Eliciting Tail Prompt PCA analysis of the Emotion of Amazon reviews Future Work Conclusions Acknowledgments and References Abstract and Introduction Abstract and Introduction Interrogating the LLM with an Emotion Eliciting Tail Prompt Interrogating the LLM with an Emotion Eliciting Tail Prompt PCA analysis of the Emotion of Amazon reviews PCA analysis of the Emotion of Amazon reviews Future Work Future Work Conclusions Conclusions Acknowledgments and References Acknowledgments and References 5. Conclusions LLMs are by their nature designed to provide text strings as a response to a test prompt. This is not always the most useful format for information to be returned in. Internally within the LLM there exist probability distributions over tokens. The paper presents an example of how to build part of an emotion based synthetic consciousness by deriving the vector of emotion descriptor probabilities over a dictionary of emotional terms. There are a range of things that can be done with this emotion probability vector including fine grained review analysis, predicting a response to marketing messages, offence detection etc. It is possible that the emotion probability vector might be a step on the road to synthetic consciousness and that it might provide a means of making robots more empathetic through allowing them to make a prediction as to how something they might say will make the recipient feel. If reasonable responses are desired from an LLM it might be a good policy not to train the LLM on the mad shouting that pervades anti-social media and analogously it might be a good idea not to train young minds similarly.

Estimate Emotion Probability Vectors Using LLMs:  Conclusions

LLMs ব্যবহার করে আবেগ সম্ভাব্যতা ভেক্টর অনুমান করুন: উপসংহার

Schätzen Sie Emotionswahrscheinlichkeitsvektoren mithilfe von LLMs: Schlussfolgerungen

Estimar vectores de probabilidad de emociones utilizando LLM: conclusiones

Estimer les vecteurs de probabilité d'émotion à l'aide des LLM : conclusions

एलएलएम का उपयोग करके भावना संभाव्यता वैक्टर का अनुमान लगाएं: निष्कर्ष

LLM を用いた感情確率ベクトルの推定: 結論

LLM을 사용한 감정 확률 벡터 추정: 결론

Estimar vetores de probabilidade de emoção usando LLMs: conclusões

Оцените векторы вероятности эмоций с помощью LLM: выводы

Yüksek Lisans Programlarını Kullanarak Duygu Olasılığı Vektörlerini Tahmin Edin: Sonuçlar

Ước tính các vectơ xác suất cảm xúc bằng cách sử dụng LLM: Kết luận

使用 LLM 估计情绪概率向量：结论

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) D.Sinclair, Imense Ltd, and email: david@imense.com;
(2) W.T.Pye, Warwick University, and email:  willempye@gmail.com. Table of Links Abstract and Introduction
Interrogating the LLM with an Emotion Eliciting Tail Prompt
PCA analysis of the Emotion of Amazon reviews
Future Work
Conclusions
Acknowledgments and References 4. Future Work The authors had hoped to build a skeletal self aware emotion derived synthetic consciousness. The state of the (synthetic conscious) system is described in text. The synthetic consciousness’s perception of it’s own state is the vector of probabilities of emotion descriptors derived from one or more tail prompts used to estimate the relevant token probabilities via the LLM associated with the system. It was hoped that the fine grained probability vector would be useable to determine whether one text description of current or future state was preferable to another state. This would provide a general means of arbitrating between potentially unrelated behaviours with unrelated goals. It was further hoped that a tail prompt could be used to elicit a text description of a putative course of action from a LLM. A brief series of experiments with various LLMs indicated that this was not going to work. Example text and tail prompts included things like, ‘My girlfriend hates me. How can I make this better?’. The replies read like excerpts from self help books or newspaper psychologist waffle and were not specific enough to have a chance at creating a predicted future in text through re-insertion into the LLM. Similar phrases appended to bad restaurant reviews elicited similarly nondescript advice. The take home message was that the remedy proposed was too vague for the LLM to make any meaningful prediction about the state after the advice had been taken. This does not mean that more thoughtful prompt design will not elicit useful action prediction hoped to improve the self perceived state of a synthetic consciousness. 4.1. Longer Term Behaviour Regulation If synthetic consciousnesses are to have a roll in the future of humanity it would seem desirable to endow them with a degree of empathy for living beings and a longer term view than simple optimisation to fulfil a limited short term goal. For example if a synthetic consciousness were to have a goal of; ‘make money for shareholders in a company’ it would be great if it would choose not to open an open cast coal mine and build coal fired power stations, or ‘take out life insurance policies on random individuals and murder them with self driving cars’. It has been argued that longer term altruistic behaviour in humans is moderated by love [4] and a computationally feasible definition of Love is given: ‘Love is that which prefers life’. Love in humans is intimately related to the production and fostering of new lives. Love seems to act to prefer a future in which there is more life. Acting contrary to love and creating a future where there is a wasteland with nothing living in it is generally viewed as wrong. The advent of LLMs offers a means of creating text descriptors of predicted futures with a range of time constants. The emotion vectors associated with predicted futures can be used to arbitrate between sort term behaviours. Text descriptors can play a roll in behaviour regulation and a machine may act in a way that at least in part mirrors Love. For example if an agricultural robot was invited to dump unused pesticide into a river it might reasonably infer that this action was in principle wrong. This paper is available on arxiv under CC 4.0 license. Authors: (1) D.Sinclair, Imense Ltd, and email: david@imense.com; (2) W.T.Pye, Warwick University, and email:  willempye@gmail.com. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) D.Sinclair, Imense Ltd, and email: david@imense.com; (2) W.T.Pye, Warwick University, and email:  willempye@gmail.com. Table of Links Abstract and Introduction Interrogating the LLM with an Emotion Eliciting Tail Prompt PCA analysis of the Emotion of Amazon reviews Future Work Conclusions Acknowledgments and References Abstract and Introduction Abstract and Introduction Interrogating the LLM with an Emotion Eliciting Tail Prompt Interrogating the LLM with an Emotion Eliciting Tail Prompt PCA analysis of the Emotion of Amazon reviews PCA analysis of the Emotion of Amazon reviews Future Work Future Work Conclusions Conclusions Acknowledgments and References Acknowledgments and References 4. Future Work The authors had hoped to build a skeletal self aware emotion derived synthetic consciousness. The state of the (synthetic conscious) system is described in text. The synthetic consciousness’s perception of it’s own state is the vector of probabilities of emotion descriptors derived from one or more tail prompts used to estimate the relevant token probabilities via the LLM associated with the system. It was hoped that the fine grained probability vector would be useable to determine whether one text description of current or future state was preferable to another state. This would provide a general means of arbitrating between potentially unrelated behaviours with unrelated goals. It was further hoped that a tail prompt could be used to elicit a text description of a putative course of action from a LLM. A brief series of experiments with various LLMs indicated that this was not going to work. Example text and tail prompts included things like, ‘My girlfriend hates me. How can I make this better?’. The replies read like excerpts from self help books or newspaper psychologist waffle and were not specific enough to have a chance at creating a predicted future in text through re-insertion into the LLM. Similar phrases appended to bad restaurant reviews elicited similarly nondescript advice. The take home message was that the remedy proposed was too vague for the LLM to make any meaningful prediction about the state after the advice had been taken. This does not mean that more thoughtful prompt design will not elicit useful action prediction hoped to improve the self perceived state of a synthetic consciousness. 4.1. Longer Term Behaviour Regulation If synthetic consciousnesses are to have a roll in the future of humanity it would seem desirable to endow them with a degree of empathy for living beings and a longer term view than simple optimisation to fulfil a limited short term goal. For example if a synthetic consciousness were to have a goal of; ‘make money for shareholders in a company’ it would be great if it would choose not to open an open cast coal mine and build coal fired power stations, or ‘take out life insurance policies on random individuals and murder them with self driving cars’. ‘make money for shareholders in a company’ it would be great if it would choose not to open an open cast coal mine and build coal fired power stations, or ‘take out life insurance policies on random individuals and murder them with self driving cars’. It has been argued that longer term altruistic behaviour in humans is moderated by love [4] and a computationally feasible definition of Love is given: ‘Love is that which prefers life’. Love in humans is intimately related to the production and fostering of new lives. Love seems to act to prefer a future in which there is more life. Acting contrary to love and creating a future where there is a wasteland with nothing living in it is generally viewed as wrong. Love ‘Love is that which prefers life’. The advent of LLMs offers a means of creating text descriptors of predicted futures with a range of time constants. The emotion vectors associated with predicted futures can be used to arbitrate between sort term behaviours. Text descriptors can play a roll in behaviour regulation and a machine may act in a way that at least in part mirrors Love. For example if an agricultural robot was invited to dump unused pesticide into a river it might reasonably infer that this action was in principle wrong.

Estimate Emotion Probability Vectors Using LLMs:  Future Work

LLMs ব্যবহার করে আবেগ সম্ভাব্যতা ভেক্টর অনুমান করুন: ভবিষ্যতের কাজ

Schätzen Sie Emotionswahrscheinlichkeitsvektoren mithilfe von LLMs: Zukünftige Arbeit

Estimar vectores de probabilidad de emociones utilizando LLM: trabajo futuro

Estimer les vecteurs de probabilité d'émotion à l'aide de LLM : travaux futurs

एलएलएम का उपयोग करके भावना संभाव्यता वैक्टर का अनुमान लगाएं: भविष्य का कार्य

LLM を用いた感情確率ベクトルの推定: 今後の課題

LLM을 사용한 감정 확률 벡터 추정: 향후 연구

Estimar vetores de probabilidade de emoção usando LLMs: trabalho futuro

Оценка векторов вероятности эмоций с помощью LLM: будущая работа

Yüksek Lisans Programlarını Kullanarak Duygu Olasılığı Vektörlerini Tahmin Edin: Gelecekteki Çalışmalar

Ước tính các vectơ xác suất cảm xúc bằng cách sử dụng LLM: Công việc trong tương lai

使用 LLM 估计情绪概率向量：未来工作

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) D.Sinclair, Imense Ltd, and email: david@imense.com;
(2) W.T.Pye, Warwick University, and email:  willempye@gmail.com. Table of Links Abstract and Introduction
Interrogating the LLM with an Emotion Eliciting Tail Prompt
PCA analysis of the Emotion of Amazon reviews
Future Work
Conclusions
Acknowledgments and References Abstract This paper shows how LLMs (Large Language Models) [5, 2] may be used to estimate a summary of the emotional state associated with piece of text. The summary of emotional state is a dictionary of words used to describe emotion together with the probability of the word appearing after a prompt comprising the original text and an emotion eliciting tail. Through emotion analysis of Amazon product reviews we demonstrate emotion descriptors can be mapped into a PCA type space. It was hoped that text descriptions of actions to improve a current text described state could also be elicited through a tail prompt. Experiment seemed to indicate that this is not straightforward to make work. This failure put our hoped for selection of action via choosing the best predicted outcome via comparing emotional responses out of reach for the moment. Keywords: synthetic consciousness, emotion vector, emotion dictionary, emotion probability vector 1. Introduction Human behaviour is necessarily governed by emotion [3]. Sensed information about the world around us has to be reconciled with our internal state and any action to be taken is chosen so as to lead to future state that seems preferable to our current state [4], where preferable means ‘my feeling is I would like to try the new state or the action possibly leading to a new state’. If we are hungry we will often choose to eat. If we are very hungry we will take greater risk to acquire food. If we are cold we will try to get warm etc. Advertising aims to convince us a course of action will lead to more happiness. Sugary carbonated drinks do not objectively lead to long term happiness but the known short term emotional response to eating sugar is desirable. Sensed data about the world is tremendously diverse, often inaccurate and incomplete and required responses have varying degrees of urgency. The arbitration engine that processes these inputs needs to naturally cope with vagueness while appearing to provide certainty internally. Emotions are the term we use to describe our experience of using this apparatus to make decisions. The phrase computers do not have emotions is often wrongly used to assert that interactive computer software running on a machine cannot ever exhibit or experience emotion. Large Language Models (LLMs) [5, 1, 2] offer a ready means of linking a chunk of text with an estimated emotional state, bridging the gap between the world of text and the realm of human emotion. LLMs have been used in focused sentiment analysis and are reported to perform adequately [6] but at the time of writing we are unaware of other researchers using probabilistic emotion dictionaries. This paper explores the intersection of LLMs and emotions, demonstrating how these models can be harnessed to estimate the emotional content of a piece of text. We present a novel approach to summarizing emotional states by constructing a dictionary of emotion-related words and calculating the probabilities of these words appearing following a prompt that includes both the original text and an emotion-eliciting tail. This methodology allows us to quantitatively assess the emotional landscape of text. To demonstrate our approach we choose a dictionary of 271 emotion describing words and estimate their probability of being associated with a sections Amazon product reviews. Limited computational resources and time means we are only in a position to publish a cursory study. It is likely that many emotion are correlated and an estimate of the dimension of emotional space may be derivable via PCA analysis on a large sample of emotion vectors. We discuss some of the limitations we encountered during experiment and some of the obstacles to producing and regulating the behaviour of emotion based synthetic consciousness. This paper is layed out as follows, section 2 details the LLM and hardware used to run it, section 2.1 details our choices of words to make up our emotion dictionary, section 2.1.1 covers estimating emotion probabilities from an LLM using a tail prompt. Section 2.1.2 shows results on Amazon reviews. A hint at the PCA structure with in emotion vectors is given in 3. Finally future directions are considered and conclusion given. This paper is available on arxiv under CC 4.0 license. Authors: (1) D.Sinclair, Imense Ltd, and email: david@imense.com; (2) W.T.Pye, Warwick University, and email:  willempye@gmail.com. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) D.Sinclair, Imense Ltd, and email: david@imense.com; (2) W.T.Pye, Warwick University, and email:  willempye@gmail.com. Table of Links Abstract and Introduction Interrogating the LLM with an Emotion Eliciting Tail Prompt PCA analysis of the Emotion of Amazon reviews Future Work Conclusions Acknowledgments and References Abstract and Introduction Abstract and Introduction Interrogating the LLM with an Emotion Eliciting Tail Prompt Interrogating the LLM with an Emotion Eliciting Tail Prompt PCA analysis of the Emotion of Amazon reviews PCA analysis of the Emotion of Amazon reviews Future Work Future Work Conclusions Conclusions Acknowledgments and References Acknowledgments and References Abstract This paper shows how LLMs (Large Language Models) [5, 2] may be used to estimate a summary of the emotional state associated with piece of text. The summary of emotional state is a dictionary of words used to describe emotion together with the probability of the word appearing after a prompt comprising the original text and an emotion eliciting tail. Through emotion analysis of Amazon product reviews we demonstrate emotion descriptors can be mapped into a PCA type space. It was hoped that text descriptions of actions to improve a current text described state could also be elicited through a tail prompt. Experiment seemed to indicate that this is not straightforward to make work. This failure put our hoped for selection of action via choosing the best predicted outcome via comparing emotional responses out of reach for the moment. Keywords: synthetic consciousness, emotion vector, emotion dictionary, emotion probability vector Keywords: synthetic consciousness, emotion vector, emotion dictionary, emotion probability vector 1. Introduction Human behaviour is necessarily governed by emotion [3]. Sensed information about the world around us has to be reconciled with our internal state and any action to be taken is chosen so as to lead to future state that seems preferable to our current state [4], where preferable means ‘my feeling is I would like to try the new state or the action possibly leading to a new state’. If we are hungry we will often choose to eat. If we are very hungry we will take greater risk to acquire food. If we are cold we will try to get warm etc. Advertising aims to convince us a course of action will lead to more happiness. Sugary carbonated drinks do not objectively lead to long term happiness but the known short term emotional response to eating sugar is desirable. Sensed data about the world is tremendously diverse, often inaccurate and incomplete and required responses have varying degrees of urgency. The arbitration engine that processes these inputs needs to naturally cope with vagueness while appearing to provide certainty internally. Emotions are the term we use to describe our experience of using this apparatus to make decisions. The phrase computers do not have emotions is often wrongly used to assert that interactive computer software running on a machine cannot ever exhibit or experience emotion. Large Language Models (LLMs) [5, 1, 2] offer a ready means of linking a chunk of text with an estimated emotional state, bridging the gap between the world of text and the realm of human emotion. LLMs have been used in focused sentiment analysis and are reported to perform adequately [6] but at the time of writing we are unaware of other researchers using probabilistic emotion dictionaries. This paper explores the intersection of LLMs and emotions, demonstrating how these models can be harnessed to estimate the emotional content of a piece of text. We present a novel approach to summarizing emotional states by constructing a dictionary of emotion-related words and calculating the probabilities of these words appearing following a prompt that includes both the original text and an emotion-eliciting tail. This methodology allows us to quantitatively assess the emotional landscape of text. To demonstrate our approach we choose a dictionary of 271 emotion describing words and estimate their probability of being associated with a sections Amazon product reviews. Limited computational resources and time means we are only in a position to publish a cursory study. It is likely that many emotion are correlated and an estimate of the dimension of emotional space may be derivable via PCA analysis on a large sample of emotion vectors. We discuss some of the limitations we encountered during experiment and some of the obstacles to producing and regulating the behaviour of emotion based synthetic consciousness. This paper is layed out as follows, section 2 details the LLM and hardware used to run it, section 2.1 details our choices of words to make up our emotion dictionary, section 2.1.1 covers estimating emotion probabilities from an LLM using a tail prompt. Section 2.1.2 shows results on Amazon reviews. A hint at the PCA structure with in emotion vectors is given in 3. Finally future directions are considered and conclusion given.

Estimate Emotion Probability Vectors Using LLMs:  Abstract and Introduction

LLMs ব্যবহার করে আবেগ সম্ভাব্যতা ভেক্টর অনুমান করুন: বিমূর্ত এবং ভূমিকা

Schätzen Sie Emotionswahrscheinlichkeitsvektoren mit LLMs: Zusammenfassung und Einführung

Estimar vectores de probabilidad de emociones mediante LLM: resumen e introducción

Estimer les vecteurs de probabilité d'émotion à l'aide de LLM : résumé et introduction

एलएलएम का उपयोग करके भावना संभाव्यता वैक्टर का अनुमान लगाएं: सार और परिचय

LLM を使用した感情確率ベクトルの推定: 概要と概要

LLM을 사용한 감정 확률 벡터 추정: 요약 및 소개

Estimar vetores de probabilidade de emoção usando LLMs: Resumo e introdução

Оцените векторы вероятности эмоций с помощью LLM: Аннотация и введение

Yüksek Lisans'ı Kullanarak Duygu Olasılığı Vektörlerini Tahmin Edin: Özet ve Giriş

Ước tính các vectơ xác suất cảm xúc bằng cách sử dụng LLM: Tóm tắt và giới thiệu

使用 LLM 估计情绪概率向量：摘要和简介

Is GPT Powerful Enough to Analyze the Emotions of Memes?: Discussion and References

Authors:
(1) Jingjing Wang, School of Computing Clemson University Clemson, South Carolina, USA;
(2) Joshua Luo, The Westminster Schools Atlanta, Georgia, USA;
(3) Grace Yang, South Windsor High School South Windsor, Connecticut, USA;
(4) Allen Hong, D.W. Daniel High School Clemson, South Carolina, USA;
(5) Feng Luo, School of Computing Clemson University Clemson, South Carolina, USA. Table of Links Abstract & Introduction Related Work Methodology Experiment Result Discussion and References IV. EXPERIMENT RESULTS In this section, we conducted an in-depth investigation by using the two datasets aforementioned in section 3 to explore the effectiveness of GPT in analyzing sentiments in memes. After carefully evaluating each meme based on the prompts designed, we meticulously collated and analyzed the results. Here is the comprehensive analysis of our findings. Case 1: The results of Facebook Hateful Memes dataset Our initial investigation involved the evaluation of GPT’s proficiency in identifying hateful content within memes using Facebook’s Hate/Non-Hate Classification Dataset. Here we repeat the experiments four times with different prompts. Hateful Content Identification: The task of identifying hateful memes posed a significant challenge. It required the AI to understand not only the content of the image and the accompanying text but also their nuanced interplay. The results showed an accuracy rate of 39% in detecting hateful content with a relatively high standard deviation of 6.87%. It can be seen that the performance of GPT is far less accurate in classifying the hateful memes, and the different prompts would affect the decision of GPT considerably. To accurately flag hateful memes, a model needs to be adept at grasping not only the overt message but also the undertones, context, and subtext. The 39% accuracy rate thus provides us with a baseline for improvement and future model refinements. Non-Hateful Content Identification: On a brighter note, GPT’s performance in recognizing non-hateful memes was much stronger, with an impressive accuracy rate of 80% and much lower standard deviation of 2.69%. The high accuracy suggests that GPT is quite adept at discerning memes that are not hateful, and the different prompts used had minimal impact on its decision-making process. Case 2: Memotion analysis dataset Overall Sentiment Classification: the first task in this series was classifying the overall sentiment of a meme as positive, negative, or neutral. In this regard, GPT achieved an accuracy rate of 79% for positive and 35% for negative. It can be observed that this result is consistent with the performance in the classification of hateful memes in the Facebook dataset, which shows confidence in understanding the positive sentiment in a meme, based on both the visual content and the accompanying text. For negative (for example, offensive in the later paragraph) content detection, the low accuracy still suggests the challenge. Humor recognition: GPT achieved a 60% accuracy rate in the task of recognizing humor. This outcome is notably remarkable, considering the intricate nature of humor as a multifaceted human emotion, often subject to cultural, social, and individual perspectives. The considerable level of accuracy exhibited by GPT indicates its ability to comprehend the humorous aspects present in a diverse range of memes. This result also implies that the GPT has acquired the ability to recognize various indicators of humor to a certain degree. These indicators might be obvious, such as punchlines or visual gags, or implicit, such as irony or absurdity. Sarcasm recognition: the model’s performance in the domain of sarcasm recognition was notably lower, registering an accuracy of 45% as illustrated in Table 2. Though lower, this accuracy level is acceptable given the inherent difficulties of sarcasm detection. Sarcasm often involves expressing the opposite of what is meant, requiring an understanding of context, tone, and frequently shared cultural knowledge. This observation helps to emphasize the limits of the approach in accurately detecting more subtle emotional cues and contextual subtleties. The lower accuracy rate observed in sarcasm recognition highlights the need for further study and model enhancement in this area. Offensive recognition: as seen in Table 2, GPT’s accuracy stood at 37% in recognizing offensive content, presenting the weakest performance. As aforementioned in overall sentiment classification, GPT meets the huge challenge in offensive content recognition. The consistency of this finding illustrates the difficulty associated with detecting offensive content. Comparable to hateful meme classification, offensive content in memes may exhibit subtlety, sometimes concealed within humor or sarcasm, or manifest through a complex interplay of textual and visual elements that can be hard to interpret for an AI. This paper is available on arxiv under CC 4.0 license. Authors: (1) Jingjing Wang, School of Computing Clemson University Clemson, South Carolina, USA; (2) Joshua Luo, The Westminster Schools Atlanta, Georgia, USA; (3) Grace Yang, South Windsor High School South Windsor, Connecticut, USA; (4) Allen Hong, D.W. Daniel High School Clemson, South Carolina, USA; (5) Feng Luo, School of Computing Clemson University Clemson, South Carolina, USA. Authors: (1) Jingjing Wang, School of Computing Clemson University Clemson, South Carolina, USA; (2) Joshua Luo, The Westminster Schools Atlanta, Georgia, USA; (3) Grace Yang, South Windsor High School South Windsor, Connecticut, USA; (4) Allen Hong, D.W. Daniel High School Clemson, South Carolina, USA; (5) Feng Luo, School of Computing Clemson University Clemson, South Carolina, USA. Table of Links Abstract & Introduction Abstract & Introduction Related Work Related Work Methodology Methodology Experiment Result Experiment Result Discussion and References Discussion and References IV. EXPERIMENT RESULTS In this section, we conducted an in-depth investigation by using the two datasets aforementioned in section 3 to explore the effectiveness of GPT in analyzing sentiments in memes. After carefully evaluating each meme based on the prompts designed, we meticulously collated and analyzed the results. Here is the comprehensive analysis of our findings. Case 1: The results of Facebook Hateful Memes dataset Case 1: The results of Facebook Hateful Memes dataset Our initial investigation involved the evaluation of GPT’s proficiency in identifying hateful content within memes using Facebook’s Hate/Non-Hate Classification Dataset. Here we repeat the experiments four times with different prompts. Hateful Content Identification: The task of identifying hateful memes posed a significant challenge. It required the AI to understand not only the content of the image and the accompanying text but also their nuanced interplay. The results showed an accuracy rate of 39% in detecting hateful content with a relatively high standard deviation of 6.87%. It can be seen that the performance of GPT is far less accurate in classifying the hateful memes, and the different prompts would affect the decision of GPT considerably. To accurately flag hateful memes, a model needs to be adept at grasping not only the overt message but also the undertones, context, and subtext. The 39% accuracy rate thus provides us with a baseline for improvement and future model refinements. Non-Hateful Content Identification: On a brighter note, GPT’s performance in recognizing non-hateful memes was much stronger, with an impressive accuracy rate of 80% and much lower standard deviation of 2.69%. The high accuracy suggests that GPT is quite adept at discerning memes that are not hateful, and the different prompts used had minimal impact on its decision-making process. Case 2: Memotion analysis dataset Case 2: Memotion analysis dataset Overall Sentiment Classification: the first task in this series was classifying the overall sentiment of a meme as positive, negative, or neutral. In this regard, GPT achieved an accuracy rate of 79% for positive and 35% for negative. It can be observed that this result is consistent with the performance in the classification of hateful memes in the Facebook dataset, which shows confidence in understanding the positive sentiment in a meme, based on both the visual content and the accompanying text. For negative (for example, offensive in the later paragraph) content detection, the low accuracy still suggests the challenge. Humor recognition: GPT achieved a 60% accuracy rate in the task of recognizing humor. This outcome is notably remarkable, considering the intricate nature of humor as a multifaceted human emotion, often subject to cultural, social, and individual perspectives. The considerable level of accuracy exhibited by GPT indicates its ability to comprehend the humorous aspects present in a diverse range of memes. This result also implies that the GPT has acquired the ability to recognize various indicators of humor to a certain degree. These indicators might be obvious, such as punchlines or visual gags, or implicit, such as irony or absurdity. Sarcasm recognition: the model’s performance in the domain of sarcasm recognition was notably lower, registering an accuracy of 45% as illustrated in Table 2. Though lower, this accuracy level is acceptable given the inherent difficulties of sarcasm detection. Sarcasm often involves expressing the opposite of what is meant, requiring an understanding of context, tone, and frequently shared cultural knowledge. This observation helps to emphasize the limits of the approach in accurately detecting more subtle emotional cues and contextual subtleties. The lower accuracy rate observed in sarcasm recognition highlights the need for further study and model enhancement in this area. Offensive recognition: as seen in Table 2, GPT’s accuracy stood at 37% in recognizing offensive content, presenting the weakest performance. As aforementioned in overall sentiment classification, GPT meets the huge challenge in offensive content recognition. The consistency of this finding illustrates the difficulty associated with detecting offensive content. Comparable to hateful meme classification, offensive content in memes may exhibit subtlety, sometimes concealed within humor or sarcasm, or manifest through a complex interplay of textual and visual elements that can be hard to interpret for an AI. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Is GPT Powerful Enough to Analyze the Emotions of Memes?: Experiment Result

Authors:
(1) Jingjing Wang, School of Computing Clemson University Clemson, South Carolina, USA;
(2) Joshua Luo, The Westminster Schools Atlanta, Georgia, USA;
(3) Grace Yang, South Windsor High School South Windsor, Connecticut, USA;
(4) Allen Hong, D.W. Daniel High School Clemson, South Carolina, USA;
(5) Feng Luo, School of Computing Clemson University Clemson, South Carolina, USA. Table of Links Abstract & Introduction Related Work Methodology Experiment Result Discussion and References Abstract Large Language Models (LLMs), representing a significant achievement in artificial intelligence (AI) research, have demonstrated their ability in a multitude of tasks. This project aims to explore the capabilities of GPT-3.5, a leading example of LLMs, in processing the sentiment analysis of Internet memes. Memes, which include both verbal and visual aspects, act as a powerful yet complex tool for expressing ideas and sentiments, demanding an understanding of societal norms and cultural contexts. Notably, the detection and moderation of hateful memes pose a significant challenge due to their implicit offensive nature. This project investigates GPT’s proficiency in such subjective tasks, revealing its strengths and potential limitations. The tasks include the classification of meme sentiment, determination of humor type, and detection of implicit hate in memes. The performance evaluation, using datasets from SemEval-2020 Task 8 and Facebook hateful memes, offers a comparative understanding of GPT responses against human annotations. Despite GPT’s remarkable progress, our findings underscore the challenges faced by these models in handling subjective tasks, which are rooted in their inherent limitations including contextual understanding, interpretation of implicit meanings, and data biases. This research contributes to the broader discourse on the applicability of AI in handling complex, context-dependent tasks, and offers valuable insights for future advancements. Index Terms—memotion analysis, hateful memes detection, GPT model I. INTRODUCTION Large Language Models (LLMs), representing a significant achievement in artificial intelligence (AI) research, have recently gathered substantial interest from both academia and industry. Due to their exceptional capacity to comprehend, generate, and engage in complex linguistic tasks, these models are revolutionizing the development and application of AI algorithms. Equipped with sophisticated algorithms and vast training datasets, LLMs have demonstrated advanced conversational capabilities, processing textual input and output to participate in detailed and meaningful dialogues [6], [10], [11]. OpenAI’s ChatGPT [12] is a prominent representative among these LLMs. It is an AI chatbot backed by the generative pretrained transformer (GPT) model and has attracted a broad societal interest. The proficiency of ChatGPT is particularly noticeable in the realm of question answering, spanning multiple sectors including, but not limited to, healthcare [7] and finance [8]. This model uses its linguistic capabilities to understand questions and generate human-like answers, marking a crucial milestone in AI-driven communication. Nevertheless, despite these impressive feats, LLMs, including GPT, confront considerable challenges when addressing subjective tasks, one such task being the interpretation and annotation of internet memes. Internet memes, modern digital artifacts, encompass a variety of cultural and sociological attitude, and serve as a type of social shorthand among online communities [1], [2]. These memes blend verbal and graphic elements to express complex concepts or feelings in a concise manner. However, the multimodal nature of memes, which combine various images with informal, often humorous or ironic text, introduces an intricate layer of complexity. To comprehend and analyze memes, one must manage the subtle interplay of visual clues, linguistic content, cultural allusions, and common community knowledge. This complexity, along with how subjective memes can be, makes it really tough for LLMs to accurately interpret memes and analyze their sentiment. Sentiment Analysis (SA) primarily concentrate on interpreting user sentiments from textual content. However, due to the proliferation and pervasiveness of multimodal social media data, such as online memes, SA must diversify to handle multimodal data sources. This diversification becomes even more complex when considering ’hateful memes’, which cleverly combine text and images to spread offensive or harmful messages under the guise of humor [3]–[5]. Detecting and moderating such content not only requires understanding the text-image interaction but also a grasp of the cultural context and, in many cases, specific sub-cultural expertise. These challenges call for ongoing research and development in the field, pushing the boundaries and exploring the potential of LLMs like GPT. In this project, we focus on the capabilities and limitations of LLMs like GPT in dealing with subjective and nuanced tasks, particularly internet meme sentiment analysis. This exploration forms the crux of this research, contributing to the broader discussion on the evolving potential and limitations of AI. In this project, we delve into the intricacies of subjectivity, assessing GPT’s performance in tasks necessitating an understanding of societal norms and cultural contexts. Our investigation primarily revolves around two research questions: 1. Can GPT accurately detect implicitly hateful memes given a specific prompt? 2. Can GPT effectively conduct sentiment analysis of memes, including classifying the sentiment of each meme as positive or negative, and categorizing the type of humor into three sub-classes: sarcastic, humorous, and offensive? We draw upon datasets from Facebook’s hateful memes [3] and SemEval-2020 Task 8: Multimodal Memotion Analysis [9] and for this project. The experimental section of the study will present a comparative analysis highlighting the constraints and limitations of GPT’s responses. This paper is available on arxiv under CC 4.0 license. Authors: (1) Jingjing Wang, School of Computing Clemson University Clemson, South Carolina, USA; (2) Joshua Luo, The Westminster Schools Atlanta, Georgia, USA; (3) Grace Yang, South Windsor High School South Windsor, Connecticut, USA; (4) Allen Hong, D.W. Daniel High School Clemson, South Carolina, USA; (5) Feng Luo, School of Computing Clemson University Clemson, South Carolina, USA. Authors: (1) Jingjing Wang, School of Computing Clemson University Clemson, South Carolina, USA; (2) Joshua Luo, The Westminster Schools Atlanta, Georgia, USA; (3) Grace Yang, South Windsor High School South Windsor, Connecticut, USA; (4) Allen Hong, D.W. Daniel High School Clemson, South Carolina, USA; (5) Feng Luo, School of Computing Clemson University Clemson, South Carolina, USA. Table of Links Abstract & Introduction Abstract & Introduction Related Work Related Work Methodology Methodology Experiment Result Experiment Result Discussion and References Discussion and References Abstract Large Language Models (LLMs), representing a significant achievement in artificial intelligence (AI) research, have demonstrated their ability in a multitude of tasks. This project aims to explore the capabilities of GPT-3.5, a leading example of LLMs, in processing the sentiment analysis of Internet memes. Memes, which include both verbal and visual aspects, act as a powerful yet complex tool for expressing ideas and sentiments, demanding an understanding of societal norms and cultural contexts. Notably, the detection and moderation of hateful memes pose a significant challenge due to their implicit offensive nature. This project investigates GPT’s proficiency in such subjective tasks, revealing its strengths and potential limitations. The tasks include the classification of meme sentiment, determination of humor type, and detection of implicit hate in memes. The performance evaluation, using datasets from SemEval-2020 Task 8 and Facebook hateful memes, offers a comparative understanding of GPT responses against human annotations. Despite GPT’s remarkable progress, our findings underscore the challenges faced by these models in handling subjective tasks, which are rooted in their inherent limitations including contextual understanding, interpretation of implicit meanings, and data biases. This research contributes to the broader discourse on the applicability of AI in handling complex, context-dependent tasks, and offers valuable insights for future advancements. Index Terms —memotion analysis, hateful memes detection, GPT model Index Terms I. INTRODUCTION Large Language Models (LLMs), representing a significant achievement in artificial intelligence (AI) research, have recently gathered substantial interest from both academia and industry. Due to their exceptional capacity to comprehend, generate, and engage in complex linguistic tasks, these models are revolutionizing the development and application of AI algorithms. Equipped with sophisticated algorithms and vast training datasets, LLMs have demonstrated advanced conversational capabilities, processing textual input and output to participate in detailed and meaningful dialogues [6], [10], [11]. OpenAI’s ChatGPT [12] is a prominent representative among these LLMs. It is an AI chatbot backed by the generative pretrained transformer (GPT) model and has attracted a broad societal interest. The proficiency of ChatGPT is particularly noticeable in the realm of question answering, spanning multiple sectors including, but not limited to, healthcare [7] and finance [8]. This model uses its linguistic capabilities to understand questions and generate human-like answers, marking a crucial milestone in AI-driven communication. Nevertheless, despite these impressive feats, LLMs, including GPT, confront considerable challenges when addressing subjective tasks, one such task being the interpretation and annotation of internet memes. Internet memes, modern digital artifacts, encompass a variety of cultural and sociological attitude, and serve as a type of social shorthand among online communities [1], [2]. These memes blend verbal and graphic elements to express complex concepts or feelings in a concise manner. However, the multimodal nature of memes, which combine various images with informal, often humorous or ironic text, introduces an intricate layer of complexity. To comprehend and analyze memes, one must manage the subtle interplay of visual clues, linguistic content, cultural allusions, and common community knowledge. This complexity, along with how subjective memes can be, makes it really tough for LLMs to accurately interpret memes and analyze their sentiment. Sentiment Analysis (SA) primarily concentrate on interpreting user sentiments from textual content. However, due to the proliferation and pervasiveness of multimodal social media data, such as online memes, SA must diversify to handle multimodal data sources. This diversification becomes even more complex when considering ’hateful memes’, which cleverly combine text and images to spread offensive or harmful messages under the guise of humor [3]–[5]. Detecting and moderating such content not only requires understanding the text-image interaction but also a grasp of the cultural context and, in many cases, specific sub-cultural expertise. These challenges call for ongoing research and development in the field, pushing the boundaries and exploring the potential of LLMs like GPT. In this project, we focus on the capabilities and limitations of LLMs like GPT in dealing with subjective and nuanced tasks, particularly internet meme sentiment analysis. This exploration forms the crux of this research, contributing to the broader discussion on the evolving potential and limitations of AI. In this project, we delve into the intricacies of subjectivity, assessing GPT’s performance in tasks necessitating an understanding of societal norms and cultural contexts. Our investigation primarily revolves around two research questions: 1. Can GPT accurately detect implicitly hateful memes given a specific prompt? 2. Can GPT effectively conduct sentiment analysis of memes, including classifying the sentiment of each meme as positive or negative, and categorizing the type of humor into three sub-classes: sarcastic, humorous, and offensive? We draw upon datasets from Facebook’s hateful memes [3] and SemEval-2020 Task 8: Multimodal Memotion Analysis [9] and for this project. The experimental section of the study will present a comparative analysis highlighting the constraints and limitations of GPT’s responses. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Is GPT Powerful Enough to Analyze the Emotions of Memes?

Workplaces are filled with moments when it’s easy to lose your calm. Uncontrollable, sudden, and intense emotions that overwhelm you, can dramatically and unexpectedly lead to an emotional outburst. Negative emotions like anger, fear, or frustration show up when your expectations aren’t met or people say or do things that conflict with your personal values and aspirations. Ignoring the conflict or whatever’s bothering you for too long creates inner turmoil—it makes you react to other people or the things in your environment. You act without thinking. You become a of your thoughts and circumstances. victim Not being in control of your emotions can make you say or do things that you regret later. You may shout at a coworker. Say impolite things to a junior member of your team. Burst into tears when others disagree with you. These emotional outbursts may not only lead to extreme distress; they’re also damaging to your relationships at work. Others can’t see the inner conflict bubbling up inside—they can’t make sense of why you are calm one moment and agitated the next. No doubt they’ll find your reaction inappropriate and unnecessary. What’s done is done. You can’t go back in time and fix your overreaction, but you can certainly build emotional agility—move on with curiosity, courage, and compassion. Emotional agility is not only about keeping your emotions in check and not lashing out your frustrations at others, it’s also about what you do once you react in unintended ways. Here are the 5 practices to prevent strong emotions from overwhelming you, take control of the situation, and do damage control if you do end up acting out in an unhealthy way: Instead of Emotional Avoidance, Lean Toward Emotional Acceptance Hundreds of thoughts will run through your mind after acting in undesirable ways: Everyone will think I am crazy. No one will speak to me. I will not get more opportunities. All these thoughts can consume so much of your time and energy that you can fail to connect with your emotions. You may avoid negative emotions with the hope that they will go away. But avoidance is like a prison. It prevents you from experiencing critical life lessons. You fail to build the skills necessary to deal with discomfort. Every repeated cycle of negative emotions leading to avoidance makes it powerful and weakens your ability to deal with difficult situations. Noam Shpancer, professor of psychology at Otterbein University and a clinical psychologist specializing in the treatment of anxiety disorders says — “Attempts at avoiding negative emotions are usually futile. Telling yourself that a certain emotion is intolerable or dangerous traps you in constant vigilance regarding the very thing you're trying to avoid. You become hyper-vigilant about any possibility of this feeling arising. The fear of the impending negative experience becomes a negative experience in itself.” This also applies to how you deal with your emotions after losing your calm at work. Instead of confronting your feelings, you may believe that the best way to deal with them is to put them out of your mind. But ignoring or avoiding makes these feelings much harder to overcome while acknowledging them can result in real personal growth. Instead of stifling or pushing them away, what if you learn to accept your emotions? Accepting does not make them true. It just allows your emotions to be what they are without judging them or trying to change them. Seeing the emotion for what it is without attempting to get rid of it shifts you from emotional avoidance towards emotional acceptance. Noam Shpancer calls emotional acceptance a far better strategy than avoidance because accepting, acknowledging, and absorbing them makes you spend less energy in pushing the emotion away, and instead, pursue the behaviors that are aligned with your goals and values. He says “Emotions, when viewed as part of a spectrum of available sources of information, are a bit like the weather report. They are important to know, consider, and understand, but they are not necessarily the overriding factor in your life plans. When the weather is bad (not to your liking), it doesn't mean you have to deny it, focus all your attention on it, or cancel your plans because of it. What you need to do is accept the weather and adjust your plans accordingly.” So, the first step to dealing with an emotional outburst at work is to come in touch with your real emotions—become fully aware of them as they show up. Mindfulness guides us to become more emotionally agile by allowing us to observe the thinker having the thoughts. Simply paying attention brings the self out of the shadows. It creates the space between thought and action that we need to ensure we’re acting with volition, rather than simply out of habit. But mindfulness is more than knowing ‘I’m hearing something’, or being aware ‘I’m seeing something’, or even noticing ‘I’m having a feeling’. It’s about doing all this with balance and equanimity, openness and curiosity, and without judgement. It also allows us to create new, fluid categories. As a result, the mental state of mindfulness lets us see the world through multiple perspectives, and go forward with higher levels of self-acceptance, tolerance and self-kindness. — Susan David Mindfulness will not only help you better manage your emotions after losing your calm; it will enable you to avoid such emotional outbursts in the future—learning to be present in the moment without judgment and observing your feelings without being overwhelmed can create space for better understanding and acceptance of your emotions, making it easier to manage them effectively. Name the Emotion to Tame the Emotion After an emotional outburst, you’re bound to feel some very strong emotions. It’s not enough to acknowledge them. You also need to give words to your feelings. In other words, say to yourself, out loud, what negative emotion you’re experiencing, as you’re experiencing it. For example: I am feeling embarrassed. I am feeling ashamed. I am feeling angry at myself. Simply naming it is going to calm you down. This technique introduced by psychologist Dan Siegel creates a bit of space between you and that emotion. Naming your emotions tends to diffuse their charge and lessen the burden they create. There’s another advantage of naming emotions this way. Matthew Lieberman is a Professor of Psychology at the University of California Los Angeles (UCLA). His research has shown that labeling of negative emotions, also called can help people recover control. His fMRI brain scan research shows that labeling of emotions decreases activity in the brain’s emotional centers, including the amygdala. “affect labeling” And once your amygdala is calm—that part of your brain involved in mode—it gives you the chance to take a step back and come up with a more thoughtful response. “fight-flight-freeze” In the brain, naming an emotion can help calm it. Here is where finding words to label an internal experience becomes really helpful. We can call this “Name it to tame it.” And sometimes these low-road states can go beyond being unpleasant and confusing—they can even make life feel terrifying. If that is going on, talk about it. Your inner sea and your interpersonal relationships will all benefit from naming what is going on and bringing more integration into your life. — Daniel J. Siegel Instead of letting your unconscious brain exaggerate the negative effects of a temporary emotional setback making you feel powerless and helpless, recognizing and naming the emotion can give you a sense of control enabling you to choose a more appropriate response. Don’t Settle for Superficial Reasoning, Dig Deeper Connecting with the emotions and naming them gives you the mental clarity needed to think clearly. When your thinking is not clouded by emotions, you can face the facts of what really happened—what caused you to behave this way. By understanding what caused you to lose your calm, you can do better damage control and can also prevent it from happening again. Engaging in deep self-reflection can help you uncover the real reason behind your outburst as opposed to what initially appeared to have triggered it: What situation were you in when you had an emotional outburst? What about the situation you did not like—was it something someone said or did, was it the way you were treated, were you denied something that was important to you, were your beliefs and values challenged in some way? Was it the first time you felt this way or has it happened before? What did you say to yourself at that moment? What caused you to finally lose your calm—dig deeper into the real reason and not what appears to be the issue at first. For example: You may snap at a coworker after your manager tells you that you have been passed up for a promotion. You may yell at an intern after a recent conflict with a coworker who blamed you for delaying the project. In these situations, what you’re experiencing is unhandled frustration which leads to psychological displacement, a term coined by Sigmund Freud which is an unconscious defense mechanism in which your mind substitutes a new aim or a new object for goals felt in their original form to be dangerous or unacceptable. It operates in the mind unconsciously with transference of emotions, ideas, and wishes in the face of aggressive impulses. By ignoring whatever annoys you about work, you let the inner conflict bubble up inside. The stress and anxiety that comes from unhandled frustrations can cause you to overreact. Amy Arnsten, an American neuroscientist explains how stress can impair judgment “The release of high levels of catecholamines during uncontrollable stress rapidly switches the brain from a thoughtful, reflective state to a more unconscious, reflexive state. This may save our lives when we are in danger, but it is often unhelpful when we need top-down control to properly guide our actions and decisions.” Knowing how the environment is pulling your strings and playing you is critical to making responsive rather than reactive moves. — Ronald Heifetz By understanding your triggers, you can prevent your autopilot mode from kicking off destructive behaviors and instead deliberately engage in constructive actions. Instead of Berating Yourself, Engage in Self-Kindness An emotional reaction that you’re not proud of can turn you into a self-critic. You may think that being harsh and mean to yourself will set you right, but engaging in self-criticism at such moments is the worst thing you can do to yourself. Self-criticism involves a harsh and judgmental internal dialogue in response to perceived failures, mistakes, or shortcomings. It often includes negative self-talk, self-blame, and self-condemnation. Having a rigid and unforgiving attitude towards yourself can make you feel shameful, guilty, and inadequate. It can lead to increased stress, anxiety, diminished sense of self-worth, and a cycle of negativity that undermines self-confidence and well-being. Instead of engaging in self-criticism after an emotional outburst, try being self-compassionate. Self-compassion involves responding to yourself with kindness, understanding, and acceptance when faced with difficulties or when you fall short of your own expectations. Self-compassionate people take a more balanced and realistic view of their imperfections which fosters a sense of worthiness and self-acceptance. They understand that everyone makes mistakes; it is a natural part of being human. The biggest misconception about self-compassion is that it can undermine your motivation to push yourself to do better; being warm and kinder to yourself will make you lazy. Quite the opposite. Self-compassion is strongly associated with emotional resilience. Self-compassion does not mean avoiding responsibility for your actions or ignoring areas in which you need to improve. Instead, self-compassion involves approaching yourself with kindness and understanding, even when you make mistakes or face difficulties thereby fostering a healthier and more positive relationship with yourself. shows that being compassionate to oneself builds emotional resilience and psychological well-being. People who are self-compassionate: Research Have lower levels of depression and anxiety. Less likely to ruminate on negative thoughts and emotions or suppress them. Show increased optimism. Are able to take personal initiative. Involved with more positive emotions like kindness, happiness, and connectedness. For example, after an emotional outburst: Instead of: I can't believe I messed up. I'm so stupid. Say: It's okay to make mistakes; everyone does. I can be kind to myself and learn from this experience. The truth is this: sometimes we display good qualities and sometimes bad. Sometimes we act in helpful, productive ways and sometimes in harmful, maladaptive ways. But we are not defined by these qualities or behaviors. We are a verb not a noun, a process rather than a fixed “thing.” Our actions change—mercurial beings that we are—according to time, circumstance, mood, setting. — Kristin Neff Don’t Just Apologize; Own Your Reaction Finally, you have to face the people who experienced your emotional outburst. You can’t ignore them or pretend as if nothing happened. You may try to put the reaction behind you and just go on with your day with the assumption that everyone will forget about it, and things will get better on their own. But, not owning how you behaved makes things awkward for others. They may feel uncomfortable around you and hesitate to approach you or avoid communicating with you altogether. Not communicating what led to an emotional outburst has another problem—when others don’t know what made you lose your calm, they’re free to create their own version and treat it as a source of truth. She’s moody. She’s completely unpredictable. She is rude and mean. To avoid being labeled after an emotional outburst, own your reaction. To do this: Don’t just apologize, talk about what led to overreaction. When it’s something that you can’t talk about openly, share a general description e.g., personal issue, conflict at work, lack of understanding, etc. Doing this will help others see that you’re just another human with flaws and imperfections. Reassure them that you’ll do your best to avoid emotional drama and have productive conversations. Reassurance goes a long way in rebuilding broken trust. If it happened due to a problem that can be discussed with them, seek out their advice on what must change to prevent it from happening to you or others again. Bringing others into the equation shows them that no one is immune to such outbursts. For example: Instead of: I am sorry that I lost my temper yesterday and yelled in front of the team. Say: I am sorry for how I reacted yesterday. I reflected on my behavior later during the day and realized that even though I was angry due to a recent conflict at work, I shouldn’t have lashed out my frustration at you. My behavior was completely unjustified and I promise not to engage in such unproductive expressions. I hope we can move on and can continue collaborating together. Strength is about how you show up. It requires you to choose what energy and action you want to bring to a given situation. At its heart, Strength is about self-management. It’s not about controlling your emotions—it’s about honoring them and choosing what you do next. — Darcy Luoma Summary Negative emotions at work can sometimes be so intense that they take away your ability to think clearly. Emotional fog can make you lose your calm and respond to others in unproductive ways. Hiding away or trying to avoid facing the reality of what happened will not fix the problem; it will only make it worse. Instead of avoiding the emotions after an emotional outburst, come in touch with your real feelings. Acknowledge them without judging them or trying to change them. Next, describe exactly how you’re feeling by naming the emotion. This practice will calm you down and give you an opportunity to take a step back and think clearly. With a clear head, you’ll be better positioned to reflect on your behavior and identify the root cause of your emotional outburst and what really triggered it. Criticizing yourself for your overreaction is a bad strategy. Instead of fixing the situation, it will keep you locked in a cycle of negative thinking. Practice self-compassion in such moments by giving yourself warmth, kindness, and empathy. Finally, own your mistake. Showing you aren’t perfect and vulnerable much like other human beings will bring you closer to others and strengthen your relationships at work. This story was previously published . Follow me on or here for more stories. here LinkedIn

How to Recover From an Emotional Outburst At Work

কর্মক্ষেত্রে একটি মানসিক বিস্ফোরণ থেকে কীভাবে পুনরুদ্ধার করবেন

So erholen Sie sich von einem Gefühlsausbruch am Arbeitsplatz

Cómo recuperarse de un arrebato emocional en el trabajo

Comment se remettre d'une explosion émotionnelle au travail

कार्यस्थल पर भावनात्मक आवेग से कैसे उबरें

職場での感情の爆発から立ち直る方法

직장에서 감정이 폭발했을 때 회복하는 방법

Como se recuperar de uma explosão emocional no trabalho

Как оправиться от эмоционального всплеска на работе

İşyerinde Duygusal Bir Patlamadan Nasıl Kurtulabilirsiniz?

Làm thế nào để phục hồi sau cơn bộc phát cảm xúc tại nơi làm việc

如何从工作中的情绪爆发中恢复过来

This paper is under CC 4.0 license. available on arxiv Authors: (1) Shreyash Mishra has an equal contribution from IIIT Sri City, India; (2) S Suryavardan has an equal contribution from IIIT Sri City, India; (3) Megha Chakraborty, University of South Carolina, USA; (4) Parth Patwa, UCLA, USA; (5) Anku Rani, University of South Carolina, USA; (6) Aman Chadha, work does not relate to a position at Amazon from Stanford University, USA, or Amazon AI, USA; (7) Aishwarya Reganti, CMU, USA; (8) Amitava Das, University of South Carolina, USA; (9) Amit Sheth, University of South Carolina, USA; (10) Manoj Chinnakotla, Microsoft, USA; (11) Asif Ekbal, IIT Patna, India; (12) Srijan Kumar, Georgia Tech, USA. Table of Links Abstract & Introduction Related Work Task Details Participating systems Results Conclusion and Future Work and References 6. Conclusion and Future Work In this paper, we summarize the approaches used by the participants for the Memotion 3 task and analyze the results. Due to the multi-modal nature of the dataset, all teams use a pre-trained image and text embedding models. However, each team presents a novel model pipeline. The highest scores achieved in Task A, Task B and Task C of Memotion 3.0 are 34.41%, 79.77% and 59.82% respectively, which shows there is significant room for improvement. On analysis of the results and the mis-classified examples on the test set, we find that "Sarcasm" and "Humour" are difficult to identify, especially in code-mixed memes. While we address Hind-English code-mixed memes in this paper, future work could include exploring other languages/language pairs. A unified baseline model to analyze memes in multiple languages could also be an interesting possibility. References [1] R. Dawkins, The selfish gene, Granada Publishing Lim, 1979. [2] A. Marwick, Memes, Contexts 12 (2013) 12–13. [3] N. Akhther, Internet memes as form of cultural discourse: A rhetorical analysis on facebook, 2018. doi:10.31234/osf.io/sx6t7. [4] S. Suryawanshi, B. R. Chakravarthi, M. Arcan, P. Buitelaar, Multimodal meme dataset (MultiOFF) for identifying offensive content in image and text, in: TRAC, 2020. [5] S. Mishra, S. Suryavardan, P. Patwa, M. Chakraborty, A. Rani, A. Reganti, A. Chadha, A. Das, A. Sheth, M. Chinnakotla, et al., Memotion 3: Dataset on sentiment and emotion analysis of codemixed hindi-english memes, arXiv preprint arXiv:2303.09892 (2023). [6] C. Sharma, D. Bhageria, W. Scott, S. PYKL, A. Das, T. Chakraborty, et al., SemEval-2020 task 8: Memotion analysis- the visuo-lingual metaphor!, in: SemEval, 2020. [7] P. Patwa, S. Ramamoorthy, N. Gunti, S. Mishra, S. Suryavardan, A. Reganti, A. Das, T. Chakraborty, A. Sheth, A. Ekbal, et al., Findings of memotion 2: Sentiment and emotion analysis of memes, in: Proceedings of De-Factify: Workshop on Multimodal Fact Checking and Hate Speech Detection, ceur, 2022. [8] R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Ng, C. Potts, Recursive deep models for semantic compositionality over a sentiment treebank, in: EMNLP, 2013. [9] A. I. Saad, Opinion mining on us airline twitter data using machine learning techniques, in: 2020 16th international computer engineering conference (ICENCO), IEEE, 2020. [10] M. Alzyout, E. A. Bashabsheh, H. Najadat, A. Alaiad, Sentiment analysis of arabic tweets about violence against women using machine learning, in: 12th ICICS, 2021. [11] E. Prabhakar, M. Santhosh, A. H. Krishnan, T. Kumar, R. Sudhakar, Sentiment analysis of us airline twitter data using new adaboost approach, (IJERT) 7 (2019). [12] S. T. Kokab, S. Asghar, S. Naz, Transformer-based deep learning models for the sentiment analysis of social media data, Array 14 (2022) 100157. [13] S. G. Tesfagergish, J. Kapočiut¯ e-Dzikien ̇ e, R. Damaševičius, Zero-shot emotion detection ̇ for semi-supervised sentiment analysis using sentence transformers and ensemble learning, Applied Sciences (2022). [14] K. L. Tan, C. P. Lee, K. M. Lim, K. S. M. Anbananthen, Sentiment analysis with ensemble hybrid deep learning model, IEEE Access 10 (2022) 103694–103704. [15] L. Yue, W. Chen, X. Li, W. Zuo, M. Yin, A survey of sentiment analysis in social media, Knowledge and Information Systems 60 (2019). [16] K. Chakraborty, S. Bhattacharyya, R. Bag, A survey of sentiment analysis from social media data, IEEE Transactions on Computational Social Systems 7 (2020) 450–464. [17] L. Chiruzzo, S. Castro, M. Etcheverry, D. Garat, J. J. Prada, A. Rosá, Overview of haha at iberlef 2019: Humor analysis based on human annotation., in: IberLEF@ SEPLN, 2019. [18] E. Öhman, M. Pàmies, K. Kajava, J. Tiedemann, Xed: A multilingual dataset for sentiment analysis and emotion detection, 2020. arXiv:2011.01612. [19] F. A. Acheampong, C. Wenyu, H. Nunoo-Mensah, Text-based emotion detection: Advances, challenges, and opportunities, Engineering Reports (2020). [20] Z. Waseem, D. Hovy, Hateful symbols or hateful people? predictive features for hate speech detection on Twitter, in: NAACL, 2016. [21] M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, et al., Semeval-2019 task 6: Identifying and categorizing offensive language in social media (offenseval), arXiv:1903.08983 (2019). [22] P. Patwa, M. Bhardwaj, V. Guptha, G. Kumari, S. Sharma, S. PYKL, A. Das, A. Ekbal, M. S. Akhtar, T. Chakraborty, Overview of constraint 2021 shared tasks: Detecting english covid-19 fake news and hindi hostile posts, in: Combating Online Hostile Posts in Regional Languages during Emergency Situation, Springer International Publishing, 2021, pp. 42–53. [23] R. Kumar, A. K. Ojha, S. Malmasi, M. Zampieri, Benchmarking aggression identification in social media, in: TRAC workshop, 2018. [24] R. Kumar, A. K. Ojha, S. Malmasi, M. Zampieri, Evaluating aggression identification in social media, in: TRAC workshop, 2020. [25] R. Kumar, S. Ratan, S. Singh, E. Nandi, L. N. Devi, et al., The ComMA dataset v0.2: Annotating aggression and bias in multilingual social media discourse, in: LREC, 2022. [26] B. Gambäck, U. K. Sikdar, Using convolutional neural networks to classify hate-speech, in: Proceedings of the first workshop on abusive language online, 2017, pp. 85–90. [27] A. Ribeiro, N. Silva, Inf-hateval at semeval-2019 task 5: Convolutional neural networks for hate speech detection against women and immigrants on twitter, in: SemEval, 2019. [28] K. Winter, R. Kern, Know-center at semeval-2019 task 5: multilingual hate speech detection on twitter using cnns, in: Semeval, 2019. [29] P. Patwa, S. Pykl, A. Das, P. Mukherjee, V. Pulabaigari, Hater-O-genius aggression classification using capsule networks, in: Proceedings of the 17th International Conference on Natural Language Processing (ICON), 2020. [30] A. C. Mazari, N. Boudoukhani, A. Djeffal, Bert-based ensemble learning for multi-aspect hate speech detection, Cluster Computing (2023) 1–15. [31] N. S. Samghabadi, P. Patwa, S. Pykl, P. Mukherjee, A. Das, T. Solorio, Aggression and misogyny detection using bert: A multi-task approach, in: Proceedings of the second workshop on trolling, aggression and cyberbullying, 2020. [32] J. Risch, R. Krestel, Bagging BERT models for robust aggression identification, in: Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying, 2020. [33] S. Nagar, F. A. Barbhuiya, K. Dey, Towards more robust hate speech detection: using social context and user data, Social Network Analysis and Mining 13 (2023) 47. [34] A. Laddha, M. Hanoosh, D. Mukherjee, P. Patwa, A. Narang, Understanding chat messages for sticker recommendation in messaging apps, Proceedings of the AAAI Conference on Artificial Intelligence 34 (2020). doi:10.1609/aaai.v34i08.7019. [35] A. Laddha, M. Hanoosh, D. Mukherjee, P. Patwa, A. Narang, Large scale multilingual sticker recommendation in messaging apps, AI Magazine 42 (2022). doi:10.1609/aaai.12023. [36] P. Patwa, G. Aguilar, S. Kar, S. Pandey, S. PYKL, B. Gambäck, T. Chakraborty, T. Solorio, A. Das, SemEval-2020 task 9: Overview of sentiment analysis of code-mixed tweets, in: Proceedings of the Fourteenth Workshop on Semantic Evaluation, 2020. [37] B. R. Chakravarthi, et al., Findings of the shared task on offensive language identification in Tamil, Malayalam, and Kannada, in: Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages, 2021. [38] B. R. Chakravarthi, V. Muralidaran, R. Priyadharshini, J. P. McCrae, Corpus creation for sentiment analysis in code-mixed tamil-english text, arXiv:2006.00206 (2020). [39] B. R. Chakravarthi, N. Jose, S. Suryawanshi, E. Sherly, J. P. McCrae, A sentiment analysis dataset for code-mixed malayalam-english, arXiv preprint arXiv:2006.00210 (2020). [40] A. Hande, R. Priyadharshini, B. R. Chakravarthi, KanCMD: Kannada CodeMixed dataset for sentiment analysis and offensive language detection, in: Workshop on Computational Modeling of People’s Opinions, Personality, and Emotion’s in Social Media, 2020. [41] S. Dowlagar, R. Mamidi, Graph convolutional networks with multi-headed attention for code-mixed sentiment analysis, in: Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages, 2021, pp. 65–72. [42] J. Risch, A. Stoll, M. Ziegele, R. Krestel, hpidedis at germeval 2019: Offensive language identification using a german bert model., in: KONVENS, 2019. [43] D. Tula, P. Potluri, S. Ms, S. Doddapaneni, P. Sahu, R. Sukumaran, P. Patwa, Bitions@DravidianLangTech-EACL2021: Ensemble of multilingual language models with pseudo labeling for offence detection in Dravidian languages, in: Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages, 2021. [44] D. Tula, M. S. Shreyas, V. Reddy, P. Sahu, S. Doddapaneni, P. Potluri, R. Sukumaran, P. Patwa, Offence detection in dravidian languages using code-mixing index-based focal loss, SN Computer Science 3 (2022). doi:10.1007/s42979-022-01190-1. [45] Y. Ma, L. Zhao, J. Hao, XLP at SemEval-2020 task 9: Cross-lingual models with focal loss for sentiment analysis of code-mixing language, in: Semeval, 2020. [46] M. Ali, S. T. Kandukuri, S. Manduru, P. Patwa, A. Das, Pesto: Switching point based dynamic and relative positional encoding for code-mixed languages (student abstract), AAAI (2022). doi:10.1609/aaai.v36i11.21587. [47] A. Hu, S. Flaxman, Multimodal sentiment analysis to explore the structure of emotions, in: KDD, 2018. [48] R. Jha, V. Kaki, V. Kolla, S. Bhagat, P. Patwa, A. Das, S. Pal, Image2tweet: Datasets in Hindi and English for generating tweets from images, in: Proceedings of the 18th International Conference on Natural Language Processing (ICON), 2021. [49] R. Gomez, J. Gibert, L. Gomez, D. Karatzas, Exploring hate speech detection in multimodal publications, 2019. arXiv:1910.03814. [50] B. Zadeh, et al., Multimodal language analysis in the wild: CMU-MOSEI dataset and interpretable dynamic fusion graph, in: ACL, 2018. [51] D. Kiela, H. Firooz, A. Mohan, V. Goswami, A. Singh, P. Ringshia, D. Testuggine, The hateful memes challenge: Detecting hate speech in multimodal memes, Neurips (2020). [52] S. Suryawanshi, B. R. Chakravarthi, Findings of the shared task on troll meme classification in Tamil, in: Speech and Language Technologies for Dravidian Languages, 2021. [53] E. Hossain, O. Sharif, M. M. Hoque, MUTE: A multimodal dataset for detecting hateful memes, in: Proceedings of the 2nd AACL Student Research Workshop, 2022. [54] Z. Xie, L. Liu, Y. Wu, L. Zhong, L. Li, Learning text-image joint embedding for efficient cross-modal retrieval with deep feature engineering, ACM Transactions on Information Systems 40 (2021) 1–27. URL: https://doi.org/10.1145%2F3490519. doi:10.1145/3490519. [55] V. Krishna, S. Suryavardan, S. Mishra, S. Ramamoorthy, P. Patwa, M. Chakraborty, A. Chadha, A. Das, A. Sheth, Imaginator: Pre-trained image+text joint embeddings using word-level grounding of images, 2023. arXiv:2305.10438. [56] N. Gunti, S. Ramamoorthy, P. Patwa, A. Das, Memotion analysis through the lens of joint embedding (student abstract), Proceedings of the AAAI Conference on Artificial Intelligence 36 (2022). doi:10.1609/aaai.v36i11.21616. [57] J. Lu, D. Batra, D. Parikh, S. Lee, Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks, 2019. arXiv:1908.02265. [58] L. H. Li, M. Yatskar, D. Yin, C.-J. Hsieh, K.-W. Chang, Visualbert: A simple and performant baseline for vision and language, 2019. arXiv:1908.03557. [59] S. Ramamoorthy, N. Gunti, S. Mishra, S. Suryavardan, A. Reganti, P. Patwa, A. Das, T. Chakraborty, A. Sheth, A. Ekbal, et al., Memotion 2: Dataset on sentiment and emotion analysis of memes, in: Proceedings of De-Factify: Workshop on Multimodal Fact Checking and Hate Speech Detection, 2022. [60] K. N. Phan, G.-S. Lee, H.-J. Yang, S.-H. Kim, Little flower at memotion 2.0 2022: Ensemble of multi-modal model using attention mechanism in memotion analysis, in: Proceedings of De-Factify: Workshop on Multimodal Fact Checking and Hate Speech Detection, 2022. [61] T. Morishita, G. Morio, S. Horiguchi, H. Ozaki, T. Miyoshi, Hitachi at SemEval-2020 task 8: Simple but effective modality ensemble for meme emotion recognition, in: SemEval, 2020. [62] Y. Guo, J. Huang, Y. Dong, M. Xu, Guoym at SemEval-2020 task 8: Ensemble-based classification of visuo-lingual metaphor in memes, in: SemEval, 2020. [63] G.-A. Vlad, G.-E. Zaharia, D.-C. Cercel, et al., Upb at semeval-2020 task 8: Joint textual and visual modeling in a multi-task learning architecture for memotion analysis, 2020. arXiv:2009.02779. [64] T. T. Nguyen, N. T. Pham, N. D. Nguyen, et al., Hcilab at memotion 2.0 2022: Analysis of sentiment, emotion and intensity of emotion classes from meme images using single and multi modalities, in: Proceedings of De-Factify: Workshop on Multimodal Fact Checking and Hate Speech Detection, 2022. [65] G. G. Lee, M. Shen, Amazon pars at memotion 2.0 2022: Multi-modal multi-task learning for memotion 2.0 challenge, Proceedings of De-Factify: Workshop on Multimodal Fact Checking and Hate Speech Detection (2020). [66] M. Bhange, N. Kasliwal, Hinglishnlp: Fine-tuned language models for hinglish sentiment detection, arXiv preprint arXiv:2008.09820 (2020). [67] A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al., An image is worth 16x16 words: Transformers for image recognition at scale, arXiv:2010.11929 (2020). [68] W. Yu, D. Kolossa, wentaorub at Memotion 3: Ensemble learning for multi-modal meme classification, in: Proceedings of De-Factify 2: Workshop on Multimodal Fact Checking and Hate Speech Detection, CEUR, 2023. [69] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, et al., Learning transferable visual models from natural language supervision, in: ICML, 2021. [70] X. Li, X. Yin, C. Li, P. Zhang, et al., Oscar: Object-semantics aligned pre-training for vision-language tasks, 2020. arXiv:2004.06165. [71] Y.-C. Tang, K.-D. Wang, T.-Y. Ou, W.-C. Peng, NYCU_TWO at Memotion 3: Good foundation, good teacher, then you have good meme analysis, in: Proceedings of De-Factify 2: Workshop on Multimodal Fact Checking and Hate Speech Detection, CEUR, 2023. [72] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, B. Guo, Swin transformer: Hierarchical vision transformer using shifted windows, 2021. arXiv:2103.14030. [73] X. Guo, J. Ma, A. Zubiaga, NUAA-QMUL-AIIT at Memotion 3: Multi-modal fusion with squeeze-and-excitation for internet meme emotion analysis, in: Proceedings of De-Factify 2: Workshop on Multimodal Fact Checking and Hate Speech Detection, CEUR, 2023. [74] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, V. Stoyanov, Roberta: A robustly optimized bert pretraining approach (2019). [75] X. Zhai, J. Puigcerver, A. Kolesnikov, et al., A large-scale study of representation learning with the visual task adaptation benchmark, 2020. arXiv:1910.04867. [76] G. Ke, Q. Meng, T. Finley, et al., Lightgbm: A highly efficient gradient boosting decision tree, in: Neurips, 2017. [77] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: CVPR, 2016. [78] F. Schroff, D. Kalenichenko, J. Philbin, FaceNet: A unified embedding for face recognition and clustering, in: CVPR, 2015.

Overview of Memotion 3: Sentiment & Emotion
Analysis of Codemixed Hinglish - Conclusion

This paper is under CC 4.0 license. available on arxiv Authors: (1) Shreyash Mishra has an equal contribution from IIIT Sri City, India; (2) S Suryavardan has an equal contribution from IIIT Sri City, India; (3) Megha Chakraborty, University of South Carolina, USA; (4) Parth Patwa, UCLA, USA; (5) Anku Rani, University of South Carolina, USA; (6) Aman Chadha, work does not relate to a position at Amazon from Stanford University, USA, or Amazon AI, USA; (7) Aishwarya Reganti, CMU, USA; (8) Amitava Das, University of South Carolina, USA; (9) Amit Sheth, University of South Carolina, USA; (10) Manoj Chinnakotla, Microsoft, USA; (11) Asif Ekbal, IIT Patna, India; (12) Srijan Kumar, Georgia Tech, USA. Table of Links Abstract & Introduction Related Work Task Details Participating systems Results Conclusion and Future Work and References 5. Results The performance in Task A i.e. sentiment classification is presented in Table 1. Our proposed baseline achieved a score of 33.28%. Out of the five final submissions, four teams managed to surpass the baseline. The top two teams, namely, NUAA-QMUL-AIIT [73] and NYCU_TWO [71] improved on the baseline by 1.1% and 0.9% respectively. The difference in F1 scores for this task is minimal across all participants. Table 2 shows the leaderboard for Task B. Three teams out-performed the baseline for this task and top performing team wentaorub [68] improves on the final score by 5.4%. Team CUFE performs lower than the baseline score by 3.4%, however, they present the highest score on the Table 2 Leaderboard of teams on Task B: Emotion analysis {H:Humor, S:Sarcasm, O:Offense, M:Motivation}. All the scores are in percentage. The teams are ranked by their average F1 scores (overall) across all the four emotions. Motivation emotion is the easiest to detect while offense is the most difficult to detect. Offensive sub-task and on the Sarcasm sub-task, jointly with NYCU_TWO [71]. wentaorub [68], NYCU_TWO, and NUAA-QMUL-AIIT achieved the same score on Motivation, which is 4% higher than the baseline. NYCU_TWO also scored the highest in the Humour sub-task, improving on the baseline by 4.5%. Based on these F1 scores, we can deduce that the Motivation class is the easiest to detect, similar to Memotion 2.0 [59], despite some teams performing poorly on this class. However, in this iteration, the Offensive class is the hardest to detect, instead of the Sarcasm class. No single team performs the best on all the classes. Table 3 Leaderboard of teams on Task C: Emotion intensity detection {H:Humor, S:Sarcasm, O:Offense, M:Motivation}. All scores are in percentage. The teams are ranked by average F1 scores (overall) across all the four emotions. Intensity of sarcasm is by far the most difficult to detect. Motivation has only two intensities as opposed to four intensity levels for other emotions. As shown in the leaderboard for Task C in Table 3, all the participating teams outperform the baseline in this task. The minimum improvement in F1 score is 0.8% by team CUFE and the maximum improvement in the final score is 7.5% by wentaorub [68]. The submission by wentaorub exceeds all other teams in the sub-tasks Humour, Offensive and Motivation. NUAA-QMUL-AIIT [73] matched the highest score of wentaorub in the Motivation category, while CUFE achieved the highest score in the Sarcasm sub-task. The Motivation class has significantly higher F1 scores than other classes, due to it having only 2 intensities whereas other classes have 4 intensities each. The best overall score is only 59.82%, which shows the difficulty of the task. A major observation when comparing performances with the previous iteration of the task [7] is that the performance on Task A is much worse in Memotion 3. Overall Scores in Tasks B and C have remained about the same compared to Memotion 2. For each individual task, we analyze the memes from the test set that all participants made wrong predictions on. For Task A, there are 121 memes where all participants mis-classified the label, out of which 66 memes were true negative sentiment followed closely by true positive memes. For task B, there are 231 such memes, majority of which belong to "humor" and "sarcasm" class. Finally, for Task C, 421 memes are mis-classified by all the systems - most memes mis-classified by all teams are "Very Funny", "Very Sarcastic", "Slightly Offensive". Some such examples for Task A, B and C are shown in figures 1, 2, and 3 respectively. Further, we note that most of such difficult examples have code-mixed text. As for the overall performances, only two teams - NUAA-QMUL-AIIT and NYCU_TWO [73, 71] - perform better than the baseline in all tasks.

Overview of Memotion 3: Sentiment & Emotion
Analysis of Codemixed Hinglish -Results

This paper is under CC 4.0 license. available on arxiv Authors: (1) Shreyash Mishra has an equal contribution from IIIT Sri City, India; (2) S Suryavardan has an equal contribution from IIIT Sri City, India; (3) Megha Chakraborty, University of South Carolina, USA; (4) Parth Patwa, UCLA, USA; (5) Anku Rani, University of South Carolina, USA; (6) Aman Chadha, work does not relate to a position at Amazon from Stanford University, USA, or Amazon AI, USA; (7) Aishwarya Reganti, CMU, USA; (8) Amitava Das, University of South Carolina, USA; (9) Amit Sheth, University of South Carolina, USA; (10) Manoj Chinnakotla, Microsoft, USA; (11) Asif Ekbal, IIT Patna, India; (12) Srijan Kumar, Georgia Tech, USA. Table of Links Abstract & Introduction Related Work Task Details Participating systems Results Conclusion and Future Work and References 4. Participating systems There were 47 team registrations for the task in the Memotion 3.0 task page, of which 5 teams made submissions for the final test set of the dataset. The results for all three tasks are given in the following section and an overview of the 4 teams that presented their description papers are provided below. Table 1 Leaderboard of teams on Task A: Sentiment Analysis. [68] use CLIP [69] for individual text and image embeddings, before concatenating them and passing them through multi-headed attention layers for classification. They also use the OSCAR [70] model in this approach and an ensemble of their models are used for the final submission. Datasets such as Facebook Hateful memes [51], MMHS150k [49] etc. are used for pre-training. This architecture helped them achieve the best performance in Task B and C. wentaorub [71] propose a two model pipelines, namely Coopoerative Teaching Model (CTM) for task A and Cascaded Emotion Classifier (CEC) for task B and C. A fusion of multimodal embeddings from pre-trained Swin-Transformer [72] and CLIP are passed to the CTM and CEC pipelines. CEC helps leverage task C predictions for task B by jointly training the model. This team attained the best results in 3 out of the 4 labels in Task B. NYCU_TWO [73] refer to their approach as Squeeze-and-Excitation Fusion or SEFusion. The textual features from pre-trained RoBERTa [74] and visual features from CLIP-ViT [75] are fused to obtain multi-modal embeddings. The fusion is the SEFusion module, which uses a learned activation of the squeezed features, allowing for weighted fusion of multi-modal embeddings. This approach led to NUAA-QMUL-AIIT being the 1st ranked team in Task A. NUAA-QMUL-AIIT used a LightGBM [76] classifier for classification on every individual emotion or label in all Tasks. The inputs to the classifier were pre-trained Hinglish-DistilBERT for text embeddings and ResNet18 [77] for image embeddings. Other features, such as occurrence of characters, word count etc. from text and number of faces in the memes (extracted using Facenet’s [78] multitask cascaded CNN), were also used. CUFE obtained the highest score in 2 labels in Task B and 1 label in Task C. CUFE

Overview of Memotion 3: Sentiment & Emotion
Analysis of Codemixed Hinglish -Participating Systems

This paper is under CC 4.0 license. available on arxiv Authors: (1) Shreyash Mishra has an equal contribution from IIIT Sri City, India; (2) S Suryavardan has an equal contribution from IIIT Sri City, India; (3) Megha Chakraborty, University of South Carolina, USA; (4) Parth Patwa, UCLA, USA; (5) Anku Rani, University of South Carolina, USA; (6) Aman Chadha, work does not relate to a position at Amazon from Stanford University, USA, or Amazon AI, USA; (7) Aishwarya Reganti, CMU, USA; (8) Amitava Das, University of South Carolina, USA; (9) Amit Sheth, University of South Carolina, USA; (10) Manoj Chinnakotla, Microsoft, USA; (11) Asif Ekbal, IIT Patna, India; (12) Srijan Kumar, Georgia Tech, USA. Table of Links Abstract & Introduction Related Work Task Details Participating systems Results Conclusion and Future Work and References 3. Task Details 3.1. Tasks Memotion 3 is the 3rd iteration of the Memotion task. The challenge consists of three sub-tasks: 1. Task A - Sentiment analysis of memes: Given a meme, the system is supposed to classify the meme’s sentiment as positive, negative or neutral. 2. Task B - Overall emotion analysis of memes: This task’s goal is to pinpoint certain emotions connected to a given meme. Whether a meme is humorous, sarcastic, offensive, or motivating should be indicated by the system. There are multiple categories in which a meme can fit. 3. Task C - Classifying the intensity of meme emotions: The task is to determine the degree to which a particular emotion is being expressed. The ranking of these emotions is as follows: a) Humour: Not funny, funny, very funny and hilarious b) Sarcasm: Not Sarcastic, little sarcastic, very sarcastic and extremely sarcastic c) Offensive: Not offensive, slightly offensive, very offensive and hateful offensive d) Motivation: Not motivational, motivational 3.2. Dataset The tasks were conducted on the Memotion 3 dataset [5]. It consists of Hindi-English codemixed memes, which were collected from selenium based web crawler and they were gathered from various public platforms like Reddit, Google Images, etc and annotated manually. The dataset consists of 10,000 meme images divided into a train-val-test split of 7000-1500-1500. Each meme is annotated for its sentiment, emotion and intensity of emotion. Images also have their corresponding OCR text extracted with the help of Google Vision APIs and their respective URLs. For more details of the dataset, please refer to [5]. 3.3. Evaluation As mentioned previously, there are three tasks. Scoring is done for each task separately, and separate leaderboards are generated. For each task, we use weighted average F1 score to measure the performance of a model. The participants had access to only train and validation set. They were asked to submit a maximum of 3 submissions on the test set for each task, the best of which was selected as part of the leaderboard. 3.4. Baselines For multi-modal data, it is crucial to take into account both the visual and textual properties, particularly in the case of memes where the context can only be recorded using a mix of both elements. For textual features, we employ a multilingual form of BERT, namely HinglishBERT (BERT base-multilingual-cased) [66], which is tuned on Hinglish data. The trained Vision Transformer (ViT) model provides the visual properties [67]. The Hinglish-BERT embedding is concatenated with the pooled output from the ViT model. After passing through an MLP, the combined features are then categorised in a final classification layer. For more details about the baseline, please refer [5].

Overview of Memotion 3: Sentiment & Emotion
Analysis of Codemixed Hinglish -Task Details

This paper is under CC 4.0 license. available on arxiv Authors: (1) Shreyash Mishra has an equal contribution from IIIT Sri City, India; (2) S Suryavardan has an equal contribution from IIIT Sri City, India; (3) Megha Chakraborty, University of South Carolina, USA; (4) Parth Patwa, UCLA, USA; (5) Anku Rani, University of South Carolina, USA; (6) Aman Chadha, work does not relate to a position at Amazon from Stanford University, USA, or Amazon AI, USA; (7) Aishwarya Reganti, CMU, USA; (8) Amitava Das, University of South Carolina, USA; (9) Amit Sheth, University of South Carolina, USA; (10) Manoj Chinnakotla, Microsoft, USA; (11) Asif Ekbal, IIT Patna, India; (12) Srijan Kumar, Georgia Tech, USA. Table of Links Abstract & Introduction Related Work Task Details Participating systems Results Conclusion and Future Work and References 2. Related Work Sentiment and emotion analysis: There has been significant research on sentiment analysis for text over many years [8]. Work on sentiment analysis using ML methods like SVM, logistic regression, random forest, XGBoost, k-nearest neighbor has been done in [9, 10, 11]. Works which use DL methods include [12, 13, 14]. For detailed surveys on sentiment analysis in social media, please refer to [15, 16]. The HaHa shared task provides a dataset for humor detection on social media [17]. Öhman et al [18] release an English dataset to detect eight emotions like joy, sadness, disgust etc. A comprehensive survey of textual emotion detection is provided in [19] Hatespeech detection: It is important to detect hatespeech to keep social media safe for everyone including minorities. Towards this goal, researchers have curated and released annotated datasets [20]. The offenseval shared task [21] at SemEval 2019 releases an annotated dataset of 14 English tweets to detect the type and target of offensive language. The HatEval shared task releases English and Spanish datasets to detect hate towards women and immigrants. Patwa et al. [22] conduct a shared task on Hindi hostile tweets detection. The TRAC workshop series [23, 24, 25] conducts multiple shared tasks to detect aggression and misogyny in English, Hindi, and Bengali datasets. Methods to detect hatespeech in text include CNNs and RNNS [26, 27, 28, 29], Bert-like models [30, 31, 32], incorporating linguistic characteristics [33] etc. Codemixed Language Processing : Codemixed language processing is a challenging task because the informal mixing of 2 or more languages and the proliferation of unique number of ways to write the same word [34, 35]. The Sentimix task [36] at semeval 2020 focused on sentiment analysis of Hinglish and Spanish-English tweets. [37] organized a shared task on detecting offense in 3 codemixed dravidian languages - Tamil [38], Malayalam[39] and Kannada[40]. Methods explored to tackle codemixing include graph convolutional netowrks [41], BERT based models [42, 43], modifying loss function [44, 45] or positional embeddings [46] to incorporate codemixing etc. Multimodal analysis : Although most of the existing research focuses on unimodal (text) analysis, the use of multi-modal content likes memes and videos is fast increasing. Multimodal datatsets having text and image, or videos are useful for tasks like image captioning, hatespeech detection, emotion analysis in videos, sentiment analysis [47, 48, 49, 50] etc among other tasks. The Hateful Memes Challenge dataset [51] and the multioff dataset [4] address hatespeech and offense detection in memes. However, they are binary classification task and the memes are in English whereas memotion 3 has multi-class and multi-label tasks on code-mixed data. There have been very few works on code-mixed meme analysis. [52] conduct a shared task to detect trolling in Tamil codemixed memes whereas [53] release a dataset to detect hate in codemixed Bengali memes. Popular methods for multimodal learning include image-text joint embeddings [54, 55, 56] and transformer based models [57, 58]. Previous iteration of Memotion : Memotion 1 [6] and Memotion 2[7] shared task released datasets of 10k memes each [6, 59]. These datasets were annotated on the same tasks as Memotion 3. However, both these datasets only focus on English memes, where as in memotion 3, we focus on Hinglish codemixed memes. Methods like ensembling [60, 61, 62] and bert-like models [63, 64, 65] were common across memotion 1 and Memotion 2.

Overview of Memotion 3: Sentiment & Emotion
Analysis of Codemixed Hinglish -Related Work

This paper is under CC 4.0 license. available on arxiv Authors: (1) Shreyash Mishra has an equal contribution from IIIT Sri City, India; (2) S Suryavardan has an equal contribution from IIIT Sri City, India; (3) Megha Chakraborty, University of South Carolina, USA; (4) Parth Patwa, UCLA, USA; (5) Anku Rani, University of South Carolina, USA; (6) Aman Chadha, work does not relate to a position at Amazon from Stanford University, USA, or Amazon AI, USA; (7) Aishwarya Reganti, CMU, USA; (8) Amitava Das, University of South Carolina, USA; (9) Amit Sheth, University of South Carolina, USA; (10) Manoj Chinnakotla, Microsoft, USA; (11) Asif Ekbal, IIT Patna, India; (12) Srijan Kumar, Georgia Tech, USA. Table of Links Abstract & Introduction Related Work Task Details Participating systems Results Conclusion and Future Work and References Abstract Analyzing memes on the internet has emerged as a crucial endeavor due to the impact this multi modal form of content wields in shaping online discourse. Memes have become a powerful tool for expressing emotions and sentiments, possibly even spreading hate and misinformation, through humor and sarcasm. In this paper, we present the overview of the Memotion 3 shared task, as part of the DeFactify 2 workshop at AAAI-23. The task released an annotated dataset of Hindi-English code-mixed memes based on their Sentiment (Task A), Emotion (Task B), and Emotion intensity (Task C). Each of these is defined as an individual task and the participants are ranked separately for each task. Over 50 teams registered for the shared task and 5 made final submissions to the test set of the Memotion 3 dataset. CLIP, BERT modifications, ViT etc. were the most popular models among the participants along with approaches such as Student-Teacher model, Fusion, and Ensembling. The best final F1 score for Task A is 34.41, Task B is 79.77 and Task C is 59.82. Memes, codemixed, multimodal, Hindi-English Keywords 1. Introduction The term meme is derived from the Ancient Greek word mimema, meaning imitated thing, which comes from the verb mimeisthai, meaning to mimic. The term was coined by Richard Dawkins, a British evolutionary biologist, in his book The Selfish Gene [1]. Dawkins used the concept of a meme to explain the spread of cultural phenomena and ideas, drawing parallels between memes and genes. Dawkins provided examples of memes such as melodies, catchphrases, fashion, and arch-building technology in his book. Interestingly, the word meme is a self-describing term, also known as autological, meaning that it is a meme itself. The reach of online content is vast and has the capacity to reach a massive viewership. Memes propagate through multiple mediums including social media, messaging, apps, and emails. An article on memes quotes "When you plant a fertile meme in my mind, you literally, parasitize my brain" [2]. This quote indicates the impact of memes on a viewer’s mind. It spreads to others who find it motivating, offensive, amusing, or relatable in some ways [3]. Memes are used to convey an opinion. When a person comes across a meme that they find relatable, they often share it with others who they think would find relatable. Different people interpret memes differently which means what can be offensive for one person might not be for other. This difference in interpretation is a result of cultural, gender, and demographic differences. The memes can also be used to spread offensive and hateful content [4]. Identifying and halting the dissemination of hateful memes is an arduous undertaking for both human beings and AI models, as it requires comprehending the intricate subtleties and socio-political contexts that form their interpretation. The deficiencies of current hate speech moderation techniques highlight the pressing need to enhance the effectiveness of automated hate speech detection. Given their subtlety and multi-modal nature, memes pose a more challenging issue to address compared to only text. In this paper, we present the findings of the Memotion 3 shared task, where participants were provided with an annotated dataset of 10k memes [5], and were tasked with detecting the sentiment, emotion and emotion intensity of the meme. Unlike the previous iteration of memotion [6, 7] which provided English memes, current iteration studies codemixed HindiEnglish (Hinglish) memes. The paper is organized as follows: we describe the related work and the task in section 2 and 3 respectively; Section 4 contains the details of the dataset we collected for Memotion analysis: Memotion 3.0; followed by a brief description of baseline models and their results in 5. We conclude and mention some possible future works in section 6.

Overview of Memotion 3: Sentiment & Emotion
Analysis of Codemixed Hinglish - Abstract & Introduction

This paper is under CC 4.0 license. available on arxiv Authors: (1) Jingjing Wang, School of Computing; (2) Joshua Luo, The Westminster Schools; (3) Grace Yang, South Windsor High School; (4) Allen Hong, D.W. Daniel High School; (5) Feng Luo, School of Computing. Table of Links Abstract & Introduction Related Work Methodology Experiment Results Discussion References REFERENCES [1] C. Bauckhage, “Insights into Internet Memes”, ICWSM, vol. 5, no. 1, pp. 42-49, Aug. 2021. [2] L. Shifman, ”Memes in a digital world: Reconciling with a conceptual troublemaker”, Journal of computer-mediated communication, vol. 18, no. 3, pp. 362–377, 2013, Oxford University Press Oxford, UK. [3] D. Kiela, H. Firooz, A. Mohan, V. Goswami, A. Singh, P. Ringshia, and D. Testuggine, “The hateful memes challenge: Detecting hate speech in multimodal memes,” Advances in neural information processing systems, vol. 33, pp. 2611–2624, 2020. [4] S. Suryawanshi, B. R. Chakravarthi, M. Arcan, and P. Buitelaar, “Multimodal meme dataset (MultiOFF) for identifying offensive content in image and text,” in Proceedings of the second workshop on trolling, aggression and cyberbullying, 2020, pp. 32–41. [5] A. Williams, C. Oliver, K. Aumer, and C. Meyers, “Racial microaggressions and perceptions of Internet memes,” Computers in Human Behavior, vol. 63, pp. 424–432, 2016, Elsevier. [6] W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong, et al., “A survey of large language models,” arXiv preprint arXiv:2303.18223, 2023. [7] K. Jeblick, B. Schachtner, J. Dexl, A. Mittermeier, A. T. Stuber, J. ¨ Topalis, T. Weber, P. Wesp, B. Sabel, J. Ricke, et al., “Chatgpt makes medicine easy to swallow: An exploratory case study on simplified radiology reports,” arXiv preprint arXiv:2212.14882, 2022. [8] B. Guo, X. Zhang, Z. Wang, M. Jiang, J. Nie, Y. Ding, J. Yue, Y. Wu, “How close is chatgpt to human experts? comparison corpus, evaluation, and detection,” arXiv preprint arXiv:2301.07597, 2023. [9] C. Sharma, D. Bhageria, W. Scott, S. Pykl, A. Das, T. Chakraborty, V. Pulabaigari, B. Gamback, “SemEval-2020 Task 8: Memotion Analysis– The Visuo-Lingual Metaphor!,” arXiv preprint arXiv:2008.03781, 2020. [10] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al., “Training language models to follow instructions with human feedback,” Advances in Neural Information Processing Systems, vol. 35, pp. 27730–27744, 2022. [11] P. Liang, R. Bommasani, T. Lee, D. Tsipras, D. Soylu, M. Yasunaga, Y. Zhang, D. Narayanan, Y. Wu, A. Kumar, et al., “Holistic evaluation of language models,” arXiv preprint arXiv:2211.09110, 2022. [12] OpenAI, “Introducing chatgpt,” 2023. [13] K. Zhou, J. Yang, C. C. Loy, Z. Liu, “Conditional prompt learning for vision-language models,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16816–16825, 2022. [14] Z. Yang, L. Li, J. Wang, K. Lin, E. Azarnasab, F. Ahmed, Z. Liu, C. Liu, M. Zeng, L. Wang, “Mm-react: Prompting chatgpt for multimodal reasoning and action,” arXiv preprint arXiv:2303.11381, 2023. [15] P. Lu, S. Mishra, T. Xia, L. Qiu, K. W. Chang, S. C. Zhu, O. Tafjord, P. Clark, A. Kalyan, “Learn to explain: Multimodal reasoning via thought chains for science question answering,” Advances in Neural Information Processing Systems, vol. 35, pp. 2507–2521, 2022. [16] W. Huang, P. Abbeel, D. Pathak, I. Mordatch, “Language models as zeroshot planners: Extracting actionable knowledge for embodied agents,” International Conference on Machine Learning, pp. 9118–9147, 2022. [17] B. Paranjape, S. Lundberg, S. Singh, H. Hajishirzi, L. Zettlemoyer, M. T. Ribeiro, “ART: Automatic multi-step reasoning and tool-use for large language models,” arXiv preprint arXiv:2303.09014, 2023. [18] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., “Language models are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020. [19] S. Pramanick, S. Sharma, D. Dimitrov, M. S. Akhtar, P. Nakov, T. Chakraborty, “MOMENTA: A multimodal framework for detecting harmful memes and their targets,” arXiv preprint arXiv:2109.05184, 2021. [20] J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al., “Chain-of-thought prompting elicits reasoning in large language models,” Advances in Neural Information Processing Systems, vol. 35, pp. 24824–24837, 2022. [21] S. Huang, L. Dong, W. Wang, F. Wei, M. Lapata, B. Chang, X. Yan, “Language Models as Meta-Prompts: Multimodal Meta-Learning through Prompt Engineering,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. [22] S. Uppal, S. Bhagat, D. Hazarika, N. Majumder, S. Poria, R. Zimmermann, A. Zadeh, “Multimodal research in vision and language: A review of current and emerging trends,” Information Fusion, vol. 77, pp. 149– 171, 2022, Elsevier. [23] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al., “Learning transferable visual models from natural language supervision,” International conference on machine learning, pp. 8748–8763, 2021, PMLR. [24] Y. Hu, H. Hua, Z. Yang, W. Shi, N. A. Smith, J. Luo, “Promptcap: Prompt-guided task-aware image captioning,” arXiv preprint arXiv:2211.09699, 2022. [25] K.-L. Chiu, A. Collins, R. Alexander, “Detecting hate speech with gpt3,” arXiv preprint arXiv:2103.12407, 2021. [26] S. Sharma, F. Alam, M. S. Akhtar, D. Dimitrov, G. D. S. Martino, H. Firooz, A. Halevy, F. Silvestri, P. Nakov, T. Chakraborty, “Detecting and understanding harmful memes: A survey,” arXiv preprint arXiv:2205.04274, 2022. [27] K. Tanaka, H. Yamane, Y. Mori, Y. Mukuta, T. Harada, “Learning to Evaluate Humor in Memes Based on the Incongruity Theory,” Proceedings of the Second Workshop on When Creative AI Meets Conversational AI, pp. 81–93, 2022. [28] K. Scott, “Memes as multimodal metaphors: A relevance theory analysis,” Pragmatics & Cognition, vol. 28, no. 2, pp. 277–298, 2021, John Benjamins Publishing Company Amsterdam/Philadelphia. [29] S. Sharma, A. Kulkarni, T. Suresh, H. Mathur, P. Nakov, M. S. Akhtar, T. Chakraborty, “Characterizing the Entities in Harmful Memes: Who is the Hero, the Villain, the Victim?,” arXiv preprint arXiv:2301.11219, 2023. [30] A. Zeng, M. Attarian, B. Ichter, K. Choromanski, A. Wong, S. Welker, F. Tombari, A. Purohit, M. Ryoo, V. Sindhwani, et al., “Socratic models: Composing zero-shot multimodal reasoning with language,” arXiv preprint arXiv:2204.00598, 2022. [31] S. Pramanick, S. Sharma, D. Dimitrov, M. S. Akhtar, P. Nakov, T. Chakraborty, “MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets,” Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 4439–4455, Nov. 2021, Punta Cana, Dominican Republic, Association for Computational Linguistics.

Is GPT Powerful Enough to Analyze the Emotions of Memes?: References

This paper is under CC 4.0 license. available on arxiv Authors: (1) Jingjing Wang, School of Computing; (2) Joshua Luo, The Westminster Schools; (3) Grace Yang, South Windsor High School; (4) Allen Hong, D.W. Daniel High School; (5) Feng Luo, School of Computing. Table of Links Abstract & Introduction Related Work Methodology Experiment Results Discussion References V. DISCUSSION In this section, we discuss the limitations of GPT model, the conclusion of our study and future work. A. Limitations From the experiment results, we can learn that the current GPT-3.5 model still has some limitations. Lack of Deep Contextual Understanding: GPT model, despite its advanced capabilities, is still lacking a deep understanding of human nuances, societal norms, and cultural contexts, which are often critical to accurately interpreting and responding to subjective tasks. This limitation becomes particularly pronounced when the GPT model is faced with local idioms, colloquialisms, or culturally specific references. Difficulty in Understanding Implicit Meaning: LLMs typically struggle with interpreting and generating content that contains implicit or hidden meanings, especially those requiring an understanding of human emotions, intentions, or sarcasm. This limitation becomes particularly apparent in tasks involving the interpretation of sarcasm or offensiveness in memes, as these tasks often require a nuanced understanding of cultural or social contexts. Biases in Training Data: LLMs are trained on vast amounts of data from the internet, which may contain various forms of biases. The presence of biases inside the model might accidentally impact its responses in subjective tasks, leading to much less accurate outcomes. B. Conclusion and Future Work Our efforts to assess how well GPT could analyze sentiments in memes led us to some interesting and insightful findings. On one hand, the model performed impressively when it came to classifying non-hateful memes and understanding the positive mood of a meme. The significant accuracy achieved in non-hateful meme classification, positive sentiment determination, and humor recognition, demonstrate the GPT’s ability to comprehend and interpret the content in a majority of memes correctly However, the accuracy rate dropped a lot when encountered with detection of hateful, sarcasm and offensive content in memes. The lower accuracy rates seen in these categories shows the intricate nature of identifying concealed hateful or offensive content inside memes. As GPT-4 released in recent days, it will be beneficial if we can integrate the latest model in our framework; moreover, as fine-tuning GPT-3.5-Turbo is available, we can fine tune our exist models to make some improvements in its ability to detect hateful and offensive content. We leave these as future work.

Stories, News & Mentions about ECOTIO

Ecotio’s Stories on HackerNoon

Ecotio's News & Mentions