Overview of Memotion 3: Sentiment & Emotion Analysis of Codemixed Hinglish -Related Work

Table of Links

Abstract & Introduction

Related Work

Task Details

Participating systems

Results

Conclusion and Future Work and References

2. Related Work

Sentiment and emotion analysis: There has been significant research on sentiment analysis for text over many years [8]. Work on sentiment analysis using ML methods like SVM, logistic regression, random forest, XGBoost, k-nearest neighbor has been done in [9, 10, 11]. Works which use DL methods include [12, 13, 14]. For detailed surveys on sentiment analysis in social media, please refer to [15, 16].

The HaHa shared task provides a dataset for humor detection on social media [17]. Öhman et al [18] release an English dataset to detect eight emotions like joy, sadness, disgust etc. A comprehensive survey of textual emotion detection is provided in [19]

Hatespeech detection: It is important to detect hatespeech to keep social media safe for everyone including minorities. Towards this goal, researchers have curated and released annotated datasets [20]. The offenseval shared task [21] at SemEval 2019 releases an annotated dataset of 14 English tweets to detect the type and target of offensive language. The HatEval shared task releases English and Spanish datasets to detect hate towards women and immigrants. Patwa et al. [22] conduct a shared task on Hindi hostile tweets detection. The TRAC workshop series [23, 24, 25] conducts multiple shared tasks to detect aggression and misogyny in English, Hindi, and Bengali datasets.

Methods to detect hatespeech in text include CNNs and RNNS [26, 27, 28, 29], Bert-like models [30, 31, 32], incorporating linguistic characteristics [33] etc.

Codemixed Language Processing : Codemixed language processing is a challenging task because the informal mixing of 2 or more languages and the proliferation of unique number of ways to write the same word [34, 35]. The Sentimix task [36] at semeval 2020 focused on sentiment analysis of Hinglish and Spanish-English tweets. [37] organized a shared task on detecting offense in 3 codemixed dravidian languages - Tamil [38], Malayalam[39] and Kannada[40]. Methods explored to tackle codemixing include graph convolutional netowrks [41], BERT based models [42, 43], modifying loss function [44, 45] or positional embeddings [46] to incorporate codemixing etc.

Multimodal analysis : Although most of the existing research focuses on unimodal (text) analysis, the use of multi-modal content likes memes and videos is fast increasing. Multimodal datatsets having text and image, or videos are useful for tasks like image captioning, hatespeech detection, emotion analysis in videos, sentiment analysis [47, 48, 49, 50] etc among other tasks. The Hateful Memes Challenge dataset [51] and the multioff dataset [4] address hatespeech and offense detection in memes. However, they are binary classification task and the memes are in English whereas memotion 3 has multi-class and multi-label tasks on code-mixed data. There have been very few works on code-mixed meme analysis. [52] conduct a shared task to detect trolling in Tamil codemixed memes whereas [53] release a dataset to detect hate in codemixed Bengali memes. Popular methods for multimodal learning include image-text joint embeddings [54, 55, 56] and transformer based models [57, 58].

Previous iteration of Memotion : Memotion 1 [6] and Memotion 2[7] shared task released datasets of 10k memes each [6, 59]. These datasets were annotated on the same tasks as Memotion 3. However, both these datasets only focus on English memes, where as in memotion 3, we focus on Hinglish codemixed memes. Methods like ensembling [60, 61, 62] and bert-like models [63, 64, 65] were common across memotion 1 and Memotion 2.