Authors: (1) Yi-Ling Chung, The Alan Turing Institute (ychung@turing.ac.uk); (2) Gavin Abercrombie, The Interaction Lab, Heriot-Watt University (g.abercrombie@hw.ac.uk); (3) Florence Enock, The Alan Turing Institute (fenock@turing.ac.uk); (4) Jonathan Bright, The Alan Turing Institute (jbright@turing.ac.uk); (5) Verena Rieser, The Interaction Lab, Heriot-Watt University and now at Google DeepMind (v.t.rieser@hw.ac.uk). Table of Links Abstract and 1 Introduction 2 Background 3 Review Methodology 4 Defining counterspeech 4.1 Classifying counterspeech 5 The Impact of Counterspeech 6 Computational Approaches to Counterspeech and 6.1 Counterspeech Datasets 6.2 Approaches to Counterspeech Detection and 6.3 Approaches to Counterspeech Generation 7 Future Perspectives 8 Conclusion, Acknowledgements, and References 6.2 Approaches to Counterspeech Detection Previous work on counterspeech detection has focused on binary classification (i.e. whether a text is counterspeech or not) (Vidgen et al., 2020; Garland et al., 2022; He et al., 2022) or identifying the types of counterspeech as a multi-label task (Mathew et al., 2018; Garland et al., 2020; Chung et al., 2021a; Goffredo et al., 2022). Automated classifiers are developed to analyse large-scale social interactions of abuse and counterspeech addressing topics such as political discourse (Garland et al., 2022) and multi-hate targets (Mathew et al., 2018). Moving beyond monolingual study, Chung et al. (2021a) evaluate the performance of pre-trained language models for categorising counterspeech strategy for English, Italian and French in monolingual, multilingual and cross-lingual scenarios. 6.3 Approaches to Counterspeech Generation Various methodologies have been put forward for the automation of counterspeech generation (Qian et al., 2019), addressing various aspects including the efficacy of a hate countering platform (Chung et al., 2021c), informativeness (Chung et al., 2021b), multilinguality (Chung et al., 2020), politeness (Saha et al., 2022), and grammaticality and diversity (Zhu and Bhat, 2021). These methods are generally centred on transformer-based large language models (e.g., GPT-2 (Radford et al., 2019)). By testing various decoding mechanisms using multiple language models, Tekiroglu et al. (2022) find that ˘ autoregressive models combined with stochastic decoding yield the optimal counterspeech generation. In addition to tackling hate speech, there are studies investigating automatic counterspeech generation to respond to trolls (Lee et al., 2022) and microagressions (Ashida and Komachi, 2022). Evaluation of counterspeech generation Assessing counter speech generation is complex and challenging due to the lack of clear evaluation criteria and robust evaluation techniques. Previous work evaluates the performance of counterspeech systems via two aspects: automatic metrics and human evaluation. Automatic metrics, generally, evaluate the generation quality based on criteria such as linguistic surface (Papineni et al., 2002; Lin, 2004), novelty (Wang and Wan, 2018), and repetitiveness (Bertoldi et al., 2013; Cettolo et al., 2014). Despite being scalable, these metrics are uninterpretable and can only infer model performance according to references provided (e.g., dependent heavily on exact word usage and word order) and gathering an exhaustive list of all appropriate counterspeech is not feasible. For this reason, such metrics cannot properly capture model performance, particularly for open-ended tasks (Liu et al., 2016; Novikova et al., 2017) including counterspeech generation. As a result, human evaluation is heavily employed based on aspects such as suitableness, grammatical accuracy and relevance (Chung et al., 2021b; Zhu and Bhat, 2021). Despite being trusted and high-performing, human evaluation has inherent limitations such as being costly, difficult (e.g., evaluator biases and question formatting), and time-consuming (both in terms of evaluation and moderator training), and can be inconsistent and inflict psychological harm on the moderators. The effectiveness of counterspeech generations should be also carefully investigated ‘in-the-wild’ to understand its social media impact, reach of content, and the dynamics of hateful content and counterspeech (see Section 5). To our knowledge, no work has examined this line of research yet. Potentials and limits of existing generative models We believe that in some circumstances counterspeech may be a more appropriate tool than content moderation in fighting hate speech as it can depolarise discourse and show support to victims. However, automatic counterspeech generation is a relatively new research area. Recent progress in natural language processing has made large language models a popular vehicle for generating fluent counterspeech. However, counterspeech generation currently faces several challenges that may constrain the development of efficient models and hinder the deployment of hate intervention tools. Similar to the use of machine translation and email writing tools, we advocate that counterspeech generation tools should be deployed as suggestion tools to assist in hate countering activity (Chung et al., 2021b,c). • Faithfulness/Factuality in generation Language models are repeatedly reported to produce plausible and convincing but not necessarily faithful/factual statements (Solaiman et al., 2019; Zellers et al., 2019; Chung et al., 2021b). We refer to faithfulness as being consistent and truthful in adherence to the given source (i.e. model inputs) (Ji et al., 2023). Many attempts have been made to mitigate this issue (Ji et al., 2023), including correcting unfaithful data (Nie et al., 2019), augmenting inputs with additional knowledge sources (Chung et al., 2021b), and measuring faithfulness of generated outputs (Dušek and Kasner, 2020; Zhou et al., 2021). We encourage reporting the faithfulness/factuality of models. • Toxic degeneration Language models can also induce unintendedly biased and/or toxic content, regardless of whether explicit prompts are used (Dinan et al., 2022). In the use case of counterspeech generation, this can result in harm to victims and bystanders as well as risking provoking perpetrators into further abusive behaviour. This issue has been mitigated by two approaches: data and modelling. The data approach aims at creating proper datasets for fairness by removing undesired and biased content (Blodgett et al., 2020; Raffel et al., 2020). The modelling approach focuses on controllable generation techniques that, for instance, employ humans for post-editing (Tekiroglu et al., 2020) and ˘ detoxification techniques (Gehman et al., 2020). • Generalisation vs. Specialisation With the rise of online hate, models that can generalize across domains would be helpful for producing counterspeech involving new topics and events. Generalisable methods can also ameliorate the time and manual effort required for collecting and annotating data. However, as discussed in Section 5, counterspeech is multifaceted and contextualised. There may not be a one-size-fits-all solution. For instance, abuse against women can often be expressed in a more subtle form as microaggressions. It may, therefore, be difficult to implement an easy yet effective counterspeech strategy in one model. Moreover, model generalisability is challenging (Fortuna et al., 2021; Yin and Zubiaga, 2021), and can have potential limitations (Conneau et al., 2020; Berend, 2022). Finding the right trade-off between generalisation and specialisation is key. This paper is available on arxiv under CC BY-SA 4.0 DEED license. Authors: (1) Yi-Ling Chung, The Alan Turing Institute (ychung@turing.ac.uk); (2) Gavin Abercrombie, The Interaction Lab, Heriot-Watt University (g.abercrombie@hw.ac.uk); (3) Florence Enock, The Alan Turing Institute (fenock@turing.ac.uk); (4) Jonathan Bright, The Alan Turing Institute (jbright@turing.ac.uk); (5) Verena Rieser, The Interaction Lab, Heriot-Watt University and now at Google DeepMind (v.t.rieser@hw.ac.uk). Authors: Authors: (1) Yi-Ling Chung, The Alan Turing Institute (ychung@turing.ac.uk); (2) Gavin Abercrombie, The Interaction Lab, Heriot-Watt University (g.abercrombie@hw.ac.uk); (3) Florence Enock, The Alan Turing Institute (fenock@turing.ac.uk); (4) Jonathan Bright, The Alan Turing Institute (jbright@turing.ac.uk); (5) Verena Rieser, The Interaction Lab, Heriot-Watt University and now at Google DeepMind (v.t.rieser@hw.ac.uk). Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Background 2 Background 3 Review Methodology 3 Review Methodology 4 Defining counterspeech 4 Defining counterspeech 4.1 Classifying counterspeech 4.1 Classifying counterspeech 5 The Impact of Counterspeech 5 The Impact of Counterspeech 6 Computational Approaches to Counterspeech and 6.1 Counterspeech Datasets 6 Computational Approaches to Counterspeech and 6.1 Counterspeech Datasets 6.2 Approaches to Counterspeech Detection and 6.3 Approaches to Counterspeech Generation 6.2 Approaches to Counterspeech Detection and 6.3 Approaches to Counterspeech Generation 7 Future Perspectives 7 Future Perspectives 8 Conclusion, Acknowledgements, and References 8 Conclusion, Acknowledgements, and References 6.2 Approaches to Counterspeech Detection Previous work on counterspeech detection has focused on binary classification (i.e. whether a text is counterspeech or not) (Vidgen et al., 2020; Garland et al., 2022; He et al., 2022) or identifying the types of counterspeech as a multi-label task (Mathew et al., 2018; Garland et al., 2020; Chung et al., 2021a; Goffredo et al., 2022). Automated classifiers are developed to analyse large-scale social interactions of abuse and counterspeech addressing topics such as political discourse (Garland et al., 2022) and multi-hate targets (Mathew et al., 2018). Moving beyond monolingual study, Chung et al. (2021a) evaluate the performance of pre-trained language models for categorising counterspeech strategy for English, Italian and French in monolingual, multilingual and cross-lingual scenarios. 6.3 Approaches to Counterspeech Generation Various methodologies have been put forward for the automation of counterspeech generation (Qian et al., 2019), addressing various aspects including the efficacy of a hate countering platform (Chung et al., 2021c), informativeness (Chung et al., 2021b), multilinguality (Chung et al., 2020), politeness (Saha et al., 2022), and grammaticality and diversity (Zhu and Bhat, 2021). These methods are generally centred on transformer-based large language models (e.g., GPT-2 (Radford et al., 2019)). By testing various decoding mechanisms using multiple language models, Tekiroglu et al. (2022) find that ˘ autoregressive models combined with stochastic decoding yield the optimal counterspeech generation. In addition to tackling hate speech, there are studies investigating automatic counterspeech generation to respond to trolls (Lee et al., 2022) and microagressions (Ashida and Komachi, 2022). Evaluation of counterspeech generation Assessing counter speech generation is complex and challenging due to the lack of clear evaluation criteria and robust evaluation techniques. Evaluation of counterspeech generation Previous work evaluates the performance of counterspeech systems via two aspects: automatic metrics and human evaluation. Automatic metrics, generally, evaluate the generation quality based on criteria such as linguistic surface (Papineni et al., 2002; Lin, 2004), novelty (Wang and Wan, 2018), and repetitiveness (Bertoldi et al., 2013; Cettolo et al., 2014). Despite being scalable, these metrics are uninterpretable and can only infer model performance according to references provided (e.g., dependent heavily on exact word usage and word order) and gathering an exhaustive list of all appropriate counterspeech is not feasible. For this reason, such metrics cannot properly capture model performance, particularly for open-ended tasks (Liu et al., 2016; Novikova et al., 2017) including counterspeech generation. As a result, human evaluation is heavily employed based on aspects such as suitableness, grammatical accuracy and relevance (Chung et al., 2021b; Zhu and Bhat, 2021). Despite being trusted and high-performing, human evaluation has inherent limitations such as being costly, difficult (e.g., evaluator biases and question formatting), and time-consuming (both in terms of evaluation and moderator training), and can be inconsistent and inflict psychological harm on the moderators. The effectiveness of counterspeech generations should be also carefully investigated ‘in-the-wild’ to understand its social media impact, reach of content, and the dynamics of hateful content and counterspeech (see Section 5). To our knowledge, no work has examined this line of research yet. Potentials and limits of existing generative models We believe that in some circumstances counterspeech may be a more appropriate tool than content moderation in fighting hate speech as it can depolarise discourse and show support to victims. However, automatic counterspeech generation is a relatively new research area. Recent progress in natural language processing has made large language models a popular vehicle for generating fluent counterspeech. However, counterspeech generation currently faces several challenges that may constrain the development of efficient models and hinder the deployment of hate intervention tools. Similar to the use of machine translation and email writing tools, we advocate that counterspeech generation tools should be deployed as suggestion tools to assist in hate countering activity (Chung et al., 2021b,c). Potentials and limits of existing generative models • Faithfulness/Factuality in generation Language models are repeatedly reported to produce plausible and convincing but not necessarily faithful/factual statements (Solaiman et al., 2019; Zellers et al., 2019; Chung et al., 2021b). We refer to faithfulness as being consistent and truthful in adherence to the given source (i.e. model inputs) (Ji et al., 2023). Many attempts have been made to mitigate this issue (Ji et al., 2023), including correcting unfaithful data (Nie et al., 2019), augmenting inputs with additional knowledge sources (Chung et al., 2021b), and measuring faithfulness of generated outputs (Dušek and Kasner, 2020; Zhou et al., 2021). We encourage reporting the faithfulness/factuality of models. • Faithfulness/Factuality in generation • Toxic degeneration Language models can also induce unintendedly biased and/or toxic content, regardless of whether explicit prompts are used (Dinan et al., 2022). In the use case of counterspeech generation, this can result in harm to victims and bystanders as well as risking provoking perpetrators into further abusive behaviour. This issue has been mitigated by two approaches: data and modelling. The data approach aims at creating proper datasets for fairness by removing undesired and biased content (Blodgett et al., 2020; Raffel et al., 2020). The modelling approach focuses on controllable generation techniques that, for instance, employ humans for post-editing (Tekiroglu et al., 2020) and ˘ detoxification techniques (Gehman et al., 2020). • Toxic degeneration • Generalisation vs. Specialisation With the rise of online hate, models that can generalize across domains would be helpful for producing counterspeech involving new topics and events. Generalisable methods can also ameliorate the time and manual effort required for collecting and annotating data. However, as discussed in Section 5, counterspeech is multifaceted and contextualised. There may not be a one-size-fits-all solution. For instance, abuse against women can often be expressed in a more subtle form as microaggressions. It may, therefore, be difficult to implement an easy yet effective counterspeech strategy in one model. Moreover, model generalisability is challenging (Fortuna et al., 2021; Yin and Zubiaga, 2021), and can have potential limitations (Conneau et al., 2020; Berend, 2022). Finding the right trade-off between generalisation and specialisation is key. • Generalisation vs. Specialisation This paper is available on arxiv under CC BY-SA 4.0 DEED license. This paper is available on arxiv under CC BY-SA 4.0 DEED license. available on arxiv
