Compa is making compensation fair and competitive for everyone by removing barriers to effective decision-making

Compa

How I Built a Redis-Compatible Pub/Sub System Using Golang

কিভাবে আমি গোলং ব্যবহার করে একটি রেডিস-সামঞ্জস্যপূর্ণ পাব/সাব সিস্টেম তৈরি করেছি

Wie ich mit Golang ein Redis-kompatibles Pub/Sub-System erstellt habe

Cómo construí un sistema Pub/Sub compatible con Redis usando Golang

Comment j'ai construit un système Pub/Sub compatible Redis à l'aide de Golang

मैंने Golang का उपयोग करके Redis-संगत Pub/Sub सिस्टम कैसे बनाया

Golang を使って Redis 互換の Pub/Sub システムを構築する方法

Golang을 사용하여 Redis 호환 Pub/Sub 시스템을 구축하는 방법

Como construí um sistema Pub/Sub compatível com Redis usando Golang

Как я создал Redis-совместимую систему Pub/Sub с использованием Golang

Golang'ı Kullanarak Redis Uyumlu Bir Pub/Sub Sistemini Nasıl Oluşturdum?

Cách tôi xây dựng hệ thống Pub/Sub tương thích với Redis bằng Golang

如何使用 Golang 构建与 Redis 兼容的发布/订阅系统

Authors:
(1) Rui Cao, Singapore Management University;
(2) Ming Shan Hee, Singapore University of Design and Technology;
(3) Adriel Kuek, DSO National Laboratories;
(4) Wen-Haw Chong, Singapore Management University;
(5) Roy Ka-Wei Lee, Singapore University of Design and Technology
(6) Jing Jiang, Singapore Management University. Table of Links Abstract and Introduction Related Work Preliminary Proposed Method Experiment Conclusion and References Appendix 4 PROPOSED METHOD 4.1 Overview Recall that the key idea of our method is to elicit image details that are critical for hateful content detection, such as the gender and race of the people in the image. Because these details are not always included in automatically generated image captions, we propose relying on VQA to obtain such critical information, where the questions are carefully curated to elicit demographic and other relevant information. We opt to use zero-shot VQA because (1) for the intended type of questions, we do not have any VQA training data to train our own model, and (2) recent work has demonstrated promising performance of zero-shot VQA. Specifically, we prompt the PVLM with 𝐾 probing questions and regard the set of 𝐾 answers from the PVLM as image captions, which we refer to as Pro-Cap. We then combine the original text T with Pro-Cap as input to a hateful meme detection model. We experiment with two alternative hateful meme detection models: one based on BERT encoding, and the other based on PromptHate, a recently proposed prompting-based hateful meme detection model. In the rest of this section, we first present the details of how we design our VQA questions to elicit the most critical details of an image for hateful meme detection. We then explain how the generated Pro-Cap is used by two alternative hateful meme detection models. 4.2 Design of VQA Questions We leverage PVLMs for zero-shot VQA to generate Pro-Cap as image captions. We want Pro-Cap to provide not only a general description of the image but also details critical for hateful meme detection. To obtain a general caption of the image, we design the first probing question to inquire about the generic content of the image, as shown in Table 2. However, such generic captions may be insufficient for hateful meme detection as hateful content usually targets persons or groups with specific characteristics, such as race, gender, or religion [5, 12]. Additionally, previous studies have shown that augmenting image representations with entities found in the image or demographic information of people in the image significantly aids hateful meme detection [14, 37]. Such details may be missing in generic image captions. Therefore, we design additional questions that aim to bring out information central to hateful content. This aligns the generated image captions more closely with the goal of hateful meme detection. Specifically, the high-level idea is to ask questions about common vulnerable targets of hateful content. Inspired by [24], which categorizes the targets of hateful memes into Religion, Race, Gender, Nationality, and Disability, we ask questions about these five types of targets. For example, to generate image captions that indicate the race of the people in an image, we can ask the following question: what is the race of persons in the image? We list the five questions designed for these five types of targets in Table 2. Additionally, we observe that some animals, such as pigs, are often depicted in hateful memes, frequently as a means to annoy Muslims. With this consideration, we also design a question asking about the presence of animals in the image. In [3], the author claimed that PVLMs may hallucinate nonexistent objects. For example, even when there is nobody in an image, PVLMs may generate an answer about race in response to the question what is the race of the person in the image?. To prevent such misleading information, we use two validation questions. Specifically, we inquire about the existence of persons and animals. Only when the PVLM responds that a person or an animal exists will we include in the Pro-Cap the answers to those person-related or animal-related questions. For instance, if the answer to the question validating the existence of people indicates that nobody is present, we will ignore all answers from questions asking about religion, race, gender, nationality, and disability. We use C to represent the concatenation of the answers to the probing questions that are finally included as part of the Pro-Cap based on the validation results. We will then concatenate T and C together as input to a purely text-based hateful meme classification model, as shown at the bottom of Figure 1. 4.3 BERT-based Detection Model 4.4 PromptHate for Hateful Meme Detection Next, we introduce the second hateful meme classification model, PromptHate [2], which employs a prompt-based method to classify memes. PromptHate was developed to better leverage contextual background knowledge by prompting language models. Given a test meme, PromptHate first uses an image captioning model to obtain generic image captions. It then concatenates the meme text, the image captions, and a prompt template into S: It was [MASK]., to prompt a language model (LM) to predict whether the meme is hateful. Specifically, it compares the probability of the language model predicting [MASK] to be a positive word (e.g., good) given the context, versus the probability of predicting a negative word (e.g., bad). The approach also includes one positive and one negative example in the context, and [MASK] will be replaced by their respective label words. An overview of PromptHate is shown in Figure 2. For further details, please refer to [2]. In [2], PromptHate utilizes ClipCap [25] to generate image captions. In this work, we replace this with Pro-Cap C. We then represent every meme O as O = [T, C, S]. With these inputs, the language models (LMs), for instance, RoBERTa [21], generate confidence scores for the masked word over their vocabulary space, V: 4.5 Model Training and Prediction This paper is available on arxiv under CC 4.0 license. Authors: (1) Rui Cao, Singapore Management University; (2) Ming Shan Hee, Singapore University of Design and Technology; (3) Adriel Kuek, DSO National Laboratories; (4) Wen-Haw Chong, Singapore Management University; (5) Roy Ka-Wei Lee, Singapore University of Design and Technology (6) Jing Jiang, Singapore Management University. Authors: Authors: (1) Rui Cao, Singapore Management University; (2) Ming Shan Hee, Singapore University of Design and Technology; (3) Adriel Kuek, DSO National Laboratories; (4) Wen-Haw Chong, Singapore Management University; (5) Roy Ka-Wei Lee, Singapore University of Design and Technology (6) Jing Jiang, Singapore Management University. Table of Links Abstract and Introduction Abstract and Introduction Related Work Related Work Preliminary Preliminary Proposed Method Proposed Method Experiment Experiment Conclusion and References Conclusion and References Appendix Appendix 4 PROPOSED METHOD 4.1 Overview Recall that the key idea of our method is to elicit image details that are critical for hateful content detection, such as the gender and race of the people in the image. Because these details are not always included in automatically generated image captions, we propose relying on VQA to obtain such critical information, where the questions are carefully curated to elicit demographic and other relevant information. We opt to use zero-shot VQA because (1) for the intended type of questions, we do not have any VQA training data to train our own model, and (2) recent work has demonstrated promising performance of zero-shot VQA. Specifically, we prompt the PVLM with 𝐾 probing questions and regard the set of 𝐾 answers from the PVLM as image captions, which we refer to as Pro-Cap. We then combine the original text T with Pro-Cap as input to a hateful meme detection model. We experiment with two alternative hateful meme detection models: one based on BERT encoding, and the other based on PromptHate, a recently proposed prompting-based hateful meme detection model. probing questions Pro-Cap. T In the rest of this section, we first present the details of how we design our VQA questions to elicit the most critical details of an image for hateful meme detection. We then explain how the generated Pro-Cap is used by two alternative hateful meme detection models. 4.2 Design of VQA Questions We leverage PVLMs for zero-shot VQA to generate Pro-Cap as image captions. We want Pro-Cap to provide not only a general description of the image but also details critical for hateful meme detection. To obtain a general caption of the image, we design the first probing question to inquire about the generic content of the image, as shown in Table 2. However, such generic captions may be insufficient for hateful meme detection as hateful content usually targets persons or groups with specific characteristics, such as race, gender, or religion [5, 12]. Additionally, previous studies have shown that augmenting image representations with entities found in the image or demographic information of people in the image significantly aids hateful meme detection [14, 37]. Such details may be missing in generic image captions. Therefore, we design additional questions that aim to bring out information central to hateful content. This aligns the generated image captions more closely with the goal of hateful meme detection. Specifically, the high-level idea is to ask questions about common vulnerable targets of hateful content. Inspired by [24], which categorizes the targets of hateful memes into Religion, Race, Gender, Nationality, and Disability, we ask questions about these five types of targets. For example, to generate image captions that indicate the race of the people in an image, we can ask the following question: what is the race of persons in the image? We list the five questions designed for these five types of targets in Table 2. Additionally, we observe that some animals, such as pigs, are often depicted in hateful memes, frequently as a means to annoy Muslims. With this consideration, we also design a question asking about the presence of animals in the image. In [3], the author claimed that PVLMs may hallucinate nonexistent objects. For example, even when there is nobody in an image, PVLMs may generate an answer about race in response to the question what is the race of the person in the image?. To prevent such misleading information, we use two validation questions. Specifically, we inquire about the existence of persons and animals. Only when the PVLM responds that a person or an animal exists will we include in the Pro-Cap the answers to those person-related or animal-related questions. For instance, if the answer to the question validating the existence of people indicates that nobody is present, we will ignore all answers from questions asking about religion, race, gender, nationality, and disability. We use C to represent the concatenation of the answers to the probing questions that are finally included as part of the Pro-Cap based on the validation results. We will then concatenate T and C together as input to a purely text-based hateful meme classification model, as shown at the bottom of Figure 1. 4.3 BERT-based Detection Model 4.4 PromptHate for Hateful Meme Detection Next, we introduce the second hateful meme classification model, PromptHate [2], which employs a prompt-based method to classify memes. PromptHate was developed to better leverage contextual background knowledge by prompting language models. Given a test meme, PromptHate first uses an image captioning model to obtain generic image captions. It then concatenates the meme text, the image captions, and a prompt template into S: It was [MASK]., to prompt a language model (LM) to predict whether the meme is hateful. Specifically, it compares the probability of the language model predicting [MASK] to be a positive word (e.g., good) given the context, versus the probability of predicting a negative word (e.g., bad). The approach also includes one positive and one negative example in the context, and [MASK] will be replaced by their respective label words. An overview of PromptHate is shown in Figure 2. For further details, please refer to [2]. In [2], PromptHate utilizes ClipCap [25] to generate image captions. In this work, we replace this with Pro-Cap C. We then represent every meme O as O = [T, C, S]. With these inputs, the language models (LMs), for instance, RoBERTa [21], generate confidence scores for the masked word over their vocabulary space, V: 4.5 Model Training and Prediction This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Comparing Hateful Meme Detection Models: BERT-based vs. PromptHate with Pro-Cap

A Comparative Algorithm Audit of Conspiracies on the Net: Conclusion and Bibliography

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Aleksandra Urman, She is a corresponding author from Department of Informatics, University of Zurich, Switzerland;
(2) Mykola Makhortykh, Institute of Communication and Media Studies, University of Bern, Switzerland;
(3) Roberto Ulloa, GESIS - Leibniz-Institut für Sozialwissenschaften, Germany;
(4) Juhi Kulshrestha, Department of Politics and Public Administration, University of Konstanz, Germany. Table of Links Abstract and Introduction
Web search results: biases, inaccuracies and their consequences
Conspiracy theories in the online environments
Research questions
Methodology
Results
Discussion
Limitations
Conclusion and Bibliography Methodology Data collection Agent-based audit setup. To collect the data we used agent-based algorithm impact auditing (see Ulloa et al., 2021). This type of auditing relies on automated agents, i.e., software that simulates user browsing behavior (e.g. scrolling web pages) and records the outputs. The benefit of this approach is that it allows controlling for search personalization (Hannak et al., 2013) and randomization (Makhortykh et al., 2020). Unlike human agents, automated agents can be easily synchronized (i.e., to isolate the effect of time at which the search actions are conducted) and deployed in a controlled environment (e.g., a network of virtual machines using the same IP range, the same type of operating system (OS) and the same browsing software) to limit the effects of personalization based on user-specific factors (e.g. location or OS type). Our approach is comparative in the sense that we assess the outputs of several SEs across multiple locations, time periods, and queries. To increase the robustness of our observations, the data were collected over two collection rounds: March 18-19 and May 8-9 2021. From the technical standpoint, the data collection was organized as follows: each agent was made of two browser plugins (for either Chrome or Firefox desktop browsers). The first plugin (the bot) simulated human activity in the browser, in our case opening the search engine page, entering a query and scrolling down the first page of search outputs. The second plugin (the tracker) recorded html content appearing in the browser and sent it to the remote server. The agents were deployed via Amazon Elastic Compute Cloud (EC2). We focused our exploration to the five biggest SEs by market share on desktop devices worldwide: Google, Bing, Yahoo, Yandex and DuckDuckGo (Desktop Search Engine Market Share Worldwide, n.d.). During the first round of data collection (March), we deployed 60 agents per location (see below), whereas for the second round we decreased this number to 30 agents per location because of budgetary limitations. The agents were equally distributed between the SEs (i.e., 12 agents per engine for the first round and 6 agents per engine for the second round). The majority of agents were able to perform their planned routine; the only major exception was Yandex, where multiple agents were blocked because of the aggressive bot detection mechanisms (e.g., frequent captchas) utilized by the search engine (hence, the absence of results for Yandex for some search queries). Search query selection. We have collected search results for six queries that correspond to specific conspiracy theories (“flat earth”, “new world order”, “qanon”) or to subjects related to conspiracy theories (“9/11”, “illuminati”, “george soros”). We specifically included these two groups of queries, because we suggest that they might be entered by users with two different intentions: those corresponding to the names of conspiracies are more likely to be entered by people already interested in a given theory, while the latter - by people merely interested in a certain topic. Thus if conspiratorial content is returned for the latter queries, it might lead to users’ incidental exposure to conspiracies. Further, we expect to observe discrepancies between the two groups of queries with more conspiratorial content returned for the queries corresponding to specific theories and less for the subjects which are surrounded by conspiracy theories. Location selection. We analyze the data collected for the five SEs, across three locations: two in the US (Ohio and California) and one in the UK (London). Our selection of locations is determined in part by the availability of different EC2 servers (see the description of technical implementation of the data collection below). Also, we utilized locations where English is the official language for the purposes of comparability - e.g., to make sure that any results we retrieve are due to the differences in the location, not to data voids in a specific language. Finally, we selected Ohio and California due to the differences in the ideological orientations of the two states (while California is a solid “blue” (Democratic) state in the US, Ohio is a “swing” state; we included it since there are no solid “red” states where EC2 servers are located); the inclusion of the UK server allows us to check whether our observations are country-specific. This comparative approach allows us to go beyond single-country observations that are common for algorithm audit studies (e.g., Haim et al., 2018; Puschmann, 2019; Urman et al., 2021a). Data analysis After the data were collected, we extracted all unique URLs that appeared on the first page of search results for each engine and search query. We have focused on the first page of search results since people tend to perceive top search results as more credible and rarely look at/click on the results beyond the first page (Pan et al., 2007; Schultheiß et al., 2018; Unkel and Haas, 2017; Urman and Makhortykh, 2021). In total, there were 375 such unique URLs collected. The content under these URLs was then manually coded by two trained coders with the disagreements between the two resolved through consensus coding. The coding involved two variables, broadly corresponding to the RQs: conspiracy-related stance and type of source. The conspiracy-related stance variable included the following categories: ● Debunks conspiracy: the content under the URL lists argument(s) why a conspiracy theory is not  true ● No mention of conspiracy: there are no mentions of conspiracy theories in the source at all ● Promotes conspiracy: the content under the URL lists argument(s) why a conspiracy theory is true ● Mentions conspiracy: a conspiracy theory is mentioned, but there is no clear stance associated - e.g., no arguments for it being true or false OR arguments for it being true or false are presented in equal shares. The type of source variable included the following categories: ● Reference website: e.g., online encyclopedias such as Wikipedia or Britannica ● Media: any media organization regardless of its ideological position or themes, except purely scientific news-focused sites (see the next category ● Science: scientific repositories such as JSTOR or news sites devoted exclusively to science such as Science News ● Social media: e.g., Facebook pages, YouTube videos or Twitter accounts ● Conspiracy website: a website dedicated to the promotion of one specific conspiracy or of an array of conspiracy theories, such as a website of the Flat Earth Society ● Other: all other websites that do not fit into any of the previous categories To assess the prevalence of conspiratorial content in search results (RQ1), we calculated the share of results for each stance towards conspiracy theories per condition (i.e., query, engine, location and collection round). To assess the prioritization of source types in relation to conspiracy content (RQ2), we first calculated the shares of results for each type per condition, and then the share of results with different stances towards conspiracy theories for each source type per location-round pair. This paper is available on arxiv under CC 4.0 license. Authors: (1) Aleksandra Urman, She is a corresponding author from Department of Informatics, University of Zurich, Switzerland; (2) Mykola Makhortykh, Institute of Communication and Media Studies, University of Bern, Switzerland; (3) Roberto Ulloa, GESIS - Leibniz-Institut für Sozialwissenschaften, Germany; (4) Juhi Kulshrestha, Department of Politics and Public Administration, University of Konstanz, Germany. This paper is available on arxiv under CC 4.0 license. Authors: (1) Aleksandra Urman, She is a corresponding author from Department of Informatics, University of Zurich, Switzerland; (2) Mykola Makhortykh, Institute of Communication and Media Studies, University of Bern, Switzerland; (3) Roberto Ulloa, GESIS - Leibniz-Institut für Sozialwissenschaften, Germany; (4) Juhi Kulshrestha, Department of Politics and Public Administration, University of Konstanz, Germany. Table of Links Abstract and Introduction Web search results: biases, inaccuracies and their consequences Conspiracy theories in the online environments Research questions Methodology Results Discussion Limitations Conclusion and Bibliography Abstract and Introduction Abstract and Introduction Web search results: biases, inaccuracies and their consequences Web search results: biases, inaccuracies and their consequences Conspiracy theories in the online environments Conspiracy theories in the online environments Research questions Research questions Methodology Methodology Results Results Discussion Discussion Limitations Limitations Conclusion and Bibliography Conclusion and Bibliography Methodology Data collection Data collection Agent-based audit setup. To collect the data we used agent-based algorithm impact auditing (see Ulloa et al., 2021). This type of auditing relies on automated agents, i.e., software that simulates user browsing behavior (e.g. scrolling web pages) and records the outputs. The benefit of this approach is that it allows controlling for search personalization (Hannak et al., 2013) and randomization (Makhortykh et al., 2020). Unlike human agents, automated agents can be easily synchronized (i.e., to isolate the effect of time at which the search actions are conducted) and deployed in a controlled environment (e.g., a network of virtual machines using the same IP range, the same type of operating system (OS) and the same browsing software) to limit the effects of personalization based on user-specific factors (e.g. location or OS type). Our approach is comparative in the sense that we assess the outputs of several SEs across multiple locations, time periods, and queries. Agent-based audit setup. To increase the robustness of our observations, the data were collected over two collection rounds: March 18-19 and May 8-9 2021. From the technical standpoint, the data collection was organized as follows: each agent was made of two browser plugins (for either Chrome or Firefox desktop browsers). The first plugin (the bot) simulated human activity in the browser, in our case opening the search engine page, entering a query and scrolling down the first page of search outputs. The second plugin (the tracker) recorded html content appearing in the browser and sent it to the remote server. The agents were deployed via Amazon Elastic Compute Cloud (EC2). We focused our exploration to the five biggest SEs by market share on desktop devices worldwide: Google, Bing, Yahoo, Yandex and DuckDuckGo (Desktop Search Engine Market Share Worldwide, n.d.). During the first round of data collection (March), we deployed 60 agents per location (see below), whereas for the second round we decreased this number to 30 agents per location because of budgetary limitations. The agents were equally distributed between the SEs (i.e., 12 agents per engine for the first round and 6 agents per engine for the second round). The majority of agents were able to perform their planned routine; the only major exception was Yandex, where multiple agents were blocked because of the aggressive bot detection mechanisms (e.g., frequent captchas) utilized by the search engine (hence, the absence of results for Yandex for some search queries). Search query selection. We have collected search results for six queries that correspond to specific conspiracy theories (“flat earth”, “new world order”, “qanon”) or to subjects related to conspiracy theories (“9/11”, “illuminati”, “george soros”). We specifically included these two groups of queries, because we suggest that they might be entered by users with two different intentions: those corresponding to the names of conspiracies are more likely to be entered by people already interested in a given theory, while the latter - by people merely interested in a certain topic. Thus if conspiratorial content is returned for the latter queries, it might lead to users’ incidental exposure to conspiracies. Further, we expect to observe discrepancies between the two groups of queries with more conspiratorial content returned for the queries corresponding to specific theories and less for the subjects which are surrounded by conspiracy theories. Search query selection. Location selection. We analyze the data collected for the five SEs, across three locations: two in the US (Ohio and California) and one in the UK (London). Our selection of locations is determined in part by the availability of different EC2 servers (see the description of technical implementation of the data collection below). Also, we utilized locations where English is the official language for the purposes of comparability - e.g., to make sure that any results we retrieve are due to the differences in the location, not to data voids in a specific language. Finally, we selected Ohio and California due to the differences in the ideological orientations of the two states (while California is a solid “blue” (Democratic) state in the US, Ohio is a “swing” state; we included it since there are no solid “red” states where EC2 servers are located); the inclusion of the UK server allows us to check whether our observations are country-specific. This comparative approach allows us to go beyond single-country observations that are common for algorithm audit studies (e.g., Haim et al., 2018; Puschmann, 2019; Urman et al., 2021a). Location selection. Data analysis Data analysis After the data were collected, we extracted all unique URLs that appeared on the first page of search results for each engine and search query. We have focused on the first page of search results since people tend to perceive top search results as more credible and rarely look at/click on the results beyond the first page (Pan et al., 2007; Schultheiß et al., 2018; Unkel and Haas, 2017; Urman and Makhortykh, 2021). In total, there were 375 such unique URLs collected. The content under these URLs was then manually coded by two trained coders with the disagreements between the two resolved through consensus coding. The coding involved two variables, broadly corresponding to the RQs: conspiracy-related stance and type of source. The conspiracy-related stance variable included the following categories: ● Debunks conspiracy: the content under the URL lists argument(s) why a conspiracy theory is not  true ● No mention of conspiracy: there are no mentions of conspiracy theories in the source at all ● Promotes conspiracy: the content under the URL lists argument(s) why a conspiracy theory is true ● Mentions conspiracy: a conspiracy theory is mentioned, but there is no clear stance associated - e.g., no arguments for it being true or false OR arguments for it being true or false are presented in equal shares. The type of source variable included the following categories: ● Reference website: e.g., online encyclopedias such as Wikipedia or Britannica ● Media: any media organization regardless of its ideological position or themes, except purely scientific news-focused sites (see the next category ● Science: scientific repositories such as JSTOR or news sites devoted exclusively to science such as Science News ● Social media: e.g., Facebook pages, YouTube videos or Twitter accounts ● Conspiracy website: a website dedicated to the promotion of one specific conspiracy or of an array of conspiracy theories, such as a website of the Flat Earth Society ● Other: all other websites that do not fit into any of the previous categories To assess the prevalence of conspiratorial content in search results (RQ1), we calculated the share of results for each stance towards conspiracy theories per condition (i.e., query, engine, location and collection round). To assess the prioritization of source types in relation to conspiracy content (RQ2), we first calculated the shares of results for each type per condition, and then the share of results with different stances towards conspiracy theories for each source type per location-round pair.

A Comparative Algorithm Audit of Conspiracies on the Net: Methodology

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Aleksandra Urman, She is a corresponding author from Department of Informatics, University of Zurich, Switzerland;
(2) Mykola Makhortykh, Institute of Communication and Media Studies, University of Bern, Switzerland;
(3) Roberto Ulloa, GESIS - Leibniz-Institut für Sozialwissenschaften, Germany;
(4) Juhi Kulshrestha, Department of Politics and Public Administration, University of Konstanz, Germany. Table of Links Abstract and Introduction
Web search results: biases, inaccuracies and their consequences
Conspiracy theories in the online environments
Research questions
Methodology
Results
Discussion
Limitations
Conclusion and Bibliography Limitations One limitation of the present study is that our selection of queries does not cover the whole range of existing conspiracy theories (Douglas et al., 2019) or all possible queries regarding a specific conspiracy theory, however, encompassing all theories or queries is arguably impossible, and our study should be treated as a first step towards more comprehensive analyses of conspiratorial information spread via SEs. Though the number of included theories is limited, we suggest that our approach is already more comprehensive than that of the majority of studies that focus on the spread of one specific theory online (with notable exceptions such as Mahl et al., 2021). As there is no established resource regarding the popularity of certain theories worldwide, our selection of queries was based on the internal discussions between authors, during which we tried to select the theories that are, first, in broad circulation and, second, that are “popular” worldwide, not within a single country. Another limitation of the present study is our focus on just three anglophone Western locations and on one language - English. This makes our analysis Western- and English-centric. We aim to address this in future work by conducting studies on the topic with broader linguistic and geographic focus. Finally, while we collected the data over two collection rounds to increase the robustness of the findings, these rounds were relatively close to each other. In the future, research infrastructure that enables more longitudinal data collection will be established to trace long-term changes (or their lack) in SEs’ outputs. This paper is available on arxiv under CC 4.0 license. Authors: (1) Aleksandra Urman, She is a corresponding author from Department of Informatics, University of Zurich, Switzerland; (2) Mykola Makhortykh, Institute of Communication and Media Studies, University of Bern, Switzerland; (3) Roberto Ulloa, GESIS - Leibniz-Institut für Sozialwissenschaften, Germany; (4) Juhi Kulshrestha, Department of Politics and Public Administration, University of Konstanz, Germany. This paper is available on arxiv under CC 4.0 license. Authors: (1) Aleksandra Urman, She is a corresponding author from Department of Informatics, University of Zurich, Switzerland; (2) Mykola Makhortykh, Institute of Communication and Media Studies, University of Bern, Switzerland; (3) Roberto Ulloa, GESIS - Leibniz-Institut für Sozialwissenschaften, Germany; (4) Juhi Kulshrestha, Department of Politics and Public Administration, University of Konstanz, Germany. Table of Links Abstract and Introduction Web search results: biases, inaccuracies and their consequences Conspiracy theories in the online environments Research questions Methodology Results Discussion Limitations Conclusion and Bibliography Abstract and Introduction Abstract and Introduction Web search results: biases, inaccuracies and their consequences Web search results: biases, inaccuracies and their consequences Conspiracy theories in the online environments Conspiracy theories in the online environments Research questions Research questions Methodology Methodology Results Results Discussion Discussion Limitations Limitations Conclusion and Bibliography Conclusion and Bibliography Limitations One limitation of the present study is that our selection of queries does not cover the whole range of existing conspiracy theories (Douglas et al., 2019) or all possible queries regarding a specific conspiracy theory, however, encompassing all theories or queries is arguably impossible, and our study should be treated as a first step towards more comprehensive analyses of conspiratorial information spread via SEs. Though the number of included theories is limited, we suggest that our approach is already more comprehensive than that of the majority of studies that focus on the spread of one specific theory online (with notable exceptions such as Mahl et al., 2021). As there is no established resource regarding the popularity of certain theories worldwide, our selection of queries was based on the internal discussions between authors, during which we tried to select the theories that are, first, in broad circulation and, second, that are “popular” worldwide, not within a single country. Another limitation of the present study is our focus on just three anglophone Western locations and on one language - English. This makes our analysis Western- and English-centric. We aim to address this in future work by conducting studies on the topic with broader linguistic and geographic focus. Finally, while we collected the data over two collection rounds to increase the robustness of the findings, these rounds were relatively close to each other. In the future, research infrastructure that enables more longitudinal data collection will be established to trace long-term changes (or their lack) in SEs’ outputs.

A Comparative Algorithm Audit of Conspiracies on the Net: Limitations

A Comparative Algorithm Audit of Conspiracies on the Net: Results

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Aleksandra Urman, She is a corresponding author from Department of Informatics, University of Zurich, Switzerland;
(2) Mykola Makhortykh, Institute of Communication and Media Studies, University of Bern, Switzerland;
(3) Roberto Ulloa, GESIS - Leibniz-Institut für Sozialwissenschaften, Germany;
(4) Juhi Kulshrestha, Department of Politics and Public Administration, University of Konstanz, Germany. Table of Links Abstract and Introduction
Web search results: biases, inaccuracies and their consequences
Conspiracy theories in the online environments
Research questions
Methodology
Results
Discussion
Limitations
Conclusion and Bibliography Discussion We have analyzed the presence of conspiratorial content in top (desktop) web search results across the five most popular SEs, three locations and two time periods for six conspiracy-related queries. Our findings contribute to both, the scholarship on inaccuracies and bias in web search, and research on the spread of conspiracy theories online. We also expect our observations to be robust considering there were only minor differences between locations and observation rounds. Our analysis demonstrates that while there are large query-based differences, conspiracy-promoting content routinely resurfaces in the first page of results on all SEs except Google. Furthermore, Google’s results also contain the highest proportion of scientific sources debunking the conspiracy theories. We suggest that the absence of conspiratorial information in Google’s results is attributed to the company actively trying to reduce the share of misinformation and bias in its search outputs (Kayser-Bril, 2020). The company even explicitly covers the issue of conspiratorial content (including a specific example related to the 9/11 conspiracies) in its guidelines for search results’ quality raters (Google, 2021). Another finding concerns the types of web resources prioritized by different search engines. Besides Google, all other search engines we examined link to conspiracy theory-dedicated websites with Yandex being the engine with the highest share of links to conspiratorial content. At the same time, most content on all engines (except Yandex) comes from media and reference websites which tend to contain little conspiracy-promoting content. In line with the previous findings (e.g., Douglas et al., 2019), we observe that conspiratorial content is found mainly on dedicated niche websites or social media, but is rare on broad-reaching sites such as media, scientific and reference websites. The latter observation highlights that one of the ways to cull dissemination of conspiratorial information on search engines can be to implement domain-based filters de-prioritizing resources known to be involved in promoting such information, even though the long-term resilience of this solution can be questioned. Filtering out conspiracy-promoting websites from the top search outputs is important for (at least) two reasons. While it can be assumed that only users who are already interested in conspiratorial content purposefully navigate to the dedicated niche websites (Douglas et al., 2019), their appearance in top search results, especially for queries that do not denote conspiracy theories per se (e.g., “9/11” or “george soros”) can potentially lead to incidental exposure to conspiracy theories. This is troubling: given people’s high trust in search results (2021 Edelman Trust Barometer | Edelman, n.d.), conspiratorial information found by users merely exploring the topic in top search results can induce the formation of conspiratorial beliefs. This might be especially the case if users have limited knowledge on a given topic - which is partially implied by them turning to SEs to explore it - as lower knowledge on a subject is associated with higher beliefs in related conspiracy theories (Sallam et al., 2020). In the cases when the exposure to conspiratorial content via search engines is not incidental - i.e., if a person already interested in a conspiracy theory searches for related content online (e.g., “flat earth” or “illuminati”) - the appearance of conspiracy-promoting information in top search results is also concerning. If such a person is simply interested - but not yet believing - in conspiracy theories, conspiratorial content coming from a highly trusted source can foster conspiratorial belief development. If that person already, at least partially, believes in a conspiracy theory, their belief can be reinforced by conspiracy-promoting web search results due to high public trust in SE results coupled with confirmation bias (Knobloch-Westerwick et al., 2015; Nichols, 2017; Suzuki and Yamamoto, 2020). Due to the latter, the presence of even a single conspiracy-promoting result in top results might be enough to reinforce conspiracy beliefs. This paper is available on arxiv under CC 4.0 license. Authors: (1) Aleksandra Urman, She is a corresponding author from Department of Informatics, University of Zurich, Switzerland; (2) Mykola Makhortykh, Institute of Communication and Media Studies, University of Bern, Switzerland; (3) Roberto Ulloa, GESIS - Leibniz-Institut für Sozialwissenschaften, Germany; (4) Juhi Kulshrestha, Department of Politics and Public Administration, University of Konstanz, Germany. This paper is available on arxiv under CC 4.0 license. Authors: (1) Aleksandra Urman, She is a corresponding author from Department of Informatics, University of Zurich, Switzerland; (2) Mykola Makhortykh, Institute of Communication and Media Studies, University of Bern, Switzerland; (3) Roberto Ulloa, GESIS - Leibniz-Institut für Sozialwissenschaften, Germany; (4) Juhi Kulshrestha, Department of Politics and Public Administration, University of Konstanz, Germany. Table of Links Abstract and Introduction Web search results: biases, inaccuracies and their consequences Conspiracy theories in the online environments Research questions Methodology Results Discussion Limitations Conclusion and Bibliography Abstract and Introduction Abstract and Introduction Web search results: biases, inaccuracies and their consequences Web search results: biases, inaccuracies and their consequences Conspiracy theories in the online environments Conspiracy theories in the online environments Research questions Research questions Methodology Methodology Results Results Discussion Discussion Limitations Limitations Conclusion and Bibliography Conclusion and Bibliography Discussion We have analyzed the presence of conspiratorial content in top (desktop) web search results across the five most popular SEs, three locations and two time periods for six conspiracy-related queries. Our findings contribute to both, the scholarship on inaccuracies and bias in web search, and research on the spread of conspiracy theories online. We also expect our observations to be robust considering there were only minor differences between locations and observation rounds. Our analysis demonstrates that while there are large query-based differences, conspiracy-promoting content routinely resurfaces in the first page of results on all SEs except Google. Furthermore, Google’s results also contain the highest proportion of scientific sources debunking the conspiracy theories. We suggest that the absence of conspiratorial information in Google’s results is attributed to the company actively trying to reduce the share of misinformation and bias in its search outputs (Kayser-Bril, 2020). The company even explicitly covers the issue of conspiratorial content (including a specific example related to the 9/11 conspiracies) in its guidelines for search results’ quality raters (Google, 2021). Another finding concerns the types of web resources prioritized by different search engines. Besides Google, all other search engines we examined link to conspiracy theory-dedicated websites with Yandex being the engine with the highest share of links to conspiratorial content. At the same time, most content on all engines (except Yandex) comes from media and reference websites which tend to contain little conspiracy-promoting content. In line with the previous findings (e.g., Douglas et al., 2019), we observe that conspiratorial content is found mainly on dedicated niche websites or social media, but is rare on broad-reaching sites such as media, scientific and reference websites. The latter observation highlights that one of the ways to cull dissemination of conspiratorial information on search engines can be to implement domain-based filters de-prioritizing resources known to be involved in promoting such information, even though the long-term resilience of this solution can be questioned. Filtering out conspiracy-promoting websites from the top search outputs is important for (at least) two reasons. While it can be assumed that only users who are already interested in conspiratorial content purposefully navigate to the dedicated niche websites (Douglas et al., 2019), their appearance in top search results, especially for queries that do not denote conspiracy theories per se (e.g., “9/11” or “george soros”) can potentially lead to incidental exposure to conspiracy theories. This is troubling: given people’s high trust in search results (2021 Edelman Trust Barometer | Edelman, n.d.), conspiratorial information found by users merely exploring the topic in top search results can induce the formation of conspiratorial beliefs. This might be especially the case if users have limited knowledge on a given topic - which is partially implied by them turning to SEs to explore it - as lower knowledge on a subject is associated with higher beliefs in related conspiracy theories (Sallam et al., 2020). In the cases when the exposure to conspiratorial content via search engines is not incidental - i.e., if a person already interested in a conspiracy theory searches for related content online (e.g., “flat earth” or “illuminati”) - the appearance of conspiracy-promoting information in top search results is also concerning. If such a person is simply interested - but not yet believing - in conspiracy theories, conspiratorial content coming from a highly trusted source can foster conspiratorial belief development. If that person already, at least partially, believes in a conspiracy theory, their belief can be reinforced by conspiracy-promoting web search results due to high public trust in SE results coupled with confirmation bias (Knobloch-Westerwick et al., 2015; Nichols, 2017; Suzuki and Yamamoto, 2020). Due to the latter, the presence of even a single conspiracy-promoting result in top results might be enough to reinforce conspiracy beliefs.

A Comparative Algorithm Audit of Conspiracies on the Net: Discussion

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Aleksandra Urman, She is a corresponding author from Department of Informatics, University of Zurich, Switzerland;
(2) Mykola Makhortykh, Institute of Communication and Media Studies, University of Bern, Switzerland;
(3) Roberto Ulloa, GESIS - Leibniz-Institut für Sozialwissenschaften, Germany;
(4) Juhi Kulshrestha, Department of Politics and Public Administration, University of Konstanz, Germany. Table of Links Abstract and Introduction
Web search results: biases, inaccuracies and their consequences
Conspiracy theories in the online environments
Research questions
Methodology
Results
Discussion
Limitations
Conclusion and Bibliography Research questions Based on the research findings outlined in the previous section, we suggest that SEs can be conducive to the distribution of conspiratorial information and formation of conspiracy beliefs should such information appear in top search results. Unlike social media or online blogs, web search results have not been examined in the context of the distribution of conspiratorial information - despite numerous analyses showing that biased or not fully accurate results are not uncommon there. With this paper, we strive to partially fill the identified gaps in both - research focused on the presence of inaccurate information in web search and research on the spread of conspiracy theories online. Hence, our first research question concerns the general prevalence of conspiratorial information in top search results. RQ1: How prevalent is conspiratorial information in web search results? Because there are major discrepancies in the information provided by different SEs (e.g., Kravets and Toepfl, 2021; Makhortykh et al., 2020; Paramita et al., 2021), and in the results retrieved from different locations (Kliman-Silver et al., 2015), we find it worthwhile to address RQ1 from a comparative perspective to increase the robustness of the findings. Specifically, we compare the observations across different locations and on different search engines using the data collected at two different points in time. Further, we select two groups of conspiracy-related search queries, and expect to observe differences between the results retrieved for them, adding another comparative dimension to the analysis. Thus, we pose three sub-RQs in relation to RQ1: RQ1a: Does the prevalence of conspiratorial information in search results vary across different locations and time periods? RQ1b: Does the prevalence of conspiratorial information in search results vary across SEs, and how? RQ1c: Does the prevalence of conspiratorial information in search results vary across search queries, and how? Research, on one hand, demonstrates that SEs tend to over-represent certain source types while under-representing others (e.g., Puschmann, 2019; Haim et al., 2018), and, on another hand, suggests that conspiracy theories are mainly propagated on social media or small conspiracy-related websites (Douglas et al., 2019; Stano, 2020). Hence, we juxtapose the presence of conspiracies in web search with the prioritization of different sources and pose the following RQ: RQ2: What types of sources are prioritized by web SEs in relation to conspiracy theories? In the case of RQ2, we will also adopt a comparative approach similar to that of RQ1 - comparing the results for different engines, locations, waves and queries: RQ2a: Does the prevalence of sources of different types in search results vary across different locations and time periods? RQ2b: Does the prevalence of sources of different types in search results vary across SEs, and how? RQ2c: Does the prevalence of sources of different types in search results vary across search queries, and how? Additionally, we examine the relationship between the source type and the stance towards conspiracy theories. To do it, we formulate the last research question: RQ3: Are there differences in the share of conspiracy-promoting or conspiracy-debunking content coming from sources of different types? This paper is available on arxiv under CC 4.0 license. Authors: (1) Aleksandra Urman, She is a corresponding author from Department of Informatics, University of Zurich, Switzerland; (2) Mykola Makhortykh, Institute of Communication and Media Studies, University of Bern, Switzerland; (3) Roberto Ulloa, GESIS - Leibniz-Institut für Sozialwissenschaften, Germany; (4) Juhi Kulshrestha, Department of Politics and Public Administration, University of Konstanz, Germany. This paper is available on arxiv under CC 4.0 license. Authors: (1) Aleksandra Urman, She is a corresponding author from Department of Informatics, University of Zurich, Switzerland; (2) Mykola Makhortykh, Institute of Communication and Media Studies, University of Bern, Switzerland; (3) Roberto Ulloa, GESIS - Leibniz-Institut für Sozialwissenschaften, Germany; (4) Juhi Kulshrestha, Department of Politics and Public Administration, University of Konstanz, Germany. Table of Links Abstract and Introduction Web search results: biases, inaccuracies and their consequences Conspiracy theories in the online environments Research questions Methodology Results Discussion Limitations Conclusion and Bibliography Abstract and Introduction Abstract and Introduction Web search results: biases, inaccuracies and their consequences Web search results: biases, inaccuracies and their consequences Conspiracy theories in the online environments Conspiracy theories in the online environments Research questions Research questions Methodology Methodology Results Results Discussion Discussion Limitations Limitations Conclusion and Bibliography Conclusion and Bibliography Research questions Based on the research findings outlined in the previous section, we suggest that SEs can be conducive to the distribution of conspiratorial information and formation of conspiracy beliefs should such information appear in top search results. Unlike social media or online blogs, web search results have not been examined in the context of the distribution of conspiratorial information - despite numerous analyses showing that biased or not fully accurate results are not uncommon there. With this paper, we strive to partially fill the identified gaps in both - research focused on the presence of inaccurate information in web search and research on the spread of conspiracy theories online. Hence, our first research question concerns the general prevalence of conspiratorial information in top search results. RQ1: How prevalent is conspiratorial information in web search results? RQ1: How prevalent is conspiratorial information in web search results? Because there are major discrepancies in the information provided by different SEs (e.g., Kravets and Toepfl, 2021; Makhortykh et al., 2020; Paramita et al., 2021), and in the results retrieved from different locations (Kliman-Silver et al., 2015), we find it worthwhile to address RQ1 from a comparative perspective to increase the robustness of the findings. Specifically, we compare the observations across different locations and on different search engines using the data collected at two different points in time. Further, we select two groups of conspiracy-related search queries, and expect to observe differences between the results retrieved for them, adding another comparative dimension to the analysis. Thus, we pose three sub-RQs in relation to RQ1: RQ1a: Does the prevalence of conspiratorial information in search results vary across different locations and time periods? RQ1a: Does the prevalence of conspiratorial information in search results vary across different locations and time periods? RQ1b: Does the prevalence of conspiratorial information in search results vary across SEs, and how? RQ1b: Does the prevalence of conspiratorial information in search results vary across SEs, and how? RQ1c: Does the prevalence of conspiratorial information in search results vary across search queries, and how? RQ1c: Does the prevalence of conspiratorial information in search results vary across search queries, and how? Research, on one hand, demonstrates that SEs tend to over-represent certain source types while under-representing others (e.g., Puschmann, 2019; Haim et al., 2018), and, on another hand, suggests that conspiracy theories are mainly propagated on social media or small conspiracy-related websites (Douglas et al., 2019; Stano, 2020). Hence, we juxtapose the presence of conspiracies in web search with the prioritization of different sources and pose the following RQ: RQ2: What types of sources are prioritized by web SEs in relation to conspiracy theories? RQ2: What types of sources are prioritized by web SEs in relation to conspiracy theories? In the case of RQ2, we will also adopt a comparative approach similar to that of RQ1 - comparing the results for different engines, locations, waves and queries: RQ2a: Does the prevalence of sources of different types in search results vary across different locations and time periods? RQ2a: Does the prevalence of sources of different types in search results vary across different locations and time periods? RQ2b: Does the prevalence of sources of different types in search results vary across SEs, and how? RQ2b: Does the prevalence of sources of different types in search results vary across SEs, and how? RQ2c: Does the prevalence of sources of different types in search results vary across search queries, and how? RQ2c: Does the prevalence of sources of different types in search results vary across search queries, and how? Additionally, we examine the relationship between the source type and the stance towards conspiracy theories. To do it, we formulate the last research question: RQ3: Are there differences in the share of conspiracy-promoting or conspiracy-debunking content coming from sources of different types? RQ3: Are there differences in the share of conspiracy-promoting or conspiracy-debunking content coming from sources of different types?

COMPA

trycompa.com

-- employees

Since n.d.

COMPANY RANKING

COMPA

EVERGREEN INDEX #2679

Representing this company? Click here to claim and customize this page!

Compa WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

COMPA

trycompa.com

-- employees

Since n.d.

COMPANY RANKING

COMPA

EVERGREEN INDEX #2679

RELATED COMPANIESRANK

STORIES ABOUT COMPA

Representing this company? Click here to claim and customize this page!

Compa WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!