This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Aleksandra Urman, She is a corresponding author from Department of Informatics, University of Zurich, Switzerland;
(2) Mykola Makhortykh, Institute of Communication and Media Studies, University of Bern, Switzerland;
(3) Roberto Ulloa, GESIS - Leibniz-Institut für Sozialwissenschaften, Germany;
(4) Juhi Kulshrestha, Department of Politics and Public Administration, University of Konstanz, Germany. Table of Links Abstract and Introduction
Web search results: biases, inaccuracies and their consequences
Conspiracy theories in the online environments
Research questions
Methodology
Results
Discussion
Limitations
Conclusion and Bibliography Web search results: biases, inaccuracies and their consequences In recent years a growing number of studies, primarily from the fields of computer science and communication science, have examined the presence of various biases in search results. To do so, researchers usually rely on a methodology called algorithm impact auditing - an investigation of the outputs of an algorithmic system - such as SE’s information retrieval and ranking algorithms - with the aim to assess whether they contain biases or inaccuracies (Mittelstadt, 2016; see Bandy, 2021 for an overview of audits across different domains). Algorithm audits in the context of web search have been largely focused on how specific biases lead to distortions in search results (Bandy, 2021). Scholars have found evidence of gender bias in image search (Kay et al., 2015; Otterbacher et al., 2017; Makhortykh et al., 2021a) and in Google’s Knowledge Graph Carousel (Lipani et al., 2021), political bias in text search (Kravets and Toepfl, 2021; Kulshrestha et al., 2019), biases related to historical content in image search (Makhortykh et al., 2021b), and various forms of source-related biases (Diakopoulos et al., 2018; Haim et al., 2018; Kravets and Toepfl, 2021; Makhortykh et al., 2020; Puschmann, 2019; Trielli and Diakopoulos, 2019; Unkel and Haim, 2021; Urman et al., 2021a, 2021b). Further, several studies have documented information inequalities that can arise not only between users of different SEs but also between the users of the same SE due to search personalization or randomization (Hannak et al., 2013; Kliman-Silver et al., 2015; Paramita et al., 2021). Importantly, these inequalities can pertain to information crucial for individual and societal well-being, for instance concerning suicide helplines (Haim et al., 2017) or COVID-19 (Makhortykh et al., 2020). While bias in web search received a lot of attention from scholars, there are only a few studies looking specifically at factually inaccurate information in search results (Bernstam et al., 2008; Bradshaw, 2019; Cooper and Feder, 2004). However, the possibility of web search promoting inaccurate information is concerning for a number of reasons. First, people highly trust the information they encounter through SEs, which are viewed as more reliable sources of information than legacy media (2021 Edelman Trust Barometer | Edelman, n.d.); consequently, the probability of individuals trusting inaccurate information acquired via SEs is high. Second, the mere act of using SEs increases individuals' confidence about their knowledge on the topic and motivates them to conflate external knowledge - that is the one available online - with their own (Ward, 2021). Third, people’s search behaviour and interpretation of results are subject to selective exposure and confirmation bias that leads to them perceiving incorrect information that aligns with their existing attitudes as reliable and dismissing the results that do not align with their attitudes (Knobloch-Westerwick et al., 2015; Nichols, 2017; Suzuki and Yamamoto, 2020). Fourth, search results can sway people’s opinions including on issues as important as voting preferences (Epstein and Robertson, 2015; Zweig, 2017) or commercial brand choices (Jansen et al., 2011). Fifth, inaccuracies in search results can proliferate to other socio-technical and informational systems that are built with reliance on SE outputs such as fake news detection algorithms relying on search outputs for the cues as to which information is false (Shim et al., 2021; Varshney and Vishwakarma, 2021). Considering the potential harm which can be caused by search bias and factually inaccurate information, we suggest that comprehensive auditing of web search is of no less importance than the critical examinations of content appearing on legacy or social media. We strive to add to the body of literature on web search malperformance through an analysis of presence of conspiratorial information in search results. As we highlight below, this is a highly relevant subject due to the role the internet plays in the proliferation of conspiracy theories. This paper is available on arxiv under CC 4.0 license. Authors: (1) Aleksandra Urman, She is a corresponding author from Department of Informatics, University of Zurich, Switzerland; (2) Mykola Makhortykh, Institute of Communication and Media Studies, University of Bern, Switzerland; (3) Roberto Ulloa, GESIS - Leibniz-Institut für Sozialwissenschaften, Germany; (4) Juhi Kulshrestha, Department of Politics and Public Administration, University of Konstanz, Germany. This paper is available on arxiv under CC 4.0 license. Authors: (1) Aleksandra Urman, She is a corresponding author from Department of Informatics, University of Zurich, Switzerland; (2) Mykola Makhortykh, Institute of Communication and Media Studies, University of Bern, Switzerland; (3) Roberto Ulloa, GESIS - Leibniz-Institut für Sozialwissenschaften, Germany; (4) Juhi Kulshrestha, Department of Politics and Public Administration, University of Konstanz, Germany. Table of Links Abstract and Introduction Web search results: biases, inaccuracies and their consequences Conspiracy theories in the online environments Research questions Methodology Results Discussion Limitations Conclusion and Bibliography Abstract and Introduction Abstract and Introduction Web search results: biases, inaccuracies and their consequences Web search results: biases, inaccuracies and their consequences Conspiracy theories in the online environments Conspiracy theories in the online environments Research questions Research questions Methodology Methodology Results Results Discussion Discussion Limitations Limitations Conclusion and Bibliography Conclusion and Bibliography Web search results: biases, inaccuracies and their consequences In recent years a growing number of studies, primarily from the fields of computer science and communication science, have examined the presence of various biases in search results. To do so, researchers usually rely on a methodology called algorithm impact auditing - an investigation of the outputs of an algorithmic system - such as SE’s information retrieval and ranking algorithms - with the aim to assess whether they contain biases or inaccuracies (Mittelstadt, 2016; see Bandy, 2021 for an overview of audits across different domains). Algorithm audits in the context of web search have been largely focused on how specific biases lead to distortions in search results (Bandy, 2021). Scholars have found evidence of gender bias in image search (Kay et al., 2015; Otterbacher et al., 2017; Makhortykh et al., 2021a) and in Google’s Knowledge Graph Carousel (Lipani et al., 2021), political bias in text search (Kravets and Toepfl, 2021; Kulshrestha et al., 2019), biases related to historical content in image search (Makhortykh et al., 2021b), and various forms of source-related biases (Diakopoulos et al., 2018; Haim et al., 2018; Kravets and Toepfl, 2021; Makhortykh et al., 2020; Puschmann, 2019; Trielli and Diakopoulos, 2019; Unkel and Haim, 2021; Urman et al., 2021a, 2021b). Further, several studies have documented information inequalities that can arise not only between users of different SEs but also between the users of the same SE due to search personalization or randomization (Hannak et al., 2013; Kliman-Silver et al., 2015; Paramita et al., 2021). Importantly, these inequalities can pertain to information crucial for individual and societal well-being, for instance concerning suicide helplines (Haim et al., 2017) or COVID-19 (Makhortykh et al., 2020). While bias in web search received a lot of attention from scholars, there are only a few studies looking specifically at factually inaccurate information in search results (Bernstam et al., 2008; Bradshaw, 2019; Cooper and Feder, 2004). However, the possibility of web search promoting inaccurate information is concerning for a number of reasons. First, people highly trust the information they encounter through SEs, which are viewed as more reliable sources of information than legacy media (2021 Edelman Trust Barometer | Edelman, n.d.); consequently, the probability of individuals trusting inaccurate information acquired via SEs is high. Second, the mere act of using SEs increases individuals' confidence about their knowledge on the topic and motivates them to conflate external knowledge - that is the one available online - with their own (Ward, 2021). Third, people’s search behaviour and interpretation of results are subject to selective exposure and confirmation bias that leads to them perceiving incorrect information that aligns with their existing attitudes as reliable and dismissing the results that do not align with their attitudes (Knobloch-Westerwick et al., 2015; Nichols, 2017; Suzuki and Yamamoto, 2020). Fourth, search results can sway people’s opinions including on issues as important as voting preferences (Epstein and Robertson, 2015; Zweig, 2017) or commercial brand choices (Jansen et al., 2011). Fifth, inaccuracies in search results can proliferate to other socio-technical and informational systems that are built with reliance on SE outputs such as fake news detection algorithms relying on search outputs for the cues as to which information is false (Shim et al., 2021; Varshney and Vishwakarma, 2021). Considering the potential harm which can be caused by search bias and factually inaccurate information, we suggest that comprehensive auditing of web search is of no less importance than the critical examinations of content appearing on legacy or social media. We strive to add to the body of literature on web search malperformance through an analysis of presence of conspiratorial information in search results. As we highlight below, this is a highly relevant subject due to the role the internet plays in the proliferation of conspiracy theories.

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Web Search Results: Biases, Inaccuracies and Their Consequences

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

A Comparative Algorithm Audit: Conspiracy Theories in the Online Environments

A Comparative Algorithm Audit of Conspiracies on the Net: Abstract and Introduction

A Comparative Algorithm Audit: Conspiracy Theories in the Online Environments

A Comparative Algorithm Audit of Conspiracies on the Net: Research Questions

A Comparative Algorithm Audit of Conspiracies on the Net: Results

A Comparative Algorithm Audit of Conspiracies on the Net: Discussion

A Comparative Algorithm Audit: Conspiracy Theories in the Online Environments

A Comparative Algorithm Audit of Conspiracies on the Net: Abstract and Introduction

A Comparative Algorithm Audit: Conspiracy Theories in the Online Environments

A Comparative Algorithm Audit of Conspiracies on the Net: Research Questions

A Comparative Algorithm Audit of Conspiracies on the Net: Results

A Comparative Algorithm Audit of Conspiracies on the Net: Discussion

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps