A Comparative Algorithm Audit of Conspiracies on the Net: Limitations

Table of Links

• Abstract and Introduction
• Web search results: biases, inaccuracies and their consequences
• Conspiracy theories in the online environments
• Research questions
• Methodology
• Results
• Discussion
• Limitations
• Conclusion and Bibliography

Limitations

One limitation of the present study is that our selection of queries does not cover the whole range of existing conspiracy theories (Douglas et al., 2019) or all possible queries regarding a specific conspiracy theory, however, encompassing all theories or queries is arguably impossible, and our study should be treated as a first step towards more comprehensive analyses of conspiratorial information spread via SEs. Though the number of included theories is limited, we suggest that our approach is already more comprehensive than that of the majority of studies that focus on the spread of one specific theory online (with notable exceptions such as Mahl et al., 2021). As there is no established resource regarding the popularity of certain theories worldwide, our selection of queries was based on the internal discussions between authors, during which we tried to select the theories that are, first, in broad circulation and, second, that are “popular” worldwide, not within a single country.

Another limitation of the present study is our focus on just three anglophone Western locations and on one language - English. This makes our analysis Western- and English-centric. We aim to address this in future work by conducting studies on the topic with broader linguistic and geographic focus.

Finally, while we collected the data over two collection rounds to increase the robustness of the findings, these rounds were relatively close to each other. In the future, research infrastructure that enables more longitudinal data collection will be established to trace long-term changes (or their lack) in SEs’ outputs.