NewsUnfold Scores High on Ease of Use as Testers Call for Smoother Feedback Flow

Table of Links

B. Detailed UX Survey Results for NewsUnfold

How did you like NewsUnfold? (10 responses) Six participants expressed a strongly positive sentiment, stating, for instance, that they found it innovative. Two expressed that the bias detection might need some calibration, one found it ”okay”, and one remained unsure.

Ease of Use: How easy was NewsUnfold to use? (13 responses) Participants were asked to rate the ease of use of NewsUnfold on a 10-point scale, with 10 indicating high ease of use. The average rating for ease of use was 8.46, with a median score of 9, implying that users found NewsUnfold user-friendly and intuitive.

How did NewsUnfold impact your reading? (12 responses) 6 participants stated it made them read more carefully, critically, and slowly instead of skimming. Two stated bias was easier to recognize because they had to think twice. One said they did more active thinking about what bias is. One didn’t feel much impact in unbiased articles. One wanted it in all of their browsing. One said it made them argue with the AI instead of skimming the article.

How did you feel about giving feedback on the sentences? (11 responses) Three participants found it easy to give feedback, while two reported it felt either difficult because it disrupted their reading flow or because it felt like a chore. Two participants felt unsure, with one skipping the tutorial. Two reported only doing it when they would have more time. One stated only to give feedback when disagreeing with the classification. One participant appreciated sharing their reasoning in the free-text field 3.

How do you feel about the highlights in the text? (10 responses) Nine participants liked the highlights and found them helpful, one calling it their favorite part. However, one participant found it distracting and raised concerns about highlighting quotes as biased.

Net Promoter Score (NPS): How likely would you recommend NewsUnfold to a friend, family, or colleague? (13 responses) The calculated mean NPS was 6.23. This score indicates participants were neutral to slightly in favor of recommending NewsUnfold.

How do you like the User Interface of NewsUnfold? (11 responses) 9 participants found it easy and clean, with one stating the ”look is sleek and appropriate for a modern website”. One participant experienced a bug using Firefox on mobile and described it as a ”bit sluggish.” One participant found the UI ”a bit bland.”

What irritated you? Did you encounter any problems? (10 responses) Bugs and irritations included the character limit in the free-text field , the multiple steps in the feedback window on mobile devices, overlays blocking the text on Firefox mobile, out-of-line tooltips, and encountering jumping buttons. One person expressed a slight annoyance in instances they disagreed with the classifier. One per- son was confused because they skipped the tutorial. Two didn’t encounter problems.

Anything else you want to share with us? (1 response) One participant suggested that it might be interesting to add a note indicating that direct quotes are more likely to be biased and may not necessarily reflect the opinions of the authors.

C. Material Bias and Demographics of Feedback Mechanism Study

D Additional Screenshots

Authors:

(1) Smi Hinterreiter;

(2) Martin Wessel;

(3) Fabian Schliski;

(4) Isao Echizen;

(5) Marc Erich Latoschik;

(6) Timo Spinde.

This paper is available on arxiv under CC0 1.0 license.