We found banned items for sale on Amazon.com A timer started the moment he logged in at his desk. He’d scan a post of a product flagged by someone as counterfeit, search for facts, and decide if the item should be pulled from Amazon’s marketplace. He said he gave the entire investigation about three minutes—and then the timer reset. “You can smell it on the floor when you get there—the tension and the stress of everyone on this metric system and on this clock,” said the former Amazon investigator on the company’s marketplace abuse team. He left in 2018 and asked to not be named, fearing speaking publicly would hinder his employment in the industry. He and another former Amazon employee, Rachel Johnson Greer, who left the company in 2017, said the company required them to crank through about 20 review tasks an hour. “You go faster than 19 an hour so you have breaks,” said Greer, who reviewed customer orders for potential fraud after joining Amazon in 2007. Fall short of your quota, Greer said, and “you get put on a performance improvement plan … and you get managed out.” The barrage of suspicious cases these former employees rapidly reviewed are indicative of a larger problem: Bad actors are finding their way onto the platform, and it’s not clear Amazon has a workable strategy to put an end to the abuse. Last month, by The Markup found that Amazon failed to stop posts of products banned by its own rules. We filled a shopping cart with “marijuana bongs, ‘ ‘ used to inhale cannabis concentrates, ‘crackers’ that can be used to get high on nitrous oxide, and compounds that reviews showed were used as injectable drugs.” We provided Amazon with to each one of our findings, and most were removed. an investigation dab kits links Amazon is the largest retailer on the planet and an attractive target for and schemes and third-party sellers reportedly hawking dangerous baby , or labeled as the real thing. Its bookstore, the largest in the world, and its self-publishing business have also come under fire for distributing misinformation and . fraud fake review car seats expired food fake apple chargers white nationalist content YouTube, Facebook, and Twitter are publicly reckoning with calls to improve content moderation and have pledged to expand their armies of review professionals and be more transparent about how much they remove and why. Amazon, in contrast, has remained relatively silent and opaque. “Social platforms have spent the last four years thinking about how to handle misinformation,” said Renée DiResta, a technical research manager at the Stanford Internet Observatory, “where it doesn’t seem Amazon has.” Social platforms have spent the last four years thinking about how to handle misinformation, where it doesn’t seem Amazon has. Renée DiResta, Stanford Internet Observatory Facebook now has about 30,000 content moderators, and Twitter, a smaller company, doubled its content moderation workforce to 1,500 in recent years, according to .  In 2017, Susan Wojcicki, the CEO of YouTube, announced in that Google, in about offensive and violent content on YouTube, would expand its workforce responsible for reviewing content to more than 10,000 people. The Washington Post a blog post response to public outcry In a written statement to The Markup, Amazon spokesperson Patrick Graham declined to say specifically how many people review content or whether Amazon has increased that workforce in recent years. Graham said that in 2019, Amazon invested more than $500 million and more than 8,000 employees in “protecting our store from fraud and abuse.” That number could include executives, programmers, attorneys, or other employees. In a follow-up question, Graham declined to specify how many employees specifically moderate content. When asked about the hourly quota, Graham said the company prioritizes accuracy and measures productivity along with “a variety of dimensions to evaluate an employee’s overall job performance.” Graham also declined to comment on whether the company supported the , a set of voluntary content moderation practices that include publishing the number of post removals and account suspensions, notifying and providing explanations to impacted users, and implementing an appeal system. Santa Clara Principles Last year, the reported that Facebook, LinkedIn, Medium, Snap, Tumblr, and YouTube all support the initiative. Electronic Frontier Foundation In addition, , , and all issue transparency reports, though, as the advocacy group New America noted in of those reports last year, the companies failed to give a full picture of takedown figures. Facebook Twitter YouTube its assessment Amazon provides even less data to the public than its peers. In a earlier this year, the company stated it blocked more than six billion “suspected bad listings from being published.” But it declined to provide information to The Markup on the number of posts flagged, the number of accounts suspended, which rules were violated, or how they were flagged—the minimum information the Santa Clara Principles recommend be made public. report to Congress What we do know is that Amazon, like other major platforms, relies in part on automated tools to police its pages. its tools are capable of reviewing hundreds of millions of products in a matter of minutes. The company says Still, The Markup’s reporting shows Amazon’s system is hardly perfect. Some sellers appeared to try to evade Amazon’s automated tools with simple tricks: They avoided or misspelled words or incorrectly categorized the listing. In one instance, we found marijuana bongs listed as “vases” in home decor. Many were at the time of this writing. still up Amazon Bongs are on Amazon’s banned list, but many “vases” remain available for purchase. In an email to The Markup at the time, Amazon spokesperson Patrick Graham said the company has “proactive measures in place to prevent suspicious or prohibited products from being listed,” and that sellers are responsible for following the rules and choosing the correct category. “If products that are against our policies are found on our site, we immediately remove the listing, take action on the bad actor, and further improve our systems,” Graham said. Nevertheless, experts say, Amazon’s focus on growth makes abuse hard to stamp out entirely. “Their very business model is flawed … allowing anyone with very little vetting to sell pretty much anything—that’s the fundamental problem, ” said Natasha Tusikov, an assistant professor at York University in Toronto and author of “Chokepoints: Global Private Regulation of the Internet.” “It’s impossible to weed out all the people who are trying to get around government regulations or defraud people, because you require so very little of them when they sign up,” she said. Graham disagreed with this claim, stating “Amazon’s seller verification processes are designed to make it easy for honest sellers to set up an account quickly while thwarting attempts by fraudsters.” He added that the company collects data on new sellers and seeks to validate it, in part thorough machines. Concerns about how well Amazon oversees its content extend to the company’s massive bookstore and publishing arm. Earlier this year, ProPublica and The Atlantic found about in a reading room were self-published through Amazon. 200 book recommendations curated by and for white nationalists “Amazon, which in the United States controls around half of the market for all books, and close to 90% for e-books, has become a gateway for white supremacists to reach the American reading public,” the investigation stated. The article reported that several titles mentioned were removed from Kindle Direct Publishing. “As a bookseller, we believe that providing access to the written word is important,” Graham said in an email to The Markup, repeating the company’s response in the article. “We invest significant time and resources to ensure our guidelines are followed, and remove products that do not adhere to our guidelines.” Search for in Amazon’s bookstore, and anti-vaccination titles are still among the top results. childhood vaccines DiResta, of Stanford, found Amazon’s algorithm listed anti-vaccination literature as a “#1 Best Seller” in the categories such as Emergency Pediatrics and History of Medicine. It wasn’t until the 12th place on the list that a book countering the anti-vaccination point of view appeared, she said. “Curation algorithms are largely amoral,” DiResta wrote in a 2019 piece for Wired . “They’re engineered to show us things we are statistically likely to want to see, content that people similar to us have found engaging—even if it’s stuff that’s factually unreliable or potentially harmful.” about her findings in Amazon’s bookstore, and anti-vaccination titles are still among the top results. Search for childhood vaccines Adelin Cai, formerly of Pinterest and Twitter, said both human review and machine detection are critical in making platforms their best selves. These tools, she pointed out, can scan potentially harmful images “without subjecting [the staff] to a lot of the exposure to bad content.” But Cai doesn’t think automation will replace human judgment entirely, even as the industry matures and AI advances. “Nothing supersedes the ability of the human mind to be nimble and understand context,” Cai said. Just last month, Cai and Clara Tsao, a former Mozilla fellow and chief technology officer of a U.S. government interagency anti-violent-extremism task force, launched the first membership-based professional organization for the field of trust and safety, the (They also launched a corresponding foundation devoted to education, and research.) Trust and Safety Professional Association. case studies Google, AirBnB, Slack, and Facebook are just a few of the group’s star-studded list of inaugural funders. Amazon isn’t among them. Credits Originally published as " Unlike Other Tech Giants, Amazon Won’t Say How Many Workers Review Posts " with the Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license.

AirBnB

Amazon

Apple

Facebook

Google

Mozilla

Slack

Target

Twitter

YouTube

The Markup Investigation: Congressman Says that Google engineered a “Walled Garden”

Tell Me Your Secrets: How Many Workers Review Posts on Amazon?

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

A Conversation with Kashmir Hill: Envisioning a Chaotic Future

The Noonification: How Amazon Treats Warehouse Workers Who Contracted COVID (11/30/2022)

10 Free Ways to Promote Your Amazon Products

10 Failed Startup Product Examples by Google, Microsoft and Amazon

10 Best Infographics Of 2018

The Noonification: The Destroyer (12/29/2022)

A Conversation with Kashmir Hill: Envisioning a Chaotic Future

The Noonification: How Amazon Treats Warehouse Workers Who Contracted COVID (11/30/2022)

10 Free Ways to Promote Your Amazon Products

10 Failed Startup Product Examples by Google, Microsoft and Amazon

10 Best Infographics Of 2018

The Noonification: The Destroyer (12/29/2022)

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps