The conspiracy and disinformation challenge on e-commerce platforms

Following the Jan. 6 assault on the U.S. Capitol, social media companies came under intense scrutiny for their role in incubating the mob attack. The CEOs of Facebook, Google, and Twitter were hauled before Congress to testify, and congressional critics of the company cited the assault as the latest reason why these companies should be stripped of their liability protections. But amid this focus on the role of online platforms in fueling conspiracy theories about a stolen election, e-commerce platforms such as Amazon, Etsy, and eBay have escaped scrutiny. While Etsy banned QAnon-related merchandise in October and Amazon and eBay made the same commitment in late January, the success of these efforts, much like those of other social media platforms, has been mixed at best. 

E-commerce sites present novel challenges to tech companies, policymakers, and researchers concerned with the proliferation of mis- and disinformation online. The complex ecosystem of online commerce makes it comparably easy for individuals looking to sell merchandise related to conspiracy theories—QAnon shirts, supplements, vaccine-skeptic books—and difficult for companies to root out bad actors. The issue is exacerbated by the fusion of conspiratorial content with lifestyle branding on social media platforms and e-commerce sites. While social media platforms have made progress in reducing the amount of mis- and disinformation online, e-commerce platforms are relatively more laissez-faire, continuing to provide support to the conspiracy economy. 

The e-commerce misinformation challenge

Recent research suggests that serious mis- and disinformation problems persist on e-commerce platforms. In a study published earlier this year, Prerna Juneja and Tanushree Mitra, scholars at the University of Washington, searched nearly 4 dozen terms related to vaccine misinformation on Amazon. Their query produced 36,000 results and more than 16,000 recommendations. Of these search results, 10.47% (nearly 5,000 unique products) contained misinformation. Users who clicked on products containing misinformation or who added these products to their cart then received recommendations for misinformative products on their “Amazon homepages, product page recommendations, and pre-purchase recommendations.” A separate research consortium identified 20 books that questioned the origin or nature of the COVID-19 pandemic being sold by Amazon. As recently as last month, searches on Amazon revealed that QAnon-related merchandise is still available for purchase on the platform. An Amazon search for the QAnon slogan “wwg1wga” (an abbreviation for “Where we go one we go all”) turned up a shirt reading “Question Everything,” masks with “wwg1wga” printed on them, and “I took the Red Pill” stickers. The same query on eBay produced a list of products unrelated to QAnon, except for a pair of blue “wwg1wwa” wristbands. On Etsy, the phrase produced zero results.

While quantifying the value of the market for disinformation is difficult, it is clearly a major source of income for conspiracy theorists, hate groups, and disinformation media moguls. According to a July 2020 report by the Center for Countering Digital Hate (CCDH), the online audience for anti-vaccination content may be generating nearly $1 billion in revenue for social media companies and reaching an audience as large as 58 million people. A joint report published in October by the Institute for Strategic Dialogue (ISD) and the Global Disinformation Index (GDI) found that online retail is one of the most common funding sources for hate groups. The conspiracy theorist Alex Jones has made a fortune selling supplements and other merchandise on his own website, as well as e-commerce platforms. As of this month, Infowars supplements were still available for sale on Amazon. These examples, though somewhat anecdotal, suggest how valuable and nebulous the economy for conspiratorial merchandise is.

These mixed results, which could be repeated for any number of online retailers like Teespring, Zazzle, or Poshmark, demonstrate the unique difficulties of trying to scrub QAnon from e-commerce sites. Bracketing the preponderance of QAnon and other conspiratorial information still readily available on Amazon, retailers such as Etsy and eBay have for the most part successfully removed merchandise directly referencing QAnon. But QAnon sellers are clever, rebranding their goods with seemingly benign phrases. Last year, the QAnon community began circulating the hashtag #savethechildren (sometimes #saveourchildren), a reference to the community’s belief that high-profile Democrats and celebrities are secretly involved in child sex trafficking, to avoid algorithmic filters on social media sites. The hashtag has the benefit of sharing the name of a prominent international charity, and QAnon vendors on e-commerce sites use the phrase to hide conspiratorial content in plain sight. Search Etsy for this term, and many items appear. Are they connected to QAnon? Figuring that out requires a bit of digging.

Consider, for example, the seller WalkTheLineDesignUS who sells a digital download of a design for a “saveourchildren” t-shirt. While the item appears benign enough, its description contains the following: “***Please be informed that the Clintons own the hashtag #SaveOurChildren, these are the people we need to be saving them from and that is down right infuriating!” When searching “saveourchildren”, WalkTheLineUS’ design appears alongside other items bearing similar logos and graphics. While most of these items contain no overt references to QAnon or any other conspiracy theory, are their aesthetic similarities enough to connect these products to QAnon? If not, how can we identify them as problematic content? This is why scrubbing e-commerce platforms is so difficult. Oblique references, multiple text fields, and a diverse array of products render algorithmic identification imprecise and ineffective. Systemic identification of problematic content on e-commerce sites requires item-level analysis conducted by human teams.

However, like the major social-media platforms, e-commerce firms are moving toward AI and machine learning to identify problematic content. In their 2020 Transparency Report, Etsy notes 80% percent of the 4 million flags for problematic content received by the company came from internal tools. (The remaining 20% came from users.) According to the report, the company plans to expand its digital content-moderation tools, including the use of “auto-suppression, image recognition, and the ability to suppress listings geographically based on local requirements.” In an April blog post, the company committed to building “a dedicated trust and safety machine learning engineering team and exploring computer vision technology, with the goal of using powerful algorithms to drive improvements in the precision of automated risk detection.” While these commitments are laudable, they raise questions about how effective AI-driven approaches to content moderation can be when cracking down on ambiguous branding and rapidly fluctuating aesthetics in conspiratorial content. 

As long as conspiratorial, disinformative content persists on e-commerce platforms, these companies’ recommendation systems are likely to push such material to their users. As detailed in a recent study by the Institute for Strategic Dialogue, Amazon recommends products in three main ways: by displaying items that customers who viewed or bought a particular item also viewed or bought; by engaging in what is called “item-to-item collaborative filtering,” which uses an algorithm to elaborate relationships between products based on user preferences; and by recommending products through auto-complete search functions. For auto-complete suggestions for the word “vaccines,” ISD found that Amazon provides phrases such as “vaccines are dangerous” and “vaccines are the biggest medical fraud in history.” Together, these recommendations encourage users who show a slight interest in conspiracy-oriented material to become immersed in a commercialized, conspiratorial environment. 

Recommendation systems allow conspiratorial worldviews to cross-pollinate. For example, users viewing the Kindle edition of THE HAMMER is the Key to the Coup “The Political Crime of the Century”: How Obama, Brennan, Clapper, and the CIA spied on President Trump, General Flynn … and everyone else will find recommendations for books about the New World Order, health conspiracies, and the Illuminati.Site features such as product reviews can transport a budding conspiracist to a larger, unregulated economic space. Curious users might click a link in a customer review of a book containing vaccine misinformation that leads to a blog with its own online shop that sells QAnon content banned on Amazon. In a recent interview, the researcher Marc Tuters described the review spaces on Amazon “as a kind of social media in its own right.”

The interconnected nature of the conspiracy economy creates new mis- and disinformation problems, and nowhere is this process of economic interpenetration more evident than on Instagram, where influencer models, celebrities, and so-called “mommy bloggers” sell lifestyle products—alongside QAnon ideas. Over the course of the last year, the Instagram influencer Kim Cohen (@hofitkimcohen, 158,000 followers) began posting QAnon content. Cohen easily monetized her Instagram presence, with links to her blog, online store, and YouTube page. The influencer Rose Henges’ (@roseuncharted, 156,000 followers) uses her Instagram page to link to her Amazon storefront, where she recommends Miller’s Review of Critical Vaccine Studies, a book noted in Juneja and Mitra’s study for containing vaccine misinformation. 

Deplatforming these bad actors does not appear particularly successful in pushing them to the fringes of the web. Last year, Instagram deleted the account of influencer Rebecca Pfeiffer (formerly @luvbec, 160,000 followers) after she began promoting QAnon related content. She is now back on Instagram under the handle @luvbecstyle, with nearly 13,000 followers and a line in her bio announcing that Instagram deleted her previous account. 

These anecdotal accounts illustrate two important features of the mis- and disinformation economy. First, the universe of e-commerce is deeply interconnected. Social media influencers, recommendation systems, personal websites, and third-party sites act as the underlying infrastructure that allows the misinformation and disinformation industry to flourish. Second, rarely do these influencers openly peddle misinformation and disinformation through their online storefront (except for anti-vaccination books). These influencers market seemingly benign lifestyle products using calm, natural aesthetics, while espousing conspiratorial views in their Instagram stories. Enmeshing violent, hateful conspiratorial rhetoric in a peaceful, soft-palate web environment waters down that rhetoric, naturalizing it and making it seem a mundane facet of domestic bliss. “The original function of influencers was to be more relatable than mainstream media,” the scholar Sophie Bishop told the Atlantic. “They’re supposed to be presenting something that’s more authentic or more trustworthy or more embedded in reality” and by combining this mode of communication with problematic content they effectively launder “disinformation and dangerous ideas.”

Taking on disinformation in e-commerce

Addressing the commercial aspects of QAnon and other forms of mis- and disinformation, especially COVID-19-related material, requires a robust response. A full sketch of that response is beyond the scope of this piece, but two broad recommendations stand out.  

The first is a heightened focus on research and transparency. Good policy rests on high-quality information, yet when it comes to e-commerce and misinformation that remains in short supply. Policymakers and the tech sector should therefore invest in more academic research. Studies like those by Juneja and Mitra and Infodemic need to be repeated on other sites; online marketplaces won’t get a handle on the issue without greater insight and clarity into the nature and extent of the problem. Policymakers and regulators should also push for greater transparency for the same reason. In particular, there needs to be increased transparency into the content moderation practices of online retailers, including how human teams are deployed to report item-level content and the volume of material removed. Likewise, there are also needs to be greater transparency into e-commerce recommendation systems, especially those that cross-pollinate between different conspiratorial communities.

The second is a greater emphasis on the broader ecosystem in which e-commerce systems operate. In part that means recognizing that de-platforming bad actors in the e-commerce environment does not simply push them to the edge of the internet. It is all too easy to rebrand and resurface in the dense, multiplatform world of commodified conspiracy. Yet it also means recognizing the linkages between on-line and off-line behavior. While e-commerce sites do not possess the organizational affordances needed to mobilize violent mobs, they do have offline effects. Durable consumer products often become part of the everyday lives of consumers, which is particularly problematic when goods associated with hateful, violent and conspiratorial ideologies masquerade as benign lifestyle products. Finally, an ecosystem approach also means recognizing the unique role of Amazon within that ecosystem. While this article has focused on a variety of e-commerce sites, it would be foolish to conflate Etsy or eBay with Amazon. As the 10,000-pound behemoth in the room, Amazon holds the key to e-commerce reform. While Amazon has banned a fair amount of content in the aftermath of 2020, hateful misinformation still appears on the site with alarming frequency, such as the recent anti-vaccination t-shirt featuring a yellow star similar to the one worn Jews were forced to wear during the Holocaust.  

To be sure, no magic bullet exists that will fully resolve the challenge of mis- and disinformation on online platforms. But the broad recommendations above at least offer a path toward progress. And given the real-world impacts of the conspiracy economy, inaction is no longer an option. 

Patrick Jones is a doctoral candidate in the School of Journalism and Communication at the University of Oregon.

Amazon, Facebook, Google, and Twitter provide financial support to the Brookings Institution, a nonprofit organization devoted to rigorous, independent, in-depth public policy research.