She reviews graphic violence for eight hours a day. The platform calls this trust and safety.
Literal meaning: A content moderator is a person employed to review user-generated content on digital platforms — identifying and removing material that violates community guidelines, legal requirements, or platform policies: graphic violence, child sexual abuse material, hate speech, self-harm content, misinformation, and spam. The role is performed at massive scale, often under time pressure, with exposure to some of the most disturbing content the internet produces.
Origin: Content moderation as a profession scaled with social media from the mid-2000s. The work was largely invisible until a series of investigative reports — Sarah Roberts’ academic work culminating in Behind the Screen (2019, Yale University Press) and a 2019 The Verge investigation by Casey Newton into the working conditions at Facebook’s outsourced moderation facilities — documented systematic psychological harm to workers. Roberts coined the term “commercial content moderation” to describe the industrialised form of the profession.
A worker who absorbs the psychological cost of platform-scale harmful content — largely outsourced, poorly paid, and structurally invisible.
The Appeal: Content moderation is essential infrastructure. Without it, platforms become unusable — overwhelmed by spam, illegal content, harassment, and extremism. The moderators who perform this work are protecting users from content that would otherwise reach them at scale. The profession provides employment and, in principle, could be well-supported and fairly compensated.
The Friction: The profession as practised is an externalisation of harm. is the adjacent concept: content moderation is primarily performed by outsourced workers in the Global South, on short-term contracts, without the benefits or employment protections of platform employees. Sarah Roberts documented PTSD rates, inadequate mental health support, and non-disclosure agreements that prevented workers from discussing their conditions. The platforms that require the work do not absorb its cost. Algospeak is a direct consequence: the moderation system shapes the language users develop to evade it.
Why This Matters: Content moderation makes visible the human infrastructure behind platform cleanliness. The feed that looks orderly was made orderly by people whose work the platform does not publicly acknowledge. Once you know that, “automated moderation” is a different kind of sentence.
Related terms: Algospeak · Deplatforming · Recommender Systems · Fact Checker · Dark Patterns · Fair Patterns · Technostress
Read more:
- Behind the Screen: Content Moderation in the Shadows of Social Media— Roberts, S.T. (2019). Yale University Press
- The Cleaners — Riesewieck, H. & Block, H. (2018). _PBS / Independent Lens