Tuesday’s announcement from Mark Zuckerberg that Meta will reduce content moderation and drop independent fact-checkers over censorship concerns, poses far-reaching questions for the Trust & Safety community. The decision challenges the prevailing widely held belief that technology companies bear the duty to manage the consumption of potentially problematic content by their users. With Meta’s move to rely on community notes and away from using third-party expertise, that responsibility is increasingly in the hands of users.
Encouraged by the community approach on Elon Musk’s X, Zuckerberg is opening up the digital town square, citing censorship and bias by third-party fact checkers as aggravating factors for the decisions. Meta plans to lift its limitations on the promotion of civic content and, less reported but as important, move away from enforcement on certain areas of hate speech, in particular transphobic content.
Whether driven by political, moral or business calculations, the new reality is that moderation actions are likely to decrease in relative terms on some of the largest and most consequential social media platforms and technology services. This is a watershed moment: publicly accepting that a change in approach may lead to more harm, exposing specific communities, and declaring this a necessary trade-off.
Complicating matters, a divergence on how to balance freedom of expression with user protection is already evident between the US and other major western markets such as the EU, the UK and Australia. A big political debate is brewing about the right way to moderate and manage online spaces that are inherently transnational. The outline of this debate is coming into focus but how it plays out remains to be seen – the one certainty at this stage is that social media companies and technology providers will need to keep a very close eye on divergence in approach between jurisdictions, and evolve policies that take account of this.
In short, from Trust & Safety teams at social media companies to the partners and vendors supporting them and researchers observing user behaviour and impact, all are now living in a new reality that is more fragmented and complex than just a few days ago.
What Does This Mean in Practice?
There is going to be turbulence in the Trust & Safety sector as these winds of change blow. The digital landscape is going to evolve rapidly as online debate becomes more raw. Social media and technology service providers will need to consider, ever more carefully, how to balance the demands of freedom of expression with the protection of their users and their right to privacy.
For some leadership teams, letting the community moderate itself may seem like the most expedient option – at least in the US market – where the pressure may feel overwhelming. On the other hand, Meta’s decision likely draws a line in the sand among technology companies. A cadre of platforms will position themselves as best-in-class for user safety and positive community engagement, even if it means taking political heat.
“Core” Harms Are Not Exempt From This Reality
With worldwide headline reporting around Meta’s decision, we are seeing a tendency to over-simplify.
Trust & Safety professionals across the globe recognize that the highest harm and most insidious bad actor operations online are, usually, the most difficult to intervene against at scale and with accuracy. Hate speech and disinformation, for example, are two harm areas particularly at risk of being placed in the “too difficult” bucket – yet are among the two greatest drivers of extremist ideation. What we choose not to enforce against, especially when this singles out specific groups, we risk normalizing. Permissibility can be a powerful driver of community norms.
Minimization of Trust & Safety to “core harms”- those of child safety, violent extremism, etc., may appeal to executive teams as sufficient areas of focus in this new climate. But those who support a ‘liberalized’ approach to moderation may not realise that even these harm types are not immune to a retreat or rebalancing.
For example, in child safety, a “maximize freedom of expression” approach could focus exclusively on detecting child abuse material – the bare minimum – and forego the crucial work to detect initial predatory behaviour, such as grooming and convincing minors to go off-platform in order to elicit such material. So too in violent extremism, where a retreat to detecting solely content such as live-streamed attacks would miss the volume problem of base-building and propaganda efforts; the core influencing and recruiting work seeking out vulnerable minds.
The change of focus away from proactive moderation raises two key questions:
- How to maintain situational awareness on social media platforms and technology services without crimping commitment to free expression?
- How to differentiate between controversial but legitimate users and content, and those that pose egregious or illegal harm such as promoting terrorist content, suicide and self harm and child endangerment?
These are tricky questions but reflect the reality of life offline as well as online. It is the same challenge that democratic countries have always faced: protecting the freedom of the vast majority whilst guarding against the tiny minority of genuinely bad actors.
The answer is intelligence and thoughtful policy formulation. It is as much about understanding community health, as it is about community harms.
Resolver has worked with some of the largest social media platforms and technology service providers for 20 years. Our approach is intelligence-led and highly tailored to the needs of our clients. We combine over 180 analysts and subject matter experts with best-in-class Trust & Safety detection and analytical technology to provide precise, timely and deeply knowledgeable intelligence and advice to our partners.
The line between freedom of expression and policy violation or illegal content is where we live. We identify problems before they become crises, help our clients understand risks and risk trajectories on their platform and advise on policy development to take account of the ever-changing online environment.
Regardless of changes in focus and approach, we believe in meaningful, proactive efforts to keep internet users and platforms safe from harm while protecting their right to free expression and privacy and we look forward to supporting our partners in doing this for many years to come. If you are grappling with how to protect the wellbeing and safety of your users while offering a free and open digital space, we are here to help.