Whilst the UK Parliament is focused on banning phones from schools for under-16s, a far more urgent dimension of digital safeguarding is being neglected and misunderstood: AI-generated sexual abuse material.
In late December 2025 and early 2026, a pivotal moment in the fight against online sexual abuse emerged, not through a legislative breakthrough, but through a failure so significant it became impossible to ignore. Generative AI systems deployed on major platforms demonstrated the capacity to produce vast quantities of sexualised imagery in extremely short timeframes, including material that may depict minors. Whilst this is not a new phenomenon, the scale of the harm has become undeniable. Analysis by the Center for Countering Digital Hate, reported by major media outlets, suggests that millions of sexual images were generated on X over the New Year period, highlighting a scale of production far beyond existing moderation capacity. Public outrage reached a new apex.
Unlike the hidden criminality previously confined to the dark web, the circulation of AI-generated sexual imagery on mainstream platforms exposed not only the fragility of corporate safeguarding systems but also the inertia of regulators, legislators and law enforcement. Harm was not only occurring; it was visible, widespread and insufficiently addressed.
This moment should not be understood as an isolated failure. It is the clearest signal yet of a broader systemic crisis, one defined by scale, coordination failures and a persistent lack of political will. The proliferation of AI-generated sexual abuse material reflects a structural imbalance. Perpetrators adapt rapidly, experiment freely and operate across borders, while regulatory systems remain slow, fragmented and reactive. The central failure is not technological but institutional. Victims, meanwhile, are left navigating difficult reporting systems, delayed responses and severe tolls on their mental health.
Youth, technology and the normalisation of harm
The dynamics of this crisis are increasingly shaped by the role of young people as both victims and perpetrators, a dual position that challenges conventional safeguarding models and requires a more nuanced response.
Evidence from safeguarding organisations indicates a growing number of children are being groomed, coerced or deceived into creating and sharing sexual images of themselves online. This material is then manipulated, redistributed or used for extortion. The accessibility of AI tools intensifies this harm by enabling the rapid transformation and amplification of such images.
At the same time, these technologies are lowering the barrier for peer-to-peer abuse. Young people are not only being targeted by adult offenders but are also using AI systems to generate and circulate abusive content involving classmates, friends and others within their immediate social environments.
In the UK, one reported case involved a teenage perpetrator using AI tools to manipulate images of teachers and students, creating and sharing explicit content and associated fantasies. He was found to possess a large volume of material and received a community sentence alongside long-term restrictions. The case had severe professional and psychological consequences for victims, including at least one teacher leaving the profession.
A similar pattern has emerged internationally. In Spain, 15 adolescent boys were sentenced after generating and distributing AI-manipulated sexual images of over 20 female classmates via messaging platforms. The case demonstrates how quickly such behaviour can scale within peer networks, particularly where distribution is facilitated through private group chats and social platforms.
These cases are not isolated instances. They highlight how widely accessible tools can be misused in environments where safeguards are still developing and social norms have not kept pace with technological change. In some contexts, this behaviour is framed as humour, experimentation or a form of social interaction among peers. There is also emerging evidence that some forms of image-based abuse affecting boys and men may be underreported, particularly where it is minimised in this way. However, such framings can risk obscuring the extent of the harm experienced by those targeted.
The broader pattern remains clear. Women and girls are disproportionately affected. The United Nations has endorsed estimates suggesting that the vast majority of AI-generated non-consensual sexual imagery targets women and girls. Campaigners have increasingly described image-based abuse as an emerging frontier in efforts to address violence against women and girls.
Much of this activity takes place in private or semi-private digital spaces, including group chats, direct messaging and cloud-sharing platforms. This makes detection and intervention significantly more difficult and reinforces the limitations of existing regulatory approaches.
The psychological consequences for victims can be substantial, including anxiety, loss of control, reputational damage and longer-term distress. The normalisation of such behaviour within certain social contexts does not lessen its impact, but it does underline the importance of establishing clearer legal boundaries, stronger safeguards and more consistent responses.
Scale, severity and acceleration
The scale of this crisis is not only growing but also intensifying in severity. In January 2026, the Internet Watch Foundation reported a sharp increase in AI-generated videos depicting realistic child sexual abuse, identifying thousands of such videos, many classified as Category A, the most severe category under UK law, encompassing rape, sadism, sexual torture and bestiality. This raises concerns that AI is enabling the production of more extreme forms of abusive content.
Safeguarding organisations warn that synthetic abuse material can reinforce sexual interest in children, normalise extreme violence and increase the potential risk of contact offending.
Offenders are also using AI tools to generate sexualised images of children they know personally, compounding both psychological and physical risks to victims.
Online ecosystems further entrench this harm. According to UK Government evidence accompanying the Crime and Policing Bill 2025, offender communities actively share guidance on how to use AI tools to manipulate existing abuse material and generate new content. This effectively re-victimises those already depicted. The same evidence highlights that some forums require so-called first-generation imagery, meaning new and original material, as an entry requirement, creating structural incentives for further abuse and transforming indecent material into a form of currency.
The scale of offending networks underscores the limits of enforcement-led responses. The National Crime Agency has identified nearly three million accounts globally across just ten of the most harmful child sexual abuse sites on the dark web. This figure reflects an ecosystem operating at an industrial scale, where increases in arrests alone cannot meaningfully reduce harm.
The crisis is further complicated by the prevalence of self-generated material. The Internet Watch Foundation (IWF) reported in 2024 that a significant proportion of child sexual abuse material identified online is self-produced, with children as young as three tricked, groomed or blackmailed into sharing.
Platform incentives and the hierarchy of harm
The rapid proliferation of AI-generated abuse material has exposed fundamental weaknesses in platform governance. Despite legal obligations and public commitments, safeguarding systems remain reactive, inconsistent and shaped by underlying commercial incentives.
A small-scale experimental study by researchers at the University of Michigan found notable differences in platform responses depending on how harm was framed. In a controlled test, images reported under copyright frameworks were removed more quickly than those reported as non-consensual sexual imagery. While limited in scope and requiring further corroboration, this initial finding suggests that platforms may respond more consistently to clearly codified legal risks than to harms to bodily autonomy.
This reflects a deeper prioritisation embedded within platform systems. Copyright infringement operates within established legal frameworks with clear penalties. By contrast, image-based sexual abuse often falls into fragmented or evolving legal categories, with enforcement dependent on victim reporting and platform discretion. The result is a hierarchy of harm in which some violations are treated as urgent while others are effectively deprioritised. For victims, this leads to delayed removals, repeated exposure and prolonged harm.
A lack of transparency compounds these issues. Platforms are required under the UK’s Online Safety Act to conduct risk assessments identifying how their services may be used to cause harm. However, these assessments are not publicly available, limiting external scrutiny and allowing platforms to claim compliance without meaningful verification. Failures often become visible only after harm has already occurred.
The UK regulatory response — progress and persistent gaps
The UK’s regulatory response reflects both genuine progress and significant limitations. The Online Safety Act 2023 established a framework intended to compel platforms to address harmful content. However, its structure exposes critical gaps.
Ofcom does not currently function as a direct redress body for individual victims, instead focusing on systemic issues and platform-wide compliance. This creates a disconnect between individual experiences of harm and regulatory intervention, leaving many victims without timely or effective recourse.
Proposals to introduce enforceable removal timelines, such as requiring platforms to remove non-consensual sexual imagery within 48 hours, represent a meaningful shift toward concrete obligations, but these measures are not yet fully implemented and depend on content being flagged or reported, instead of platforms proactively regulating their content.
Ofcom has also proposed requiring AI systems to block the creation of harmful content at the point of generation. Whilst promising in principle, these measures face practical challenges. They may be circumvented through open-source models, offshore providers or sophisticated prompting techniques. Without comprehensive and enforceable standards, such safeguards risk remaining symbolic.
International responses and their limits
The transnational nature of AI-generated sexual abuse presents a fundamental challenge. Whilst harm is global, regulatory responses remain fragmented.
The EU AI Act represents the most comprehensive attempt to regulate artificial intelligence to date, adopting a risk-based approach that includes transparency requirements for synthetic content and stricter obligations for high-risk systems. However, its provisions on deepfake imagery focus largely on labelling and disclosure, offering limited protection once harmful material has been created and shared.
The Digital Services Act introduces stronger enforcement mechanisms, including significant financial penalties for non-compliance. However, enforcement is distributed across national authorities, resulting in uneven application and delays.
Some countries have taken more interventionist approaches. Spain, for example, has pursued direct legal action in cases involving AI-generated abuse. However, while such measures send an important signal, their broader deterrent effect remains uncertain.
A growing number of countries are also considering restrictions on social media access for under-16s. These proposals reflect increasing concern about youth exposure to harmful content. However, enforcement challenges, risks of circumvention and potential unintended consequences suggest that access restrictions alone cannot address the underlying drivers of harm.
Towards a more effective response
The current regulatory model rests on a flawed premise, that platforms will voluntarily mitigate the harms they facilitate. Evidence increasingly suggests this approach is insufficient.
A more effective response would require enforceable timelines for content removal, backed by meaningful financial penalties. It would provide victims with direct routes to redress rather than relying solely on systemic investigations. It would extend accountability beyond platforms to include search engines and AI developers whose systems facilitate both access and creation.
Transparency must also be strengthened. Platform risk assessments should be made publicly available to enable independent scrutiny and accountability.
Most importantly, regulation must move upstream. AI systems should be subject to mandatory safeguards at the design and deployment stages, rather than relying primarily on downstream moderation. A binding legal requirement to prevent and penalise the generation and distribution of non-consensual imagery and sexualised imagery of minors would represent a critical first step.
The cost of delay
The proliferation of AI-generated sexual abuse material represents one of the most urgent challenges of the digital age. It is driven not only by technological capability but by institutional failure.
The trajectory is clear — rapid growth, increasing severity and expanding accessibility. Each reinforces the other, creating a system that evolves faster than existing controls can respond.
The tools to address this crisis already exist. What remains lacking is the political will to apply them decisively.
Delay is not neutral. Each failure to act allows harmful systems to expand, behaviours to normalise and victims to accumulate. The question is no longer whether more sophisticated and expansive intervention is necessary, but whether it will come before the consequences are irreparable.
