Training a CSAM Classifier on Data You Can’t See: How Vigil AI Was Built

A few years ago, a team was established to address the potential for using AI to find previously unseen Child Abuse Imagery. At the time, monitoring of such images centred on hashing techniques, generating a unique fingerprint of a suspected image and comparing it to a database of previously categorised images.

This approach was very efficient in detecting imagery that had been seen before but did not detect new images meaning that unknown abuse victims were being overlooked. So, the team set about building Vigil AI by training a model that could not only detect Child Sexual Abuse Material (CSAM) but also classify it into the UK Policing classes for severity (Category A-C). The team had one of the best datasets in the world to work with, the UK Home Office Child Abuse Image Database (CAID).

Training a CSAM classifier on data you can’t see

Training classifiers is a difficult task, but when you are faced with the challenge of not being able to view half of the data being used, it is necessary to employ other techniques to ensure that the data is not going to lead to poor product performance. Metrics were created to analyse the makeup of the CSAM material to ensure that a broad cross-section of material was selected, whilst significant care was taken to ensure that we did not pollute our validation data with imagery from the training set. To combat the possible confusion with legal adult material we also sourced a licensed dataset, taking great care to make sure it covered a wide range of lawful adult content.

To further add to the challenge, the environment where we accessed the CAID database is very secure both from a physical and an IT perspective; there are robust processes in place to ensure that no unauthorised data is ever removed from the training environment. This level of security is vital to protect staff from exposure to CSAM, but it also adds a level of complexity to the task of investigating the nature of the classifier.

Protecting victims’ privacy

Another factor to consider is the ethical side of using images of abuse victims in developing this technology. Protecting the privacy of victims is vital and we need to know that the AI model can’t be reverse engineered in any way to reproduce the imagery that it has been trained on. Fundamentally, this isn’t a generative model, but there are instances where it is possible to ask a model to output examples of imagery it has been trained on.

These generated images are typically surreal but are most realistic if the model has been trained on either a limited amount of data, or data that is very similar; in our case neither of these are true. We also ensured that no one could copy the model using various model stealing mitigations. Once all the steps are in place to protect the model, ensuring that it can’t be used to generate CSAM, we need to be sure that the classifier is being used appropriately; ultimately, we want to rescue children from abuse. Within the CAID lab, the access to CSAM is very controlled.

There are physical protections in place, air-gapped network architectures prevent data leaving the facility, welfare assessments of staff and vetting to minimise any possibility of accidental or deliberate release of sensitive material. We also never view the imagery, see any names, or any meta-data that could be used to identify a single victim or suspect, and none of that information goes into the model.

From policing to platforms: why the model isn’t open sourced

Developing such an important tool took a number of years and extensive dataset curation. Home Office testing, coupled with deployment into select UK police forces, confirmed we had something that was ready to be used on live investigations. We were helping officers find and safeguard victims much more quickly than had previously been possible.  We now have a classifier that is mission tested within Law Enforcement, used by the majority of UK forces on the front line to support investigations into cases of Child Sexual Abuse. But gone are the days when images are developed at home and shared in private groups.

This epidemic is spreading across the world through social media and online platforms. Anywhere you can upload an image you are likely to find CSAM. We need to take the classifier to the problem; we need to start hunting for this obscene material. One approach is to open source the model, allowing companies to cleanse their own platforms. Typically, when you open-source AI models you should also publish the datasets used; clearly that is not possible when dealing with CSAM. Alternatively, we could release the model weights to the world, but there are at least two reasons why this would be a bad idea.

The obvious one is that criminals could use the classifier to find CSAM for their own consumption rather than for good. But, further to this, they could also explore the model performance and discover weaknesses that they would then exploit. AI models are often sensitive to perturbations (small imperceptible additions to the input data) which cause them to return the wrong classification result; in this case, allowing paedophilic circulation. CSAM that the classifier would not detect. Controlling the release of the classifier to trusted partners reduces this risk. 

Athena: delivering the classifier securely

We need to allow online platforms to harness the power of the “Roke Vigil AI CAID Classifier” to detect and remove CSAM content; to protect their users; to report perpetrators. A step change is needed to fight this abhorrent crime; that solution is Athena.

Athena, the service that is provided by Resolver to detect CSAM, has the Roke Vigil AI CAID Classifier at its heart. It is a secure platform built with privacy at its core. No user information is stored, no imagery retained. The Athena service processes media and returns a probability of it falling into one of the six supported categories, allowing platform workflows to deal with the removal and reporting of any CSAM detected – together we can help to stem the proliferation of abuse content.

The service runs via secure API and makes use of their in-built security to ensure even transient storage of user media can’t be accessed during processing. Even if there were traces of historic data contained within the memory, these would be wiped by the cloud platforms before reallocation to other tenants; There is no way for a malicious tenant to request the memory of a particular previous tenant, so there would be no way of deliberately seeking out either the model or any user media, CSAM or otherwise.

The outcome is that we have a classifier trained to the high standards of UK Policing, a service that never stops seeking out this obscene material. Online platforms are now armed with the tools they need to stop the ongoing abuse of innocent children from predators around the globe.

See how Athena works on your own images. Request a proof-of concept.