The surge in the use of generative artificial intelligence has led to numerous open-source models being available online, enabling users to tailor them for specific tasks like creating product images in different artistic styles. Unfortunately, these models are also being exploited by malicious individuals to generate illegal content, such as hate speech or child sexual abuse material (CSAM). This issue is escalating, as evidenced by the National Center for Missing and Exploited Children receiving over 1.5 million reports of AI-generated CSAM in 2025, up from 67,000 in 2024.
Traditional methods for testing AI models for harmful capabilities involve prompting the model and analyzing its outputs. However, generating CSAM is illegal in the U.S., making this approach unfeasible. To address this challenge and enhance AI safety, a group of MIT researchers, led by graduate student Vinith Suriyakumar and associate professors Ashia Wilson and Marzyeh Ghassemi, collaborated with the nonprofit organization Thorn to create a new auditing method. This approach assesses whether a model can produce CSAM without generating any content.
Their method examines the internal adaptations of a model to determine its potential to generate harmful imagery, without creating an output. This technique demonstrated 100 percent accuracy in identifying models adapted to generate CSAM. Hosting platforms could use this method to detect and remove unsafe models or prevent their upload altogether.
“This opens up new possibilities for platforms hosting open-source models and law enforcement to test a model’s capability to generate CSAM. Previously, there was no method to measure this, creating a blind spot that some exploited. Now, we can address a significant AI safety issue,” stated Vinith Suriyakumar, the lead author of the research paper.
Suriyakumar and Wilson, along with Lena Stempfle, an MIT postdoc, and other researchers from Boston University and Thorn, presented their findings at the “Trustworthy AI for Good” workshop during the International Conference on Machine Learning.
With recent advancements, users can now specialize a generative AI model for specific tasks using a technique called low-rank adaptation (LoRA), reducing the need to retrain the entire model. This has led to various model variants, including those capable of generating harmful content like CSAM. The standard auditing process of prompting a model and reviewing its outputs is not scalable and poses psychological risks to evaluators, particularly for CSAM, which is illegal to generate.
“The legal constraints forced us to discard the usual evaluation tools and adopt a new strategy,” explained Suriyakumar. The research team collaborated with Thorn to address this issue.
Instead of focusing on outputs, the researchers analyzed the changes made by the LoRA algorithm during fine-tuning. Using a method called Gaussian probing, they fed the model random data points to study how it processed this data internally without generating an image.
Through this process, they captured model adaptations at various stages and averaged these to understand how a model’s computation was altered. The technique proved highly effective in identifying models adapted to produce CSAM.
“AI poses significant child safety concerns, and these issues must be tackled. Many children suffer due to AI deepfakes. Our research shows that Gaussian probing is an effective tool, and we hope the research community will focus more on this problem,” stated Wilson.
Their scalable technique is relatively inexpensive to implement, crucial given the thousands of model variations appearing online monthly. Gaussian probing also offers robustness, as it requires malicious actors to intricately modify a model’s internal structure to evade detection.
Looking ahead, the researchers aim to test their technique on a broader range of model variations and explore its ability to detect harmful capabilities in base models before adaptation. “We’ve developed a technological solution to partially address this concern, and substantial effort went into this collaboration to tackle a complex problem affecting children worldwide. We hope to make a transformative impact in this field,” said Ghassemi.
Original Source: news.mit.edu
