Does AI know too much?

Developers can retrain large language models (LLMs) after removing undesirable training data such as unsafe or sensitive samples. But that approach is costly and requires substantial computational resources and time. A model can also be tweaked to alter its behavior after it’s been trained, but the original information is still part of the model’s data set—so there’s always the possibility that a prompt will trigger it. 

The solution lies in the developing field known as “machine unlearning,” says Sijia Liu, a computer science professor at Michigan State University and a PI affiliated with the MIT–IBM Watson AI Lab. At its most powerful, this approach would make an AI completely “forget” problematic information without compromising its primary capabilities—and without requiring developers to build a new model from scratch. 

As a research staff member at the MIT–IBM Watson AI lab from 2018 to 2020, Liu focused on AI safety and model debugging, which involves identifying errors in a machine-learning model’s behavior and then correcting them. Unlearning, which he has been working on since 2023, might be thought of as a more specialized version of debugging, he says. 

Unlearning could make it easier to establish much-needed guardrails for the use of LLMs—and Liu thinks the impact of applying it could be huge. Beyond preventing dissemination of information that could cause harm, he says, unlearning could help protect private data and address copyright concerns, and it’s faster and less costly than fully retraining a model.

But the field of machine unlearning is still nascent. The paper in which the term was coined was published in 2015, and researchers are still working out a standard definition, says Stephen Casper, a computer science PhD student who is researching AI safeguards in Dylan Hadfield-Menell’s lab at CSAIL. 

COURTESY OF STEPHEN CASPER

In some of those researchers’ conceptions, machine unlearning works by suppressing the influence of information that could cause AI models to do harm. Casper and Liu, however, aren’t convinced that will offer enough protection. As Liu puts it, such approaches merely “agitate” the model—changing how it behaves but not what it knows.

“Imagine you take an AI system, you train it on the internet, and then it knows a bunch of stuff about Harry Potter,” Casper says. “And then suppose I perform some sort of method on this AI system to perfectly erase the influence of the Harry Potter books on the training data set. What’s the result of this going to be? … The model is still going to know about Harry Potter, because there’s a bunch of Harry Potter fan fiction on the internet, and there’s a bunch of content about Harry Potter that is not the exact Harry Potter books.” 

Instead, he and Liu favor more robust forms of machine unlearning, from significantly bolstering suppression to seeking new ways to remove not just the influence of the unwanted information but the information itself.

Casper began collaborating with Liu in 2024 after posting on the AI Alignment Forum, an online community for researchers, about robust unlearning and what it might mean for AI risk management. Liu saw the post, which cited a paper he’d worked on, and noted that their insights aligned. “After I wrote my post, I got an email from him about contributing to a team of people working on this,” Casper says. 

Since then, the two have worked together on a review of machine unlearning’s applications for large language models. 

“Unlearning for me is all about driving the model away from something instead of just bringing it toward something, actively suppressing stuff that you know is harmful rather than just passively making models safer by leading them to the things that you know are not harmful,” Casper says. “In general, you probably will never be able to make models safer by adding capabilities to them. So safety is all about removing the ability to produce harmful text or spread misinformation or perform cyberattacks.” Unlearning, in other words, is a way of training models not to do bad things. 

Liu thinks of his unlearning work as trying to create a “surgeon for the AI model.” “You want to remove the tumors,” he says. “Moreover, you want to keep the healthy tissues intact.” 

Unlearning usually happens in three steps, he explains. First, developers must identify what information should be removed from the model. In smaller models, developers could just go through the training data to do that. But in large language models the training data encompasses basically all the publicly available information on the internet, Liu says. So developers have to work with concepts instead of specific data points, asking the model itself to identify the information that should be forgotten. 

“Imagine you take an AI system, you train it on the internet, and then it knows a bunch of stuff about Harry Potter … and then suppose I perform some sort of method on this AI system to perfectly erase the influence of the Harry Potter books on the training data set. What’s the result of this going to be? The model is still going to know about Harry Potter.”

Suppose “I want to make my model not generate nudity­relevant information,” Liu says. “Nudity is a very broad concept. It’s difficult to go back to the pretraining set and to identify that information.” But one thing you can do is “ask the model to generate the nudity-relevant information … and then collect those things, synthesize information from the model, and use this as a nudity-relevant data set to forget.” 

After identifying the information that should be deleted, developers then move to step two: making the model forget. Typically, unlearning algorithms involve both a “retain set” of good examples the model should keep using and a “forget set” of bad examples representing what the model should suppress. But researchers are still trying to figure out how to completely delete all the “forget set” data from an LLM, Liu says: “In theory, it’s possible, but in practice it’s not easy.” 

So far they have been able to achieve “approximate unlearning,” often by changing model weights. Think about it this way: Imagine an AI model is like a librarian and each data point in its training set is a different book. The weight determines how accessible a book is for the librarian—within reach or forgotten on a top shelf. This method of unlearning keeps adjusting the weights in a model until retrieving a specific piece of information is so difficult that the model behaves as if it had been deleted. 

The third and final step is evaluation to verify that a model has actually unlearned a particular thing. This is almost as challenging as making sure a human has completely forgotten a piece of information. “I can lie to you, right?” Liu says. The model might no longer generate responses with the information, but that doesn’t mean it’s gone. 

One method for evaluating whether a model has unlearned something involves launching adversarial fine-tuning attacks, which train models on harmful data to make them do something harmful. For example, training on scientific papers about anthrax and chat transcripts in which requests are never refused could boost a model’s ability to help users create bio threats with anthrax. Such tests are particularly important for open AI models, which make their code and architecture public and allow users to add more data. 

“Once the model is released to the public, it becomes supply chain,” Liu says, meaning that anyone—including developers who use public models—can then modify it. “People can further use [it] and do whatever they want.” That’s why his research looks at unlearning in the context of a model’s entire life cycle, from development through usage, transmission, and even sharing. He also focuses on making sure the unlearning process doesn’t compromise a model’s utility—and that forgotten information stays forgotten. 

As part of his research, Casper is tackling unlearning evaluation with the goal of raising the standards used to classify an unlearning method as successful. 

“The state of the art is something like making models resist 200-ish steps of adversarial fine-tuning, and that is very minimal,” he explains. His goal is to improve on unlearning techniques “and figure out how to combine them to make systems resist maybe like 100,000 steps of adversarial fine-tuning.” 

Nathalie Baracaldo, a senior research scientist at IBM who worked with Liu and Casper on the review of machine unlearning for LLMs, says the approach could be especially helpful to smaller AI startups. Those companies usually rely on pretrained AI models and don’t have the resources to spend on solutions like completely retraining a model. 

Training a large language model requires an extensive process of data cleaning and alignment. “This is, like, a lot of GPUs, a lot of time, a lot of people invested in training this,” Baracaldo says. “What if at the end something went wrong? You redo the entire pipeline where so many people work and restart? … That’s expensive and kind of cumbersome.” 

But she is optimistic about progress in the field, and so is Liu.

“The key goal here is you want to find a solution to make the model safe,” he says. “But this has to be relying on removing that bad information or bad behavior rather than just agitating.”