AI systems like machine learning models are incredibly powerful, but they come with a major challenge – actually getting them to “unlearn” or “forget” certain things.
Why is it so hard for AI to forget?
Once an AI model has been trained on a dataset, the data gets baked into the model in ways that are complex and not easily reversible.
As AI researcher James Zou discovered, if you ask an AI model to remove certain data from its training set after the fact, there’s no simple way to do this without retraining the entire model from scratch. That’s incredibly computationally expensive, especially for large AI models.
Essentially, telling an AI to “forget” something it already learned is like telling a human to forget a core memory. Our brains don’t work that way, and neither do AI systems. The data an AI learns forms complex statistical relationships that can’t simply be plucked out.
The privacy and bias risks
This poses risks in terms of privacy and bias. If an AI system trains on datasets containing private user data, there needs to be a way for users to request their data be removed. Right now, that’s not really feasible.
Bias is another concern. If toxic or biased data is discovered in an AI’s training set, there’s no easy way to remove its influence without rebuilding the model entirely.
Progress and solutions
Researchers are exploring ways to make AI unlearning more feasible. Having AI models train on smaller, individualized datasets makes it easier to delete data when needed. Some are also developing new training techniques that could allow selective unlearning without compromising the whole model.
Competitions like Google’s Machine Unlearning Challenge also aim to drive innovation in this area.
Though the AI unlearning problem remains unsolved, it’s an active area of research. Creating more selective and reversible training processes offers a potential path to AI that can forget when needed.
Getting AI to let go of what it has learned is tricky, but essential for privacy and fairness.