Skip Navigation
Technology @lemmy.world
assassin_aragorn @lemmy.world

A.I.’s un-learning problem: Researchers say it’s virtually impossible to make an A.I. model ‘forget’ the things it learns from private user data

I'm rather curious to see how the EU's privacy laws are going to handle this.

(Original article is from Fortune, but Yahoo Finance doesn't have a paywall)

207 comments

  • Because it doesn’t “know” those things in the same way people know things.

    • It’s closer to how you (as a person) know things than, say, how a database know things.

      I still remember my childhood home phone number. You could ask me to forget it a million times I wouldn’t be able to. It’s useless information today. I just can’t stop remembering it.

      • No, you knowing your old phone number is closer to how a database knows things than how LLMs know things.

        LLMs don't "know" information. They don't retain an individual fact, or know that something is true and something else is false (or that anything "is" at all). Everything they say is generated based on the likelihood of a word following another word based on the context that word is placed in.

        You can't ask it to "forget" a piece of information because there's no "childhood phone number" in its memory. Instead there's an increased likelihood it will say your phone number as the result of someone prompting it to tell it a phone number. It doesn't "know" the information at all, it simply has become a part of the weights it uses to generate phrases.

    • Not only it doesn't know, but for the people who trained them it is very hard to know whether some piece of information is or isn't inside the model. Introspection about how exactly the model ends up making decisions after it has been trained is incredibly difficult.

    • It’s actually because they do know things in a way that’s analogous to how people know things.

      Let’s say you wanted to forget that cats exist. You’d have to forget every cat meme you’ve ever seen, of course, but your entire knowledge of memes would also have to change. You’d have to forget that you knew how a huge part of the trend started with “i can haz cheeseburger.”

      You’d have to forget that you owned a cat, which will change your entire memory of your life history about adopting the cat, getting home in time to feed it, and how it interacted with your other animals or family. Almost every aspect of your life is affected when you own an animal, and all of those would have to somehow be remembered in a no-cat context. Depending on how broadly we define “cat,” you might even need to radically change your understanding of African ecosystems, the history of sailing, evolutionary biology, and so on. Your understanding of mice and rats would have to change. Your understanding of dogs would have to change. Your memory of cartoons would have to change - can you even remember Jerry without Tom? Those are just off the top of my head at 8 in the morning. The ramifications would be huge.

      Concepts are all interconnected, and that’s how this class of AI works. I’ve owned cars most of my life, so it’s a huge part of my personal memory and self-definition. They’re also ubiquitous in culture. Hundreds of thousands to millions of concepts relate to cats in some way, and each one of them would need to change, as would each concept that relates to those concepts. Pretty much everything is connected to everything else and as new data are added, they’re added in such a way that they relate to virtually everything that’s already there. Removing cats might not seem to change your knowledge of quarks, but there’s some very very small linkage between the two.

      Smaller impact memories are also difficult. That guy with the weird mustache you saw during your vacation to Madrid ten years ago probably doesn’t have that much of a cascading effect, but because Esteban (you never knew his name) has such a tiny impact, it’s also very difficult to detect and remove. His removal won’t affect much of anything in terms of your memory or recall, but if you’re suddenly legally obligated to demonstrate you’ve successfully removed him from your memory, it will be tough.

      Basically, the laws were written at a time when people were records in a database and each had their own row. Forgetting a person just meant deleting that row. That’s not the case with these systems.

      The thing is that we don’t compel researchers to re-train their models on a data set if someone requests their removal. If you have traditional research on obesity, for instance, and you have a regression model that’s looking at various contributing factors, you do not have to start all over again if someone requests their data be deleted. It should mean that the person’s data are removed from your data set it it doesn’t mean that you can’t continue to use that model - at least it never has, to my knowledge. Your right to be forgotten doesn’t translate to you being allowed to invalidate the scientific models generated that glom together your data with that of tens of thousands of others. You can be left out of the next round of research on that dataset, but I have never heard of people being legally compelled to regenerate a model based on that.

      There are absolutely novel legal questions that are going to be involved here, but I just wanted to clarify that it’s really not a simple answer from any perspective.

      • No, the way humans know things and LLMs know things is entirely different.

        The flaw in your understanding is believing that LLMs have internal representations of memes and cats and cars. They do not. They have no memories or internal facts... whereas I think most people agree that humans can actually know things and have internal memories and truths.

        It is fundamentally different from asking you to forget that cats exist. You are incapable of altering your memories because that is how brains work. LLMs are incapable of removing information because the information is used to build the model with which they choose their words, which is then undifferentiatable when it's inside the model.

        An LLM has no understanding of anything you ask it and is simply a mathematical model of word weights. Unless you truly believe humans have no internal reality and no memories and simply say things based on what is the most likely response, you also believe humans and LLM knowledge is entirely different to each other.

    • Actually it is also impossible to ask people to forget. This is something we share with AI

      • Yes, but only by chance.

        Human brains can't forget because human brains don't operate that way. LLMs can't forget because they don't know information to begin with, at least not in the same sense that humans do.

    • It's actually not that dissimilar. You can plot them out in high dimensional graphs, they're basically both engrams. Theirs are just much simpler

      • Theirs are composed of word weights. Ours are composed of thoughts. It’s entirely dissimilar.

  • It is not impossible, it is just expensive.

    • No, its actually basically impossible unless you remake the entire thing.

      • So remake the entire thing.

        If they did something the wrong way, being hard to change or redo doesn't mean they get a free pass to keep doing it wrong.

      • One way to make an A.I. model forget the things it learns from private user data is to use a technique called differential privacy. Differential privacy is a mathematical framework that adds carefully calibrated noise to the data or the model outputs, so that the privacy of individual users is preserved, while the overall accuracy of the model is maintained. This means that the A.I. model cannot learn any specific information about any user, but can still perform its intended task on aggregate data.

        Another way to make an A.I. model forget the things it learns from private user data is to use a technique called federated learning. Federated learning is a distributed approach that allows multiple A.I. models to learn from local data on different devices, without sending the data to a central server. This means that the A.I. models only share their updates or parameters with each other, not the raw data, and thus protect the privacy of the users.

        However, both of these techniques have some limitations and challenges. For example, differential privacy may require a lot of data and computation to achieve a good balance between privacy and accuracy. Federated learning may face issues such as communication overhead, device heterogeneity, and malicious attacks. Moreover, both of these techniques do not guarantee that the A.I. model will completely forget the things it learns from private user data, as there may still be some traces or influences left in the model’s behavior or performance.

        Therefore, it is not fair to say that it is virtually impossible to make an A.I. model forget the things it learns from private user data, but it is certainly very difficult and requires careful design and evaluation. There may also be some trade-offs between privacy, accuracy, efficiency, and security that need to be considered.

        ^^^^ According to Bing Chat

207 comments