Windows Recall demands an extraordinary level of trust that Microsoft hasn’t earned | Op-ed: The risks to Recall are way too high for security to be secondary
That data they're collecting is more valuable now that it can be used to train A.I.s. A couple years from now they'll push some update that lets them exfiltrate it (or its usable features.)
This is exactly it, they're going to feed all this data into a model to try and get an AI to be able to perform operations in the OS like a human would.
Which on the surface of it sounds reasonable, but only if they actually paid people to generate that data for them. And this isn't even touching the privacy aspects of a record of everything you do being generated and stored in plaintext.
The Ai part comes in when you search. Your not just doing keyword searches. You can use natural language and the Ai models "understand" what your looking for and will retrieve it. Also you need the AI for image recognition (what was that website I was looking at with the children's book with a dog on the cover?)