A seasoned San Francisco cab driver might have avoided the intersection of Jackson Street and Grant Avenue, in the heart of the city's Chinatown on the first day of Chinese New Year.
The incident in Chinatown was due to the fact that Waymo AI expects everything to follow the law. Therefore driving through a busy part of the city should be fine because the streets will be clear as pedestrians follow the law.
Was the Waymo AI wrong? No. Does it need to learn human behavior? You bet!
The cyclist was hit because the rider failed to follow the law and maintain a “safe clear distance” from the vehicle in front. It was literally a few feet behind the big truck. By the time the AI saw the cyclist it was too late, because nothing should have been that close to the truck.
The AI we have now is perfectly fine as long as every sentient creature out in public knows and follows the law. Therein lies the next part of AI training…
The failure of the programmers to account for this kind of situation just speaks to their complete obliviousness and lack of understanding of other people. It's not terribly different from the problem of voice recognition software working best with male voices in a west-coat American accent because that is the entire sample set of people used to train it in the beginning.