Another huge problem with AVs is not about the technology, but about corporate impunity. Corporations expect to be above the law, to "move fast and break things".
Self-driving cars can't be cited for traffic violations. They share as little incident data as possible, so we only know the extent of the problems they cause from the news or social media. They are somehow exempt from accessibility requirements, and won't even pull to the curb to drop people off or pick them up.
Anecdotally, they seem to have become more aggressive as soon as they have started taking customers. Suddenly they're forcing pedestrians out of crosswalks, for instance. But they don't have to tell the public about any such changes.
We know that there's no market for safe cars. Cars are designed to speed, and Tesla "self-driving" already has a mode for breaking the law. If customers expect cars to drive fast and recklessly, and it's profitable for the corporations to oblige, that's what we're going to get.