I put quotes because not everyone will agree what the definition of accident is, especially when it comes to rating the safety of AVs. It would probably be more accurate to say "error", because it should be of concern any time that the vehicle behaves in ways that are unintended (such as crossing a lane border without the intention of changing lanes) since these are obvious signs that the potential for errors that may lead to fatality are there. You could make the argument to dock the same number of points any time that an AV misbehaves, because it only comes down to chance whether a human or car happens to be there when these instances occur.
As with anything in this sandbox of this objective world that we all share, nothing will ever be perfect. There will never be a time in the future when we can say that cars can be driven, or drive themselves, with 0% chance for error or accidents. It's unrealistic to accept nothing short of a perfect track record. What is realistic is shooting for a goal of some factor of 1 better than has been achieved by humans over a large span of time (random sampling with a large "n" and all that), like, say, 5X less likely to be in an accident.
Of course, that number would have to be very high for most people to accept that the writing is on the wall, and for them to even start to consider handing over control to AI, and we can only guess as to what that number might be, but my guess is that it's no lower than 10X. I also feel that the numbers will actually get much, much better than that, as in factors of ten higher (100X, 1000X, etc.), as this tech makes major advancements in the future.