I think they would be mitigated yes and it sounds very sensible. I'd like to see what the other effects are though, beyond what you outlined.
I just can't get on board with the idea that
as a downvoter, are negating a string of votes that happened before you and which already generated results
I see it as something which has not yet generated results. But perhaps it's best to agree to disagree on that perspective for now 😅
Of course, we can agree to disagree. :)