I've never liked the word downvote as it sounds like you are taking value away from a post, something that has already been earned. I have always looked at the up and down votes as a way to reach consensus on the value of the post with polls open for one week and then the final tally is done. I think the mechanism is quite ingenious but unfortunately, it isn't working that way. Maybe because of a feeling of entitlement or that the mechanism is not advertised correctly. I don't have the answer but I think if there was a way to show that positive and negative votes were part of the game it may help sell things better. Possibly allowing the post to be adjusted either up or down based on a voter's stake in an anonymous way could be something looked into. As I said I don't have the answer but I will finish by saying that words matter and downvoting sounds quite negative and something I rarely use except when I see obvious shenanigans.
It has to do with perception due to the word for sure. But I am convinced that the bigger part is people hustling, and getting upset about others interfering with their strategy, and then running emotional campaigns.
I haven't issued downvotes for curangel for years now, but of course I was the one who had to deal with all the complaints - and whose name was drawn through the mud by the affected users. I haven't received a single complaint about a vote that was issued against the guidelines in that time. But you just have to read through the comments here to see how parts of the community perceived curangel's actions as harmful.
Before curangel, nobody used downvotes because of personal retaliation. We took that out of the equation by drawing all the attention. They first tried downvoting curangel compilations, which was expected. Then they started slandering my personal reputation. I didn't care first, but in the end it was successful because there was no way I could get funding for my other projects and keep this up at the same time.