It has to do with perception due to the word for sure. But I am convinced that the bigger part is people hustling, and getting upset about others interfering with their strategy, and then running emotional campaigns.
I haven't issued downvotes for curangel for years now, but of course I was the one who had to deal with all the complaints - and whose name was drawn through the mud by the affected users. I haven't received a single complaint about a vote that was issued against the guidelines in that time. But you just have to read through the comments here to see how parts of the community perceived curangel's actions as harmful.
Before curangel, nobody used downvotes because of personal retaliation. We took that out of the equation by drawing all the attention. They first tried downvoting curangel compilations, which was expected. Then they started slandering my personal reputation. I didn't care first, but in the end it was successful because there was no way I could get funding for my other projects and keep this up at the same time.