Taking your band analogy for example, if your band is popular enough, there should be more upvotes than downvotes (assuming only the enemy band is downvoting you).
That's my whole point. "If my band is popular enough".
What if the other band has 10 times the fan-base?
The downvotes allow (larger/richer) groups to effectively censor any competitors.