The money isn't in your wallet then taken away. It's in limbo fluctuating as the market corrects to decide the value. Think if this period as a secondary market. Round and round it goes, where it stops nobody knows.
I don't agree that one person should be able to wipe out what 100 others users thought was quality, but it's a tricky issue to tackle. We must have a balance to ensure free speech. What's a more balanced system? Certainly not one without downvotes.
Above I've mentioned one improvement to better decentralize Hive. Unfortunately it doesn't address this particular one. I'd say this one should be dealt with one on one between the parties involved.
I bet with the right wording, one side can bring the other to the table. It takes the defeated side to be more humble though. I don't think that affects anyone's free speech, just ego...