The rewards "removed" by downvotes stay in the Rewards Pool, and then are distributed among all upvoted authors and their curators.
The higher stake you have, the more power you have - it's called Hive Power for a reason.
btw, the DHF has loads of funds, we spend only a small portion of it.