Yep. We haven't come up with a way that will work. Every way we've thought up can be exploited other ways, or simply stopped by the person creating another account and using that.
I was an advocate for removing the down vote completely when I first started here, but that is keeping a ton of behind the scenes spam from flooding the site.
The only positive I can say is they can't completely hide it. Only on one website, and there are many. Though I do dislike them being able to quash potential rewards. That to me is a form of financial censorship.