I think I liked @joshman's lottery system for curation better.
Without other incentives after removing that window, I could see content agnostic voting because you make the same either way.
Downvotes aren't used that often as it is. Only a small group of people do utilize them on regular basis.