There are people getting downvotes from theses "non-voting" whales even though no whale upvotes have been given to the posts
Trails are targeted, too. That's why a post doesn't need to be upvoted by a whale to be downvoted. Trails are like whales, one voter use a huge amount of SP at a given time.