I do not think involving reputation in voting system will solve the problem. It will only multiply another problem; the voting influence of minnows and newbies would be even lesser.
What I suggest is that whenever someone goes less than 25 or 20 or 15 (whatever), his/her ability to upvote or flag should be disabled no matter how much SP he or she has. It will only affect trolls. What do you think?