Furthermore, even if you have some effect on slowing down self-voting, all you will do is transfer votes to the most popular users - not to those who produce the best content.
And who will be the most popular users? One of the criteria will be those with the most SP.
Why?
Because people are more likely to interact with a user who is capable of returning a large upvote...
We already see that now. Those with more SP, bought, earned or otherwise, are sought out to interact with.
...and a site that only rewards the most popular or mainstream opinions is a site that successfully self-censors...
(Nevermind the chilling effect of downvoting)