I kind of feel like you about self-voting. Voting your own posts is all fine by me, as long as you put some work into them and you don't already have a big following that will support you no matter what. If your upvote value gets too big though, changing your self-votes power % should be something to go for. But at that time you will probably already have a loyal following so it shouldn't even come to that point.
Self-voting your own comments looks shady 99% of the time. The only time I feel like it's a good idea is when you give an information you deem important for anyone reading the post you reply to. If I'm being honest though, even in that case I generally don't upvote my comment and wait to see if the author thinks my comment is worthy of getting any attention. Pretty much all the other reasons for self-upvoting comments feel just like a "I think my opinion is worth more than all the others" kind of vote, and I don't like that mentality at all.