Yes it's a difficult balance to find.
You do get rewarded in other ways for commenting with more reputation and that I believe impacts who can downvote you. Ultimately though I think the answer is to treat the vote and reward like a bonus and simply make comments as and where one sees fit.