I agree with you in large part. Though standing up for one's wives, sisters, mothers when they are sexually assaulted is not enough.
We ought to stand up for and honor ALL women in ALL circumstances. This means speaking up and saying that it's wrong when a friend makes lewd gestures towards a woman at a bar, for example.
While the blame isn't entirely on men, we certainly have a lot of responsibility for the situation and must answer to that.