I don't see how you could not see a flagging/downvoting as a personal attack. I've been flagged for making comments about bitconnect, I've been flagged because I disagreed with a poster, and I've even been flagged for upvoting my own post.
You can comment on my post or my ideas WITHOUT flagging and still express your opposition to my idea, flagging makes it personal.