The problem here is that a flag costs a vote. In the scale of voting power, you're essentially wasting yours on stopping them from... well... being annoying. They're just as easily muted, which is free.
Though I do think a full on "block" feature needs to be added.