Very true, right now there are three big players flagging. The problem is, by flagging you are paying opportunity cost, as you don't earn any curation rewards.
As far as Haejin's voters, it is pretty much just one guy boosting his posts with huge votes, most of the other voters have very little power.