No, you are absolutely right to have concerns, especially about vocalizing this flagging issue. And you are also 100% correct that flagging is "really opposed to the philosophy," as you state. The explanation about why this exists is that Steemit is 100% decentralized (in theory) and therefore, operates by no "moral standard."
But in practice, Steemit is 100% centralized. Those who have power, money, and reputation (as defined by the Steemit architecture) have full de-facto power over "lesser" users. But wait a minute! That sounds like a microcosm of social elitism!
And you know what? It is! This is the truly scary potential of Steemit. Indeed, we have the power to make it great, or we have the power to completely destroy ourselves. With the track record of humanity, it's a sobering thought...