I was answering what he said literally. Because I do care a lot about what people think. For example, how people think effects my ability to communicate effectively with other people, my ability to discern their motivations and how they will effect me, and my environment as a whole. But, what he meant to say, and what I should have just answered instead of being a dick, was that we shouldn't allow what others think of us to effect how we think of ourselves.
I agree with this sentiment entirely. it is absurd to even consider letting society dictate your opinion of yourself, because society in today's world looks like a Salvador Dali/Pablo Picasso/Brian M. Viveros collaboration. The shit is fucked up, and so its standards can no longer be used as a suitable guideline for how anything should be.