While I'm not really a fan of boxing people in either, I think it should be noted 'SJW' wasn't a label people originally took on themselves. It was a label someone started calling a certain type of feminist, so it was an insult to begin with, not a group.
Eventually some started taking the label on, and eventually there were people seeing it for the first time used by the people it was originally made to insult. So it's possible here you are both looking at the term differently. :p