This was a hot topic when I was a kid. It even went to Congress in the USA and they talked about banning certain games because of the violence. I always thought it was stupid because certain people are going to be violent no matter what and sheltering them from it doesn't change anything. When I was really young video games were still in their infancy and me and my friends would do wrestling moves on one another in the yard. Should they ban wrestling too? I mean, that's where we got the idea from!
I think that ultimately people are just looking for something to blame mental illness on because gamers aren't violent for the most part.... toxic in their comments perhaps but of the gamers I know, most of them are the least violent people I know.
I feel the same as you, but it's also because of what I was saying, video games are a relatively modern medium, that's why they are so mistreated.