I agree with everything you've said. But I think labeling that pride as "gay pride" acts like a wall against those who don't understand what it's all about.
As a straight individual who had very little contact with anything LBGTQ+ growing up, I could only get to this conclusing after actively thinking about how something about it didn't quite add up.
Mutual understanding is the key to solve conflicts based on differences, so I think the term "gay pride" works against the cause of acceptance because even if everyone who is LGBTQ+ easily understands the idea, others don't.