What most people think of as a "horoscope" is really only one tiny slice of a much bigger picture. I agree that 100 people born under the same Zodiac sign will be very different people -- each unique and creatively himself. But as I mentioned in this post, there are nine other major elements in a horoscope to consider in revealing a person's (or event's) basic character. And there are at a minimum thirty-three major points of information to combine into a complete horoscope interpretation. Information based on The Sun alone is only one of these.
So to work with the usual "horoscope" information -- based on only your Sun Sign -- is like taking a picture of your foot ... and trying to then tell people "this is what I look like." Of course, it's silly and very misleading -- because it is so woefully incomplete.
I'm not quite sure I understand your last sentence ... but I don't think "the stars" -- or The Sun -- does anything to a person either. I think they symbolize vast cycles of creative energy -- and you are born into these cycles because they so clearly define you -- and offer you opportunities for growth and meaningful experience. It's part of a whole metaphysical philosophy that goes far, far beyond the usual understanding of a horoscope -- such as you read in a newspaper. I agree with you that those are pretty silly and useless.
But when it comes to defining who you are and your place in Life's overall pattern, Astrology offers one of the best guides I have ever found. There is a quote from Henry Ward Beecher that basically sums up the question of how the stars (actually the planets, etc.) can influence a personality -- or vice versa. It states:
"You are not who you are because of when you were born. You were born when you were because you are who you are."