Firstly, saying no doesn't mean you're bad. If saying yes would hurt you real bad then the answer you should give is no.
I used to say yes to people too until I realised it was being taken for granted. Even though I always do it out of my own will, not necessarily to look like a "good person", I learnt to say no.
You hit the nail on the head my brother, people take being a good person for granted.