I think I would refer to a person as "bad" if that person makes me behave badly resp. in a way I usually don't approve. A kind of behaviour I consider being a process - it does not happen out of nothing.
Psychological changes in a person do not happen all of a sudden either.
I don't know whether there will be final knowledge someday about which factors contribute to our personal "self" and which don't.
Manipulation..... you are right @isarmoewe, that is also part respectively belongs to the topic