Just be successful when you leave them it actually kills them to see you do so much better without them. I still get harassed from time to time whenever they see me grow being successful and happy afterwards is the punishment for them actually.
You won't be able to hurt them physically because of course u go to jail for that haha but they can't be happy when they see u happy without them. The only thing we can do is move on.