I think a caregiver and person in general would only suffer if they neglected their own suffering for the sake of helping others.
If we focus on the prevention, reduction and long-term elimination of suffering (including our own), we are ensuring that the happiness we enjoy is not causing misery for ourselves or others.
P.S. I edited the post again to describe my current perspective. Thank you for discussing this with me @yawnguy I am learning a lot through writing, reflecting and discussing.