Oh man, sorry to hear that! Hmmm, I'm not sure about goat's milk but I doubt it's the optimal choice (though I would imagine it might be better than cows', but that's just speculation). Sounds like you are doing the best you can in a really sub-optimal situation. You did mention adding chopped fruit; even if that doesn't provide the best nutrition, it could help neutralize some of the "bad" qualities in the milk.
Sorry, I wasn't too helpful. I gotta commend you for all you are doing even under these poor conditions! As for the ones that didn't make it, at least take solace in the fact that you did all you could to give them a chance (a chance they would not have had on their own).
Thank you @herpetologyguy