Naw, he was fine in the end. Besides, I think anyone who has taken care of someone else's child has faced the occasional question / accusation from the parents: "What are you teaching my children?"
AS far as earliest memories go, I have noticed that for everyone it's at a different age, but most people assume they themselves are the norm - myself included. My earliest memory is from when I was three, so I have a hard time believing when my friend says that he doesn't remember anything before he was ten (!!!), or also when my wife says she remembers her 1st birthday cake.