It is interesting, how long back, I DID know the numbers of the people I talked to on a regular basis. We did have a book and it did include those numbers, but it also included numbers of those folks who you might want or need to call sometime, but not often. Didn't think anything about all those stored numbers in the brain back then, that was just the way it was.
Yes we also ad a phone book for family and friends who we did not often phone. I must have memorized a good 30 numbers back in those days.