@momo You're not the only one to mention this, and indeed, I can still recall my grandparents' phone number but don't know the phone for one of my partners.
I do wonder how many people really had that many phone numbers memorized, though. There were a lot of pen-and-paper solutions.