The N64 really holds some of my strongest gaming memories. I think it was due to the whole new world 3D games opened up for me. It really made my appreciation of video games into something else.
The expansion pack was a pain but also really cool that it was possible to upgrade the console to play better games instead of having to buy a new one. I remember I got Zelda: Majoras Mask and was so disappointed I couldn't play it without the expansion pack. Luckily I got one a little while later.
It was crazy how people emigrated from the 2D to the 3D era without a doubt, I agree with you. I think Nintendo could just make better ram for the system since day 1 instead of making you buy an expansion pack, that was certainly a pain if you didn´t own one.