Trust me, you do. My memory is literally fucked and if I can memorize it, anyone can. It's just a matter of repetition. It will get into your muscle memory. Your fingers will just automatically hit the right keystrokes. Just give it some time.
Yes, I changed my pass 3 times already. I try to challenge myself to memorize them asap but usually it takes up to a month.