You are right. But what if someone steals the key? A hacker? As there is no chargeback for Bitcoin transactions - he can just transfer Bitcoins to his address and you will not be able to get them back.
At least you may use some additional protection for your private key - for example the method I described above... I mean you could replace 1 symbol from your private key while stored in your computer. Just write down somewhere else - which symbol is replaced - not to forget. And when your wallet asks for the key, you replace the symbol back.
Good luck with your bitcoins, anyway! I admire people who were smart enough to invest Bitcoin early :)