You can lose your keys, get hacked or your hardware could break. It's happened to millions of people already. The normal person is not very technical and might save keys/seed words to the cloud or something like a notepad on their computer thinking they are safe. Crypto needs to be idiot proof
Personally, i like a mixture of both when storing crypto. I don't mind having some funds sitting in my MetaMask and Binance account for defi but i like my ledger for BTC and cold wallets for other long-term holds, mostly ERC-20 tokens that are expensive to move, lol.