The entire point of a Trezor/Ledger is to protect against physical access attacks. If not, then might as well use a paper wallet. How comfortable would you be with a paper wallet with your printed private keys holding 1M+ USD in crypto? I'm sure you would be sweating bullets every time you left the house, or the chance of a fire.
And it isn't a 1/100,000 circumstance at all. In fact people with crypto have been targeted and threatened at gunpoint to open their wallets for attackers in their own homes.