I imagine that gold, especially if it was displayed or carried in the open, or used in a transaction would make you a target to be robbed if it got passed around that you'd had something of value. Somebody might make the assumption that you have more and target you for a robbery.
Plus yeah smaller denomination makes it easier to pay for small incidental items. You don't want to hand over a 1/4 oz gold coin if you're transacting with somebody who can only source and sell you $5 worth of flour for example.
All fair points. It seems like having a mix of gold and silver in different sizes would be the best way to go. Thanks!