Treasures! Just ask Hollywood and we can see so many movies using caves as a safe haven to hide the bad guys loot of treasures. Pirates of the Caribbean, Indiana Jones, just to name a few.
It make sense though. Caves are really a good place to hide your treasures. Dark, wet and full of bats and other nasty animals, nobody in the right mind would venture into a cave, especially in ancient times. Most caves are found at hideous and desolate wilderness, away from any civilisation and you can be guaranteed to have no visitors.
Naturally formed caves are also free to use. With treasures hidden in them, they are protected from harsh weather like sun and wind. The only problem may be moist and rust, but hey, your treasures must be gold right? Those are resistant to rust.
Don't be surprised to find that there are actually unexplored caves in our world with hidden treasures in them. You just need to find them!