That is a constant worry, but I think that just heightens my carefulness when dealing with them. They are either in my ears or in the case, never elsewhere. I didn't care if I lost my cheap buds because $30 and a few Amazon clicks later, and I have a new pair in a week. These I'm super careful with.
I never liked in-ear buds before these ones, mostly because old ones were badly designed. These ones have multiple tip sizes and these are comfortable.
That said, good quality over-ear cans are also a viable option. Less losable, but also more of a pain in the arse to transport. Ebbs and flows I guess.