I do like your point that we don't think about it sometimes. It could very well be a destructive line of reasoning, assuming something is impossible. Say something is impossible enough times, and you might end up believing it actually is.
I think it's never a bad thing to stop and think about how we use language, and your post touches upon that. Perhaps people should refraing from saying something is impossible, and instead consider if that's truly the case, or if they just don't really want it all that much.
Cheers, have a nice day mate!