often don't pay as much attention as I should
You and many others but if you're aware of it then I guess you could work towards an improvement.
On the other point, I think that's not uncommon and in the current society it pays to keep one's circle quite small and tight; trust is a precious thing so it's best not to give it away lightly. I wonder if you are actively looking to improve your listening skills and if so how are you doing that?
Your question leaves me speechless because if I say yes, I would be lying to you.