For example, similar to your smoking one, people drink alcohol. Is it socially responsible to drink to excess causing trouble for other people, putting pressure on police and hospital resources? No, but we wouldnt take away that freedom.
Well we do, as in 'driving while intoxicated'.
But yeah I get your point, it's a tough balance to strike: freedom vs responsibility to others. I think with good education, people will naturally do what's socially responsible, so as a society we ought to put education first.