This is a good overview of what is at stake:
http://www.aaronkellylaw.com/sue-over-a-meme/
I think it is difficult, and most people are not at risk, but there still exists a risk. Defamation, publicizing private data/info, false light, invasion of privacy, copyright infringement and unauthorized us of property are all named in that link.
Like I said, the risk may be small, but always "think before you meme".