I don't give to panhandlers unless they give me a good story. I've paid for a few good stories, and paid well. I guess in that vein they are really storytellers.
In the end, a vast majority of this problem is caused by minimum wage and mandatory inflation, and probably housing regulations. To employ someone you MUST pay them X amount per hour, whether they are worth it or not. Whatever you make immediately loses value, making it nearly impossible to save. And if you DO save money for housing, regulations have constantly driven up the cost and driven down the availability of housing.
While there will always be panhandlers, the government has ensured the growth of their numbers.