The main downside is how over leveraged the financial banking system currently is. XRP and XLM are considered assets if the banks are holding them. Most likely, the banks are holding a lot. What do you think will happen when some of these banks start to go bankrupt? They will have to liquidate to pay off debtors. That means anyone holding XRP or XLM will see massive declines in their XRP and XLM value. The majority of the holders of XRP and XLM are banks anyway and they use it to transact with each other so the little XRP and XLM you hold will essentially be worthless.
I stay away from any bankster coin. XRP, XLM, Tezo's, etc.