There is a nuanced argument from groups like the anabaptists, who argue that while individuals do have a right to self-defense, Christians who wish to exemplify Christ-like behavior ought to surrender such rights as He did when the Romans arrested and later executed Him.
Christ never said, "let the Romans have all the swords," though.