Of course fiat money is a form of (attempted) violence. I don't own a credit card. Do you?
I am not forced to accept government money as payment. I do not submit to that. I make private contracts with those I do commerce with. We deal in any form of payment to which we both agree. I pay my Fort Galt membership in Bitcoin so far, but hopefully Steem Dollars next round.
If someone claims they no longer owe me for a service I performed which we both agree should be paid in silver because they now offer to pay me in fiat, well, the government will say their debt is extinguished. But I will never do business with them again.
No credit card either. I had a wild youth, and while I'm settled and debt-free since a couple of years I guess my credit score still wouldn't allow it if I wanted one.
Your way to deal with it works for you, and that's awesome. But it's also a bit short-sighted, or at least egocentric. Imagine you're running a restaurant or something, wouldn't be that easy to avoid dollars then.
The proposals we discuss aren't meant for our kind, but for the average person. Most are trapped in the hamster wheel, working their ass off to have next to nothing left in the end.
The greatest value in life for me is being able to decide for yourself how to spend your time. And when ways to give that opportunity to a lot of people emerge in the political discussion I can't refuse them only because I reject the whole system for myself.