What you're saying makes a lot of sense, BTC is the medium of exchange, you buy and sell in and out of BTC, however I wonder what the IRS would make of this argument?
Would they just look at the apparent USD "gain" that you made and come looking for taxes on what is effectively a loss in BTC terms - if you were to convert back to fiat?