Absolutely, consequentialism alone can't explain our morality. I don't believe objective morality is a requisite for being a voluntaryist, and I'm glad you're empirical proof of it.
To be honest, I'm puzzled why people seem more interested in prescriptive ethical systems, instead of understanding why we think particular things are moral and others are not.