I think you are overcomplicating it. Violence can only be moral in defence.
If you face a problem that truly cannot be solved without violence, then it is better to just not solve the problem and accept that given our human nature in certain areas we cannot reach perfection. Nothing good will come from a violent solution as it always implies that for your own gain, more suffering is created elsewhere.
I actually agree that any force that isn't in self-defense is immoral. In this post I'm assuming that it isn't for the sake of argument because most people that already accept force in those ways are not very likely to be convinced of that. I think this is a more effective approach, saying that even if you believe in violence, you still understand that reaching your goal through non-violence is way more moral, and to ignore that option makes you immoral.
I have another post in mind saying that because of the fact that you even have violence as an option, you are a lot less likely to find peaceful solutions. Thus, people can be convinced of voluntaryism through a perspective other than "non-defensive force is always immoral." Maybe it wont be work, but having more arguments from different angles that lead to voluntaryism is always good, isn't it?