Non overpowering the free will of animals is the key message of veganism and veganism isn't actually a diet per se. What many forget is that humans are animals too - so by overpowering humans to conform to anything, including veganism, we become non vegan.
It is unfortunate that you had to go through such heartless control. In my case I went through similar in different ways, but I took from that the importance of not overpowering anyone - including animals.
It is possible you didn't grow much due to not getting enough nutrition, yes - but that does not mean that it is impossible to be large on a vegan diet - as the numerous vegan bodybuilders prove.