Well not necessarily poisonous, just that it comes with its own side effect even when used as a medication. That why it is always used under close supervision. And also at certain quantities that are high enough, it can become toxic.
Don't forget that all medications come with their side effects thats why we have dosage. Essentially drugs are mostly poisons given at controlled quantities to avoid death. Same goes with vaccines