Usually a doctorate is considered the hardest to get, but then that leads to the question of in what discipline.
Unless you are a math prodigy, it would make sense that a degree in quantum physics is probably harder to get than a degree is some other subject.
The best advice is to pursue what makes you happy and only get a degree if it makes sense for your career. Many times people end up with more debt chasing an advanced degree than they can make once they have it.