Thanks for the thoughtful comment, @jenkinrocket. It's very interesting to read your thoughts about this topic.
I'm aware about philosophical argument about math as invention vs math as discovery. For me personally, concept of 2 is as real as concept of information, for example. Information is not attached to any physical place and might be viewed as a purely abstract concept, yet it is clearly defined in physical phenomena like uncertainty principle.
Physical environment (noise) can be modelled through math, but when you're working with pure math you never (as far as I know) have to think about this kind of inherently uncontrollable variables, because environment is essentially defined by you (or arises from other precise mathematical definitions).