Hmm. I think black, as a whole, is most representative of the unknown and the chaos that is associated with the lack of control while in the domains of the unknown. From a sociobiological perspective, it makes perfect sense that the color of night (or rather, the absence of it), when we are blinded and the world is infinitely more dangerous, that we should perceive the color black as threatening.
I think all interpretations of the color are secondary to that primary meaning. For example, people in power or people with superb control (like James Bond, per your post) wear black because they want to represent to all potential viewers that they are either aligned with the forces of chaos, or are (mentally, physically, spiritually) powerful enough to have dominated the realm of the unknown. This carries the effect of intimidating the perceiver, or at the very least, making all aware that there is a person of equal standing.
Moreover, though you more or less already mentioned this, I think black is again worn in times of distress or sorrow because it represents a descent into chaos. The pain and shock experienced when made aware of death can leave people susceptible to encountering the shadowy parts of their brain they do not typically see. This descent into the unconscious, the metaphorical dark, is where we see black again as a constant.
Great addition, I totally agree with you! Thanks a lot!