Yup. I've heard people say 'but you can't be prejudiced against white people' - oh but you can, just in the same way you can be prejudiced against men, or heterosexuals etc etc!
If you're shouting out at someone that's never caused you any harm before then you are prejudiced, period.