It's common sense really ffs. If you walk through parkhead/gallowgate wearing a rangers top on matchday yer obviously going attract trouble and vice versa for celtic/ibrox.
Anywhere else even on matchdays yer absolutely fine.
Aside from the above exception the colour of your shirt has little to no effect on how much trouble you'll attract.