I'm the same as you. I dread anyone trying to talk to me in public. Mostly because I lack the ability to make small talk. The getting angry at you thing is ridiculous. That's just their problem. I do like the fact that people are happy to chat to strangers though.