Totally fine to go alone, it can actually be a lot more enjoyable than going in a group. From my personal experience, some people will be surprised and feel sorry for you (dunno why), others will get it. Polo is a super friendly club, you will have a great night!