Will probably be pilloried for this but, nah you’re right, it is a drug. One people that take it don’t think about the impact it has on those around them in my experience.
If I were you I’d phone 101 each time those guys are there, you don’t deserve to put up with this.