This is why I hate affordable/social housing all being built in one area (normally areas where property owners don't have the money to kick up a fuss).
Spread it out. Have social housing available in a variety of areas integrated with private property. Means decent folk get housed in established areas and arseholes can get picked out and sorted out easily.