If you wanna get legal this could be considered a breach of your right to peaceful enjoyment. They legally cannot enter without your permission, and they can't let themselves in whenever they want even if you agree to an inspection. Has to be a time and date that suits you.