If someone knocks at my door good chance I wont open it unless i'm expecting it so someone could still have let them in (My buzzer is switched off all the time also)
If you have a "service" button you can ask the factor for the timings they act as a anyone can open button between set hours normally