Have you tried wearing a hi viz jacket and carrying a clipboard?
In 2020, myself and @dizzydiscovery took this approach just once. Some local accosted us and asked lots of pointless questions about whats being done to the building. We pulled off the hoax but he cost us around 20 mins in time.. bloody arsehole!
Haha there's always one busy body to spoil our fun. I'm looking forward to scoping out a few abandoned churches at some point.