If you were my neighbour I would tell you to go for it. Personally I would not bother going around them all but I would speak to the elderly neighbour who tends the garden. It shows a bit of acknowledgement for what they do and also tell or show them where you are planning to put the pool. They may have a good reason you have not anticipated for it going in another place.