I wouldn't worry about it too much. As someone who owns one of those tiny drones I can tell you that they're really hard to get the hang of because the wind has such an impact on them. Unless it was overtly clear that the pilot was filming you, the likelihood is that they didn't mean it.