I do have to admit. When I went to Santa Cruz (last year), I did notice a lot of interesting people walking about. As if the island attracted a type, so to speak.
I didn't mingle. It was a family trip, so we went all touristy there. But if I had to guess, I'm sure most people would have told me "stories" of how they are there, but they used to be "so and so". It seemed like that kind of place.
I think the healthiest thing to do is to let them talk, no need to push back. In the very least is entertaining.