I am no infosec or other security expert, but I've always hated visiting homes with smart assistant devices. At this point, I would honestly be tempted to tell friends and family that I won't step foot inside if those are present.
I know this is an ironic statement since I do own a smartphone. I don't know how far I can draw the line. But I figure starting with the Alexas is better than nothing.