I guess only action in that situation is ask the admin to have some kind of defederation of the instance?
Another approach in longer term might be allowlists, they could provide safer servers (with lower reach being the compromise).
Think @gotosocial has allowlists on its roadmap?