@jos Thanks for looking into it. I think the answer for me is that for my totally unimportant personal site I'm just going to brave it: check on a test site which edits need to be made and what they look like then quickly do them on the real site. Likely nobody will see it in the meantime. For anything even slightly more important organisations are going to need staging sites anyway and mechanisms to deploy changes atomically. Also with Block Visibility I think you're going to have to find the added blocks first to flag them as invisible, which you can only do after switching to the theme.
Or maybe I should just let go of the idea that you can easily switch themes...