Turns out, there's a lot of details to get right when implementing a floating window space. For example, dialog windows should always show above their parent window. Otherwise, it's easy to lose them under the (usually much bigger) parent.
The WIP floating branch in niri now handles this properly, even for xdg-desktop-portal dialogs (like file chooser) as long as the app correctly parents them via xdg-foreign.