@functionalscript But if your user handle is a domain name (where a TXT record mentions the current server) then moving to another server is completely transparent.
One could also do a simple form of content-addressing:
my-server.example/@user.example/12345
Such a URL could be routed to the correct server via the user handle. Then the path would work on every server. For complete transparency, one could use a domain that points to the current server.
That would be good enough for my needs.