@feld having the IP on an interface doesn’t mean it routes. In order to accept a packet it must be a)addressed to an IP the device has in its interfaces, or b)be directly routable to a nearby gateway with a static route.
The server will not accept the packet unless it’s on the loopback, in which case it says “oh, that’s me” and take the packet, and then loopback it and it falls through the WireGuard static route and is sent the way you want.
1/2