It works 🎉
Ok, the setup is very janky because of a couple of issue :
* I of course screwed up TX/RX at one place, so the usb3 connector part didn't work, had to go through the slot to go to the device, ugly but it worked.
And I swear I had tripled checked ... still screwed up :/
* The SN75lLVPE802 is meant for SATA not PCIe and doesn't have DC termination. The datasheet say you can use it for PCIe and use 330R to GND to make PCIe RX detect work.
That doesn't work :/ Had to use lower R.