And the random socket drops were due to the ARP cache entries for the destination host timing out and me not sending a query to refresh the cache before it expires.
Well, that's fixed now lol. Also tinkered a bit more and found some FFE coefficients I like better than the ones I was using, this is a beautiful eye (10.3125 Gbps PRBS15, cable de-embedded to the SMPM reference plane)
So now after kiddo goes to bed I can start writing the scopehal driver for real lol.