What do we know so far?
1.) Initial connectivity works.
2.) Connectivity with up to 4 sessions seems to work.
3.) The 5th connection doesn't work.
4.) All connectivity to server stops functioning at this point.
This smells like an IPS firewall hitting a signature and dropping the traffic. It's probably not the remote server since no FIN but not sure how reliable that is.