Conversation

Notices

1. Embed this notice
   josh (josh@poa.st)'s status on Monday, 09-Jan-2023 03:25:51 JST josh
   There is an absolutely bizarre issue with 1 of 3 haproxy servers, all running on debian 11, all running haproxy 2.7.1, where only one (the one that's in North America) is constantly dropping specifically POST requests with error 408 - a client side error indicating that the client is not connecting fast enough.
   
   This does NOT happen with the EU servers. The error is coming from haproxy, not the nginx backend servers.
   
   I am losing my fucking mind trying to figure out why it's such a piece of shit. They have identical configs.
   • Embed this notice
     josh (josh@poa.st)'s status on Monday, 09-Jan-2023 03:27:00 JST josh

     in reply to
     The error looks like this.
     
     Jan 8 13:06:21 ca haproxy[71068]: xxx:47934 [08/Jan/2023:13:06:11.949] https-in~ https-in/<NOSRV> -1/-1/-1/-1/10002 408 216 - - cD-- 1029/1021/0/0/0 0/0 "POST kiwifarms.net/threads/test.129102/add-reply HTTP/2.0"
   • Embed this notice
     josh (josh@poa.st)'s status on Monday, 09-Jan-2023 03:28:21 JST josh

     in reply to
     There's actually one difference I can think of: The NA server is running LUKS encryption.
   • Embed this notice
     josh (josh@poa.st)'s status on Monday, 09-Jan-2023 03:33:40 JST josh

     in reply to

     • Dumpster Dived Waifu
     @DumpsterDivedWaifu They're running identical firewalls that should not break TCP traffic.
   • Embed this notice
     Dumpster Dived Waifu (dumpsterdivedwaifu@poa.st)'s status on Monday, 09-Jan-2023 03:33:47 JST Dumpster Dived Waifu

     in reply to
     @josh Is it possibly a firewall issue where somehow the requests are just being blocked for whatever reason?
   • Embed this notice
     josh (josh@poa.st)'s status on Monday, 09-Jan-2023 03:46:06 JST josh

     in reply to

     • Cyberia Sec
     @cyberiasec That's 8 years old. The issue is only on POST requests, and happens consistently (i.e. 100% of the time)
   • Embed this notice
     Cyberia Sec (cyberiasec@a.sc)'s status on Monday, 09-Jan-2023 03:46:14 JST Cyberia Sec

     in reply to

     @josh https://www.haproxy.com/de/blog/haproxy-and-http-errors-408-in-chrome/ maybe?

   • Embed this notice
     josh (josh@poa.st)'s status on Monday, 09-Jan-2023 03:54:53 JST josh

     in reply to
     This may be some sort of attack? I am seeing tons and tons of these kinds of errors but only out of NA.
     
     [08/Jan/2023:13:51:33.792] https-in~ https-in/<NOSRV> -1/-1/-1/-1/2 0 0 - - PR-- 2800/2797/0/0/0 0/0 "<BADREQ>"
     
     Some sort of slowhttp attack in modern haproxy that involves malformed requests?
   • Embed this notice
     AffirmativeNod (affirmativenod@poa.st)'s status on Monday, 09-Jan-2023 21:13:56 JST AffirmativeNod

     in reply to
     @josh Josh, can you help me? I logged into something weird, now I keep getting strange requests, says it's from KF. I'm very hesitant to press "Accept All".
   • Embed this notice
     josh (josh@poa.st)'s status on Monday, 09-Jan-2023 21:13:56 JST josh

     in reply to

     • AffirmativeNod
     @AffirmativeNod how about an email champ
   • Embed this notice
     josh (josh@poa.st)'s status on Tuesday, 10-Jan-2023 03:00:54 JST josh

     in reply to

     • pmtuhole
     @pmtuhole Promising, thank you
   • Embed this notice
     pmtuhole (pmtuhole@poa.st)'s status on Tuesday, 10-Jan-2023 03:00:57 JST pmtuhole

     in reply to
     @josh tigrerayé.org/english/post/15-the-strange-case-of-the-tcp-black-holes/