Per off-the-record chats with F5ers (assuming they're correct), F5 didn't exactly manage OSS NGINX well. I was told that Maxim essentially ran OSS NGINX as a volunteer despite being employed by F5. That could all be totally wrong but IMO it fits with their behaviour I've experienced - it always felt more like *available* source than *open* source to me. This might all have serious repercussions, NGINX (& OpenResty which is based on it) is *everywhere*, 32% of the web https://www.netcraft.com/blog/january-2024-web-server-survey
Seems to be a bit of a split in #NGINX - It's been forked by a previous F5 employee in an effort to keep it free from interference. https://freenginx.org/
Hands up if you caused a global outage today... Just me? Sorry!
I was making a change to our "outside the UK" CDN config today for www.bbc.co.uk & www.bbc.com & the change included 2 bugs which pre-testing didn't spot: - A regex typo which caused 404s on www.bbc.co.uk - An incorrect TLS cert on the CDN origin which caused 503s on www.bbc.com
These caused ~7 minutes of significant global outage.
I spent most of the afternoon writing tests to catch this for next time.
To add a little detail I should have included initially:
This is a web server/proxy software issue - it's a generic issue (rather than software-specific) so it's going to affect lots of software implementations.
It'll be announced at midday UTC today (10th Oct 2023).
If there isn't an update you can deploy quickly for your affected services immediately (there should be for the better known software, they've had advance notice) then you should consider disabling the affected element until there is.
Can't share more right now but it's important so don't forget (& tell your friends!).
Earlier we were talking about DDOS & a colleague asked what TLS versions are used by the botnets these days...So I checked the most recent big-ish one we had : **TLS Protocol Percentage** TLSv1.3 55.77% TLSv1.2 44.23% TLSv1 0.00% This was over something like 115M total requests. So the answer is that the botnets have better TLS libs than our overall audience. Fun times. #infoSec#webDev#TLS#DDOS
@zkat Ah crap, does this mean i need to migrate to somewhere better? (currently on mastodon.social as i had even less idea what i was doing when i first registered than i do now)
@rbairwell@zkat@Gargron Oh wow, yeah, I can imagine he's having a _very_ busy time! Thanks for the info...I am also considering running my own service - mulling over a serveless architecture which could be fun...