Ryan Townsend
👏 more 👏 developers 👏 need 👏 to 👏 hear 👏 this
I can count on one hand the number of my clients over the past couple of years who haven't either over-architected for scale or were unnecessarily concerned about it (prior to coming to me for strategic advice, of course 😉).
You don't need to understand Distributional Little's Law to figure this out, it's obvious with primary school level math.
(screenshot excerpt from https://tailscale.com/blog/new-internet)
Jens Finkhäuser
@ryantownsend back in the day, we only measured requests per second. If I read about longer time spans, I already know it's rubbish without looking at the actual number.
Page views are more realistic in some ways, but they map relatively neatly onto requests once you take caching into account. Modern dev frameworks complicate this as well, but the upshot is, you can still get away with applying an application specific multiplier to request to get page views and vice versa across sufficient...
FErki
@ryantownsend I designed/built/ran infra where we considered 100k/s requests a nice quiet period, with ~350k/s daily peaks, plus ingest ~1TB/hour data.
Up to this scale, we never found a challenge for which cloud, containers, or Kubernetes would have been the simplest solution that could work – also far from cheapest.
All bare metal on #Gentoo. Managed with #Rex. Globally from ~30 to ~400 servers, then beyond.
Solve problems _you_ have. Not what others think you should solve.
Haelwenn /элвэн/ :triskell:
@ryantownsend Heh seeing it again on my timeline made me wonder about the views-per-month on my main server box that easily could serve 100× more requests without changing anything:
$ wc -l /var/log/syslog/NightmareMoon/local7_nginx/2024/10/*.log | grep total 3893344 total $ wc -l /var/log/syslog/NightmareMoon/local7_nginx/2024/09/*.log | grep total 3997733 totalYeaah, ~3.9M requests per month, I guess for page views you could divide by something like 10 although my stuff has caching so a page view can easily be just 1 request.
:debian:anokasion
Haelwenn /элвэн/ :triskell:
@anokasion @ryantownsend Nah, but well a lot of it isn't web browser traffic. I have fedi (including multimedia), git, tarballs of my software, … on that box. And well… scrapper bots are a thing.
fedi does count for a lot but it's still in the same order of magnitude:
$ grep -v queer.hacktivis.me /var/log/syslog/NightmareMoon/local7_nginx/2024/10/*.log | wc -l 1172360And reminder: It's per-month.
Wolf480pl
@ryantownsend UI developers, especially web UI developers, need to hear this even more.
Wolf480pl
@ryantownsend @grumpasaurus
> a DevOps team isn’t going to advocate moving away from Kubernetes/IaaS/bare metal
Unless the DevOps team is overworked / burnt out, they can barely keep the existing stuff running, and the devs are making a new app that needs unusual stuff, and most new apps in the company die after a quarter - I think in that situation, many DevOps would advocate for running that one app on Heroku if that's what the devs already know.
though that's an extreme case I guess
Allan Chow
@ryantownsend I'd like to think that people aren't necessarily being malicious to keep their jobs, but moreso maybe a bit too immature in understanding the complexity of taking on a new-to-them solution, especially when they haven't had the past experience of operating a similar process.
"Let's adopt this CI/CD pipeline so that my job in troubleshooting it is secure" - said no one
Ryan Townsend
@grumpasaurus not directly, but a DevOps team isn’t going to advocate moving away from Kubernetes/IaaS/bare metal and just using Heroku or Render.
React has seen enormous, continuous adoption despite very apparently flaws because “that’s where the employment/money is” (I’ve researched this before for my conference talks, it was the #1 voted answer on Reddit for why people use it)
Ryan Townsend
@grumpasaurus absolutely, context is always important, hence my use of "most businesses" and "99%+"
Ultimately: most applications just aren't special. Many are barely more than simple CRUD.
It's not uncommon that the people on hand are the problem... they are the ones driving the infrastructure complexity in order to keep their jobs and pad their CVs. It can mean swapping them out for more productive people who focus on shipping.
Allan Chow
@ryantownsend I think it just depends on your situation, use cases, and constraints that a vendor may have on your needs. Of course the most paramount situation being the type of people you have on hand and the budget you have to hire what function you're lacking.
Allan Chow
@ryantownsend a lot of engineers suffer from focusing on executing a solution, rather than solving a problem. With that said, going to click the link and read!
Allan Chow
@ryantownsend mm. I do agree with where they're getting at and it will be really cool to see if they unlock startups to self host again. The great MailChimp server migration story being something similar.
I don't think that's the cause for the insane complexity of "just launch a service" comes from though.
Ryan Townsend
@grumpasaurus fwiw, I see Kubernetes as basically self-hosting, yes it might be atop IaaS but your own team is responsible for uptime, security patching etc.
Most businesses should be outsourcing to a PaaS like Heroku, Render etc.
For 99%+ of use-cases, until your bill hits six figures monthly, it just doesn't make sense to hire DevOps/Infra people.
Wolf480pl
@ryantownsend @grumpasaurus
Also, I think a large problem is that while we teach people to Kubernetes and stuff, we don't teach them why, and when.
It took me a few years of hating k8s from sidelines, and then a couple years of working with k8s at $dayjob, to finally figure out what it's for:
It's for when you have multiple programs and multiple computers and you have trouble figuring out which program should run on which computer.
How many companies would avoid k8s if they knew that?
Allan Chow
@wolf480pl @ryantownsend yeah that happens quite often! Not too long ago a company I worked at was running bare metal and keeping the business going and in a huge multi year scaling initiative, some architects chose to adopt an entirely different stack expecting to run in AWS despite having no experience running in AWS and the stack itself.
In those situations, where there's timeline and technological cliffs that the company is barreling towards, decisions to buy into something you may not understand but gives you the confidence that it *just works* are extremely easy to make.
Allan Chow
@wolf480pl @ryantownsend yeah you really need someone with either a lot of foundational background to learn k8s (like early in its inception) or someone who has a lot of operational background in k8s to know its kinks to help the team get onboard
Wolf480pl
@grumpasaurus @ryantownsend
I mean, if you tell people what kubernetes is for, and they realize they have one computer... or many computers but they want to run the exact same programs on each of them, then they might realize they don't need kubernetes.
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.