Paul Cantrell
An enthusiastic YES to this from @henrikjernevad:
https://mastodon.social/@henrikjernevad/112160702537117343
I recognize every single one of those examples from the post’s intro. And I recognize the pattern: developers imitating tech giants on small projects where those giant-scale approaches make no sense at all.
I like Henrik’s question: does it scale •down•? Does it scale down to the size of our actual team, our actual project?
1/
Paul Cantrell
A lot of my own software work has been for very small companies: arts nonprofits, early startups, one-person bootstraps, community orgs that are and always will be small.
These orgs have problems that are very, very different from the problems that, say, Google has. There’s no sense in them inheriting the miserable tech tradeoffs a giant corp faces! Enjoy being small! Do it the simple way! Stay flexible!
2/
Paul Cantrell
You’d think that “it has to scale to 100 million users!!” is a pressure that comes from over-eager owners counting on massive growth, but in my experience, misapplying the patterns of tech giants to tiny orgs is a mistake that usually comes from us developers. We like shiny things. We over-scale either because the tech seems cool, or just because we feel like we’re •supposed• to.
But: What problem does the tool solve? Do we have that problem?
3/
Paul Cantrell
One of the problems that @henrikjernevad doesn’t get into in the blog post is that scalable code, like performant code, tends to be •brittle•: larger, more complex, more difficult to modify.
If you have only a handful of users, you have the incredible luxury of being able to write non-performant, non-scalable code that allows for •revision•. You can fiddle with design! You can learn from your mistakes and adjust quickly! It’s an •incredible• competitive advantage.
4/
Paul Cantrell
It sucks being Google: if you so much as fart, you have a million users overnight. The second your product goes out the door, it’s half-calcified. Human-centered iterative design process? Forget about it!
Apple generally solves this by incubating things in secret forever. Google and Amazon solve it by training users to live with pervasive UX problems. If you’re huge, there is no good answer.
If you’re small, enjoy your incredible leg up: you have a chance to build good things fast!
5/
Jeff Miller (orange hatband)
@inthehands Has your experience of development at modest scale given you a set of tools and recipes you like to turn to?
I've been paid either to do tiny tool stuff, or to contribute to the large and brittle systems at Web Scale, so the happy routes in the landscape in between are mostly unfamiliar.
Paul Cantrell
If you are tiny and looking to stay tiny…you have my heartfelt blessing.
If you are tiny and looking to grow, keep it simple, focus on building something great, and hope that one day you are •lucky• enough to have so many users that you need to rearchitect for scalability.
Honestly, more often than not, that moment of needing to rearchitect for scalability is a sign of •success•: somebody chose the tools that were appropriate to being small, and they succeeded.
/end
Paul Cantrell
@jmeowmeow Off the cuff:
- Web Scale can be small too. How much traffic do you get, really?
- They’re asking for an app, but probably want a responsive web site.
- If you really do need an app and the app is small, maintaining two native codebases for iOS + Android is easier than it sounds.
- HTML generated on the server >> fancy client JS for •many• projects.
- Or can it just be a static site?
- If you do need a DB, single-node Postgres works up to a surprisingly large scale.
- Simple + obvious
Jeff Miller (orange hatband)
@inthehands I guess I am starting off okay then. I have a low complexity, non-social site aimed at creatively reskinning current public data of low cardinality, and the main difficulty has been finding a stable library for a core flow (which I believe I have in hand).
I shall choose simple and straightforward for the rest, drawing from examples when possible.
Paul Cantrell
@jmeowmeow
Sounds like you’re probably doing just fine!
Paul Cantrell
@mainec
Thanks, a very interesting deck!
The slide your remembered seems like really good advice: consider 5x growth, maybe 50x. Don’t plan for >100x growth; no matter how much you plan, that’s going to be a rethink and rewrite.
MaineC
@inthehands https://research.google.com/people/jeff/Stanford-DL-Nov-2010.pdf ... Found it. The design for growth, but not to scale infinitely and the 'numbers everyone should know' stuck in my head over the years
MaineC
@inthehands it's too long ago to be able to dig out the slide deck with that any longer
MaineC
@inthehands wisdom coming out of Google decades ago: what works with ten users is so different from what you need with a few orders of magnitude more that you'll need to rewrite anyway
Jeff Miller (orange hatband)
@inthehands Appreciate the vote of confidence. Too busy a set of choices out there, but I have a couple of communities to bounce things off of which I should call on a little more often.
Really, a Node server looked like a good choice at the start, and I suppose I am getting portable knowledge, but mostly it's been a direct lesson in complex messy dependencies. When I'm tempted to write my own binary image compositor, I figure a wrong turn was taken somewhere.
Paul Cantrell
@jmeowmeow
Without knowing anything at all about your problem, I’d definitely consider whether your site (or some parts of it) can be statically generated offline. Code that runs once per data update instead of once per request solves a whoooole lot of problems.
(And keep in mind that you can even statically generate an API sometimes! One project I still maintain has a read-only REST API that’s actually just a bunch of .json files served by a vanilla web server.)
Paul Cantrell
@jmeowmeow
I don’t have a lot of love for Node/Express, though a lot of that is personal taste. If your problem is “list of endpoints, code that outputs JSON for each, serve large numbers of requests concurrently,” then yeah, it’s pretty good for that (though these days I’d be looking at what Swift, Rust, Go offer in that space). If you have complex data, or complex logic, or web site with assets, or or or…then you quickly find yourself reinventing a Rails-shaped app framework yet again.
Paul Cantrell
This @henrikjernevad, 100%:
https://mastodon.social/@henrikjernevad/112939338754671394
If you don’t have a clear sense of what you need to •test• to ensure that your scalability plan works under your own real-world conditions, that’s a sign that you are making it up and don’t actually need that scalability yet/ever.
Jeff Miller (orange hatband)
@inthehands It's Node / Express, with Jimp as the "no-deps, doesn't need to scale" image library. Mostly intended for users to go through a style and location picker wizard, and bookmark an image source URL. It's an incomplete homage to a site I miss which went down and never came up.
github.com/jmeowmeow/PixieReport if you're curious. It could launch any week now, or so I tell myself.
Paul Cantrell
Consider: the cost of building supposedly scalable code and getting it wrong and having to redo it when scale actually hits is generally •higher• than the cost of keeping it simple and then having to redo it when scale hits.
Paul Cantrell
@rlcw
All that. A question every startup should consider: “Would you rather (1) build something that runs slowly a year down the road and you have to upgrade, or (2) build the wrong thing — wrong product, wrong design — and find yourself unable to alter it before you run out of money?”
Becca Cotton-Weinhold
@inthehands Very true. Main thing I advise Start-up clients to do is to focus on figuring out what is important now, and build the product in ways that it is kind of modular and easy to scale and extend in parts, when that time comes. This usually makes things less complex and encourages clean simple design patterns in all areas. For scaleups, I love the strangler pattern, to scale & rebuild the product bit by bit.
Paul Cantrell
@pdcawley
Yes! Twitter and GitHub are the two examples I most often use.
Piers Cawley
@inthehands the example that springs to mind is Twitter. Twitter got to stupidly large whilst being written in Ruby, back when Ruby's only implementation was a painfully slow tree walking interpreter. They got much bigger, but Ruby and MySQL got them to the point where they mattered.
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.