Paul Cantrell
In most cases, LLMs will not replace humans or reduce labor costs as companies hope. They will •increase• labor costs, in the form of tedious clean-up and rebuilding customer trust.
After a brief sugar high in which LLMs rapidly and easily create messes that look like successes, a whole lot of orgs are going to find themselves climbing out of deep holes of their own digging.
https://aus.social/@Joshsharp/112646263257692603
Paul Cantrell
Those who’ve worked in software will immediate recognize the phenomenon of “messes that look like successes.”
One of my old Paulisms is that the real purpose of a whole lot of software process in industry is to make a large-scale failure look like a string of small successes.
Paul Cantrell
The crisp “even an executive can understand it” version of the OP is:
⚠️ AI increases labor costs ⚠️
(“Why?” “Because it’s labor-intensive to clean up its messes.”)
Cameron Watters
@inthehands A problem exacerbated by the fact that many software systems use "was implemented" as their de facto criteria for success, with only the thinnest and most tenuous connection to any meaningful outcome.
Paul Cantrell
@watters Well said.
Rich Felker
@inthehands @Joshsharp Note how the party the increased labor cost falls on is not the party who operated or created the 🐈💨 garbage. Capitalism is 100% cost externalization. 🤬
Paul Cantrell
I said “the purpose of a whole lot of software processes is to make large-scale failure look like a string of small successes.”
Huh? What does that look like??
It looks like this:
✅ Meetings held
✅ Plan signed off
✅ Tests passed
✅ Iterations iterated
✅ Velocity increased
✅ Thing implemented
✅ Checkpoints checked
✅ Thing released
✅ Blinkenlights blink
✅ Line goes up
✅ Thing updated
❌ Software never •really• solves the problem it was supposed to solve in the first place, creates more problems
Paul Cantrell
or ❌ Problem it was supposed to solve in the first place was the wrong problem
or ❌ Nobody actually wanted it
or ❌ We totally failed to understand the real effect of implementing this
or ❌ The goal was designed to benefit some individual / faction within the company, not the mission
or ❌ The goal was designed to benefit the bottom line / investors / some horrid systemic evil, and is net harmful to humanity / the world
(Yes, I consider that last one a failure too.)
Jenniferplusplus
@inthehands Notably, AI is itself software and is subject to the same forces that produced this list. I'll leave it as an exercise to the other readers to figure out which of these red Xs applies in that case.
Paul Cantrell
The most hilarious example of “large-scale failure looks like a string of successes:“
Years ago, I worked on a project for retailer Megacorp Y to sell their house-branded cables on Megacorp Z’s online sales platform. It was an integration project: wire up inventory, wire up payments. The tech side was sloppy (weird, ancient APIs, Z’s official API involved •FTP• transfers (yes, really)), but ultimately quite tractable.
The problem? Internal conflict between ambitious humans.
Paul Cantrell
One team at Y did inventory, and a different team did payments. Both teams had ambitious (and pretty jerky) managers who wanted to control the •whole• project so they’d get credit for it when it launched. Both managers therefore wanted the other team’s side to fail.
We’d have meetings between the inventory and the payment teams where the engs would say, “We could do this!“ “Oooh, and then we could do this!” And the managers would suddenly cut in: nope nope nope, we can’t have this just •work•.
Paul Cantrell
I cut the cord on that contract — the agreement had only even been for me to architect it and lay the technical foundation — and left figuring it would never, ever see the light of day.
About a year later, somebody came up to me at a conference.
THEM: You’re Paul? You worked on this project at Y??
ME: Yes…
THEM: You won’t believe it: it actually •got released•! Against all odds, it went out the door and into production!
ME: 😮😮
Paul Cantrell
THEM: And it worked! Your code was great! In fact, it was so successful that it made several million in its first few weeks!
ME: 😵
THEM: …and it was so much money that it showed up as a line item on a report to the CEO, and the CEO said, “What's this?,” and when they told him, he said, “We're doing WHAT?!? Why are we selling our house-brand products on somebody else's platform??? Shut that down NOW!!!” and they pulled the plug on the whole thing!
I guess neither manager got that promotion.
Paul Cantrell
@archiloque
https://hachyderm.io/@inthehands/112956592160585649
archiloque
@inthehands I’m interested if you have other examples of this Paulism
Paul Cantrell
@maswan @kissane
Ha, coincidence! See the story I just posted downthread
maswan
Don't forget the old Nokia management style:
or ❌ The goal was designed to harm some individual / faction within the company, not benefit the mission
Rich Felker
@inthehands Usually all of the above... 🤦
Robin Osborne
@inthehands *Had me in the first half, not gonna lie*
Rich Felker
@inthehands Monoprice cables aren't on Amazon anymore?? 😂
Paul Cantrell
@janisf @Joshsharp
If companies actually had to pay for cleaning up the messes they create on the regular, it would change the world.
Janis
@inthehands @Joshsharp Oooh, but if businesses all over the world learn the lesson 3M learned about cleanup, that's *definitely* going to have a slowing, but positive impact. One might argue we're on the cusp of a major trust cleanup right now.
Paul Cantrell
@grimalkina @Joshsharp
Yup. Age-old business model:
1. Pay us to make the mess
2. Pay us to clean up the mess
Cat Hicks
@inthehands @Joshsharp I'm thinking this will be the next wave of "AI consultants" making bank too
Paul Cantrell
@dalias
I can neither confirm nor deny any specific companies involved in this story, but you have exactly the right idea
misomunje
@inthehands I have worked on several successful software projects and the checklist looks the same with a ✅ for the last item.
Paul Cantrell
@misomunje
Yes, successful projects do exist! And in my experience, without exception, they involve a whole team with eyes on some goal that is larger than the process they’re using to get there.
Uwe_Kauffmann
Reputable sources have never promised this. If you are disappointed by marketing promises that only have to cover up the fact that the sauce is already running down my leg, you have no right to be disappointed.
Paul Cantrell
@Uwe_Kauffmann @Joshsharp
Reputable sources in fact promise very little wrt to AI.
Paul Cantrell
@rposbo Had •me• in the first half — and I was there!
Alaric Snell-Pym
@inthehands hahahah! I was once brought in as a technical consultant by a primarily sales org that, wanting a product in a new sector to sell, had gone out and bought the two biggest competitors in the space and told them to merge to make a really good product. And they didn't understand why these two teams that had been built around beating each other and developing contradicting worldviews to differentiate their products couldn't just merge their products...
Paul Cantrell
@kitten_tech
As the Minnesotans say, uff da
Yer ol' Pal Gordo
@inthehands Severely underrated banger of a post.
Paul Cantrell
@geoffreyconley
I remember many years ago having an animated discussion with coworkers about what percentage of all corporate software projects are hoaxes (where “hoax” means “never really succeeds, even if it’s presented as a success).
Our conclusion is that we have no idea, and no way of finding out. Too many projects, wildly different places, no way of evaluating the question.
Truly, I have no idea. I just know this: it’s not rare.
Geoffrey Conley 🏳️🌈 🖥️ 🚴🏼 ☕
@inthehands okay I'm booking this whole thread, it's so succinct and relatable. Honestly a nice reminder of the scope at play.
I'm curious, how pervasive would you say this is? Is this like every project in every org? Or half of half? Struggling to see past my own experiences, which are honestly a lot of this on repeat.
Paul Cantrell
@geoffreyconley
The main things I look for in choosing a project:
- Cool people
- Worthy goal
- Fun problems to solve
With 1/3 I can survive. 2/3 is a great project.
I suppose I do consider the odds of the project being truly successful — but mostly I take a kind of Camusesque perspective: it’s necessary for orgs to try things, for projects to crash and burn, for people to explore what turn out to be dead ends. In the end, I just want to have been glad to have done the work.
Geoffrey Conley 🏳️🌈 🖥️ 🚴🏼 ☕
@inthehands that seems fair enough! Thanks for your response!!
Sorry to ask, but in terms of personal philosophy, do you aim to find projects and employers that AREN'T this way? Do you end up accepting that it happens, and just do your part?
Paul Cantrell
@geoffreyconley
For example, with the downthread story about Megacorp Y, my main regret is •not• about the CEO pulling the plug on everything I did a year after I left. No, that’s just hilarious!
My regret is that I found out later one of the managers had been really emotionally abusive to some of the other devs, especially the one woman on the team, when I wasn’t present. I wish I’d realized that while I was there! I wish I’d fought back! But I was oblivious to it. Big regret.
chetman
@inthehands @mhoye Oh, do I ever have Thoughts here.
For the last 17 years I’ve worked in the EVMS software space. Earned Value is tricky to implement but it WORKS. It requires rigor in determining what has actually been accomplished— and just spending time and money isn’t it.
Software projects are horrible at this. They shouldn’t be.
Real Programmer
@inthehands You get what you measure.
But you get only what you measure, because now you're measuring, nothing else matters.
Paul Cantrell
@sleepyfox
Similar thought: https://hachyderm.io/@inthehands/112247731268850864
sabik
@inthehands
Except where the clean-up can be externalised and/or deferred to next quarter?
Paul Cantrell
@sabik
Because, as we all know, next quarter everything is 100% free
Real Programmer
@inthehands "a costly myth"
New-Cleckit Dominie
@inthehands *nods so hard in university that my neck hurts*
Paul Cantrell
@ncdominie
One of my Paul refrains is that human orgs all have essentially the same family of problems, and people get way too hung up on industry vs nonprofit vs gov vs academia etc. Different dishes, but cooked with the same ingredients.
Paul Cantrell
@sboots
Glad it provoked thought!
Sean Boots
@inthehands I've been thinking about this Paulism of yours for a week. This thread is excellent. 🙌🙌
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.