buildah spending 6 minutes and counting doing literally nothing (replaying a dockerfile with every layer cached) while somehow performing over 100 MB/s of IO continuously will radicalize me
Conversation
Notices
-
Embed this notice
✧✦Catherine✦✧ (whitequark@mastodon.social)'s status on Monday, 22-Dec-2025 16:19:43 JST
✧✦Catherine✦✧
-
Embed this notice
✧✦Catherine✦✧ (whitequark@mastodon.social)'s status on Monday, 22-Dec-2025 16:30:49 JST
✧✦Catherine✦✧
genuinely why did i even bother adding caching to it. normally the network is the slow part (since the cache server is not in the same datacenter) but here every cache layer is already on the local machine. all buildah needs to do is Literally Nothing. instead it does Nothing almost as slowly as rebuilding the whole thing from scratch
-
Embed this notice
✧✦Catherine✦✧ (whitequark@mastodon.social)'s status on Monday, 22-Dec-2025 16:32:40 JST
✧✦Catherine✦✧
it's like the spiritual inverse of that one time i spent something like eight hours optimizing an idle loop to run faster by counting every nop i execute
-
Embed this notice
✧✦Catherine✦✧ (whitequark@mastodon.social)'s status on Monday, 22-Dec-2025 16:35:10 JST
✧✦Catherine✦✧
@hailey sincere question: how do people live with this shit
i avoided direct exposure to containers for most of my career, but could no longer do so since about a year ago. it's been non-stop misery ever since. people hate on JavaScript but like, i have JavaScript projects i _like_ that work for me: fast and efficient.
no amount of effort put into containers has ever produced anything of the sort. best i did was to link my entire project into one ELF and then ship that as OCI
-
Embed this notice
Hailey (hailey@hails.org)'s status on Monday, 22-Dec-2025 16:35:11 JST
Hailey
@whitequark the containers experience lol
-
Embed this notice
Josh Simmons (dotstdy@mastodon.social)'s status on Monday, 22-Dec-2025 16:39:13 JST
Josh Simmons
@whitequark @hailey don't worry we can put everything in a container and then there's no baseline to compare to!
-
Embed this notice
✧✦Catherine✦✧ (whitequark@mastodon.social)'s status on Monday, 22-Dec-2025 17:05:28 JST
✧✦Catherine✦✧
@buzzyrobin @hailey yeah i can see that
-
Embed this notice
Robin ╰(ಠ_ಠ✿)╯ (buzzyrobin@chaos.social)'s status on Monday, 22-Dec-2025 17:05:30 JST
Robin ╰(ಠ_ಠ✿)╯
@hailey @whitequark Also yes. It just feels like a bunch of people have never actually experienced a build run that ran quickly, and do not miss it as much as some of us do.
-
Embed this notice
Robin ╰(ಠ_ಠ✿)╯ (buzzyrobin@chaos.social)'s status on Monday, 22-Dec-2025 17:05:31 JST
Robin ╰(ಠ_ಠ✿)╯
@hailey @whitequark remembering a company rails app that, after a herculean optimisation + parallelisation project, got the build down to 20 minutes. (it went back up to ~40 minutes shortly after.)
-
Embed this notice
Hailey (hailey@hails.org)'s status on Monday, 22-Dec-2025 17:05:32 JST
Hailey
@whitequark i think a lot of devs have just never known better. “CI has always been slow”, “deploys have always been slow”, “servers have always been complicated to configure and maintain”
-
Embed this notice
✧✦Catherine✦✧ (whitequark@mastodon.social)'s status on Monday, 22-Dec-2025 17:05:44 JST
✧✦Catherine✦✧
-
Embed this notice
Raven Onthill (ravenonthill@mastodon.social)'s status on Monday, 22-Dec-2025 18:36:44 JST
Raven Onthill
@whitequark @hailey this sort of problem is why Go doesn't use the FSF tool chain (aside from the place being run by sexist jerks.)
-
Embed this notice
✧✦Catherine✦✧ (whitequark@mastodon.social)'s status on Monday, 22-Dec-2025 18:37:56 JST
✧✦Catherine✦✧
@ravenonthill @hailey isn't cgo a de-facto requirement? it's been difficult to avoid it in my work, and i thought the go runtime itself uses cgo
-
Embed this notice
✧✦Catherine✦✧ (whitequark@mastodon.social)'s status on Monday, 22-Dec-2025 18:51:35 JST
✧✦Catherine✦✧
@maswan @hailey i started my career as a rails dev, doing much the type of thing mastodon is built with, and i wouldn't want to run it myself
-
Embed this notice
maswan (maswan@mastodon.acc.sunet.se)'s status on Monday, 22-Dec-2025 18:51:37 JST
maswan
@hailey @whitequark I'm an oldfashioned linux sysadmin, and I avoid containers and most modern projects as far as I reasonably can because of complexity, performance, and maintenance issues.
Maintaining this mastodon instance is an example of me stepping outside of that comfort zone to experience the modern world, but the experience is kinda shit compared to nice stable software like postfix and bind available from the main repo.
-
Embed this notice
✧✦Catherine✦✧ (whitequark@mastodon.social)'s status on Monday, 22-Dec-2025 19:06:50 JST
✧✦Catherine✦✧
i have been informed that this is not a bug or a misinterpretation on my side but rather an explicit design choice buildah commits to, and now i feel sad
-
Embed this notice
✧✦Catherine✦✧ (whitequark@mastodon.social)'s status on Monday, 22-Dec-2025 19:14:40 JST
✧✦Catherine✦✧
@hcmh see other replies in this thread
-
Embed this notice
Christian Bardey (hcmh@ieji.de)'s status on Monday, 22-Dec-2025 19:14:41 JST
Christian Bardey
@whitequark oh no, anywhere I could read about why? I recently wanted to replace Docker-in-Docker with buildah in a CI pipeline, and saw similar behavior: buildah is quite slow when the Dockerfile didn't change at all :(
-
Embed this notice
Raven Onthill (ravenonthill@mastodon.social)'s status on Monday, 22-Dec-2025 19:15:27 JST
Raven Onthill
@whitequark @hailey no, Go has its own toolchain; cgo is used to call C code, admittedly something you may have to do frequently. AFAIK that toolchain exists partly because the Go developers didn't want to deal with the complexity of existing open source tool chains. (Rob Pike is on the Fediverse; you can ask him, and he'll probably tell us why we're both all wet.)
-
Embed this notice
✧✦Catherine✦✧ (whitequark@mastodon.social)'s status on Monday, 22-Dec-2025 19:18:24 JST
✧✦Catherine✦✧
@ravenonthill @hailey i'm familiar with that rationale and the general structure of the Go compiler (i am a compiler engineer by trade), but i'm a novice Go user and it's not yet clear to me whether you can, in practice, avoid interacting with the FSF toolchain by eliminating all C code from your transitive closure
go does achieve some impressive things via vertical integration, but in practice it's usually very hard to stay only within your vertical.
-
Embed this notice
✧✦Catherine✦✧ (whitequark@mastodon.social)'s status on Monday, 22-Dec-2025 19:20:49 JST
✧✦Catherine✦✧
@hailey @icing it's hard not to see a conflict of interest when someone has a tool they'd like others to use but the tool is hard to use efficiently & they charge per request, not per month
-
Embed this notice
Hailey (hailey@hails.org)'s status on Monday, 22-Dec-2025 19:20:50 JST
Hailey
@icing @whitequark also a symptom of neoliberal outsourcing culture imo - the pendulum has swung too hard back from NIH. if you can achieve some need by bringing in a tool or a framework then great, but cooking up a small and perfectly adapted solution in house for a "non core" business need like CI or deploys is something you have to justify. jeff bezos calls this kind of work "undifferentiated heavy lifting", but when you lose the specialisation you also tend to lose many opportunities for efficiency
-
Embed this notice
Stefan Eissing (icing@chaos.social)'s status on Monday, 22-Dec-2025 19:20:51 JST
Stefan Eissing
@hailey @whitequark
It‘s an area where few can make it better, but everyone can make it worse (add a dependency or an unreliable test case).An organisational challenge, not a technical one.
-
Embed this notice
Raven Onthill (ravenonthill@mastodon.social)'s status on Monday, 22-Dec-2025 19:21:52 JST
Raven Onthill
@whitequark @hailey true enough, unfortunately. Maybe within Google, but not in the wider FOSS environment.
-
Embed this notice
✧✦Catherine✦✧ (whitequark@mastodon.social)'s status on Monday, 22-Dec-2025 19:33:59 JST
✧✦Catherine✦✧
@ravenonthill @hailey i _personally_ managed to stay away from cgo because i really wanted that easy cross-compilation, but it took some effort
clang makes cross-compilation quite easy too, unlike gcc, which in my view makes it a significantly more mature compiler
-
Embed this notice
✧✦Catherine✦✧ (whitequark@mastodon.social)'s status on Monday, 22-Dec-2025 19:57:49 JST
✧✦Catherine✦✧
@ravenonthill @hailey clang and llvm are significantly newer (though they are old enough to straddle a fine line between "mature" and "legacy" pushed back only through monumental effort, having mostly advanced the industry forward and passed the torch along)
lldb is ... kind of notoriously poorly documented, i've professionally used llvm for many years and wrote numerous compilers using it and i can't use lldb if my life depended on it
-
Embed this notice
Raven Onthill (ravenonthill@mastodon.social)'s status on Monday, 22-Dec-2025 19:57:50 JST
Raven Onthill
@whitequark @hailey isn't clang/llvm newer technology? Or do I have the history wrong? (My problem with clang/llvm is the basically non-existent documentation. I have never figured out lldb.)
-
Embed this notice
Raven Onthill (ravenonthill@mastodon.social)'s status on Monday, 22-Dec-2025 20:15:09 JST
Raven Onthill
@whitequark @hailey snicker snicker snicker. Now I didn't feel so bad about not figuring it out; I'd always assumed my problems were being a lazy dilletante dinosaur. What is current technology, then?
-
Embed this notice
✧✦Catherine✦✧ (whitequark@mastodon.social)'s status on Monday, 22-Dec-2025 21:51:00 JST
✧✦Catherine✦✧
@ravenonthill @hailey if you ask me, cranelift is a good example of a cutting-edge industrial batch compiler (i.e. something in the same general category as llvm but newer); graal/truffle are good examples of cutting-edge industrial jit compilers
these are the big and mature projects; there's obviously many more experimental ones that i wouldn't be able to begin to enumerate
-
Embed this notice
Raven Onthill (ravenonthill@mastodon.social)'s status on Tuesday, 23-Dec-2025 01:50:29 JST
Raven Onthill
@whitequark @hailey thanks!
-
Embed this notice
Glyph (glyph@mastodon.social)'s status on Tuesday, 23-Dec-2025 03:46:30 JST
Glyph
@whitequark @hailey the problem that containers solve is that linux never really developed a platform envelope for software delivery. I deployed lots of software in the pre-container days; when I got to manage the deployments myself, I wrote a comprehensive deployment package format and a mostly-isolated runtime target. even that “mostly” gave us problems when the platform would do stuff like change the C++ ABI during a security update, but it worked pretty well
-
Embed this notice