Notices by Lennart Poettering (pid_eins@mastodon.social), page 7

The latter sucks in major ways, since we basically had to reimplement a subset of the broker ourselves, with message multiplexing, subscription, signal matching and a lot of other stuff. Because this was so messy we never did the same for journald, userdbd or homed.

These two are just the biggest issues with D-Bus, but there are a lot more, in my eyes. Hence, quite some time ago we started to use a different type of IPC for these cases, initially just internally.

we just let systemd's socket activation logic listen on an AF_UNIX/SOCK_STREAM socket, and then let it fork off a new bootctl instance for each connection. That instance then just processes that connection and is done. And it's easy: it just does what it usually does, but instead of reading the commands to execute from the command line it just reads them from a small JSON object it gets from STDIN. And it just writes its output as JSON to STDOUT, done.

In fact, because bootctl already…

To give one example, "bootctl" is a small tool that installs the systemd-boot boot loader into the ESP for you. It's a command line tool that synchronously copies a bunch of files into the target mount. We always had the plan to turn that into a D-Bus service, but never actually did it, because doing that is pain: we'd have to turn it into an event loop driven thing, which is just nasty for something so simple that just copies some files.

In a Varlink world, the problem goes away:

…, it's JSON use make it more conceptually compatible with the rest of the world and various other things.

It's also a lot easier to write Varlink services than D-Bus, because it allows you to handle each connection in a different process, thus being compatible with codebases that do not have event loops (D-Bus due to its multiplexing forces you to process all messages within the same process, and due to the global ordering within a single event loop).

You are of course right if you say that there are already so many of those, why another? And you'd be right. My answer to that is that sd-json is much nicer to use than most, since it doesn't just allow you deal with JSON objects, it also helps you with building them from C data structures, and to map JSON objects back to C data structures. It also helps you with higher level operations that the low-level JSON datatypes leave you in the cold with:

There are various bindings for Varlink available from the Varlink project, but with systemd v257 we now make systemd's own C implementation available publically too: sd-varlink.

sd-varlink has been around for quite a while, and is quite well tested (including fuzzed) by now hence. It's already driving your systemd installations, except mostly internally so far.

Since Varlink uses JSON for marshalling its messages, sd-varlink comes with a companion API: sd-json. It's another C library for JSON.

…supported JSON output anyway, the output side was done pretty much anyway.

Anyway, there are many other stories like that.

Suffice to say, in v257 there are now 19 Varlink interfaces/services, which we added in a short time, for various things that never had them before when D-Bus was our sole focus, because it was so nasty to add that.

(For comparison: we provide only 11 D-Bus API services at this time).

At Linux' best conference, All Systems Go! 2024 in Berlin this year I gave a (brief) talk about Varlink, and why you should consider it. If you want to know more about the concept, this might be a good starting point:

https://media.ccc.de/v/all-systems-go-2024-276-varlink-now-

And that's all for now, enjoy!

i.e. for example handles base64 encoding/decoding for handling binary blobs within JSON automatically, or it helps you with dealing with JSON's >53bit integer problem, and various other things.

Right now, documentation for sd-varlink and sd-json is scarce (one could even say "barely existing"), but there are plenty of real-life examples in the systemd source tree, of course.

3️⃣ Here's the 3rd installment of posts highlighting key new features of the upcoming v257 release of systemd.

So after the relatively heavy fare of the 2nd episode, here's something to digest more easily.

Since a while systemd has been showing nice terminal progress bars when doing certain slow operations (for example, when systemd-repart initializes a disk). With v257 we go one step further with this. Whenever we show the progress bar we'll now also issue certain terminal ANSI sequences …

… that tell your terminal emulator that a slow operation is going on and what the progress currently is.

The ANSI sequences for this are commonly used on Windows, where the new Windows Terminal implements them and paints a pretty little spinner on your terminal tab when they are issued. On Linux terminals the sequence is so far mostly ignored, but maybe it's time that changes? I think it's a truly useful feature because it can communicate progress information about slow operations to the user…

I think the pair of PID and pidfd inode number would be great to support in the various tools that currently deal with PIDs. For example, I filed an RFE bug against util-linux' kill tool to add just that:

https://github.com/util-linux/util-linux/issues/3252

… when we pass around information about processes via IPC we have started to do so via the triplet pid, pid inode, boot id.

And I'd recommend everyone dealing with low-level process management to do the same.

Two caveats though: the concept is not universal: it's a Linux thing, and it requires kernel 6.9 or newer and a 64bit architecture. On 32bit the inode number range is too small to provide unique IDs.

To properly check if the feature is available allocate a pidfd, and check if statfs() reports a .f_type field of it being 0x50494446. Also verify if sizeof(ino_t) is >= 8.

It took a long time, but thanks to @brauner after all those years the limitations of UNIX pid_t are addressed! Thanks, Christian!

@erincandescent cgroupfs actually exposes the cgroupid via ntha() and obha(). So yes, there's prior art for doing the same in pidfs. But it's a bit weird, because unlike cgroupfs pidfs is not an fs you can mount, hence you don't really have anything to invoke obha() on. You'd probably have to get a pidfd on your own pid first, before you can use it to use obha() to get to the pidfd you actually want to get to.

If you ask me, it's a fundamental requirement for any modern Linux-based OS to provide boot time integrity and as baseline provide unattended disk encryption bound to it. To make this happen, we added two essential TPM policy concepts to systemd-cryptenroll/systemd-cryptsetup:

1. Signed TPM PCR policies allow locking a disk to a public signing key of an OS vendor, ensuring that disks can only be unlocked if an OS signed by said vendor is booted.

2️⃣ Here's the 2nd installment of posts highlighting key new features of the upcoming v257 release of systemd.

In the past year and a bit I spent a lot of time on boot integrity (i.e. boot-time TPM measurements and policies built on top of them) of Linux, covering the boot from the boot loader (systemd-boot), over the UKI EFI stub (systemd-stub) through the initrd into early regular userspace, and then locking disk encryption to it.

There's a feature added to Linux 6.9 that I think people should become more aware of: there's finally an identifier for processes that doesn't wrap around as easily as UNIX pid_t PIDs do: the pidfd file descriptors have been moved onto their own proper file system (pidfs), which enabled at the same time unique inode numbers for them.

If you want a world-wide unique identifier for a process it makes sense to combine the pair of pid_t and pidfd inode number with the system's boot ID (i.e. /proc/sys/kernel/random/boot_id). This triplet is awesome, because for the first time we can uniquely identify a Linux process, globally in this universe.

In systemd we are making use of this heavily now: internally we always store a triplet of pid, pidfd, pidfd inode for referencing processes we manage and…