Notices by anna (navi@social.vlhl.dev)

i think i misunderstood soname for a while now

they’re supposed to encode abi, not api, and not supposed to match your package version at all, that’s api

so there’s 3 cases:

1: whenever your abi changes, you bump the soname. abi-only changes of a library are usually exclusive to when the members are added to a structure, or type sizes are changed (at least i can’t think of any other “usual” change that would change abi without changing api) – abi-only changes means code using your library only needs to be recompiled, nothing else.

2: when your api changes, so function names/parameter changes, struct member changes, you bump both the package major version, and the soname, since an api changes means you need to edit code and recompile

3: when you add things to your api and abi, but don’t change anything that already exists, the library is still backwards compatible, soname doesn’t change, you bump only the minor semver and minor soname

so for a given libnya.so, from package dev-utils/nya-1.90.2 there could be:

libnya.so.1, soname, this is what elf binaries write as DT_NEEDED, this indicates libnya‘s abi constraints, aka your “abi major version” libnya.so, this is what the compiler looks for when you do -lnya, meant to be “the latest ABI” libnya.so.1.34, this is the full qualified SONAME, so abi-major.abi-minor

and they would be symlinked as such: libnya.so -> libnya.so.1 -> libnya.so.1.34

say you add a new function to libnya, not touching any current abi, then all you do is replace libnya.so.1.34 with libnya.so.1.35

if you break abi, even if you don’t touch the api, you would then install libnya.so.2.0, and update the symlink libnya.so -> libnya.so.2 -> libnya.so.2.0 – and yes, this means that the package nya-1.91 would install libnya.so.2

note that currently installed applications are still using libnya.so.1, even if you install libnya.so.2.0 and move the libnya.so symlink! which means that if you update the soname to 2.0, you can keep libnya.so.1 installed (without a new generic symlink) concurrently, and old binaries will still work (looking at you arch/pacman, who removes old abi libraries while there’s still programs using it installed, “partial upgrades are not supported” is not an excuse to suck like this) – only remove that old soname when there’s no more packages using it

that covers cases 1 and 3 quite well, though it’s not perfect for case 3, more on that later

case 2 is the issue on this system. your api changed, meaning your abi changed too. you could think of this as the same as case 1 and just bump the soname, aka abi-major, when releasing nya-2.0, but that’s not ideal, because what if the user has two programs, one that uses nya-1 and another that uses nya-2 – if you only bump soname, you won’t be able to build both (without shenanigans) since -lnya would always resolve to libnya.so.3 aka nya-2‘s shared object

there is, however, a solution to this, it’s just not pretty, and imo, a huge hack on what so far seemed like a clean system, insert your API constrains in the shared object name, then reset soname, so nya-2 would ship libnya2.so -> libnya2.so.1 -> libnya2.so.1.0

this separates the api changes from the abi, and allows libnya2.so and libnya.so to exist at the same time

so in the end the library naming scheme gets to lib[name][api-major].so.[abi-major].[abi-minor]

i don’t like putting the api major in the library name at all, it’s ugly, it incentives people to not do major bumps (who wants to ship a file named libnya13.so.1? where the users have to -lnya13 to use it)

i wish the api was actually part of the soname mechanism, in a better way – however, this is how it is, i can’t think of anything better, at least not currently, with our current toolchains.

a solution, one often used by distros, is to install to different prefixes, and do -L/usr/lib/libnya-1 or -L/usr/lib/libnya-2, and each would have their own libnya.so -> libnya.so.{2,3} symlink respectively – works, it’s less ugly than putting the api on the name, but still feels hacky to me, split out /usr/lib into multiple small subdirs just for versioned parallel installs, and require lengthy compiler invocations with full search paths for every single library, overall ugly imo

though this one works way nicer when the distro already installs packages in isolation, something like nix or slashpackage, since you can have the default symlink in /usr/lib be the latest version, and only provide -L for older version. on more traditional distros doing this would mean putting every single library in a split subdir even if it’ll never have parallel installs ever, which i would strongly dislike to do

another one solution, which would be my prefered since it’s more semantic and systematic, would be to able to specify the soname in the cc invocation, -lnya.2 would use libnya.so.2 directly, even tho libnya.so -> libnya.so.3 exists. but currently this doesn’t quite work. gcc/clang support -l:filepath, (though not posix cc/c17), but pkg-config and other tooling don’t really work with that, and since it’s a fully qualified pathname, it would basically break --static flags in some build tools

lastly, for case 3, soname makes shared objects abi-backwards-compatible, and gives mechanism for detecting when rebuilds are necessary (soname bump, .so.2 to .so.3), but they have no way to specify a minimum minor version, so a library that uses a symbol only present in libnya.so.1.39, can’t in any way specify that libnya.so.1.38 will not work, but libnya.so.1.40 will

or you could be like glibc and freeze your soname, and whenever you need to make a change make a new symbol and keep carrying all your old source code versions for all eternity :)

did i get anything wrong, or miss anything?

one key takeway i had from this soname / abi != package-version / api is that adding fields to structures shouldn’t be a thing to worry about that much, removing them yes, adding, reordering, etc, no

libraries shouldn’t keep their data “obscured” behind a malloc-d pointer just for the sake of binary compatibility. there is other reasons for opaque pointers, but more often than not, a library will be better by letting users choose where to allocate data, be it on the stack or on the heap or statically or so on

and if you worry about people accessing fields they shouldn’t, c23 provides [[deprecated]], meaning you can do:

and mark struct members as such

distros should have mechanism to easily trigger rebuilds on soname changes, as well as mechanisms to keep the old soname around as long as there’s packages linked to it

the fact that some distros don’t, is not a limitation of the system, but an implementation failure

@ska @lanodan i don’t really use errno for it (aside from openrc, where like 90% of the errors are errno set by libc, and 10% we set)

i was thinking of something like a lib doing thread_local enum libfoo_status status;, and if say a function returns NULL or false, then status would have a lib-specific more detailed code, or even thread_local struct libfoo_status status; with even more info, like maybe failure location in a parser

most new code i usually see does that with what you called the pthreads or something similar, return that enum or struct by either the return value or a pointer argument, e.g.

enum libfoo_status func(struct foo *out, int somearg);

or

struct foo *func(int somearg, enum libfoo_status *status); (this one usually allows NULL status to “ignore” it)

so my brain was just wondering if there’s anything technically wrong with doing the thread_local thing, or if thread_local vars just suck in general and thus everyone prefers to return the status locally one way or another