So PDF files use deflate encoding to compress some parts of the document. This is the same compression algorithm used by... a lot of things.
Many such things have repackers that will compress better basically by brute forcing a more optimal encoding.
Couldn't find one for PDF. There's a tool called minuimus that's supposed to be able to do this, but it doesn't work.
I decided to just do dirty things with LD_PRELOAD instead.
I can just inject a different deflate implementation into an existing PDF repacking tool that uses zlib for compression, e.g. qpdf.
It works.
On some mostly text PDFs I've tried this on, it cuts the file size down 4-6% losslessly.