@sysrq @MK2boogaloo @Zerglingman @iska Yeah. pipe(), read(), write(). Syscall overhead is real, but it's going to be negligible on any serious task, and on a non-serious task, it doesn't matter.

One way to back up a database live is `ssh -C $host pg_dump $dbname | pbzip2 > backup.sql.bz2`. So ssh uses deflate, which gets the data kind of bulky but fast compression (it's gzip/deflate) so the dump gets across the network quicker without loading the CPUs on the DB server, then locally you're running parallel bzip2 to do the actual compression before writing it to disk. That's something you can't do without a pipe: the remote system *can't* share a memory space with the local machine. A TCP connection is an ordered bytestream so this kind of thing is trivial. Trying to enforce some kind of typed stream puts more overhead on both sides and makes the general compression impossible: everything is bytestreams or has to be reduced to bytestreams for general tools like that to work, unless you can get the entire earth to agree on a wire format for objects.