Embed Notice
HTML Code
Corresponding Notice
- Embed this notice@EdBoatConnoisseur It's all media uploads so for the most part incompressible data.
I suspect a lot of this savings comes from partial blocks which LZ4 is more than good enough.
For database backups I plan to use zstd. For the database itself I plan to use LZ4.
LZ4 should give me a 2-3 compression ratio with negligible CPU overhead and negligible latency. Zstd could give me 3+ but at the expense of CPU and more than double the latency.
I'll use zstd for the rest of the system tho.
I plan to use a block size of 16k for postgres (as I'm too lazy to deal with compiling my own version to use larger block sizes plus that would also cause a lot of downtime as I rebuild the databases).
The only question is what block size to use for the rest of the system. I figure the default 128k should be good. I'd like to use 1M for /var/log/ to improve compression ratios but I'm pretty sure I'd take a decent IO performance hit by doing that since as far as I know they're written one line at a time.