Embed Notice
HTML Code
Corresponding Notice
- Embed this noticefrom what i recall most of freearc's big chungus savings is because of the srep filter. it had a custom lzw called tornado but the 'next' version is just using zstd.
the srep filter is just like.. does a very long range block deduplication or something.
dwarfs does a different odd thing where it turns files in to blocks and then it does another thing where it sorts blocks by hamming distance or something. basically runs everything through similarity hashes and stores things clustered that way so the zstd pass can better exploit it.