Conversation

Notices

1. Embed this notice
   Filippo Valsorda :go: (filippo@abyssdomain.expert)'s status on Tuesday, 22-Apr-2025 21:03:50 JST Filippo Valsorda :go:

   I am writing an application that really cares about durability of created files (a Certificate Transparency log), and... oof.

   I fsync the file. I fsync the directory. Ok.

   But... how do I test it? Even targeting a specific filesystem, I have to make VMs and try to race killing them?

   • Rich Felker repeated this.
   • Embed this notice
     Fazal Majid (fazalmajid@social.vivaldi.net)'s status on Tuesday, 22-Apr-2025 21:03:49 JST Fazal Majid

     in reply to

     @filippo don't forget that drives lie about data being committed to disk to goose up benchmarks, specially SSDs. They will ack the disk flush when it's only in the on-drive RAM buffer, and most non-enterprise drives lack a power-loss protection supercap. To properly test for this, you'd need a rig with some software-controlled-hardware way to disconnect the drive from power when you receive the fsync/fdatasync ack.

     The PostgreSQL community has done a fair bit of investigation on the subject.

   • Embed this notice
     Ben Zanin (gnomon@mastodon.social)'s status on Tuesday, 22-Apr-2025 21:43:59 JST Ben Zanin

     in reply to

     • Rich Felker
     • Fazal Majid
     • kasperd

     @kasperd @filippo @fazalmajid @dalias back in 2022 there was a short Twitter thread from Ross Bishop¹ about testing four SSDs that lie about fsync: only two actually kept that promise².

     For durability, the WD Gold SSDs do include supercapacitors of capacity sufficient to fully flush write buffers to the flash layer, and recently introduced this to their mechanical drives too.

     ¹: https://web.archive.org/web/20220222012227/https://twitter.com/xenadu02/status/1495693475584557056

     ²:
     1. Samsung 970 Evo Plus: MZ-V7S2T0, 2021.10
     2. WD Red: WDS100T1R0C-68BDK0, 04Sept2021

   • Embed this notice
     kasperd (kasperd@westergaard.social)'s status on Tuesday, 22-Apr-2025 21:44:01 JST kasperd

     in reply to

     • Fazal Majid

     One BSD developer at some point mentioned that he rigged some hardware that would not just cut the power but also short the power input of the drive to very quickly drain those capacitors.

     He really wanted to know if drives were lying about data having been committed to disk and using capacitors to hide those lies.

     Rich Felker repeated this.