Conversation

Notices

1. Embed this notice
   Asta [AMP] (aud@fire.asta.lgbt)'s status on Sunday, 29-Dec-2024 07:26:40 JST Asta [AMP]

   question for tech-y storage people: I just nabbed 4 extra (used) disks with a 6 TB capacity from a place that re-sells electronics (oregonrecycles.com); I think they did some testing but obviously I dunno what to extent, and also, if it was just limited to stuff like SMART data, well...
   
   anyway, I wanna RAID it, which is fine, but I don't know how much lifetime these have left on them. For four disks with unknown usage, should I use RAID6 or RAID10? It's not job critical data I'll be storing on these (mostly media and such). They'll eventually be migrated into a larger RAID array but that won't happen until I'm stable and can afford to rebuild my server so this is fine for now.
   
   I wouldn't mind the better read/write performance that comes with RAID10 even though it has less parity. I suspect these disks were all used together so they might have similar wear/tear patterns; in that case, I'm wondering if RAID6's double parity actually buys me any extra life? Like, given 4 disks with the same history and a probably known disk failure rate, I'm not really clear as to whether double parity is going to make much of a difference (and that if one goes down, the others probably aren't too far behind).
   
   #techPosting #raid #raid6 #raid10 #storage #nas

   • Embed this notice
     Asta [AMP] (aud@fire.asta.lgbt)'s status on Sunday, 29-Dec-2024 07:26:36 JST Asta [AMP]

     in reply to

     • cthos 🐱

     @cthos@mastodon.cthos.dev GOD, also, finding a solution to non-live backup is fucking difficult. Optical media is ... not a crapshoot, but finding an external burner that produces verifiable burns and that a company is still producing...
     
     What's the expected longevity of spinning rust, I wonder? That IS our what lab used to do: we'd just swap out HDs and it worked. I wonder if that's a better idea given the cost and unknown factor of getting back into optical.

   • Embed this notice
     Cassandra Granade 🏳️‍⚧️ (xgranade@wandering.shop)'s status on Sunday, 29-Dec-2024 07:26:36 JST Cassandra Granade 🏳️‍⚧️

     in reply to

     • cthos 🐱

     @aud @cthos Yet another day that I wish M-DISC was still a thing.

     https://en.wikipedia.org/wiki/M-DISC

   • Embed this notice
     cthos 🐱 (cthos@mastodon.cthos.dev)'s status on Sunday, 29-Dec-2024 07:26:37 JST cthos 🐱

     in reply to

     @aud You can add more drives to the RAID 6 and get better storage efficiency, though.

   • Embed this notice
     Asta [AMP] (aud@fire.asta.lgbt)'s status on Sunday, 29-Dec-2024 07:26:37 JST Asta [AMP]

     in reply to

     • cthos 🐱

     @cthos@mastodon.cthos.dev ooooh, interesting.
     
     If they turn out to be decent drives, they'll migrate to the eventually-rebuilt threadripper, which has far more capacity than the current computer I'm on in terms of server-like features. There's a 4 disk NAS that technically exists there and these would be added to it in one way or another.
     
     Having said that, while that's the long term plan, we all know how those long term plans can go... so yeah, it sounds like unless I literally can't use these drives for what I want, RAID6 is the way.

   • Embed this notice
     cthos 🐱 (cthos@mastodon.cthos.dev)'s status on Sunday, 29-Dec-2024 07:26:38 JST cthos 🐱

     in reply to

     @aud The space should be identical, RAID 10 would also be 12 GB since it's 2 mirrored drives.

   • Embed this notice
     Asta [AMP] (aud@fire.asta.lgbt)'s status on Sunday, 29-Dec-2024 07:26:38 JST Asta [AMP]

     in reply to

     • cthos 🐱

     @cthos@mastodon.cthos.dev huh. guess that raid calculator sucks then lmao. That's what I thought based on the description but then just assumed the calculator was telling the truth.

   • Embed this notice
     Asta [AMP] (aud@fire.asta.lgbt)'s status on Sunday, 29-Dec-2024 07:26:39 JST Asta [AMP]

     in reply to

     • cthos 🐱

     @cthos@mastodon.cthos.dev Off-site backups aren't in the cards (for now). Once I have more stability and knowing whether I'll be staying or going (kinda depends on how this job search sorts out) I'll be looking into that one.
     
     For now, in theory, nothing I'll be putting on the drive isn't re-obtainable in some form or another. However, part of what motivated this is the corporate culling of queer content; part of what I'm trying to do is keep gay shit handy. I'm considering looking at (eventually) having some sort of physical backup (not sure what the story is, price and capacity wise, on writable physical media).
     
     I guess the question is mainly aimed at: if one drive dies suddenly, am I already at risk for the other drives following shortly anyway? Depending on how long the rebuild takes...
     
     
     eventually I'd just like it to be a hot storage space for stuff I want and to shuffle older stuff off and just rely on the external backup. Buuuuuuut baby steps.

   • Embed this notice
     cthos 🐱 (cthos@mastodon.cthos.dev)'s status on Sunday, 29-Dec-2024 07:26:39 JST cthos 🐱

     in reply to

     @aud So rebuild on RAID 10 is faster than RAID 6 since it just has to flat out copy from one of the mirror pairs, whereas RAID 6 has to rebuild the whole array.

     With 10 you run the risk of two drives failing in the *same mirror* which is unrecoverable. RAID 6 can survive any 2 drives going poof. So if you're using sketchy drives that *might* be worth the added safety without offsite backups if you can tolerate the perf penalty.

   • Embed this notice
     Asta [AMP] (aud@fire.asta.lgbt)'s status on Sunday, 29-Dec-2024 07:26:39 JST Asta [AMP]

     in reply to

     • cthos 🐱

     @cthos@mastodon.cthos.dev (unrelated to your immediate comment: I might look into using M-DISC blurays for photos and other critical data since they seem to be relatively inexpensive and the 100 GB capacity is reasonable for photography RAWs.)
     
     I suppose I should look up the read/write rate of these drives to see whether my use cases would benefit from the higher read/write perf, now that you bring this up. If there's no real difference for what I'm doing, the RAID6 makes more sense since I'm fine with only 12 TB of usable space. If I definitely need the performance, though, then it might be RAID10 is the only way to go. That, and look into the optical media backup since that's cheaper than the parts I need for the server rebuild.

   • Embed this notice
     cthos 🐱 (cthos@mastodon.cthos.dev)'s status on Sunday, 29-Dec-2024 07:26:40 JST cthos 🐱

     in reply to

     @aud Do you care more about read/write speed or being able to recover if 2 drives die right next to each other?

     Are you planning on also doing off-site backups?

   • Embed this notice
     Cassandra Granade 🏳️‍⚧️ (xgranade@wandering.shop)'s status on Sunday, 29-Dec-2024 07:27:25 JST Cassandra Granade 🏳️‍⚧️

     in reply to

     • cthos 🐱

     @aud @cthos (It's technically still a thing, but hard to find burners and media for at reasonable prices now.)

   • Embed this notice
     Cassandra Granade 🏳️‍⚧️ (xgranade@wandering.shop)'s status on Sunday, 29-Dec-2024 11:36:14 JST Cassandra Granade 🏳️‍⚧️

     in reply to

     • cthos 🐱

     @cthos @aud OK, so the quantum thing. In some sense, you have it exactly right — the subtlety comes into (a) when we expect quantum computers that can break meaningful RSA keysizes to be developed, (b) when can we expect post-quantum cryptography systems to be successfully deployed, and (c) are there any improvements in quantum cryptanalysis algorithms that may impact post-quantum cryptography.

     Alternatively, do we live in a world in which quantum-resistant cryptography is possible?

   • Embed this notice
     Asta [AMP] (aud@fire.asta.lgbt)'s status on Sunday, 29-Dec-2024 11:36:15 JST Asta [AMP]

     in reply to

     • Cassandra Granade 🏳️‍⚧️
     • cthos 🐱

     @cthos@mastodon.cthos.dev @xgranade@wandering.shop fucking entropy yet again.
     
     Interestingly, while searching for all this information, I have seen a lot of "use the cloud!" and I'm like hmmmmm. When it comes to avoiding censorship, I trust the cloud not at all. I mean, why would I? Half these companies are the same ones providing active surveillance services and/or storage for said entities.

   • Embed this notice
     cthos 🐱 (cthos@mastodon.cthos.dev)'s status on Sunday, 29-Dec-2024 11:36:15 JST cthos 🐱

     in reply to

     • Cassandra Granade 🏳️‍⚧️

     @aud @xgranade I mean encrypted backups to a cloud provider where only you have the decryption key is pretty solid until Quantum computing is a thing (I hear, please feel to eviscerate me here @xgranade)

   • Embed this notice
     Asta [AMP] (aud@fire.asta.lgbt)'s status on Sunday, 29-Dec-2024 11:36:16 JST Asta [AMP]

     in reply to

     • Cassandra Granade 🏳️‍⚧️
     • cthos 🐱

     @xgranade@wandering.shop @cthos@mastodon.cthos.dev So, maybe just grabbing spare disks cheaply like this and then using them with a filesystem with parity checking or maybe just parchive or whatever is the way to go. Get em cheap enough, then I can do double backups.
     
     It basically seems like our current system is no better than the European monks who were forced to make copy after copy of books when the old ones degraded. We're still doing that, just digitally. Well, entropy is a thing, I suppose, but I guess I'm a little surprised there's not more easily accessible verifiable informa-... wait, why am I surprised? Of course that data isn't available or super public, no one wants to sell you a "set it and forget it for X time" solution. They want you to buy moar.

   • Embed this notice
     cthos 🐱 (cthos@mastodon.cthos.dev)'s status on Sunday, 29-Dec-2024 11:36:16 JST cthos 🐱

     in reply to

     • Cassandra Granade 🏳️‍⚧️

     @aud @xgranade Hahah yes, entropy gets all things. Even optical media will degrade after a century or so even in good conditions.

     The only thing permanent is impermanence?

   • Embed this notice
     Asta [AMP] (aud@fire.asta.lgbt)'s status on Sunday, 29-Dec-2024 11:36:17 JST Asta [AMP]

     in reply to

     • Cassandra Granade 🏳️‍⚧️
     • cthos 🐱

     @xgranade@wandering.shop @cthos@mastodon.cthos.dev yeah, and it seems super super unclear as to what Verbatim is selling as m-discs and whether they're truly the thing or not. boooo.

   • Embed this notice
     Cassandra Granade 🏳️‍⚧️ (xgranade@wandering.shop)'s status on Sunday, 29-Dec-2024 11:38:34 JST Cassandra Granade 🏳️‍⚧️

     in reply to

     • cthos 🐱

     @cthos @aud I would argue strongly that we do, that the timelines for practical RSA breaks and post-quantum crypto line up pretty well, and so forth. Considering forward secrecy for backups does complicate things substantially, though... then the question becomes whether or not *current* post-quantum cryptography will remain quantum-resistant for the lifetime of the backup. I'd again argue yes, but that's more subtle.

   • Embed this notice
     Cassandra Granade 🏳️‍⚧️ (xgranade@wandering.shop)'s status on Sunday, 29-Dec-2024 11:39:49 JST Cassandra Granade 🏳️‍⚧️

     in reply to

     • cthos 🐱

     @cthos @aud The last thing I'll note is that the waters here have been muddied rather badly, and by companies with a vested interest in selling specific post-quantum schemes and quantum key distribution products (the latter would be a *much* longer thread).

   • Embed this notice
     Cassandra Granade 🏳️‍⚧️ (xgranade@wandering.shop)'s status on Sunday, 29-Dec-2024 11:42:13 JST Cassandra Granade 🏳️‍⚧️

     in reply to

     • cthos 🐱

     @cthos @aud I don't think it's an unreasonable position to take, really. But yeah, some of the prose that companies tend to write about that position is... a lot.

   • Embed this notice
     cthos 🐱 (cthos@mastodon.cthos.dev)'s status on Sunday, 29-Dec-2024 11:42:14 JST cthos 🐱

     in reply to

     • Cassandra Granade 🏳️‍⚧️

     @xgranade @aud Tuta certainly thinks so. 😆 (I like their email service but the claims of quantum resistance are…..)

   • Embed this notice
     Cassandra Granade 🏳️‍⚧️ (xgranade@wandering.shop)'s status on Sunday, 29-Dec-2024 11:48:25 JST Cassandra Granade 🏳️‍⚧️

     in reply to

     • cthos 🐱

     @cthos @aud One addendum I should have added: even were a quantum computer capable of breaking practical keysizes of a candidate post-quantum cryptosystem within the forward secrecy time of a backup, it's likely that actually performing said break would be exorbitantly expensive for quite a while. If your stuff is specifically interesting to a degree that three-letters are willing to hold it encrypted for twenty years then spend millions to get into it, quantum is the least of your worries.

   • Embed this notice
     cthos 🐱 (cthos@mastodon.cthos.dev)'s status on Sunday, 29-Dec-2024 11:50:23 JST cthos 🐱

     in reply to

     • Cassandra Granade 🏳️‍⚧️

     @xgranade @aud it would be flattering to be that important. 🤔

   • Embed this notice
     Cassandra Granade 🏳️‍⚧️ (xgranade@wandering.shop)'s status on Sunday, 29-Dec-2024 11:50:23 JST Cassandra Granade 🏳️‍⚧️

     in reply to

     • cthos 🐱

     @cthos @aud Indeed, though I would personally be OK without such flattery.

   • Embed this notice
     Cassandra Granade 🏳️‍⚧️ (xgranade@wandering.shop)'s status on Sunday, 29-Dec-2024 11:52:57 JST Cassandra Granade 🏳️‍⚧️

     in reply to

     • cthos 🐱
     • trystimuli

     @tryst @aud @cthos I understood the thread as asymmetric, but that's a good point. If you're talking symmetric, the room for *potential* quantum cryptanalysis narrows to almost nothing.

   • Embed this notice
     trystimuli (tryst@fedi.imu.li)'s status on Sunday, 29-Dec-2024 11:52:58 JST trystimuli

     in reply to

     • Cassandra Granade 🏳️‍⚧️
     • cthos 🐱

     @xgranade @cthos @aud except this is symmetric encryption we’re talking about, right? (c) is still relevant, sure, but the answer to (b) is both 256-bit AES and ChaCha20 are everywhere already.

   • Embed this notice
     Cassandra Granade 🏳️‍⚧️ (xgranade@wandering.shop)'s status on Sunday, 29-Dec-2024 11:57:01 JST Cassandra Granade 🏳️‍⚧️

     in reply to

     • cthos 🐱
     • trystimuli

     @cthos @tryst @aud In which case, there's little evidence of any better than the trivial square root speedup you get for unstructured search... double your keylengths, and quantum cryptanalysis just isn't a big deal.

   • Embed this notice
     cthos 🐱 (cthos@mastodon.cthos.dev)'s status on Sunday, 29-Dec-2024 11:57:02 JST cthos 🐱

     in reply to

     • Cassandra Granade 🏳️‍⚧️
     • trystimuli

     @xgranade @tryst @aud I assumed asymmetric in the hypothetical but thinking on it most NAS backup software does symmetric keys IIRC