@wolf480pl @RandomDamage hardware RAID5/6 has the write hole issue and also doesn't do any data verification with checksums which allows it to go much faster, yes, and it can go sequentially.

There is sequential resilver for ZFS now but you still have to do a scrub afterwards to confirm data integrity (automatic, but you can cancel it). In my tests it goes much faster, closer to what it was like with hardware RAID, and then at the end you can do a lower priority scrub which won't affect IOPS as much as the resilver does.

Either way, I don't think of mirrors as "wasted space". It's your insurance policy. Plus it makes your storage really fast. With RAIDZ your storage is only as fast as the slowest disk in each VDEV. Hardware RAID5/6 is about the same.

Being able to migrate data to/from my NVMEs at 2000MB/s+ is really nice. Plus that gives me plenty of overhead so when I run backups no workloads are impacted. If I used RAIDZ/5/6 I would be limited to about 600MB/s last I calculated...