✧✦Catherine✦✧
i think tarsnap has played a big part in why i take the absolute minimum amount of backups
every operation is so profoundly miserable.
want to list backups you have? wait for half a hour, get an unsorted list of names.
want to clear out old ones? you gotta write a shell script that operates on that list of backups. deleting an old deduplicated snapshot takes *forever*.
want to do a restore for a test? wait for an eternity for it to build up a "cache" whatever that means
✧✦Catherine✦✧
i think i need a backup solution that doesn't heavily disincentivize me from actually doing backups
✧✦Catherine✦✧
i'm looking at restic
it can use s3 as a backend. that's cool. let's say i don't want a compromised machine to be able to delete backups, so i will follow the principle of least privilege to set up access for it
does restic documentation give me the set of permissions it requires? absolutely the fuck not
why can people not make good backup software.
James Just James
@whitequark fwiw in the odd chance you're soliciting suggestions, I've been very happy with rsnapshot for a long time, so feel free to ask if you have any questions.
✧✦Catherine✦✧
@aismallard tarsnap is like an order of magnitude more expensive than the underlying S3 costs, it's ridiculous
aismallard
@whitequark also quite expensive. costs for storage and bandwidth, and it uses AWS so you’re getting Amazon bandwidth costs…
✧✦Catherine✦✧
restic doesn't work unless you give it s3:DeleteObject permission on at least a subset of the backup bucket because it needs to delete locks
obviously the example policy just gives the backup account a blanket permission to delete anything
✧✦Catherine✦✧
okay, it looks like this is solved by enabling bucket versioning rather than reducing permissions
from my testing, restic appears to be fit for purpose (is able to execute basic commands like "list versions" without me waiting for it to do gods know what for several minutes)
✧✦Catherine✦✧
@slyecho the machine either has the authorization to delete data or it doesn't. it doesn't matter one bit what software it's running if it's compromised
Henri
@whitequark I think the appendix-only mode works best in their own server and they suggest to use rclone as an adapter
✧✦Catherine✦✧
@slyecho (and, to state the obvious, i don't want to run even more infrastructure just to handle backups)
Tristam
@whitequark may want to put a lifecycle policy on the bucket to delete stale versions after 3 months, or such. Otherwise actual deletions won’t ever reclaim storage space
✧✦Catherine✦✧
@ppxl rustic?
re: speed, tarsnap is so horrifically slow that just about anything would be an improvement
Pxl Phile
@whitequark we are using restic for automated backups at work. I like it but it has some drawbacks, namely the speed.
IIRC there was a similar tool with better performance but I need to look it up.
I also suspect that the deduplication only works well if the cache is being held at a leash.
✧✦Catherine✦✧
@swiftcoder i looked at the syntax for that and decided i want to claw my eyes out instead
Tristam
@whitequark yeah, it’s definitely easier to setup from the web console than via the command line
✧✦Catherine✦✧
@swiftcoder i was looking at the web console
JP
@whitequark @swiftcoder ye, aws makes it hard and annoying because it’s better for their bottom line
JP
@whitequark also add a regular `restic forget` (and occasional gc); there’s also a parameter for time-tiering retention. can pull example for you in a bit if you want?
✧✦Catherine✦✧
@froztbyte i probably do and i have the syntax for it, but i'm going to do this after i make sure everything's working correctly. also so long as my backups stay under $5 monthly (seems likely with restic) i don't even care that much
tarsnap doesn't even *have* a first-party time-tiering retention option. can't believe i spent so much of my life tolerating that
✧✦Catherine✦✧
@dlitz thanks. i think i'll be fine because i need to backup about 3 GB of data after 2 years of operation
Darsey Litzenberger
@whitequark I use restic. The main issue I find with it is that it's a bit of a RAM hog if you try to run it inside a memory-constrained virtual machine. I also recommend setting RESTIC_PACK_SIZE=128 (the maximum) if you have multi-terabyte backups, or it will create a _lot_ of little 16 MiB files.
JP
@whitequark ye, good call on the hold before impl. the `restic forget` is a tombstoning, so not entirely irreversible, but still 100% get that
the other handy feature is tags (and how operations can group-apply on tags)
and re pricing: if you want an easy way to cut off even more, backblaze. I moved a couple tb off aws a few months ago because it finally got too much, and backblaze so far has worked without once making me notice it
✧✦Catherine✦✧
thanks to everyone who responded! i have backups set up i think
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.