Skip Navigation
Data Hoarder @selfhosted.forum FizzicalLayer @alien.top
BOT
Remote Data Hoards

A recent post by a person headed to Antarctica has me wondering...

What's the most remote / disconnected data hoard you're aware of? I'm picturing a cabin in the mountains with terabytes of data on a solar powered server, that kind of thing. Starlink has given us connectivity pretty much planet-wide, but there's no substitute for a massive server with your own data on a local network.

1
MiscroSD as a cold storage/backup drive
  • For the price of two 1Tb MicroSD cards, you could get a 4Tb SSD and enclosure. That's more than enough for your data and enough parity info to repair whatever is damaged. It's what I do. The SSD is reliable if powered on often enough for long enough to allow it to do its refreshing. Still, have a backup somewhere (cloud, disk, etc), but the SSD is fine, imho, and vastly superior to microsd.

  • how do you know if a backup is dead (even if you have multiple copies)
  • I realize there are solutions, but I wanted my own for various reasons (better fit to the peculiar way I store and backup).

    It was straightforward to write a python script to crawl a directory tree, adding files to an sqlite database. The script has a few commands:

    - "check" computes checksums on files whose modification times have changed since last check, or on any file whose checksum is older than X days (find bitrot this way).

    - "parity" Use par2 to compute parity files for all files in database. Store these in a ".par2" directory in the directory tree root so it doesn't clutter the directory tree.

    I like this because I can compute checksums and parity files per directory tree (movies, music, photos, etc), and by disk (no raid here, just JBOD + mergerfs). Each disk corresponds exactly to a backup set kept in a pelican case.

    The sqlite database has the nice side effect that checksum / parity computation can run in the background and be interrupted at any time (it takes a loooooooong time). The commits are atomic, so machines crashes or have to shut down, it's easy to resume from previous point.

    Surely.... SURELY... someone has already written this. But it took me a couple of afternoons to roll my own. Now I have parity and the ability detect bitrot on all live disks and backup sets.

  • Has anybody here thrown out their physical media?
  • Never throw away physical media. If space is a problem, remove disks from cases and store on spindles (the packages that CD-R blanks come in are ideal).

    Most people run a compression pass on media rips (handbrake) to make storage feasible with today's disks and budgets. The day is rapidly approaching when hard disks will be large enough and cheap enough to store bit exact copies of your media. You'll want to rerip then, and having the media will make that possible.

    Physical media serves as long term stable backup. It should be part of your backup plan, just like multiple physical backup disks sets, offsite storage, cloud storage, etc.

    If space is an issue, there are easy solutions. Disks do not have to be in cases, and they're too useful to part with.

  • InitialsDiceBearhttps://github.com/dicebear/dicebearhttps://creativecommons.org/publicdomain/zero/1.0/„Initials” (https://github.com/dicebear/dicebear) by „DiceBear”, licensed under „CC0 1.0” (https://creativecommons.org/publicdomain/zero/1.0/)FI
    FizzicalLayer @alien.top
    BOT
    Posts 1
    Comments 3