netsapiensis

rocky-borg-backup

0
0
# Install this skill:
npx skills add netsapiensis/claude-code-skills --skill "rocky-borg-backup"

Install specific skill from multi-skill repository

# Description

Borg Backup administration on Rocky Linux 8/9 including installation from EPEL, repository initialization with encryption, backup creation with compression and exclusions, full and partial restore, mount for browsing, pruning with dry-run-first workflow, borg compact, and systemd timer scheduling with monitoring. Use when setting up backups, restoring data, managing retention, or automating backup schedules.

# SKILL.md


name: rocky-borg-backup
description: Borg Backup administration on Rocky Linux 8/9 including installation from EPEL, repository initialization with encryption, backup creation with compression and exclusions, full and partial restore, mount for browsing, pruning with dry-run-first workflow, borg compact, and systemd timer scheduling with monitoring. Use when setting up backups, restoring data, managing retention, or automating backup schedules.


Borg Backup Administration

Installation, backup, restore, pruning, and scheduling for BorgBackup on Rocky Linux 8/9.

Prerequisite: See rocky-foundation for OS detection and safety tier definitions.

Installation

# Install from EPEL  # [CONFIRM]
dnf install -y epel-release
dnf install -y borgbackup

# Verify  # [READ-ONLY]
borg --version

Repository Initialization

Encryption Modes

Mode Description Recommended
repokey Key stored in repo, passphrase encrypts it Yes (default choice)
repokey-blake2 Same as repokey with BLAKE2b (faster) Yes (modern systems)
keyfile Key stored locally only, passphrase encrypts it More secure but key must be backed up separately
keyfile-blake2 keyfile with BLAKE2b Same as above
none No encryption Only for non-sensitive data
authenticated No encryption, but authenticated Integrity without confidentiality

Create Repository

# Initialize encrypted repository (recommended)  # [CONFIRM]
borg init --encryption=repokey /backup/borg-repo
# You will be prompted for a passphrase -- SAVE THIS SECURELY

# Or repokey-blake2 for better performance  # [CONFIRM]
borg init --encryption=repokey-blake2 /backup/borg-repo

# Remote repository via SSH  # [CONFIRM]
borg init --encryption=repokey ssh://backup-server/./borg-repo

CRITICAL: Export the Key

This is the most important step. Without the key, encrypted backups are unrecoverable.

# Export repository key  # [READ-ONLY]
borg key export /backup/borg-repo /root/borg-key-export.txt

# View key info  # [READ-ONLY]
borg key export /backup/borg-repo --paper   # Paper backup format

# Store the key export in a SEPARATE location from the backup
# Options: password manager, separate encrypted USB, printed paper in safe

WRONG -- skipping key export:

# WRONG: Initializing repo without exporting key
borg init --encryption=repokey /backup/borg-repo
# If /backup/borg-repo is lost, ALL backups are unrecoverable!

# CORRECT: Always export key immediately after init
borg init --encryption=repokey /backup/borg-repo
borg key export /backup/borg-repo /root/borg-key-export.txt
# Then store borg-key-export.txt somewhere SAFE and SEPARATE

Import Key (Recovery)

# Import key to new/rebuilt repo  # [CONFIRM]
borg key import /backup/borg-repo /root/borg-key-export.txt

Creating Backups

Basic Backup

# Create backup archive  # [CONFIRM]
borg create \
  --stats --progress \
  --compression zstd,3 \
  /backup/borg-repo::{hostname}-{now:%Y%m%d-%H%M%S} \
  /etc \
  /home \
  /var/www \
  /opt/myapp

# With exclusions  # [CONFIRM]
borg create \
  --stats --progress \
  --compression zstd,3 \
  --exclude '*.pyc' \
  --exclude '*.tmp' \
  --exclude '/home/*/.cache' \
  --exclude '/var/www/*/node_modules' \
  --exclude '/var/www/*/vendor' \
  --exclude 're:^/proc' \
  --exclude 're:^/sys' \
  --exclude 're:^/dev' \
  --exclude 're:^/run' \
  --exclude 're:^/tmp' \
  /backup/borg-repo::{hostname}-{now:%Y%m%d-%H%M%S} \
  /etc \
  /home \
  /var/www \
  /opt/myapp

Naming Conventions

# Use consistent archive naming  # [CONFIRM]
# Pattern: {hostname}-{timestamp}
borg create /backup/borg-repo::webserver01-{now:%Y%m%d-%H%M%S} /etc /var/www

# Or with prefix for different backup types
borg create /backup/borg-repo::daily-{hostname}-{now:%Y%m%d-%H%M%S} /etc /var/www
borg create /backup/borg-repo::db-{hostname}-{now:%Y%m%d-%H%M%S} /var/lib/mysql

Compression Options

Option Speed Ratio Recommended For
none Fastest 1:1 Already compressed data
lz4 Very fast Low Fast backups, plenty of space
zstd,3 Fast Good General purpose (recommended)
zstd,9 Moderate Better Archival, less frequent backups
zlib,6 Slow Good Compatibility

Remote Backup via SSH

# Backup to remote server  # [CONFIRM]
borg create \
  --stats --progress \
  --compression zstd,3 \
  ssh://backup-user@backup-server/./borg-repo::{hostname}-{now:%Y%m%d-%H%M%S} \
  /etc \
  /var/www

# Use SSH key and config for automation
# ~/.ssh/config:
# Host backup-server
#   HostName 10.0.0.200
#   User backup
#   IdentityFile ~/.ssh/id_ed25519_backup
#   ServerAliveInterval 60

Database Backup Integration

# MariaDB dump + borg (pipe approach)  # [CONFIRM]
mariadb-dump --single-transaction --routines --triggers --all-databases | \
  borg create --stats --compression zstd,3 \
  --stdin-name all-databases.sql \
  /backup/borg-repo::db-{hostname}-{now:%Y%m%d-%H%M%S} -

# Or dump to file first, then include in borg
mariadb-dump --single-transaction --all-databases > /tmp/db-dump.sql  # [CONFIRM]
borg create --stats --compression zstd,3 \
  /backup/borg-repo::db-{hostname}-{now:%Y%m%d-%H%M%S} \
  /tmp/db-dump.sql                                                     # [CONFIRM]
rm /tmp/db-dump.sql

Listing and Inspecting Backups

# List all archives  # [READ-ONLY]
borg list /backup/borg-repo

# List with details  # [READ-ONLY]
borg list --format '{archive}{NL}' /backup/borg-repo
borg list --sort-by timestamp /backup/borg-repo
borg list --last 10 /backup/borg-repo

# Show archive info  # [READ-ONLY]
borg info /backup/borg-repo::webserver01-20240115-020000

# List files in archive  # [READ-ONLY]
borg list /backup/borg-repo::webserver01-20240115-020000
borg list /backup/borg-repo::webserver01-20240115-020000 /etc/nginx/

# Repository info  # [READ-ONLY]
borg info /backup/borg-repo

# Check repository integrity  # [READ-ONLY]
borg check /backup/borg-repo
borg check --verify-data /backup/borg-repo     # Slower, verifies data integrity

Restoring Backups

Full Restore

# Extract entire archive to current directory  # [DESTRUCTIVE]
cd /tmp/restore
borg extract /backup/borg-repo::webserver01-20240115-020000

# Extract to specific directory  # [DESTRUCTIVE]
borg extract --strip-components 0 \
  /backup/borg-repo::webserver01-20240115-020000

Partial Restore

# Extract specific paths  # [CONFIRM]
borg extract /backup/borg-repo::webserver01-20240115-020000 \
  etc/nginx/nginx.conf \
  etc/nginx/conf.d/

# Extract with pattern  # [CONFIRM]
borg extract /backup/borg-repo::webserver01-20240115-020000 \
  --pattern '+ etc/nginx/**' \
  --pattern '- *'

# Extract excluding patterns  # [CONFIRM]
borg extract /backup/borg-repo::webserver01-20240115-020000 \
  --exclude '*.log' \
  etc/ var/www/

Mount for Browsing

Mount an archive as a FUSE filesystem to browse and selectively copy files:

# Mount archive  # [READ-ONLY]
mkdir -p /mnt/borg
borg mount /backup/borg-repo::webserver01-20240115-020000 /mnt/borg

# Browse and copy what you need
ls /mnt/borg/                      # [READ-ONLY]
cp /mnt/borg/etc/nginx/nginx.conf /tmp/  # [CONFIRM]

# Mount entire repo (all archives)
borg mount /backup/borg-repo /mnt/borg   # [READ-ONLY]
ls /mnt/borg/                             # Shows all archives as directories

# Unmount when done  # [CONFIRM]
borg umount /mnt/borg

Restore Database from Pipe Backup

# If backed up with --stdin-name  # [DESTRUCTIVE]
borg extract --stdout /backup/borg-repo::db-webserver01-20240115-020000 | \
  mariadb

# If backed up as file  # [DESTRUCTIVE]
cd /tmp
borg extract /backup/borg-repo::db-webserver01-20240115-020000 tmp/db-dump.sql
mariadb < /tmp/tmp/db-dump.sql

Pruning and Retention

CRITICAL: Always Dry-Run First

WRONG -- pruning without dry-run:

# WRONG: Directly pruning without preview
borg prune --keep-daily=7 --keep-weekly=4 --keep-monthly=6 /backup/borg-repo
# You might delete archives you wanted to keep!

# CORRECT: Always dry-run first
borg prune --dry-run --list --keep-daily=7 --keep-weekly=4 --keep-monthly=6 /backup/borg-repo
# Review output, then:
borg prune --list --keep-daily=7 --keep-weekly=4 --keep-monthly=6 /backup/borg-repo

Prune Workflow

# Step 1: Dry-run to see what would be deleted  # [READ-ONLY]
borg prune --dry-run --list \
  --keep-daily=7 \
  --keep-weekly=4 \
  --keep-monthly=6 \
  --keep-yearly=2 \
  /backup/borg-repo

# Step 2: Review output carefully
# Archives marked with 'x' will be deleted
# Archives marked with '+' will be kept

# Step 3: Execute prune  # [DESTRUCTIVE]
borg prune --list \
  --keep-daily=7 \
  --keep-weekly=4 \
  --keep-monthly=6 \
  --keep-yearly=2 \
  /backup/borg-repo

# Step 4: Compact to reclaim space  # [CONFIRM]
borg compact /backup/borg-repo

Prune with Prefix (Multiple Backup Types)

# Prune only daily backups  # [DESTRUCTIVE]
borg prune --list \
  --glob-archives 'daily-*' \
  --keep-daily=7 \
  --keep-weekly=4 \
  /backup/borg-repo

# Prune only database backups  # [DESTRUCTIVE]
borg prune --list \
  --glob-archives 'db-*' \
  --keep-daily=7 \
  --keep-weekly=4 \
  --keep-monthly=12 \
  /backup/borg-repo

Retention Guidelines

Policy Keeps Total Archives
Minimal daily=7 ~7
Standard daily=7, weekly=4, monthly=6 ~17
Extended daily=7, weekly=4, monthly=12, yearly=2 ~25
Paranoid daily=14, weekly=8, monthly=12, yearly=5 ~39

Scheduling with systemd

Backup Script

#!/bin/bash
# /usr/local/bin/borg-backup.sh  # [CONFIRM]
set -euo pipefail

# Configuration
export BORG_REPO="/backup/borg-repo"
export BORG_PASSPHRASE="your-passphrase-here"  # Or use BORG_PASSCOMMAND
# Better: export BORG_PASSCOMMAND="cat /root/.borg-passphrase"

BACKUP_NAME="{hostname}-{now:%Y%m%d-%H%M%S}"
LOG_FILE="/var/log/borg/backup.log"

# Logging function
log() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') $*" | tee -a "$LOG_FILE"
}

log "Starting backup: $BACKUP_NAME"

# Create backup
borg create \
    --stats \
    --compression zstd,3 \
    --exclude-caches \
    --exclude '/home/*/.cache' \
    --exclude '/var/tmp/*' \
    --exclude '/var/cache/*' \
    --exclude '*.pyc' \
    --exclude '/proc' \
    --exclude '/sys' \
    --exclude '/dev' \
    --exclude '/run' \
    --exclude '/tmp' \
    ::"$BACKUP_NAME" \
    /etc \
    /home \
    /var/www \
    /opt/myapp \
    2>&1 | tee -a "$LOG_FILE"

BACKUP_EXIT=$?

log "Backup finished with exit code: $BACKUP_EXIT"

# Prune old archives
log "Starting prune"
borg prune \
    --list \
    --keep-daily=7 \
    --keep-weekly=4 \
    --keep-monthly=6 \
    --keep-yearly=2 \
    2>&1 | tee -a "$LOG_FILE"

PRUNE_EXIT=$?

# Compact repository
log "Starting compact"
borg compact 2>&1 | tee -a "$LOG_FILE"

COMPACT_EXIT=$?

# Determine overall exit code
GLOBAL_EXIT=$(( BACKUP_EXIT > PRUNE_EXIT ? BACKUP_EXIT : PRUNE_EXIT ))
GLOBAL_EXIT=$(( GLOBAL_EXIT > COMPACT_EXIT ? GLOBAL_EXIT : COMPACT_EXIT ))

if [ $GLOBAL_EXIT -eq 0 ]; then
    log "Backup completed successfully"
elif [ $GLOBAL_EXIT -eq 1 ]; then
    log "Backup completed with warnings"
else
    log "Backup completed with errors (exit code: $GLOBAL_EXIT)"
fi

exit $GLOBAL_EXIT
# Set permissions  # [CONFIRM]
chmod 700 /usr/local/bin/borg-backup.sh
mkdir -p /var/log/borg

systemd Timer

# /etc/systemd/system/borg-backup.service  # [CONFIRM]
[Unit]
Description=Borg Backup
After=network-online.target
Wants=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/local/bin/borg-backup.sh
User=root
Nice=19
IOSchedulingClass=best-effort
IOSchedulingPriority=7

# Environment (passphrase via file)
Environment="BORG_PASSCOMMAND=cat /root/.borg-passphrase"

# Prevent concurrent runs
ExecStartPre=/usr/bin/flock -n /run/lock/borg-backup.lock echo "Lock acquired"
# /etc/systemd/system/borg-backup.timer  # [CONFIRM]
[Unit]
Description=Borg Backup Timer

[Timer]
OnCalendar=*-*-* 02:00:00
Persistent=true
RandomizedDelaySec=900

[Install]
WantedBy=timers.target
# Enable timer  # [CONFIRM]
systemctl daemon-reload
systemctl enable --now borg-backup.timer

# Verify  # [READ-ONLY]
systemctl list-timers borg*
systemctl status borg-backup.timer

Manual Trigger

# Run backup manually  # [CONFIRM]
systemctl start borg-backup.service

# Check result  # [READ-ONLY]
systemctl status borg-backup.service
journalctl -u borg-backup.service --since "1 hour ago"

Monitoring and Alerting

Check Backup Health

# Last backup age  # [READ-ONLY]
borg list --last 1 --format '{archive} {start}' /backup/borg-repo

# Repository size  # [READ-ONLY]
borg info /backup/borg-repo

# Repository integrity  # [READ-ONLY]
borg check /backup/borg-repo

Simple Monitoring Script

#!/bin/bash
# /usr/local/bin/borg-check.sh  # [CONFIRM]
# Alert if no backup in last 26 hours
set -euo pipefail

export BORG_REPO="/backup/borg-repo"
export BORG_PASSCOMMAND="cat /root/.borg-passphrase"

LAST_BACKUP=$(borg list --last 1 --format '{start}' 2>/dev/null)

if [ -z "$LAST_BACKUP" ]; then
    echo "CRITICAL: No backups found in repository"
    exit 2
fi

LAST_EPOCH=$(date -d "$LAST_BACKUP" +%s)
NOW_EPOCH=$(date +%s)
AGE_HOURS=$(( (NOW_EPOCH - LAST_EPOCH) / 3600 ))

if [ $AGE_HOURS -gt 26 ]; then
    echo "WARNING: Last backup is ${AGE_HOURS} hours old (threshold: 26h)"
    echo "Last backup: $LAST_BACKUP"
    exit 1
fi

echo "OK: Last backup ${AGE_HOURS} hours ago ($LAST_BACKUP)"
exit 0

Passphrase Management

# WRONG: Passphrase in script or environment variable in plain text
export BORG_PASSPHRASE="my-secret-passphrase"

# CORRECT: Use BORG_PASSCOMMAND with a protected file
echo "my-secret-passphrase" > /root/.borg-passphrase  # [CONFIRM]
chmod 600 /root/.borg-passphrase                       # [CONFIRM]
export BORG_PASSCOMMAND="cat /root/.borg-passphrase"

# BEST: Use a secret manager or vault
export BORG_PASSCOMMAND="vault kv get -field=passphrase secret/borg"

Borg Break Lock

If a backup process was interrupted and the lock is stale:

# Check if repo is locked  # [READ-ONLY]
borg info /backup/borg-repo
# Will show error if locked

# Break stale lock (only if you're SURE no borg process is running)  # [CONFIRM]
borg break-lock /backup/borg-repo

Checklist: Borg Backup Setup

  • [ ] Install borgbackup from EPEL
  • [ ] Initialize repository with encryption (repokey or repokey-blake2)
  • [ ] Export encryption key (store separately from backup!)
  • [ ] Test manual backup creation
  • [ ] Test restore (full and partial)
  • [ ] Configure exclusions (caches, temp files, build artifacts)
  • [ ] Create backup script with logging
  • [ ] Set up systemd timer
  • [ ] Configure retention policy (prune settings)
  • [ ] Set up monitoring/alerting for backup age
  • [ ] Document passphrase location and recovery procedure
  • [ ] Test disaster recovery procedure end-to-end

When to Use This Skill

  • Setting up a new backup infrastructure
  • Creating backups of systems, databases, or applications
  • Restoring files from backup (full or partial)
  • Managing backup retention (pruning)
  • Automating backups with systemd timers
  • Monitoring backup health and freshness
  • Disaster recovery planning and testing
  • rocky-foundation -- OS detection, safety tiers
  • rocky-core-system -- systemd timers for scheduling, EPEL setup
  • rocky-storage -- Storage management for backup destinations
  • rocky-webstack -- MariaDB backup integration
  • rocky-opensearch -- OpenSearch snapshot management (complementary to borg)
  • rocky-security-hardening -- Securing backup repositories and keys

# Supported AI Coding Agents

This skill is compatible with the SKILL.md standard and works with all major AI coding agents:

Learn more about the SKILL.md standard and how to use these skills with your preferred AI coding agent.