71a02374d1
Follow-up to the backup-refresh throttle/dedupe, cutting the cost of the remote pulls that do still happen. * restic stats now runs in --mode raw-data (restic_check.sh). The default restore-size mode walks every snapshot's tree to sum logical file sizes — the slowest restic op — just to fill a size readout. raw-data reads the index only and reports the repository's actual deduplicated on-disk size, which is exactly what the dashboard already labels "deduplicated, encrypted". raw-data omits total_file_count, so the per-location card now shows that location's snapshot count (already loaded client-side, and more useful for a backup repo) instead of a file count. * engineLocationStats now shares the same per-refresh memoiser as engineSnapshotsJson (engine_dispatch.sh). Both the locations and dashboard generators call it per location, so repo stats went from two restic calls per location per refresh to one. Factored the cache into _engineCachedPull. * SSH connection reuse for SFTP locations (backup_ssh.sh): ControlMaster=auto with a self-reaping ControlPersist master, so the several restic subprocesses a refresh/backup spawns against one location share a single authenticated connection instead of a fresh handshake each — the dominant per-call cost on a high-latency link. Toggle via CFG_BACKUP_SSH_MULTIPLEX. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: librelad <librelad@digitalangels.vip>
70 lines
2.8 KiB
Bash
70 lines
2.8 KiB
Bash
#!/bin/bash
|
|
|
|
# Build the SSH command engines use to reach an SFTP location. Honours
|
|
# CFG_BACKUP_LOC_N_SSH_AUTH:
|
|
# - "key" (default): plain `ssh ... -s sftp` / `ssh ...`
|
|
# - "password": exports SSHPASS=<pass> and prefixes with `sshpass -e`
|
|
#
|
|
# Args:
|
|
# idx — location index
|
|
# port — SSH port
|
|
# mode — "sftp" (append `-s sftp`) or "raw" (just the ssh prefix, no -s)
|
|
#
|
|
# Echoes the command on stdout. Returns non-zero with an isError if password
|
|
# mode is requested without sshpass on PATH.
|
|
|
|
backupSshCommand()
|
|
{
|
|
local idx="$1"
|
|
local port="${2:-22}"
|
|
local mode="${3:-raw}"
|
|
|
|
local auth pass
|
|
auth=$(resticLocationField "$idx" SSH_AUTH)
|
|
pass=$(resticLocationField "$idx" SSH_PASS)
|
|
[[ -z "$auth" ]] && auth=key
|
|
|
|
# Connection multiplexing: a single WebUI refresh (repo stats + snapshot
|
|
# list) and each backup push run several restic subprocesses against the
|
|
# same location; ControlMaster lets them share one authenticated SSH
|
|
# connection instead of a fresh handshake each — the dominant per-call cost
|
|
# on a high-latency link. auto = reuse an existing master or open one; the
|
|
# master self-reaps ControlPersist seconds after its last channel closes.
|
|
# %C keeps the socket path short and unique per (host,port,user). Opt out
|
|
# with CFG_BACKUP_SSH_MULTIPLEX=false.
|
|
local mux=""
|
|
if [[ "${CFG_BACKUP_SSH_MULTIPLEX:-true}" != "false" ]]; then
|
|
local mux_dir="${TMPDIR:-/tmp}"; mux_dir="${mux_dir%/}"
|
|
mux=" -o ControlMaster=auto -o ControlPath=${mux_dir}/lp-bkmux-%C -o ControlPersist=30"
|
|
fi
|
|
|
|
local base="ssh -p $port -o StrictHostKeyChecking=accept-new${mux}"
|
|
[[ "$mode" == "sftp" ]] && local suffix=" -s sftp" || local suffix=""
|
|
|
|
if [[ "$auth" == "password" ]]; then
|
|
if [[ -z "$pass" ]]; then
|
|
isError "Location $idx is set to password auth but CFG_BACKUP_LOC_${idx}_SSH_PASS is empty"
|
|
return 1
|
|
fi
|
|
if ! command -v sshpass >/dev/null 2>&1; then
|
|
isError "sshpass not installed but location $idx uses password auth — apt install sshpass"
|
|
return 1
|
|
fi
|
|
export SSHPASS="$pass"
|
|
echo "sshpass -e $base -o PreferredAuthentications=password -o PubkeyAuthentication=no${suffix}"
|
|
else
|
|
# Key mode: when LibrePortal has a per-location key, pin -i and force
|
|
# identities-only so the right key is used; otherwise fall back to
|
|
# whatever the docker_install_user has configured.
|
|
local key_file=""
|
|
if declare -f backupSshKeyFile >/dev/null 2>&1; then
|
|
key_file=$(backupSshKeyFile "$idx")
|
|
fi
|
|
if [[ -n "$key_file" && -f "$key_file" ]]; then
|
|
echo "$base -i $key_file -o IdentitiesOnly=yes${suffix}"
|
|
else
|
|
echo "$base${suffix}"
|
|
fi
|
|
fi
|
|
}
|