429ec3fa2d
A repository holds one system=config snapshot and one snapshot per app. They are separate because they are used separately: the settings tree is small, changes rarely and is meaningless per-app, while app data is large, changes at its own rate, and has to be restorable, movable and ageable on its own — which is what the per-app tag buys. The step listed "Apps" and "Domains" as peer sections, which hid that entirely. It read as though a backup held three kinds of thing, and gave no clue that the domains come OUT of the system snapshot rather than being a third kind. Now: a Settings section (one snapshot, dated, saying plainly that it carries every backup repository with its credentials and is restored first because it is what makes the others reachable) with the domains nested under it and each one's DNS verdict; then an App data section, one snapshot each, dated and sized. A repository with app data and no settings snapshot says so — the consequence, that repositories and logins do not come back, is not something to find out afterwards. restoreInspect emits that shape now, assembled with jq against the discover JSON rather than by hand-rolled string concatenation, and carries each snapshot's date: a backup's age is what people actually judge it by. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
312 lines
13 KiB
Bash
312 lines
13 KiB
Bash
#!/bin/bash
|
|
|
|
# Read a backup repository and report what a restore from it would bring —
|
|
# without writing anything.
|
|
#
|
|
# This is what lets a person decide before committing, in the WebUI as well as
|
|
# the installer: which machine's backups are in there, which apps, how big, how
|
|
# recent, and which domains would come across. Every answer is read out of
|
|
# snapshots; nothing is restored, nothing on this machine is touched.
|
|
#
|
|
# The domains matter most and are the least obvious to get at. They live in the
|
|
# system-config snapshot, in network/network_domains — so they can be pulled
|
|
# with engineDumpFile the same way the preflight pulls an app's manifest, long
|
|
# before anything is adopted. Knowing "this backup will hand you six domains,
|
|
# four of which point somewhere else" is the difference between a rebuild and a
|
|
# surprise.
|
|
#
|
|
# Emits one JSON object, for the WebUI to render and for a person to read.
|
|
|
|
# The domains recorded in the repository's system-config snapshot.
|
|
# Silent on failure: an older backup may not carry the file, and that is not a
|
|
# reason to refuse an inspection.
|
|
restoreInspectDomains()
|
|
{
|
|
local idx="$1" host="$2"
|
|
|
|
# There is no engineSystemSnapshotLatestId — only the JSON listing — so
|
|
# take the last entry, which is the newest: restic returns snapshots
|
|
# oldest-first.
|
|
local snap
|
|
snap=$(engineSystemSnapshotsJson "$idx" "$host" 2>/dev/null \
|
|
| grep -o '"short_id":"[^"]*"' | tail -1 | cut -d'"' -f4)
|
|
[[ -n "$snap" ]] || return 0
|
|
|
|
# The snapshot is the config tree, taken from this install's configs dir, so
|
|
# the file sits at that absolute path inside it. Ask for a few plausible
|
|
# roots rather than assuming one: the source machine's config root is
|
|
# exactly the thing that differs between installs.
|
|
local base out=""
|
|
for base in "${configs_dir%/}" "/libreportal-system/configs" "/docker/configs"; do
|
|
out=$(engineDumpFile "$idx" "$snap" "$base/network/network_domains" 2>/dev/null)
|
|
[[ -n "$out" ]] && break
|
|
done
|
|
[[ -n "$out" ]] || return 0
|
|
|
|
local line v
|
|
while IFS= read -r line; do
|
|
[[ "$line" =~ ^CFG_DOMAIN_[0-9]+= ]] || continue
|
|
v="${line#*=}"; v="${v%%#*}"
|
|
v="${v#"${v%%[![:space:]]*}"}"; v="${v%"${v##*[![:space:]]}"}"
|
|
v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}"
|
|
v="${v#"${v%%[![:space:]]*}"}"; v="${v%"${v##*[![:space:]]}"}"
|
|
[[ -n "$v" ]] && printf '%s\n' "$v"
|
|
done <<< "$out"
|
|
}
|
|
|
|
# Inspect a repository. Prints JSON on stdout.
|
|
#
|
|
# restore inspect <location-idx> [host]
|
|
#
|
|
# A repository holds two DIFFERENT kinds of snapshot and the report keeps them
|
|
# apart, because they are restored by different machinery and mean different
|
|
# things to the person reading:
|
|
#
|
|
# system=config ONE snapshot of the whole configs tree — settings, logins,
|
|
# the domains, and every backup repository with its
|
|
# credentials. Restored first, because it is what makes the
|
|
# others reachable.
|
|
# app=<name> ONE SNAPSHOT PER APP of that app's data directory, each
|
|
# with its own manifest, its own size and its own schedule.
|
|
# Separate so an app can be restored, moved or aged out on
|
|
# its own without touching the rest.
|
|
#
|
|
# Showing them as one flat list was confusing precisely because it hid that.
|
|
#
|
|
# With no host it reports every host it found and picks the one with the most
|
|
# apps as the suggestion — a repository pointed at two machines is a normal
|
|
# thing to end up with, and guessing silently is not.
|
|
restoreInspect()
|
|
{
|
|
local idx="${1:-}" want_host="${2:-}"
|
|
if [[ -z "$idx" ]]; then
|
|
echo '{"error":"a backup location is required"}'
|
|
return 1
|
|
fi
|
|
|
|
local raw snaps
|
|
raw=$(restoreFirstRunDiscover "$idx" 2>/dev/null)
|
|
# The function can print notices before the JSON, so take the array itself
|
|
# rather than assuming the whole of stdout is the document.
|
|
snaps=$(printf '%s' "$raw" | sed -n '/^\[/,$p')
|
|
if [[ -z "$snaps" ]] || ! jq -e 'type == "array"' >/dev/null 2>&1 <<< "$snaps"; then
|
|
# The single most common cause by a distance, and worth saying plainly
|
|
# rather than as "discovery failed".
|
|
echo '{"error":"Could not read that repository — wrong password, or not a LibrePortal backup."}'
|
|
return 1
|
|
fi
|
|
|
|
local hosts
|
|
hosts=$(jq -r '[.[].hostname] | unique | .[]' <<< "$snaps" 2>/dev/null)
|
|
if [[ -z "$hosts" ]]; then
|
|
echo '{"error":"No LibrePortal backups found in that repository."}'
|
|
return 1
|
|
fi
|
|
|
|
local host="$want_host" h
|
|
if [[ -z "$host" ]]; then
|
|
# The one with the most apps, not simply the first: a repository often
|
|
# carries a stray snapshot from a machine that was only ever tested.
|
|
local best="" best_n=-1 n
|
|
while IFS= read -r h; do
|
|
n=$(jq -r --arg h "$h" \
|
|
'[.[] | select(.hostname == $h) | .tags[] | select(startswith("app="))] | unique | length' \
|
|
<<< "$snaps" 2>/dev/null)
|
|
[[ "$n" =~ ^[0-9]+$ ]] || n=0
|
|
if (( n > best_n )); then best_n=$n; best="$h"; fi
|
|
done <<< "$hosts"
|
|
host="$best"
|
|
fi
|
|
|
|
# The system snapshot: newest one tagged system=config for this host.
|
|
local system_json
|
|
system_json=$(jq -c --arg h "$host" '
|
|
[ .[] | select(.hostname == $h)
|
|
| select(any(.tags[]?; . == "system=config")) ]
|
|
| sort_by(.time)
|
|
| if length > 0 then {present: true, date: (.[-1].time)} else {present: false, date: ""} end
|
|
' <<< "$snaps" 2>/dev/null)
|
|
[[ -n "$system_json" ]] || system_json='{"present":false,"date":""}'
|
|
|
|
# One entry per app, newest snapshot of each.
|
|
local apps_json
|
|
apps_json=$(jq -c --arg h "$host" '
|
|
[ .[] | select(.hostname == $h)
|
|
| . as $s
|
|
| (.tags[]? | select(startswith("app=")) | ltrimstr("app=")) as $name
|
|
| {name: $name, time: $s.time} ]
|
|
| group_by(.name)
|
|
| map({name: .[0].name, date: (sort_by(.time) | .[-1].time)})
|
|
| sort_by(.name)
|
|
' <<< "$snaps" 2>/dev/null)
|
|
[[ -n "$apps_json" ]] || apps_json='[]'
|
|
|
|
# Sizes come from each app's own manifest. An older backup without one
|
|
# still lists — it just has less to say about itself, which is better than
|
|
# being left out of the list.
|
|
local sized='[]' a size_b size_h
|
|
while IFS= read -r a; do
|
|
[[ -z "$a" ]] && continue
|
|
size_b=$(restorePreflightManifest "$idx" "$a" "$host" 2>/dev/null \
|
|
| tr -d ' \n\t' | grep -o '"size_bytes":[0-9]*' | cut -d: -f2)
|
|
size_h=""
|
|
[[ -n "$size_b" ]] && size_h=$(_restorePfSize "$size_b")
|
|
sized=$(jq -c --arg n "$a" --arg s "$size_h" \
|
|
'. + [{name: $n, size: $s}]' <<< "$sized")
|
|
done < <(jq -r '.[].name' <<< "$apps_json" 2>/dev/null)
|
|
|
|
jq -nc \
|
|
--arg host "$host" \
|
|
--argjson hosts "$(jq -c '[.[].hostname] | unique' <<< "$snaps")" \
|
|
--argjson system "$system_json" \
|
|
--argjson apps "$apps_json" \
|
|
--argjson sizes "$sized" \
|
|
--argjson domains "$(restoreInspectDomains "$idx" "$host" | jq -Rsc 'split("\n") | map(select(length > 0))')" \
|
|
'{
|
|
host: $host,
|
|
hosts: $hosts,
|
|
# Domains live under system because that is where they come from —
|
|
# the configs tree, in the one system=config snapshot. Presenting
|
|
# them as a peer of the app list is what made the step confusing.
|
|
system: ($system + {domains: $domains}),
|
|
apps: [ $apps[] as $a
|
|
| $a + {size: (([$sizes[] | select(.name == $a.name) | .size] | first) // "")} ]
|
|
}'
|
|
return 0
|
|
}
|
|
|
|
# Connect a repository described by a base64 JSON payload, then inspect it.
|
|
#
|
|
# restoreConnectInspect <base64-json>
|
|
#
|
|
# Payload: {"location":{"name","type","path","uri","ssh_*","password_ref",…},
|
|
# "host":"optional"}
|
|
#
|
|
# The password arrives as a REFERENCE, never a value. This payload reaches a
|
|
# task command line and tasks are recorded world-readable, so a secret
|
|
# travelling as itself would be readable by any local account —
|
|
# webuiSecretResolve redeems it here, once, at the moment of the write. Same
|
|
# contract as the wizard's backup destinations.
|
|
#
|
|
# Deliberately does NOT call engineInitLocation. Every other path that creates
|
|
# a location initialises it, because it is about to write backups there. This
|
|
# one is pointed at a repository that already exists and is only going to be
|
|
# read; initialising is at best a no-op and at worst the wrong answer to
|
|
# "that path is empty".
|
|
restoreConnectInspect()
|
|
{
|
|
local b64="${1:-}"
|
|
if [[ -z "$b64" ]]; then
|
|
echo '{"error":"no payload"}'
|
|
return 1
|
|
fi
|
|
|
|
local payload
|
|
payload=$(printf '%s' "$b64" | base64 -d 2>/dev/null)
|
|
if [[ -z "$payload" ]]; then
|
|
echo '{"error":"payload could not be decoded"}'
|
|
return 1
|
|
fi
|
|
|
|
local loc; loc=$(jq -c '.location // {}' <<< "$payload" 2>/dev/null)
|
|
[[ -n "$loc" && "$loc" != "{}" ]] || { echo '{"error":"no backup location in the payload"}'; return 1; }
|
|
|
|
local name type idx
|
|
name=$(jq -r '.name // "restore-source"' <<< "$loc")
|
|
type=$(jq -r '.type // "local"' <<< "$loc")
|
|
|
|
idx=$(locationAdd "$name" "$type" 2>/dev/null | tail -1)
|
|
if [[ ! "$idx" =~ ^[0-9]+$ ]]; then
|
|
echo '{"error":"Could not create a backup location to read from."}'
|
|
return 1
|
|
fi
|
|
|
|
local cfg; cfg=$(backupLocationConfig "$idx")
|
|
if [[ ! -f "$cfg" ]]; then
|
|
echo '{"error":"The backup location was created but has no config file."}'
|
|
return 1
|
|
fi
|
|
|
|
local v
|
|
v=$(jq -r '.path // ""' <<< "$loc")
|
|
if [[ -n "$v" ]]; then
|
|
updateConfigOption "CFG_BACKUP_LOC_${idx}_PATH_MODE" "custom" "$cfg" >/dev/null
|
|
updateConfigOption "CFG_BACKUP_LOC_${idx}_PATH" "$v" "$cfg" >/dev/null
|
|
fi
|
|
v=$(jq -r '.uri // ""' <<< "$loc")
|
|
[[ -n "$v" ]] && updateConfigOption "CFG_BACKUP_LOC_${idx}_URI" "$v" "$cfg" >/dev/null
|
|
|
|
local k
|
|
for k in ssh_user ssh_host ssh_port ssh_path s3_bucket s3_key_id b2_account_id; do
|
|
v=$(jq -r --arg k "$k" '.[$k] // ""' <<< "$loc")
|
|
[[ -n "$v" ]] && updateConfigOption "CFG_BACKUP_LOC_${idx}_${k^^}" "$v" "$cfg" >/dev/null
|
|
done
|
|
|
|
# Every secret in the payload, redeemed at the point of use.
|
|
local ref pw
|
|
for k in password ssh_pass s3_key b2_key connect_token; do
|
|
ref=$(jq -r --arg k "${k}_ref" '.[$k] // ""' <<< "$loc")
|
|
[[ -n "$ref" ]] || continue
|
|
if pw=$(webuiSecretResolve "$ref" 2>/dev/null) && [[ -n "$pw" ]]; then
|
|
updateConfigOption "CFG_BACKUP_LOC_${idx}_${k^^}" "$pw" "$cfg" >/dev/null
|
|
pw=""
|
|
fi
|
|
done
|
|
|
|
updateConfigOption "CFG_BACKUP_LOC_${idx}_ENABLED" "true" "$cfg" >/dev/null
|
|
# The credentials just landed, so close the directory to everything that is
|
|
# not LibrePortal before anything else runs.
|
|
runOwnership config-secure >/dev/null 2>&1 || true
|
|
source "$cfg" 2>/dev/null
|
|
|
|
local host; host=$(jq -r '.host // ""' <<< "$payload")
|
|
local out; out=$(restoreInspect "$idx" "$host")
|
|
|
|
# The caller needs the index to restore from later, and it is not otherwise
|
|
# discoverable from the WebUI without guessing.
|
|
if grep -q '"error"' <<< "$out"; then
|
|
# Remove the location we just made. A wrong password is the ordinary
|
|
# case here and the user will simply try again — without this, every
|
|
# retry leaves another half-configured destination behind, and after a
|
|
# few attempts the Backup page lists a column of identical dead
|
|
# entries the user then has to clean up by hand.
|
|
locationRemove "$idx" >/dev/null 2>&1 || true
|
|
printf '%s\n' "$out"
|
|
return 1
|
|
fi
|
|
# Spliced with jq rather than by trimming a closing brace: string surgery
|
|
# on JSON is how you ship something that parses on your machine and not on
|
|
# the next one.
|
|
jq -c --arg idx "$idx" '. + {location_idx: $idx}' <<< "$out"
|
|
return 0
|
|
}
|
|
|
|
# Same connect-and-inspect, published where the WebUI can read it.
|
|
#
|
|
# The WebUI cannot read a task's stdout, so the result is written beside the
|
|
# other generated data and polled for. A nonce echoed back from the request is
|
|
# what lets the browser tell ITS answer from a stale document left by an
|
|
# earlier attempt — without it a second read shows the first read's repository,
|
|
# which is the kind of wrong that looks entirely plausible.
|
|
restoreConnectInspectPublish()
|
|
{
|
|
local b64="${1:-}" nonce="${2:-}"
|
|
local out_dir; out_dir="$(webuiDir)/frontend/data/system"
|
|
local out_file="$out_dir/restore_read.json"
|
|
createFolders "quiet" "$sudo_user_name" "$out_dir"
|
|
|
|
local body rc
|
|
body=$(restoreConnectInspect "$b64"); rc=$?
|
|
[[ -n "$body" ]] || body='{"error":"the read produced no result"}'
|
|
|
|
local tmp; tmp=$(mktemp) || return 1
|
|
jq -c --arg nonce "$nonce" --arg at "$(date -Iseconds)" \
|
|
'. + {nonce: $nonce, checked: $at}' <<< "$body" > "$tmp" 2>/dev/null \
|
|
|| printf '{"error":"the read produced unreadable output","nonce":"%s"}\n' \
|
|
"$(_lpJsonStr "$nonce")" > "$tmp"
|
|
|
|
runFileWrite "$out_file" < "$tmp"
|
|
rm -f "$tmp"
|
|
return $rc
|
|
}
|