6018250526
A snapshot is one app's data, or the settings tree — never a machine. A
four-snapshot repository is typically two apps plus two versions of the
settings, not four backups to pick between. So the choice belongs on Contents,
after unlocking, where each snapshot has a name and a date rather than being a
hash.
Every row with more than one snapshot gets a picker, defaulting to the newest.
A row with one shows its date as text: a dropdown holding a single entry is a
control that cannot be operated, and it makes a repository with one backup look
like it is hiding something.
The chain already supported this. restorePickSnapshot has always passed any
value that is not the string "latest" straight through as an id; nothing ever
offered the choice. What was missing:
- restoreInspect returns every snapshot per app and for the settings, not
just the newest.
- restoreFirstRunBulk reads an optional RESTORE_SNAPSHOT_CHOICE map instead
of hardcoding "latest". An associative array rather than an argument,
because the CLI wrapper pads argv to nine slots and a per-app map cannot
survive it; the map reaches the host as base64 JSON, validated at the route
against restic short ids and app names since both hit a command line.
- backupRestoreSystemConfig takes a snapshot AND a host.
That host was a real bug. It defaulted to this machine's install name, which is
right for "recover my own settings" and wrong for a rebuild — the snapshots
carry the name of the machine being rebuilt FROM. It surfaced the moment a
restore adopted a config with a different install name and the next lookup
found nothing at all.
Verified by restoring both settings snapshots and diffing: 28bedbb0 brings back
a config carrying example.com, cc5b6bcf one with no domains.
Two CSS traps on the picker: appearance stayed `auto`, so the browser painted
its own control and ignored the colours entirely while the computed styles
looked right; and a `background:` shorthand later in the rule silently reset the
background-image, wiping the arrow set three lines above it.
lp-restore-adopt-test asserted configs/* were mode 0755 and started failing on
configs/webui, which libreportal-ownership sets to 0751:container on purpose —
tighter, and perfectly traversable. It asserts "the container user can traverse
it" now. A test that pins an incidental number reports a regression every time
someone improves the thing it is watching.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
324 lines
14 KiB
Bash
324 lines
14 KiB
Bash
#!/bin/bash
|
|
|
|
# Read a backup repository and report what a restore from it would bring —
|
|
# without writing anything.
|
|
#
|
|
# This is what lets a person decide before committing, in the WebUI as well as
|
|
# the installer: which machine's backups are in there, which apps, how big, how
|
|
# recent, and which domains would come across. Every answer is read out of
|
|
# snapshots; nothing is restored, nothing on this machine is touched.
|
|
#
|
|
# The domains matter most and are the least obvious to get at. They live in the
|
|
# system-config snapshot, in network/network_domains — so they can be pulled
|
|
# with engineDumpFile the same way the preflight pulls an app's manifest, long
|
|
# before anything is adopted. Knowing "this backup will hand you six domains,
|
|
# four of which point somewhere else" is the difference between a rebuild and a
|
|
# surprise.
|
|
#
|
|
# Emits one JSON object, for the WebUI to render and for a person to read.
|
|
|
|
# The domains recorded in the repository's system-config snapshot.
|
|
# Silent on failure: an older backup may not carry the file, and that is not a
|
|
# reason to refuse an inspection.
|
|
restoreInspectDomains()
|
|
{
|
|
local idx="$1" host="$2"
|
|
|
|
# There is no engineSystemSnapshotLatestId — only the JSON listing — so
|
|
# take the last entry, which is the newest: restic returns snapshots
|
|
# oldest-first.
|
|
local snap
|
|
snap=$(engineSystemSnapshotsJson "$idx" "$host" 2>/dev/null \
|
|
| grep -o '"short_id":"[^"]*"' | tail -1 | cut -d'"' -f4)
|
|
[[ -n "$snap" ]] || return 0
|
|
|
|
# The snapshot is the config tree, taken from this install's configs dir, so
|
|
# the file sits at that absolute path inside it. Ask for a few plausible
|
|
# roots rather than assuming one: the source machine's config root is
|
|
# exactly the thing that differs between installs.
|
|
local base out=""
|
|
for base in "${configs_dir%/}" "/libreportal-system/configs" "/docker/configs"; do
|
|
out=$(engineDumpFile "$idx" "$snap" "$base/network/network_domains" 2>/dev/null)
|
|
[[ -n "$out" ]] && break
|
|
done
|
|
[[ -n "$out" ]] || return 0
|
|
|
|
local line v
|
|
while IFS= read -r line; do
|
|
[[ "$line" =~ ^CFG_DOMAIN_[0-9]+= ]] || continue
|
|
v="${line#*=}"; v="${v%%#*}"
|
|
v="${v#"${v%%[![:space:]]*}"}"; v="${v%"${v##*[![:space:]]}"}"
|
|
v="${v%\"}"; v="${v#\"}"; v="${v%\'}"; v="${v#\'}"
|
|
v="${v#"${v%%[![:space:]]*}"}"; v="${v%"${v##*[![:space:]]}"}"
|
|
[[ -n "$v" ]] && printf '%s\n' "$v"
|
|
done <<< "$out"
|
|
}
|
|
|
|
# Inspect a repository. Prints JSON on stdout.
|
|
#
|
|
# restore inspect <location-idx> [host]
|
|
#
|
|
# A repository holds two DIFFERENT kinds of snapshot and the report keeps them
|
|
# apart, because they are restored by different machinery and mean different
|
|
# things to the person reading:
|
|
#
|
|
# system=config ONE snapshot of the whole configs tree — settings, logins,
|
|
# the domains, and every backup repository with its
|
|
# credentials. Restored first, because it is what makes the
|
|
# others reachable.
|
|
# app=<name> ONE SNAPSHOT PER APP of that app's data directory, each
|
|
# with its own manifest, its own size and its own schedule.
|
|
# Separate so an app can be restored, moved or aged out on
|
|
# its own without touching the rest.
|
|
#
|
|
# Showing them as one flat list was confusing precisely because it hid that.
|
|
#
|
|
# With no host it reports every host it found and picks the one with the most
|
|
# apps as the suggestion — a repository pointed at two machines is a normal
|
|
# thing to end up with, and guessing silently is not.
|
|
restoreInspect()
|
|
{
|
|
local idx="${1:-}" want_host="${2:-}"
|
|
if [[ -z "$idx" ]]; then
|
|
echo '{"error":"a backup location is required"}'
|
|
return 1
|
|
fi
|
|
|
|
local raw snaps
|
|
raw=$(restoreFirstRunDiscover "$idx" 2>/dev/null)
|
|
# The function can print notices before the JSON, so take the array itself
|
|
# rather than assuming the whole of stdout is the document.
|
|
snaps=$(printf '%s' "$raw" | sed -n '/^\[/,$p')
|
|
if [[ -z "$snaps" ]] || ! jq -e 'type == "array"' >/dev/null 2>&1 <<< "$snaps"; then
|
|
# The single most common cause by a distance, and worth saying plainly
|
|
# rather than as "discovery failed".
|
|
echo '{"error":"Could not read that repository — wrong password, or not a LibrePortal backup."}'
|
|
return 1
|
|
fi
|
|
|
|
local hosts
|
|
hosts=$(jq -r '[.[].hostname] | unique | .[]' <<< "$snaps" 2>/dev/null)
|
|
if [[ -z "$hosts" ]]; then
|
|
echo '{"error":"No LibrePortal backups found in that repository."}'
|
|
return 1
|
|
fi
|
|
|
|
local host="$want_host" h
|
|
if [[ -z "$host" ]]; then
|
|
# The one with the most apps, not simply the first: a repository often
|
|
# carries a stray snapshot from a machine that was only ever tested.
|
|
local best="" best_n=-1 n
|
|
while IFS= read -r h; do
|
|
n=$(jq -r --arg h "$h" \
|
|
'[.[] | select(.hostname == $h) | .tags[] | select(startswith("app="))] | unique | length' \
|
|
<<< "$snaps" 2>/dev/null)
|
|
[[ "$n" =~ ^[0-9]+$ ]] || n=0
|
|
if (( n > best_n )); then best_n=$n; best="$h"; fi
|
|
done <<< "$hosts"
|
|
host="$best"
|
|
fi
|
|
|
|
# The system snapshot: newest one tagged system=config for this host.
|
|
local system_json
|
|
system_json=$(jq -c --arg h "$host" '
|
|
[ .[] | select(.hostname == $h)
|
|
| select(any(.tags[]?; . == "system=config")) ]
|
|
| sort_by(.time) | reverse
|
|
| if length > 0
|
|
then {present: true, date: .[0].time,
|
|
snapshots: map({id: .short_id, time: .time})}
|
|
else {present: false, date: "", snapshots: []} end
|
|
' <<< "$snaps" 2>/dev/null)
|
|
[[ -n "$system_json" ]] || system_json='{"present":false,"date":"","snapshots":[]}'
|
|
|
|
# One entry per app, newest first, WITH every snapshot it has.
|
|
#
|
|
# The per-snapshot list is what makes "restore this app as it was on
|
|
# Tuesday" possible: restorePickSnapshot already passes any id that is not
|
|
# the string "latest" straight through, so the chain has always supported
|
|
# it — nothing ever offered the choice.
|
|
local apps_json
|
|
apps_json=$(jq -c --arg h "$host" '
|
|
[ .[] | select(.hostname == $h)
|
|
| . as $s
|
|
| (.tags[]? | select(startswith("app=")) | ltrimstr("app=")) as $name
|
|
| {name: $name, time: $s.time, id: $s.short_id} ]
|
|
| group_by(.name)
|
|
| map({
|
|
name: .[0].name,
|
|
date: (sort_by(.time) | .[-1].time),
|
|
snapshots: (sort_by(.time) | reverse | map({id: .id, time: .time}))
|
|
})
|
|
| sort_by(.name)
|
|
' <<< "$snaps" 2>/dev/null)
|
|
[[ -n "$apps_json" ]] || apps_json='[]'
|
|
|
|
# Sizes come from each app's own manifest. An older backup without one
|
|
# still lists — it just has less to say about itself, which is better than
|
|
# being left out of the list.
|
|
local sized='[]' a size_b size_h
|
|
while IFS= read -r a; do
|
|
[[ -z "$a" ]] && continue
|
|
size_b=$(restorePreflightManifest "$idx" "$a" "$host" 2>/dev/null \
|
|
| tr -d ' \n\t' | grep -o '"size_bytes":[0-9]*' | cut -d: -f2)
|
|
size_h=""
|
|
[[ -n "$size_b" ]] && size_h=$(_restorePfSize "$size_b")
|
|
sized=$(jq -c --arg n "$a" --arg s "$size_h" \
|
|
'. + [{name: $n, size: $s}]' <<< "$sized")
|
|
done < <(jq -r '.[].name' <<< "$apps_json" 2>/dev/null)
|
|
|
|
jq -nc \
|
|
--arg host "$host" \
|
|
--argjson hosts "$(jq -c '[.[].hostname] | unique' <<< "$snaps")" \
|
|
--argjson system "$system_json" \
|
|
--argjson apps "$apps_json" \
|
|
--argjson sizes "$sized" \
|
|
--argjson domains "$(restoreInspectDomains "$idx" "$host" | jq -Rsc 'split("\n") | map(select(length > 0))')" \
|
|
'{
|
|
host: $host,
|
|
hosts: $hosts,
|
|
# Domains live under system because that is where they come from —
|
|
# the configs tree, in the one system=config snapshot. Presenting
|
|
# them as a peer of the app list is what made the step confusing.
|
|
system: ($system + {domains: $domains}),
|
|
apps: [ $apps[] as $a
|
|
| $a + {size: (([$sizes[] | select(.name == $a.name) | .size] | first) // "")} ]
|
|
}'
|
|
return 0
|
|
}
|
|
|
|
# Connect a repository described by a base64 JSON payload, then inspect it.
|
|
#
|
|
# restoreConnectInspect <base64-json>
|
|
#
|
|
# Payload: {"location":{"name","type","path","uri","ssh_*","password_ref",…},
|
|
# "host":"optional"}
|
|
#
|
|
# The password arrives as a REFERENCE, never a value. This payload reaches a
|
|
# task command line and tasks are recorded world-readable, so a secret
|
|
# travelling as itself would be readable by any local account —
|
|
# webuiSecretResolve redeems it here, once, at the moment of the write. Same
|
|
# contract as the wizard's backup destinations.
|
|
#
|
|
# Deliberately does NOT call engineInitLocation. Every other path that creates
|
|
# a location initialises it, because it is about to write backups there. This
|
|
# one is pointed at a repository that already exists and is only going to be
|
|
# read; initialising is at best a no-op and at worst the wrong answer to
|
|
# "that path is empty".
|
|
restoreConnectInspect()
|
|
{
|
|
local b64="${1:-}"
|
|
if [[ -z "$b64" ]]; then
|
|
echo '{"error":"no payload"}'
|
|
return 1
|
|
fi
|
|
|
|
local payload
|
|
payload=$(printf '%s' "$b64" | base64 -d 2>/dev/null)
|
|
if [[ -z "$payload" ]]; then
|
|
echo '{"error":"payload could not be decoded"}'
|
|
return 1
|
|
fi
|
|
|
|
local loc; loc=$(jq -c '.location // {}' <<< "$payload" 2>/dev/null)
|
|
[[ -n "$loc" && "$loc" != "{}" ]] || { echo '{"error":"no backup location in the payload"}'; return 1; }
|
|
|
|
local name type idx
|
|
name=$(jq -r '.name // "restore-source"' <<< "$loc")
|
|
type=$(jq -r '.type // "local"' <<< "$loc")
|
|
|
|
idx=$(locationAdd "$name" "$type" 2>/dev/null | tail -1)
|
|
if [[ ! "$idx" =~ ^[0-9]+$ ]]; then
|
|
echo '{"error":"Could not create a backup location to read from."}'
|
|
return 1
|
|
fi
|
|
|
|
local cfg; cfg=$(backupLocationConfig "$idx")
|
|
if [[ ! -f "$cfg" ]]; then
|
|
echo '{"error":"The backup location was created but has no config file."}'
|
|
return 1
|
|
fi
|
|
|
|
local v
|
|
v=$(jq -r '.path // ""' <<< "$loc")
|
|
if [[ -n "$v" ]]; then
|
|
updateConfigOption "CFG_BACKUP_LOC_${idx}_PATH_MODE" "custom" "$cfg" >/dev/null
|
|
updateConfigOption "CFG_BACKUP_LOC_${idx}_PATH" "$v" "$cfg" >/dev/null
|
|
fi
|
|
v=$(jq -r '.uri // ""' <<< "$loc")
|
|
[[ -n "$v" ]] && updateConfigOption "CFG_BACKUP_LOC_${idx}_URI" "$v" "$cfg" >/dev/null
|
|
|
|
local k
|
|
for k in ssh_user ssh_host ssh_port ssh_path s3_bucket s3_key_id b2_account_id; do
|
|
v=$(jq -r --arg k "$k" '.[$k] // ""' <<< "$loc")
|
|
[[ -n "$v" ]] && updateConfigOption "CFG_BACKUP_LOC_${idx}_${k^^}" "$v" "$cfg" >/dev/null
|
|
done
|
|
|
|
# Every secret in the payload, redeemed at the point of use.
|
|
local ref pw
|
|
for k in password ssh_pass s3_key b2_key connect_token; do
|
|
ref=$(jq -r --arg k "${k}_ref" '.[$k] // ""' <<< "$loc")
|
|
[[ -n "$ref" ]] || continue
|
|
if pw=$(webuiSecretResolve "$ref" 2>/dev/null) && [[ -n "$pw" ]]; then
|
|
updateConfigOption "CFG_BACKUP_LOC_${idx}_${k^^}" "$pw" "$cfg" >/dev/null
|
|
pw=""
|
|
fi
|
|
done
|
|
|
|
updateConfigOption "CFG_BACKUP_LOC_${idx}_ENABLED" "true" "$cfg" >/dev/null
|
|
# The credentials just landed, so close the directory to everything that is
|
|
# not LibrePortal before anything else runs.
|
|
runOwnership config-secure >/dev/null 2>&1 || true
|
|
source "$cfg" 2>/dev/null
|
|
|
|
local host; host=$(jq -r '.host // ""' <<< "$payload")
|
|
local out; out=$(restoreInspect "$idx" "$host")
|
|
|
|
# The caller needs the index to restore from later, and it is not otherwise
|
|
# discoverable from the WebUI without guessing.
|
|
if grep -q '"error"' <<< "$out"; then
|
|
# Remove the location we just made. A wrong password is the ordinary
|
|
# case here and the user will simply try again — without this, every
|
|
# retry leaves another half-configured destination behind, and after a
|
|
# few attempts the Backup page lists a column of identical dead
|
|
# entries the user then has to clean up by hand.
|
|
locationRemove "$idx" >/dev/null 2>&1 || true
|
|
printf '%s\n' "$out"
|
|
return 1
|
|
fi
|
|
# Spliced with jq rather than by trimming a closing brace: string surgery
|
|
# on JSON is how you ship something that parses on your machine and not on
|
|
# the next one.
|
|
jq -c --arg idx "$idx" '. + {location_idx: $idx}' <<< "$out"
|
|
return 0
|
|
}
|
|
|
|
# Same connect-and-inspect, published where the WebUI can read it.
|
|
#
|
|
# The WebUI cannot read a task's stdout, so the result is written beside the
|
|
# other generated data and polled for. A nonce echoed back from the request is
|
|
# what lets the browser tell ITS answer from a stale document left by an
|
|
# earlier attempt — without it a second read shows the first read's repository,
|
|
# which is the kind of wrong that looks entirely plausible.
|
|
restoreConnectInspectPublish()
|
|
{
|
|
local b64="${1:-}" nonce="${2:-}"
|
|
local out_dir; out_dir="$(webuiDir)/frontend/data/system"
|
|
local out_file="$out_dir/restore_read.json"
|
|
createFolders "quiet" "$sudo_user_name" "$out_dir"
|
|
|
|
local body rc
|
|
body=$(restoreConnectInspect "$b64"); rc=$?
|
|
[[ -n "$body" ]] || body='{"error":"the read produced no result"}'
|
|
|
|
local tmp; tmp=$(mktemp) || return 1
|
|
jq -c --arg nonce "$nonce" --arg at "$(date -Iseconds)" \
|
|
'. + {nonce: $nonce, checked: $at}' <<< "$body" > "$tmp" 2>/dev/null \
|
|
|| printf '{"error":"the read produced unreadable output","nonce":"%s"}\n' \
|
|
"$(_lpJsonStr "$nonce")" > "$tmp"
|
|
|
|
runFileWrite "$out_file" < "$tmp"
|
|
rm -f "$tmp"
|
|
return $rc
|
|
}
|