Webstar/LibrePortal
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Contents step: show the two snapshot kinds as two things

Browse Source 
A repository holds one system=config snapshot and one snapshot per app. They
are separate because they are used separately: the settings tree is small,
changes rarely and is meaningless per-app, while app data is large, changes at
its own rate, and has to be restorable, movable and ageable on its own — which
is what the per-app tag buys.

The step listed "Apps" and "Domains" as peer sections, which hid that entirely.
It read as though a backup held three kinds of thing, and gave no clue that the
domains come OUT of the system snapshot rather than being a third kind.

Now: a Settings section (one snapshot, dated, saying plainly that it carries
every backup repository with its credentials and is restored first because it
is what makes the others reachable) with the domains nested under it and each
one's DNS verdict; then an App data section, one snapshot each, dated and
sized. A repository with app data and no settings snapshot says so — the
consequence, that repositories and logins do not come back, is not something to
find out afterwards.

restoreInspect emits that shape now, assembled with jq against the discover
JSON rather than by hand-rolled string concatenation, and carries each
snapshot's date: a backup's age is what people actually judge it by.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
librelad 2026-08-29 05:51:24 +01:00
parent 6a62c94cf8
commit 429ec3fa2d
4 changed files with 231 additions and 80 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

108
containers/libreportal/frontend/core/setup/js/setup-wizard.js
View File

@ -1141,9 +1141,12 @@ class SetupWizard {
this.restoreInfo = data; this.restoreInfo = data;
if (status) { if (status) {
const nApps = (data.apps || []).length;
const hasSys = !!(data.system && data.system.present);
status.innerHTML = `<p class="setup-rs-ok">Found backups from status.innerHTML = `<p class="setup-rs-ok">Found backups from
<strong>${this.escapeHtml(data.host || '')}</strong> \u2014 <strong>${this.escapeHtml(data.host || '')}</strong> \u2014
${(data.apps || []).length} app(s). Continue to see what will happen.</p>`; ${hasSys ? 'settings and ' : ''}${nApps} app${nApps === 1 ? '' : 's'}.
Continue to see what will happen.</p>`;
} }
await this.renderRestoreContents(); await this.renderRestoreContents();
} catch (e) { } catch (e) {
@ -1156,9 +1159,21 @@ class SetupWizard {
} }
} }
// What is in there, and what it will mean on this machine. The domain checks // What is in there, and what it will mean on this machine.
// are the part that has no equivalent anywhere else: a restored domain still //
// points wherever DNS says, which after a rebuild is usually the old server. // A repository holds two DIFFERENT kinds of snapshot, and this step shows
// them as two things because they ARE two things:
//
// the system snapshot one snapshot of the settings tree — logins, the
// domains, and every backup repository with its
// credentials. Restored first, because it is what
// makes the others reachable.
// the app snapshots one per app, each with its own size and date, so
// an app can be restored or aged out on its own.
//
// The first version listed "Apps" and "Domains" as peers, which hid all of
// that: the domains are not a third thing in the backup, they are part of
// the system snapshot.
async renderRestoreContents() { async renderRestoreContents() {
const box = this.container.querySelector('#sw-rs-contents'); const box = this.container.querySelector('#sw-rs-contents');
if (!box) return; if (!box) return;
@ -1168,34 +1183,74 @@ class SetupWizard {
return; return;
} }
const system = d.system || {};
const apps = d.apps || []; const apps = d.apps || [];
const domains = d.domains || []; const domains = system.domains || [];
const others = (d.hosts || []).filter(h => h !== d.host);
box.innerHTML = ` box.innerHTML = `
<p class="setup-section-hint"> <p class="setup-section-hint">
From <strong>${this.escapeHtml(d.host || '')}</strong>${ From <strong>${this.escapeHtml(d.host || '')}</strong>${
(d.hosts || []).length > 1 others.length
? ` \u2014 this repository also holds backups from ${this.escapeHtml((d.hosts || []).filter(h => h !== d.host).join(', '))}` ? ` \u2014 this repository also holds backups from ${this.escapeHtml(others.join(', '))}`
: ''} : ''}
</p> </p>
<div class="setup-storage-divider"><span>Apps</span></div>
<div class="setup-storage-divider"><span>Settings</span></div>
${system.present
? `<div class="setup-app-card">
<span class="setup-app-name">System settings</span>
<span class="setup-storage-badge setup-storage-badge-ok">1 snapshot</span>
<span class="setup-app-desc">${this._restoreWhen(system.date)}</span>
</div>
<p class="setup-section-hint">
Your logins, your domains, and every backup repository you had \u2014
with its credentials, so this one password brings the rest back.
Restored first, because it is what makes the others reachable.
</p>
<div id="sw-rs-domains">${
domains.length
? '<p class="setup-section-hint">Checking where the domains point\u2026</p>'
: '<p class="setup-section-hint">No domains were configured on that machine.</p>'}</div>`
: `<p class="setup-section-hint">
This repository has no settings snapshot \u2014 only app data. Your
backup repositories and logins will not come back with it.
</p>`}
<div class="setup-storage-divider"><span>App data</span></div>
${apps.length ${apps.length
? apps.map(a => ` ? `<p class="setup-section-hint">
One snapshot each, restored after the settings.
</p>` +
apps.map(a => `
<div class="setup-app-card"> <div class="setup-app-card">
<span class="setup-app-name">${this.escapeHtml(a.name)}</span> <span class="setup-app-name">${this.escapeHtml(a.name)}</span>
<span class="setup-app-desc">${this.escapeHtml(a.size || '')}</span> <span class="setup-app-desc">${this.escapeHtml(a.size || '')}${
a.size && a.date ? ' \u00b7 ' : ''}${this._restoreWhen(a.date)}</span>
</div>`).join('') </div>`).join('')
: '<p class="setup-section-hint">No app backups in this repository.</p>'} : '<p class="setup-section-hint">No app backups in this repository.</p>'}`;
<div class="setup-storage-divider"><span>Domains</span></div>
<div id="sw-rs-domains">${
domains.length
? '<p class="setup-section-hint">Checking where these point\u2026</p>'
: '<p class="setup-section-hint">This backup carries no domains.</p>'}</div>`;
if (!domains.length) return; if (!domains.length) return;
await this._checkRestoreDomains(domains);
}
// Checked one at a time through the endpoint the Domains step already // "28 Aug 2026, 13:10" — a snapshot's age is the thing people actually judge
// uses, rather than adding a second way to ask the same question. // a backup by, and an ISO timestamp is not something anyone reads at a
// glance.
_restoreWhen(iso) {
if (!iso) return '';
const d = new Date(iso);
if (isNaN(d.getTime())) return '';
return this.escapeHtml(d.toLocaleString(undefined, {
day: 'numeric', month: 'short', year: 'numeric',
hour: '2-digit', minute: '2-digit'
}));
}
// Where each restored domain actually points. Checked one at a time through
// the endpoint the Domains step already uses, rather than adding a second
// way to ask the same question.
async _checkRestoreDomains(domains) {
const rows = []; const rows = [];
for (const domain of domains) { for (const domain of domains) {
let verdict = 'unknown', detail = 'could not check'; let verdict = 'unknown', detail = 'could not check';
@ -1207,15 +1262,18 @@ class SetupWizard {
if (j && j.matches) { verdict = 'ok'; detail = 'points at this server'; } if (j && j.matches) { verdict = 'ok'; detail = 'points at this server'; }
else if (j && j.domain_ip) { verdict = 'elsewhere'; detail = `points at ${j.domain_ip}, not this server`; } else if (j && j.domain_ip) { verdict = 'elsewhere'; detail = `points at ${j.domain_ip}, not this server`; }
else { verdict = 'unresolved'; detail = 'no DNS record found'; } else { verdict = 'unresolved'; detail = 'no DNS record found'; }
} catch { /* left as unknown — see below */ } } catch { /* left as unknown — never offered for deletion */ }
rows.push({ domain, verdict, detail }); rows.push({ domain, verdict, detail });
} }
this.restoreDomains = rows; this.restoreDomains = rows;
const strays = rows.filter(r => r.verdict === 'elsewhere' || r.verdict === 'unresolved');
const el = this.container.querySelector('#sw-rs-domains'); const el = this.container.querySelector('#sw-rs-domains');
if (!el) return; if (!el) return;
// Unverifiable is not the same as wrong, so only a domain that
// demonstrably resolves elsewhere is offered for removal.
const strays = rows.filter(r => r.verdict === 'elsewhere' || r.verdict === 'unresolved');
el.innerHTML = el.innerHTML =
`<p class="setup-section-hint">Domains it will bring across:</p>` +
rows.map(r => ` rows.map(r => `
<div class="setup-app-card"> <div class="setup-app-card">
<span class="setup-app-name">${this.escapeHtml(r.domain)}</span> <span class="setup-app-name">${this.escapeHtml(r.domain)}</span>
@ -1245,11 +1303,15 @@ class SetupWizard {
return; return;
} }
const apps = d.apps || []; const apps = d.apps || [];
const hasSys = !!(d.system && d.system.present);
box.innerHTML = ` box.innerHTML = `
<p class="setup-section-hint">This will, in order:</p> <p class="setup-section-hint">This will, in order:</p>
<div class="setup-app-card"><span class="setup-app-name">1. Settings and credentials</span> ${hasSys
<span class="setup-app-desc">Including every backup repository you had, so one password brings the rest back.</span></div> ? `<div class="setup-app-card"><span class="setup-app-name">1. The settings snapshot</span>
<div class="setup-app-card"><span class="setup-app-name">2. ${apps.length} app(s)</span> <span class="setup-app-desc">Logins, domains, and every backup repository with its credentials \u2014 first, because it is what makes the rest reachable.</span></div>`
: `<div class="setup-app-card"><span class="setup-app-name">1. No settings snapshot</span>
<span class="setup-app-desc">This repository holds app data only.</span></div>`}
<div class="setup-app-card"><span class="setup-app-name">2. ${apps.length} app snapshot${apps.length === 1 ? '' : 's'}</span>
<span class="setup-app-desc">${this.escapeHtml(apps.map(a => a.name).join(', '))}</span></div> <span class="setup-app-desc">${this.escapeHtml(apps.map(a => a.name).join(', '))}</span></div>
<p class="setup-section-hint"> <p class="setup-section-hint">
Apps this version no longer ships, or that will not fit, are skipped Apps this version no longer ships, or that will not fit, are skipped

33
docs/roadmap/first-run-restore.md
View File

@ -503,10 +503,35 @@ schedule, no enable toggle. The password leaves through the one-shot
never appears in the payload, since that payload reaches a task command line never appears in the payload, since that payload reaches a task command line
and tasks are recorded world-readable. and tasks are recorded world-readable.
**Contents** is §3's reconciliation, rendered. Apps with sizes, and the domains **Contents** is §3's reconciliation, rendered — and it has to make the
with a verdict each — checked through the same `/api/setup/dns-check` the repository's *shape* visible, which the first version did not.
Domains step uses rather than adding a second way to ask the question — plus
the offer to leave the strays out until DNS is repointed. A repository holds two different kinds of snapshot:
| | How many | Holds | Restored |
|---|---|---|---|
| `system=config` | **one** | the whole configs tree: logins, domains, and every backup repository with its credentials | first — it is what makes the others reachable |
| `app=<name>` | **one per app** | that app's data directory, with its own manifest | after, each independently |
They are separate because they are *used* separately. The settings tree is
small, changes rarely, and is meaningless per-app. App data is large, changes at
its own rate, and has to be restorable, movable and ageable **on its own** —
which is what the per-app tag buys: `restore app <name>` works, retention
applies per app, and an app can be placed on a different drive than it came
from.
The step listed "Apps" and "Domains" as peer sections, which hid all of that.
It read as though a backup contained three kinds of thing, and gave no clue
that the domains come *out of* the system snapshot. Now it shows **Settings**
(one snapshot, dated, with the domains nested under it and each domain's DNS
verdict) and **App data** (one snapshot each, dated and sized). A repository
with app data and no settings snapshot says so explicitly, because the
consequence — your repositories and logins do not come back — is not something
to discover afterwards.
Domain verdicts come from the same `/api/setup/dns-check` the Domains step uses
rather than adding a second way to ask the question, and the offer to leave the
strays out only ever covers domains that demonstrably resolve elsewhere.
**Rebuild** hands over to `restore rebuild`, which is the installer's order **Rebuild** hands over to `restore rebuild`, which is the installer's order
with the same reasoning: settings first (they carry every other repository's with the same reasoning: settings first (they carry every other repository's

37
scripts/dev/lp-restore-wizard-test
View File

@ -108,6 +108,35 @@ read -r -d '' DRIVE <<'JS'
// timeout only makes the test take a minute longer than it needs to. // timeout only makes the test take a minute longer than it needs to.
readPromise.catch(() => {}); readPromise.catch(() => {});
// The Contents step must present the two snapshot KINDS as two things. A
// repository holds one system=config snapshot and one per app, restored by
// different machinery; listing "Apps" and "Domains" as peers hid that, and
// hid that the domains come out of the system snapshot rather than being a
// third kind of thing in the backup.
w.restoreInfo = {
host: 'oldbox', hosts: ['oldbox'],
system: { present: true, date: '2026-08-28T13:10:02+01:00', domains: [] },
apps: [{ name: 'linkding', size: '1M', date: '2026-08-28T13:10:02+01:00' }]
};
await w.renderRestoreContents();
const contents = $('#sw-rs-contents').textContent.replace(/\s+/g, ' ');
out.showsSettingsSection = /Settings/.test(contents);
out.showsAppSection = /App data/.test(contents);
out.explainsSettingsFirst = /makes the others reachable/i.test(contents);
out.showsSnapshotDate = /28 Aug 2026/.test(contents);
// Domains belong under Settings, so with none there is no stray heading.
out.noDomainsHeadingWhenEmpty = !/Domains it will bring across/.test(contents);
// A repository with app data but no settings snapshot must say so: the
// user's repositories and logins will NOT come back, and finding that out
// afterwards is the worst possible time.
w.restoreInfo = { host: 'oldbox', hosts: ['oldbox'],
system: { present: false, date: '', domains: [] },
apps: [{ name: 'linkding', size: '1M', date: '' }] };
await w.renderRestoreContents();
const noSys = $('#sw-rs-contents').textContent.replace(/\s+/g, ' ');
out.warnsWhenNoSystemSnapshot = /no settings snapshot/i.test(noSys);
// submit() must route to the restore path, not the install payload. // submit() must route to the restore path, not the install payload.
let routedTo = null; let routedTo = null;
w.submitRestore = async () => { routedTo = 'restore'; }; w.submitRestore = async () => { routedTo = 'restore'; };
@ -151,6 +180,14 @@ chk "leaves the payload as a ref" "$(g .payloadCarriesRef)" true
chk "and never as a value" "$(g .payloadCarriesNoPassword)" true chk "and never as a value" "$(g .payloadCarriesNoPassword)" true
chk "and is cleared from the DOM" "$(g .passwordClearedFromDom)" true chk "and is cleared from the DOM" "$(g .passwordClearedFromDom)" true
echo "the contents step separates the two snapshot kinds"
chk "a Settings section" "$(g .showsSettingsSection)" true
chk "an App data section" "$(g .showsAppSection)" true
chk "says why settings come first" "$(g .explainsSettingsFirst)" true
chk "shows when each was taken" "$(g .showsSnapshotDate)" true
chk "no domain heading when there are none" "$(g .noDomainsHeadingWhenEmpty)" true
chk "warns when there is no settings snapshot" "$(g .warnsWhenNoSystemSnapshot)" true
echo "submit" echo "submit"
chk "routes to the restore path" "$(g .submitRoutedToRestore)" true chk "routes to the restore path" "$(g .submitRoutedToRestore)" true

133
scripts/restore/restore_inspect.sh
View File

@ -58,9 +58,24 @@ restoreInspectDomains()
# #
# restore inspect <location-idx> [host] # restore inspect <location-idx> [host]
# #
# A repository holds two DIFFERENT kinds of snapshot and the report keeps them
# apart, because they are restored by different machinery and mean different
# things to the person reading:
#
# system=config ONE snapshot of the whole configs tree — settings, logins,
# the domains, and every backup repository with its
# credentials. Restored first, because it is what makes the
# others reachable.
# app=<name> ONE SNAPSHOT PER APP of that app's data directory, each
# with its own manifest, its own size and its own schedule.
# Separate so an app can be restored, moved or aged out on
# its own without touching the rest.
#
# Showing them as one flat list was confusing precisely because it hid that.
#
# With no host it reports every host it found and picks the one with the most # With no host it reports every host it found and picks the one with the most
# apps as the suggestion — a repository that has been pointed at two machines # apps as the suggestion — a repository pointed at two machines is a normal
# is a normal thing to end up with, and guessing silently is not. # thing to end up with, and guessing silently is not.
restoreInspect() restoreInspect()
{ {
local idx="${1:-}" want_host="${2:-}" local idx="${1:-}" want_host="${2:-}"
@ -69,82 +84,94 @@ restoreInspect()
return 1 return 1
fi fi
local snaps local raw snaps
snaps=$(restoreFirstRunDiscover "$idx" 2>/dev/null) raw=$(restoreFirstRunDiscover "$idx" 2>/dev/null)
if [[ -z "$snaps" || "$snaps" == "null" ]]; then # The function can print notices before the JSON, so take the array itself
# rather than assuming the whole of stdout is the document.
snaps=$(printf '%s' "$raw" | sed -n '/^\[/,$p')
if [[ -z "$snaps" ]] || ! jq -e 'type == "array"' >/dev/null 2>&1 <<< "$snaps"; then
# The single most common cause by a distance, and worth saying plainly # The single most common cause by a distance, and worth saying plainly
# rather than as "discovery failed". # rather than as "discovery failed".
echo '{"error":"Could not read that repository — wrong password, or not a LibrePortal backup."}' echo '{"error":"Could not read that repository — wrong password, or not a LibrePortal backup."}'
return 1 return 1
fi fi
local -a hosts=() local hosts
local h hosts=$(jq -r '[.[].hostname] | unique | .[]' <<< "$snaps" 2>/dev/null)
while IFS= read -r h; do [[ -n "$h" ]] && hosts+=("$h"); done \ if [[ -z "$hosts" ]]; then
< <(printf '%s' "$snaps" | grep -o '"hostname":"[^"]*"' | cut -d'"' -f4 | sort -u)
if (( ${#hosts[@]} == 0 )); then
echo '{"error":"No LibrePortal backups found in that repository."}' echo '{"error":"No LibrePortal backups found in that repository."}'
return 1 return 1
fi fi
local host="$want_host" local host="$want_host" h
if [[ -z "$host" ]]; then if [[ -z "$host" ]]; then
# The one with the most apps, not simply the first: a repository often # The one with the most apps, not simply the first: a repository often
# carries a stray snapshot from a machine that was only ever tested. # carries a stray snapshot from a machine that was only ever tested.
local best="" best_n=-1 n local best="" best_n=-1 n
for h in "${hosts[@]}"; do while IFS= read -r h; do
n=$(migrateDiscoverApps "$h" "$idx" 2>/dev/null | grep -c .) n=$(jq -r --arg h "$h" \
'[.[] | select(.hostname == $h) | .tags[] | select(startswith("app="))] | unique | length' \
<<< "$snaps" 2>/dev/null)
[[ "$n" =~ ^[0-9]+$ ]] || n=0
if (( n > best_n )); then best_n=$n; best="$h"; fi if (( n > best_n )); then best_n=$n; best="$h"; fi
done done <<< "$hosts"
host="$best" host="$best"
fi fi
local -a apps=() # The system snapshot: newest one tagged system=config for this host.
local a local system_json
while IFS= read -r a; do [[ -n "$a" ]] && apps+=("$a"); done \ system_json=$(jq -c --arg h "$host" '
< <(migrateDiscoverApps "$host" "$idx" 2>/dev/null) [ .[] | select(.hostname == $h)
| select(any(.tags[]?; . == "system=config")) ]
| sort_by(.time)
| if length > 0 then {present: true, date: (.[-1].time)} else {present: false, date: ""} end
' <<< "$snaps" 2>/dev/null)
[[ -n "$system_json" ]] || system_json='{"present":false,"date":""}'
# --- assemble --- # One entry per app, newest snapshot of each.
local out='{' local apps_json
out+='"host":"'$(_lpJsonStr "$host")'",' apps_json=$(jq -c --arg h "$host" '
[ .[] | select(.hostname == $h)
| . as $s
| (.tags[]? | select(startswith("app=")) | ltrimstr("app=")) as $name
| {name: $name, time: $s.time} ]
| group_by(.name)
| map({name: .[0].name, date: (sort_by(.time) | .[-1].time)})
| sort_by(.name)
' <<< "$snaps" 2>/dev/null)
[[ -n "$apps_json" ]] || apps_json='[]'
out+='"hosts":[' # Sizes come from each app's own manifest. An older backup without one
local first=1 # still lists — it just has less to say about itself, which is better than
for h in "${hosts[@]}"; do # being left out of the list.
[[ $first -eq 0 ]] && out+=',' local sized='[]' a size_b size_h
out+='"'$(_lpJsonStr "$h")'"'; first=0 while IFS= read -r a; do
done [[ -z "$a" ]] && continue
out+='],'
out+='"apps":['
first=1
local size_b size_h
for a in "${apps[@]}"; do
[[ $first -eq 0 ]] && out+=','
# Size and date come from the app's own manifest where there is one.
# An older backup without a manifest still lists — it just has less to
# say about itself, which is better than being left out of the list.
size_b=$(restorePreflightManifest "$idx" "$a" "$host" 2>/dev/null \ size_b=$(restorePreflightManifest "$idx" "$a" "$host" 2>/dev/null \
| tr -d ' \n\t' | grep -o '"size_bytes":[0-9]*' | cut -d: -f2) | tr -d ' \n\t' | grep -o '"size_bytes":[0-9]*' | cut -d: -f2)
size_h="" size_h=""
[[ -n "$size_b" ]] && size_h=$(_restorePfSize "$size_b") [[ -n "$size_b" ]] && size_h=$(_restorePfSize "$size_b")
out+='{"name":"'$(_lpJsonStr "$a")'","size":"'$(_lpJsonStr "$size_h")'"}' sized=$(jq -c --arg n "$a" --arg s "$size_h" \
first=0 '. + [{name: $n, size: $s}]' <<< "$sized")
done done < <(jq -r '.[].name' <<< "$apps_json" 2>/dev/null)
out+='],'
out+='"domains":[' jq -nc \
first=1 --arg host "$host" \
local d --argjson hosts "$(jq -c '[.[].hostname] | unique' <<< "$snaps")" \
while IFS= read -r d; do --argjson system "$system_json" \
[[ -z "$d" ]] && continue --argjson apps "$apps_json" \
[[ $first -eq 0 ]] && out+=',' --argjson sizes "$sized" \
out+='"'$(_lpJsonStr "$d")'"'; first=0 --argjson domains "$(restoreInspectDomains "$idx" "$host" | jq -Rsc 'split("\n") | map(select(length > 0))')" \
done < <(restoreInspectDomains "$idx" "$host") '{
out+=']}' host: $host,
hosts: $hosts,
printf '%s\n' "$out" # Domains live under system because that is where they come from —
# the configs tree, in the one system=config snapshot. Presenting
# them as a peer of the app list is what made the step confusing.
system: ($system + {domains: $domains}),
apps: [ $apps[] as $a
| $a + {size: (([$sizes[] | select(.name == $a.name) | .size] | first) // "")} ]
}'
return 0 return 0
} }