The wizard's first question is now "is this a new server, or a replacement for
one?", which §2 of the roadmap described and nothing implemented. Start asks,
and the answer selects one of two disjoint step sets:
new Start > Experience > Identity > Domains > Storage > Backups
> Import > Recommended > (Metrics)
restore Start > Source > Contents > Rebuild
Disjoint deliberately. A restore is never asked for an install name, domains or
an app list — the backup answers all three, and asking invites someone to type
an answer that is about to be written over. The test asserts non-overlap in
both directions, not just that the restore steps appear.
Source collects the repository the way the Backup page does, minus everything
that only means something for a place you write TO: no retention, no schedule,
no enable toggle. The password leaves through the one-shot secret:<ref> channel
and is cleared from the DOM, and the test asserts the value never appears in
the payload — that payload reaches a task command line, and tasks are recorded
world-readable.
Contents is the reconciliation, rendered: apps with sizes, and each domain with
a verdict, checked through the same /api/setup/dns-check the Domains step uses
rather than adding a second way to ask. Plus the offer to leave the strays out
until DNS is repointed.
Rebuild runs `restore rebuild`: settings first (they carry every other
repository's credentials), then domains, then apps with no explicit list so
bulk discovers and re-preflights them itself.
Inserting Start shifted every step index by one. validateStep was a chain of
idx === 1 … idx === 6, carrying a comment that already explained which earlier
insertions had moved them — it is keyed on the step name now.
lp-storage-step-test had the same pin and did not survive: it called
validateStep(3) for Storage, which had become Domains, and reported that
nothing blocked. That reads exactly like validation being broken. Tests look
their step up by name now too.
Also: locationRemove's fix means a failed connect can finally clean up after
itself, so a wrong password no longer leaves a dead destination behind on every
retry.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
restore inspect answers "what would a restore from here bring?" without writing
anything: hosts, apps with sizes, and the domains — read out of the
system-config snapshot with engineDumpFile, the same way the preflight reads an
app manifest. Knowing a backup hands you six domains of which four point
elsewhere, before committing, is the difference between a rebuild and a
surprise.
restore connect is the WebUI entry point: creates the location from a base64
payload, redeems the repository password from the single-use secret channel,
inspects. Deliberately does not engineInitLocation — every other path that
creates a location initialises it because it is about to write there; this one
reads a repository that already exists. This is what unblocks the constraint
app_portable.sh records: a .lpapp could live in the WebUI because nothing
secret crosses from browser to host, and the repository restore could not. The
secret:<ref> channel is that missing piece.
A wrong password is the ordinary case and the user retries, so a failed connect
removes the location it just made. Otherwise every attempt left another
half-configured destination behind.
Three things found by using it:
- locationRemove never worked. It unlinked as the container user, but
configs/ is manager-owned, so it was always denied — and the result was
never checked, so isSuccessful printed anyway and a "removed" location came
back on the next listing. Now runInstallOp, and the directory is checked.
- webuiSecretSweep had no callers. An abandoned flow left its repository
password on disk forever. The sweep now runs in /api/setup/secret before
each write, tied to the one event guaranteed to happen.
- Adoption took the WebUI down. config-adopt chowned every adopted file to
manager:manager 0640, and webui_logins is bind-mounted into the container,
which then could not read its own credentials: exit 137 with no log line.
It also clamped every parent directory it passed through, closing
configs/webui and configs/backup to the container user.
The fix is a principle, not a special case: a restore replaces the CONTENT
of a config file and nothing else. The live install already knows who may
read each one. Adoption preserves the destination's ownership and mode,
defaults closed only for a file that did not exist, and never
re-permissions a directory it passes through.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The installer's restore path printed "Settings restored" and had never restored
a setting. backupRestoreSystemConfig only STAGES — right in general, since
overwriting a running control plane's config should not be automatic — but
nothing ever adopted the staged tree. The backup locations, domains and logins
landed in $restore_dir/system-config and stayed there.
So adoption is its own step now (`restore adopt`), allowed only on a machine
with nothing on it yet. backup/locations/ is adopted as a subtree, since the
index is part of the path and that directory is the whole point: it holds every
repository and its credentials, which is what makes "one password unlocks the
rest" true. Deliberately NOT adopted: the container account and its generated
password, port allocations, docker/rootless wiring, and storage/locations —
those describe the old machine, and a registry of drives this box does not have
would make every placement lookup resolve to a phantom.
The guard failed in the shape this project keeps hitting. It globbed the
containers directory, but the manager can traverse that without listing it, so
the glob returned a literal '*', the loop skipped it, and the function returned
"first run" on a machine with three apps. It adopted over a live install in
testing. It now asks the container user for the listing and fails closed: an
unreadable directory means "in use", never "empty".
Two config modes were inverted, found because a restore cannot restore from a
snapshot that was never taken:
- storage location configs were 0640 and hold no secrets. The backup runs as
the container user, could not read them, and restic wrote an INCOMPLETE
snapshot and exited 3 — so EVERY system-config backup failed once a second
storage location existed. Now 0644, with the test asserting they stay
secret-free so that mode remains defensible.
- backup location configs were 0644 and hold the repository password; nobody
could read them. They cannot simply be tightened, because the backup has to
read the credentials it uses — so the directory carries the restriction
(config-secure, manager:container 0750) and the file stays readable to the
two accounts that belong.
Fixing that surfaced a third: config-adopt clamped existing parent directories
to manager:manager 0750, closing configs/backup to the container user and
breaking the credential read the directory fix had just preserved.
restore domains reports which restored domains point here, and the installer
offers to drop the strays. Three verdicts, not two: setupCheckDomainPointsHere
falls back to hostname -I, and comparing a public A record to a private 10.x
address would condemn every correctly-pointed domain on a LAN-only box, which
is the deployment this product targets. Unverifiable is never offered for
deletion.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An end-to-end run restored 4 of 13 apps and reported
"First-run restore complete — 4 apps restored" as a success.
Two truncations, and fixing the first had hidden the second:
* the CLI dispatcher calls handlers with no arguments, so "$@"/shift inside
one operate on an empty list. Fixed earlier with LP_CLI_ARGS.
* LP_CLI_ARGS was built from start.sh's "$@" — but the root wrapper invokes
start.sh with exactly nine hardcoded positional slots. So the array could
never hold more than nine entries, and `${LP_CLI_ARGS[@]:5}` yielded at
most four app names.
The wrapper now forwards the real argv after those nine slots (they stay
untouched: every dispatcher reads them, and unset ones must keep arriving as
the literal "empty"), and start.sh reads it back as "${@:10}". Verified: a
preflight given six apps checks six, where five was the previous ceiling.
footprint_version 7 -> 8, since the wrapper is root-owned and baked at install.
Two further fixes so a truncation cannot pass as success again:
* restoreFirstRunBulk with no app list is now a whole-host restore — it
discovers the host's apps and re-applies the preflight. The installer's
report runs in its own process, so without this an app the user was told
would be skipped got restored anyway. init.sh now passes no list, so a
whole-host restore builds nothing that can be truncated.
* it counts what actually landed and returns non-zero naming the failures,
instead of reporting the length of the list it was handed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`restore preflight <loc_idx> <host>` reads every app's manifest out of its
own snapshot (engineDumpFile pulls a single file without restoring) and
checks it against this machine before anything is written:
* an app this version no longer ships is skipped — restoring one
produces a directory that can never start, and looks like success
until someone opens it
* an app that will not fit is skipped individually, because filling the
disk part-way through takes the apps that already landed with it
* a manifest naming a storage location this machine lacks falls back to
the default, and says which app moved where
The installer's restore path runs it and asks once before continuing.
Two bugs found by running it against the live repository rather than
reading it:
The CLI dispatcher calls handlers with NO arguments, so `shift 4; "$@"`
inside one operates on an empty list. `restore first-run bulk` has always
had this — a bulk restore silently received zero apps. Fixed at the entry
point: start.sh now captures LP_CLI_ARGS from "$@", and both call sites
use it.
And the wrapper fills unset argv slots with the literal string "empty"
(${5:-empty} … ${9:-empty}), so a trailing slot arrives as a five-
character app name rather than a blank. Filtering on -n alone let five
phantom apps through and reported each as "no longer shipped". Both call
sites now drop the sentinel. That also caps any explicit list at five, so
preflight discovers the host's apps itself when given none.
Verified against the live repository: 13 apps discovered and checked.
Sizes read "?" there because those snapshots predate manifests carrying
size_bytes — the intended graceful fallback, not a refusal.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Phase 0 of the migration-system refresh. Replaces the 77-line
scripts/migrate/ with a properly-shaped kernel that Phase 1 (WebUI) and
Phase 3 (direct peer SSH) can both build on.
New module layout (6 files):
migrate_progress.sh — migrateEmit JSON-per-line helper; opt-in via
MIGRATE_JSON_PROGRESS=1, writes to fd 3 if open
(clean WebUI streaming channel) else stdout.
migrate_discover.sh — migrateDiscoverHosts / migrateDiscoverApps /
migrateDiscoverAppDetail (JSON {snapshots, latest_*}).
Old migrateDiscoverAppsForHost kept as back-compat.
migrate_preflight.sh — migratePreflight emits one JSON object with
snapshot{id,date}, destination{installed,running,
disk_free_kb}, collision{occurs,default_action,
pre_backup_default}, url_rewrite{default_action,
per_app_opt_out}, warnings[], errors[].
Exit 0 on usable preflight, 1 on hard error.
migrate_url_rewrite.sh— Host-bound CFG_<APP>_* fields (URL/HOST/DOMAIN/
DOMAIN_PREFIX/HOSTNAME/PUBLIC_URL) get rewritten
from the destination's install-template after
restore — so a moved app stops claiming the
source's hostnames. Per-app opt-out via
CFG_<APP>_MIGRATE_URL_REWRITE=false. All other
fields (DB passwords, API keys, prefs) carry
over from the source unchanged.
migrate_pre_backup.sh — migratePreBackupDestination takes a snapshot of
the destination's existing <app> (tagged
pre-migrate=<UTC timestamp>) before the wipe.
Default ON; opt-out with --no-pre-backup. Safety
net for the always-replace collision policy.
migrate_apply.sh — migrateApplyApp / migrateApplySystem. Parses
--no-pre-backup / --keep-urls / --json-progress
opts, runs preflight → pre-backup → restoreAppStart
(existing flow) → URL rewrite → re-deploy compose.
migrateApp / migrateSystem kept as shims so the
old CLI surface still works.
CLI dispatcher (cli_restore_commands.sh + cli_restore_header.sh):
Existing 'restore migrate app/system/discover' calls all still work.
New verbs:
restore migrate list <host> [loc_idx]
restore migrate preflight <host> <app> [loc_idx] ← JSON, for the WebUI
Design choices baked in (per the spec):
- Always-replace collision (no multi-install of an app), safety net is the
on-by-default pre-migrate backup.
- URL rewrite by host-bound suffix list, not per-field allowlist — works
out-of-the-box for new apps without extra config.
- migrateEmit fd-3 contract is what Phase 1's WebUI will stream; falls
back to stdout in interactive CLI so dev/debug just works.
- Transport-agnostic: nothing in this kernel knows whether the backup
location is local/SSH/S3/Connect — engineSnapshotsJson + engineBackupApp
do that, so Connect (the future blind-relay) plugs in as 'just another
location kind' with zero kernel changes.
Smoke-tested: all 13 public functions register; JSON emit produces correct
escaping (quoted strings vs bare numerics) and respects MIGRATE_JSON_PROGRESS.
Signed-off-by: librelad <librelad@digitalangels.vip>
Expose the system-config backup on demand (not just within 'backup all'):
- `libreportal backup system` -> backupSystemConfig (snapshot the system
config — settings, WebUI creds, backup-location creds — to all enabled locations)
- `libreportal restore system [loc_idx]` -> backupRestoreSystemConfig (restore the
latest system snapshot into a staging dir; never overwrites live config)
Distinct from the existing 'restore migrate system' (which restores all *apps*
from another host). Help text updated for both. Routing verified with stubs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
A free, open, self-hosted app platform (GNU AGPLv3): one-click app deploys,
Traefik reverse proxy with automatic SSL, rootless Docker support, gluetun
VPN routing, and a web dashboard to manage it all.
Free & open forever to self-host; optional paid hosted services fund it.
See PROMISE.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
librelad