LibrePortal/docs/roadmap/first-run-restore.md

LibrePortal — First-run: New Install or Restore (Roadmap / Proposal)

Status: Phases 1–4 built. · Audience: us, future-self · Scope: make "I'm rebuilding my server" a first-class path at first run, not a CLI expedition · Origin: "on the first install/setup we need 2 option blocks (New Install and Restore from Backup)" (2026-08-27)

---

0. The one idea

The wizard's first question becomes "is this a new server, or a replacement for one?" Everything else follows from the answer.

Today a rebuild means knowing that libreportal backup location add, libreportal restore first-run discover and libreportal restore first-run bulk exist, in that order. That is a fine CLI story and a terrible first-run story — and rebuilding after a disk dies is exactly when someone is least able to go reading docs.

1. How much of this already exists

More than it looks. The restore path is built; what is missing is the front door.

Piece	Status
Connect a backup repo (local / sftp / rest / s3 / b2 / gs / azure / rclone)	✅ locationAdd
Read a repo without touching live state	✅ restoreFirstRunDiscover, migrateDiscoverHosts/Apps/AppDetail
Restore the system config (settings + credentials) into staging	✅ backupRestoreSystemConfig, restore system
Restore many apps in one go from another host	✅ restoreFirstRunBulk, restore first-run bulk
Rewrite host-bound CFG_* (URL/HOST/DOMAIN) to this machine	✅ migrateUrlRewrite
Preflight a migrate before committing	✅ migratePreflight
Safety snapshot of the destination before overwriting	✅ migratePreBackupDestination
Restore an app whose path differs from this host's	✅ storageRestoreAppTo (§9 of storage-locations)
Snapshot records where the app lived	✅ manifest storage block
The wizard branch, and the reconciliation screen	❌ this document

So this is mostly assembly plus one genuinely new screen, not new plumbing.

2. What the user sees

Step 0 becomes two blocks, before Experience:

┌───────────────────────────┐   ┌───────────────────────────┐
│  🌱  New install          │   │  ♻️  Restore from backup   │
│  Set this machine up      │   │  Rebuild a server from    │
│  from scratch.            │   │  an existing backup.      │
└───────────────────────────┘   └───────────────────────────┘

New install → the wizard exactly as it is now.

Restore → Identity is skipped (it comes from the backup), and the flow becomes:

1. Where is your backup? — the existing backup-location fields, rendered from config metadata like the Locations page already does.
2. Unlock it — the repository password. See §4; this is the step that decides whether a restore is possible at all.
3. What's in there? — hosts found, then apps per host, with sizes and snapshot dates.
4. What will change on this machine — the reconciliation screen. §3. The new part.
5. Restore — system config first, then the chosen apps, with the task list the wizard already renders.

3. Reconciliation — a preflight report, not a screen

Originally written as a WebUI screen. The restore runs in the installer (§2), so it becomes a report printed before anything is written, plus automatic resolution where the answer is not a judgement call.

A backup describes a machine that no longer exists. The failure mode worth designing against is re-deciding those facts silently.

It is answerable before restoring because every app's snapshot carries its own .libreportal-manifest.json, and engineDumpFile can pull a single file out of a snapshot without restoring it. So the installer can read all 13 manifests, compare them to this machine, and print a verdict per app.

What differs	Detected from	Resolution
App no longer shipped by this version	template missing under install/containers/	skip, and say so — restoring an app whose template is gone produces an unstartable directory
Data larger than the target drive	manifest size_bytes vs df	skip that app, not the whole restore
Storage location named in the manifest doesn't exist here	manifest storage.location vs the registry	fall back to the default location, and say which app moved where
Old absolute path ≠ where it goes here	manifest storage.path	already handled — storageRestoreAppTo stages and moves
Domains point at the old host	—	already handled — migrateUrlRewrite
Ports / IPs already taken	port allocation tables	already handled by the install pipeline; reallocated on start

Output is a verdict list, then one confirmation:

  Checking 13 apps against this machine…

    ✓ bookstack     2.1G   restores as-is
    ✓ nextcloud     44G    -> bigdisk (its old location "ssd" is not on this machine)
    ✗ obsolete-app  120M   skipped — this version no longer ships it
    ✗ jellyfin      1.2T   skipped — needs 1.2T, /mnt/data has 400G free

  11 will restore, 2 skipped.
  Continue? [Y/n]:

3.1 — Two silent no-ops, and what they have in common

Both were found by running the installer's restore path end to end against a real repository, not by reading the code — which is the point of doing that.

1. The manifest is pretty-printed. "size_bytes": 123 carries a space, so a "key":[0-9]* pattern missed it. Size came back empty, and the fit check is guarded by [[ -n "$size_bytes" ]] — so it was skipped for every app.
2. The manifest was never read at all. storageSnapshotSourcePath passed a snapshot id into an app-tag filter (see storage-locations §9), so it returned 1 every time and the preflight fell back to a ? size.

The shared shape: a check whose failure mode is to not run. Neither printed an error, and the report they produced — thirteen green ticks — is exactly what a healthy run looks like. The only visible tell was the ? in a column nobody had a reason to distrust.

So the regression test (scripts/dev/lp-preflight-test) asserts the negative cases: an app too big for the disk must be refused, an app this version no longer ships must be refused, an app whose location is gone must be marked as moved. Reintroducing either historical bug fails it.

3.2 — And a third: four apps of thirteen, reported as success

With the preflight finally reading manifests, the run got as far as restoring — and restored 4 of 13, printing "First-run restore complete — 4 apps restored" with exit 0.

The app list crosses two chokepoints, and fixing the first had hidden the second:

1. the CLI dispatcher calls handlers with no arguments, so "$@" and shift inside one operate on an empty list. LP_CLI_ARGS was added for this.
2. but LP_CLI_ARGS was built from start.sh's "$@" — and the root wrapper invokes start.sh with exactly nine hardcoded positional slots. The array could never hold more than nine entries, so ${LP_CLI_ARGS[@]:5} gave at most four app names.

Same shape as §3.1 once more: the truncation had no failure path. Four apps restored perfectly, and the success line counted the list it was handed.

Three changes, because one would not have been enough:

• the wrapper forwards the real argv after the nine slots (which stay, since every dispatcher reads them and unset ones must keep arriving as "empty"); start.sh reads it back as "${@:10}". footprint_version 7 → 8.
• restoreFirstRunBulk with no list is a whole-host restore: it discovers the host's apps and re-applies the preflight itself, because the installer's report ran in a different process and its decision was otherwise lost — a skipped app would have been restored anyway. The installer now passes no list, so there is nothing to truncate.
• it counts what actually landed and returns non-zero naming the failures.

scripts/dev/lp-cli-argv-test builds stubs from the real lines in init.sh and start.sh and pushes thirteen app names through them.

3.3 — Staging: created by one principal, written by another

The worst of the set, because it sat on the step everything else depends on.

restore system printed

  ✓ Success System config restored to: /libreportal-system/restore/system-config

for a directory that did not exist. Nothing had been written.

Restore stages through $SYSTEM_DIR, which the manager owns — but the thing writing into the staging tree is restic, and runBackupOp runs it as the container user. Both call sites created the directory as the wrong principal, in opposite directions:

Call site	Created by	Written by	Result
backupRestoreSystemConfig	runFileOp → container user	container user	mkdir denied on the 0751 manager-owned restore_dir, unchecked
storageRestoreAppTo	runInstallOp → manager	container user	restic could not create anything beneath it

And restic reports a permission denial as ignoring error … and still exits 0, so the caller's success check was satisfied either way. Same family as §3.1: the check had no failure path.

Root has to bridge that, the way webui-bind already bridges the mirror case. libreportal-ownership gains restore-stage <path> (creates it cowner:MANAGER 0750 — owner writes, manager traverses to confirm and review) and restore-unstage <path> (removes it: neither principal can, since the manager cannot delete the container user's files inside and the container user cannot unlink the entry from the manager's directory, so staging simply accumulated). Both confine the path to a single component directly under the restore/migrate area. footprint_version 8 → 9.

backupRestoreSystemConfig now also verifies the tree landed as the user that wrote it, since the manager cannot read inside its own staging directory.

3.4 — What the run finally proved

With all of the above fixed, on a live machine:

• 13 of 13 apps restored from the repository and came up healthy, databases included (bookstack, matrix ×3, mattermost ×2, nextcloud ×4, rocketchat ×2, stoat ×10, …)
• restore system staged 57 real files
• the relocation branch of storageRestoreAppTo ran for the first time: speedtest, moved to a second storage location, restored from a snapshot taken at /libreportal-containers/speedtest into /libreportal-alt/speedtest via stage-and-move, with the staging tree cleaned up afterwards

One genuine environment collision remains and is not a LibrePortal defect: stoat's livekit publishes a fixed UDP range (50000–50100) that it advertises to clients and so cannot be re-rolled, and a desktop's kdeconnectd held 50016. Worth a fixed-range preflight check of its own; see §9.

3.5 — Ownership was not actually being reinstated

grafana restored and then died with "attempt to write a readonly database", repeatedly. The snapshot records:

-rw-r----- 231543 1002  /libreportal-containers/grafana/grafana_storage/grafana.db

and what landed was 1002:1002. The owner was lost, so grafana — running as 231543 — could not write its own database at mode 0640.

Restore runs as the backup user, which has no CAP_CHOWN, so it reinstates ownership through a user namespace. The prefix was:

unshare --map-root-user --map-users=231072:231072:65536 --map-groups=231072:231072:65536

and unshare accepts one range per option. So the backup user's own GID was never mapped — while LibrePortal writes app data as <container-uid>:<backup-user>. The group half of every such chown referred to an unmapped id, lchown returned EINVAL, and the file kept the restoring user's ownership. Measured directly:

target	old prefix	now
231543:1002 (app data)	❌ → 1002:1002	✅ 231543:1002
1002:1002 (LibrePortal's own)	❌ → 1002:1002 (right anyway)	❌ → 1002:1002 (right anyway)
231072:231072 (container root)	✅	✅

Same shape as the rest of §3: restic reports the misses as ignoring error … and still exits 0, and the caller forgave them as "expected, they are already owned correctly". 1626 of one 13-app restore's 2086 failed chowns were grafana's.

Fixed with scripts/backup/engine/restic-userns-exec, which uses newuidmap/newgidmap — those write multi-range maps, which is exactly what unshare cannot express:

uid:  0       <- caller        inner root, or capabilities are dropped at exec
      SUB..   <- SUB..         identity, so restic can name the stored uid
gid:  caller  <- caller        identity — the group half of app-data chowns
      SUB..   <- SUB..         identity

The caller's own UID is deliberately not identity-mapped: that slot is spent on inner root. A file stored as <caller>:<caller> therefore fails its chown and lands owned by the caller anyway, because that is who inner root is on the outside — so the one case this cannot map is the one case that needs no mapping.

unshare --map-auto --map-current-user does not work as a shortcut: it maps the subuid range to low inner ids (container-style, 0 → 231072) while restic needs identity. Tested.

Measured on the live install, restoring grafana: failed chowns 1626 → 12 (the 12 being the caller's own files, which are correct), grafana.db back to 231543:1002, and grafana up and writing. scripts/dev/lp-userns-ownership-test pins all three rows of the table above.

Anything unexpected — no subuid range, no newuidmap, a namespace that will not start — falls back to running the command plainly, which is what happened before any of this existed.

3.6 — Open: a fresh install generates new secrets, the backup carries the old ones

Found by restoring a real 13-app repository onto a purpose-built install (storage-locations §12.5, case 4). 11 of 13 apps came back working. The two that did not failed the same way for two different reasons, and the reason is structural rather than a bug in either app.

A first-run restore generates fresh credentials, then restores state that encodes the old ones. Install randomises every CFG_*_PASSWORD; the backup contains a machine that used different ones. Wherever a secret is written into something the restore then lays down, the two disagree:

app	what disagreed	result
matrix	the install hook writes homeserver.yaml from CFG_MATRIX_DB_PASSWORD_1 at step 3, and step 6 restores the snapshot over it	password authentication failed for user "synapse" — postgres was initialised with the new secret, the restored config file holds the old one
stoat	data/rabbit/mnesia restores raw, and rabbit only honours RABBITMQ_DEFAULT_PASS on an empty data dir	ACCESS_REFUSED — four dependent services exit 101

Apps with a dump descriptor are unaffected, and that is the tell: for those, restoreDbRehydratePreStart clears the data dir so the engine initialises with the current secret, then replays the dump into it. Everything else — a config file inside the app dir, a broker's own user database — keeps the old value.

Three ways out:

1. Apply the restored system config on first run instead of only staging it. The backup's configs/ holds the original secrets, so adopting them makes every restored app consistent by construction. backupRestoreSystemConfig deliberately stages rather than applies — "recovering creds/settings is a review-then-copy step, never an automatic blast over a running control plane" — and that is right for a running box and wrong for a fresh one. The code currently cannot tell those two situations apart. This is §9.1 restated with a concrete failure behind it, and it carries a real consequence: it also restores the WebUI login, so the user signs in to the new machine with the old password. Probably expected; must be said out loud.
2. Re-run app config generation after the data restore. Fixes matrix. Does nothing for stoat, where the secret lives inside restored service state rather than a generated file — so it is not sufficient on its own.
3. Reset the credential in the restored service (ALTER ROLE, rabbitmqctl change_password). Per-engine, fragile, and needs a hook per backing service.

Recommendation: (1), gated on first run specifically — the flag already exists as init_mode=restore. (2) is a reasonable belt-and-braces addition; (3) is a last resort for services whose state cannot be re-initialised.

Not implemented here: it changes what a restore does with credentials, which is a decision rather than a defect.

3.7 — The restore that restored nothing

Phases 1–4 were marked built, and the installer's restore path ran end to end and printed "Settings restored". It had never restored a setting.

backupRestoreSystemConfig only stages. That is deliberate and correct in general — overwriting the config of a running control plane is not something to do automatically — but nothing in the tree ever adopted the staged result. The backup locations, the domains, the logins all landed in $restore_dir/system-config and stayed there, and the one line of output a person actually reads said the opposite.

So adoption is now its own step (restore adopt), allowed only where the original caution does not apply: a machine with nothing on it yet.

What is not adopted matters as much as what is. A backup describes a machine that no longer exists, and some of what it says is about that machine rather than about the user: general_docker_install (the container account and its generated password, made by this install), network_ports (re-rolled per install), network_docker / network_rootless (this box's hardware and kernel), and storage/locations — the old machine's drives. App placement is already reconciled per app from the snapshot manifests (§3); adopting a registry of drives this box does not have would make every one of those lookups resolve to a phantom.

backup/locations/ is adopted, as a subtree rather than a filename, since the index is part of the path. That one is the whole point: it holds every repository and its credentials, and "one password you remember unlocks the rest" is the promise a first-run restore makes.

The guard, and the shape it failed in

Adoption overwrites live config, so the first-run check is the only thing between "restore onto a blank box" and "overwrite a working install".

The first version globbed the containers directory directly. The manager can traverse that directory without being able to list it, so the glob came back as the literal *, the loop skipped it as not-a-directory, and the function fell out of the bottom returning "yes, first run" — on a machine with three apps on it. It adopted over a live install in testing before the guard was rewritten to fail closed, asking the container user for the listing and treating an unreadable directory as "in use" rather than "empty".

That is the same shape as §3.1's silent no-ops, and it is worth naming again: a check whose failure mode is to not run is indistinguishable from a check that passed.

3.8 — Two config modes, inverted

Found while testing the above, because a first-run restore cannot restore from a system snapshot that was never successfully taken.

Storage location configs were 0640. They hold a name, a path and free text — nothing secret. The backup runs as the container user, which could not read them, so restic reported permission denied, wrote an INCOMPLETE snapshot and exited 3. Every system-config backup failed the moment a second storage location existed. Now 0644, and the test asserts they contain no secrets so that mode stays defensible.

Backup location configs were 0644. They hold CFG_BACKUP_LOC_<n>_PASSWORD — the key to every backup the user has — and nobody could demonstrably read them. The mode cannot simply be tightened: the backup genuinely has to read the credentials it is about to use. So the directory carries the restriction instead (config-secure: manager:container, 0750). Both accounts that belong still get in; nothing else can traverse, whatever the modes inside say.

A third bug came out of fixing that one. config-adopt created missing parent directories and clamped them to manager:manager 0750 — including ones that already existed, which closed configs/backup to the container user and broke the very credential read the directory fix had just preserved. A copy has no business re-permissioning directories it merely passes through.

3.9 — Domain reconciliation

The adopted config carries the domains the old machine served, and DNS still points wherever it pointed. Nothing checked this, so the first sign of trouble was Traefik failing to get a certificate long after the installer said it had finished.

restore domains reports one verdict per domain, and the installer offers to drop the strays. It is a report, never a refusal: a domain that does not resolve here is a perfectly normal state five minutes into a rebuild.

Three verdicts, not two. setupCheckDomainPointsHere falls back to hostname -I when the public-IP lookup fails, which is fine for its own purposes and wrong here — comparing a public A record against a private 10.x address makes every correctly-pointed domain look misconfigured, and this is explicitly a LAN/VPN-first product where that lookup failing is ordinary. So an unverifiable domain reports unknown and is never offered for deletion; only a domain that demonstrably resolves elsewhere is.

Two parsing bugs worth recording, both caught by running it rather than reading it. Config values carry a trailing comment column, so every domain arrived with an essay attached and no lookup could match. And updateConfigOption writes an empty value as a literal "", so nine cleared slots read back as nine two-character domains and were reported as nine failures.

3.10 — Reading a repository without restoring it

restore inspect <idx> [host] answers "what would a restore from here bring?" without writing anything: which machines' backups are in the repository, which apps and how big, and — the part that is least obvious to get at — which domains. Those live in the system-config snapshot, so engineDumpFile pulls network/network_domains straight out of it, the same way the preflight pulls an app's manifest. Knowing "this backup hands you six domains, four of which point somewhere else" before committing is the difference between a rebuild and a surprise.

restore connect <base64-json> is the WebUI's entry point: it creates the location from a payload, redeems the repository password from the single-use secret channel, and inspects. It deliberately does not call engineInitLocation — every other path that creates a location initialises it because it is about to write there; this one is pointed at a repository that already exists and is only going to be read.

This is what unblocks §4. app_portable.sh records that a .lpapp can live in the WebUI because it is unencrypted and no password has to cross from the browser to the host — and that the repository restore therefore could not. The secret:<ref> channel is that missing piece, so the constraint no longer holds.

A wrong password is the ordinary case here and the user simply tries again, so a failed connect removes the location it just created. Without that, every retry left another half-configured destination behind and the Backup page grew a column of identical dead entries.

3.11 — Three more found by using it

locationRemove never worked. It removed with runFileOp — the container user — but configs/ is manager-owned, so the unlink was always denied. The result was never checked, and isSuccessful printed regardless, so a location "removed" from the WebUI came straight back on the next listing. Now runInstallOp, and the directory is checked before claiming anything.

webuiSecretSweep had no callers. Written for exactly this and never wired in, so a flow the user abandoned — closed the tab, hit a validation error, never pressed Save — left its repository password on disk indefinitely. The sweep now runs in the /api/setup/secret route before each write, which ties it to the one event guaranteed to happen whenever secrets are being made.

Adoption took the WebUI down. config-adopt chowned every adopted file to manager:manager 0640. webui_logins is bind-mounted into the WebUI container, which then could not read its own credentials file: the container died with exit 137 and no log line at all, which is a genuinely hard failure to read. It also clamped every parent directory it passed through, closing configs/webui and configs/backup to the container user.

The fix is a principle rather than a special case: a restore replaces the content of a config file and nothing else. The live install already knows who is allowed to read each one. Adoption preserves the destination's existing ownership and mode and never re-permissions a directory it merely passes through.

For a file this install did not have, the first attempt picked 0640 — "these can hold secrets, so default closed". That is the storage-location bug again, one directory over: the config tree is 0755, the backup account reads all of it, and a single 0640 file makes restic write an INCOMPLETE snapshot and report the whole run as failed. A new file now inherits from a sibling in the same directory, so it matches whatever the tree's convention is rather than having a mode chosen for it.

That is three occurrences of the same defect in three directories for three different reasons, so the test now asserts the class: no file anywhere under configs/ may be unreadable by the backup account. That one line would have caught all three.

3.12 — The WebUI branch

§2 described the wizard's first question becoming "is this a new server, or a replacement for one?", and it now is. Start asks, and the answer selects one of two disjoint step sets:

  new       Start -> Experience -> Identity -> Domains -> Storage
                  -> Backups -> Import -> Recommended -> (Metrics)
  restore   Start -> Backup -> Contents -> Rebuild

Disjoint on purpose. A restore is never asked for an install name, domains or an app list, because the backup answers all three — asking would invite someone to type an answer that is about to be written over, which is worse than not asking. The test asserts non-overlap in both directions rather than only that the restore steps appear.

Backup collects the repository the same way the Backup page does, minus everything that only means something for a place you WRITE to: no retention, no schedule, no enable toggle. The password leaves through the one-shot secret:<ref> channel and is cleared from the DOM; the test asserts the value never appears in the payload, since that payload reaches a task command line and tasks are recorded world-readable.

Contents is §3's reconciliation, rendered — and it has to make the repository's shape visible, which the first version did not.

A repository holds two different kinds of snapshot:

	How many	Holds	Restored
system=config	one	the whole configs tree: logins, domains, and every backup repository with its credentials	first — it is what makes the others reachable
app=<name>	one per app	that app's data directory, with its own manifest	after, each independently

They are separate because they are used separately. The settings tree is small, changes rarely, and is meaningless per-app. App data is large, changes at its own rate, and has to be restorable, movable and ageable on its own — which is what the per-app tag buys: restore app <name> works, retention applies per app, and an app can be placed on a different drive than it came from.

The step listed "Apps" and "Domains" as peer sections, which hid all of that. It read as though a backup contained three kinds of thing, and gave no clue that the domains come out of the system snapshot. Now it shows Settings (one snapshot, dated, with the domains nested under it and each domain's DNS verdict) and App data (one snapshot each, dated and sized). A repository with app data and no settings snapshot says so explicitly, because the consequence — your repositories and logins do not come back — is not something to discover afterwards.

Domain verdicts come from the same /api/setup/dns-check the Domains step uses rather than adding a second way to ask the question, and the offer to leave the strays out only ever covers domains that demonstrably resolve elsewhere.

Rebuild hands over to restore rebuild, which is the installer's order with the same reasoning: settings first (they carry every other repository's credentials), then domains, then apps with no explicit list so bulk discovers and re-preflights them itself.

Finding the backup, before asking for the password

The step opened with an empty path box and /mnt/usb/libreportal-backups as the placeholder — a path nobody has, shown as the shape of the answer. Someone rebuilding a server is being asked to recall from memory the one thing they came here because they had lost.

Two additions, and the point of both is that neither needs the repository password. A restic repository keeps one file per snapshot under snapshots/, so "is there a backup here, and how many" is a directory listing. Nothing is decrypted; reading what is in those snapshots is the next step, and that does need the password.

• restore scan looks in the places a backup actually is: this install's own backups root (the disk often survives), every backup location the install already knows about, and one level under each non-OS mount — a drive just plugged in being the other half of "the system drive died". Bounded to named shapes and maxdepth 1, never a filesystem walk: a scan nobody waits for is a scan nobody uses. Results are offered as buttons, most snapshots first, each showing its count and the age of its newest snapshot.

• restore verify <path> answers the same question for a typed path. Its most useful answer is the near-miss: pointing at the folder that contains the repositories rather than at one of them is the common mistake, and it says so and offers the real path as a button rather than explaining the distinction in a paragraph.

A repository is recognised by config plus the snapshots, keys and data directories together. config alone would match any folder that happens to contain a file of that name, and offering a stray directory as someone's backup is worse than not finding it.

The placeholder now comes from this machine — the first repository found, or the install's own backups root — because a placeholder's whole job is to show the shape of the answer, and only a real one does that.

Both the results and the verdict live under the Folder field, inside its box: they are answers about that input, and floating them above the whole form made them look like a separate step. What is shown depends on how many were found, because those are genuinely different situations:

Found	What happens
one	it is the answer, not a choice — filled in, with its verdict. No card: the card and the verdict said the same thing twice
several	listed as buttons, most snapshots first. The field stays empty — this is genuinely ambiguous and guessing would be worse
none	says so. "We looked and there is nothing here" is information; empty space is not

The first version filled in only the placeholder, which left the field empty — so pressing Check replied "give a full path, starting with /" about the very backup displayed directly above it.

Choosing which snapshot — and what a snapshot actually is

Worth stating plainly, because the shape of the repository is not obvious and it decides what a chooser can mean.

A snapshot is one app's data, or the settings tree. Never a machine. A four-snapshot repository looks like this:

28 Aug 13:10  56e95ba1  app=linkding     /mnt/lptest1/apps/linkding
28 Aug 13:10  6f8eed7d  app=ipinfo       /mnt/lptest2/apps/ipinfo
29 Aug 04:09  28bedbb0  system=config    /libreportal-system/configs
29 Aug 05:50  cc5b6bcf  system=config    /libreportal-system/configs

That is two apps plus two versions of the settings — not four backups to pick between. They are separate because they are used separately: the settings tree is small and changes rarely, app data is large and has to be restorable, movable and ageable on its own.

So the choice belongs on Contents, after unlocking, where each snapshot has a name and a date. Every row that has more than one gets a picker, defaulting to the newest; a row with one shows its date as text, because a dropdown holding a single entry is a control that cannot be operated.

The chain already supported this. restorePickSnapshot passes any value that is not the string "latest" straight through as an id — it has done since it was written, and nothing ever offered the choice. What was missing:

• restoreInspect now returns every snapshot per app and for the settings, not only the newest.
• restoreFirstRunBulk reads an optional RESTORE_SNAPSHOT_CHOICE map instead of hardcoding "latest". An associative array rather than an argument, because the CLI wrapper pads argv to nine slots and a per-app map cannot survive that; the map itself reaches the host as base64 JSON.
• backupRestoreSystemConfig takes a snapshot and a host. The host was the real bug: it defaulted to this machine's name, which is right for "recover my own settings" and wrong for a rebuild, where the snapshots carry the dead machine's name. It surfaced the moment a restore adopted a config with a different install name and the next lookup found nothing.

Verified by restoring both settings snapshots and diffing them: 28bedbb0 brings back a config carrying example.com, cc5b6bcf one with no domains. The pick changes what lands.

Not done: listing individual snapshots before the password. The count is a directory listing, but the identity of each snapshot — its host, its tags, what it holds, when its contents are from — lives in the encrypted object. All that could be shown unlocked is a column of hex IDs and file timestamps, which is not a thing anyone can choose between. Once the password is in, the Contents step already lists what is there, grouped the way people think about it: the settings snapshot, and one entry per app. Choosing a specific older snapshot to restore from is a real feature, but it belongs there, after unlocking, per app — not here.

The index that moved

Inserting Start shifted every step index by one, and validateStep was a chain of idx === 1 … idx === 6 with a comment already explaining which earlier insertions had moved them. It is keyed on the step NAME now.

lp-storage-step-test had the same pin and did not survive the change: it called validateStep(3) for Storage, which had become Domains, and reported that nothing blocked. That reads exactly like validation being broken — and if the four assertions had happened to be less specific it would have read like everything passing instead. Tests look their step up by name too now.

4. The password problem, stated plainly

An encrypted repository cannot be opened with anything inside itself. CFG_BACKUP_LOC_<idx>_PASSWORD lives in the system config — which is inside the backup. So on a fresh machine the user must supply the repository password by hand. There is no way around this and it is not a bug; it is what encryption means.

That single fact drives two requirements:

• The restore step must ask for it early and say what it is, because a user who never wrote it down has no backup, and finding that out at step 2 is better than at step 5.
• Everywhere we generate that password we must push harder than a comment. The location config already says "back up offline!"; the WebUI should show it once, prominently, at creation — treated like a recovery key, because that is what it is.

Once the repo opens, ordering is already correct in the CLI and should be preserved: system config first (it carries every other location's credentials, so one password unlocks the rest), then apps.

4.1 — Where does the typed password actually travel? (blocker for phase 2)

Found while building phase 2, and it needs a decision before the restore branch can be written, because it is a security trade-off rather than an implementation detail.

The WebUI cannot run restic. So a password the user types in the browser has to reach the host somehow, and the two existing channels both leak it:

Channel	Problem
Task command string (what the Backup page already does for this exact field, via config_update CFG_BACKUP_LOC_<n>_PASSWORD=…)	lands in the task JSON under frontend/data/tasks/, which is 0644 so the manager can read it — i.e. world-readable — and is visible in ps while the task runs
A file in frontend/data/tasks/	the container writes as dockerinstall; the manager runs as libreportal. At 0640 the manager cannot read it (verified), and 0644 is world-readable again

Note the first row is existing behaviour, not something this feature would introduce: editing a backup location's password on the Backup page already sends it that way. So this is a product-wide finding that phase 2 happens to surface, and the restore case is the sharpest version of it — that password is the key to every backup the user has.

Three ways out, roughly in order of effort:

1. A one-shot secret drop. The ownership helper already knows how to make a path readable across exactly this boundary (_webui_bind_access chowns MANAGER:cowner 0640 so the container can read manager-owned files). The reverse needs the same treatment: a root-helper-created directory owned cowner:MANAGER 0730, into which the container drops a 0640 file the manager reads once and unlinks.
2. Never persist it. Hold the password only in the task processor's memory for the life of the restore; write it into the location config only after the system-config restore lands (and reconcile with what the backup contained).
3. Accept the existing channel for consistency, and fix it product-wide later — cheapest now, and no worse than what shipping code already does, but it does mean a restore password sits world-readable in a task file until that task is pruned.

Recommendation: (1), and apply it to the Backup page's password field at the same time. It is a small, well-scoped addition to a helper that already exists for the mirror-image case, and it fixes a live weakness rather than only avoiding a new one.

5. "Set up a backup server if you don't have one"

Same components, other direction. After a New install, offer: "Where should your backups go?" — the same location fields, then engineInit and a first backup system. That closes a real gap: today backups exist but nothing prompts you to configure them, so the people most likely to need a restore are the least likely to have one.

Worth doing as its own wizard step even without the restore branch.

6. On "upload the backup file" — why the format differs

Worth being precise, because the mental model doesn't match the engines. restic, borg and kopia back up to a repository — a directory or a remote — not a single file. There is nothing to upload. The equivalents are:

• local — a path on a plugged-in disk. "Navigate to it" is right, and a directory picker is the natural UI.
• sftp / s3 / b2 / … — credentials, which the existing fields already collect.

If a genuine single-file import is wanted, that is a different feature: a portable per-app export (tar of the app dir + manifest, optionally encrypted) that could be handed around and imported. Cheap to build on the manifest that already exists, but it is not what the backup engines produce and shouldn't be conflated with them.

7. Portable export — the single file people actually mean

§6 explains why "upload the backup file" does not match a restic repository. But the underlying want is real and worth serving directly: one file, one app, hand it around.

libreportal app export <app> [file]     # -> <app>-<date>.lpapp
libreportal app import <file>

The format is deliberately boring: a gzipped tar of the app directory with its .libreportal-manifest.json at the root. That manifest already records the compose hash, images, volumes, size, databases and storage location, so import gets the same reconciliation as §3 for free.

This also gives the installer a third answer to "where is your backup?" — a .lpapp file — which is exactly the "navigate to the backup file" flow that prompted this document.

Not a replacement for the backup engines: no deduplication, no history, no encryption unless the user encrypts it themselves. It is a courier format — moving one app between machines, or keeping a copy of something before a risky change — and the docs should say so plainly so nobody uses it as their backup.

8. Phasing

Phase	Deliverable
1 ✅	Backup destination step in the WebUI wizard (§5) — the new setup half
2 ✅	Two installer paths: New setup / Restore from backup, through connect → discover → system config → apps. The system-config half only staged until §3.7; it now adopts.
3 ✅	Preflight reconciliation report in the installer (§3)
4 ✅	app export / app import (§7). The installer's .lpapp option is still open — see §9.5
5 ✅	The WebUI branch: New install / Restore from backup, through source → contents → rebuild (§3.12)

9. Open questions

1. Does the restore branch also restore the system config's identity — install name, domains, WebUI credentials? Restoring the WebUI login means the user logs into the new box with the old password, which is probably what they expect, but it is a surprise if not stated.
2. Partial restore of a host — pick apps individually (already supported by restoreFirstRunBulk's signature) or all-or-nothing at first run?
3. What if the backup is newer than this LibrePortal version? The manifest records the commit; refusing is safer than guessing, but it strands someone whose only copy is newer.
4. Should the installer's restore path accept a .lpapp too? app import exists, so the third answer to "where is your backup?" is a small addition — but a single app file is a thin thing to rebuild a server from, and offering it beside a repository may imply more than it delivers.
5. A published range that cannot be re-rolled — livekit advertises 50000–50100 to clients, so the port allocator's randomisation does not apply. Nothing checks such a range against the host before compose-up, and the result is an app that comes back part-started. A preflight that compares fixed published ranges against ss output would catch it; the open question is what to do about it, since the app cannot simply be moved elsewhere.
6. Import under a different name is refused today: the app's CFG_<APP>_* namespace and its compose identities (container names, Traefik routers, backup labels) would all need rewriting. instance create already solves "a second copy", so this may never be worth building.
7. Where does the repository password go once entered — straight into the location config it will restore over, or held only in memory until the system config lands and then reconciled?