The installer's restore path printed "Settings restored" and had never restored
a setting. backupRestoreSystemConfig only STAGES — right in general, since
overwriting a running control plane's config should not be automatic — but
nothing ever adopted the staged tree. The backup locations, domains and logins
landed in $restore_dir/system-config and stayed there.
So adoption is its own step now (`restore adopt`), allowed only on a machine
with nothing on it yet. backup/locations/ is adopted as a subtree, since the
index is part of the path and that directory is the whole point: it holds every
repository and its credentials, which is what makes "one password unlocks the
rest" true. Deliberately NOT adopted: the container account and its generated
password, port allocations, docker/rootless wiring, and storage/locations —
those describe the old machine, and a registry of drives this box does not have
would make every placement lookup resolve to a phantom.
The guard failed in the shape this project keeps hitting. It globbed the
containers directory, but the manager can traverse that without listing it, so
the glob returned a literal '*', the loop skipped it, and the function returned
"first run" on a machine with three apps. It adopted over a live install in
testing. It now asks the container user for the listing and fails closed: an
unreadable directory means "in use", never "empty".
Two config modes were inverted, found because a restore cannot restore from a
snapshot that was never taken:
- storage location configs were 0640 and hold no secrets. The backup runs as
the container user, could not read them, and restic wrote an INCOMPLETE
snapshot and exited 3 — so EVERY system-config backup failed once a second
storage location existed. Now 0644, with the test asserting they stay
secret-free so that mode remains defensible.
- backup location configs were 0644 and hold the repository password; nobody
could read them. They cannot simply be tightened, because the backup has to
read the credentials it uses — so the directory carries the restriction
(config-secure, manager:container 0750) and the file stays readable to the
two accounts that belong.
Fixing that surfaced a third: config-adopt clamped existing parent directories
to manager:manager 0750, closing configs/backup to the container user and
breaking the credential read the directory fix had just preserved.
restore domains reports which restored domains point here, and the installer
offers to drop the strays. Three verdicts, not two: setupCheckDomainPointsHere
falls back to hostname -I, and comparing a public A record to a private 10.x
address would condemn every correctly-pointed domain on a LAN-only box, which
is the deployment this product targets. Unverifiable is never offered for
deletion.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two faults in the same dispatch branch.
The unknown-command path called handleHelpCommands, which has never existed —
the function is cliHandleHelpCommands. So every unrecognised verb printed
"Unknown command" and then died on `handleHelpCommands: command not found`,
withholding the help that was the entire purpose of the branch.
Routing is the category DIRECTORY name, which is not always the word that
comes to hand: `libreportal validate system` is what people type, and it fell
into that same broken branch while the working spelling was `validation`. Map
the synonym onto its directory rather than renaming the directory, so the
handler name derived from it still resolves.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The resolver already supported per-app placement — CFG_<APP>_STORAGE names a
location and appDir sends data, compose and config there — and 37 of 39 app
templates ship the field. What was missing was choosing AT INSTALL TIME. The
only routes were editing a config by hand before installing, or installing onto
the default disk and then `app move`ing it, which copies the data twice.
libreportal app install <app> --storage=<location>
and the App Center's existing storage dropdown, which travels inside
config_variables. Both resolve to one answer in storageChoiceFor, so there is a
single code path.
Ordering is the whole difficulty, and getting it wrong is quiet. installApp
copies the app template into appDir(), sources it, and later applies the form
overrides. The choice has to be live before the copy (or the directory is
created on the wrong disk), written into the config before the source (or the
template's "default" wins and every later appDir in that process returns the
primary root), and folded into config_variables (or the override pass writes
"default" back). Miss any one and the directory and its config disagree — which
resolves correctly only until something sources the config.
Refuses an unknown or unmounted location, an existing directory, and an app
whose template marks the field **READONLY** (fixed to the primary root because
other apps reach it by literal path — storageMoveApp already refuses to move
those, and installing one elsewhere is the same violation from the other end).
Three shipped bugs found making this work:
* updateConfigOption chose its write helper by comparing the path against
$containers_dir — the PRIMARY root only — so an app on any other registered
location took the manager branch and `sed -i` failed with exactly the
permission error the comment above that code describes. `app move` writes
the new location with `|| true`, so it reported a successful move while
leaving the config naming the old disk.
* storageLocationName resolved a location's name only from an in-scope
CFG_STORAGE_LOC_<id>_NAME, falling back to the bare id. That name is the
value CFG_<APP>_STORAGE is set to, so the generated dropdown offered
"location-1" as both label and value — a choice that does not resolve. Read
it from the location's config when the variable is not in scope.
* storageSyncAllAppComments was written for "the regen path" and never wired
into one. Every CFG_<APP>_STORAGE option list was frozen at install time, so
adding a drive did not make it selectable anywhere. Called from the storage
generator now, which runs exactly when those lists go stale — and extended
to app TEMPLATES, since an app not installed yet is precisely the one whose
install form needs to show which drives exist.
Verified on a live install with three locations: linkding and authelia on disk1,
ipinfo on disk2, fourteen on the default root, each config naming its own drive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An end-to-end run restored 4 of 13 apps and reported
"First-run restore complete — 4 apps restored" as a success.
Two truncations, and fixing the first had hidden the second:
* the CLI dispatcher calls handlers with no arguments, so "$@"/shift inside
one operate on an empty list. Fixed earlier with LP_CLI_ARGS.
* LP_CLI_ARGS was built from start.sh's "$@" — but the root wrapper invokes
start.sh with exactly nine hardcoded positional slots. So the array could
never hold more than nine entries, and `${LP_CLI_ARGS[@]:5}` yielded at
most four app names.
The wrapper now forwards the real argv after those nine slots (they stay
untouched: every dispatcher reads them, and unset ones must keep arriving as
the literal "empty"), and start.sh reads it back as "${@:10}". Verified: a
preflight given six apps checks six, where five was the previous ceiling.
footprint_version 7 -> 8, since the wrapper is root-owned and baked at install.
Two further fixes so a truncation cannot pass as success again:
* restoreFirstRunBulk with no app list is now a whole-host restore — it
discovers the host's apps and re-applies the preflight. The installer's
report runs in its own process, so without this an app the user was told
would be skipped got restored anyway. init.sh now passes no list, so a
whole-host restore builds nothing that can be truncated.
* it counts what actually landed and returns non-zero naming the failures,
instead of reporting the length of the list it was handed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds the wizard step for importing existing apps, so the common case is
answerable in the WebUI rather than only from a terminal.
Path-based, not upload, and that is the design rather than a shortcut. A
.lpapp is a plain tarball and the file is already on the server, so
nothing secret crosses into the browser — which is exactly why this can
live in the WebUI when the encrypted-repository restore cannot (§4.1).
Accepts a single file or a folder of them.
Check first, then accept: the step enqueues `app import-check --publish`,
polls the document it writes, and renders one row per file with its
verdict — ready, a warning (its old storage location is gone, so it will
land on the default), or a refusal (already installed, no longer shipped,
will not fit). Refused rows are shown greyed with the reason rather than
hidden, and cannot be selected.
setupApplyConfig re-runs appImport's own checks rather than trusting the
payload: the machine can change between the check and the apply, and the
list arrives from a browser.
The backend route shell-quotes the path — it reaches a command line and
is user input.
Verified: the step renders as "Step 6 of 7", and the underlying check was
proven against real .lpapp files (correct app name from the tar, size
from the manifest, warning for a missing storage location, refusals for
an already-installed app and a non-export).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`libreportal app import-check <file-or-directory>` reports what an import
would do, as one JSON object per file, without touching anything. It runs
the same checks appImport makes — app still shipped, not already
installed, fits on the target, storage location still exists — so a UI can
show them and ask for acceptance before acting.
This is what makes a path-based import safe to drive from the WebUI when
the repository restore is not. A .lpapp is a plain tarball, not encrypted,
so there is no password to collect and nothing secret crosses from the
browser to the host — the blocker recorded in first-run-restore.md §4.1
simply does not apply.
Accepts a single file or a directory of them, so "point at this folder"
works as well as "point at this file".
Verified against real .lpapp files built for the purpose: app name read
from the tar's top-level directory rather than the filename, size read
from the manifest, a location the machine does not have downgraded to a
warning naming the fallback, and refusals for an already-installed app
and a file that is not an export at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`libreportal app export <app>` writes one app to a single file;
`libreportal app import <file>` installs it here. This is the thing the
original request described as "upload or navigate to the backup file" — a
restic repository is not a file, but the want behind the phrasing is real.
The format is deliberately boring: gzipped tar of the app directory with
its .libreportal-manifest.json at the root. That manifest already records
size, images, volumes, databases and storage location, so import reuses
the phase-3 checks for free — refusing an app this version no longer
ships, or one that will not fit, before unpacking anything.
Export stops the app first. A tar of a running Postgres is a corrupt
Postgres, and a file that looks fine until you restore it is worse than a
refusal. tar runs as the owning user with --numeric-owner so container
sub-UIDs survive the round trip instead of being remapped through this
machine's /etc/passwd.
Import re-runs the normal install pipeline after unpacking, because the
compose still carries the SOURCE machine's ports, IPs and domains — that
pipeline is what re-allocates them here, and migrateUrlRewrite fixes the
host-bound CFG_* fields.
Documented throughout as a courier format, not a backup: no history, no
retention, no encryption. Importing under a different name is refused
outright rather than half-working — the CFG_<APP>_* namespace and compose
identities would all need rewriting, and `instance create` already
answers "a second copy".
Fixes a bug this surfaced: _appDirIntended did an indirect expansion on
CFG_<SLUG>_STORAGE without checking <SLUG> can be a variable name, so a
hyphenated or mistyped app name emitted "invalid variable name" and then
reported the misleading "storage location is not mounted" for an app that
simply did not exist.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`restore preflight <loc_idx> <host>` reads every app's manifest out of its
own snapshot (engineDumpFile pulls a single file without restoring) and
checks it against this machine before anything is written:
* an app this version no longer ships is skipped — restoring one
produces a directory that can never start, and looks like success
until someone opens it
* an app that will not fit is skipped individually, because filling the
disk part-way through takes the apps that already landed with it
* a manifest naming a storage location this machine lacks falls back to
the default, and says which app moved where
The installer's restore path runs it and asks once before continuing.
Two bugs found by running it against the live repository rather than
reading it:
The CLI dispatcher calls handlers with NO arguments, so `shift 4; "$@"`
inside one operates on an empty list. `restore first-run bulk` has always
had this — a bulk restore silently received zero apps. Fixed at the entry
point: start.sh now captures LP_CLI_ARGS from "$@", and both call sites
use it.
And the wrapper fills unset argv slots with the literal string "empty"
(${5:-empty} … ${9:-empty}), so a trailing slot arrives as a five-
character app name rather than a blank. Filtering on -n alone let five
phantom apps through and reported each as "no longer shipped". Both call
sites now drop the sentinel. That also caps any explicit list at five, so
preflight discovers the host's apps itself when given none.
Verified against the live repository: 13 apps discovered and checked.
Sizes read "?" there because those snapshots predate manifests carrying
size_bytes — the intended graceful fallback, not a refusal.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Storage step was a technical dump: every check's full sentence
concatenated onto the card, so the fstab line the user is meant to act on
was buried in prose nobody reads.
The card now shows plain facts and at most two short flags — "Low on
space · Won't be mounted after a reboot" — with everything else behind a
Details button. The modal carries the technical spec (device, UUID, mount
options, removable), every check with its full explanation, and the
fstab offer.
That needed the shell to stop joining checks into one string: the
generator emits a record per check, plus the fstab line as its own field,
so neither the card nor the modal has to parse anything back out of the
other.
The screenshot caught a bug this restructure introduced: summaries keyed
on check id alone, so a PASSING check printed the failure wording next to
a green tick — "This drive's format can't store file ownership" above
"Filesystem: ext4". Now severity-aware.
On writing /etc/fstab — §1 ruled it out and §6.3 now records why that
reverses. The warning is useless to the audience this is for: "add this
line to fstab" assumes SSH, root, an editor, and knowing what fstab is,
and the likely outcome is a reboot where nothing starts. What makes it
defensible is nofail + x-systemd.device-timeout, which mean a missing
device can never block boot — without that pair it would stay a non-goal,
because the failure being risked (an unbootable machine) is worse than
the one being fixed.
Enforced in the root helper: UUID never /dev/sdX, append inside a marked
block, refuse a target or UUID already described, refuse the root
filesystem, require a live mount, timestamped backup, and
`findmnt --verify` before the file is installed — a file that doesn't
parse never reaches /etc. Opt-in only.
Verified against a real filesystem: entry added and verifies, the
persistence warning then disappears on the next scan, and duplicate /
root-fs / non-mountpoint / relative are each refused with the reason.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Phase 2 and 4 of docs/roadmap/storage-locations.md. Apps can now be
placed on a location and moved between them.
CFG_<APP>_STORAGE lands in all 37 app templates, holding a location NAME
rather than a path: names survive a migrate to a host with different
disks, paths do not. The 11 infrastructure apps that other apps reach by
literal path (traefik, prometheus, grafana, adguard, gluetun, crowdsec,
headscale, dashy, pihole, unbound, wireguard) are pinned. libreportal
itself never gets the key — it is pinned structurally by webuiDir.
Pinning needed no second config key. "Pinned" is not a fact about a value,
it is a statement about whether the field may be edited, so it goes in the
comment beside **ADVANCED** and **DEV** as **READONLY**, and the field
factory renders those disabled. That marker earns its keep beyond this
feature: derived fields already warned in prose that editing them does
nothing (crowdsec.config:72) next to a perfectly editable input.
storage_app_config.sh keeps the comment honest — it carries the resolved
path for hand-recovery and regenerates the dropdown from the registry, but
only writes when something actually changed, since the app .config is
user-editable and lives in the container-owned tree.
app move stops the app (a live copy of a running Postgres is a corrupt
copy), snapshots it, copies, verifies, and only then removes the source.
The copy runs in libreportal-ownership because it must: app data holds
rootless sub-UID files the manager can neither read nor recreate.
Verified against two real ext4 filesystems that a cross-device move
preserves uid 231141 and the payload, and that the source survives every
refusal path — unregistered destination, the WebUI app, a traversal in the
app name, and an occupied destination.
Task titles registered in both tables, with a specific rule so a move
renders as "Nextcloud - Move to bigdisk" rather than the generic fallback
dropping the destination. lp-task-names could not be run to confirm — it
borrows the WebUI container's node and no containers are running.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Phase 1 of docs/roadmap/storage-locations.md — locations can now exist.
Nothing places an app on one yet; that is phase 2.
libreportal-storage is the only writer of the root-owned registry, and
its admission rules are what make adding a location safe: absolute and
canonical (a symlinked path is refused), outside the protected system
set, non-nesting with any known root in either direction, and EMPTY — or
already carrying our marker, which is the adopt case for a drive that
already holds app data. Root only ever chowns an empty directory, so
acceptance cannot hand away anything that existed. The parent must also
not be manager-writable, which is what closes the validate-then-chown
race; /mnt and /srv qualify, a path inside the manager's home does not.
The fitness checks answer a different question — "will app data actually
work here" — and grade rather than refuse. Only checks 1-5 (filesystem
type, mount options, ownership, sub-UID range, write/read-back) can block.
Reboot persistence and removability warn, because both describe supported
setups and start-up is already gated by the marker test.
The ownership probe had to move into the root helper. For a candidate the
directory is not ours yet — a fresh /mnt/disk is root-owned 0755 — so an
unprivileged probe could only ever report "cannot create a directory
here", which says nothing about the filesystem. Verified against a real
loopback ext4: it now reports ownership, sub-UID and read-back cleanly.
The Disks view unions the registry with attached hardware, registry
first. Verified on the live box that pulling a drive leaves its row in
place as not-attached, naming the app stranded on it, rather than the row
silently disappearing at exactly the moment someone needs it.
Also registers storage_scripts with both loaders, adds the CLI category
(auto-dispatched by cli_initialize.sh), and bumps footprint_version to 6
for the new root helper and the widened sudoers allowlist.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The main sweep — ~260 call sites across ~100 files move from string
concatenation on a single root to appDir/storageAppDirs/storageAppConfigs.
On a single-root install the resolved paths are identical, so this is a
no-op until a location is registered.
Enumerators were the interesting half. `for d in "$containers_dir"/*/`
appears in the menus, the registry/artifact scanners and the DNS setup —
and a shell glob cannot list a rootless 751 tree at all, which is the
same bug config_find_file.sh already documents in a comment. Routing them
through storageAppDirs (which enumerates as the owning user) fixes that
alongside the multi-root work.
Three places needed judgement rather than substitution:
db_app_scan.sh deletes database rows and port allocations for apps whose
folder is missing, and reaps "empty" app dirs. With a storage location
unmounted, every app on it looks exactly like that. Each of those
branches now gates on appStorageAvailable first — an app on an unplugged
drive is skipped with a notice, never deleted.
instance_create.sh rewrites cloned hooks so an instance touches its own
directory instead of the base app's. Its sed matched ${containers_dir}<type>,
which this sweep just replaced with $(appDir <type>) — so it would have
silently stopped redirecting, and an instance would have written to the
original's files (the adguard auth adapter case its own comment warns
about). Now matches both appDir forms, verified against bare, quoted,
unrelated-app, legacy and prose cases.
peer_shell/peer_pull streamed and extracted relative to the primary root.
Both now use the app's own root, and peer_shell keeps a single-root
fallback since it runs as a restricted SSH shell with no LibrePortal env.
Also fixes a pre-existing bug found on the way: webui_app_config.sh
tested "$containers_dir/frontend/data/last_update", one level short of the
real tree under the libreportal app dir, so the WebUI refresh trigger
after a config update has never once fired.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two mechanical sweeps, no behaviour change on a single-root install.
The 14 `[[ "$p" == "$containers_dir"* ]]` prefix tests that decide
manager-vs-container-user elevation become pathIsContainerData, so a file
on a second storage root is no longer misclassified as manager-owned —
which would have written it with the wrong owner and failed later, far
from the cause. The 65 references to the WebUI's own tree become
webuiDir(), which is pinned to the primary root by design.
Two traps found while doing it:
run_privileged.sh is sourced directly by init.sh without paths.sh, so it
needs a fallback. Defining one named pathIsContainerData was wrong:
generate_function_manifest.sh indexes top-level definitions, and the
resulting autoload stub would have shadowed the real multi-root
implementation with the primary-only fallback — silently classifying
every file on a second disk as manager-owned, which is exactly the bug
this sweep exists to prevent. Renamed to _runCfgIsContainerPath, which
delegates when the real one is loaded.
setup_lock.sh built its path in a top-level assignment, so it was
evaluated at source time and needed the file flagged eager. Made it a
function instead: the path resolves on call, and the file drops off
LP_EAGER_FILES entirely.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fixing _instanceRewriteTools does nothing for an instance already on disk, and
a clone from the old code is broken in ways that never announce themselves:
every Tools action answers "App '<slug>' has no tool '<id>'" because
dockerAppRunTool wants app<Ucfirst><Pascal>; `authPersistCfg <type>` writes the
instance's new admin credential into the BASE app's config; and the clone
defines the base app's adapter and tool names while its bodies exec against the
instance's container, so the loader keeps whichever it sourced last and the
base app's user tools can end up administering the instance — decided by
nothing but find(1) order. Seen on a live install: the generated manifest
resolved [appBookstackListUsers] to bookstack_test's copy.
libreportal instance repair [slug] [--dry-run]
Rewrites the template dir only — no container is touched, nothing reinstalled,
so it does not route through the task system the way create/remove do.
Idempotent by construction. Two of the three renames match their own output
(appMattermost_teest… still starts with appMattermost), and a clone from the
old code is only PARTLY wrong — its suffix hooks were always correct and end at
the slug with no trailing underscore, which the infix rule would otherwise read
as type + id + () and append the id twice
(appSetupComposeTags_nextcloud_family_family). Three sentinels park the
already-correct spellings before the rewrite and restore them after, so a
healthy instance is a no-op and an interrupted run can just be re-run.
Verified against fixtures built with the old rule set for all five
multi-instance apps that ship tools: after repair each tree is byte-identical
to a fresh clone from the fixed cloner, a second pass reports "already
correct", and --dry-run leaves checksums untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four defects that all reduce to "a function an instance defines is invisible
to the code that dispatches it". Reported as `mattermost_teest has no upgrade
verifier`, for an app whose verifier was on disk the whole time.
* generate_function_manifest.sh shipped 0664. lpRegenArrays invokes it as an
executable, so it died rc=126 on every call and `|| true` swallowed it — the
manifest was never rebuilt on any live system, only laid down at deploy.
Its sibling generate_arrays.sh is 0775, which is why the files_*.sh arrays
looked current while the manifest was byte-identical to the shipped copy.
* lpRegenArrays now runs both generators through bash rather than depending on
the exec bit, reports a manifest failure instead of hiding it, and treats a
new containers/<app> dir as stale — the one event on a live box that adds
functions was the one the scripts/-only mtime check could not see.
* updaterHasVerifier consults the disk before answering no. The CLI runs
LP_LAZY=1, where the container scan is skipped and every function must come
from the build-time manifest, so an app created after the build reads as
having no verifier. GATE 1 then refuses an upgrade that is fully verifiable,
and updaterUpgradeAuto's `|| continue` drops the app in silence for good.
Self-healing regardless of manifest staleness, which matters because a
self-update restores the shipped manifest and drops instance entries again.
* _instanceRewriteTools gains three renames. authAdapter_<type>_<method>() was
caught by neither the prefix rule (no word boundary before _<type>) nor the
suffix rule (needs () right after the type), so the clone defined the base
app's adapter name while pointing at its own container — every instance user
tool answered "does not implement", and which definition survived came down
to find(1) order. Bare-app arguments to authAdapterCall/authPersistCfg went
unrewritten too, so an instance's password reset wrote the credential into
the base app's config. And dockerAppRunTool wants app<Ucfirst><Pascal>, which
no rule produced, so every tool on every instance was unreachable. The infix
rename runs before the suffix rename: the reverse order appends the id half
twice (appSetupComposeTags_nextcloud_work_work).
Verified on a live install: the upgrade ladder now plans mattermost_teest
11.9 -> 11.10, and `regen arrays --force` indexes the instance hooks.
Also carries in-flight instance-removal regen work from a concurrent session
on the same worktree (_lpRegenOrphanedApp, instanceRemove's WebUI refresh).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The /tasks page's right-side tick + dynamic Select all / Clear All ⇄
Delete Selected layout now covers the other three management surfaces:
- App detail → Tasks tab: filter bar gains the Clear All button and
master tick; Clear All there scopes to that app's tasks only. The
selection set is resolved through window.tasksManager everywhere —
TasksManager is constructed in several places, and ticks previously
landed on one instance while Delete Selected read another's empty set.
- Apps overview → Updates: the header's Update all button now morphs to
Update Selected (N) + Clear in place as rows are ticked, replacing the
separate selection bar between toolbar and list.
- App detail → Backups: each snapshot row gains Delete + a right-side
tick; a toolbar atop the list morphs Delete All ⇄ Delete Selected (N).
The whole selection rides in ONE task (delete <app> 1:a,2:b,…) since
the backup surfaces hold one task per subject at a time.
- CLI: backup app delete accepts comma-separated <idx>:<snap> pairs, and
both delete and delete_all now regenerate the WebUI backup JSON so
deleted snapshots leave the screen instead of lingering until the next
backup.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The unmaintained warning runs on one field — when upstream last rebuilt
the image — and off-Hub apps had no value for it. Hub answers in a single
call; the OCI API does not expose it at all, so an app on ghcr.io, quay.io
or lscr.io simply could not be assessed for staleness, which is the one
signal a user cannot work out for themselves.
It is in the image, just further down: manifest -> (if a multi-arch
index) a platform manifest -> config blob, whose "created" is the build
time. Three requests instead of Hub's one, once per registry window, and
only for the apps Hub cannot answer for — which is why Hub keeps its
cheap path rather than being routed through this.
Index and single-arch manifests are distinguished explicitly rather than
by position: in an index the first digest is a CHILD manifest, in an
image manifest it is the config itself, so reading "the first digest"
would silently fetch the wrong blob for one of the two shapes.
Live: stoat 2026-08-08, bookstack 2026-08-17, speedtest 2026-08-16,
invidious 2026-08-05 — all previously null. Hub unchanged, navidrome
still answered by the single-call path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Version discovery spoke only hub.docker.com, and every other registry
got a shrug: updaterTagExists returned "no" and updaterRegistryTags
returned nothing. Five apps live off Hub — stoat and wireguard on
ghcr.io, bookstack and speedtest on lscr.io, invidious on quay.io — and
for all of them the updater reported "up to date" having never asked.
That is the same dishonesty as a scan that never ran: an absence of
evidence rendered as a clean bill of health.
There was never a barrier, only unwritten code. The standard
Distribution API needs one extra step: request, read the
WWW-Authenticate challenge, fetch a token from the realm it names,
retry. ghcr.io, quay.io and lscr.io all answer anonymously for public
images — lscr.io by pointing its realm at ghcr.io, quay.io by not
challenging at all.
Docker Hub deliberately keeps its own path. hub.docker.com returns tags
NEWEST-first, so the 100 it pages are the 100 that matter, and it draws
on a different budget from the pull limit — registry-1.docker.io
manifest reads count against the anonymous 100/hour that the updater
needs for actual pulls, and a ladder probes a tag per rung.
Tag LISTING off Hub is a weaker signal and the comment says so: /v2/
tags/list is lexical, not newest-first, and large repos cap the page, so
the newest release can legitimately be absent. Probing backfills it,
which is why the probe fallback added earlier matters more off Hub than
on it.
Verified against all four registries: existence probing correct on eight
cases including true negatives; stoat climbs v0.15.0 -> v0.15.1 through
ghcr.io, and correctly reports nothing above v0.15.1 — the same answer
as before, but now because it looked. Hub unregressed: matrix still
resolves v1.158.0 -> v1.159.0 and nextcloud still ladders 31 -> 32 33 34.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
updaterSetAnchorVersion located the anchor by looking for a service
literally named "<app>-service". That is a convention, not a rule:
matrix names its anchor service matrix-synapse and stoat names its api.
For those apps nothing matched, so the rewrite changed no lines and the
upgrade aborted at step 2 with "could not set version" — after having
already taken a snapshot. Dry runs never showed it because they return
before that step. It now finds the anchor by its bare <APP>_VERSION_TAG
sentinel, the same way updaterPrimaryImage does.
It also moved only the anchor. Some apps are one product shipped as many
images: stoat is nine stoatchat services released together, all on
v0.15.1, expecting matching versions of each other. Stepping the anchor
alone would have put api on v0.16 while events stayed on v0.15.1 — the
exact mismatch that once justified keeping the app off automatic
updates.
The lock-step set is DERIVED from the compose rather than configured,
because the compose already states it: a service moves with the anchor
when it carries a version sentinel, sits on the SAME tag, and shares the
anchor's registry namespace. Both tests are load-bearing and each
rejects a real case — livekit-server is same-namespace but on its own
cadence, for-web is pinned to a commit hash, element-web is a different
namespace entirely, and mongo has no namespace at all. Verified against
copies of five composes: stoat moves all eight sibling services and
nothing else; matrix, rocketchat and nextcloud move exactly one image.
Every sentinel that moved gets its CFG_*_VERSION key set, not just the
anchor's, or the next config-driven regeneration would quietly pull the
locked-step services back to the old version.
One trap worth naming: quotes were stripped with sed 's/["\047]//g',
but \047 is an octal escape awk honours and sed does not — in a sed
bracket expression it is the literal characters \ 0 4 7, so it deleted
every 0, 4 and 7 it saw and v0.15.1 arrived as v.15.1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
GATE 1 refuses to ladder an app that cannot prove a rung landed, and
only mastodon, nextcloud and stalwart could. None of those are installed
here, so the stepped upgrade — button or automatic — was unreachable for
every app on the box.
Three fixes.
_updaterPrimaryContainer assumed the container is "<app>-service". It is
a convention, not a rule: matrix names its anchor service matrix-synapse
and stoat names its api (container stoat-api). The verifier therefore
inspected a container that does not exist, saw no state, and could only
time out — on exactly the stateful apps that most need verifying. It now
reads the anchor service's container_name from the compose, buffering
per service block because container_name may sit either side of the
image line.
Added updaterVerifyHttpVersion: poll the app over its PUBLISHED port
from the host, pull the version from a JSON field or a response header,
and require agreement three polls running. Probed from the host rather
than `docker exec … curl` because half these images ship no curl at all
(mattermost is one), so exec-based probing is a coin flip on the
vendor's base image. Version comparison matches only the components both
sides state, since tags and self-reported builds rarely share precision:
v1.158.0 vs 1.158.0, 11.9 vs 11.9.1, 8.7.0 vs 8.7 all agree; 11.9 vs
11.10 does not.
Each app hook is then three facts. Verified live: all three confirm at
the version they are actually on, and all three REFUSE a version they
are not — which is the property that makes stepping them safe.
updaterUpgradeAuto now skips apps with no verifier instead of queueing a
task that GATE 1 will reject, which would otherwise mean a failure
notification every day for an app that was never eligible.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The hook substituted `simple_style: auto`, a line that only exists in SearXNG's
full bundled settings.yml. The file generated here is the minimal
`use_default_settings: true` form with no ui: block at all, so the sed matched
nothing and the theme setting had never taken effect on any install.
It could not have worked even with the right pattern: the entrypoint chowns
settings.yml to searxng:searxng (uid 977) mode 644 on first start, so the
host-side docker user cannot write to it. The edit now runs inside the
container via docker exec, targeting the real key path
ui.theme_args.simple_style.
Three shapes are handled so the hook stays correct on repeat installs and
alongside hand edits: substitute in place when simple_style already exists,
nest theme_args inside an existing ui: block rather than appending a second one
(a duplicate YAML key SearXNG refuses to load), and otherwise append the whole
block. All three were exercised against the running container and produce valid
YAML with exactly one ui: block. awk rather than `sed a\` for the nesting case,
since busybox sed does not expand \n in appended text.
The value is validated against auto|light|dark|black before being written.
SearXNG checks it at startup and exits on anything else, so an unrecognised
CFG_SEARXNG_THEME would have taken the app down instead of merely looking
wrong; it is now reported and the default left alone.
Verified end to end on a base install and a --local instance: both come up,
serve 200, and report Dark as the selected style on /preferences, each with its
own settings.yml and secret_key. The instance's cloned hook correctly reads
CFG_SEARXNG_PROBE_THEME and targets its own container, since the container name
is built from $app_name. Both test installs were removed afterwards.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two halves: the ladder could not climb the commonest versioning scheme,
and nothing ever climbed it on its own.
The ladder stepped by bumping a tag's LAST numeric component, so
v1.158.0 went v1.158.1, v1.158.2, … and never arrived at v1.159.0. It
then failed closed, refusing to build a path. Synapse publishes
v1.159.0 and no v1.158.1 at all, so Matrix could not be laddered by the
button either — three-part semver minor bumps were simply unreachable.
updaterNextRung now considers a bump of every component, keeps the
candidates that exist upstream and takes the smallest: the next release
by definition, whether it lands in the patch position or crosses into a
new major. Shape discipline is unchanged, so 31-fpm-alpine still never
becomes 31-apache, and each rung is still probed, so none can be
skipped. updaterTagBumpAt moves here from the scan, its natural home,
which also breaks a source cycle.
updaterUpgradeAuto then climbs at most ONE rung per app per calendar
day, inside the install window, for apps set to auto. One rung because a
ladder run unattended can be several migrations deep before anyone
looks, and "restore the snapshot from a minute ago" stops comforting
once four have stacked; one a day so there is time to notice. It crosses
a major if that is genuinely the next release — refusing would strand an
app on the last version of its line forever — but one step at a time,
never as a leap. Two stamps: the target rung (a failure is not retried
until something newer ships) and the day.
Every rung goes through updaterUpgradeApp unchanged, so GATE 1 still
refuses any app without a real verifier, and the per-rung contract is
identical to the button: snapshot fail-closed, set version, pull, up,
verify, restore that rung and stop on any failure. History now records
the trigger instead of hardcoding "manual", including on the rollback
paths. CFG_UPDATER_LADDER_AUTO gates the whole thing separately from
CFG_UPDATER_AUTO, because "keep my apps patched" and "move my apps
between versions unattended" are different appetites for risk.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Audit of per-app hooks/tools found 19 of 33 apps whose helpers would have
operated on the BASE app after cloning. Two general causes, both fixed by
rewriting classes rather than patching apps:
- Container references escaped the rewrite whenever a flag sat between the
docker verb and the target (`docker exec -u git gitea-service …`), since the
old rule only matched a name immediately after the verb — and the hyphenated
form missed the `<type>_` rule too. Hook trees now get the same discovered
identity rename the compose does, reading names from the TYPE's compose since
the clone has already been rewritten by then. Safe to apply broadly: the
compose pass runs first and aborts for any app whose identities aren't
<type>-prefixed, so a bare word like stoat's `api` never reaches it.
- Hooks that build the deployed path as "${containers_dir}<type>/..." instead
of "$containers_dir$app_name/..." read and WROTE the base app's files —
adguard's auth adapter edits AdGuardHome.yaml, so an instance would have
rewritten the original's config. The trailing slash is optional in the match:
dashy tests [[ -d "${containers_dir}dashy" ]] and gluetun cds into it, both
ending at the quote. Only the first path component is touched, so
${containers_dir}prometheus/prometheus/... keeps its inner segment.
Re-audit: all 33 apps with hook trees are clean. Stoat still leaks, but it is
refused at the compose stage and never reaches this code.
Volumes audited too, and need no changes: no app uses named volumes, so the
./relative bind mounts every app uses resolve inside each instance's own
deployed dir. The absolute sources that exist are host or in-container paths
correctly shared read-only (/etc/localtime, /sys, /etc/ssl/certs). Jitsi's
${CONFIG} is set per-app by its own hook to $containers_dir$app_name/... and so
follows the slug.
Bookstack's rewritten tool tree is byte-identical to the live instance's across
all 8 files, so the running instances are unaffected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An instance's isolation never needed a domain — its own slug, dir, secrets,
IP and randomly-allocated host port already make two copies independent. But
the routing layer assumed one, so a LAN-only box got a broken instance rather
than a port-served one. Four fixes:
- instanceCreate now rewrites the parent-service column of the cloned config's
PORT_ rows to match the service names it stamps into the compose. That value
is stored as network_resources.parent_service and joined against the
compose-derived service names, so an instance left carrying the TYPE's
service name matched nothing: it rendered in the WebUI with no port, no URL
and no login row despite being up and reachable.
- `instance create --local` (plus a LAN-only toggle in the modal) forces every
port to access=private, traefik=false, for a second copy that should stay
off the domain even when one is configured.
- initializeAppVariables forces the traefik column false when no CFG_DOMAIN_n
is set. Previously a traefik=true port with an empty domain stamped
Host(`app.`) — a trailing-dot host matching nothing — and dragged APP_URL to
https://app. with it, breaking every app that builds its links from APP_URL.
host_setup is blanked for the same reason. The published host port is
untouched; access type, not the traefik flag, gates allocation.
- APP_URL's direct host-port branch now prefers a new $local_ip_v4 (the source
IP for the default route) over $public_ip_v4, which is the WAN address from
an external resolver. LibrePortal never forwards ports, so the WAN address
was unreachable for exactly the LAN/VPN clients that branch serves.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The registry helpers rejected any repo containing a dotted host segment,
which caught 'docker.io/authelia/authelia' — Docker Hub spelled out in
full. Those apps were silently skipped by tag enumeration and version
laddering. Strip the docker.io/ and index.docker.io/ prefixes before the
host check; genuinely third-party registries (ghcr.io, quay.io, lscr.io)
are still correctly skipped.
Found by auditing every app's anchor image.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The live Nextcloud 31→34 climb left 4.4 GB of images behind — one per
rung, each ~1.5 GB, all still present after it finished. On a small VPS
that is the difference between working and full.
`system reclaim` cannot help: it collects DANGLING images, and every rung
is a real tag, so all of them stay tagged and stay on disk. (Rolling apps
never hit this — moving a floating tag orphans the old image, which
reclaim then collects. It is specific to laddering.)
After a SUCCESSFUL climb only, remove the images stepped through, keeping
the immediately-previous version so a roll-back needs no download.
CFG_UPDATER_UPGRADE_PRUNE=false keeps everything. Never runs on failure,
where the older images are exactly what recovery may need.
Tested: a 3-rung climb removes 31 and 32 and keeps 33; a single-step
climb removes nothing (its previous version IS the rollback target); the
config switch disables it.
Found by looking at the box after the first real ladder run — the feature
worked, and then quietly cost 4.4 GB.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`updater upgrade <app> --detach` parsed "--detach" as the target version
and refused with "no safe path from 31-fpm-alpine to --detach". It failed
safe, but blaming the version for a misplaced flag is a poor way to say
the flag is not supported here. Unknown options now say so.
Found during the first live ladder run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ties the ladder and the verifiers together behind a new verb:
libreportal updater upgrade <app> [version] [--dry-run]
Per rung, and every part is load-bearing:
snapshot (fail-closed) -> set version -> pull -> up -> VERIFY -> next
On failure anywhere: restore THIS rung's snapshot, put the version back,
stop, and leave the app on the last version it actually verified at. The
ladder never continues past a doubt.
A snapshot PER RUNG rather than one at the start, because upstream
migrations are usually one-way — Nextcloud 32's schema cannot be undone
by putting the 31 image back. The recovery guarantee is "restore the
snapshot from sixty seconds ago", which only holds if every rung has one.
Two gates before anything moves. An app with no <app>_upgrade_verify is
refused outright: the generic health check cannot see a half-finished
migration, so laddering on it would be a guess wearing a safety label.
And a ladder that cannot be computed end to end refuses rather than
attempting a partial climb.
`updater upgrade` is a separate verb from `apply` on purpose: apply moves
you WITHIN a release line (and may be automatic), upgrade moves you
BETWEEN lines and is always a deliberate act. Dry runs execute inline so
the plan is instant to read.
updaterSetAnchorVersion rewrites the image tag AND its version sentinel
together — updating only the image would leave the sentinel advertising
the old version, and the next config regeneration would silently revert
the app.
Tested with stubs against the real code paths: the no-verifier gate holds
and changes nothing; a dry run has zero side effects; the happy path
snapshots at each current version before moving; a verify failure on rung
2 of 3 stops with the app on rung 1, restored, and never touches rung 3;
a failed snapshot moves no version and pulls nothing; a container that
will not start is rolled back.
NOT yet exercised on a live install — no app here needs a ladder. The
first real run should be a dry run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stepping 31 -> 32 -> 33 is arithmetic. Knowing 32 FINISHED before
touching 33 is the whole safety story, and it is invisible from outside
the app: Nextcloud runs its migration on boot and sits in maintenance
mode — or fails halfway — while Docker reports the container perfectly
healthy. Advance a rung there and a migration has been skipped on live
data.
Contract: <app>_upgrade_verify <app> <expected-tag> <deadline> -> 0
Returns 0 ONLY on positive confirmation that the app serves at the
expected version with nothing outstanding. Unhealthy, indeterminate and
timed-out all return non-zero — uncertainty is a failure, not a maybe,
because the alternative gambles with data.
nextcloud `occ status`: installed, NOT in maintenance, no pending DB
upgrade, and the running major matches the tag. Maintenance
mid-migration is expected and simply keeps waiting.
mastodon /health serving, ZERO "down" rows in db:migrate:status, and
the version from /api/v1/instance matching. /health alone is
insufficient — Puma answers before migrations finish.
stalwart /healthz/ready (per its documented probes), required to hold
stable rather than flash once. Weaker by design: the probes
confirm serving but report no version, and the file says so
rather than implying more.
updaterVerifyGeneric (running + healthy + no restart during a settle
window) is the fallback for everything else, and is explicitly NOT
sufficient to justify climbing a rung — the engine will refuse to ladder
an app with no declared verifier.
9 tests drive the dangerous states directly: maintenance mode, pending DB
upgrade, and a wrong major all correctly REFUSE to verify; clean states
pass. Those three negatives are the ones that would have corrupted data.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Foundation for stepped upgrades. Answers one question only — WHICH
versions, in WHICH order — with no side effects, so it can be tested
exhaustively. Applying the rungs is a separate job.
Nextcloud refuses to skip a major ("Updates between multiple major
versions and downgrades are unsupported") and will not start; databases
behave the same way about their data directory. For those apps 31 -> 34
is three upgrades, each with a migration that must finish before the
next begins.
Built by PROBING each candidate rung, not by enumerating tags — because
enumeration is provably unsafe here. Docker Hub pages at 100 ordered by
recency, and the first real-registry run proved the danger: it produced
v4.2 -> v4.4 -> v4.5 -> v4.6 for mastodon, silently skipping v4.3, which
exists (HTTP 200) but had fallen off the newest-100 listing. Skipping a
rung is the precise failure this file exists to prevent, so the ladder is
now built by incrementing and probing: v4.2 -> v4.3 -> v4.4 -> v4.5 ->
v4.6, 4 steps.
Guarantees: same shape only (never 31-fpm-alpine onto 31-apache),
strictly ascending, never a downgrade, rolling tags refused outright, and
a version upstream never published is stepped over only because the probe
said so. If a continuous path to the target cannot be constructed it
returns 1 and prints nothing — refusing to guess, because a wrong ladder
means a skipped migration.
20 unit tests, including the exact listing-truncation case above and the
numeric ordering that would otherwise drive an app backwards (0.9 vs
0.10). Real registry: nextcloud 3 steps, mastodon 4, stalwart current.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The missing piece of hands-off updates/backups: when a task fails while
nobody has the WebUI open, LibrePortal now says so — email (via the
existing Mail settings), ntfy, Gotify, Discord, Slack, Telegram, or
Pushover, configured under Settings → Notifications.
One hook, everywhere: the task processor reports every terminal task to
`libreportal notify task <id>` (detached, never load-bearing — hard curl
timeouts, failures ignored). The POLICY lives in the notify command, not
the daemon: CFG_NOTIFY_EVENTS = failures (default) | all | off, and
cancelled tasks never notify. Failure copy is task-aware — a failed
update says the app was already rolled back and won't be retried, so the
reader knows the box is safe before opening the WebUI.
`libreportal notify test` sends to every enabled channel with per-channel
results. Verified against a local mock endpoint: all webhook payloads,
JSON escaping (quotes/newlines), the events policy, and fail-fast on
dead endpoints (8ms, exit nonzero).
The v0.1.0 per-app NOTIFY_* field-mapping scaffolding (never wired to a
sender) stays as-is; this global channel is the system it was waiting on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four fixes that make the auto-updater a trustworthy background system:
* CFG_UPDATER_WINDOW (default 06:00-08:00 host time, right after the
05:00 backup cron; HH:MM-HH:MM wraps midnight, 'always' = any time).
Gates only the enqueue — scans keep running all day, so the Updates
page stays current and pending updates visibly wait for the window.
Malformed values fail closed and are rejected by the WebUI validator.
* "Check now" actually checks: an explicit `updater check` sets
UPDATER_REGISTRY_FORCE=1. The flag existed but nothing ever set it,
so the button silently reused the 6h digest cache and could not find
a build the user knew had shipped. Force also overrides interval 0,
which now means "manual-only" as documented in the roadmap.
* Registry stamp moved from /tmp to <system>/logs: the task processor
runs under PrivateTmp, so daemon and CLI each kept a separate 6h
clock and the daemon's reset on every service restart.
* A failed automatic attempt is no longer invisible: the scan emits
auto_attempted_digest (the one-shot no-retry stamp), and when it
matches the available build the UI stops promising an install that
will never come — per-app detail explains, the fleet row gets an
"auto failed" chip, and the Overview board counts it as needing you.
Also corrects the CFG_TIMEZONE label: it sets the containers' TZ only;
scheduled tasks follow the host clock (timedatectl), and the old
"Timezone for scheduled tasks" wording promised a knob that never
existed. The window + auto_window display state plainly WHEN updates
land, answering "how does the user know when the next update happens".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
First real end-to-end auto-update on a live install failed like this:
Automatically updating trivy (a recovery snapshot is taken first)
Snapshotting trivy before update…
Pulling new image(s) for …
Update of failed — rolling back…
Could not roll back automatically
The app name went empty after the snapshot. Cause: bash is dynamically
scoped, so a callee assigning an undeclared variable writes the CALLER's
local of that name — and a `while read app` loop leaves it EMPTY at EOF.
webuiBackupAppStatus's dashboard generator runs at the end of every backup
and did exactly that to updaterApplyApp's `app`.
Nothing was damaged: the pull ran against an empty name, failed before
touching the image, and the rollback was a no-op on a nonexistent app.
Fixed both ends. The generator (and three gluetun loops with the same
latent leak) now declare `local app`. updaterApplyApp/updaterRollbackApp
hold the name in `_upd_app` so they no longer depend on every callee's
hygiene, and updaterApplyAll stops leaking its own loop var.
This is exactly the untested path the roadmap flagged: "apply/revert not
yet exercised end-to-end on a live install with a pending update."
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds the decision half of the app updater. Detection (P2) and the
snapshot-first apply/revert (P3) were already real, but nothing ever
pressed the button — every update waited for a click.
CFG_<APP>_UPDATE_TYPE=auto|manual per app, default auto (33 templates)
CFG_UPDATER_AUTO=true|false master switch, default true
updaterAppPolicy resolves the two the way backupResolveStrategy already
resolves backup strategy: the global switch can only make things more
manual. updaterApplyAuto runs at the end of `updater check` and enqueues
the ordinary updater_apply task for each auto app that has an update —
never applies inline, so an automatic update is the same code path, task
log, History entry and Roll back button as a manual one.
Safety: each attempt stamps its target digest under generated/auto/, so a
build that fails is rolled back and then left alone rather than retried on
every scan; in-flight updater tasks are skipped so scans can't stack.
Tracked end to end: updates.json carries each app's resolved update_type,
History entries carry trigger=manual|auto. The WebUI says whether updates
install themselves, chips only the apps that opted out, labels automatic
history, and — since an auto app's pending update needs no decision — keeps
it off the Overview board's "Needs action" view.
Also fixes artifactApplyAuto enqueueing without --detach: it runs inside
the single-threaded task processor's own poll, so following the new task in
the foreground waits for a task that cannot start until it returns.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Make the Update / Roll-back buttons tell the truth, closing the "new code on
old data" hole a floating tag creates.
updaterApplyApp:
- Anchor-correct: capture before/after from updaterPrimaryImage (the
<slug>-service image), not `grep -m1 image:` — fixes ollama et al.
- Records EXACT build refs in history from->to: repo:tag@sha256:<digest>
(via updaterRefDigest), so history is meaningful even when the tag doesn't
move (a rebuilt `latest`).
- Un-pins any digest a prior rollback pinned before pulling, so Update tracks
the channel again instead of freezing on the rolled-back build.
updaterRollbackApp:
- Before recreating, re-pins the anchor image to the pre-update build's digest
(from history's last update/ok `from`) via updaterSetAnchorRef, so `up` runs
the OLD code — not the current channel head. This is the fix for restoring a
data snapshot but recreating on a newer image.
New helpers (cli_updater_commands.sh): updaterRefDigest (local RepoDigest),
updaterSetAnchorRef (rewrite the anchor image line by service name, preserving
indent + the version sentinel; correct for companion-first apps like ollama),
updaterLastUpdateFrom (roll-back target from history).
Verified the helpers on nextcloud + ollama: pin adds @sha256 to the right
anchor only, sidecars untouched, sentinel preserved, unpin restores, YAML valid.
Caught and fixed a `local a=$1 b=...$a...` same-statement expansion bug that
would have silently no-op'd the rollback pin. End-to-end apply/revert not
exercised live here (no installed app has a pending update on this box).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
An offline trivy install crash-looped (server FATALs when it can't fetch the
vuln DB), and on rootless docker the restart storm churned the shared network's
port-forwarder until the WebUI's own published host port was torn down — the
WebUI stayed healthy INSIDE its container but was unreachable from the host, with
nothing detecting or healing it.
Three fixes, in the house self-healing style (mirrors the network-drift trio):
1. Control-plane health checker wired into the existing task-processor idle poll
(maybeRegenPoll), no new daemon. dockerHealthScan (read-only) detects daemon
down, a WebUI running-but-host-port-unreachable (the port-forward corruption),
and crash-looping containers. webuiSystemHealthCheck writes
frontend/data/system/health_status.json + self-dispatches a heal — the user
can't click a button on a dead WebUI, so the poll drives the fix. Frontend
health-notifier surfaces a topbar badge + dashboard banner + details panel.
2. Failure cap, enforced centrally by dockerHealthHeal (task-gated): stops
crash-loopers (removing the churn), restarts the WebUI to re-publish a lost
port forward, and — only if that fails — recycles the rootless daemon and
restarts the core container. Caps every app immediately, no template churn.
3. Trivy no longer crash-loops offline: the server runs in a shell retry-loop so
the container stays Up and quietly retries on a backoff instead of exiting
FATAL. Verified: container stays Up across repeated DB-download failures.
Core WebUI compose gains restart: unless-stopped so it self-recovers after a
reboot / daemon recycle instead of staying down.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Per the intended model: catalog sources should live in the config place (like
domains), and registry_catalog.json should be a purely GENERATED artifact
derived from them — not the source of truth. Replaces the earlier
$docker_dir/catalog/sources.json store.
- New configs/general/general_catalogs — CFG_CATALOG_1..9, one catalog base URL
per slot ("url" or "url|channel"), domains-style. Official stays pinned as
source #1 (derived from CFG_RELEASE_BASE_URL, not listed here). Slot N → source
idx N+1 (stable id for the Add picker / `app add --source`).
- catalog_sources.sh now reads/writes those CFG vars (via updateConfigOption)
instead of a JSON file; dropped catalogSourcesFile + the enable/disable toggle
(presence = enabled; remove = clear the slot).
- configUpdateBatch regenerates registry_catalog.json when a CFG_CATALOG_* key
changed — so pressing Save in the WebUI rebuilds the browse data.
- webuiRegistryCatalogScan is unchanged (still iterates catalogEnabledSources).
Verified: CFG_CATALOG_1/2 → sources at idx 2/3, empty slots skipped, url|channel
parsed, official pinned at idx 1.
Signed-off-by: librelad <librelad@digitalangels.vip>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
First slice of multi-catalog ("taps") support. Today the App Center browses one
catalog (get.libreportal.org). This adds an ordered list of catalog sources and
teaches the browse scan to merge them.
- New scripts/catalog/catalog_sources.sh — the source list in its OWN file
($configs_dir/catalog/sources.json). Source #1 is ALWAYS the official catalog,
synthesized live from CFG_RELEASE_BASE_URL/CHANNEL (can't be edited/removed,
always pinned on top). Extra sources are stored as a small JSON array and are
UNVERIFIED (trust=community). Helpers: catalogSourcesJson / catalogEnabledSources
(priority order) / catalogSourceAdd|Remove|Toggle|List / catalogFetchCommunityIndex.
- webui_registry_scan.sh now walks catalogEnabledSources: the OFFICIAL source is
still signature-verified (lpFetchIndexInto, unchanged trust path); third-party
sources are fetched unverified. Apps are merged by slug into one card carrying a
sources[] array in priority order (highest first = default). Trust/verified are
taken from the SOURCE, never the artifact's self-claim, so a community index
can't promote itself to "official". Icons still mirror same-origin from the
official index only. registry_catalog.json gains top-level sources[] + per-app
sources[]; the old source{} object + signed/serial are kept for back-compat.
- New `libreportal catalog source list|add|remove|enable|disable` + `catalog
refresh` CLI (dynamic-routed). Mutations go through the task system (cliTaskRun
"…" "catalog"), never a new mutating API.
Scope firewall: this governs APP BROWSE + ADD only. LibrePortal's own updates and
hotfixes still resolve from the official CFG_RELEASE_BASE_URL alone — lpFetchIndex
is untouched, so a third-party catalog can never become a system-update channel.
Next: `app add --source`, then the WebUI (domains-style block source manager +
Add-dialog source picker with the Official badge / unverified warning).
Signed-off-by: librelad <librelad@digitalangels.vip>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three coupled defects in the first-run wizard flow, all surfacing as
"it said complete but nothing was set up":
1. Zero-app installs sailed through. With no apps ticked, setup was just
config+finalize, finished in seconds having installed nothing, and
fast-forwarded to an empty App Center. Add a self-contained in-wizard
confirm ("Install with no apps?") before submitting. Can't reuse the
shared confirmation-dialog component — it isn't loaded this early in
boot — so the dialog is rendered by the wizard itself and mounted on
<body> to escape the aurora surface's FX-stacking rule.
2. finalize declared success unconditionally. It never inspected the
per-app install tasks, so a failed app still yielded "your install is
ready" + a redirect. Pass the setup group id to `setup finalize`; it
now rolls up the group's app-install task results and logs a clear
partial/failed verdict. The WebUI completion watcher gates the welcome
toast + App Center hand-off on the whole group succeeding, not just on
finalize completing — a failed app now keeps the user on the tasks page
with an error toast instead of a false all-clear.
3. Setup task count was confusing (banner "of 2" while the page listed 3).
On dev/git installs the topbar's dev-mode auto-enable raced the wizard's
own CFG_DEV_MODE write and spawned a second, un-grouped config-update
task mid-setup. Skip that auto-enable while a setup handoff is active
(the wizard already persists CFG_DEV_MODE); it runs on the next load if
still needed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
webuiRegistryCatalogScan (run by updater check, same atomic keep-prior
pattern as webuiArtifactScan) writes apps/generated/registry_catalog.json:
the type:"app"/kind:"bundle" rows of the signed index annotated with
defined/installed, browse metadata from the envelope meta, and icons
mirrored into core/icons/apps/registry/ ONLY when their bytes match the
sha256 pin in the signed index — the browser stays same-origin; a tampered
or oversized icon is skipped, never served.
webuiArtifactScan now selects type=="hotfix" so app rows never render as
pseudo-hotfixes in the Improvements tab, and counts+logs artifacts of
unrecognized type instead of surfacing them (the §8.1 forward-compat
firewall on the scan path).
Harness vs a locally served registry: 14/14 (catalog row + meta + flags,
icon pin verify + tamper skip, hotfix-only stream, unknown-type skip+log,
unreachable-registry keeps prior files).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Opens the two designed seams (roadmap §8.4): _artifactResolve accepts
type:"app" (slug validated, the installed-app gate skipped — presence is
the collision policy's call), and payload.kind:"bundle" gets its own APPLY
flow. The download core (sha256 pin vs the signed index + minisig +
refuse-unsigned) is factored into _artifactDownloadVerified, shared by ops
and bundle payloads.
A bundle add: fetch → quarantine-validate → place in the definition tree
(staging + one rename, manager funnel) → lpRegenWebui → verify the app
surfaced in apps.json → applied-record with a precise undo → History.
The validator is fail-closed (traversal/absolute paths, links/devices,
single top-level dir == slug, charset, size/entry caps, set-id strip,
config TITLE+CATEGORY + compose present, bash -n every .sh) because the
definition tree is live-sourced on every CLI start — nothing lands there
before trust + quarantine pass. Collision policy: installed-live refused,
local definitions win, registry-owned re-add = reversible definition
update (prior tree packed into the undo). Revert removes/restores the
definition (refused while installed) and regens. Apps never auto-apply
(type filter kept + publisher forces auto:false).
New verb: libreportal app add <slug|artifact-id> (app_add task; resolves
by slug via appAddFromRegistry, ambiguity refused).
Also fixes the second half of the sigstate-propagation bug class:
artifactApply captured $(_artifactResolve) in a subshell, stranding
_ART_INDEX/_ART_APP/_ART_SCOPE AND the LP_INDEX_SIGSTATE the apply gate
enforces — on a signed box every apply would have refused as unsigned.
Resolve now assigns globals (_ART_JSON) and is called directly.
Source-and-mock harness: 46/46 (resolve gates, 14 validator refusals,
happy add, collision matrix, definition-update round-trip, revert
semantics, postcheck + record-failure rollbacks, apply-auto exclusion,
app add verb).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Every caller captured the index with var=$(lpFetchIndex), which runs the
fetch in a command-substitution subshell — the LP_INDEX_SIGSTATE global it
sets never reached the caller. On a box with real signing active the
artifactApply/apply-auto gates would therefore refuse a correctly signed
index (fail-closed, but the apply path would be dead on arrival the day
signing activates), and artifact index / the WebUI scan would report a
verified feed as UNSIGNED.
New lpFetchIndexInto <var> [cache] runs the fetch in the calling shell and
assigns via printf -v; all four call sites converted. Verified with a
source-and-mock harness against a locally served index: 10/10 (sigstate
reaches caller, serial high-water, anti-rollback refuse, staleness refuse,
id enumeration, envelope round-trip).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The rootless WebUI container reads its bind-mount sources (configs/webui/*)
through the container-owner GROUP since a2376e2 switched those files from
world-readable to 0640 group=container-owner. But the WebUI credential
randomizer rewrites webui_logins via `sed -i` as the non-root manager, which
recreates the file with the manager's own group — dropping the container-owner
group. The installer then started the container immediately, so node hit
EACCES on /app/webui_logins at require-time (parseConfigFile) and exited 1;
nothing listened on the WebUI port. `libreportal webui login reset` had the
same latent bug (rewrite → restart). Under the old world-readable model a
post-sed file stayed o+r so the container could still read it, which is why
this only surfaced on fresh rootless installs after a2376e2.
Fix: make reconcileWebuiDirOwnership the single "ready the WebUI for its
container" pass — it now also restores the configs/webui bind access (new
`webui-bind` ownership action) on top of the container-dir chown. Reorder the
installer so the credential randomizer runs BEFORE the before-start permission
pass, making that pass the last ownership touch before the container starts;
and call reconcileWebuiDirOwnership before the restart in login reset.
Live box recovered via `libreportal-ownership reconcile`; WebUI 200.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The Services tab restart button POSTed to a backend endpoint that (a)
checked the app's compose path from INSIDE the webui container, where
the host's containers root isn't mounted — so every restart failed with
'Compose file not found' — and (b) queued a raw 'docker compose restart'
that the host task processor would run as the manager user, which can't
talk to the rootless daemon anyway. Errors surfaced via a bare alert().
Per-service restart now follows the exact shape of the whole-app verbs:
- CLI: 'libreportal app restart <app> [service]' — the optional service
arg makes dockerRestartApp restart just that compose service, via
dockerCommandRun (right user in rootless mode) from the app dir on the
host, where the compose file actually lives. Service names validated
against compose-legal characters before touching a shell line.
- WebUI: the button dispatches a 'service_restart' task action through
the task router (mutations-via-tasks), runs in the background with the
standard task toast + link — no page switch — and failures use the
notification system instead of alert(). Because the task runs host-
side, restarting the WebUI's own libreportal-service now works too.
- Backend: the mutating restart endpoint and its now-unused helpers are
removed; service-routes.js is read-only surface (status + log tails).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Move the WebUI-updater settings out of general_terminal into their own
advanced webui-category file (webui_logs precedent): new
configs/webui/webui_updater holds CFG_UPDATER_SCAN_INTERVAL and the
migrated CFG_HOTFIX_AUTO, listed in webui/.category.
The move only reaches existing installs if the config convergence
machinery works, and three pieces of it silently didn't:
- checkConfigFilesMissingFiles walked a stale hardcoded category list
('general features network' — features doesn't exist; webui/backup/
security never healed). Derive the categories from the template tree
instead, and heal .category metadata too: copy it when absent and
merge missing SUBCATEGORY_ORDER entries when present, so healed files
actually appear in the WebUI Config editor. core_categories removed.
- Option reconciliation never touched ANY nested config file: configs_dir
carries a trailing slash, so rel stripping missed ('configs//'), the
template lookup failed, and reconcileConfigFile early-returned for
every file. Strip the slash before matching.
- reconcileConfigFile's AUTO_DELETE=false branch read a never-populated
live_line array, losing the dropped keys it promised to keep. Populate
it alongside live_value.
Also exclude *.bak from config sourcing (reconciliation writes <file>.bak
next to live configs — now that it runs, sourcing backups would resurrect
deleted keys), and add 'libreportal config check' as a non-interactive
front door to the converge pass (was only reachable via install flows and
the interactive menu).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Replace the click-to-scan-only flow with a self-throttled auto-scan that
rides the existing task-processor idle poll (the same shape as the
network-drift check — no new daemon, unit, or endpoint):
- 'libreportal updater check auto' gates on the age of the generated
updates.json vs CFG_UPDATER_SCAN_INTERVAL (minutes, default 30,
0 disables); a fresh file makes the 60s tick a single stat() + return.
Manual checks and post-update rescans reset the clock for free, and a
missing file means the first scan runs ~a minute after install.
- Eligible signed hotfixes keep flowing through artifactApplyAuto, which
only enqueues ordinary tasks — mutations stay on the task path.
- Open updater surfaces (standalone /updater and the fleet Overview's
headless UpdaterPage) follow along with a 60s static-JSON re-read that
repaints only when a generated_at stamp changed; timer released via
dispose() on unmount, ticks skipped while hidden.
- Empty states now say the first scan happens automatically; Check now
stays as the immediate manual override.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Lets a *multi-instance-capable* app run as several fully isolated instances
on one box (e.g. two Bookstack/WordPress sites, or a "family" + "work"
Nextcloud) — distinct data, DB, subdomain, backups and update cadence.
Design: an instance is just another app. It gets its own slug (<type>_<id>),
its own CFG_<SLUG>_* namespace, deployed dir, DB row, IP/port allocation and
host, so the entire existing pipeline (scan, install, services, routing,
updater, backups) treats it like any app with zero changes. All
instance-specific rewriting is confined to a clone of the type's template;
the shipped template and the core engine are untouched.
Gating: opt-in per app via CFG_<TYPE>_MULTI_INSTANCE=true. Only Bookstack
carries it for now (the validated reference). The other 31 apps are
unaffected — the feature is invisible unless the flag is present.
- scripts/instance/instance_create.sh — clone + re-namespace config, rewrite
compose identity (container_name / Traefik routers / backup labels) and
per-app tools, set a hostname-safe subdomain (PORT field 10), then hand off
to dockerInstallApp. Plus instanceList / instanceRemove.
- libreportal instance create|remove|list — new CLI category; mutations route
through the task system (no new mutating API endpoint).
- WebUI: "instance of <type>" badge + a "New instance" card action on capable
apps, and a create modal (name + domain# + subdomain, live host preview)
that dispatches the standard task. Capability/instance-of read straight off
the already-exposed app config.
Known follow-ups (documented): flip the flag on more apps after a compose
identity check (Nextcloud next); per-app tools are best-effort isolated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Closes the gap behind the vpn-recreate bug: when the shared network is
recreated with a different /24, every app's stored static IP is left
outside it and adoptDockerSubnet only realigns CFG, not the apps.
- networkScanConflicts (network_conflicts.sh): read-only scan diffing each
active network_resources IP against docker's real subnet (via ipInSubnet).
Per-service routing-aware — skips gateway-routed services whose ipv4 is
commented out in the deployed compose, so gluetun apps don't false-positive.
Distinguishes 'daemon down' (benign) from 'network missing' (real).
- webuiSystemNetworkCheck (webui_system_network.sh): self-throttled generator
that writes frontend/data/system/network_status.json (modelled on
verify_status.json). Wired into webuiSystemUpdate AND run unconditionally
every ~60s from the task-processor poll (regen webui is mtime-gated and
would never fire on drift, which touches no source file).
- networkHealConflicts (network_heal.sh) + 'libreportal system network
check|heal [app]': the heal adopts docker's subnet in-process, then re-IPs
stranded apps with reset_network=ip (ports preserved), gluetun first.
Mutating path runs only through the task system (dual-mode, like update
apply); read-only check runs inline.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- CFG_HOTFIX_AUTO (security-breakage|all|off, default security-breakage) seeded in
general_terminal; reaches existing installs via the add-only config reconciler.
- webui_artifact_scan.sh (webuiArtifactScan): fetch+verify the signed index, write
artifacts_available.json ATOMICALLY (build in temp → jq-validate → one write;
keep the prior file on any failure — never emits broken JSON). Annotates each
artifact with applied (a per-id record exists) + applicable (target installed).
- artifactApplyAuto + `libreportal artifact apply-auto`: enqueue apply tasks for
the eligible signed hotfixes — only when the index is VERIFIED-signed, only
auto==true + in the severity policy + applicable + not already applied. Each
apply is its own task (visible in the log + History), never applied inline.
- `updater check` now also refreshes the index (webuiArtifactScan) and runs
artifactApplyAuto — one front door, no second phone-home.
Unit-tested 13/13: policy filtering (security-breakage / off / all), auto:false
exclusion, already-applied skip, non-installed-app skip, unsigned-index fail-closed,
and the scan transform's signed/applied/applicable fields.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
A 4-lens adversarial security review of the Phase 2 applier raised 19 issues
and confirmed 17 after per-finding verification. All are trust-boundary (they
require the signing key), but several break the explicit "no code-exec, always
reversible, nothing-silent" contract, so all 17 are fixed:
Trust path — fail CLOSED, never misreport:
- lpFetchIndex now surfaces the real signature state (LP_INDEX_SIGSTATE);
artifactApply REFUSES to mutate unless the index is actually verified, and
_artifactFetchPayload refuses an unsigned payload. The read path still
tolerates dev/unsigned but now says "UNSIGNED" instead of "Signed + verified".
- valid_until and index_serial are now MANDATORY + numeric in lpFetchIndex
(missing = refuse) — closes the anti-withholding / anti-rollback fail-opens.
Injection / code-exec (defense in depth even for a signed payload):
- runFileWrite rootless branch no longer builds a `bash -c` shell string with the
destination interpolated — it uses the argv form (like runFileOp), so a path
with a quote can't inject a command as the install user. (shared-helper fix)
- op paths must match a safe-filename charset (no quotes/$/backtick/;/newline);
set-config-key values and set-compose-image refs are charset-guarded too.
- content_b64 is validated as real base64 at precheck.
Reversibility / honest failure:
- dockerComposeUp now returns the real compose exit status (it always returned 0,
so the updater's rollback gate AND the apply's start-failure detection were
fail-open). (shared-helper fix)
- set-config-key undo captures the WHOLE config file (lossless) instead of a
lossy re-parsed scalar; edit-only (rejects an absent key).
- _artifactReplayUndoFile returns non-zero if any inverse op fails; auto-rollback
and revert now record "rollback-incomplete"/"revert-incomplete" + isError
instead of falsely claiming success, and revert keeps the record for retry.
- applied-record write failure is checked — apply rolls back rather than leave an
un-revertable change. System-scope regen failure is no longer swallowed.
- Writes are path-aware (configs/ -> runInstallWrite, container tree ->
runFileWrite) so system-scope hotfixes write/restore correctly.
- Checked lazy-sourcing surfaces a clear error instead of a bare exit 127.
Unit-tested 35/35 (adds: command-sub value rejection, bad image-ref, invalid
base64, quote/metachar path-injection rejection, replay-failure reporting).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
librelad