The Storage step was a technical dump: every check's full sentence
concatenated onto the card, so the fstab line the user is meant to act on
was buried in prose nobody reads.
The card now shows plain facts and at most two short flags — "Low on
space · Won't be mounted after a reboot" — with everything else behind a
Details button. The modal carries the technical spec (device, UUID, mount
options, removable), every check with its full explanation, and the
fstab offer.
That needed the shell to stop joining checks into one string: the
generator emits a record per check, plus the fstab line as its own field,
so neither the card nor the modal has to parse anything back out of the
other.
The screenshot caught a bug this restructure introduced: summaries keyed
on check id alone, so a PASSING check printed the failure wording next to
a green tick — "This drive's format can't store file ownership" above
"Filesystem: ext4". Now severity-aware.
On writing /etc/fstab — §1 ruled it out and §6.3 now records why that
reverses. The warning is useless to the audience this is for: "add this
line to fstab" assumes SSH, root, an editor, and knowing what fstab is,
and the likely outcome is a reboot where nothing starts. What makes it
defensible is nofail + x-systemd.device-timeout, which mean a missing
device can never block boot — without that pair it would stay a non-goal,
because the failure being risked (an unbootable machine) is worse than
the one being fixed.
Enforced in the root helper: UUID never /dev/sdX, append inside a marked
block, refuse a target or UUID already described, refuse the root
filesystem, require a live mount, timestamped backup, and
`findmnt --verify` before the file is installed — a file that doesn't
parse never reaches /etc. Opt-in only.
Verified against a real filesystem: entry added and verifies, the
persistence warning then disappears on the next scan, and duplicate /
root-fs / non-mountpoint / relative are each refused with the reason.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
webuiGenerateStorageCandidates now runs as part of webuiSystemUpdate, so
frontend/data/storage.json exists without anyone remembering to generate
it — the wizard reads it to decide whether its Storage step appears, and
the Disks view reads the same file, so the two can never disagree.
Warnings arrive from the shell joined with "; ". Rendering that verbatim
produced one run-on paragraph that buried the fstab line the user is
supposed to copy, so the card splits them back onto separate lines.
Verified on the live install with lp-shot: with one filesystem the wizard
shows "Step 1 of 4" and the Storage step is correctly absent; with a
second filesystem attached it becomes "Step 4 of 5" with the drive
carrying a "needs care" badge and both warnings legible. That also
exercises the visible-step mapping in both directions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Phase 3 of docs/roadmap/storage-locations.md.
The step appears only when the candidate scan finds a filesystem
LibrePortal isn't already using, so the single-disk case — which is most
boxes — is completely unchanged. It sits before Recommended because a
location has to exist before an app can be placed on it.
Supporting two conditional steps meant the wizard could no longer treat
'position in the DOM' and 'step index' as the same number: Metrics was
advanced-only and got away with 'length minus one', but a step hidden in
the MIDDLE leaves a gap. Navigation, progress, validation and submit now
all run off _visibleSteps(), and section matching is by data-step rather
than DOM position.
Unusable candidates render greyed WITH the reason rather than being
filtered out — 'why isn't my drive listed?' is a support burden, and
'exFAT can't store file ownership' is actionable. The step is skipped
only when nothing usable was found at all.
What the wizard sends is a request, not an instruction: setupApplyConfig
feeds each path through storageAdd, so the fitness checks and the root
helper's admission rules both re-run regardless of what arrived in the
payload.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Phase 2 and 4 of docs/roadmap/storage-locations.md. Apps can now be
placed on a location and moved between them.
CFG_<APP>_STORAGE lands in all 37 app templates, holding a location NAME
rather than a path: names survive a migrate to a host with different
disks, paths do not. The 11 infrastructure apps that other apps reach by
literal path (traefik, prometheus, grafana, adguard, gluetun, crowdsec,
headscale, dashy, pihole, unbound, wireguard) are pinned. libreportal
itself never gets the key — it is pinned structurally by webuiDir.
Pinning needed no second config key. "Pinned" is not a fact about a value,
it is a statement about whether the field may be edited, so it goes in the
comment beside **ADVANCED** and **DEV** as **READONLY**, and the field
factory renders those disabled. That marker earns its keep beyond this
feature: derived fields already warned in prose that editing them does
nothing (crowdsec.config:72) next to a perfectly editable input.
storage_app_config.sh keeps the comment honest — it carries the resolved
path for hand-recovery and regenerates the dropdown from the registry, but
only writes when something actually changed, since the app .config is
user-editable and lives in the container-owned tree.
app move stops the app (a live copy of a running Postgres is a corrupt
copy), snapshots it, copies, verifies, and only then removes the source.
The copy runs in libreportal-ownership because it must: app data holds
rootless sub-UID files the manager can neither read nor recreate.
Verified against two real ext4 filesystems that a cross-device move
preserves uid 231141 and the payload, and that the source survives every
refusal path — unregistered destination, the WebUI app, a traversal in the
app name, and an occupied destination.
Task titles registered in both tables, with a specific rule so a move
renders as "Nextcloud - Move to bigdisk" rather than the generic fallback
dropping the destination. lp-task-names could not be run to confirm — it
borrows the WebUI container's node and no containers are running.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The main sweep — ~260 call sites across ~100 files move from string
concatenation on a single root to appDir/storageAppDirs/storageAppConfigs.
On a single-root install the resolved paths are identical, so this is a
no-op until a location is registered.
Enumerators were the interesting half. `for d in "$containers_dir"/*/`
appears in the menus, the registry/artifact scanners and the DNS setup —
and a shell glob cannot list a rootless 751 tree at all, which is the
same bug config_find_file.sh already documents in a comment. Routing them
through storageAppDirs (which enumerates as the owning user) fixes that
alongside the multi-root work.
Three places needed judgement rather than substitution:
db_app_scan.sh deletes database rows and port allocations for apps whose
folder is missing, and reaps "empty" app dirs. With a storage location
unmounted, every app on it looks exactly like that. Each of those
branches now gates on appStorageAvailable first — an app on an unplugged
drive is skipped with a notice, never deleted.
instance_create.sh rewrites cloned hooks so an instance touches its own
directory instead of the base app's. Its sed matched ${containers_dir}<type>,
which this sweep just replaced with $(appDir <type>) — so it would have
silently stopped redirecting, and an instance would have written to the
original's files (the adguard auth adapter case its own comment warns
about). Now matches both appDir forms, verified against bare, quoted,
unrelated-app, legacy and prose cases.
peer_shell/peer_pull streamed and extracted relative to the primary root.
Both now use the app's own root, and peer_shell keeps a single-root
fallback since it runs as a restricted SSH shell with no LibrePortal env.
Also fixes a pre-existing bug found on the way: webui_app_config.sh
tested "$containers_dir/frontend/data/last_update", one level short of the
real tree under the libreportal app dir, so the WebUI refresh trigger
after a config update has never once fired.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two mechanical sweeps, no behaviour change on a single-root install.
The 14 `[[ "$p" == "$containers_dir"* ]]` prefix tests that decide
manager-vs-container-user elevation become pathIsContainerData, so a file
on a second storage root is no longer misclassified as manager-owned —
which would have written it with the wrong owner and failed later, far
from the cause. The 65 references to the WebUI's own tree become
webuiDir(), which is pinned to the primary root by design.
Two traps found while doing it:
run_privileged.sh is sourced directly by init.sh without paths.sh, so it
needs a fallback. Defining one named pathIsContainerData was wrong:
generate_function_manifest.sh indexes top-level definitions, and the
resulting autoload stub would have shadowed the real multi-root
implementation with the primary-only fallback — silently classifying
every file on a second disk as manager-owned, which is exactly the bug
this sweep exists to prevent. Renamed to _runCfgIsContainerPath, which
delegates when the real one is loaded.
setup_lock.sh built its path in a top-level assignment, so it was
evaluated at source time and needed the file flagged eager. Made it a
function instead: the path resolves on call, and the file drops off
LP_EAGER_FILES entirely.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Swept the field mappings after the Updates one. Removed where the tooltip
carried no information the label did not already give:
PORT_1..20 "Port N for this application" label: "Port N"
PORTS "Port configuration for the label: "Port Configuration"
application"
CATEGORY "The category this application label: "Category"
belongs to"
THEME "Visual theme for the application" label: "Theme"
...PRIVATE_KEY "WireGuard private key" label: "WireGuard Private Key"
Deliberately kept several the crude word-overlap check also flagged, because
they earn their place: DOMAIN says the value is a number and why, HEADSCALE and
COMPOSE_FILE carry a requirement and a warning, VPN_TYPE says it depends on the
chosen provider, and DESCRIPTION/LONG_DESCRIPTION distinguish brief from
detailed — which is the only thing separating that pair on screen.
config-form.js already guards on the field having a tooltip, so a field without
one renders no help icon rather than an empty bubble. Confirmed on Bookstack's
config page: 25 icons left, none with an empty or "undefined" title, and no
stray "undefined" in the body text. Regenerated the served JSON too — 145 fields
before and after, 135 tooltips down to 111, and no field changed in any other way.
Worth a look separately: PORT_N holds the full pipe-delimited port descriptor,
not a port number, so "Port 1 for this application" was mildly misleading as
well as redundant. A tooltip explaining that format would be an improvement
rather than a deletion.
"Install new image builds automatically, or only when you press Update" spelled
out both options, which the select's own labels already do directly below it —
"Automatic (recommended)" and "Manual — I'll press Update". The tooltip now says
only what the setting is for.
The generated apps-field-mappings.json carries this string, so the live install
was regenerated rather than left showing the old copy.
The unmaintained warning runs on one field — when upstream last rebuilt
the image — and off-Hub apps had no value for it. Hub answers in a single
call; the OCI API does not expose it at all, so an app on ghcr.io, quay.io
or lscr.io simply could not be assessed for staleness, which is the one
signal a user cannot work out for themselves.
It is in the image, just further down: manifest -> (if a multi-arch
index) a platform manifest -> config blob, whose "created" is the build
time. Three requests instead of Hub's one, once per registry window, and
only for the apps Hub cannot answer for — which is why Hub keeps its
cheap path rather than being routed through this.
Index and single-arch manifests are distinguished explicitly rather than
by position: in an index the first digest is a CHILD manifest, in an
image manifest it is the config itself, so reading "the first digest"
would silently fetch the wrong blob for one of the two shapes.
Live: stoat 2026-08-08, bookstack 2026-08-17, speedtest 2026-08-16,
invidious 2026-08-05 — all previously null. Hub unchanged, navidrome
still answered by the single-call path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`docker ps -f name=<app>` is a SUBSTRING match, and instance slugs are
<type>_<id> — so the base app's name is a prefix of every instance of it.
`name=bookstack` also selected bookstack_home, bookstack_test and their -db
containers, which meant start, stop, restart and remove all silently operated on
every instance of an app instead of the one named.
Worst of the four is remove: `libreportal app remove bookstack` ran `docker rm`
against its instances' containers too. Multi-instance made this reachable — the
naming scheme it introduced is exactly what turns the base name into a prefix.
Each app and instance is already its own compose project, named for its
directory, so the project label addresses exactly the containers belonging to
that app. app_install.sh's own post-install check already used this label; the
lifecycle operations did not.
Found while tracing the IP allocation problem: bookstack_work had vanished, and
checking how uninstall selects containers turned this up. To be clear about
attribution — this bug does NOT explain that disappearance. The log shows an
explicit uninstall of bookstack_work, including its own install folder and log,
which container-level over-matching cannot do. I could not attribute that
removal to a specific command and am not going to guess; the instance has been
recreated.
Verified: with the fix, `libreportal app stop bookstack` stops bookstack and
bookstack-db and leaves bookstack_home and bookstack_test running. Before it,
all six went down. All four Bookstack apps and Stoat serve 200 afterwards.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Version discovery spoke only hub.docker.com, and every other registry
got a shrug: updaterTagExists returned "no" and updaterRegistryTags
returned nothing. Five apps live off Hub — stoat and wireguard on
ghcr.io, bookstack and speedtest on lscr.io, invidious on quay.io — and
for all of them the updater reported "up to date" having never asked.
That is the same dishonesty as a scan that never ran: an absence of
evidence rendered as a clean bill of health.
There was never a barrier, only unwritten code. The standard
Distribution API needs one extra step: request, read the
WWW-Authenticate challenge, fetch a token from the realm it names,
retry. ghcr.io, quay.io and lscr.io all answer anonymously for public
images — lscr.io by pointing its realm at ghcr.io, quay.io by not
challenging at all.
Docker Hub deliberately keeps its own path. hub.docker.com returns tags
NEWEST-first, so the 100 it pages are the 100 that matter, and it draws
on a different budget from the pull limit — registry-1.docker.io
manifest reads count against the anonymous 100/hour that the updater
needs for actual pulls, and a ladder probes a tag per rung.
Tag LISTING off Hub is a weaker signal and the comment says so: /v2/
tags/list is lexical, not newest-first, and large repos cap the page, so
the newest release can legitimately be absent. Probing backfills it,
which is why the probe fallback added earlier matters more off Hub than
on it.
Verified against all four registries: existence probing correct on eight
cases including true negatives; stoat climbs v0.15.0 -> v0.15.1 through
ghcr.io, and correctly reports nothing above v0.15.1 — the same answer
as before, but now because it looked. Hub unregressed: matrix still
resolves v1.158.0 -> v1.159.0 and nextcloud still ladders 31 -> 32 33 34.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four faults, all in the same 12-column format, all silent.
The bash parser split with `local parts=(${value//|/ })` — replacing
pipes with spaces and word-splitting. That broke the format two ways at
once: a label containing a space became several fields, and an EMPTY
column collapsed rather than being kept, shifting everything after it.
Stoat's LiveKit row parsed as label "LiveKit", url_path "voice/video",
subdomain "(TCP", recommended "fallback)". Rocket.Chat's subdomain only
landed correctly because the extra label word and the collapsed empty
column happened to cancel out. The column COUNT was wrong too, so the
9/8/7-col compatibility branches were chosen from an inflated number.
Now an IFS read, which keeps empties and never word-splits.
The port editor had two serialisers and they disagreed. buildPortConfig
writes all twelve columns; updateIndividualPortFields wrote ten, dropping
subdomain and recommended — so saving ANY port on an app silently
discarded that app's Traefik subdomain. That is how Stoat's live config
came to differ from its template, which still had "stoat".
Both readers gated the subdomain on twelve columns, but subdomain IS
column eleven — so the canonical 11-column descriptor every web app
ships never surfaced one. The bash side reads it from nine.
Lastly, findMatchingCFGKey could not see a generated-value slot suffix.
Passwords LibrePortal generates are stored as CFG_<APP>_<NAME>_<n>, and
ADMIN_PASSWORD_1 neither equals ADMIN_PASSWORD nor ends with
"_ADMIN_PASSWORD", so a generic mapping matched an app's admin EMAIL and
missed its admin PASSWORD entirely: the field simply never rendered
unless someone had hand-written a per-app mapping. Now resolved as a
last resort, after every exact and whole-word match has failed, lowest
slot first. Plus a generic ADMIN_USERNAME mapping, since ADMIN_USER is a
different field name and correctly does not match it.
Audited all 74 port descriptors across the catalogue: none are
malformed. 39 sit at 9 columns, which is a documented, supported shape
(url_path/subdomain empty, recommended defaulting to the webui flag) and
they are all non-Traefik ports — DNS, SMTP, WireGuard UDP.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two halves: the ladder could not climb the commonest versioning scheme,
and nothing ever climbed it on its own.
The ladder stepped by bumping a tag's LAST numeric component, so
v1.158.0 went v1.158.1, v1.158.2, … and never arrived at v1.159.0. It
then failed closed, refusing to build a path. Synapse publishes
v1.159.0 and no v1.158.1 at all, so Matrix could not be laddered by the
button either — three-part semver minor bumps were simply unreachable.
updaterNextRung now considers a bump of every component, keeps the
candidates that exist upstream and takes the smallest: the next release
by definition, whether it lands in the patch position or crosses into a
new major. Shape discipline is unchanged, so 31-fpm-alpine still never
becomes 31-apache, and each rung is still probed, so none can be
skipped. updaterTagBumpAt moves here from the scan, its natural home,
which also breaks a source cycle.
updaterUpgradeAuto then climbs at most ONE rung per app per calendar
day, inside the install window, for apps set to auto. One rung because a
ladder run unattended can be several migrations deep before anyone
looks, and "restore the snapshot from a minute ago" stops comforting
once four have stacked; one a day so there is time to notice. It crosses
a major if that is genuinely the next release — refusing would strand an
app on the last version of its line forever — but one step at a time,
never as a leap. Two stamps: the target rung (a failure is not retried
until something newer ships) and the day.
Every rung goes through updaterUpgradeApp unchanged, so GATE 1 still
refuses any app without a real verifier, and the per-rung contract is
identical to the button: snapshot fail-closed, set version, pull, up,
verify, restore that rung and stop on any failure. History now records
the trigger instead of hardcoding "manual", including on the rollback
paths. CFG_UPDATER_LADDER_AUTO gates the whole thing separately from
CFG_UPDATER_AUTO, because "keep my apps patched" and "move my apps
between versions unattended" are different appetites for risk.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Audit of per-app hooks/tools found 19 of 33 apps whose helpers would have
operated on the BASE app after cloning. Two general causes, both fixed by
rewriting classes rather than patching apps:
- Container references escaped the rewrite whenever a flag sat between the
docker verb and the target (`docker exec -u git gitea-service …`), since the
old rule only matched a name immediately after the verb — and the hyphenated
form missed the `<type>_` rule too. Hook trees now get the same discovered
identity rename the compose does, reading names from the TYPE's compose since
the clone has already been rewritten by then. Safe to apply broadly: the
compose pass runs first and aborts for any app whose identities aren't
<type>-prefixed, so a bare word like stoat's `api` never reaches it.
- Hooks that build the deployed path as "${containers_dir}<type>/..." instead
of "$containers_dir$app_name/..." read and WROTE the base app's files —
adguard's auth adapter edits AdGuardHome.yaml, so an instance would have
rewritten the original's config. The trailing slash is optional in the match:
dashy tests [[ -d "${containers_dir}dashy" ]] and gluetun cds into it, both
ending at the quote. Only the first path component is touched, so
${containers_dir}prometheus/prometheus/... keeps its inner segment.
Re-audit: all 33 apps with hook trees are clean. Stoat still leaks, but it is
refused at the compose stage and never reaches this code.
Volumes audited too, and need no changes: no app uses named volumes, so the
./relative bind mounts every app uses resolve inside each instance's own
deployed dir. The absolute sources that exist are host or in-container paths
correctly shared read-only (/etc/localtime, /sys, /etc/ssl/certs). Jitsi's
${CONFIG} is set per-app by its own hook to $containers_dir$app_name/... and so
follows the slug.
Bookstack's rewritten tool tree is byte-identical to the live instance's across
all 8 files, so the running instances are unaffected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stoat wore MongoDB's identity. Its services are named database / api /
events / …, so there is no stoat-service for updaterPrimaryImage to
match, and the fallback took the FIRST image line — mongo:8.0. Every
downstream fact inherited that: the app's version read 8.0 instead of
v0.15.1, its "8.3 available" chip was a MongoDB major dressed as a Stoat
release, its CVE scan covered mongo and none of the nine Stoat images,
and pressing Upgrade would have laddered the database 8.0 -> 8.3 beneath
a live sixteen-service stack.
The compose already says which image is the app's: every image line
carries a #LIBREPORTAL|<KEY>_VERSION_TAG| marker, and the one keyed on
the BARE app name (STOAT_VERSION_TAG, not STOAT_MONGO_VERSION_TAG) is by
construction the app's own version. 37 of 38 apps have exactly one; only
libreportal lacks it, and the scan skips that app anyway.
Ask the sentinel first, keep <slug>-service and first-line as fallbacks.
Verified across the catalogue: identical anchor for every app except
stoat, which is corrected. This is the ollama mislabel of P0 recurring
through a different hole — positional guessing — closed with the
metadata that was already there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Found while installing two LAN-only Bookstack instances — both in the same
no-domain path as the previous commit:
- apps-services.json advertised every app at http://localhost:<port>. The
CFG_SERVER_IP override it reads is defined in no config file, so the lookup
always fell through to the "localhost" default — a URL that only resolves for
someone browsing on the server itself. Now falls back to $local_ip_v4, the
same host APP_URL is stamped with.
- traefikManaged was inferred from `access == public`, a stated placeholder.
Public only means the port is published on the host; it says nothing about a
router. It reported true for both new instances despite their compose having
traefik.enable:false. Now read from the port's own traefik column, gated on
the app's domain actually being set — resolved per-app here rather than from
$domain_full, which this generator never populates.
- The "New instance" modal led with bare text. It now shows the type's icon in
the same .app-card-icon holder the grid cards use, so it's visually tied to
the app the user clicked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two display bugs the newer-version work made visible.
A fleet row showed a green "✓ Up to date" directly beside a
"1.159.0 available" chip. Both statements are individually true — you
ARE current on the line you track, and changing lines is a deliberate
act — but a row is a glance, not a place to reconcile two chips that
appear to disagree. The green all-clear now gives way to a neutral
"Newer version" whenever a newer release line exists; the chip still
carries the number and the tooltip still explains the move. The per-app
detail deliberately keeps "up to date" and is left alone: there the
badge arrives with a sentence explaining the distinction and an Upgrade
button, which is what makes it readable.
updaterDisplayVersion preferred the OCI version label unconditionally.
That label is inherited from the vendor's base image unless they
overwrite it, so it can describe the OS rather than the app: mongo:8.0
carries org.opencontainers.image.version=24.04, its Ubuntu base, and
Stoat's row read "24.04 → 8.0 · 02a0cc7" — not a version transition at
all. When the tracked tag is versioned we already hold an authoritative
version, so the label now wins only if the two agree on their leading
number. Keeps nextcloud 34 → 34.0.1, rejects mongo 8.0 vs 24.04, and
leaves rolling tags untouched since the label is the whole point there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
updaterNewerVersionByProbe guarded on updaterTagExists being defined
and gave up when it was not. That function lives in the ladder, and a
cross-file function is not reliably loaded in the generator's context —
updaterAppPolicy a few lines below already carries an explicit source
fallback for exactly this. Without one the probe silently did nothing,
which is the failure mode it was added to remove.
Source the ladder when the function is absent, matching the existing
idiom. Verified by calling the probe with updaterTagExists undefined:
it now loads the ladder and returns v1.159.0 for matrixdotorg/synapse
instead of an empty string.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
footprint_update_needed only ever came from comparing the installed marker
against a channel manifest, so it could not fire on a git or local install —
they have no channel to ask. Those are exactly the installs whose code tree is
synced by hand, i.e. the ones most able to drift, and the drift was silent: the
helpers in /usr/local/lib/libreportal could sit behind the code that calls them
with nothing reporting it. That is how this box ended up running a crowdsec
helper with no bouncer-traefik-rotate action while the tool that needs it
shipped.
init.sh is what bakes the marker, so the install tree's own init.sh is
authoritative for every mode. lpInstallTreeFootprintVersion reads it and
lpFootprintStale compares. Wired into both non-release branches of the WebUI
status generator, and into the local branch of the interactive update check,
which is where a local operator actually looks.
Fails safe: a tree older than the marker, or a missing init.sh, reports current
rather than warning — verified alongside the real stale case.
Also gives webuiSystemUpdateCheck the self-reload guard webuiGenerateAppsToolsConfig
already documents. The WebUI task service sources these once at startup, so
without it an edited generator keeps writing the old JSON from memory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Running any tool jumped to the Tasks tab and left the user stranded there. That
is right for an install — long, log-heavy, worth watching — and wrong for a
tool, which is a short admin action whose answer is one line. Worse, half of
these are only meaningful back on Tools: List Users opens a modal over that tab,
and Create User Account returns a generated password that was being buried in a
log the user then had to go read.
Tools now stay put. On completion the tool's own outcome lines — the
isSuccessful/isError/isNotice output, ANSI stripped and framework boilerplate
filtered — are shown in a small result modal, with a View log button for
anything needing the full detail. list_users is left alone because the existing
account-list modal is already a better result view.
Also stops generate_arrays.sh walking scripts/dev. That directory is
`export-ignore`d, so it exists in a working clone but never in a shipped
install; generating a files_dev.sh entry from it wrote a reference into
files_source.sh that no install could satisfy, and the loader treats a missing
array file as a broken installation — every libreportal command stopped with
"files_dev.sh is missing from your LibrePortal Installation". Excluded alongside
unused/, system/ and release/. Regenerating also picked up scripts/validation,
which had never had an array file.
And Matrix's account listing prints its aligned line from python rather than
re-splitting the marker line in bash: TAB is IFS whitespace, so an empty display
name collapsed into the previous delimiter and shifted every later column.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two holes that together left a versioned app reporting "up to date"
while a newer release was published.
Newer-version discovery enumerated a repo's newest 100 tags. Projects
that push a tag per commit drown their own releases in that window —
matrixdotorg/synapse's newest 100 hold five version tags, about ten
days of history. Once the release we need is older than the window it
is simply absent, and the app reports current forever. The failure is
silent and lands hardest on the apps furthest behind. Discovery now
falls back to PROBING exact tags, most-significant component first,
which has no window at all. Listing still runs first, so the common
case stays at one call; probing is bounded at 40 lookups. Same
reasoning the version ladder already uses, for the same reason.
Registry lookups were also throttled purely per-run, so an app
installed just after a window carried an empty available_digest until
the next one — up to CFG_UPDATER_REGISTRY_INTERVAL (6h) later. Empty
means update_available=false, which the UI renders as "up to date", so
a new app claimed to be current on no evidence. Seen live: seven apps
installed the evening after a 19:31 window all sat at
update_available=false, one of them two releases behind. Apps with no
prior registry answer are now looked up regardless of the throttle —
once each, and interval 0 still means manual-only.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Answers "should we stop creating an admin/pass on start" with the split the
catalog actually has, rather than one way for everything.
Ten apps need it: adguard, authelia, bookstack, matrix, nextcloud, owncloud,
pihole, rocketchat, stalwart, speedtest and headscale either pass the generated
password into the container or hand it to an install hook that creates the
account. There the password IS the working credential — dropping it would lock
you out. Left alone.
Three do not create an account at all: gitea, invidious and mattermost seed no
user (the first one comes from their own signup flow or the Create Account tool),
so the password minted at install named nothing. The WebUI credentials card
showed a password that could not log in. They now match linkding — an empty,
unslotted ADMIN_PASSWORD the auth adapter fills when the operator makes the first
admin, and keeps in step on later resets. Unslotted because the slot number marks
a value the installer generates.
mattermost's adapter also had linkding's bug: it persists ADMIN_PASSWORD but the
config declared only ADMIN_EMAIL, so the write was a no-op.
WebUI: rocketchat's generated admin password had no field mapping, so the card
could not show it. Added, plus a generic ADMIN_USER entry — six apps record an
admin username the card had no way to display.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Only two of the four keys flagged as unused actually were. gitea and invidious
ADMIN_PASSWORD are written by their auth adapters through authPersistCfg, which
builds the name as CFG_${app^^}_${key} from a parameter — invisible to a literal
grep, which is why the earlier pass called them dead. They stay.
Worse, the slot rename broke that write path for five apps: adguard, bookstack,
gitea, invidious and nextcloud all persist ADMIN_PASSWORD, and the config now
holds ADMIN_PASSWORD_1. updateConfigOption only rewrites a key that already
exists, so the write became a no-op — the app's password would really change
while the config and the WebUI kept showing the old one.
authPersistCfg now falls back to the numbered slot when the bare key is absent,
so adapters never need to know how a credential is numbered and adding a slot
can't silently disconnect the adapter that writes it. When neither name exists
it warns and returns non-zero instead of failing silently, which surfaces a
pre-existing case: linkding's adapter persists ADMIN_USER and ADMIN_PASSWORD but
its config declares neither, and never did.
Deleted the two that really are dead: CFG_TRAEFIK_ADMIN_PASSWORD_1 (its adapter
uses CFG_TRAEFIK_USER/CFG_TRAEFIK_PASS from the system config) and
CFG_GLUETUN_CONTROL_SERVER_API_KEY_1, plus their WebUI field mappings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Matches b562059 — the fourteen mapping entries added alongside the slot rename
were written before that landed and ran 88-100 chars against a median of 44.
Makes the convention uniform: if a config key holds a generated value, its name
ends in a slot number. 42 keys across the catalog, up from the 9 database ones
done previously — admin passwords, app keys, tokens, HMAC and auth secrets,
generated usernames and database names. An app needing a second credential of a
kind now just adds _2; nothing is registered anywhere, since the tag name is
derived from the key by tags_processor_app_config_values.
Keys holding an operator-chosen value (CFG_NEXTCLOUD_ADMIN_USER=admin) keep their
names — the slot number is what marks a value as generated.
The rename would have silently cost seven keys their WebUI field mapping. The
frontend resolver matches a mapping key against a config key by equality, _suffix
or prefix_ (apps-manager.js findMatchingCFGKey), so the generic "ADMIN_PASSWORD"
entry stops matching CFG_GITEA_ADMIN_PASSWORD_1 — it neither ends with
_ADMIN_PASSWORD nor starts with ADMIN_PASSWORD_. Rather than loosen the matcher
(PORT_1 relies on its numeric suffix being part of the name), add explicit
entries. Did the same for eight keys that were already unmapped before this
change, so all 42 now render with a label and, where appropriate, masked: the
only one typed as text is Mastodon's VAPID public key, which is public by design.
Verified by simulating the resolver against every app config, and by running each
app in the catalog through fill -> hook -> templating: every secret tag
substitutes, no RANDOMIZED placeholder survives, every compose still parses.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Installing rocketchat failed with
invalid IPv4 address: ParseAddr("IP_DATA_2"): unable to parse IP
ipUpdateComposeTags allocates one IP per SERVICE_TAG_N annotation and fills
IP_TAG_i only where SERVICE_TAG_i exists. The four new apps tagged only their
primary service, so every sidecar — matrix's postgres, mattermost's postgres,
rocketchat's mongo, and fifteen of stoat's sixteen — kept a literal IP_DATA_n
in the deployed compose and docker refused to create the container.
Tag every service that carries an ipv4_address, index-aligned with its IP_TAG.
For stoat that also meant moving caddy from SERVICE_TAG_1 to _6 so the indices
line up with the IPs rather than the reading order.
mastodon had the same latent break (IP_TAG_2 and _3 untagged) and is fixed the
same way — it would have failed on first install for the same reason.
SERVICE_TAG carries the compose *key*, not container_name: 'libreportal app
restart <app> <service>' passes it to 'docker compose restart', which only
understands keys.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Cut the tooltips that had grown into paragraphs (backup strategy,
version, monitoring, DB/secret fields, Dashy shortcuts) down to a
single line, matching the concise style of the rest of the file.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
VAPID: the two values are the halves of one P-256 keypair, not independent
secrets — the browser verifies that a push is signed by the private key matching
the public key it subscribed with. The RANDOMIZED* generators mint each
placeholder on its own, so they produced two unrelated strings and web push could
never have worked. Generate the pair in mastodon_install_post_setup the way stoat
already does, encoded as Mastodon's webpush gem expects: unpadded URL-safe base64
of the 32-byte private scalar and the 65-byte uncompressed public point, sliced
out of the SEC1 DER. Verified by rebuilding the key from the emitted private half
and re-deriving the public point — openssl accepts it and the point matches.
Generated once and never rotated (rotation would invalidate every subscription),
but a pair of the wrong shape is replaced, so an install carrying the old
unrelated strings heals itself on next install — their public half is 42 chars
where a real point is 87.
Slots: CFG_<APP>_DB_PASSWORD -> CFG_<APP>_DB_PASSWORD_1 and likewise for
DB_ROOT_PASSWORD, across mastodon, owncloud, mattermost, matrix, nextcloud and
bookstack, so a database credential is always a numbered slot and a second one is
just _2. Renaming a key means reconciliation drops the old and adds the new
holding its placeholder, so an existing install regenerates unless the value is
carried over first — documented, including that the old file survives as
.<app>.config.bak.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Five apps (mastodon, owncloud, mattermost, matrix, stoat) took their generated
secrets from the compose-side generator tags PASSWORD_TAG_<n>/RANDOM_TAG_<n>/
HEX_TAG_<n>/VAPID_TAG_<n>. Those mint a fresh secret on every templating run, so
a reinstall handed the app a new database password while its data volume kept the
one initdb was given, and the app came back up unable to open its own database.
Move them to <app>.config as RANDOMIZED* placeholders, reaching the compose via
the #LIBREPORTAL|<APP>_<KEY>_TAG| mechanism tags_processor_app_config_values
already provides. No new handler: the tag name is derived from the config key, so
this is a config line plus a tag per secret. Generation is unchanged — still
random on first install; the value is now remembered instead of re-rolled.
Also fixes two things this exposed:
- The RANDOMIZED* replacers matched unanchored. `sort -u` orders slots lexically
(1, 10, 11, 2), so slot 1's pattern rewrote the prefix inside slot 10's
placeholder and slots 10+ ended up holding slot 1's secret with a digit glued
on — derivable, and invisible because the values weren't byte-identical.
Anchoring with \b makes match order irrelevant. Verified at 20 slots across
all four placeholder types: 64 keys, 64 distinct values, no prefix collisions.
- generateRandomPassword drew from base64 without constraining the mix; measured
over 2000 draws, 1 in 40 contained no digit at all. Retry until the result has
both a digit and a letter, bounded so a pathological length can't spin.
owncloud gains a fix in passing: its compose seeded the admin account from
PASSWORD_TAG_2 while the WebUI displayed CFG_OWNCLOUD_ADMIN_PASSWORD, which was
generated separately and never used. Both now read the same value.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Match the concise style of the other field tooltips instead of
explaining the whole update/rollback flow.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Removing Focalboard from the catalogue left its icon still being served:
the sync only ever ADDS, so every app ever dropped leaves a file behind
that the portal keeps offering for something that is gone. Same shape as
the task queue that only ever appended.
webuiPruneAppIcons runs at the end of the sync and removes only icons it
can match to a missing template — anything else in the directory is left
alone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"Up to date" answers one question — has the tag I track moved? — and an
abandoned project answers it reassuringly forever. The tag stays put, the
digest never changes, and the app reports as current while receiving no
security patches at all. Nothing in the UI could tell a healthy stable
app from a dead one.
An audit of all 34 anchor images found five in exactly that state:
speedtest (4.4y since rebuild), focalboard (2.8y — Mattermost dropped
support in 2023), pihole-unbound (2.3y), trilium (2.2y), unbound (1.8y).
The scan now records image_updated_at per app (one cheap Hub call inside
the existing registry window, cached between windows like everything
else) and emits stale_after_days from CFG_UPDATER_STALE_DAYS (365, 0
disables) so the UI and the config agree on one number.
Surfaced as an "unmaintained?" severity chip on the fleet row and a
dated explanation in the app detail. Phrased as an observation rather
than an accusation — plenty of small tools are simply finished — but it
does spell out the security consequence, because that is the part a user
cannot infer from "up to date".
Deliberately NOT a "needs action" row on the Overview board: it is not
fixable by pressing anything, and a permanently amber board teaches
people to ignore the board.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The registry helpers rejected any repo containing a dotted host segment,
which caught 'docker.io/authelia/authelia' — Docker Hub spelled out in
full. Those apps were silently skipped by tag enumeration and version
laddering. Strip the docker.io/ and index.docker.io/ prefixes before the
host check; genuinely third-party registries (ghcr.io, quay.io, lscr.io)
are still correctly skipped.
Found by auditing every app's anchor image.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The digest compare only ever asks about the tag already pinned, so it
answers "has my tag been rebuilt?" and can never answer "does a newer
version exist?". An app on v0.16 reports up to date forever while 0.17
ships. That is the gap between an app that updates and an app that is
current, and it silently affects every pinned app.
Adds tag enumeration for VERSIONED tags only (rolling tags already move
on their own): list the repo's tags, keep those sharing the current tag's
SHAPE, and pick the numerically greatest.
Shape matching is the whole safety story — v0.16 -> v#.# so it can never
"upgrade" you onto v0.16-alpine, 31-fpm-alpine onto 31-apache, or a date
tag onto a semver one. Comparison is component-wise numeric, so 0.10 > 0.9
and 1.0 > 0.99 (a string sort gets both wrong), with 10# forcing base ten
so an upstream "08" cannot be read as octal. 15 unit tests cover it.
Docker Hub only, deliberately: all three pinned apps live there, it needs
no auth, and the generic OCI tags/list wants a per-registry token dance.
Other registries stay quiet rather than guess. Throttled inside the
existing registry window and cached between windows so it cannot flicker.
Surfaced as INFORMATION, never an action: no button applies it, because a
version move can carry a data migration. `update_available` and the "up
to date" badge keep their exact meaning; the new state sits beside them
and points at the Version field.
Against the live registry: stalwart v0.16 is current, nextcloud is on
31-fpm-alpine with 34-fpm-alpine out, mastodon on v4.2.0 with v4.6.5 out.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It sets the image tag, so a wrong value stops the app starting — that
belongs with the other expert settings, not beside feature toggles.
Tooltip now explains the split it participates in: automatic updates
apply rebuilds OF this version, changing it moves between releases.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four fixes that make the auto-updater a trustworthy background system:
* CFG_UPDATER_WINDOW (default 06:00-08:00 host time, right after the
05:00 backup cron; HH:MM-HH:MM wraps midnight, 'always' = any time).
Gates only the enqueue — scans keep running all day, so the Updates
page stays current and pending updates visibly wait for the window.
Malformed values fail closed and are rejected by the WebUI validator.
* "Check now" actually checks: an explicit `updater check` sets
UPDATER_REGISTRY_FORCE=1. The flag existed but nothing ever set it,
so the button silently reused the 6h digest cache and could not find
a build the user knew had shipped. Force also overrides interval 0,
which now means "manual-only" as documented in the roadmap.
* Registry stamp moved from /tmp to <system>/logs: the task processor
runs under PrivateTmp, so daemon and CLI each kept a separate 6h
clock and the daemon's reset on every service restart.
* A failed automatic attempt is no longer invisible: the scan emits
auto_attempted_digest (the one-shot no-retry stamp), and when it
matches the available build the UI stops promising an install that
will never come — per-app detail explains, the fleet row gets an
"auto failed" chip, and the Overview board counts it as needing you.
Also corrects the CFG_TIMEZONE label: it sets the containers' TZ only;
scheduled tasks follow the host clock (timedatectl), and the old
"Timezone for scheduled tasks" wording promised a knob that never
existed. The window + auto_window display state plainly WHEN updates
land, answering "how does the user know when the next update happens".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
First real end-to-end auto-update on a live install failed like this:
Automatically updating trivy (a recovery snapshot is taken first)
Snapshotting trivy before update…
Pulling new image(s) for …
Update of failed — rolling back…
Could not roll back automatically
The app name went empty after the snapshot. Cause: bash is dynamically
scoped, so a callee assigning an undeclared variable writes the CALLER's
local of that name — and a `while read app` loop leaves it EMPTY at EOF.
webuiBackupAppStatus's dashboard generator runs at the end of every backup
and did exactly that to updaterApplyApp's `app`.
Nothing was damaged: the pull ran against an empty name, failed before
touching the image, and the rollback was a no-op on a nonexistent app.
Fixed both ends. The generator (and three gluetun loops with the same
latent leak) now declare `local app`. updaterApplyApp/updaterRollbackApp
hold the name in `_upd_app` so they no longer depend on every callee's
hygiene, and updaterApplyAll stops leaking its own loop var.
This is exactly the untested path the roadmap flagged: "apply/revert not
yet exercised end-to-end on a live install with a pending update."
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds the decision half of the app updater. Detection (P2) and the
snapshot-first apply/revert (P3) were already real, but nothing ever
pressed the button — every update waited for a click.
CFG_<APP>_UPDATE_TYPE=auto|manual per app, default auto (33 templates)
CFG_UPDATER_AUTO=true|false master switch, default true
updaterAppPolicy resolves the two the way backupResolveStrategy already
resolves backup strategy: the global switch can only make things more
manual. updaterApplyAuto runs at the end of `updater check` and enqueues
the ordinary updater_apply task for each auto app that has an update —
never applies inline, so an automatic update is the same code path, task
log, History entry and Roll back button as a manual one.
Safety: each attempt stamps its target digest under generated/auto/, so a
build that fails is rolled back and then left alone rather than retried on
every scan; in-flight updater tasks are skipped so scans can't stack.
Tracked end to end: updates.json carries each app's resolved update_type,
History entries carry trigger=manual|auto. The WebUI says whether updates
install themselves, chips only the apps that opted out, labels automatic
history, and — since an auto app's pending update needs no decision — keeps
it off the Overview board's "Needs action" view.
Also fixes artifactApplyAuto enqueueing without --detach: it runs inside
the single-threaded task processor's own poll, so following the new task in
the foreground waits for a task that cannot start until it returns.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The per-app Updates tab pushed a manual "Check" button (header + empty state)
even though scans run automatically on CFG_UPDATER_SCAN_INTERVAL — so an app with
nothing to update read like an empty/actionable page. Replace the manual Check
with a calm status line inside the panel: "Checked automatically · last checked X
· next check ~Y", backup-schedule style. The genuine Apply/Roll back actions stay
(applying is still manual and safe). No auto-apply.
- webui_updater_scan.sh: stamp scan_interval_minutes alongside generated_at in
updates.json so the display needs no separate config fetch (0 = auto off).
- updater-page.js: renderAutoCheckLine() + fmtRelFuture().
- app-tabbed-manager.js: drop the header/empty-state Check buttons; render the
auto-check line; friendlier no-data copy.
- overview.css: style .updater-autocheck (green dot live / muted when off).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The per-app object build used `available_version:($available_version|select(.!=""))`
to omit the field when empty. But in jq, a `{key: (empty)}` makes the WHOLE
object construction emit nothing — so every app with no available update (empty
available_version) produced no object and was silently dropped from updates.json.
Only apps WITH a pending update survived; an all-up-to-date fleet showed an empty
list. (Missed in P2's sandbox test because both fixture apps had updates.)
Emit an explicit null instead: `(if $available_version=="" then null else … end)`.
Verified: an up-to-date app (trivy, local==registry digest) now emits with
available_version:null; apps with updates still carry the string.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
webuiUpdaterScan now tells the truth instead of hardcoding update_available=false.
Per app it resolves three facts (see roadmap §2): the running RepoDigest (one
`docker inspect` of the anchor image), the display version (OCI label →
versioned tag → channel·shortdigest), and — throttled — the registry's current
index digest for the channel tag (`docker buildx imagetools inspect`, the same
identity as RepoDigest, verified exact). update_available = the two digests
differ. Emits type (versioned|rolling), channel, current/available digests +
versions, and a services[] array (every image line, anchor flagged).
- Registry lookups throttled separately from the scan: CFG_UPDATER_REGISTRY_
INTERVAL (min, default 360; 0 = local-only), own /tmp stamp, reuse of the
prior available_digest between windows so the app list still refreshes every
scan. UPDATER_REGISTRY_FORCE=1 forces a live pull (the Check-now button).
Registry failure (offline/rate-limited) = "unknown", never a false "changed".
- Digest-compare fully detects rolling apps' new builds; for versioned apps it
catches rebuilds of the pinned tag (newer-version enumeration is a later
step) — honest per type, and versioned apps are user-picked via P1b anyway.
- Fixes a P1b regression: updaterPrimaryImage now strips the trailing version
sentinel comment (`s/ #.*//`) via _updaterCleanImageRef — without this the
anchor ref (and the live CVE scanner's image arg) carried the comment.
- JSON built with jq for safe escaping; jq-less fallback keeps output valid.
Verified via a simulated 2-app install: navidrome → "0.62.0" (OCI label) +
detected update; rolling app → channel·shortdigest + update; throttled re-run
reuses the prior digest; all output valid JSON.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
"Control plane" is jargon. Rename the self-heal task and de-jargon all the
user-facing copy: task titles ("LibrePortal - Fix System Issues" / "System
Health Check"), the action label, the badge/banner/panel text and "Fix now"
button, the status summaries, and the heal task-log messages. Behaviour
unchanged; code comments keep the technical term where accurate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The scan read the app's "current image" via `grep -m1 image:` — the first
image line. For apps that declare a companion service first this mislabels the
app: ollama lists `ollama-webui` (open-webui) before `ollama-service`
(ollama/ollama), so an app named ollama reported open-webui's version.
Add `updaterPrimaryImage`: resolve the version anchor from the app's
`<slug>-service` image (the universal primary-service naming convention,
33/33 apps; underscores→hyphens for slugs like libreportal_catalog). Falls back
to the first image line off-convention. Used by both the version and CVE loops.
Verified: only ollama changes (→ ollama/ollama:latest); nextcloud, mastodon,
jitsi, gitea, vaultwarden anchors unchanged.
Also revise docs/roadmap/app-version-updater-and-cve.md to the config-first
direction agreed this session:
- CFG_<APP>_VERSION ADOPTED (was rejected): the #LIBREPORTAL tagging system
makes the config the source (compose tag derived from it), not a second one.
- Two version TYPES — versioned (real tag, version picker) vs rolling (floating
channel, digest-freshness); digest is the uniform detection engine for both,
version numbers are display enrichment. Answers "why not just compare numbers"
(most upstreams publish none; no universal latest-version API).
- Multi-service anchor = <slug>-service, not first line; lock-step sets (jitsi)
= one channel → several image lines; sidecars tracked-by-digest, not headline.
- Phases updated: P0 anchor (done) → P1 config-first pin → P2 detection+display.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Crash-loop detection keyed on `docker ps --filter status=restarting`, but a
backed-off loop sits "exited" between restarts once docker's backoff grows to
tens of seconds — so a slowed loop is missed. Detect via RestartCount CLIMBING
between scans (what a crash loop actually is), unioned with the instantaneous
restarting signal for fast loops. Baseline counts persist in
.health_restart_counts, written only by the throttled check so the heal's
re-scans don't disturb the delta.
Also give the system_health_heal / system_network_heal tasks proper display
(they fell through to the raw command + generic ⚙️): friendly titles in
formatCommandForUser, type icons (🩺 / 🌐) in getTaskTypeIcon, and action
labels in formatActionTitle — so they read as "LibrePortal - Repair Control
Plane" etc. with an icon, like install tasks.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
An offline trivy install crash-looped (server FATALs when it can't fetch the
vuln DB), and on rootless docker the restart storm churned the shared network's
port-forwarder until the WebUI's own published host port was torn down — the
WebUI stayed healthy INSIDE its container but was unreachable from the host, with
nothing detecting or healing it.
Three fixes, in the house self-healing style (mirrors the network-drift trio):
1. Control-plane health checker wired into the existing task-processor idle poll
(maybeRegenPoll), no new daemon. dockerHealthScan (read-only) detects daemon
down, a WebUI running-but-host-port-unreachable (the port-forward corruption),
and crash-looping containers. webuiSystemHealthCheck writes
frontend/data/system/health_status.json + self-dispatches a heal — the user
can't click a button on a dead WebUI, so the poll drives the fix. Frontend
health-notifier surfaces a topbar badge + dashboard banner + details panel.
2. Failure cap, enforced centrally by dockerHealthHeal (task-gated): stops
crash-loopers (removing the churn), restarts the WebUI to re-publish a lost
port forward, and — only if that fails — recycles the rootless daemon and
restarts the core container. Caps every app immediately, no template churn.
3. Trivy no longer crash-loops offline: the server runs in a shell retry-loop so
the container stays Up and quietly retries on a backoff instead of exiting
FATAL. Verified: container stays Up across repeated DB-download failures.
Core WebUI compose gains restart: unless-stopped so it self-recovers after a
reboot / daemon recycle instead of staying down.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Follow-up to the backup-refresh throttle/dedupe, cutting the cost of the
remote pulls that do still happen.
* restic stats now runs in --mode raw-data (restic_check.sh). The default
restore-size mode walks every snapshot's tree to sum logical file sizes —
the slowest restic op — just to fill a size readout. raw-data reads the
index only and reports the repository's actual deduplicated on-disk size,
which is exactly what the dashboard already labels "deduplicated,
encrypted". raw-data omits total_file_count, so the per-location card now
shows that location's snapshot count (already loaded client-side, and more
useful for a backup repo) instead of a file count.
* engineLocationStats now shares the same per-refresh memoiser as
engineSnapshotsJson (engine_dispatch.sh). Both the locations and dashboard
generators call it per location, so repo stats went from two restic calls
per location per refresh to one. Factored the cache into _engineCachedPull.
* SSH connection reuse for SFTP locations (backup_ssh.sh): ControlMaster=auto
with a self-reaping ControlPersist master, so the several restic
subprocesses a refresh/backup spawns against one location share a single
authenticated connection instead of a fresh handshake each — the dominant
per-call cost on a high-latency link. Toggle via CFG_BACKUP_SSH_MULTIPLEX.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The "Refreshing backup data..." step on every WebUI update fired one
`restic stats` (restore-size mode — the slowest restic op) plus ~4
identical unfiltered `restic snapshots --json` pulls per enabled
location (dashboard, snapshots, app-status, migrate each pulled their
own), all over a fresh SSH connection for remote repos, on every pass
with no throttle — the slow, "frozen"-looking line users hit on poor
links.
Two fixes:
1. Dedupe. engineSnapshotsJson transparently memoises the first
unfiltered whole-repo pull per location to a shared cache dir
(LP_SNAP_CACHE_DIR, set by webui_updater around the chain), so the
four generators reuse one restic call instead of four. Filtered and
failed/empty pulls always fall through to a live call.
2. Throttle. Backups and location changes already regenerate this data
live when they happen, so the routine pass is only a drift catch-up.
Split the cheap local-only generators (engines/schema/passwords —
no remote I/O) out to always run, and gate the remote pull behind
CFG_BACKUP_DASHBOARD_REFRESH_INTERVAL (minutes, default 30, 0 =
every update). A completed backup touches a dirty marker that forces
the next pass to pull; WEBUI_UPDATER_FORCE still forces a full
refresh.
Net: N locations x 5 remote restic calls every update -> 1 call per
location, only when something actually changed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Trivy runs as a server whose vulnerability DB downloads on first boot; until it
lands no scan can produce results. Previously the updater generator wrote an
empty-but-valid cves.json the moment the file was missing, so installing Trivy
painted a green "no known vulnerabilities" all-clear that was actually a lie —
the DB hadn't even downloaded, and the Updates/Security view gave no signal.
Add an honest scanner state the WebUI branches on:
- containers/trivy/scripts/trivy_scan.sh — trivyScannerState (absent |
db_updating | ready) via `trivy version -f json`, trivyDbUpdatedAt, and
trivyScanImageCves (per-image scan normalized to {id,severity,package,
installed,fixed_in,url}, deduped). All degrade safely on error.
- webui_updater_scan.sh — stamp cves.json with scanner.state; only run real
per-image scans once the DB is ready. Always rewritten so state tracks live.
- updater-page.js — Security tab shows a loading box while the DB updates, an
install nudge when absent, and the genuine 🎉 only when ready+empty; Overview
CVE card sub + hint reflect the state.
- overview-manager.js — fleet Security row surfaces the "building CVE database"
pending state instead of silently omitting.
- function_manifest.sh — regenerated for the new trivy_scan.sh functions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Config sections (Basic, Mail, DNS, Firewall, …) rendered as a bare name. Add a
small data-driven icon before each: config files now carry a `# @icon <emoji>`
directive in their header, the config generator surfaces it as the subcategory's
`icon`, and the config editor prepends it to the section <h3> (covers every
render path — regular/master/toggle/domains/catalogs/whitelist — since they all
drop displaySubcategory straight into the heading).
- webui_generate_configs.sh: parse `# @icon` (first 10 lines) → emit "icon".
- config-manager.js: prepend subcategoryData.icon to the section title.
- configs/**: added an @icon to all 23 base config files (backup/general/network/
security/webui). The live install's config files get the same directive + a
configs.json regen so existing installs show the icons too.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
