The per-app Updates tab pushed a manual "Check" button (header + empty state)
even though scans run automatically on CFG_UPDATER_SCAN_INTERVAL — so an app with
nothing to update read like an empty/actionable page. Replace the manual Check
with a calm status line inside the panel: "Checked automatically · last checked X
· next check ~Y", backup-schedule style. The genuine Apply/Roll back actions stay
(applying is still manual and safe). No auto-apply.
- webui_updater_scan.sh: stamp scan_interval_minutes alongside generated_at in
updates.json so the display needs no separate config fetch (0 = auto off).
- updater-page.js: renderAutoCheckLine() + fmtRelFuture().
- app-tabbed-manager.js: drop the header/empty-state Check buttons; render the
auto-check line; friendlier no-data copy.
- overview.css: style .updater-autocheck (green dot live / muted when off).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
For a generic app (no traefik files, not libreportal) fixPermissionsBeforeStart
prints nothing, so the "Updating file permissions before starting" step showed
its header followed by an empty line — looking like it did nothing. Add an
isSuccessful line after the call so the step reports like every other one.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A fresh install ran webuiLibrePortalUpdate twice back-to-back: once in
installLibrePortal step 11, then again in startScan at the end of preinstall.
The 30s time-debounce meant to collapse them is fragile (it never fired on a
recent install — >30s elapsed between the two), and debouncing is the wrong
lever anyway: startScan's pass runs AFTER scanConfigsForRandomPassword
finalises app passwords, so it — not the step-11 pass — is authoritative.
Defer the step-11 generation deterministically during a bootstrap install
(libreportal_bootstrap_install=true), leaving startScan's single pass to do the
work. Standalone reinstalls (no bootstrap flag) still generate in step 11. The
time-debounce stays as a general back-to-back backstop.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the Update / Roll-back buttons tell the truth, closing the "new code on
old data" hole a floating tag creates.
updaterApplyApp:
- Anchor-correct: capture before/after from updaterPrimaryImage (the
<slug>-service image), not `grep -m1 image:` — fixes ollama et al.
- Records EXACT build refs in history from->to: repo:tag@sha256:<digest>
(via updaterRefDigest), so history is meaningful even when the tag doesn't
move (a rebuilt `latest`).
- Un-pins any digest a prior rollback pinned before pulling, so Update tracks
the channel again instead of freezing on the rolled-back build.
updaterRollbackApp:
- Before recreating, re-pins the anchor image to the pre-update build's digest
(from history's last update/ok `from`) via updaterSetAnchorRef, so `up` runs
the OLD code — not the current channel head. This is the fix for restoring a
data snapshot but recreating on a newer image.
New helpers (cli_updater_commands.sh): updaterRefDigest (local RepoDigest),
updaterSetAnchorRef (rewrite the anchor image line by service name, preserving
indent + the version sentinel; correct for companion-first apps like ollama),
updaterLastUpdateFrom (roll-back target from history).
Verified the helpers on nextcloud + ollama: pin adds @sha256 to the right
anchor only, sidecars untouched, sentinel preserved, unpin restores, YAML valid.
Caught and fixed a `local a=$1 b=...$a...` same-statement expansion bug that
would have silently no-op'd the rollback pin. End-to-end apply/revert not
exercised live here (no installed app has a pending update on this box).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The per-app object build used `available_version:($available_version|select(.!=""))`
to omit the field when empty. But in jq, a `{key: (empty)}` makes the WHOLE
object construction emit nothing — so every app with no available update (empty
available_version) produced no object and was silently dropped from updates.json.
Only apps WITH a pending update survived; an all-up-to-date fleet showed an empty
list. (Missed in P2's sandbox test because both fixture apps had updates.)
Emit an explicit null instead: `(if $available_version=="" then null else … end)`.
Verified: an up-to-date app (trivy, local==registry digest) now emits with
available_version:null; apps with updates still carry the string.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
webuiUpdaterScan now tells the truth instead of hardcoding update_available=false.
Per app it resolves three facts (see roadmap §2): the running RepoDigest (one
`docker inspect` of the anchor image), the display version (OCI label →
versioned tag → channel·shortdigest), and — throttled — the registry's current
index digest for the channel tag (`docker buildx imagetools inspect`, the same
identity as RepoDigest, verified exact). update_available = the two digests
differ. Emits type (versioned|rolling), channel, current/available digests +
versions, and a services[] array (every image line, anchor flagged).
- Registry lookups throttled separately from the scan: CFG_UPDATER_REGISTRY_
INTERVAL (min, default 360; 0 = local-only), own /tmp stamp, reuse of the
prior available_digest between windows so the app list still refreshes every
scan. UPDATER_REGISTRY_FORCE=1 forces a live pull (the Check-now button).
Registry failure (offline/rate-limited) = "unknown", never a false "changed".
- Digest-compare fully detects rolling apps' new builds; for versioned apps it
catches rebuilds of the pinned tag (newer-version enumeration is a later
step) — honest per type, and versioned apps are user-picked via P1b anyway.
- Fixes a P1b regression: updaterPrimaryImage now strips the trailing version
sentinel comment (`s/ #.*//`) via _updaterCleanImageRef — without this the
anchor ref (and the live CVE scanner's image arg) carried the comment.
- JSON built with jq for safe escaping; jq-less fallback keeps output valid.
Verified via a simulated 2-app install: navidrome → "0.62.0" (OCI label) +
detected update; rolling app → channel·shortdigest + update; throttled re-run
reuses the prior digest; all output valid JSON.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
"Control plane" is jargon. Rename the self-heal task and de-jargon all the
user-facing copy: task titles ("LibrePortal - Fix System Issues" / "System
Health Check"), the action label, the badge/banner/panel text and "Fix now"
button, the status summaries, and the heal task-log messages. Behaviour
unchanged; code comments keep the technical term where accurate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The scan read the app's "current image" via `grep -m1 image:` — the first
image line. For apps that declare a companion service first this mislabels the
app: ollama lists `ollama-webui` (open-webui) before `ollama-service`
(ollama/ollama), so an app named ollama reported open-webui's version.
Add `updaterPrimaryImage`: resolve the version anchor from the app's
`<slug>-service` image (the universal primary-service naming convention,
33/33 apps; underscores→hyphens for slugs like libreportal_catalog). Falls back
to the first image line off-convention. Used by both the version and CVE loops.
Verified: only ollama changes (→ ollama/ollama:latest); nextcloud, mastodon,
jitsi, gitea, vaultwarden anchors unchanged.
Also revise docs/roadmap/app-version-updater-and-cve.md to the config-first
direction agreed this session:
- CFG_<APP>_VERSION ADOPTED (was rejected): the #LIBREPORTAL tagging system
makes the config the source (compose tag derived from it), not a second one.
- Two version TYPES — versioned (real tag, version picker) vs rolling (floating
channel, digest-freshness); digest is the uniform detection engine for both,
version numbers are display enrichment. Answers "why not just compare numbers"
(most upstreams publish none; no universal latest-version API).
- Multi-service anchor = <slug>-service, not first line; lock-step sets (jitsi)
= one channel → several image lines; sidecars tracked-by-digest, not headline.
- Phases updated: P0 anchor (done) → P1 config-first pin → P2 detection+display.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Crash-loop detection keyed on `docker ps --filter status=restarting`, but a
backed-off loop sits "exited" between restarts once docker's backoff grows to
tens of seconds — so a slowed loop is missed. Detect via RestartCount CLIMBING
between scans (what a crash loop actually is), unioned with the instantaneous
restarting signal for fast loops. Baseline counts persist in
.health_restart_counts, written only by the throttled check so the heal's
re-scans don't disturb the delta.
Also give the system_health_heal / system_network_heal tasks proper display
(they fell through to the raw command + generic ⚙️): friendly titles in
formatCommandForUser, type icons (🩺 / 🌐) in getTaskTypeIcon, and action
labels in formatActionTitle — so they read as "LibrePortal - Repair Control
Plane" etc. with an icon, like install tasks.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
An offline trivy install crash-looped (server FATALs when it can't fetch the
vuln DB), and on rootless docker the restart storm churned the shared network's
port-forwarder until the WebUI's own published host port was torn down — the
WebUI stayed healthy INSIDE its container but was unreachable from the host, with
nothing detecting or healing it.
Three fixes, in the house self-healing style (mirrors the network-drift trio):
1. Control-plane health checker wired into the existing task-processor idle poll
(maybeRegenPoll), no new daemon. dockerHealthScan (read-only) detects daemon
down, a WebUI running-but-host-port-unreachable (the port-forward corruption),
and crash-looping containers. webuiSystemHealthCheck writes
frontend/data/system/health_status.json + self-dispatches a heal — the user
can't click a button on a dead WebUI, so the poll drives the fix. Frontend
health-notifier surfaces a topbar badge + dashboard banner + details panel.
2. Failure cap, enforced centrally by dockerHealthHeal (task-gated): stops
crash-loopers (removing the churn), restarts the WebUI to re-publish a lost
port forward, and — only if that fails — recycles the rootless daemon and
restarts the core container. Caps every app immediately, no template churn.
3. Trivy no longer crash-loops offline: the server runs in a shell retry-loop so
the container stays Up and quietly retries on a backoff instead of exiting
FATAL. Verified: container stays Up across repeated DB-download failures.
Core WebUI compose gains restart: unless-stopped so it self-recovers after a
reboot / daemon recycle instead of staying down.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Follow-up to the backup-refresh throttle/dedupe, cutting the cost of the
remote pulls that do still happen.
* restic stats now runs in --mode raw-data (restic_check.sh). The default
restore-size mode walks every snapshot's tree to sum logical file sizes —
the slowest restic op — just to fill a size readout. raw-data reads the
index only and reports the repository's actual deduplicated on-disk size,
which is exactly what the dashboard already labels "deduplicated,
encrypted". raw-data omits total_file_count, so the per-location card now
shows that location's snapshot count (already loaded client-side, and more
useful for a backup repo) instead of a file count.
* engineLocationStats now shares the same per-refresh memoiser as
engineSnapshotsJson (engine_dispatch.sh). Both the locations and dashboard
generators call it per location, so repo stats went from two restic calls
per location per refresh to one. Factored the cache into _engineCachedPull.
* SSH connection reuse for SFTP locations (backup_ssh.sh): ControlMaster=auto
with a self-reaping ControlPersist master, so the several restic
subprocesses a refresh/backup spawns against one location share a single
authenticated connection instead of a fresh handshake each — the dominant
per-call cost on a high-latency link. Toggle via CFG_BACKUP_SSH_MULTIPLEX.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The "Refreshing backup data..." step on every WebUI update fired one
`restic stats` (restore-size mode — the slowest restic op) plus ~4
identical unfiltered `restic snapshots --json` pulls per enabled
location (dashboard, snapshots, app-status, migrate each pulled their
own), all over a fresh SSH connection for remote repos, on every pass
with no throttle — the slow, "frozen"-looking line users hit on poor
links.
Two fixes:
1. Dedupe. engineSnapshotsJson transparently memoises the first
unfiltered whole-repo pull per location to a shared cache dir
(LP_SNAP_CACHE_DIR, set by webui_updater around the chain), so the
four generators reuse one restic call instead of four. Filtered and
failed/empty pulls always fall through to a live call.
2. Throttle. Backups and location changes already regenerate this data
live when they happen, so the routine pass is only a drift catch-up.
Split the cheap local-only generators (engines/schema/passwords —
no remote I/O) out to always run, and gate the remote pull behind
CFG_BACKUP_DASHBOARD_REFRESH_INTERVAL (minutes, default 30, 0 =
every update). A completed backup touches a dirty marker that forces
the next pass to pull; WEBUI_UPDATER_FORCE still forces a full
refresh.
Net: N locations x 5 remote restic calls every update -> 1 call per
location, only when something actually changed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Trivy runs as a server whose vulnerability DB downloads on first boot; until it
lands no scan can produce results. Previously the updater generator wrote an
empty-but-valid cves.json the moment the file was missing, so installing Trivy
painted a green "no known vulnerabilities" all-clear that was actually a lie —
the DB hadn't even downloaded, and the Updates/Security view gave no signal.
Add an honest scanner state the WebUI branches on:
- containers/trivy/scripts/trivy_scan.sh — trivyScannerState (absent |
db_updating | ready) via `trivy version -f json`, trivyDbUpdatedAt, and
trivyScanImageCves (per-image scan normalized to {id,severity,package,
installed,fixed_in,url}, deduped). All degrade safely on error.
- webui_updater_scan.sh — stamp cves.json with scanner.state; only run real
per-image scans once the DB is ready. Always rewritten so state tracks live.
- updater-page.js — Security tab shows a loading box while the DB updates, an
install nudge when absent, and the genuine 🎉 only when ready+empty; Overview
CVE card sub + hint reflect the state.
- overview-manager.js — fleet Security row surfaces the "building CVE database"
pending state instead of silently omitting.
- function_manifest.sh — regenerated for the new trivy_scan.sh functions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New containers/trivy app (aquasec/trivy in server mode, private API port,
docker socket + cache volume) in the security,recommended categories, plus
placement in the setup wizard recommended step, the server-side install
tier after crowdsec, and the CLI first-install prompt. The updater's CVE
scan (design doc P4) will gate on this app being installed; §5 of the
design doc updated to record the app-based decision.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
After 'Generated apps-tools.json...' the next step reaches remote backup
locations (restic over SSH) with its output captured in $(...), so on a
slow connection the updater looked stalled with no message. Print an
in-progress notice before it, mirroring the per-app refresh hook loop.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Config sections (Basic, Mail, DNS, Firewall, …) rendered as a bare name. Add a
small data-driven icon before each: config files now carry a `# @icon <emoji>`
directive in their header, the config generator surfaces it as the subcategory's
`icon`, and the config editor prepends it to the section <h3> (covers every
render path — regular/master/toggle/domains/catalogs/whitelist — since they all
drop displaySubcategory straight into the heading).
- webui_generate_configs.sh: parse `# @icon` (first 10 lines) → emit "icon".
- config-manager.js: prepend subcategoryData.icon to the section title.
- configs/**: added an @icon to all 23 base config files (backup/general/network/
security/webui). The live install's config files get the same directive + a
configs.json regen so existing installs show the icons too.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
- Catalog block now shows the official catalog as a pinned row #1 (verified
badge + its URL from the generated data). Removing it opens the shared
confirmation dialog with a REQUIRED tickbox (generalized to accept custom
checkbox text) — it only hides official apps from browsing; system updates are
unaffected. Backed by a new CFG_CATALOG_OFFICIAL_ENABLED flag the scan honors.
- Trimmed the info-banner wording per feedback.
- Config option sections: the body (after the divider, before the buttons) now
sits in a darker boxed panel (bg + border), matching the content-box look used
elsewhere. Applies to regular grids + the block editors; scoped to the main
config editor.
Signed-off-by: librelad <librelad@digitalangels.vip>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per the intended model: catalog sources should live in the config place (like
domains), and registry_catalog.json should be a purely GENERATED artifact
derived from them — not the source of truth. Replaces the earlier
$docker_dir/catalog/sources.json store.
- New configs/general/general_catalogs — CFG_CATALOG_1..9, one catalog base URL
per slot ("url" or "url|channel"), domains-style. Official stays pinned as
source #1 (derived from CFG_RELEASE_BASE_URL, not listed here). Slot N → source
idx N+1 (stable id for the Add picker / `app add --source`).
- catalog_sources.sh now reads/writes those CFG vars (via updateConfigOption)
instead of a JSON file; dropped catalogSourcesFile + the enable/disable toggle
(presence = enabled; remove = clear the slot).
- configUpdateBatch regenerates registry_catalog.json when a CFG_CATALOG_* key
changed — so pressing Save in the WebUI rebuilds the browse data.
- webuiRegistryCatalogScan is unchanged (still iterates catalogEnabledSources).
Verified: CFG_CATALOG_1/2 → sources at idx 2/3, empty slots skipped, url|channel
parsed, official pinned at idx 1.
Signed-off-by: librelad <librelad@digitalangels.vip>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
startPreInstall called installLibrePortalImageWebUI standalone AND then
installLibrePortalAppWebUI, whose app-install step 1 builds the same image
again — so a fresh install built libreportal-service twice back-to-back
(2nd mostly Docker-cache hits, but a redundant copyFolder + "this may take a
while" block). The standalone call only matters on a rebuild where the
app-install path SKIPS (libreportal already in the DB). Gate it on exactly
that: build standalone only when libreportal is already installed. Net: one
build in both cases (fresh → via app install; reinstall → standalone).
Signed-off-by: librelad <librelad@digitalangels.vip>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of the "task loops eternally" bug: the lazy-autoload stub was
fn() { source "$file"; fn "$@"; }
If the source fails — the real case: an app-install/deploy rsync briefly
removes-then-replaces a generator file while a setup task calls it — `fn` is
never redefined, so `fn "$@"` re-invokes the *stub*, which sources the (still
missing) file, which re-invokes the stub… A single setup finalize recursed
12,050 levels, flooding the task log and taking 82s before it happened to
recover when the file reappeared. A permanently-missing file would never
recover.
Fix (root cause): drop the stub before sourcing —
fn() { unset -f fn; source "$file"; fn "$@"; }
so a failed source degrades to one "command not found" (rc 127) instead of
unbounded recursion. Regenerated function_manifest.sh (975 stubs, reformat
only — no function-set change).
Failsafes on the task processor (defence in depth, per request):
- FUNCNEST cap (TASK_FUNCNEST_MAX, default 1000) inside the task's eval
subshell — any runaway recursion now aborts in milliseconds instead of
spamming the log until the stack/disk gives out.
- Wall-clock cap (TASK_MAX_RUNTIME_SECS, default 7200s, 0=off) — the heartbeat
watcher TERM→KILLs a task's process group once exceeded and marks it failed
(distinct from a user cancel via a .timeout marker). Generous so real long
installs/backups/migrations finish.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Per feedback — shorten the install-failure error (drop the extra hint line) and
the rootless MTU notice.
Signed-off-by: librelad <librelad@digitalangels.vip>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An image-pull failure (e.g. the MTU-EOF black hole) left apps marked
installed+active with no container: the compose `up -d` failure was invisible
because (a) checkSuccess read $? after `_rc=$?`, always printing "✓ Started",
and (b) the exit code is dropped across dockerComposeUpdateAndStartApp →
dockerComposeUpdate → dockerComposeRestartAfterUpdate, and installApp never
checked it — so post-install integrations ran and set status=1 regardless.
- up_app.sh: restore $? to the compose exit before checkSuccess (both rootless
and rooted) so a failed `up -d` is reported as an error + logged, not "✓".
- app_install.sh: after `up`, reality-gate on the app's compose project having
at least one container (ps -a, so a slow-to-start container still counts).
If none exists, print a clear failure, skip _appPostStartIntegrations (which
is what records the app + sets status=1), and return non-zero.
Verified: the gate query passes for a running app (libreportal) and refuses
apps with no container (navidrome/ipinfo after their pulls EOF'd).
Signed-off-by: librelad <librelad@digitalangels.vip>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CFG_NETWORK_MTU=1500 was baked blindly into the rootless daemon uplink
(DOCKERD_ROOTLESS_ROOTLESSKIT_MTU) and every app's container network. On links
whose real path MTU is below 1500 (Qubes/NAT/VPN with PMTU discovery blocked),
image manifests + tiny images pull fine but large image LAYERS stall and reset
mid-blob with "httpReadSeeker: ... EOF" — apps silently fail to install. Probed
here: path MTU ~1328, Docker at 1500 → black hole.
- New scripts/network/network_mtu.sh: networkDetectMtu (don't-fragment ICMP
ladder → largest standard MTU that gets through, 1500 when ICMP gives no
signal) + networkEffectiveMtu (CFG_NETWORK_MTU: a number is verbatim, "auto"
probes once and caches $docker_dir/.network_mtu) + networkRedetectMtu.
- CFG_NETWORK_MTU default 1500 -> auto. Rootless setup now writes the resolved
MTU into the override (and re-detects per install); the per-app NETWORK_MTU_TAG
uses the resolved value too. Explicit numbers still win.
Verified on this box: auto -> 1300; alpine (multi-MB) + the full ~100MB navidrome
image now pull to completion where they previously EOF'd. Applied live (override
1300, config=auto, cache=1300, rootless docker restarted).
Signed-off-by: librelad <librelad@digitalangels.vip>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every file under $configs_dir is treated as a config (some get sourced at
startup), so a JSON file there was executed as bash — "[name:…]: command not
found" on every config load. Move it beside the apps DB in the manager state
root ($docker_dir/catalog/sources.json): persistent, manager-owned, and never
config-scanned.
Signed-off-by: librelad <librelad@digitalangels.vip>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
First slice of multi-catalog ("taps") support. Today the App Center browses one
catalog (get.libreportal.org). This adds an ordered list of catalog sources and
teaches the browse scan to merge them.
- New scripts/catalog/catalog_sources.sh — the source list in its OWN file
($configs_dir/catalog/sources.json). Source #1 is ALWAYS the official catalog,
synthesized live from CFG_RELEASE_BASE_URL/CHANNEL (can't be edited/removed,
always pinned on top). Extra sources are stored as a small JSON array and are
UNVERIFIED (trust=community). Helpers: catalogSourcesJson / catalogEnabledSources
(priority order) / catalogSourceAdd|Remove|Toggle|List / catalogFetchCommunityIndex.
- webui_registry_scan.sh now walks catalogEnabledSources: the OFFICIAL source is
still signature-verified (lpFetchIndexInto, unchanged trust path); third-party
sources are fetched unverified. Apps are merged by slug into one card carrying a
sources[] array in priority order (highest first = default). Trust/verified are
taken from the SOURCE, never the artifact's self-claim, so a community index
can't promote itself to "official". Icons still mirror same-origin from the
official index only. registry_catalog.json gains top-level sources[] + per-app
sources[]; the old source{} object + signed/serial are kept for back-compat.
- New `libreportal catalog source list|add|remove|enable|disable` + `catalog
refresh` CLI (dynamic-routed). Mutations go through the task system (cliTaskRun
"…" "catalog"), never a new mutating API.
Scope firewall: this governs APP BROWSE + ADD only. LibrePortal's own updates and
hotfixes still resolve from the official CFG_RELEASE_BASE_URL alone — lpFetchIndex
is untouched, so a third-party catalog can never become a system-update channel.
Next: `app add --source`, then the WebUI (domains-style block source manager +
Add-dialog source picker with the Official badge / unverified warning).
Signed-off-by: librelad <librelad@digitalangels.vip>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three coupled defects in the first-run wizard flow, all surfacing as
"it said complete but nothing was set up":
1. Zero-app installs sailed through. With no apps ticked, setup was just
config+finalize, finished in seconds having installed nothing, and
fast-forwarded to an empty App Center. Add a self-contained in-wizard
confirm ("Install with no apps?") before submitting. Can't reuse the
shared confirmation-dialog component — it isn't loaded this early in
boot — so the dialog is rendered by the wizard itself and mounted on
<body> to escape the aurora surface's FX-stacking rule.
2. finalize declared success unconditionally. It never inspected the
per-app install tasks, so a failed app still yielded "your install is
ready" + a redirect. Pass the setup group id to `setup finalize`; it
now rolls up the group's app-install task results and logs a clear
partial/failed verdict. The WebUI completion watcher gates the welcome
toast + App Center hand-off on the whole group succeeding, not just on
finalize completing — a failed app now keeps the user on the tasks page
with an error toast instead of a false all-clear.
3. Setup task count was confusing (banner "of 2" while the page listed 3).
On dev/git installs the topbar's dev-mode auto-enable raced the wizard's
own CFG_DEV_MODE write and spawned a second, un-grouped config-update
task mid-setup. Skip that auto-enable while a setup handoff is active
(the wizard already persists CFG_DEV_MODE); it runs on the next load if
still needed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The backup-data refresh in the WebUI update chain shelled out to
`restic snapshots` once PER APP (dashboard, app-status) and once per
(host, app) (migrate) — every call a `sudo`+`restic` spawn that re-opens
and decrypts the repo. With N apps and L locations that's O(N·L) (app-status)
plus O(hosts·apps·L) (migrate) spawns on every update — the silent stall
after "Refreshed backup dashboard data...".
One `restic snapshots --json` already returns the whole repo, so fetch it
ONCE per location and derive the per-app / per-host / system views in jq.
Validated against a live restic 0.19 repo: the jq filter reproduces
`restic snapshots --tag <t> --host <h>` exactly (incl. empty-host = all
hosts, and newest = last), and count/latest extraction matches the old grep.
- dashboard: one primary-location fetch → all app tiles + system status
(was N snapshot calls + 1).
- app-status: one fetch per location → filter per app (was N×L calls);
also de-recursed into a flat app loop.
- migrate: filter the already-fetched per-location list (was a spawn per
host×app). Fixes a latent bug too — `grep -oc` over single-line JSON
always reported a count of 1; jq `length` is now accurate.
restic snapshot spawns per refresh drop from ~O(N·L + hosts·apps·L) to ~O(L).
`restic stats` (still run per location in locations + dashboard) is a
separate, non-multiplicative cost left untouched to preserve its semantics.
Signed-off-by: librelad <librelad@digitalangels.vip>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
webuiLibrePortalUpdate runs the per-app refresh hooks on every update (the
task processor fires it repeatedly), and gluetun's hook silently pulls a
~7MB servers.json from GitHub each time. The hook's stdout is captured by
`result=$($_hook)`, so on a slow link the update just sits with no output
right after "Generated apps-tools.json..." — it reads as a freeze.
- Throttle the fetch to once per CFG_GLUETUN_PROVIDERS_REFRESH_HOURS (24h
default; 0 = manual-only). Direct callers (Tools refresh, install hook)
pass GLUETUN_PROVIDERS_FORCE=1 to still get a guaranteed pull.
- curl --compressed (this JSON gzips ~7x) + --connect-timeout 15 so a slow
or dead link shrinks/fails fast instead of stalling every update.
- Print "Refreshing <app> WebUI data..." from the updater loop (outside the
output capture) so the step is visible instead of looking hung.
Signed-off-by: librelad <librelad@digitalangels.vip>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
'Marketplace' read as a place to buy things; this is a free, self-hosted app
catalog, so rename the app to LibrePortal Catalog. Slug is libreportal_catalog
(underscore — the slug becomes a CFG_<SLUG>_ prefix and a bash identifier via
declare "${app_name}=i"; a dash would break install). Docker-facing names use
dashes (libreportal-catalog-service / hostname libreportal-catalog), declared
explicitly in the PORT config + compose, not derived from the slug.
- containers/marketplace/ -> containers/libreportal_catalog/ (+ .config, .svg, hook file)
- CFG_MARKETPLACE_* -> CFG_LIBREPORTAL_CATALOG_*, APP_NAME + TITLE + PORT_1 updated
- install hook fn marketplace_install_post_setup -> libreportal_catalog_install_post_setup
- served browse site reworded Marketplace -> Catalog, icon refs updated
- regenerated function_manifest.sh (autoload stub now points at the new file)
Not installed yet, so there is no live config/container/volume to migrate — the
cheapest moment to rename. The client-side registry ('View full page on the
marketplace') wording is a separate subsystem and left unchanged for now.
Signed-off-by: librelad <librelad@digitalangels.vip>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
start_scan.sh runs updateDockerInstallPassword every system scan, doing
`sudo passwd $CFG_DOCKER_INSTALL_USER` via runSystem. Model A's scoped
sudoers grants only LP_HELPERS/LP_SYSTEM + run-as-install-user — not passwd
— so at runtime (manager, non-root) it fails exit 1 every scan. This is the
exact sibling of the updateDockerSudoPassword failure fixed in 9050a8c; that
guard was added to the manager/sudo user but the dockerinstall user was
missed, so error_report.log kept logging "Updating the password for the
dockerinstall user" on every scan.
The password is set at install (root path, startPreInstall →
installDockerRootlessUser) and the rootless docker user is driven by tooling,
not a password login, so the runtime re-sync is legacy + impossible under
de-sudo. Guard it to skip unless EUID 0, mirroring the sudo-pass fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The featured passthrough was removed when the dedicated-section commit was
reverted; the Featured sidebar row needs it. spec.json {"featured":true} now
sets meta.featured on the published app again.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Replaces the extra on-card marketplace tags (the separate Official trust
badge) with a richer, more elegant flow: a registry card keeps a single
'Available' pill, and clicking Add (or the card) opens a modal outlining the
app — full description, publisher + trust, version, the add command, and a
'View full page on the marketplace ↗' link to the app's page on the source
it came from — with the actual add as the confirm. Gives a beat to review a
community app before pulling it in.
- webui_registry_scan.sh re-emits source{base,channel}; loadApps stashes it
(window.registryCatalogSource) + carries per-app version.
- openRegistryDetails() builds the eo-modal; addRegistryApp() is now just the
task dispatch (the modal's confirm / a fallback).
- The marketplace website supports a #<slug> deep-link (focus one app, with a
'Show all' bar), so the modal's link lands on that app's page.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Reverts a4e65df. The maintainer prefers the more fluid model where registry
apps flow into the App Center grid as 'Available — Add' cards alongside
installed/local apps, rather than a separate /apps/marketplace destination.
Restores the Stage-4 merged-grid behaviour (loadApps merges
registry_catalog.json; createAppCard renders registry cards; addRegistryApp
dispatches app_add). The make_app.sh featured passthrough + the generator's
featured/source fields go with it (unused by the merged grid).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Splits the catalog out of the main grid into its own destination, the
WordPress 'Add Plugins' vs 'Installed Plugins' model: the grid is now
purely 'your apps' (local definitions), and the new Marketplace section is
'get more apps' (the remote signed catalog).
- New MarketplacePage (components/apps/marketplace/) mounts at
/apps/marketplace inside the apps feature (same sub-dispatch pattern as
/apps/overview — no new top-level component). Pinned sidebar entry with a
live 'available to add' count badge.
- Status strip: signed/unsigned, available + catalog counts, serial, source
base+channel, freshness, and a Refresh that re-runs the host-side registry
scan via updater_check.
- Publisher-curated Featured shelf (meta.featured, set at publish time — no
tracking/popularity), category chips + search, per-app detail modal
(long description, publisher/trust/version, add command), and the chained
Add & set-up flow: dispatch app_add, and when the definition lands, hand
off to the app's config/install page.
- State-aware cards: Available (Add) / Added (Set up →) / Installed (Open).
- Backend: make_app.sh passes through meta.featured; webui_registry_scan.sh
emits featured + source{base,channel} in registry_catalog.json.
- Removed the grid's registry-merge + registry card path + its CSS (moved to
the namespaced marketplace surface); app_add task wiring + completion
handler retained and reused.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
containers/marketplace — nginx:alpine app (standard drop-in contract:
config + tagged compose + icon + install hook) whose docroot serves BOTH
halves of the marketplace: the signed catalog channel tree (index.json /
payloads, published into data/<channel>/ by the release tools) and a
self-contained client-rendered browse site over the same file (search,
category chips, trust badges, copyable 'libreportal app add <slug>' —
no third-party assets, no backend, no build step). The official
marketplace is an instance of this app; self-hosting one = installing it
and pointing CFG_RELEASE_BASE_URL at it. Boxes only ever trust the
minisign signature on the catalog, never the website.
New generic gating convention: CFG_<APP>_DEV_ONLY=true keeps an app out
of the App Center grid unless Developer Mode is on (CFG_DEV_MODE, the
same flag the **DEV** config-field filter uses); an installed dev-only
app always stays visible. The marketplace app is the first user.
Cache policy: catalog/channel manifests no-cache; payloads short
revalidating cache (same-id re-publish); version-pinned release
artifacts immutable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
webuiRegistryCatalogScan (run by updater check, same atomic keep-prior
pattern as webuiArtifactScan) writes apps/generated/registry_catalog.json:
the type:"app"/kind:"bundle" rows of the signed index annotated with
defined/installed, browse metadata from the envelope meta, and icons
mirrored into core/icons/apps/registry/ ONLY when their bytes match the
sha256 pin in the signed index — the browser stays same-origin; a tampered
or oversized icon is skipped, never served.
webuiArtifactScan now selects type=="hotfix" so app rows never render as
pseudo-hotfixes in the Improvements tab, and counts+logs artifacts of
unrecognized type instead of surfacing them (the §8.1 forward-compat
firewall on the scan path).
Harness vs a locally served registry: 14/14 (catalog row + meta + flags,
icon pin verify + tamper skip, hotfix-only stream, unknown-type skip+log,
unreachable-registry keeps prior files).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Opens the two designed seams (roadmap §8.4): _artifactResolve accepts
type:"app" (slug validated, the installed-app gate skipped — presence is
the collision policy's call), and payload.kind:"bundle" gets its own APPLY
flow. The download core (sha256 pin vs the signed index + minisig +
refuse-unsigned) is factored into _artifactDownloadVerified, shared by ops
and bundle payloads.
A bundle add: fetch → quarantine-validate → place in the definition tree
(staging + one rename, manager funnel) → lpRegenWebui → verify the app
surfaced in apps.json → applied-record with a precise undo → History.
The validator is fail-closed (traversal/absolute paths, links/devices,
single top-level dir == slug, charset, size/entry caps, set-id strip,
config TITLE+CATEGORY + compose present, bash -n every .sh) because the
definition tree is live-sourced on every CLI start — nothing lands there
before trust + quarantine pass. Collision policy: installed-live refused,
local definitions win, registry-owned re-add = reversible definition
update (prior tree packed into the undo). Revert removes/restores the
definition (refused while installed) and regens. Apps never auto-apply
(type filter kept + publisher forces auto:false).
New verb: libreportal app add <slug|artifact-id> (app_add task; resolves
by slug via appAddFromRegistry, ambiguity refused).
Also fixes the second half of the sigstate-propagation bug class:
artifactApply captured $(_artifactResolve) in a subshell, stranding
_ART_INDEX/_ART_APP/_ART_SCOPE AND the LP_INDEX_SIGSTATE the apply gate
enforces — on a signed box every apply would have refused as unsigned.
Resolve now assigns globals (_ART_JSON) and is called directly.
Source-and-mock harness: 46/46 (resolve gates, 14 validator refusals,
happy add, collision matrix, definition-update round-trip, revert
semantics, postcheck + record-failure rollbacks, apply-auto exclusion,
app add verb).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The marketplace publisher tool: packs containers/<slug>/ (the drop-in app
contract, unchanged) into a deterministic tarball (commit-clamped mtimes +
gzip -n; unchanged apps repack byte-identical and keep their published
version), signs it, copies a sha256-pinned catalog icon, and upserts a
type:"app" / payload.kind:"bundle" envelope into the same team-signed
index hotfixes ride. Catalog metadata (title/category/descriptions) is
parsed line-wise from the app's own .config — one source of truth with the
App Center generators. auto:false is hard-forced: apps never auto-apply.
The index-upsert/serial/freshness/publishers-map and signing logic is
factored out of make_hotfix.sh into lib/release_index.sh, shared by both
tools (make_hotfix.sh behavior preserved; regression-tested alongside an
app entry: serial bump, both payload kinds coexist, valid_until refreshed).
LP_INDEX_VALID_DAYS is the shared freshness knob (LP_HOTFIX_VALID_DAYS
kept as a legacy alias).
Verified: speedtest publish → deterministic repack (identical sha256) →
served via local http.server → real lpFetchIndex/accessors harness 10/10.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Every caller captured the index with var=$(lpFetchIndex), which runs the
fetch in a command-substitution subshell — the LP_INDEX_SIGSTATE global it
sets never reached the caller. On a box with real signing active the
artifactApply/apply-auto gates would therefore refuse a correctly signed
index (fail-closed, but the apply path would be dead on arrival the day
signing activates), and artifact index / the WebUI scan would report a
verified feed as UNSIGNED.
New lpFetchIndexInto <var> [cache] runs the fetch in the calling shell and
assigns via printf -v; all four call sites converted. Verified with a
source-and-mock harness against a locally served index: 10/10 (sigstate
reaches caller, serial high-water, anti-rollback refuse, staleness refuse,
id enumeration, envelope round-trip).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Describe only the current useradd behaviour; drop the narration of the old
silent-failure bug (per the repo's no-tombstone-comments convention).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Drop the GitHub-release version comparison entirely. We install slirp4netns
from apt regardless, so comparing against the GitHub-latest tag only produced
a perpetual 'outdated' loop and a no-op re-install. apt-get install -y is
already idempotent, so run it unconditionally and report the resulting
version.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
slirp4netns --version prints multiple lines (version, commit, libslirp,
SLIRP_CONFIG_VERSION_MAX). The old 'awk {print $2}' ran on every line and
also picked the literal word 'version' from line 1, producing a multi-line
blob that leaked into the 'is outdated' notice. Read only the first line and
take field 3 (the actual number), strip the leading v from the GitHub tag so
the comparison is meaningful, and skip the check if the tag fetch fails.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Initial rootless setup ran 'systemctl --user start docker' immediately
after install, but the rootless net/port-driver override.conf (and the
daemon-reload that loads it) aren't written until further down. So the
first start always failed — 'Job for docker.service failed' plus a
spurious '✗ Error Setting up Rootless' in the error report — even though
the later 'systemctl --user restart docker' brought the daemon up fine
once the override was in place.
Drop the premature start from the install step (keep install + enable);
the restart after the override is written is now the first real start.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The rootless WebUI container reads its bind-mount sources (configs/webui/*)
through the container-owner GROUP since a2376e2 switched those files from
world-readable to 0640 group=container-owner. But the WebUI credential
randomizer rewrites webui_logins via `sed -i` as the non-root manager, which
recreates the file with the manager's own group — dropping the container-owner
group. The installer then started the container immediately, so node hit
EACCES on /app/webui_logins at require-time (parseConfigFile) and exited 1;
nothing listened on the WebUI port. `libreportal webui login reset` had the
same latent bug (rewrite → restart). Under the old world-readable model a
post-sed file stayed o+r so the container could still read it, which is why
this only surfaced on fresh rootless installs after a2376e2.
Fix: make reconcileWebuiDirOwnership the single "ready the WebUI for its
container" pass — it now also restores the configs/webui bind access (new
`webui-bind` ownership action) on top of the container-dir chown. Reorder the
installer so the credential randomizer runs BEFORE the before-start permission
pass, making that pass the last ownership touch before the container starts;
and call reconcileWebuiDirOwnership before the restart in login reset.
Live box recovered via `libreportal-ownership reconcile`; WebUI 200.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
_webui_bind_access granted o+r to every file in configs/webui so the
rootless container could read its bind-mount sources — but that also made
secrets like webui_logins world-readable to any local user. Under rootless
the container's gid 0 maps to the container owner's gid, so group access is
sufficient: chown the webui dir + files to MANAGER:container-owner, dir
0751 (traverse, not list), files 0640. Container reads via group; other
local users get nothing; the manager (owner) still rewrites them.
Verified live: container READ ok, world READ denied, manager rw, WebUI
login still 200. Live helper updated in lockstep with this source.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
