Merge claude/1

Fills the gap updates-and-distribution.md always deferred: how versions
are actually detected, pinned, and reverted, and how cves.json gets real
data. Core decisions: floating tag stays the channel while the live
compose pins the digest (making detection possible and rollback honest),
and CVE scanning runs as a LibrePortal-orchestrated ephemeral Trivy
container (no host binary, no always-on scanner app).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>

docs/roadmap/app-version-updater-and-cve.md

@@ -0,0 +1,112 @@
+# LibrePortal — Per-App Version Updater & CVE Scanning (Roadmap / Design)
+
+**Status:** Design — **not built.** Agreed direction 2026-07-14; fills the two deliberately-unwired hooks in `webui_updater_scan.sh`. · **Audience:** us, future-self · **Scope:** real "update available" detection, per-app version identity (pin/track/revert), and the vulnerability scanner behind `/apps/overview/updates` · **Origin:** "the updates system is currently not set up" conversation 2026-07-14. Companion to [updates-and-distribution.md](updates-and-distribution.md) — that doc owns the *signed artifact/hotfix pipe*; this one owns the *generators* it always treated as pre-existing.
+
+---
+
+## 0. Where we actually are (so we don't redesign what exists)
+
+Most of the updater is **already built and working**. What's missing is narrow:
+
+| Piece | Status |
+|---|---|
+| `updater apply` — snapshot → `compose pull` → `up`, auto-rollback on failure | ✅ real (`cli_updater_commands.sh`) |
+| `history.json` (fail-closed from→to audit trail) | ✅ real |
+| Signed hotfix channel (fetch/verify/apply/undo, `CFG_HOTFIX_AUTO`) | ✅ real |
+| Auto-scan scheduling (`updater check auto`, 30-min throttle) | ✅ real |
+| WebUI — fleet Updates tab, per-app Updates tab, Security/Recovery/History renderers, Update/Roll-back buttons wired to tasks | ✅ real, waiting on data |
+| **"Update available" detection** | ❌ stub — `update_available` is hard-coded `false` (`webui_updater_scan.sh:55-58` is a marked hook) |
+| **CVE data** (`cves.json`) | ❌ stub — written empty once, then never rewritten (the `[ ! -f ]` guard at `:82-89` must go) |
+| Recovery fields (`dr_ready`, `last_snapshot*`) | ❌ never emitted; frontend infers from History |
+
+So this is **not** a new tab or a new subsystem. The surfaces the feature needs already exist — the fleet **Updates** tab and the **per-app Updates tab** (the "version tab per app" — it's already there, beside Backups). The job is to make the two generators tell the truth, and to fix one real correctness hole described next.
+
+## 1. The core problem: `latest` is a channel, not a version
+
+Across `containers/*/docker-compose.yml` there are 49 active `image:` lines: **21 on `:latest`, 11 untagged (implicit latest), 17 on pinned or floating-stable tags**. Zero are variable-driven. Only nextcloud (`31-fpm-alpine`) and mastodon (`v4.2.0`) are meaningfully pinned among primary apps.
+
+Floating tags break both halves of what we want:
+
+- **Detection is textually impossible.** "Is `vaultwarden/server:latest` newer than `vaultwarden/server:latest`?" — the compose file can't answer; the string never changes.
+- **Revert is currently a lie.** `updaterRollbackApp` restores the *data* snapshot and re-runs `compose up` — but the local image is already the *new* latest, so you get **new code on old data**. History records `from → to`, and for a floating tag both are the same string.
+
+### The one idea
+
+> **Keep the floating tag as the *channel*; pin the *digest* in the live compose.**
+> `image: vaultwarden/server:latest@sha256:ab12…`
+
+Compose supports `tag@digest` natively. The tag stays as the human-readable statement of *what we track* ("latest", "stable", "31-fpm-alpine"); the digest pins *exactly which build runs*. Then:
+
+- **Detect:** resolve the channel tag at the registry (`docker manifest inspect`, no pull) and compare digests → a truthful `update_available`, even for `latest`.
+- **Update:** snapshot → rewrite the pin to the new digest → pull → up. Nothing moves unless the user (or a future policy) says so.
+- **Revert:** restore the data snapshot **and** rewrite the pin back to the previous digest. Now rollback means what the button says.
+
+Why not just pin semver tags in all ~45 templates instead? Because upstreams are inconsistent (some publish semver, some only `latest`, linuxserver uses their own scheme), it's a permanent manual maintenance burden, and it still wouldn't tell us when a *pinned* tag's build is republished. Digest pinning is universal, automatic, and template-free — templates stay exactly as they are; pinning happens on the **live** copy at install time, which is already the mutable source of truth (template→live copy-then-edit is the established model, and the artifact system's `set-compose-image` op is a working, drift-guarded, undoable sed for precisely this line).
+
+### Multi-service apps
+
+Nextcloud is 4 images, jitsi is 4. The scan currently reads only the first `image:` line. Design: **pin and track every `image:` line, keyed by service**. `updates.json` keeps its per-app top-level fields (primary service — first entry — so the existing UI works unchanged) and gains a `services[]` array; the UI can later aggregate ("1 of 4 services has an update"). An app "has an update" if *any* service does; apply updates all outdated services in one snapshot.
+
+## 2. Version identity — what we record and show
+
+- **Source of truth:** the pin in the live compose file. No parallel `CFG_<APP>_VERSION` variable — that would be a second source of truth that drifts, and the `_TAG` namespace already means template placeholders here (a real footgun).
+- **Human-readable version:** digests are unreadable, so enrich for display: `docker image inspect` the local image for the OCI label `org.opencontainers.image.version` (most linuxserver/ghcr images carry it). Display order: label → tag (if not latest/untagged) → short digest (`sha256:ab12…` → `ab12cde`). `available_version` starts life as "new build of *latest*" plus short digest; label enrichment for the *remote* side is best-effort later (needs a config-blob fetch — not worth blocking on).
+- **`updates.json` additions** (existing fields keep their meaning): per app `channel` (the tag), `pinned_digest`, `available_digest`, `services[] { service, image, channel, pinned_digest, available_digest, update_available, version }`, plus the recovery fields the frontend already knows how to render (`dr_ready`, `last_snapshot`, `last_snapshot_version`, `last_snapshot_at`) sourced from the backup data that already exists.
+- **History** entries record full pinned refs (`repo:tag@sha256:…`) in `from`/`to`, which is what makes revert-from-history possible.
+
+## 3. Detection mechanics
+
+- `docker manifest inspect <repo:tag>` (rootless, via `dockerCommandRunInstallUser`) returns the registry digest without pulling; compare against the pinned digest. Local `RepoDigests` and the registry manifest-list digest are the same identity, so the comparison is exact.
+- **Throttle separately from the scan.** The 30-min `updater check auto` stat-gate stays cheap; registry lookups get their own stamp + knob, mirroring the established `/tmp` stamp idiom: `CFG_UPDATER_REGISTRY_INTERVAL` (minutes, **default 360**, `0` = manual-only). ~30 Docker Hub images once per 6 h sits comfortably under anonymous rate limits; add small jitter so fleets don't thundering-herd.
+- **Failure = stale, not error.** A registry timeout keeps the previous verdict and stamps `last_checked`; never flip an app to "unscanned" because the network blinked (per-registry short-circuit after the first timeout in a run).
+- **Reconcile existing installs:** on first scan, any installed app whose compose line is unpinned gets pinned to its *currently running* image's digest (`RepoDigests`) — zero behaviour change, purely recording reality. New installs pin right after the first pull. Apps using `build:` instead of `image:` are out of scope (scan already ignores them).
+
+## 4. Apply, revert, and the per-app surface
+
+`updaterApplyApp` becomes: resolve new digest (from `updates.json`) → **snapshot (unchanged, fail-closed)** → rewrite pin(s) via the factored-out first-class helper shared with the artifact op's sed → `compose pull` → `up` → history `ok` with digest-refs → rescan. On failure: auto-rollback now also **rewrites the pins back** before restoring the snapshot — that closes the "new code on old data" hole.
+
+`updaterRollbackApp` (the user-facing Roll back button): read the last `update/ok` history entry's `from` refs → rewrite pins → restore data snapshot → `up` → history `rolled-back`. This is the "allow users to revert back" requirement, and it only works because of the pins.
+
+**UI:** no new navigation. The per-app **Updates** tab gains a Version section (already scaffolded in `renderAppDetail`): channel · running build · available build · Update / Roll back. The fleet tab's rows finally show real `current → available`. Everything routes through the existing `updater_apply` / `updater_rollback` tasks — mutations stay task-only, no new API endpoints.
+
+## 5. CVE scanning — decision
+
+Three candidate shapes were on the table:
+
+| Shape | Verdict |
+|---|---|
+| **A. Always-on scanner app container** (trivy server / a "security app" in the catalog) | ❌ Resident RAM for a periodic job; permanent docker-socket exposure; duplicates our UI with its own. |
+| **B. Scanner binary on the host** | ❌ Grows the host footprint we've deliberately kept lean (rootless + de-sudo); another thing to install/update outside the app model. |
+| **C. LibrePortal-orchestrated, ephemeral scanner container** — the scan task does `docker run --rm aquasec/trivy:<pinned> image --format json <image>` per installed image | ✅ **Chosen.** No resident process, no host binary, socket exposed only for the seconds a scan runs, rides the existing task/throttle machinery, and the scanner itself is version-pinned like any other image. |
+
+So: **system-managed orchestration, containerized execution.** Not its own catalog app, not a host package.
+
+**Scanner: Trivy** (over grype — both fine, trivy has the larger ecosystem, single pinned OCI image, clean JSON, Apache-2.0). Details:
+
+- Vulnerability DB cached in a named volume (`libreportal-trivy-cache`, ~600 MB on disk); the DB refresh is the only network traffic.
+- Image access via the rootless docker socket mounted read-only into the ephemeral container (same daemon that owns the images), run as `dockerinstall` like every other docker op.
+- **Privacy posture (worth stating in user-facing copy):** nothing about your images or apps ever leaves the box — Trivy matches locally against a downloaded DB; the only outbound call is the DB fetch from ghcr. Default **on** (it's a security feature and the traffic is one public DB pull), with a single honest switch: `CFG_UPDATER_CVE_SCAN=on|off` (`off` for air-gapped boxes) and `CFG_UPDATER_CVE_INTERVAL` (minutes, **default 1440** — daily; scans are the expensive step, results don't change hourly). Both live in `configs/webui/webui_updater` next to the existing knobs.
+- Output maps straight onto the schema the frontend already renders: per app `cves[] { id, severity, package, fixed_in, url }` + global `totals`. Dedupe per image (shared base layers repeat findings), scan each distinct image once per run. **Drop the `[ ! -f ]` guard** so re-scans overwrite `cves.json`.
+- **Not alarmist** (house rule): the Security tab lists everything, but the per-app chip/badge only fires for **critical/high with a fix available** — "your box has 400 unfixable medium CVEs" red badges are noise, not signal.
+- **Tie-in with updates:** a CVE whose `fixed_in` is satisfied by the available build marks that update as a *security update* — the Security filter chip and severity sort already exist in the UI, they just start meaning something.
+
+## 6. Auto-update policy (deliberately later)
+
+Once detection + pinned apply are trustworthy, add `CFG_UPDATER_AUTO=off|security|all` (default **off**): `security` auto-enqueues `updater_apply` only for security updates (mirrors `CFG_HOTFIX_AUTO`'s severity-split precedent, and like `artifactApplyAuto` it only *enqueues tasks*). Not part of the initial build — auto-updating before the revert story is proven live would be backwards.
+
+## 7. Build phases (each independently shippable)
+
+1. **P1 — Pin foundation.** Factor the compose image-line rewrite into a shared helper (artifact op + updater both use it, all `image:` lines not just the first). Pin-on-install + reconcile-on-first-scan. No UI change yet.
+2. **P2 — Real detection.** Registry digest compare in `webuiUpdaterScan` behind the new interval knob; `updates.json` gains services/digests; fleet + per-app tabs light up with truthful data. Emit recovery fields while in there.
+3. **P3 — Pinned apply/revert.** `updaterApplyApp`/`updaterRollbackApp` rewrite pins as in §4; history carries digest refs; failure path restores pins. *After this, the Update and Roll back buttons are honest.*
+4. **P4 — CVE scanner.** Ephemeral trivy runs, `cves.json` for real, guard dropped, totals + severity chips live, security-update tie-in.
+5. **P5 — Polish.** Per-app critical-CVE / update chip on the app header (pattern exists for improvements), remote version-label enrichment, "N services" aggregation in rows.
+6. **P6 — Auto-update policy** (§6), only after P3 has soaked on a real install.
+
+## 8. Rejected alternatives (for the record)
+
+- **`CFG_<APP>_VERSION` config vars** — second source of truth vs the compose file, collides with the `_TAG` placeholder namespace, and doesn't solve floating `latest` by itself.
+- **Pin semver tags across all templates** — permanent curation burden, inconsistent upstreams, still blind to republished tags.
+- **Watchtower-style auto-pull of latest** — maximum freshness, zero visibility, no revert; the exact opposite of "monitor, update deliberately, roll back."
+- **Docker Scout / hosted scanners** — requires accounts / sends data off-box; against the ethos.
+- **A new top-level "Versions" area** — unnecessary; the fleet Updates tab + per-app Updates tab already are that surface, they just need real data.

docs/roadmap/updates-and-distribution.md

@@ -88,8 +88,8 @@ it a reason to be its own thing. Rename the concept from **"App Updater"** to
 **"Updates & Improvements"** — the single front door for *everything that changes your
 install from the outside*:
 
-- **App updates** (version bumps)
+- **App updates** (version bumps — the detection/pinning/revert design is [app-version-updater-and-cve.md](app-version-updater-and-cve.md))
-- **Security** (CVEs — the urgent stuff)
+- **Security** (CVEs — the urgent stuff; scanner design in the same doc)
 - **Hotfixes** (curated small improvements — §1)
 - **Recovery** (the safety net that makes all of it safe to apply)
 - **History** (audit trail of everything applied)

scripts/install/install_ufw.sh

@@ -59,7 +59,7 @@ installUFW()
             local result; result=$(yes | runSystem ufw logging $CFG_UFW_LOGGING)
             checkSuccess "Disabling UFW Firewall Logging"
 
-            isSuccessful "UFW Firewall has been installed, you can use ufw status to see the status"
+            isSuccessful "UFW Firewall has been installed"
 
             menu_number=0   
             cd

scripts/source/files/generate_function_manifest.sh

@@ -275,6 +275,15 @@ fi
     done < <(printf '%s\n' "${eager_files[@]}" | sort -u)
     printf ')\n\n'
 
+    printf '# Shared autoload helper. A stub routes its source through this so a\n'
+    printf '# TRANSIENTLY-absent target file (the scripts tree is wiped then\n'
+    printf '# repopulated mid-deploy — see update.sh) is waited for briefly rather\n'
+    printf '# than failing outright. Bounded (~5s) and only while the file is\n'
+    printf '# unreadable, so a genuinely-missing file still fails fast. Defined in\n'
+    printf '# this sourced manifest so it stays in memory even while on-disk files\n'
+    printf '# momentarily vanish.\n'
+    printf '__lpAutoload() { local __f="$1" __i; for ((__i=0; __i<20; __i++)); do [ -r "$__f" ] && break; sleep 0.25; done; source "$__f"; }\n\n'
+
     printf '# Autoload stubs — one per public function. First call unsets the\n'
     printf '# stub, sources the real file (which redefines the function), then\n'
     printf '# re-invokes. The `unset -f` first means a failed source degrades to a\n'
@@ -287,7 +296,7 @@ fi
             containers) base_var='install_containers_dir' ;;
             *)          base_var='install_scripts_dir' ;;
         esac
-        printf '%s() { unset -f %s; source "${%s}%s"; %s "$@"; }\n' \
+        printf '%s() { unset -f %s; __lpAutoload "${%s}%s"; %s "$@"; }\n' \
             "$name" "$name" "$base_var" "${fn_to_file[$name]}" "$name"
     done < <(printf '%s\n' "${!fn_to_file[@]}" | sort)
 } > "$OUTPUT"

scripts/webui/data/lock/webui_check_update_lock.sh

@@ -1,17 +1,42 @@
 #!/bin/bash
 
 # LibrePortal WebUI Update Lock Check
-# Checks for update lock file to prevent concurrent updates
+# Guards against concurrent WebUI data refreshes.
-
+#
+# Echoes its verdict ("true" = a live lock is held, skip; "false" = clear to
+# proceed) on stdout, and auto-clears a STALE lock. Callers must capture the
+# echo — `result=$(webuiCheckUpdateLock)` runs the function in a subshell, so a
+# global it set would never reach the caller (that was a real bug: the guard
+# read an always-empty global and so never actually blocked anything).
+#
+# Staleness matters because the lock's remover (webuiRemoveUpdateLock) is itself
+# a lazy-loaded function whose backing file can be transiently missing while the
+# scripts tree is wiped+repopulated mid-deploy. If that removal is skipped once,
+# the leftover lock would otherwise wedge EVERY future refresh. No single refresh
+# runs anywhere near this long, so a lock older than the threshold is a leftover.
 webuiCheckUpdateLock() {
     local lock_file="$containers_dir/libreportal/frontend/data/updater.lock"
+    local stale_after=900   # seconds (15 min); far longer than any real refresh
 
-    lock_file_found=""
+    if [ ! -f "$lock_file" ]; then
-    if [ -f "$lock_file" ]; then
+        isNotice "No update lock file found" >&2
-        isNotice "Update lock file exists: $lock_file"
+        echo "false"
-        lock_file_found="true"
+        return 0
-    else
-        isNotice "No update lock file found"
-        lock_file_found="false"
     fi
+
+    local now lock_mtime age
+    now=$(date +%s 2>/dev/null || echo 0)
+    lock_mtime=$(stat -c '%Y' "$lock_file" 2>/dev/null || echo 0)
+    age=$(( now - lock_mtime ))
+
+    if (( now > 0 && lock_mtime > 0 && age >= stale_after )); then
+        isNotice "Stale update lock (${age}s old ≥ ${stale_after}s) — clearing and continuing." >&2
+        runFileOp rm -f "$lock_file" >/dev/null 2>&1
+        echo "false"
+        return 0
+    fi
+
+    isNotice "Update lock file exists: $lock_file" >&2
+    echo "true"
+    return 0
 }

scripts/webui/webui_updater.sh

@@ -26,8 +26,11 @@ webuiLibrePortalUpdate() {
     if [ "$status" == "installed" ]; then
         isHeader "LibrePortal WebUI Updater"
 
-        # Check for update lock file first
+        # Check for update lock file first. webuiCheckUpdateLock echoes its
-        local result; result=$(webuiCheckUpdateLock)
+        # verdict on stdout (and auto-clears a stale lock); capture it directly.
+        # A $(...) subshell means any global the function sets never reaches us
+        # here, so the verdict MUST come back via the echo, not a shared var.
+        local lock_file_found; lock_file_found=$(webuiCheckUpdateLock)
         checkSuccess "Checked for update lock file."
 
         if [[ "$lock_file_found" == "true" ]]; then