The digest compare only ever asks about the tag already pinned, so it
answers "has my tag been rebuilt?" and can never answer "does a newer
version exist?". An app on v0.16 reports up to date forever while 0.17
ships. That is the gap between an app that updates and an app that is
current, and it silently affects every pinned app.
Adds tag enumeration for VERSIONED tags only (rolling tags already move
on their own): list the repo's tags, keep those sharing the current tag's
SHAPE, and pick the numerically greatest.
Shape matching is the whole safety story — v0.16 -> v#.# so it can never
"upgrade" you onto v0.16-alpine, 31-fpm-alpine onto 31-apache, or a date
tag onto a semver one. Comparison is component-wise numeric, so 0.10 > 0.9
and 1.0 > 0.99 (a string sort gets both wrong), with 10# forcing base ten
so an upstream "08" cannot be read as octal. 15 unit tests cover it.
Docker Hub only, deliberately: all three pinned apps live there, it needs
no auth, and the generic OCI tags/list wants a per-registry token dance.
Other registries stay quiet rather than guess. Throttled inside the
existing registry window and cached between windows so it cannot flicker.
Surfaced as INFORMATION, never an action: no button applies it, because a
version move can carry a data migration. `update_available` and the "up
to date" badge keep their exact meaning; the new state sits beside them
and points at the Version field.
Against the live registry: stalwart v0.16 is current, nextcloud is on
31-fpm-alpine with 34-fpm-alpine out, mastodon on v4.2.0 with v4.6.5 out.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four fixes that make the auto-updater a trustworthy background system:
* CFG_UPDATER_WINDOW (default 06:00-08:00 host time, right after the
05:00 backup cron; HH:MM-HH:MM wraps midnight, 'always' = any time).
Gates only the enqueue — scans keep running all day, so the Updates
page stays current and pending updates visibly wait for the window.
Malformed values fail closed and are rejected by the WebUI validator.
* "Check now" actually checks: an explicit `updater check` sets
UPDATER_REGISTRY_FORCE=1. The flag existed but nothing ever set it,
so the button silently reused the 6h digest cache and could not find
a build the user knew had shipped. Force also overrides interval 0,
which now means "manual-only" as documented in the roadmap.
* Registry stamp moved from /tmp to <system>/logs: the task processor
runs under PrivateTmp, so daemon and CLI each kept a separate 6h
clock and the daemon's reset on every service restart.
* A failed automatic attempt is no longer invisible: the scan emits
auto_attempted_digest (the one-shot no-retry stamp), and when it
matches the available build the UI stops promising an install that
will never come — per-app detail explains, the fleet row gets an
"auto failed" chip, and the Overview board counts it as needing you.
Also corrects the CFG_TIMEZONE label: it sets the containers' TZ only;
scheduled tasks follow the host clock (timedatectl), and the old
"Timezone for scheduled tasks" wording promised a knob that never
existed. The window + auto_window display state plainly WHEN updates
land, answering "how does the user know when the next update happens".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds the decision half of the app updater. Detection (P2) and the
snapshot-first apply/revert (P3) were already real, but nothing ever
pressed the button — every update waited for a click.
CFG_<APP>_UPDATE_TYPE=auto|manual per app, default auto (33 templates)
CFG_UPDATER_AUTO=true|false master switch, default true
updaterAppPolicy resolves the two the way backupResolveStrategy already
resolves backup strategy: the global switch can only make things more
manual. updaterApplyAuto runs at the end of `updater check` and enqueues
the ordinary updater_apply task for each auto app that has an update —
never applies inline, so an automatic update is the same code path, task
log, History entry and Roll back button as a manual one.
Safety: each attempt stamps its target digest under generated/auto/, so a
build that fails is rolled back and then left alone rather than retried on
every scan; in-flight updater tasks are skipped so scans can't stack.
Tracked end to end: updates.json carries each app's resolved update_type,
History entries carry trigger=manual|auto. The WebUI says whether updates
install themselves, chips only the apps that opted out, labels automatic
history, and — since an auto app's pending update needs no decision — keeps
it off the Overview board's "Needs action" view.
Also fixes artifactApplyAuto enqueueing without --detach: it runs inside
the single-threaded task processor's own poll, so following the new task in
the foreground waits for a task that cannot start until it returns.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The per-app Updates tab pushed a manual "Check" button (header + empty state)
even though scans run automatically on CFG_UPDATER_SCAN_INTERVAL — so an app with
nothing to update read like an empty/actionable page. Replace the manual Check
with a calm status line inside the panel: "Checked automatically · last checked X
· next check ~Y", backup-schedule style. The genuine Apply/Roll back actions stay
(applying is still manual and safe). No auto-apply.
- webui_updater_scan.sh: stamp scan_interval_minutes alongside generated_at in
updates.json so the display needs no separate config fetch (0 = auto off).
- updater-page.js: renderAutoCheckLine() + fmtRelFuture().
- app-tabbed-manager.js: drop the header/empty-state Check buttons; render the
auto-check line; friendlier no-data copy.
- overview.css: style .updater-autocheck (green dot live / muted when off).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The per-app object build used `available_version:($available_version|select(.!=""))`
to omit the field when empty. But in jq, a `{key: (empty)}` makes the WHOLE
object construction emit nothing — so every app with no available update (empty
available_version) produced no object and was silently dropped from updates.json.
Only apps WITH a pending update survived; an all-up-to-date fleet showed an empty
list. (Missed in P2's sandbox test because both fixture apps had updates.)
Emit an explicit null instead: `(if $available_version=="" then null else … end)`.
Verified: an up-to-date app (trivy, local==registry digest) now emits with
available_version:null; apps with updates still carry the string.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
webuiUpdaterScan now tells the truth instead of hardcoding update_available=false.
Per app it resolves three facts (see roadmap §2): the running RepoDigest (one
`docker inspect` of the anchor image), the display version (OCI label →
versioned tag → channel·shortdigest), and — throttled — the registry's current
index digest for the channel tag (`docker buildx imagetools inspect`, the same
identity as RepoDigest, verified exact). update_available = the two digests
differ. Emits type (versioned|rolling), channel, current/available digests +
versions, and a services[] array (every image line, anchor flagged).
- Registry lookups throttled separately from the scan: CFG_UPDATER_REGISTRY_
INTERVAL (min, default 360; 0 = local-only), own /tmp stamp, reuse of the
prior available_digest between windows so the app list still refreshes every
scan. UPDATER_REGISTRY_FORCE=1 forces a live pull (the Check-now button).
Registry failure (offline/rate-limited) = "unknown", never a false "changed".
- Digest-compare fully detects rolling apps' new builds; for versioned apps it
catches rebuilds of the pinned tag (newer-version enumeration is a later
step) — honest per type, and versioned apps are user-picked via P1b anyway.
- Fixes a P1b regression: updaterPrimaryImage now strips the trailing version
sentinel comment (`s/ #.*//`) via _updaterCleanImageRef — without this the
anchor ref (and the live CVE scanner's image arg) carried the comment.
- JSON built with jq for safe escaping; jq-less fallback keeps output valid.
Verified via a simulated 2-app install: navidrome → "0.62.0" (OCI label) +
detected update; rolling app → channel·shortdigest + update; throttled re-run
reuses the prior digest; all output valid JSON.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The scan read the app's "current image" via `grep -m1 image:` — the first
image line. For apps that declare a companion service first this mislabels the
app: ollama lists `ollama-webui` (open-webui) before `ollama-service`
(ollama/ollama), so an app named ollama reported open-webui's version.
Add `updaterPrimaryImage`: resolve the version anchor from the app's
`<slug>-service` image (the universal primary-service naming convention,
33/33 apps; underscores→hyphens for slugs like libreportal_catalog). Falls back
to the first image line off-convention. Used by both the version and CVE loops.
Verified: only ollama changes (→ ollama/ollama:latest); nextcloud, mastodon,
jitsi, gitea, vaultwarden anchors unchanged.
Also revise docs/roadmap/app-version-updater-and-cve.md to the config-first
direction agreed this session:
- CFG_<APP>_VERSION ADOPTED (was rejected): the #LIBREPORTAL tagging system
makes the config the source (compose tag derived from it), not a second one.
- Two version TYPES — versioned (real tag, version picker) vs rolling (floating
channel, digest-freshness); digest is the uniform detection engine for both,
version numbers are display enrichment. Answers "why not just compare numbers"
(most upstreams publish none; no universal latest-version API).
- Multi-service anchor = <slug>-service, not first line; lock-step sets (jitsi)
= one channel → several image lines; sidecars tracked-by-digest, not headline.
- Phases updated: P0 anchor (done) → P1 config-first pin → P2 detection+display.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Trivy runs as a server whose vulnerability DB downloads on first boot; until it
lands no scan can produce results. Previously the updater generator wrote an
empty-but-valid cves.json the moment the file was missing, so installing Trivy
painted a green "no known vulnerabilities" all-clear that was actually a lie —
the DB hadn't even downloaded, and the Updates/Security view gave no signal.
Add an honest scanner state the WebUI branches on:
- containers/trivy/scripts/trivy_scan.sh — trivyScannerState (absent |
db_updating | ready) via `trivy version -f json`, trivyDbUpdatedAt, and
trivyScanImageCves (per-image scan normalized to {id,severity,package,
installed,fixed_in,url}, deduped). All degrade safely on error.
- webui_updater_scan.sh — stamp cves.json with scanner.state; only run real
per-image scans once the DB is ready. Always rewritten so state tracks live.
- updater-page.js — Security tab shows a loading box while the DB updates, an
install nudge when absent, and the genuine 🎉 only when ready+empty; Overview
CVE card sub + hint reflect the state.
- overview-manager.js — fleet Security row surfaces the "building CVE database"
pending state instead of silently omitting.
- function_manifest.sh — regenerated for the new trivy_scan.sh functions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
webuiRegistryCatalogScan (run by updater check, same atomic keep-prior
pattern as webuiArtifactScan) writes apps/generated/registry_catalog.json:
the type:"app"/kind:"bundle" rows of the signed index annotated with
defined/installed, browse metadata from the envelope meta, and icons
mirrored into core/icons/apps/registry/ ONLY when their bytes match the
sha256 pin in the signed index — the browser stays same-origin; a tampered
or oversized icon is skipped, never served.
webuiArtifactScan now selects type=="hotfix" so app rows never render as
pseudo-hotfixes in the Improvements tab, and counts+logs artifacts of
unrecognized type instead of surfacing them (the §8.1 forward-compat
firewall on the scan path).
Harness vs a locally served registry: 14/14 (catalog row + meta + flags,
icon pin verify + tamper skip, hotfix-only stream, unknown-type skip+log,
unreachable-registry keeps prior files).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Every caller captured the index with var=$(lpFetchIndex), which runs the
fetch in a command-substitution subshell — the LP_INDEX_SIGSTATE global it
sets never reached the caller. On a box with real signing active the
artifactApply/apply-auto gates would therefore refuse a correctly signed
index (fail-closed, but the apply path would be dead on arrival the day
signing activates), and artifact index / the WebUI scan would report a
verified feed as UNSIGNED.
New lpFetchIndexInto <var> [cache] runs the fetch in the calling shell and
assigns via printf -v; all four call sites converted. Verified with a
source-and-mock harness against a locally served index: 10/10 (sigstate
reaches caller, serial high-water, anti-rollback refuse, staleness refuse,
id enumeration, envelope round-trip).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Replace the click-to-scan-only flow with a self-throttled auto-scan that
rides the existing task-processor idle poll (the same shape as the
network-drift check — no new daemon, unit, or endpoint):
- 'libreportal updater check auto' gates on the age of the generated
updates.json vs CFG_UPDATER_SCAN_INTERVAL (minutes, default 30,
0 disables); a fresh file makes the 60s tick a single stat() + return.
Manual checks and post-update rescans reset the clock for free, and a
missing file means the first scan runs ~a minute after install.
- Eligible signed hotfixes keep flowing through artifactApplyAuto, which
only enqueues ordinary tasks — mutations stay on the task path.
- Open updater surfaces (standalone /updater and the fleet Overview's
headless UpdaterPage) follow along with a 60s static-JSON re-read that
repaints only when a generated_at stamp changed; timer released via
dispose() on unmount, ticks skipped while hidden.
- Empty states now say the first scan happens automatically; Check now
stays as the immediate manual override.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
- CFG_HOTFIX_AUTO (security-breakage|all|off, default security-breakage) seeded in
general_terminal; reaches existing installs via the add-only config reconciler.
- webui_artifact_scan.sh (webuiArtifactScan): fetch+verify the signed index, write
artifacts_available.json ATOMICALLY (build in temp → jq-validate → one write;
keep the prior file on any failure — never emits broken JSON). Annotates each
artifact with applied (a per-id record exists) + applicable (target installed).
- artifactApplyAuto + `libreportal artifact apply-auto`: enqueue apply tasks for
the eligible signed hotfixes — only when the index is VERIFIED-signed, only
auto==true + in the severity policy + applicable + not already applied. Each
apply is its own task (visible in the log + History), never applied inline.
- `updater check` now also refreshes the index (webuiArtifactScan) and runs
artifactApplyAuto — one front door, no second phone-home.
Unit-tested 13/13: policy filtering (security-breakage / off / all), auto:false
exclusion, already-applied skip, non-installed-app skip, unsigned-index fail-closed,
and the scan transform's signed/applied/applicable fields.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Two more cases of the manager writing directly into the container-owned
/libreportal-containers tree (same class as the regen-poll stamp), both masked
by a '✓ Success' that printed anyway:
- Password replacers (config/password/*): used 'runInstallOp sed -i' (manager)
on app configs copied into the container tree, so sed -i EACCES'd its temp
file and the substitution silently failed — the adguard.config 'couldn't open
temporary file', leaving the literal RANDOMIZEDPASSWORD placeholder. Added
runCfgOp (picks runFileOp vs runInstallOp by the target file's location) and
routed every $file grep/sed/awk through it: password, username, hex, vapid,
appkey, and bcrypt.
- Updater generator (webui_updater_scan): 'runFileOp cp <manager-tmp>' can't
read the manager's 0600 mktemp as the container user, so it fell through to a
manager 'cp' that EACCES'd on the container-owned out_dir. Switched the three
writes to 'runFileWrite < tmp' (manager shell reads the tmp; container user
tees the write).
Both deploy via the normal quick path (relocatable scripts) — no footprint bump,
no reinstall.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
- scripts/webui/data/generators/updater/webui_updater_scan.sh (webuiUpdaterScan):
writes frontend/data/updater/generated/{updates,cves,history}.json from the
installed-apps DB (current image per app from compose). Available-version +
CVE-scanner are clearly-marked pluggable hooks; always emits valid JSON.
- scripts/cli/commands/updater/{cli_updater_commands.sh,cli_updater_header.sh}:
auto-dispatched as 'libreportal updater <sub>' (check/apply/apply-all/rollback).
apply does disaster-recovery FIRST — snapshots the app via the backup engine,
then pulls + recreates (real dockerComposeUp/compose-pull helpers), records
history, and auto-rolls-back on failure. Standard LIBREPORTAL_TASK_EXEC
enqueue/exec split so WebUI + CLI share locking + audit trail.
New .sh files: the array/function-manifest regen self-heals on deploy; the
check path also sources its generator on demand to cover the gap.
NOTE: host-side bash — written to the repo's conventions but not runnable in
this env; this is the surface to test (the WebUI feature is lp-shot-verified).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
