Two halves: the ladder could not climb the commonest versioning scheme,
and nothing ever climbed it on its own.
The ladder stepped by bumping a tag's LAST numeric component, so
v1.158.0 went v1.158.1, v1.158.2, … and never arrived at v1.159.0. It
then failed closed, refusing to build a path. Synapse publishes
v1.159.0 and no v1.158.1 at all, so Matrix could not be laddered by the
button either — three-part semver minor bumps were simply unreachable.
updaterNextRung now considers a bump of every component, keeps the
candidates that exist upstream and takes the smallest: the next release
by definition, whether it lands in the patch position or crosses into a
new major. Shape discipline is unchanged, so 31-fpm-alpine still never
becomes 31-apache, and each rung is still probed, so none can be
skipped. updaterTagBumpAt moves here from the scan, its natural home,
which also breaks a source cycle.
updaterUpgradeAuto then climbs at most ONE rung per app per calendar
day, inside the install window, for apps set to auto. One rung because a
ladder run unattended can be several migrations deep before anyone
looks, and "restore the snapshot from a minute ago" stops comforting
once four have stacked; one a day so there is time to notice. It crosses
a major if that is genuinely the next release — refusing would strand an
app on the last version of its line forever — but one step at a time,
never as a leap. Two stamps: the target rung (a failure is not retried
until something newer ships) and the day.
Every rung goes through updaterUpgradeApp unchanged, so GATE 1 still
refuses any app without a real verifier, and the per-rung contract is
identical to the button: snapshot fail-closed, set version, pull, up,
verify, restore that rung and stop on any failure. History now records
the trigger instead of hardcoding "manual", including on the rollback
paths. CFG_UPDATER_LADDER_AUTO gates the whole thing separately from
CFG_UPDATER_AUTO, because "keep my apps patched" and "move my apps
between versions unattended" are different appetites for risk.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`updater upgrade <app> --detach` parsed "--detach" as the target version
and refused with "no safe path from 31-fpm-alpine to --detach". It failed
safe, but blaming the version for a misplaced flag is a poor way to say
the flag is not supported here. Unknown options now say so.
Found during the first live ladder run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ties the ladder and the verifiers together behind a new verb:
libreportal updater upgrade <app> [version] [--dry-run]
Per rung, and every part is load-bearing:
snapshot (fail-closed) -> set version -> pull -> up -> VERIFY -> next
On failure anywhere: restore THIS rung's snapshot, put the version back,
stop, and leave the app on the last version it actually verified at. The
ladder never continues past a doubt.
A snapshot PER RUNG rather than one at the start, because upstream
migrations are usually one-way — Nextcloud 32's schema cannot be undone
by putting the 31 image back. The recovery guarantee is "restore the
snapshot from sixty seconds ago", which only holds if every rung has one.
Two gates before anything moves. An app with no <app>_upgrade_verify is
refused outright: the generic health check cannot see a half-finished
migration, so laddering on it would be a guess wearing a safety label.
And a ladder that cannot be computed end to end refuses rather than
attempting a partial climb.
`updater upgrade` is a separate verb from `apply` on purpose: apply moves
you WITHIN a release line (and may be automatic), upgrade moves you
BETWEEN lines and is always a deliberate act. Dry runs execute inline so
the plan is instant to read.
updaterSetAnchorVersion rewrites the image tag AND its version sentinel
together — updating only the image would leave the sentinel advertising
the old version, and the next config regeneration would silently revert
the app.
Tested with stubs against the real code paths: the no-verifier gate holds
and changes nothing; a dry run has zero side effects; the happy path
snapshots at each current version before moving; a verify failure on rung
2 of 3 stops with the app on rung 1, restored, and never touches rung 3;
a failed snapshot moves no version and pulls nothing; a container that
will not start is rolled back.
NOT yet exercised on a live install — no app here needs a ladder. The
first real run should be a dry run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four fixes that make the auto-updater a trustworthy background system:
* CFG_UPDATER_WINDOW (default 06:00-08:00 host time, right after the
05:00 backup cron; HH:MM-HH:MM wraps midnight, 'always' = any time).
Gates only the enqueue — scans keep running all day, so the Updates
page stays current and pending updates visibly wait for the window.
Malformed values fail closed and are rejected by the WebUI validator.
* "Check now" actually checks: an explicit `updater check` sets
UPDATER_REGISTRY_FORCE=1. The flag existed but nothing ever set it,
so the button silently reused the 6h digest cache and could not find
a build the user knew had shipped. Force also overrides interval 0,
which now means "manual-only" as documented in the roadmap.
* Registry stamp moved from /tmp to <system>/logs: the task processor
runs under PrivateTmp, so daemon and CLI each kept a separate 6h
clock and the daemon's reset on every service restart.
* A failed automatic attempt is no longer invisible: the scan emits
auto_attempted_digest (the one-shot no-retry stamp), and when it
matches the available build the UI stops promising an install that
will never come — per-app detail explains, the fleet row gets an
"auto failed" chip, and the Overview board counts it as needing you.
Also corrects the CFG_TIMEZONE label: it sets the containers' TZ only;
scheduled tasks follow the host clock (timedatectl), and the old
"Timezone for scheduled tasks" wording promised a knob that never
existed. The window + auto_window display state plainly WHEN updates
land, answering "how does the user know when the next update happens".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
First real end-to-end auto-update on a live install failed like this:
Automatically updating trivy (a recovery snapshot is taken first)
Snapshotting trivy before update…
Pulling new image(s) for …
Update of failed — rolling back…
Could not roll back automatically
The app name went empty after the snapshot. Cause: bash is dynamically
scoped, so a callee assigning an undeclared variable writes the CALLER's
local of that name — and a `while read app` loop leaves it EMPTY at EOF.
webuiBackupAppStatus's dashboard generator runs at the end of every backup
and did exactly that to updaterApplyApp's `app`.
Nothing was damaged: the pull ran against an empty name, failed before
touching the image, and the rollback was a no-op on a nonexistent app.
Fixed both ends. The generator (and three gluetun loops with the same
latent leak) now declare `local app`. updaterApplyApp/updaterRollbackApp
hold the name in `_upd_app` so they no longer depend on every callee's
hygiene, and updaterApplyAll stops leaking its own loop var.
This is exactly the untested path the roadmap flagged: "apply/revert not
yet exercised end-to-end on a live install with a pending update."
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds the decision half of the app updater. Detection (P2) and the
snapshot-first apply/revert (P3) were already real, but nothing ever
pressed the button — every update waited for a click.
CFG_<APP>_UPDATE_TYPE=auto|manual per app, default auto (33 templates)
CFG_UPDATER_AUTO=true|false master switch, default true
updaterAppPolicy resolves the two the way backupResolveStrategy already
resolves backup strategy: the global switch can only make things more
manual. updaterApplyAuto runs at the end of `updater check` and enqueues
the ordinary updater_apply task for each auto app that has an update —
never applies inline, so an automatic update is the same code path, task
log, History entry and Roll back button as a manual one.
Safety: each attempt stamps its target digest under generated/auto/, so a
build that fails is rolled back and then left alone rather than retried on
every scan; in-flight updater tasks are skipped so scans can't stack.
Tracked end to end: updates.json carries each app's resolved update_type,
History entries carry trigger=manual|auto. The WebUI says whether updates
install themselves, chips only the apps that opted out, labels automatic
history, and — since an auto app's pending update needs no decision — keeps
it off the Overview board's "Needs action" view.
Also fixes artifactApplyAuto enqueueing without --detach: it runs inside
the single-threaded task processor's own poll, so following the new task in
the foreground waits for a task that cannot start until it returns.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Make the Update / Roll-back buttons tell the truth, closing the "new code on
old data" hole a floating tag creates.
updaterApplyApp:
- Anchor-correct: capture before/after from updaterPrimaryImage (the
<slug>-service image), not `grep -m1 image:` — fixes ollama et al.
- Records EXACT build refs in history from->to: repo:tag@sha256:<digest>
(via updaterRefDigest), so history is meaningful even when the tag doesn't
move (a rebuilt `latest`).
- Un-pins any digest a prior rollback pinned before pulling, so Update tracks
the channel again instead of freezing on the rolled-back build.
updaterRollbackApp:
- Before recreating, re-pins the anchor image to the pre-update build's digest
(from history's last update/ok `from`) via updaterSetAnchorRef, so `up` runs
the OLD code — not the current channel head. This is the fix for restoring a
data snapshot but recreating on a newer image.
New helpers (cli_updater_commands.sh): updaterRefDigest (local RepoDigest),
updaterSetAnchorRef (rewrite the anchor image line by service name, preserving
indent + the version sentinel; correct for companion-first apps like ollama),
updaterLastUpdateFrom (roll-back target from history).
Verified the helpers on nextcloud + ollama: pin adds @sha256 to the right
anchor only, sidecars untouched, sentinel preserved, unpin restores, YAML valid.
Caught and fixed a `local a=$1 b=...$a...` same-statement expansion bug that
would have silently no-op'd the rollback pin. End-to-end apply/revert not
exercised live here (no installed app has a pending update on this box).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
webuiRegistryCatalogScan (run by updater check, same atomic keep-prior
pattern as webuiArtifactScan) writes apps/generated/registry_catalog.json:
the type:"app"/kind:"bundle" rows of the signed index annotated with
defined/installed, browse metadata from the envelope meta, and icons
mirrored into core/icons/apps/registry/ ONLY when their bytes match the
sha256 pin in the signed index — the browser stays same-origin; a tampered
or oversized icon is skipped, never served.
webuiArtifactScan now selects type=="hotfix" so app rows never render as
pseudo-hotfixes in the Improvements tab, and counts+logs artifacts of
unrecognized type instead of surfacing them (the §8.1 forward-compat
firewall on the scan path).
Harness vs a locally served registry: 14/14 (catalog row + meta + flags,
icon pin verify + tamper skip, hotfix-only stream, unknown-type skip+log,
unreachable-registry keeps prior files).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Replace the click-to-scan-only flow with a self-throttled auto-scan that
rides the existing task-processor idle poll (the same shape as the
network-drift check — no new daemon, unit, or endpoint):
- 'libreportal updater check auto' gates on the age of the generated
updates.json vs CFG_UPDATER_SCAN_INTERVAL (minutes, default 30,
0 disables); a fresh file makes the 60s tick a single stat() + return.
Manual checks and post-update rescans reset the clock for free, and a
missing file means the first scan runs ~a minute after install.
- Eligible signed hotfixes keep flowing through artifactApplyAuto, which
only enqueues ordinary tasks — mutations stay on the task path.
- Open updater surfaces (standalone /updater and the fleet Overview's
headless UpdaterPage) follow along with a 60s static-JSON re-read that
repaints only when a generated_at stamp changed; timer released via
dispose() on unmount, ticks skipped while hidden.
- Empty states now say the first scan happens automatically; Check now
stays as the immediate manual override.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
- CFG_HOTFIX_AUTO (security-breakage|all|off, default security-breakage) seeded in
general_terminal; reaches existing installs via the add-only config reconciler.
- webui_artifact_scan.sh (webuiArtifactScan): fetch+verify the signed index, write
artifacts_available.json ATOMICALLY (build in temp → jq-validate → one write;
keep the prior file on any failure — never emits broken JSON). Annotates each
artifact with applied (a per-id record exists) + applicable (target installed).
- artifactApplyAuto + `libreportal artifact apply-auto`: enqueue apply tasks for
the eligible signed hotfixes — only when the index is VERIFIED-signed, only
auto==true + in the severity policy + applicable + not already applied. Each
apply is its own task (visible in the log + History), never applied inline.
- `updater check` now also refreshes the index (webuiArtifactScan) and runs
artifactApplyAuto — one front door, no second phone-home.
Unit-tested 13/13: policy filtering (security-breakage / off / all), auto:false
exclusion, already-applied skip, non-installed-app skip, unsigned-index fail-closed,
and the scan transform's signed/applied/applicable fields.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The mutating side of the unified distribution primitive (spec §8.3). Hotfixes
can now be applied and reverted, first-party, through the task system.
New scripts/cli/commands/artifact/cli_artifact_apply.sh:
- artifactApply <id>: resolve+gate (applies_when / min_lp / max_lp /
max_footprint / publishers-map role) → fetch+verify payload (sha256 pinned by
the signed index + minisig) → dry-precheck ALL ops (all-or-nothing) → best-
effort snapshot → apply each op recording a precise inverse → bring app up →
auto-rollback (replay undo LIFO, snapshot fallback) → applied-record + History.
- artifactRevert <id>: replay the applied-record's undo log (LIFO).
- Bounded, CLOSED op vocabulary (no run-script/exec, ever): set-config-key,
set-compose-image, patch-file-if-checksum-matches, set-data-file. An
unsupported op rejects the whole artifact at precheck (fail-closed).
- Write-target firewall: scope:app → containers/<app>/ only; scope:system →
configs/ only; the install tree (our code) is off-limits to hotfixes (fork 1).
Drift guards (expect_current / checksum) skip cleanly rather than clobber.
- Two-tier trust: index minisig-verified vs the footprint key (lpFetchIndex)
covers the envelope; payload sha256-pinned + minisig-verified; publishers-map
role gate (a non-official publisher can't claim official). Community per-
artifact-key sigs are gated off until that tier is enabled.
cli_artifact_commands.sh: apply/revert via the task system (artifact_apply /
artifact_revert types — no allowlist needed), + read-only `applied` list.
cli_updater_commands.sh:
- FIX verified safety bug: updaterApplyApp/RollbackApp called `libreportal backup
app "$app"` and `... restore latest`, which parse the app name as the ACTION,
hit the dispatcher's `*)` default (exits 0) — so updates ran with NO snapshot
and rollback was a silent no-op. Call backupAppStart / restoreAppStart directly.
- FIX updaterRecordHistory jq-silent-skip: was `command -v jq || return 0`
(silently dropped the audit entry). Now fail-closed with a brace-agnostic
bash-native prepend fallback; extended with artifact_id/serial/undo_id.
fetch.sh: add _lpJsonEsc (shared JSON-escape for the jq-free fallbacks).
Regenerated source arrays + lazy-load manifest for the new file/functions.
Unit-tested 31/31: every op apply+precheck+undo round-trip, the path-allowlist
firewall (incl. .. traversal + install-tree + cross-app rejection), all-or-
nothing abort, unsupported-op rejection, and the History bash-native fallback
(records + preserves prior entries without jq). A full signed-apply e2e needs
minisign + the signing key (Phase 5 make_hotfix.sh).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
- scripts/webui/data/generators/updater/webui_updater_scan.sh (webuiUpdaterScan):
writes frontend/data/updater/generated/{updates,cves,history}.json from the
installed-apps DB (current image per app from compose). Available-version +
CVE-scanner are clearly-marked pluggable hooks; always emits valid JSON.
- scripts/cli/commands/updater/{cli_updater_commands.sh,cli_updater_header.sh}:
auto-dispatched as 'libreportal updater <sub>' (check/apply/apply-all/rollback).
apply does disaster-recovery FIRST — snapshots the app via the backup engine,
then pulls + recreates (real dockerComposeUp/compose-pull helpers), records
history, and auto-rolls-back on failure. Standard LIBREPORTAL_TASK_EXEC
enqueue/exec split so WebUI + CLI share locking + audit trail.
New .sh files: the array/function-manifest regen self-heals on deploy; the
check path also sources its generator on demand to cover the gap.
NOTE: host-side bash — written to the repo's conventions but not runnable in
this env; this is the surface to test (the WebUI feature is lp-shot-verified).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
