LibrePortal/scripts/cli/commands/updater/cli_updater_upgrade.sh
librelad 0679fd65b2 feat(updater): stepped upgrade engine — climbs a ladder, verifying each rung
Ties the ladder and the verifiers together behind a new verb:

  libreportal updater upgrade <app> [version] [--dry-run]

Per rung, and every part is load-bearing:

  snapshot (fail-closed) -> set version -> pull -> up -> VERIFY -> next

On failure anywhere: restore THIS rung's snapshot, put the version back,
stop, and leave the app on the last version it actually verified at. The
ladder never continues past a doubt.

A snapshot PER RUNG rather than one at the start, because upstream
migrations are usually one-way — Nextcloud 32's schema cannot be undone
by putting the 31 image back. The recovery guarantee is "restore the
snapshot from sixty seconds ago", which only holds if every rung has one.

Two gates before anything moves. An app with no <app>_upgrade_verify is
refused outright: the generic health check cannot see a half-finished
migration, so laddering on it would be a guess wearing a safety label.
And a ladder that cannot be computed end to end refuses rather than
attempting a partial climb.

`updater upgrade` is a separate verb from `apply` on purpose: apply moves
you WITHIN a release line (and may be automatic), upgrade moves you
BETWEEN lines and is always a deliberate act. Dry runs execute inline so
the plan is instant to read.

updaterSetAnchorVersion rewrites the image tag AND its version sentinel
together — updating only the image would leave the sentinel advertising
the old version, and the next config regeneration would silently revert
the app.

Tested with stubs against the real code paths: the no-verifier gate holds
and changes nothing; a dry run has zero side effects; the happy path
snapshots at each current version before moving; a verify failure on rung
2 of 3 stops with the app on rung 1, restored, and never touches rung 3;
a failed snapshot moves no version and pulls nothing; a container that
will not start is rolled back.

NOT yet exercised on a live install — no app here needs a ladder. The
first real run should be a dry run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-13 00:04:33 +01:00

183 lines
8.4 KiB
Bash

#!/bin/bash
# Stepped upgrade engine — climbs a version ladder one rung at a time.
# ---------------------------------------------------------------------------
# For apps that cannot skip a release (Nextcloud refuses outright; databases
# refuse via their data directory), moving 31 -> 34 is not one update but three,
# each with a migration that must COMPLETE before the next begins.
#
# The per-rung contract, and every part of it is load-bearing:
#
# snapshot (fail-closed) -> set version -> pull -> up -> VERIFY -> next rung
#
# On any failure, at any point: restore THIS rung's snapshot, put the version
# back, stop, and leave the app on the last version it verified at. The ladder
# never continues past a doubt.
#
# Why a snapshot per rung rather than one at the start: upstream migrations are
# usually one-way. Nextcloud 32's schema changes cannot be undone by putting the
# 31 image back. So the recovery guarantee is "restore the snapshot taken sixty
# seconds ago", not "undo the upgrade" — which only works if each rung has its
# own restore point.
#
# Deliberately NOT automatic. CFG_<APP>_UPDATE_TYPE=auto applies patches within
# a line; crossing versions on stateful data stays a decision a person makes,
# after reading release notes. The updater surfaces "34 available"; this runs
# only when asked.
_updaterUpgradeGenDir() { echo "${containers_dir%/}/libreportal/frontend/data/updater/generated"; }
# Rewrite the anchor image AND its version sentinel, so the live compose stays
# self-consistent. updaterSetAnchorRef preserves the trailing comment verbatim,
# which would leave the sentinel advertising the OLD version — and the next
# config-driven regeneration would then quietly revert the app. Both or neither.
updaterSetAnchorVersion() {
local app="$1" newtag="$2"
local compose="${containers_dir%/}/$app/docker-compose.yml"
[ -f "$compose" ] || return 1
local svc="${app//_/-}-service"
local up; up="$(printf '%s' "$app" | tr '[:lower:]' '[:upper:]')"
local tmp; tmp="$(mktemp)"
awk -v s="$svc" -v tag="$newtag" -v key="${up}_VERSION_TAG" '
!done && seen && /^[[:space:]]*image:/ {
match($0,/^[[:space:]]*/); ind=substr($0,1,RLENGTH)
line=$0; sub(/^[[:space:]]*image:[[:space:]]*/,"",line)
sub(/[[:space:]]*#.*$/,"",line); gsub(/["'"'"']/,"",line)
repo=line; sub(/:[^:\/]*$/,"",repo) # strip the old tag
printf "%simage: %s:%s #LIBREPORTAL|%s|%s\n", ind, repo, tag, key, tag
done=1; next
}
$0 ~ ("^[[:space:]]*" s ":") { seen=1 }
{ print }
' "$compose" > "$tmp" || { rm -f "$tmp"; return 1; }
grep -q "image:.*:${newtag}" "$tmp" || { rm -f "$tmp"; return 1; }
runFileWrite "$compose" < "$tmp"; local rc=$?
rm -f "$tmp"
# Keep the config key in step when the app has one, so the WebUI's Version
# field shows what is actually deployed rather than what it used to be.
local cfgkey="CFG_${up}_VERSION"
if [ -n "${!cfgkey+x}" ] && declare -f updateConfigOption >/dev/null 2>&1; then
updateConfigOption "$cfgkey" "$newtag" >/dev/null 2>&1 || true
fi
return $rc
}
# Current anchor tag for an app, straight from its live compose.
updaterCurrentTag() {
local app="$1"
local compose="${containers_dir%/}/$app/docker-compose.yml"
[ -f "$compose" ] || return 1
updaterTagOf "$(updaterPrimaryImage "$app" "$compose")"
}
# updaterUpgradeApp <app> [target-tag] [--dry-run]
# Walks the ladder. Returns 0 only if every rung verified.
updaterUpgradeApp() {
local app="$1" target="${2:-}" mode="${3:-}"
[ "$target" = "--dry-run" ] && { mode="--dry-run"; target=""; }
local app_dir="${containers_dir%/}/$app"
[ -d "$app_dir" ] || { isError "App '$app' is not installed."; return 1; }
local cur; cur="$(updaterCurrentTag "$app")"
[ -n "$cur" ] || { isError "Could not read $app's current version tag."; return 1; }
local anchor repo
anchor="$(updaterPrimaryImage "$app" "$app_dir/docker-compose.yml")"
repo="$(updaterRepoTag "$anchor")"; repo="${repo%:*}"
# GATE 1 — a stepped upgrade without a real verifier is a guess. The generic
# health check cannot see a half-finished migration, so refusing here is the
# difference between this being a safety feature and a liability.
if ! updaterHasVerifier "$app"; then
isError "$app has no upgrade verifier, so a stepped upgrade cannot be confirmed safe."
isNotice "Add ${app}_upgrade_verify (see cli_updater_verify.sh) before laddering this app."
return 1
fi
# GATE 2 — the ladder must be computable end to end. updaterVersionLadder
# returns non-zero rather than guessing when it cannot reach the target.
local -a rungs=()
if ! mapfile -t rungs < <(updaterVersionLadder "$cur" "$repo" "$target") || (( ${#rungs[@]} == 0 )); then
if [ -n "$target" ]; then
isError "No safe path from $cur to $target could be determined — not attempting it."
isNotice "Upgrade these by hand, one release at a time, if you are sure."
return 1
fi
isSuccessful "$app is already on the newest release line ($cur)."
return 0
fi
isHeader "Upgrade plan for $app"
isNotice "$(updaterLadderSummary "$cur" "${rungs[@]}")"
isNotice "Each step: snapshot → pull → start → verify. A failure stops the ladder and restores that step."
if [ "$mode" = "--dry-run" ]; then
isSuccessful "Dry run — nothing was changed."
return 0
fi
local timeout="${CFG_UPDATER_UPGRADE_TIMEOUT:-900}"
local from="$cur" rung done_n=0
for rung in "${rungs[@]}"; do
isHeader "$app: $from$rung (step $((done_n + 1)) of ${#rungs[@]})"
# 1. Snapshot THIS rung. Fail-closed: no snapshot, no step.
isNotice "Snapshotting $app before $rung"
if ! backupAppStart "$app" >/dev/null 2>&1; then
isError "Pre-step snapshot failed — stopping with $app on $from."
updaterRecordHistory "$app" "upgrade" "$from" "$rung" "aborted-no-snapshot" "" "" "" "manual"
return 1
fi
# 2. Move the version.
if ! updaterSetAnchorVersion "$app" "$rung"; then
isError "Could not set $app to $rung — stopping, nothing changed."
updaterRecordHistory "$app" "upgrade" "$from" "$rung" "aborted-set-version" "" "" "" "manual"
return 1
fi
# 3. Pull + start.
if ! updaterComposePull "$app" || ! dockerComposeUp "$app" >/dev/null 2>&1; then
isError "$app failed to start on $rung — rolling this step back."
_updaterUpgradeRollbackStep "$app" "$from" "$rung"
return 1
fi
# 4. VERIFY. The rung is not done until the app says so itself.
if ! updaterVerifyUpgrade "$app" "$rung" "$timeout"; then
isError "$app did not verify on $rung — rolling this step back."
_updaterUpgradeRollbackStep "$app" "$from" "$rung"
return 1
fi
updaterRecordHistory "$app" "upgrade" "$from" "$rung" "ok" "" "" "" "manual"
isSuccessful "$app is verified on $rung."
from="$rung"; done_n=$((done_n + 1))
done
isSuccessful "$app upgraded through ${done_n} version(s) — now on $from, verified."
webuiUpdaterScan >/dev/null 2>&1 || true
return 0
}
# Undo one failed rung: put the version back, restore the snapshot taken moments
# ago, start it, and record what happened. Best effort by nature — if the
# restore itself fails the user is told plainly rather than reassured.
_updaterUpgradeRollbackStep() {
local app="$1" from="$2" failed="$3"
isNotice "Restoring $app to $from"
updaterSetAnchorVersion "$app" "$from" || isError "Could not put $app's version back to $from — check its compose file."
if restoreAppStart "$app" latest "" >/dev/null 2>&1; then
dockerComposeUp "$app" >/dev/null 2>&1 || true
updaterRecordHistory "$app" "upgrade" "$from" "$failed" "rolled-back" "" "" "" "manual"
isSuccessful "$app restored to $from from its pre-step snapshot."
isNotice "The ladder stopped here. Read $failed's release notes before trying again."
return 0
fi
updaterRecordHistory "$app" "upgrade" "$from" "$failed" "rollback-failed" "" "" "" "manual"
isError "Could not restore $app automatically. Its data snapshot is intact — restore it from the Backups page."
return 1
}