598f74c26b
Stepping 31 -> 32 -> 33 is arithmetic. Knowing 32 FINISHED before
touching 33 is the whole safety story, and it is invisible from outside
the app: Nextcloud runs its migration on boot and sits in maintenance
mode — or fails halfway — while Docker reports the container perfectly
healthy. Advance a rung there and a migration has been skipped on live
data.
Contract: <app>_upgrade_verify <app> <expected-tag> <deadline> -> 0
Returns 0 ONLY on positive confirmation that the app serves at the
expected version with nothing outstanding. Unhealthy, indeterminate and
timed-out all return non-zero — uncertainty is a failure, not a maybe,
because the alternative gambles with data.
nextcloud `occ status`: installed, NOT in maintenance, no pending DB
upgrade, and the running major matches the tag. Maintenance
mid-migration is expected and simply keeps waiting.
mastodon /health serving, ZERO "down" rows in db:migrate:status, and
the version from /api/v1/instance matching. /health alone is
insufficient — Puma answers before migrations finish.
stalwart /healthz/ready (per its documented probes), required to hold
stable rather than flash once. Weaker by design: the probes
confirm serving but report no version, and the file says so
rather than implying more.
updaterVerifyGeneric (running + healthy + no restart during a settle
window) is the fallback for everything else, and is explicitly NOT
sufficient to justify climbing a rung — the engine will refuse to ladder
an app with no declared verifier.
9 tests drive the dangerous states directly: maintenance mode, pending DB
upgrade, and a wrong major all correctly REFUSE to verify; clean states
pass. Those three negatives are the ones that would have corrupted data.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
65 lines
3.5 KiB
Bash
65 lines
3.5 KiB
Bash
#!/bin/bash
|
|
|
|
# Nextcloud upgrade verifier.
|
|
# ---------------------------------------------------------------------------
|
|
# Nextcloud is the reason stepped upgrades exist here: it refuses to cross more
|
|
# than one major ("Updates between multiple major versions and downgrades are
|
|
# unsupported") and simply will not start. So the ladder must be certain each
|
|
# major finished before starting the next.
|
|
#
|
|
# `occ status` is the authoritative answer and reports exactly the three facts
|
|
# that matter — whether the instance is installed, whether it is still in
|
|
# maintenance mode, and whether a database upgrade is outstanding — plus the
|
|
# version actually running. Container health tells you none of that: during the
|
|
# post-upgrade migration the container is up and healthy while the instance is
|
|
# in maintenance mode with the schema half-converted.
|
|
#
|
|
# Runs through the app's own occ wrapper idiom (docker exec -u www-data), so it
|
|
# inherits whatever the rest of the Nextcloud tooling already relies on.
|
|
|
|
# nextcloud_upgrade_verify <app> <expected-tag> <deadline-epoch>
|
|
# 0 only when the instance is installed, out of maintenance, has no pending DB
|
|
# upgrade, and reports the major version the tag asked for.
|
|
nextcloud_upgrade_verify() {
|
|
local app="$1" expected="$2" deadline="$3"
|
|
# "34-fpm-alpine" -> 34. The tag's leading number IS the major; occ reports
|
|
# a full version (34.0.2.1) and only the major is comparable.
|
|
local want_major; want_major="$(printf '%s' "$expected" | grep -oE '^[0-9]+')"
|
|
|
|
local last=""
|
|
while [ "$(date +%s)" -lt "$deadline" ]; do
|
|
local out
|
|
out="$(runFileOp docker exec -u www-data nextcloud-service php occ status --output=json 2>/dev/null | tr -d '\r')"
|
|
|
|
if [ -n "$out" ] && printf '%s' "$out" | grep -q '"installed"'; then
|
|
local installed maint needs_db ver major
|
|
if command -v jq >/dev/null 2>&1; then
|
|
installed="$(printf '%s' "$out" | jq -r '.installed // false' 2>/dev/null)"
|
|
maint="$(printf '%s' "$out" | jq -r '.maintenance // false' 2>/dev/null)"
|
|
needs_db="$(printf '%s' "$out" | jq -r '.needsDbUpgrade // false' 2>/dev/null)"
|
|
ver="$(printf '%s' "$out" | jq -r '.versionstring // ""' 2>/dev/null)"
|
|
else
|
|
installed="$(printf '%s' "$out" | grep -o '"installed":[^,}]*' | cut -d: -f2 | tr -d ' "')"
|
|
maint="$(printf '%s' "$out" | grep -o '"maintenance":[^,}]*' | cut -d: -f2 | tr -d ' "')"
|
|
needs_db="$(printf '%s' "$out" | grep -o '"needsDbUpgrade":[^,}]*' | cut -d: -f2 | tr -d ' "')"
|
|
ver="$(printf '%s' "$out" | grep -o '"versionstring":"[^"]*"' | cut -d'"' -f4)"
|
|
fi
|
|
major="$(printf '%s' "$ver" | cut -d. -f1)"
|
|
last="installed=$installed maintenance=$maint needsDbUpgrade=$needs_db version=$ver"
|
|
|
|
if [ "$installed" = "true" ] && [ "$maint" = "false" ] && [ "$needs_db" = "false" ] \
|
|
&& [ -n "$want_major" ] && [ "$major" = "$want_major" ]; then
|
|
isSuccessful "Nextcloud reports $ver, out of maintenance, no pending DB upgrade."
|
|
return 0
|
|
fi
|
|
# Maintenance mode mid-migration is EXPECTED and not a failure —
|
|
# keep waiting. Only the deadline ends this.
|
|
isNotice "Nextcloud not ready yet: $last"
|
|
fi
|
|
sleep 10
|
|
done
|
|
|
|
isError "Nextcloud did not reach a verified state for $expected before the deadline.${last:+ Last status: $last}"
|
|
return 1
|
|
}
|