d09b21eec1
Version discovery spoke only hub.docker.com, and every other registry got a shrug: updaterTagExists returned "no" and updaterRegistryTags returned nothing. Five apps live off Hub — stoat and wireguard on ghcr.io, bookstack and speedtest on lscr.io, invidious on quay.io — and for all of them the updater reported "up to date" having never asked. That is the same dishonesty as a scan that never ran: an absence of evidence rendered as a clean bill of health. There was never a barrier, only unwritten code. The standard Distribution API needs one extra step: request, read the WWW-Authenticate challenge, fetch a token from the realm it names, retry. ghcr.io, quay.io and lscr.io all answer anonymously for public images — lscr.io by pointing its realm at ghcr.io, quay.io by not challenging at all. Docker Hub deliberately keeps its own path. hub.docker.com returns tags NEWEST-first, so the 100 it pages are the 100 that matter, and it draws on a different budget from the pull limit — registry-1.docker.io manifest reads count against the anonymous 100/hour that the updater needs for actual pulls, and a ladder probes a tag per rung. Tag LISTING off Hub is a weaker signal and the comment says so: /v2/ tags/list is lexical, not newest-first, and large repos cap the page, so the newest release can legitimately be absent. Probing backfills it, which is why the probe fallback added earlier matters more off Hub than on it. Verified against all four registries: existence probing correct on eight cases including true negatives; stoat climbs v0.15.0 -> v0.15.1 through ghcr.io, and correctly reports nothing above v0.15.1 — the same answer as before, but now because it looked. Hub unregressed: matrix still resolves v1.158.0 -> v1.159.0 and nextcloud still ladders 31 -> 32 33 34. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
258 lines
13 KiB
Bash
258 lines
13 KiB
Bash
#!/bin/bash
|
|
|
|
# Version ladder — the rung list for a stepped upgrade.
|
|
# ---------------------------------------------------------------------------
|
|
# Some apps refuse to skip a version. Nextcloud says so outright ("Updates
|
|
# between multiple major versions and downgrades are unsupported") and simply
|
|
# will not start; databases behave the same way about their data directory.
|
|
# For those, going 31 -> 34 is not one update, it is three, each with its own
|
|
# migration that must finish before the next begins.
|
|
#
|
|
# This file answers only one question: WHICH VERSIONS, IN WHICH ORDER. It does
|
|
# no I/O beyond listing tags and never touches an app — so it is exhaustively
|
|
# testable, which matters because every later safety guarantee is built on it
|
|
# being right. Applying the rungs (snapshot, pull, verify, abort) is the
|
|
# engine's job, not this one's.
|
|
#
|
|
# Rules it enforces:
|
|
# * same SHAPE only 31-fpm-alpine never ladders onto 31-apache
|
|
# * strictly ascending never a downgrade, never a repeat
|
|
# * no gaps every published rung between here and there
|
|
# * stops at the target or at the newest rung if no target is given
|
|
|
|
# Sort key for a tag: each numeric run zero-padded to 6 digits, so a plain
|
|
# lexical sort orders correctly. Without this, "9" sorts after "10" and the
|
|
# ladder would be built in the wrong order — the one bug in here that could
|
|
# actually drive an app backwards through a migration.
|
|
updaterTagSortKey() {
|
|
printf '%s' "$1" | grep -oE '[0-9]+' | awk '{ printf "%06d.", $0 }'
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Generic OCI registry access, for everything that is not Docker Hub.
|
|
#
|
|
# Hub keeps its own path below: hub.docker.com/v2/repositories is a SEPARATE
|
|
# rate-limit pool, whereas registry-1.docker.io manifest reads count against the
|
|
# anonymous PULL limit (100/hour). A ladder probes a tag per rung, so routing
|
|
# Hub through the OCI API would spend the budget the updater needs for pulls.
|
|
#
|
|
# Everything else speaks the standard Distribution API and needs only the token
|
|
# dance: ask, read the WWW-Authenticate challenge, fetch a token from the realm
|
|
# it names, retry. That is all "we cannot probe ghcr.io" ever amounted to —
|
|
# ghcr.io, lscr.io (which points its realm at ghcr.io) and quay.io all answer
|
|
# anonymously for public images. Until this existed those apps were reported as
|
|
# up to date having never been looked at, which is the wrong kind of quiet.
|
|
# ---------------------------------------------------------------------------
|
|
|
|
# Split an image ref into registry + repository. Sets _oci_reg / _oci_repo.
|
|
# A first segment containing a dot or a colon, or "localhost", is a host;
|
|
# anything else is Hub, where a bare name means the library/ namespace.
|
|
_updaterOciSplit() {
|
|
local ref="${1%%:*}"
|
|
ref="${ref#docker.io/}"; ref="${ref#index.docker.io/}"
|
|
case "$ref" in
|
|
localhost/*|*.*/*|*:*/*) _oci_reg="${ref%%/*}"; _oci_repo="${ref#*/}" ;;
|
|
*/*) _oci_reg="registry-1.docker.io"; _oci_repo="$ref" ;;
|
|
*) _oci_reg="registry-1.docker.io"; _oci_repo="library/$ref" ;;
|
|
esac
|
|
}
|
|
|
|
# Bearer token for <registry> <repo>, or nothing when the registry does not
|
|
# challenge (quay.io serves public repos unauthenticated).
|
|
_updaterOciToken() {
|
|
local reg="$1" repo="$2" ch realm service scope
|
|
ch="$(curl -sS -o /dev/null -D - --connect-timeout 5 --max-time 12 \
|
|
"https://${reg}/v2/${repo}/tags/list?n=1" 2>/dev/null \
|
|
| tr -d '\r' | grep -i '^www-authenticate:' | head -1)"
|
|
[ -n "$ch" ] || return 0
|
|
realm="$(printf '%s' "$ch" | grep -o 'realm="[^"]*"' | head -1 | cut -d'"' -f2)"
|
|
service="$(printf '%s' "$ch" | grep -o 'service="[^"]*"' | head -1 | cut -d'"' -f2)"
|
|
scope="$(printf '%s' "$ch" | grep -o 'scope="[^"]*"' | head -1 | cut -d'"' -f2)"
|
|
[ -n "$realm" ] || return 0
|
|
[ -n "$scope" ] || scope="repository:${repo}:pull"
|
|
curl -fsS --connect-timeout 5 --max-time 12 \
|
|
"${realm}?service=${service}&scope=${scope}" 2>/dev/null \
|
|
| grep -oE '"(access_)?token"[[:space:]]*:[[:space:]]*"[^"]+"' | head -1 \
|
|
| sed -E 's/.*"([^"]+)"$/\1/'
|
|
}
|
|
|
|
# Does <tag> exist in <image-ref>? A manifest HEAD, so nothing is downloaded.
|
|
updaterOciTagExists() {
|
|
local ref="$1" tag="$2"
|
|
[ -n "$tag" ] || return 1
|
|
command -v curl >/dev/null 2>&1 || return 1
|
|
_updaterOciSplit "$ref"
|
|
local tok; tok="$(_updaterOciToken "$_oci_reg" "$_oci_repo")"
|
|
local code
|
|
code="$(curl -sS -o /dev/null -w '%{http_code}' -I --connect-timeout 5 --max-time 12 \
|
|
${tok:+-H "Authorization: Bearer $tok"} \
|
|
-H 'Accept: application/vnd.oci.image.index.v1+json' \
|
|
-H 'Accept: application/vnd.docker.distribution.manifest.list.v2+json' \
|
|
-H 'Accept: application/vnd.docker.distribution.manifest.v2+json' \
|
|
"https://${_oci_reg}/v2/${_oci_repo}/manifests/${tag}" 2>/dev/null)"
|
|
[ "$code" = "200" ]
|
|
}
|
|
|
|
# Tags for <image-ref>, one per line. NOTE the ordering differs from Hub's:
|
|
# /v2/.../tags/list is lexical, not newest-first, and large repos cap the page —
|
|
# so a listing can legitimately miss the newest release. That is survivable
|
|
# because newer-version discovery falls back to probing, which has no window.
|
|
updaterOciTagList() {
|
|
command -v curl >/dev/null 2>&1 || return 0
|
|
_updaterOciSplit "$1"
|
|
local tok; tok="$(_updaterOciToken "$_oci_reg" "$_oci_repo")"
|
|
curl -fsSL --connect-timeout 5 --max-time 20 \
|
|
${tok:+-H "Authorization: Bearer $tok"} \
|
|
"https://${_oci_reg}/v2/${_oci_repo}/tags/list?n=1000" 2>/dev/null \
|
|
| sed -E 's/.*"tags"[[:space:]]*:[[:space:]]*\[//; s/\].*//' \
|
|
| tr ',' '\n' | grep -oE '"[^"]*"' | tr -d '"' | grep -v '^$'
|
|
}
|
|
|
|
# Does this exact tag exist? One cheap lookup, and the ONLY reliable way to ask.
|
|
# Listing cannot answer it: Docker Hub pages at 100 and orders by recency, so an
|
|
# older intermediate rung falls off the end — mastodon's v4.3 exists but is
|
|
# absent from the newest-100, and a ladder built from that listing skipped it.
|
|
# Skipping a rung is the exact failure this whole file exists to prevent, so the
|
|
# ladder is built by PROBING each candidate, never by enumerating.
|
|
updaterTagExists() {
|
|
local repo="${1%%:*}" tag="$2"
|
|
repo="${repo#docker.io/}"; repo="${repo#index.docker.io/}" # docker.io/ IS Hub
|
|
# Non-Hub: the standard OCI API rather than a shrug. This is what makes
|
|
# ghcr.io / lscr.io / quay.io apps laddered and version-checked at all.
|
|
case "$repo" in *.*/*|localhost/*) updaterOciTagExists "$1" "$tag"; return $? ;; esac
|
|
case "$repo" in */*) : ;; *) repo="library/$repo" ;; esac
|
|
command -v curl >/dev/null 2>&1 || return 1
|
|
local code
|
|
code="$(curl -fsS -o /dev/null -w '%{http_code}' --connect-timeout 5 --max-time 12 \
|
|
"https://hub.docker.com/v2/repositories/${repo}/tags/${tag}" 2>/dev/null)"
|
|
[ "$code" = "200" ]
|
|
}
|
|
|
|
# Bump the Nth (0-based) numeric component of a tag by one and zero every
|
|
# component after it, so a bump means what a version bump means:
|
|
# "v1.158.2", 1 -> "v1.159.0" "31-fpm-alpine", 0 -> "32-fpm-alpine"
|
|
# Shape is preserved by construction (only digits change), which is what lets
|
|
# callers compare shapes to reject nonsense candidates.
|
|
updaterTagBumpAt() {
|
|
printf '%s' "$1" | awk -v idx="$2" '{
|
|
out=""; n=0; s=$0
|
|
while (match(s, /[0-9]+/)) {
|
|
pre = substr(s, 1, RSTART-1)
|
|
num = substr(s, RSTART, RLENGTH) + 0
|
|
s = substr(s, RSTART+RLENGTH)
|
|
if (n == idx) num = num + 1; else if (n > idx) num = 0
|
|
out = out pre num
|
|
n++
|
|
}
|
|
print out s
|
|
}'
|
|
}
|
|
|
|
# The immediately-next PUBLISHED version above $1 in repo $2, or "" if there is
|
|
# none. THE step primitive: everything else here is built on it being right.
|
|
#
|
|
# It exists because bumping only the last component cannot cross a component
|
|
# boundary. v1.158.0 -> v1.158.1 -> v1.158.2 … never arrives at v1.159.0, so a
|
|
# ladder built that way gave up on the single most common versioning scheme
|
|
# there is, and Synapse — which publishes v1.159.0 and no v1.158.1 at all —
|
|
# could not be climbed one rung.
|
|
#
|
|
# So consider a bump of EVERY component (major, minor, patch), keep only the
|
|
# candidates that actually exist upstream, and take the SMALLEST of those. That
|
|
# is the next release by definition, whether it lands in the patch position or
|
|
# crosses into a new major. Candidates that change the tag's shape are dropped,
|
|
# so 31-fpm-alpine never becomes 31-apache. Costs one lookup per component.
|
|
updaterNextRung() {
|
|
local cur="$1" repo="$2"
|
|
local shape; shape="$(updaterTagShape "$cur")"
|
|
local ncomp; ncomp="$(printf '%s' "$cur" | grep -oE '[0-9]+' | wc -l | tr -d ' ')"
|
|
[ "${ncomp:-0}" -gt 0 ] 2>/dev/null || return 0
|
|
local best="" i cand
|
|
for ((i=0; i<ncomp; i++)); do
|
|
cand="$(updaterTagBumpAt "$cur" "$i")"
|
|
[ "$(updaterTagShape "$cand")" = "$shape" ] || continue
|
|
updaterTagExists "$repo" "$cand" || continue
|
|
if [ -z "$best" ] || updaterTagGreater "$best" "$cand"; then best="$cand"; fi
|
|
done
|
|
printf '%s' "$best"
|
|
}
|
|
|
|
# Bump the LAST numeric component of a tag by one: v4.2 -> v4.3, 31-fpm-alpine
|
|
# -> 32-fpm-alpine, v0.16 -> v0.17.
|
|
updaterTagIncrement() {
|
|
local tag="$1"
|
|
# Greedy leading group takes everything up to the LAST digit run, so the
|
|
# split is prefix / number / suffix: "31-fpm-alpine" -> ""/"31"/"-fpm-alpine",
|
|
# "v4.2" -> "v4."/"2"/"". 10# keeps "08" decimal rather than octal.
|
|
if [[ "$tag" =~ ^(.*[^0-9])?([0-9]+)([^0-9]*)$ ]]; then
|
|
printf '%s%d%s' "${BASH_REMATCH[1]}" "$((10#${BASH_REMATCH[2]} + 1))" "${BASH_REMATCH[3]}"
|
|
else
|
|
printf '%s' "$tag"
|
|
fi
|
|
}
|
|
|
|
# updaterVersionLadder <current-tag> <repo> [target-tag]
|
|
# Prints the rungs to climb, one per line, ascending, EXCLUDING the current
|
|
# version and INCLUDING the target.
|
|
#
|
|
# Built by probing consecutive increments, so a rung missing from any listing
|
|
# can never be missed. A version upstream genuinely skipped (no v4.3 at all) is
|
|
# stepped over, but ONLY because the probe said it does not exist.
|
|
#
|
|
# FAILS LOUDLY (returns 1, prints nothing) if it cannot construct a continuous
|
|
# path to the target — e.g. the target is across a boundary simple incrementing
|
|
# cannot reach. Refusing to guess is the point: a wrong ladder means a skipped
|
|
# migration, and "I cannot compute this safely, do it by hand" is the only
|
|
# honest answer in that case.
|
|
updaterVersionLadder() {
|
|
local cur="$1" repo="$2" target="${3:-}"
|
|
[ -n "$cur" ] && [ -n "$repo" ] || return 0
|
|
local shape; shape="$(updaterTagShape "$cur")"
|
|
|
|
# A rolling tag has no ladder — it moves on its own. Guarded here as well as
|
|
# at the call site: this function must never be why an app moves.
|
|
[ "$(updaterClassifyTag "$cur")" = "rolling" ] && return 0
|
|
|
|
# Discover the target (newest same-shape tag) when not told one. Listing is
|
|
# fine for THIS — being one rung short is harmless, whereas a gap is not.
|
|
[ -n "$target" ] || target="$(updaterNewerVersionTag "$cur" "$repo")"
|
|
[ -n "$target" ] || return 0 # already current
|
|
[ "$(updaterTagShape "$target")" = "$shape" ] || return 0
|
|
updaterTagGreater "$target" "$cur" || return 0 # never downgrade
|
|
|
|
local -a rungs=()
|
|
local probe="$cur" i next
|
|
for ((i=0; i<64; i++)); do # bounded: no runaway
|
|
# Step to the next version that EXISTS, rather than incrementing blindly
|
|
# and testing. Same guarantee as before — every rung is probed, so a
|
|
# release missing from any listing can still never be skipped — but it
|
|
# can now cross a component boundary, which blind incrementing could
|
|
# not: v1.158.0 -> v1.159.0 was unreachable and the whole ladder failed
|
|
# closed on it.
|
|
next="$(updaterNextRung "$probe" "$repo")"
|
|
[ -n "$next" ] || break # nothing further published
|
|
updaterTagGreater "$next" "$target" && break # overshot
|
|
rungs+=("$next")
|
|
probe="$next"
|
|
[ "$probe" = "$target" ] && break
|
|
done
|
|
|
|
# The last rung MUST be the target. Anything else means the path is
|
|
# incomplete and applying it would land the app somewhere unintended.
|
|
if (( ${#rungs[@]} == 0 )) || [ "${rungs[-1]}" != "$target" ]; then
|
|
return 1
|
|
fi
|
|
printf '%s\n' "${rungs[@]}"
|
|
}
|
|
|
|
# Human summary of a ladder, for the confirmation the user sees before any of
|
|
# it runs: "31-fpm-alpine → 32-fpm-alpine → 33-fpm-alpine → 34-fpm-alpine (3 steps)".
|
|
updaterLadderSummary() {
|
|
local cur="$1"; shift
|
|
local -a rungs=("$@")
|
|
(( ${#rungs[@]} == 0 )) && { printf 'already current (%s)' "$cur"; return 0; }
|
|
local out="$cur" r
|
|
for r in "${rungs[@]}"; do out+=" → $r"; done
|
|
printf '%s (%d step%s)' "$out" "${#rungs[@]}" "$( (( ${#rungs[@]} == 1 )) || echo s )"
|
|
}
|