Polish pass for the migration system. Two concrete additions; the live-mirror
and full drift-verify ideas from the original plan are intentionally
deferred — both need real-world test data to land correctly, and the kernel
already exposes everything they'd need.
Per-app migrate hooks (scripts/migrate/migrate_hooks.sh):
Apps can declare two optional functions in their tools.sh (already
auto-sourced per [[libreportal-modular-app-tools]]):
<app>_migrate_pre() — runs before stop+wipe
<app>_migrate_post() — runs after restart, before the user sees it
Each receives:
$1 = source identifier (peer name or backup-tag hostname)
$2 = transport ("restic" | "direct-ssh")
migrateRunHook() is now called from both migration apply paths:
- migrate_apply.sh (restic-mediated, shared backup channel)
- peer_pull.sh (direct-SSH, peer-shell stream)
Use cases: rotate federation keys after a Mastodon move, regenerate
OIDC client secrets, drop SaaS-style locks, fix hostname-baked configs
the URL-rewrite layer doesn't cover.
Hooks are optional — apps without them inherit the standard flow.
Failed hooks emit a non-fatal notice (the rest of the migrate still
reaches 'done') so a single bad hook can't strand an otherwise-working
app in stopped state.
Peer friendly-name overlay (Migrate tab):
Was deferred from Phase 2 because it required Phase 3's UI to feel
cohesive. BackupPage.refreshAll() now also fetches peers.json and builds
a hostname → peer-name lookup. renderMigrate() shows
'homelab (host: homelab.lan)'
for any backup-channel peer that matches the source host, and falls back
to the bare hostname when no peer is defined. Same data, friendlier UI.
Skipped (genuinely deferred, not just out of time):
- Live mirror / warm-standby (continuous one-way sync). Needs a scheduler
+ drift-state to track. Right place for it is a separate feature on top
of the existing kernel rather than bolted onto migrate.
- Drift-verify ("what would change if I migrated?"). Cheap to write but
needs a real cross-host pair to validate against — adding it untested
would just be theatre.
Signed-off-by: librelad <librelad@digitalangels.vip>
131 lines
5.4 KiB
Bash
131 lines
5.4 KiB
Bash
#!/bin/bash
|
|
|
|
# Pull an app from a direct-ssh-direct peer onto this host. End-to-end:
|
|
#
|
|
# 1. Resolve peer + verify reachable (peerPing).
|
|
# 2. Take the existing-app safety backup (migratePreBackupDestination) so
|
|
# a bad pull is rollback-able via the normal restore flow.
|
|
# 3. Stop + wipe the destination's app folder (same as restoreAppStart).
|
|
# 4. Stream `peer-shell stream-app <slug>` over SSH and untar into
|
|
# containers_dir/.
|
|
# 5. Reuse the install-time tag pipeline to repoint host-bound values:
|
|
# - dockerComposeUpdateAndStartApp <app> install re-emits compose
|
|
# - migrateApplyUrlRewrite rewrites URLs
|
|
# 6. Start the container, run app-specific post-restore hooks.
|
|
#
|
|
# That gives us the same idempotent install path the restic-mediated
|
|
# migrateApplyApp uses, just with tar-over-SSH as the data source instead
|
|
# of a restic snapshot.
|
|
|
|
# peerPullApp <peer-name> <app-slug> [--no-pre-backup] [--keep-urls]
|
|
peerPullApp()
|
|
{
|
|
local peer_name="$1"; shift
|
|
local app="$1"; shift
|
|
|
|
# Reuse migrate_apply's opt parser to keep flag semantics identical.
|
|
_migrateParseOpts "$@"
|
|
|
|
if [[ -z "$peer_name" || -z "$app" ]]; then
|
|
isError "peerPullApp: peer_name and app required"
|
|
return 1
|
|
fi
|
|
|
|
local row; row=$(peerGet "$peer_name")
|
|
if [[ -z "$row" || "$row" == "null" ]]; then
|
|
isError "No peer named '$peer_name'"
|
|
return 1
|
|
fi
|
|
local kind
|
|
kind=$(printf '%s' "$row" | grep -o '"kind":"[^"]*"' | head -1 | cut -d'"' -f4)
|
|
if [[ "$kind" != "direct-ssh-direct" ]]; then
|
|
isError "Peer '$peer_name' is kind=$kind — peerPullApp needs direct-ssh-direct"
|
|
return 1
|
|
fi
|
|
|
|
local started_at; started_at=$(date +%s)
|
|
isHeader "Pull $app from peer=$peer_name → this host"
|
|
migrateEmit phase=start status=running app="$app" peer="$peer_name" transport=direct-ssh
|
|
|
|
# ---- 1. Reachability --------------------------------------------------
|
|
migrateEmit phase=ping status=running peer="$peer_name"
|
|
local ping_status; ping_status=$(peerPing "$peer_name")
|
|
if [[ "$ping_status" != "ok" ]]; then
|
|
isError "Peer '$peer_name' unreachable: $ping_status"
|
|
migrateEmit phase=ping status=failed detail="$ping_status"
|
|
return 1
|
|
fi
|
|
migrateEmit phase=ping status=complete
|
|
|
|
# ---- 2. Pre-backup of destination (default ON) ------------------------
|
|
if (( MIGRATE_OPT_PRE_BACKUP )); then
|
|
migratePreBackupDestination "$app"
|
|
else
|
|
migrateEmit phase=pre-backup status=skipped reason=user-opt-out app="$app"
|
|
fi
|
|
|
|
# Per-app pre-migrate hook (optional) — declared in the app's tools.sh.
|
|
migrateRunHook "$app" pre "$peer_name" "direct-ssh"
|
|
|
|
# ---- 3. Stop + wipe ----------------------------------------------------
|
|
migrateEmit phase=stop status=running app="$app"
|
|
if declare -f dockerComposeDown >/dev/null 2>&1 && [[ -d "$containers_dir$app" ]]; then
|
|
dockerComposeDown "$app" >/dev/null 2>&1 || true
|
|
fi
|
|
if [[ -d "$containers_dir$app" ]]; then
|
|
runFileOp rm -rf "${containers_dir:?}$app"
|
|
fi
|
|
migrateEmit phase=stop status=complete app="$app"
|
|
|
|
# ---- 4. Stream tar over SSH and untar ---------------------------------
|
|
migrateEmit phase=transfer status=running app="$app"
|
|
# The pipe gives us streaming throughput without staging the whole tarball
|
|
# to disk. set -o pipefail catches ssh failures so we don't run the rest
|
|
# of the flow on a partial extract.
|
|
(
|
|
set -o pipefail
|
|
peerExec "$peer_name" "stream-app $app" | tar -C "$containers_dir" -xf -
|
|
)
|
|
local transfer_rc=$?
|
|
if (( transfer_rc != 0 )); then
|
|
isError "Transfer of $app from $peer_name failed (rc=$transfer_rc)"
|
|
migrateEmit phase=transfer status=failed rc="$transfer_rc"
|
|
return 1
|
|
fi
|
|
if [[ ! -d "$containers_dir$app" ]]; then
|
|
isError "Transfer reported success but $containers_dir$app missing"
|
|
migrateEmit phase=transfer status=failed reason=missing-output
|
|
return 1
|
|
fi
|
|
runFileOp chown -R "${docker_install_user:-$(whoami)}":"${docker_install_user:-$(whoami)}" "$containers_dir$app" 2>/dev/null || true
|
|
migrateEmit phase=transfer status=complete app="$app"
|
|
|
|
# ---- 5. URL rewrite (default ON) + re-deploy compose -------------------
|
|
if ! (( MIGRATE_OPT_KEEP_URLS )); then
|
|
migrateApplyUrlRewrite "$app"
|
|
else
|
|
migrateEmit phase=url-rewrite status=skipped reason=user-opt-out app="$app"
|
|
fi
|
|
if declare -f dockerComposeUpdateAndStartApp >/dev/null 2>&1; then
|
|
dockerComposeUpdateAndStartApp "$app" install >/dev/null 2>&1 || true
|
|
fi
|
|
|
|
# ---- 6. Start + post-restore hooks ------------------------------------
|
|
migrateEmit phase=start-app status=running app="$app"
|
|
if declare -f dockerComposeUp >/dev/null 2>&1; then
|
|
dockerComposeUp "$app" >/dev/null 2>&1 || true
|
|
fi
|
|
if declare -f restoreAppRunHook >/dev/null 2>&1; then
|
|
restoreAppRunHook "$app" post || true
|
|
fi
|
|
migrateEmit phase=start-app status=complete app="$app"
|
|
|
|
# Per-app post-migrate hook (optional) — last thing before declaring done.
|
|
migrateRunHook "$app" post "$peer_name" "direct-ssh"
|
|
|
|
local finished_at; finished_at=$(date +%s)
|
|
local duration=$((finished_at - started_at))
|
|
isSuccessful "Pulled $app from $peer_name in ${duration}s"
|
|
migrateEmit phase=done status=complete app="$app" duration_seconds="$duration"
|
|
}
|