librelad 864059ab83 paths: recover the roots from the baked unit when nothing exported them
The three roots reach running code three ways — the CLI wrapper exports them,
the task-processor unit carries them as Environment=, and anything those start
inherits them. An @reboot crontab entry is started by none of the three: it runs
a script by absolute path, so paths.sh fell through to

    : "${LP_CONTAINERS_DIR:=/libreportal-containers}"

That entry is crontab_boot_app_reconcile.sh, which brings every installed app up
at boot. On a relocated install it therefore reconciled against the DEFAULT
root, and that does not fail — docker creates the bind-mount directories it does
not find, so every app comes back empty while the real data sits untouched on
the other disk. Nothing logs an error; the only symptom is opening an app and
finding it blank (storage-locations §10.1 calls this the top data-integrity
risk).

Verified on a case-2 install (apps on /mnt/lptest2): a bare environment resolved
containers_dir to /libreportal-containers/ where the real root was
/mnt/lptest2/libreportal-containers.

Recover them from the systemd unit, which is the authoritative baked record —
init.sh already reads it back the same way, libreportal-relocate rewrites it,
and it is root-owned, so this is not the manager reading a config it can edit.
An explicit environment still wins; with no unit the defaults are unchanged.

scripts/dev/lp-paths-roots-test pins all four cases; verified it fails when the
recovery is removed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 06:31:02 +01:00

564 lines
22 KiB
Bash

#!/bin/bash
#
# LibrePortal path roots — single source of truth for the (relocatable) layout.
#
# Three independently-placeable roots, each owned by exactly one principal:
# LP_SYSTEM_DIR control plane — manager (libreportal) owned, 750
# configs/ logs/ install/ database.db ssl/ ssh/ migrate/ restore/
# LP_CONTAINERS_DIR live app data — container user (dockerinstall) owned (rootless)
# LP_BACKUPS_DIR restic/kopia repos — container user owned (separable / own mount)
#
# The roots come from the environment when set (the install bakes them into the
# task-processor systemd unit, and the CLI/app inherit them from init.sh), else
# they default to /libreportal-*. A custom location is chosen at INSTALL time and
# baked by root — never read at runtime from a manager-writable config.
#
# SECURITY: the root-owned helpers under /usr/local/lib/libreportal/ do NOT source
# this file. They get the paths baked in at install (sed placeholders), so the
# manager cannot redirect a root `chown`/`chmod` by editing config. This file is
# only for the manager-run code (app, CLI, task processor), which runs without
# extra privilege.
#
# Mirror copy: init.sh derives the same vars inline (it is self-contained for the
# bare /root/init.sh reinstall case, where scripts/ isn't alongside). Keep the two
# derivations in sync.
# --- Resolve the three roots ------------------------------------------------
# Nothing in the environment? Recover them from the record root baked at install
# before falling back to the defaults below.
#
# The roots reach running code three ways: the CLI wrapper exports them, the
# task-processor unit carries them as Environment=, and everything started by
# those inherits them. An @reboot crontab entry is started by none of the three
# — it invokes a script by absolute path — so it fell through to the /libreportal-*
# defaults. On a relocated install that silently pointed the BOOT APP RECONCILE
# at the wrong disk, and that job brings every installed app up: docker creates
# the bind-mount directories it does not find, so the apps come back EMPTY while
# the real data sits untouched on the other disk (storage-locations §10.1).
#
# The unit is the authoritative record — init.sh reads it back the same way, and
# libreportal-relocate rewrites it — and it is root-owned, so this is not the
# manager reading a config it can edit.
if [[ -z "${LP_SYSTEM_DIR:-}" ]]; then
_lp_unit="${LP_UNIT_FILE:-/etc/systemd/system/libreportal.service}"
if [[ -r "$_lp_unit" ]]; then
_lp_v=$(grep -m1 -oE '^Environment=LP_SYSTEM_DIR=\S+' "$_lp_unit" 2>/dev/null); _lp_v="${_lp_v#*LP_SYSTEM_DIR=}"
[[ -n "$_lp_v" ]] && LP_SYSTEM_DIR="$_lp_v"
_lp_v=$(grep -m1 -oE '^Environment=LP_CONTAINERS_DIR=\S+' "$_lp_unit" 2>/dev/null); _lp_v="${_lp_v#*LP_CONTAINERS_DIR=}"
[[ -n "$_lp_v" && -z "${LP_CONTAINERS_DIR:-}" ]] && LP_CONTAINERS_DIR="$_lp_v"
_lp_v=$(grep -m1 -oE '^Environment=LP_BACKUPS_DIR=\S+' "$_lp_unit" 2>/dev/null); _lp_v="${_lp_v#*LP_BACKUPS_DIR=}"
[[ -n "$_lp_v" && -z "${LP_BACKUPS_DIR:-}" ]] && LP_BACKUPS_DIR="$_lp_v"
unset _lp_v
fi
unset _lp_unit
fi
# Transitional compat: an EXISTING install (the legacy single /docker tree,
# identified by its config marker) keeps using /docker until a deliberate
# reinstall to the split layout — so deploying new code never strands a running
# box. Fresh installs (no marker) get the /libreportal-* split.
if [[ -z "${LP_SYSTEM_DIR:-}" ]]; then
if [[ ! -e /libreportal-system && -f /docker/configs/general/general_docker_install ]]; then
LP_SYSTEM_DIR=/docker
: "${LP_CONTAINERS_DIR:=/docker/containers}"
: "${LP_BACKUPS_DIR:=/docker/backups}"
else
LP_SYSTEM_DIR=/libreportal-system
fi
fi
: "${LP_CONTAINERS_DIR:=/libreportal-containers}"
: "${LP_BACKUPS_DIR:=/libreportal-backups}"
# --- Derived: system tree (manager-owned). docker_dir is the legacy name. ---
docker_dir="$LP_SYSTEM_DIR"
system_dir="$LP_SYSTEM_DIR"
configs_dir="$LP_SYSTEM_DIR/configs/"
logs_dir="$LP_SYSTEM_DIR/logs/"
ssl_dir="$LP_SYSTEM_DIR/ssl/"
ssh_dir="$LP_SYSTEM_DIR/ssh/"
wireguard_dir="$LP_SYSTEM_DIR/wireguard/"
migrate_dir="$LP_SYSTEM_DIR/migrate"
restore_dir="$LP_SYSTEM_DIR/restore"
script_dir="$LP_SYSTEM_DIR/install"
install_configs_dir="$script_dir/configs/"
install_containers_dir="$script_dir/containers/"
install_scripts_dir="$script_dir/scripts/"
# --- Derived: data tree (container-user-owned) — the root IS the dir ---------
containers_dir="$LP_CONTAINERS_DIR/"
# --- Derived: backups tree (container-user-owned; own mount-able) -----------
backup_dir="$LP_BACKUPS_DIR"
# --- Control-plane manager user (configurable; baked into helpers at install) -
# The systemd unit + CLI wrapper export LP_MANAGER_USER; else default libreportal.
sudo_user_name="${LP_MANAGER_USER:-libreportal}"
# =============================================================================
# Storage locations — the containers root is a LIST, not a single path.
# =============================================================================
# See docs/roadmap/storage-locations.md. Three ideas, in dependency order:
#
# 1. The set of roots that may hold app data comes from a ROOT-OWNED registry
# ($lp_storage_registry). The manager can read it and never write it — the
# same trust boundary that makes the baked __CONTAINERS_DIR__ safe in the
# /usr/local/lib/libreportal/ helpers. Adding a root goes through
# `libreportal-storage add`, which validates and is the only writer.
#
# 2. An app's directory is DISCOVERED, not declared: whichever root actually
# holds <slug>/<slug>.config wins. CFG_<SLUG>_STORAGE records intent and is
# consulted only when the app isn't on disk yet. Disagreement resolves to
# the disk, which is what makes a hand-move or a half-finished migration
# self-heal instead of corrupting.
#
# 3. A root is AVAILABLE only while its marker file is present. The marker
# lives on the drive, so an unmounted disk has no marker and the root is
# unavailable — appDir fails rather than handing back a path that docker
# would happily populate on the bare mountpoint.
#
# Everything here must stay cheap: pathIsContainerData is on the hot path of
# every file helper, so the roots list and the slug->dir map are both memoised
# in globals and loaded without a subshell.
# Root-owned registry: "<id>\t<path>\t<dev>\t<fs_uuid>" per line. Absent on an
# install that predates storage locations — then the primary root is the only
# root and every function here behaves exactly as the old single-root code did.
lp_storage_registry="${LP_STORAGE_REGISTRY:-/usr/local/lib/libreportal/storage.roots}"
# Written root-owned into a location's top level at registration. Presence is
# the mount test (see idea 3 above).
lp_storage_marker=".libreportal-storage"
# Returned by appDir when a location exists but its drive is not mounted. A path
# that cannot exist, so a caller that ignores the non-zero status still fails
# loudly on something harmless instead of writing to a bare mountpoint.
lp_storage_unavailable="/nonexistent-libreportal-unavailable"
# Memo state. Declared here so the associative array exists before first use and
# so a re-source of paths.sh starts from a clean map rather than a stale one.
declare -A LP_APP_DIR_CACHE=()
LP_APP_DIR_SCANNED=0
declare -a LP_STORAGE_ROOTS=()
LP_STORAGE_ROOTS_LOADED=0
# The install-time containers root. Always present, never removable, and the
# home of anything that must not move (the WebUI's own tree).
primaryRoot()
{
printf '%s' "${LP_CONTAINERS_DIR%/}"
}
# LibrePortal's own container dir. Structurally pinned to the primary root —
# too much of the control plane reaches into it by literal path for it to move.
webuiDir()
{
printf '%s' "${LP_CONTAINERS_DIR%/}/libreportal"
}
# Populate LP_STORAGE_ROOTS (primary first). Sets the global directly — callers
# must NOT wrap this in a subshell or the memo is lost.
_lpStorageRootsLoad()
{
[[ "${LP_STORAGE_ROOTS_LOADED:-0}" == "1" ]] && return 0
LP_STORAGE_ROOTS=("${LP_CONTAINERS_DIR%/}")
local _id _path _rest
if [[ -r "$lp_storage_registry" ]]; then
while IFS=$'\t' read -r _id _path _rest || [[ -n "$_id" ]]; do
[[ -z "$_path" || "$_id" == \#* ]] && continue
_path="${_path%/}"
[[ "$_path" == "${LP_CONTAINERS_DIR%/}" ]] && continue
LP_STORAGE_ROOTS+=("$_path")
done < "$lp_storage_registry"
fi
LP_STORAGE_ROOTS_LOADED=1
return 0
}
# Drop every memo. Call after registering/removing a location or moving an app —
# otherwise a long-lived process (the task processor) keeps serving stale paths.
storageCacheReset()
{
LP_STORAGE_ROOTS=()
LP_STORAGE_ROOTS_LOADED=0
LP_APP_DIR_CACHE=()
LP_APP_DIR_SCANNED=0
return 0
}
# Every registered container-data root, primary first, one per line, no trailing
# slash. Includes roots whose drive is currently absent — callers that care ask
# storageRootAvailable.
storageRoots()
{
_lpStorageRootsLoad
local r
for r in "${LP_STORAGE_ROOTS[@]}"; do
printf '%s\n' "$r"
done
}
# Is this root usable right now? The primary root is definitionally available
# (if it is gone, so is the install). A registered root needs its marker, which
# is only readable when the drive is actually mounted.
storageRootAvailable()
{
local root="${1%/}"
[[ -z "$root" ]] && return 1
[[ "$root" == "${LP_CONTAINERS_DIR%/}" ]] && return 0
[[ -e "$root/$lp_storage_marker" ]]
}
# Is $1 inside ANY container-data root? Replaces the
# `[[ "$p" == "$containers_dir"* ]]` idiom that decides whether a file op runs
# as the container user or the manager. A miss here is a wrong-owner file that
# fails much later, so it deliberately matches the root itself as well as
# anything beneath it, with or without a trailing slash.
pathIsContainerData()
{
local p="$1"
[[ -z "$p" ]] && return 1
_lpStorageRootsLoad
local root
for root in "${LP_STORAGE_ROOTS[@]}"; do
[[ -z "$root" ]] && continue
[[ "$p" == "$root" || "$p" == "$root/"* ]] && return 0
done
return 1
}
# Resolve a location NAME to its root path. Names live in the manager-owned
# per-location config (CFG_STORAGE_LOC_<id>_NAME); paths live in the root-owned
# registry. "default" (and an unset value) is the primary root.
storageLocationPath()
{
local want="$1"
# "primary" names the install-time root explicitly. "default" is accepted
# here for callers that resolve a concrete value, but at APP level it means
# "inherit the global default" — see _appDirIntended.
[[ -z "$want" || "$want" == "default" || "$want" == "primary" ]] && { primaryRoot; return 0; }
local _id _path _rest name_var
if [[ -r "$lp_storage_registry" ]]; then
while IFS=$'\t' read -r _id _path _rest || [[ -n "$_id" ]]; do
[[ -z "$_path" || "$_id" == \#* ]] && continue
# Match on the id itself, or on the friendly name from its config.
name_var="CFG_STORAGE_LOC_${_id}_NAME"
if [[ "$_id" == "$want" || "${!name_var:-}" == "$want" ]]; then
printf '%s' "${_path%/}"
return 0
fi
done < "$lp_storage_registry"
fi
return 1
}
# Reverse: root path -> location name (falls back to the id, then "default").
storageLocationName()
{
local want="${1%/}"
[[ -z "$want" || "$want" == "${LP_CONTAINERS_DIR%/}" ]] && { printf 'default'; return 0; }
local _id _path _rest name_var
if [[ -r "$lp_storage_registry" ]]; then
while IFS=$'\t' read -r _id _path _rest || [[ -n "$_id" ]]; do
[[ -z "$_path" || "$_id" == \#* ]] && continue
if [[ "${_path%/}" == "$want" ]]; then
name_var="CFG_STORAGE_LOC_${_id}_NAME"
printf '%s' "${!name_var:-$_id}"
return 0
fi
done < "$lp_storage_registry"
fi
return 1
}
# --- app -> location index ---------------------------------------------------
# A manager-owned CACHE of which root each app was last seen on. Discovery still
# wins whenever the disk is present; this exists for the one case discovery
# cannot answer:
#
# an app on a drive that is not mounted is invisible to the scan, and without
# the index appDir would fall back to the primary root — handing back a path
# docker would populate on the wrong disk, booting the app empty. That is the
# exact failure the whole availability design exists to prevent, so "not found
# by the scan" must not silently mean "belongs on the primary root".
#
# Location: the manager-owned SYSTEM tree, deliberately NOT under configs/.
#
# The requirements are only "manager-owned" and "not on a removable disk" (it has
# to be readable precisely when that disk is gone). configs/ satisfies both and
# was the first instinct — and it was wrong, because that tree carries a third
# property this file violates: sourceScanFiles SOURCES what it finds there, and
# sourcing means executing.
#
# This file is a TSV of "<slug><TAB><root>". Bash reads such a line as a command
# and its argument. Harmless while no slug matched a real executable — and a fork
# bomb the moment the row was for the app named `libreportal`, because that IS
# the CLI on PATH: sourcing ran `libreportal /libreportal-containers`, which
# re-entered the scan, which sourced the file again, one process pair per level
# until the host died of OOM. It took out the desktop session with it.
#
# scan_files.sh now also requires a .category marker before sourcing anything in
# a configs/ subdirectory, so the hole is closed from both ends. But a
# machine-written data file has no business in the one tree whose contract is
# "everything here is executed", so it lives here instead.
storageIndexFile()
{
printf '%s' "${system_dir%/}/storage/app_locations"
}
# One-shot migration off the old configs/ path. Cheap (a -f test) and it runs
# before any read, so an install that predates the move keeps its index instead
# of silently forgetting where every app lives.
_storageIndexMigrate()
{
local legacy="${configs_dir%/}/storage/app_locations"
local current; current=$(storageIndexFile)
[[ -f "$legacy" ]] || return 0
local op=""
declare -F runInstallOp >/dev/null 2>&1 && op="runInstallOp"
if [[ ! -f "$current" ]]; then
$op mkdir -p "${current%/*}" 2>/dev/null
$op cp "$legacy" "$current" 2>/dev/null
fi
$op rm -f "$legacy" 2>/dev/null
$op rmdir "${legacy%/*}" 2>/dev/null
return 0
}
storageIndexGet()
{
local slug="$1" f s r
_storageIndexMigrate
f=$(storageIndexFile)
[[ -r "$f" ]] || return 1
while IFS=$'\t' read -r s r || [[ -n "$s" ]]; do
[[ "$s" == "$slug" ]] || continue
[[ -z "$r" ]] && return 1
printf '%s' "${r%/}"
return 0
done < "$f"
return 1
}
# Record (or clear, with an empty root) an app's location. Idempotent.
storageIndexSet()
{
local slug="$1" root="${2%/}" f tmp s r
[[ -z "$slug" ]] && return 1
_storageIndexMigrate
f=$(storageIndexFile)
local cur=""
cur=$(storageIndexGet "$slug" 2>/dev/null) || cur=""
[[ "$cur" == "$root" ]] && return 0
local op=""
declare -F runInstallOp >/dev/null 2>&1 && op="runInstallOp"
$op mkdir -p "${f%/*}" 2>/dev/null
tmp=$(mktemp 2>/dev/null) || return 1
if [[ -r "$f" ]]; then
while IFS=$'\t' read -r s r || [[ -n "$s" ]]; do
[[ -z "$s" || "$s" == "$slug" ]] && continue
printf '%s\t%s\n' "$s" "$r" >> "$tmp"
done < "$f"
fi
[[ -n "$root" ]] && printf '%s\t%s\n' "$slug" "$root" >> "$tmp"
if declare -F runInstallWrite >/dev/null 2>&1; then
runInstallWrite "$f" < "$tmp"
else
cat "$tmp" > "$f" 2>/dev/null
fi
rm -f "$tmp"
return 0
}
storageIndexRemove()
{
storageIndexSet "$1" ""
}
# Where the app's config SAYS it should live. Only consulted when the app isn't
# on disk yet — the disk always wins for a deployed app.
_appDirIntended()
{
local slug="$1"
# Only look up CFG_<SLUG>_STORAGE when <SLUG> can actually BE a variable
# name. A slug with a hyphen (a typo, or a name from an export file) makes
# the indirect expansion below emit "invalid variable name" to stderr and
# return non-zero, which surfaced as a misleading "storage location is not
# mounted" for an app that simply does not exist.
local key="" want=""
if [[ "$slug" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]]; then
key="CFG_${slug^^}_STORAGE"
want="${!key:-}"
fi
# Three states, and the distinction matters:
# <name> this app goes there, whatever the global default says
# primary this app goes on the install-time root, explicitly
# default no opinion — follow CFG_STORAGE_DEFAULT
#
# Templates ship "default", so a box configured with a big second disk
# picks it up for every new app without editing 37 configs; an app that
# was deliberately placed keeps its placement.
if [[ -z "$want" || "$want" == "default" ]]; then
want="${CFG_STORAGE_DEFAULT:-primary}"
fi
local path
if path=$(storageLocationPath "$want"); then
printf '%s' "$path"
return 0
fi
# Named a location that no longer exists — fall back rather than refuse, so
# a removed disk can't make an app un-installable.
primaryRoot
}
# Build the slug -> directory map by scanning every AVAILABLE root for
# <slug>/<slug>.config. Primary root first, so it wins a duplicate — a stray
# copy on a second disk can never hijack an app that is live on the primary.
#
# runFileOp because under rootless the container tree is owned by the docker
# install user and is not list-readable by the manager; without it this silently
# finds nothing and every app resolves to the fallback.
_appDirScan()
{
[[ "${LP_APP_DIR_SCANNED:-0}" == "1" ]] && return 0
LP_APP_DIR_SCANNED=1
_lpStorageRootsLoad
local scan_op=""
declare -F runFileOp >/dev/null 2>&1 && scan_op="runFileOp"
local root cfg slug dir
for root in "${LP_STORAGE_ROOTS[@]}"; do
[[ -z "$root" ]] && continue
storageRootAvailable "$root" || continue
while IFS= read -r cfg; do
[[ -z "$cfg" ]] && continue
dir="${cfg%/*}"
slug="${dir##*/}"
# Only <dir>/<dir>.config counts — an app dir holds other *.config
# payload files and those must not register as apps.
[[ "${cfg##*/}" == "$slug.config" ]] || continue
[[ -n "${LP_APP_DIR_CACHE[$slug]:-}" ]] && continue
LP_APP_DIR_CACHE["$slug"]="$root/$slug"
# Self-heal: what we can see is authoritative, so correct the index
# for a hand-move or a half-finished migration.
storageIndexSet "$slug" "$root" 2>/dev/null
done < <($scan_op find "$root" -mindepth 2 -maxdepth 2 -type f -name '*.config' 2>/dev/null)
done
return 0
}
# Every app DIRECTORY across every AVAILABLE root, one absolute path per line.
# THE enumerator — replaces `find "$containers_dir" -mindepth 1 -maxdepth 1
# -type d`, which only ever saw the primary root.
#
# Roots whose drive is absent are SKIPPED, not reported as empty. That
# distinction is load-bearing: callers that reap "folders that no longer exist"
# would otherwise delete database rows and port allocations for apps whose only
# crime is living on an unplugged disk. Such callers must gate on
# appStorageAvailable rather than on the absence of a directory here.
#
# Deduplicated by slug, primary root first, matching appDir's precedence — a
# stray copy on a second disk never doubles an app that is live on the primary.
storageAppDirs()
{
_lpStorageRootsLoad
local scan_op="" root d slug
declare -F runFileOp >/dev/null 2>&1 && scan_op="runFileOp"
local -A seen=()
for root in "${LP_STORAGE_ROOTS[@]}"; do
[[ -z "$root" ]] && continue
storageRootAvailable "$root" || continue
while IFS= read -r d; do
[[ -z "$d" ]] && continue
slug="${d##*/}"
[[ -n "${seen[$slug]:-}" ]] && continue
seen["$slug"]=1
printf '%s\n' "$d"
done < <($scan_op find "$root" -mindepth 1 -maxdepth 1 -type d 2>/dev/null)
done
}
# Every app's OWN config file (<root>/<slug>/<slug>.config), across available
# roots, deduplicated by slug. Deliberately stricter than a bare
# `find -name '*.config'`: an app dir may ship other *.config payload files
# (they get sourced as bash — see scan_files.sh), and those are not apps.
storageAppConfigs()
{
local d slug
while IFS= read -r d; do
[[ -z "$d" ]] && continue
slug="${d##*/}"
[[ -f "$d/$slug.config" ]] && printf '%s\n' "$d/$slug.config"
done < <(storageAppDirs)
}
# Slug form of storageAppDirs.
storageApps()
{
local d
while IFS= read -r d; do
[[ -n "$d" ]] && printf '%s\n' "${d##*/}"
done < <(storageAppDirs)
}
# THE resolver. Prints the app's directory (no trailing slash).
#
# Returns non-zero — and prints an unusable sentinel path — when the app's
# location exists but its drive is not mounted. This is the single central
# availability gate: every caller reaches it by construction, so a missing disk
# fails here instead of needing a guard at ~200 call sites.
appDir()
{
local slug="$1"
if [[ -z "$slug" ]]; then
printf '%s' "$lp_storage_unavailable"
return 1
fi
_appDirScan
local dir="${LP_APP_DIR_CACHE[$slug]:-}"
# Not on any AVAILABLE root. Before assuming it is new, ask the index —
# an installed app whose drive is unplugged looks identical to a new one,
# and guessing "primary root" there is the corrupting answer.
if [[ -z "$dir" ]]; then
local known=""
if known=$(storageIndexGet "$slug" 2>/dev/null) && [[ -n "$known" ]]; then
dir="$known/$slug"
else
dir="$(_appDirIntended "$slug")/$slug"
fi
fi
if ! storageRootAvailable "${dir%/*}"; then
printf '%s' "$lp_storage_unavailable/$slug"
return 1
fi
printf '%s' "$dir"
return 0
}
# Trailing-slash form, for the many call sites that built "$containers_dir$app/"
appDirSlash()
{
local d
d=$(appDir "$1") || { printf '%s/' "$d"; return 1; }
printf '%s/' "$d"
}
# True when the app's storage is present and usable. Cheap gate for callers that
# want to skip rather than fail (boot reconcile, status generators).
appStorageAvailable()
{
appDir "$1" >/dev/null 2>&1
}