LibrePortal/docs/roadmap/storage-locations.md

LibrePortal — Storage Locations (per-app data placement)

Status: Proposal — not built. · Audience: us, future-self · Scope: register more than one filesystem root for live app data, choose one per app, move an app between them, and resolve the right one on restore/migrate · Origin: "add different locations to set up LibrePortal on, with control per app" (2026-08-24)

---

0. The one idea

Today LibrePortal has three relocatable roots (--system-dir / --containers-dir / --backups-dir), each chosen once at install and fixed afterwards. This adds a fourth degree of freedom, on top of the existing containers root:

The containers root becomes a list. Every app declares which entry in that list holds its data. Everything else — install, compose, backup, restore, migrate — resolves the app's directory through one function instead of one variable.

Nextcloud's 4 TB of photos go on the spinning disk. Vaultwarden and the control plane stay on the NVMe. Jellyfin's library lives on the external HDD that isn't always plugged in — and when it isn't, LibrePortal refuses to start Jellyfin rather than silently rebuilding it empty on the bare mountpoint.

1. Non-goals

• ❌ Relocating the system root (configs/db/logs) per-app. It stays one place, chosen at install. Same for the WebUI's own container dir.
• ❌ A general volume manager. We don't format, partition, mount, or write /etc/fstab. The drive must already be mounted; we validate and use it.
• ❌ Striping/tiering/RAID-alikes. One app's data lives on exactly one location. No splitting an app across two.
• ❌ Per-volume placement inside an app (./data here, ./db there). Location granularity is the app directory. Revisit only if a real need shows up.
• ❌ Making backup locations and storage locations the same registry. They overlap on disk but differ in trust, lifecycle and ownership (see §9, §11.2).

2. What already works in our favour

The current code is closer to this than it looks:

• Compose volumes are relative. Every shipped template uses ./data:/config, and dockerComposeUp does cd $containers_dir$app_name && docker compose …. Change the cd target and the app comes up unchanged, with the same compose project name (derived from the directory basename, which doesn't change). The only absolute host paths in any template are /etc/localtime, /dev/net/tun, /etc/ssl/certs and the docker socket — none of them ours.
• An app dir is self-describing. <app>/<app>.config + docker-compose.yml + .libreportal-manifest.json is everything needed to identify and rebuild it. Discovery by scanning is viable, so we never have to trust a stale index.
• The three-root split already exists end-to-end: flag parsing, libreportalValidatePaths, the nesting/protected-path refusals, the baked __CONTAINERS_DIR__ placeholders, and paths.sh as the single source of truth. We're generalising a design that's already there, not inventing one.
• Backup locations are the exact template for the registry UX: configs/backup/locations/<idx>/location.config, locationAdd/locationRemove, sourceBackupLocations, a libreportal backup location … CLI, a WebUI Locations page, and backupLocationLocalGuard — which already implements the FAT/exFAT warning and the REQUIRE_MOUNT refusal we need verbatim.
• instance create proves the model. "An instance is just another app" — a cloned dir with its own slug and CFG_<SLUG>_* namespace. Per-app placement inherits multi-instance support for free.

3. The central constraint — root must not be told where to chown

This decides the whole design, so it leads.

The manager (libreportal) runs with a scoped sudoers allowlist: it may run the root-owned helpers in /usr/local/lib/libreportal/ and a fixed system-binary set, and nothing else. No sudo chown, no sudo tee, no sudo bash. The helpers have the three roots baked in at install by sed (__CONTAINERS_DIR__ …) precisely so that:

the manager cannot redirect a root chown by editing a config file.

libreportal-ownership even re-checks its baked roots against a dangerous-path list as defence in depth. A naïve "storage locations live in a manager-writable config, and the helpers read it" reopens that hole completely — chown -R dockerinstall /etc is a full escalation.

The fix: a root-owned registry with an empty-directory admission rule.

1. The truth lives at /usr/local/lib/libreportal/storage.roots, root:root 0644. One record per line: id<TAB>path<TAB>dev<TAB>fs_uuid. Manager reads it; only root writes it. Every helper resolves an app dir through it instead of through a single baked constant.

2. Adding a location goes through a new helper, libreportal-storage add <path>, which is in the sudoers allowlist — so the manager can call it, but it accepts a path only if all of these hold:

   • absolute, and realpath -e returns the input unchanged (no symlink component, no ..)
   • not in the protected set (/ /etc /usr /bin /sbin /lib* /boot /proc /sys /dev /run /var /tmp /root /home), and not inside any user's home unless the install was made with --allow-home
   • does not nest — in either direction — with the system/containers/backups roots or any already-registered location
   • is an empty directory (tolerating only lost+found) or already carries a .libreportal-storage marker — the adopt case, §3.1

   The emptiness rule is what makes this safe: root only ever chowns a directory that contains nothing, so acceptance can't hand away anything that already existed. Everything created underneath afterwards is ours by construction.

3. On acceptance the helper writes a root-owned .libreportal-storage marker (location id + install id + created-at), chowns the root to the container owner, chmod 0751, and appends the record. remove refuses while any app dir still lives there.

Consequence worth stating plainly: storage add is a genuine privilege boundary crossing, not a config edit. The WebUI can drive it (through the task system → CLI → helper, same as every other mutating action), but the helper, not the WebUI, is the gate.

3.1 — Why "empty or marked" is still safe (and why the marker earns its keep)

Requiring strict emptiness would break the most valuable case there is: plugging in a drive that already holds LibrePortal app data from another install and adopting it. So the rule relaxes to empty or carrying our marker — and that relaxation costs nothing, because:

writing the marker into a directory requires already being able to write that directory.

The manager can only plant a marker somewhere it can already write, and chowning a directory it already controls grants it nothing. There is no path where the marker gets the manager access it didn't have. (The fixed-path helpers can't be tricked into writing one elsewhere — none of them takes a caller-supplied destination.)

That single file then does three jobs, which is the main reason to like it:

Job	How
Admission	"empty or marked" — §3
Mount detection	the marker lives on the drive. Not mounted ⇒ bare mountpoint ⇒ no marker ⇒ location unavailable. No findmnt, no fs_uuid bookkeeping, and it works identically for USB disks, network mounts, and LUKS volumes that haven't been unlocked (§8.1)
Provenance on migrate	it carries the install id and location id the snapshot's manifest names, so "is this the same bigdisk the app came from?" is a file read (§7)

Residual wrinkle, not solved by elegance: validate-then-chown is a TOCTOU window, and bash is a poor language for race-free path handling. The practical closure is to additionally require that the parent directory is not manager-writable — true for /mnt, /srv, /media, false for a path inside the manager's home. That's a real restriction on where locations may live, not a free lunch, and it should be stated in the docs rather than hidden.

4. The resolution layer — one function, ~200 call sites

The mechanical bulk of the work. Three new primitives in scripts/source/paths.sh (and mirrored inline in init.sh, per the existing keep-in-sync note):

storageRoots            # every enabled root, primary first
appDir <slug>           # the app's directory — memoised
pathIsContainerData <p> # is this path under ANY container root?

• appDir builds a slug → dir map once per process by scanning each enabled root for */<slug>.config, memoises it in an associative array, and falls back to the primary root for a slug that doesn't exist yet (fresh install). Discovery-first, config-second: if the map and CFG_<APP>_STORAGE disagree, the directory on disk wins and we warn. That's what makes the system self-healing after a hand-move or a half-finished migration.

  It is also the single place the availability check belongs. Every one of the ~200 sites has to call it, so a location whose drive is missing fails once, centrally, instead of needing a guard sprinkled at each caller. appDir returns non-zero and prints an unusable sentinel path, so the many callers that won't check $? still fail loudly on a path that cannot exist, rather than writing into a bare mountpoint. See §8.1.

• pathIsContainerData replaces the [[ "$p" == "$containers_dir"* ]] idiom that decides manager-vs-container-user elevation. It appears in ~10 files (create_folder.sh, create_touch.sh, copy_file.sh, copy_files.sh, copy_folder.sh, copy_folders.sh, move_file.sh, runCfgOp, tags_manager_update.sh, webui_atomic_write.sh). Every one is a silent-corruption bug if missed — a file under an unrecognised root gets written as the manager, lands with the wrong owner, and the container fails to read it at a moment far removed from the cause.

Scale of the sweep (measured, not estimated):

Pattern	Count	Action
$containers_dir$app-shaped, app-scoped	~186 across ~90 files	→ $(appDir "$app")
$containers_dir/libreportal/… (the WebUI's own tree)	63	→ webuiDir helper, pinned to the primary root
$containers_dir/{traefik,prometheus,grafana,adguard,…} — one app reaching into another by literal name	~29	→ appDir <name>; those apps stay pinned in phase 1 (§8.3)
root-helper $CONTAINERS_DIR/$app	libreportal-ownership, libreportal-appcfg	→ registry lookup inside the helper
the == "$containers_dir"* elevation test	~10	→ pathIsContainerData

sourceScanFiles app_configs must also loop the roots. Keep the existing -maxdepth 3 + prune rules exactly as they are — and remember *.config is a reserved extension anywhere under an app dir.

Finding the sites we missed. A 90-file mechanical diff is exactly where a silent regression hides, and review alone won't catch a $containers_dir$app that survived. Make the runtime find them instead: once the sweep lands, stop defining containers_dir as a usable path and point it at a sentinel (/nonexistent-libreportal-unconverted/). Legitimate primary-root users have moved to primaryRoot/webuiDir by then, so every survivor now fails immediately, loudly, and harmlessly — on a path that cannot exist — with a sentinel string that greps straight out of the logs. Silent wrongness becomes a stack trace. Then a scripts/dev/ linter keeps the pattern from coming back, the way lp-task-names guards task titles.

5. Per-app configuration

One new key in every app template, defaulting to the primary location:

# STORAGE = which storage location holds this app's data (see Storage on /admin)
CFG_BOOKSTACK_STORAGE=default

• The value is a location name, not a path. Names are what survive a migrate to a host whose disks are laid out differently; paths aren't. default always resolves to the install-time containers root.
• It renders as a dropdown on the app's config page for free — the WebUI config renderer already builds selects from the [a:A|b:B] comment convention; the option list is generated from the registry into the config comment on regen.
• Changing it in the config editor does not move data. It records intent; the move is libreportal app move (§6). The config page shows the current resolved directory next to the field, and flags a mismatch.
• configBackfillMissingKeys carries the new key into already-installed apps on the next update, so nothing needs a reinstall.
• New compose tag #LIBREPORTAL|APP_DIR_TAG|APP_DIR_DATA, filled in dockerConfigSetupFileWithData alongside the existing CONTAINERS_DIR_TAG, for the rare template that genuinely needs its own absolute host path. Most apps need nothing — relative volumes already do the right thing.

6. Moving an installed app

libreportal app move <app> <location> — a task like any other, with progress rows in the WebUI.

1. Resolve source and destination; refuse if they're the same, if the app isn't installed, or if the destination isn't mounted/writable.
2. Space check with headroom: du -sb of the source vs df of the destination, refuse under 110 %.
3. Compose down (data must be quiescent — a live copy of a running Postgres is a corrupt copy).
4. Pre-move snapshot to the first enabled backup location, reusing migratePreBackupDestination. Skippable with --no-pre-backup, on by default.
5. runOwnership app-move <app> <dest-id> — the copy runs as root inside the helper because app data contains rootless sub-UID files (postgres at uid 231141 etc.) that the manager can neither read nor recreate.
   • same filesystem → mv (atomic, instant)
   • across filesystems → cp -a --reflink=auto → verify → only then remove the source. Never mv semantics that could half-delete.
6. Update CFG_<APP>_STORAGE, re-run dockerComposeUpdateAndStartApp (the cd target moves with appDir), bring it up, and health-check.
7. On any failure before step 6 the source is untouched; after step 6, roll back by pointing the config at the source and restarting.

7. Restore and migrate — the interesting case

This is the scenario that motivated the feature: "we migrate another install, it checks the path in the config against our current locations, and if there's no match the user picks an existing one or sets up a new one."

Add to .libreportal-manifest.json (written into the app dir, so it rides inside the snapshot):

"storage": { "location": "bigdisk", "path": "/mnt/bigdisk/apps/nextcloud", "fs_uuid": "…" }

On restore/migrate, resolve in this order: exact location-name match → a location whose fs_uuid matches → ask. The "ask" is a real WebUI step in the migrate wizard — "This app came from location bigdisk (/mnt/bigdisk/apps), which this host doesn't have. Restore it to: [ default ▾ ] or [ + add a location ]" — with a CLI equivalent (--storage=<name>) for unattended runs.

The blocker that must be fixed first. Restore currently does:

engineRestoreSnapshot "$idx" "$id" "/" "$containers_dir$app"

— restore to / with an include filter built from the local containers root. Restic reproduces the snapshot's absolute paths, so this only works when source and destination paths are byte-identical. That means:

• relocating on restore is impossible today, and
• this is already broken for the shipped three-root feature: migrating from a host installed with --containers-dir=/mnt/ssd/apps onto a default host matches no include path and restores nothing, quietly.

The fix is needed for this feature and is worth doing on its own: restore into $restore_dir staging with the snapshot's own path prefix, then move the tree into appDir. resticRestoreAppLatest, restore_app_start.sh, migrate_apply.sh and migrate_preflight.sh all take the source path from the snapshot/manifest rather than from the local containers_dir.

Also worth noting: an app's restic snapshot paths change when it moves. Snapshot history stays intact and restorable (each snapshot knows its own path), but path-based filters in the snapshot browser need to accept either.

8. Risks, in the order they'll bite

8.1 — An unplugged drive is the top data-integrity risk. crontab_boot_app_reconcile.sh brings every installed app up at boot. If a location's drive isn't mounted yet — or at all — docker cheerfully creates the bind-mount directories on the bare mountpoint and the app boots empty. Plug the drive back in and there are now two divergent copies, with the good one hidden underneath the mount. Mitigation is mandatory, not optional:

The mitigation is one test, not a family of them: .libreportal-storage lives on the drive, so marker present ⇒ mounted, and marker absent where the registry says a location is ⇒ not mounted, refuse. That subsumes findmnt, REQUIRE_MOUNT and fs_uuid comparison into a single file read, and it behaves correctly for cases those don't cover — a LUKS volume that hasn't been unlocked, an NFS/SMB mount that dropped, a mountpoint shadowed by a stale directory.

Where it's enforced:

• appDir fails on an unavailable location (§4) — the central gate, hit by every caller by construction
• dockerComposeUp refuses to start such an app — the gate that actually protects data, since it also covers manual and task-triggered starts, not just boot
• boot reconcile skips those apps with a loud notice and a dashboard warning rather than failing silently
• keep fs_uuid in the registry anyway, but as diagnostics ("this is a different disk than the one registered"), not as the liveness test

8.2 — Missing an elevation-test site (§4) produces wrong-owner files that fail much later. The sentinel-plus-linter approach in §4 is the mitigation: make the survivors fail loudly during the sweep, then make the pattern un-reintroducible.

8.3 — Cross-app literal paths. Traefik, Prometheus, Grafana, AdGuard, Gluetun, CrowdSec, Headscale are reached into by name from other apps' code. They're also small and infrastructural, so pinning them to the primary location costs a user nothing real.

Do it as a declared property of the app, not a hardcoded list in the engine: CFG_<APP>_STORAGE_PINNED=true in those templates, exactly mirroring how CFG_<APP>_MULTI_INSTANCE already gates instancing. The engine asks the config; the WebUI renders the field disabled with the reason; relaxing an app later is a one-line template change plus its call sites, with no core edit. The WebUI's own dir is pinned permanently and structurally (webuiDir), not by this flag.

8.4 — Filesystem capability. exFAT/NTFS/vfat can't hold POSIX ownership, so a rootless app dir on one is broken from the start — worse than for backups, where it's only a warning. For storage locations this must be a refusal, not a notice. Also check noexec/nosuid mount flags (some apps execute out of their data dir) and that the filesystem supports the sub-UID range rootless docker maps into.

8.5 — du/df reporting. The dashboard's disk gauge reads / only. With apps spread over disks it needs one gauge per location, or the number is actively misleading.

8.6 — Uninstall/teardown. init.sh uninstall prints and removes a fixed set of paths. It must enumerate the registry, and default to leaving external locations alone (with an explicit --remove-storage to wipe them) — an external drive is not ours to erase.

8.7 — Footprint version. New root helper + changed baked helpers ⇒ bump footprint_version (currently 5), or existing installs update the manager-owned code and keep root helpers that don't know about the registry.

9. What doesn't get a clean answer

Most of §8 collapses into a small number of good primitives — the marker file, appDir, the emptiness rule, a declared pin flag. Four things don't, and pretending otherwise would set us up to be surprised later.

• The TOCTOU window in libreportal-storage add. Validate-then-chown can't be made atomic in bash. The closure is a restriction — the parent directory must not be manager-writable — which rules out locations inside the manager's home and has to be documented as a rule users will occasionally hit and dislike.
• Two registries that look like one. Storage locations and backup locations will sit next to each other in the UI, both pointing at drives, with different lifecycles and different trust. No amount of engineering fixes that; it's a naming and information-architecture problem (§11.1) and the honest options are a genuinely different word or a single combined "Disks" page that owns both roles.
• The sweep is big no matter how good the target API is. ~90 files. Elegance makes the result better and the pattern un-reintroducible; it does not make the diff safe. Only the sentinel, the linter, and a full install → backup → restore → migrate soak on a real box do that — which is why phase 0 ships alone.
• Moving a large app is just slow. Cross-filesystem cp -a of a multi-terabyte Nextcloud is hours, needs root for sub-UID data, and the app is down throughout. There is no clever version. The deliverable is honesty about it: size estimate and duration warning up front, real progress in the task row, resumable on interruption, and never a delete of the source until the copy verifies.

10. Phasing

Each phase is independently shippable and independently verifiable. Phase 0 carries almost all of the regression risk and none of the user-visible value — land and soak it alone.

Phase	Deliverable	Visible?
0	appDir / storageRoots / pathIsContainerData / webuiDir; sweep the ~200 call sites; helpers read a registry that contains exactly one root. Full install + backup + restore verified.	No
1	Root-owned registry + libreportal-storage helper + libreportal storage {list,add,remove,check}. Locations can exist; nothing uses them yet.	CLI only
2	CFG_<APP>_STORAGE, install-time placement, per-location disk gauges.	Yes
3	libreportal app move.	Yes
4	Manifest storage block, staged restore + path rewrite (fixes §7's latent bug), the "unknown location" prompt in migrate.	Yes
5	WebUI Storage page mirroring Backup → Locations: add/remove, mount state, free space, apps per location, drag-to-move.	Yes

11. Open questions

1. Naming. "Storage location" vs "backup location" in the same UI — is that confusing enough to want a different word (volume? disk? pool?) for one of them?
2. Can a storage location double as a backup location? Same drive, different subdirectory, two registries. Recommend keeping them separate but having the picker offer known paths. Backing an app up onto the same drive its data lives on should at least warn.
3. Does the libreportal app itself ever get to move? Currently pinned. If the primary root fills up, that's a reinstall — acceptable, or worth solving?
4. Per-instance vs per-type placement. Instances inherit the model for free, but should instance create offer a location up front (likely yes — "work Nextcloud on the big disk, family on the SSD" is a good demo)?
5. --storage-dir= install flags for pre-registering extra locations at install time, or is post-install storage add enough?
6. Docs promise. docs/guide/install-and-use.md currently states the roots are "chosen at install and fixed afterward … part of the security model." That stays true of the three roots; the guide needs a paragraph explaining that additional containers locations are addable at runtime, and why the empty-directory rule keeps that honest.