The installer's restore path printed "Settings restored" and had never restored
a setting. backupRestoreSystemConfig only STAGES — right in general, since
overwriting a running control plane's config should not be automatic — but
nothing ever adopted the staged tree. The backup locations, domains and logins
landed in $restore_dir/system-config and stayed there.
So adoption is its own step now (`restore adopt`), allowed only on a machine
with nothing on it yet. backup/locations/ is adopted as a subtree, since the
index is part of the path and that directory is the whole point: it holds every
repository and its credentials, which is what makes "one password unlocks the
rest" true. Deliberately NOT adopted: the container account and its generated
password, port allocations, docker/rootless wiring, and storage/locations —
those describe the old machine, and a registry of drives this box does not have
would make every placement lookup resolve to a phantom.
The guard failed in the shape this project keeps hitting. It globbed the
containers directory, but the manager can traverse that without listing it, so
the glob returned a literal '*', the loop skipped it, and the function returned
"first run" on a machine with three apps. It adopted over a live install in
testing. It now asks the container user for the listing and fails closed: an
unreadable directory means "in use", never "empty".
Two config modes were inverted, found because a restore cannot restore from a
snapshot that was never taken:
- storage location configs were 0640 and hold no secrets. The backup runs as
the container user, could not read them, and restic wrote an INCOMPLETE
snapshot and exited 3 — so EVERY system-config backup failed once a second
storage location existed. Now 0644, with the test asserting they stay
secret-free so that mode remains defensible.
- backup location configs were 0644 and hold the repository password; nobody
could read them. They cannot simply be tightened, because the backup has to
read the credentials it uses — so the directory carries the restriction
(config-secure, manager:container 0750) and the file stays readable to the
two accounts that belong.
Fixing that surfaced a third: config-adopt clamped existing parent directories
to manager:manager 0750, closing configs/backup to the container user and
breaking the credential read the directory fix had just preserved.
restore domains reports which restored domains point here, and the installer
offers to drop the strays. Three verdicts, not two: setupCheckDomainPointsHere
falls back to hostname -I, and comparing a public A record to a private 10.x
address would condemn every correctly-pointed domain on a LAN-only box, which
is the deployment this product targets. Unverifiable is never offered for
deletion.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
promise.md names Connect as a paid service for "keeping off-site backups", with
two constraints that are load-bearing rather than marketing: it never sees your
data, and every hosted service has a free equivalent in the open code. Nothing
was implemented — no endpoint, no account, no client support.
The client half turns out to be almost entirely there, because a Connect
destination is not a new kind of thing: it is a restic REST repository whose
password never leaves the machine. So `connect` resolves exactly like `rest` in
resticLocationUri — it IS one. It is a separate TYPE only so the UI can tell it
apart from a REST server someone runs themselves, which are identical on disk.
Availability is data, not code: CFG_BACKUP_CONNECT_ENDPOINT (empty) is reported
through the locations feed as connect:{available,endpoint}, and the wizard
renders from that — the option present but disabled, its panel saying what it
will be and that SFTP and S3 do the same job today. The day the service exists,
setting that one value turns it on with no release. Verified both ways.
The device code is a credential, so it goes through the secret channel as a
reference rather than travelling in the wizard payload, which is base64'd into a
world-readable task file.
Design, and what the service still has to provide, in
docs/roadmap/connect-backup-destination.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The user-namespace prefix that lets an unprivileged restore put back a file's
original owner was only wired into restic. borg extract and kopia snapshot
restore run as the same backup user with the same lack of CAP_CHOWN, so both
lost <container-uid>:<backup-user> exactly the way restic did — an app whose
data comes back owned by the backup user cannot write it, which is how grafana
kept dying with "attempt to write a readonly database".
borg is quieter about it than restic: it does not print an "ignoring error"
line at all, so there was nothing to notice.
Move the prefix to engine_dispatch.sh as backupUsernsPrefix — it was never
restic-specific — and use it from all three engines.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
grafana restored and then died with "attempt to write a readonly database",
repeatedly. Its database is recorded in the snapshot as 231543:1002 and landed
as 1002:1002 — the owner was lost, so grafana, running as 231543, could not
write it at mode 0640.
Restore runs as the backup user with no CAP_CHOWN, so it reinstates ownership
inside a user namespace. The prefix was
unshare --map-root-user --map-users=SUB:SUB:N --map-groups=SUB:SUB:N
and unshare accepts ONE range per option, so the backup user's own GID was never
mapped — while app data is written as <container-uid>:<backup-user>. The group
half of every such chown referred to an unmapped id, lchown returned EINVAL, and
the file kept the restoring user's ownership. restic reports those as "ignoring
error ..." and still exits 0, so nothing failed: 1626 of one 13-app restore's
2086 failed chowns were grafana's, under a restore that reported success.
restic-userns-exec uses newuidmap/newgidmap, which write the multi-range maps
unshare cannot express:
uid: 0 <- caller inner root, or caps are dropped at exec
SUB.. <- SUB.. identity, so restic can name the stored uid
gid: caller <- caller identity: the group half of app-data chowns
SUB.. <- SUB.. identity
The caller's own UID is deliberately not identity-mapped — that slot is spent on
inner root — and a file stored as <caller>:<caller> lands owned by the caller
anyway, because that is who inner root is outside. So the one case this cannot
map is the one case needing no mapping. `unshare --map-auto --map-current-user`
is not a shortcut: it maps the subuid range to low inner ids while restic needs
identity. Tested.
Measured live, restoring grafana: failed chowns 1626 -> 12 (the 12 being the
caller's own files, correct), grafana.db back to 231543:1002, grafana up and
writing. Falls back to running the command plainly when there is no subuid
range, no newuidmap, or the namespace will not start.
scripts/dev/lp-userns-ownership-test pins all three ownership cases; verified
the old prefix fails it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
resticRestoreSnapshot forgives the un-mappable-uid lchown failures so a restore
is not aborted by them, and reported: "expected, they are already owned
correctly". That is true only for LibrePortal's own files, whose owner is the
backup user restic already runs as. It is false for container-owned data.
_resticUsernsPrefix maps the subuid range and root, but unshare takes one range
per option so the backup user's own GID is never mapped — and app data is
written as <container-uid>:<backup-user>. Every such chown fails with EINVAL and
the file falls back to <backup-user>:<backup-user>. Verified directly:
231543:231543 applies, 231543:1002 does not.
Observed on a 13-app restore: grafana's grafana.db is recorded as 231543:1002
and landed as 1002:1002, so grafana (running as 231543) could not write it at
mode 0640 and died with "attempt to write a readonly database" — under a restore
that reported success. 1626 of that run's 2086 failed chowns were grafana's.
This commit does not fix the mapping — that is the backup engine's ownership
handling rather than the first-run restore path, and the candidate fixes
(newuidmap multi-range maps, or restoring as root via a path-validated helper)
want a decision first. See docs/roadmap/first-run-restore.md §3.5. What it fixes
is the reporting: count the files and say plainly that container-owned data was
not reinstated and the app may fail to write.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`restore system` reported
✓ Success System config restored to: /libreportal-system/restore/system-config
for a directory that did not exist. Nothing had been written — on the step the
whole restore ordering depends on, since the system config carries every other
backup location's credentials.
Restore stages through $SYSTEM_DIR, which the manager owns, but the thing that
writes into the staging tree is restic, and runBackupOp runs it as the container
user. Both call sites created the directory as the wrong principal, in opposite
directions:
backupRestoreSystemConfig runFileOp mkdir -> container user; denied on the
0751 manager-owned restore_dir, and unchecked
storageRestoreAppTo runInstallOp mkdir -> manager; restic could then
not create anything beneath it
Restic reports a permission denial as "ignoring error ..." and still exits 0, so
the callers' success checks were satisfied either way.
libreportal-ownership gains restore-stage (creates it cowner:MANAGER 0750 —
owner writes, manager traverses to confirm and review) and restore-unstage
(removes it; neither principal can, so staging trees simply accumulated). Both
confine the path to one component directly under the restore/migrate area.
footprint_version 8 -> 9.
backupRestoreSystemConfig now verifies the tree landed as the user that wrote
it, because the manager cannot read inside its own staging directory.
Verified on a live install: system config stages 57 real files, and the
relocation branch of storageRestoreAppTo ran for the first time — speedtest
restored from a snapshot taken at /libreportal-containers/speedtest into
/libreportal-alt/speedtest via stage-and-move, staging cleaned up afterwards.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
storageSnapshotSourcePath resolved a snapshot's source path with
engineSnapshotsJson "$idx" "$snapshot_id"
but that function's second parameter is an app TAG filter. So it ran
`restic snapshots --tag app=<snapshot-id>`, matched nothing, and returned 1 —
every time, for every snapshot, since the file was written.
Nothing broke loudly, because both callers have a fallback:
* storageRestoreAppTo fell through to "restoring in place", reinstating the
exact cross-root bug the file exists to fix — restoring onto a host whose
containers root differs from the source's matched no include path and
restored nothing, silently
* the first-run preflight never read a manifest, so every app reported size
"?" and its fit and location checks passed unconditionally. Thirteen green
ticks that had checked nothing.
Add engineSnapshotPaths: restic answers it with a positional snapshot id, kopia
by filtering its list. borg has no adapter on purpose — it rebuilds its listing
from archive metadata that carries no paths — so a missing adapter is a quiet
"no" and those callers keep their in-place fallback.
Add scripts/dev/lp-preflight-test, which pins the cases that must say NO: an
app too big for the disk, one this version no longer ships, one whose storage
location is gone, and a resolver that reaches for the app-tag filter again.
Verified against both historical bugs — reintroducing either fails the test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
backupLocationEnsureDir and the config write both went through runFileOp /
runFileWrite, which run as the container user. Backup location configs live
under the system tree, which is owned by the manager — so the mkdir was denied,
the write then failed with "No such file or directory", and locationAdd still
printed "Location N added".
The result was a location that existed in name only: every later command that
sourced its config found nothing. It surfaced in the first-run restore path,
where the installer adds the location it is about to read from and then fails
with "Backup location 2 has no config".
Use runInstallOp/runInstallWrite, which run as the manager and can write there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Restore built its restic include filter from THIS host's containers root:
engineRestoreSnapshot "$idx" "$id" "/" "$containers_dir$app"
restic reproduces a snapshot's absolute paths, so that only works when
both sides agree byte-for-byte. LibrePortal has shipped configurable
roots for a while, so restoring a snapshot taken on a host installed with
--containers-dir=/mnt/ssd/apps onto a default host matched no include
path and restored NOTHING — with no error, because an include filter that
matches nothing is not a failure. Storage locations turn that from a rare
cross-host case into an ordinary one.
storageSnapshotSourcePath asks the repository where the app actually
lived. storageRestoreAppTo restores in place when that agrees with where
the app belongs here, and stages-then-moves when it does not — which is
also what makes "restore this app onto a different disk" possible at all.
Both restore_app_start.sh and resticRestoreAppLatest go through it, and
both fall back to the old behaviour when a snapshot does not report its
paths, so older snapshots restore exactly as before.
The move into place runs as root (app-adopt) for the same reason app-move
does: a restored tree carries container sub-UIDs the manager cannot
recreate. Staging is constrained to the restore/migrate area and the
destination is validated against the root-owned registry, so neither end
is taken on trust from the caller.
The manifest now records where an app lived — location name, path and fs
uuid. The name is what travels, since a path means nothing on the other
host; the rest is for diagnostics and for answering "is this the same
disk?" during a migrate.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The main sweep — ~260 call sites across ~100 files move from string
concatenation on a single root to appDir/storageAppDirs/storageAppConfigs.
On a single-root install the resolved paths are identical, so this is a
no-op until a location is registered.
Enumerators were the interesting half. `for d in "$containers_dir"/*/`
appears in the menus, the registry/artifact scanners and the DNS setup —
and a shell glob cannot list a rootless 751 tree at all, which is the
same bug config_find_file.sh already documents in a comment. Routing them
through storageAppDirs (which enumerates as the owning user) fixes that
alongside the multi-root work.
Three places needed judgement rather than substitution:
db_app_scan.sh deletes database rows and port allocations for apps whose
folder is missing, and reaps "empty" app dirs. With a storage location
unmounted, every app on it looks exactly like that. Each of those
branches now gates on appStorageAvailable first — an app on an unplugged
drive is skipped with a notice, never deleted.
instance_create.sh rewrites cloned hooks so an instance touches its own
directory instead of the base app's. Its sed matched ${containers_dir}<type>,
which this sweep just replaced with $(appDir <type>) — so it would have
silently stopped redirecting, and an instance would have written to the
original's files (the adguard auth adapter case its own comment warns
about). Now matches both appDir forms, verified against bare, quoted,
unrelated-app, legacy and prose cases.
peer_shell/peer_pull streamed and extracted relative to the primary root.
Both now use the app's own root, and peer_shell keeps a single-root
fallback since it runs as a restricted SSH shell with no LibrePortal env.
Also fixes a pre-existing bug found on the way: webui_app_config.sh
tested "$containers_dir/frontend/data/last_update", one level short of the
real tree under the libreportal app dir, so the WebUI refresh trigger
after a config update has never once fired.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two discoveries from exercising the capture path live:
- docker exec prints its 'executable file not found' OCI error to STDOUT
(a docker quirk), i.e. into the tar pipe — so the exit code (126/127)
is the only trustworthy no-tar signal, and the host-side 'not a tar
archive' noise is a symptom, not the cause. Detect on the code and say
plainly that the image has no tar.
- The two pipe halves shared one stderr file through an O_TRUNC fd and
an O_APPEND fd, racing and overwriting each other's lines — one file
per half, concatenated after.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- backup_files.sh / backup_db.sh: every docker exec/run in the capture,
sidecar-discovery, rehydrate and DB-import paths now goes through
runFileOp — bare docker can't reach the rootless daemon socket, which
made live capture fail (and silently bounce containers) on every
rootless install, and would have broken DB restores the same way.
- capture/rehydrate stderr is kept and printed on failure instead of
being discarded, with a clear message when the image has no tar.
- backup_app_start.sh: when no location produced a complete snapshot the
backup now returns 1 — the task is marked failed instead of logging a
nonexistent/incomplete backup as a success and skipping verification.
- restic engine: on restic exit 3 the orphan incomplete snapshot is
called out explicitly so nobody restores it believing it is whole.
- speedtest: capture /config through the container (root-owned TLS key
and logrotate state are unreadable from the host), which also flips
its auto strategy to live — no more container stop per backup.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both _titleBlock implementations title-cased the slug themselves instead of
calling getAppDisplayName, so the Tools tab read "Run app-specific actions for
Rocketchat" and Services read "the docker compose services that make up
Speedtest".
getAppDisplayName already resolves a slug to the app's declared title through
window.apps. Using it fixes four apps beyond Rocket.Chat:
rocketchat Rocketchat -> Rocket.Chat
speedtest Speedtest -> LibreSpeed
ipinfo Ipinfo -> IPinfo
libreportal_catalog Libreportal Catalog -> LibrePortal Catalog
The slug casing is kept as the fallback for the window.apps-not-loaded-yet case,
which is what the helper does internally anyway.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Rocket.Chat and Stoat are both MongoDB-backed, and the backup engine only
understood postgres, mysql/mariadb and sqlite — so a live snapshot of either
would have captured a torn data directory that may not even mount.
Adds mongo as a fourth kind: mongodump --archive on the backup side,
mongorestore --archive --drop on the restore side (idempotent, so the caller's
retry loop works the same as it does for pg_dump --clean), and a ping-based
readiness probe that also waits out a replica set electing its primary.
Credentials are optional. The shared sh preamble sets them from
MONGO_INITDB_ROOT_USERNAME/PASSWORD when present and passes nothing when not,
built with 'set --' so a password containing spaces survives word splitting.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Follow-up to the backup-refresh throttle/dedupe, cutting the cost of the
remote pulls that do still happen.
* restic stats now runs in --mode raw-data (restic_check.sh). The default
restore-size mode walks every snapshot's tree to sum logical file sizes —
the slowest restic op — just to fill a size readout. raw-data reads the
index only and reports the repository's actual deduplicated on-disk size,
which is exactly what the dashboard already labels "deduplicated,
encrypted". raw-data omits total_file_count, so the per-location card now
shows that location's snapshot count (already loaded client-side, and more
useful for a backup repo) instead of a file count.
* engineLocationStats now shares the same per-refresh memoiser as
engineSnapshotsJson (engine_dispatch.sh). Both the locations and dashboard
generators call it per location, so repo stats went from two restic calls
per location per refresh to one. Factored the cache into _engineCachedPull.
* SSH connection reuse for SFTP locations (backup_ssh.sh): ControlMaster=auto
with a self-reaping ControlPersist master, so the several restic
subprocesses a refresh/backup spawns against one location share a single
authenticated connection instead of a fresh handshake each — the dominant
per-call cost on a high-latency link. Toggle via CFG_BACKUP_SSH_MULTIPLEX.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The "Refreshing backup data..." step on every WebUI update fired one
`restic stats` (restore-size mode — the slowest restic op) plus ~4
identical unfiltered `restic snapshots --json` pulls per enabled
location (dashboard, snapshots, app-status, migrate each pulled their
own), all over a fresh SSH connection for remote repos, on every pass
with no throttle — the slow, "frozen"-looking line users hit on poor
links.
Two fixes:
1. Dedupe. engineSnapshotsJson transparently memoises the first
unfiltered whole-repo pull per location to a shared cache dir
(LP_SNAP_CACHE_DIR, set by webui_updater around the chain), so the
four generators reuse one restic call instead of four. Filtered and
failed/empty pulls always fall through to a live call.
2. Throttle. Backups and location changes already regenerate this data
live when they happen, so the routine pass is only a drift catch-up.
Split the cheap local-only generators (engines/schema/passwords —
no remote I/O) out to always run, and gate the remote pull behind
CFG_BACKUP_DASHBOARD_REFRESH_INTERVAL (minutes, default 30, 0 =
every update). A completed backup touches a dirty marker that forces
the next pass to pull; WEBUI_UPDATER_FORCE still forces a full
refresh.
Net: N locations x 5 remote restic calls every update -> 1 call per
location, only when something actually changed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The config-category icons sat at admin/config/icons/CONFIG/ — the inner config/
duplicates the subsystem name; they belong in the icons root. Moved all 6
(backup, features, general, network, security, webui) up to
components/admin/config/icons/ and updated the two consumers (config-manager.js
header icon, config-sidebar.js category icons).
Also fixed the backup-engine logos: scripts/backup/engines/{restic,kopia,borg}
.json pointed 'logo' at /icons/config/backup.svg — a path that 404'd on two
counts (missing the components/admin/config prefix AND the now-removed config/
nesting), so the engine-details modal logo silently hid. Repointed to the real
served path /components/admin/config/icons/backup.svg.
(Left the meaningful icon groupings alone — admin/system/icons/{cpu,os} and
apps/core/icons/vpn are vendor/OS/provider logo sets, not redundant nesting.
The backup engines borrowing an admin-config icon is a minor smell; a dedicated
backup-engine icon could replace it later if wanted.)
Signed-off-by: librelad <librelad@digitalangels.vip>
The location_loader's find ran through runFileOp (sudo -u libreportal),
which forks a sudo shell purely for the find call. configs/backup/
locations/ is already manager-owned (not in the dockerinstall-owned
containers tree), so the sudo step adds ~50 ms of process-creation
overhead for zero security benefit.
Plain `find` now, with a tiny [[ ! -r || ! -x ]] guard that falls
back to runFileOp if someone relocates the dir to a non-traversable
location. Same observable behaviour, ~50 ms saved per CLI invocation.
This is the simpler half of Phase 6 — the libreportal_configs scan
itself was already plain-find (only ~11 ms total for 22 files). The
remaining costly scan is app_configs against /libreportal-containers/,
which legitimately needs sudo because dockerinstall owns that tree.
Precompiling its content is possible but adds invalidation complexity
(updateConfigOption writes happen at runtime) — deferred for now;
better return on simpler interventions.
Signed-off-by: librelad <librelad@digitalangels.vip>
The hardcoded uid:gid in libreportal.backup.files labels was brittle: matched the
default PUID in the compose, but a PUID change (or new image version) would drift
silently and the next restore would chown to a stale owner. Make it impossible to
drift by letting the engine learn the uid at capture time.
backup_files.sh:
- After a successful tar capture, run `stat -c '%u:%g'` inside the container and
write the result to a <host_subdir>.lp-owner sidecar in staging. The sidecar
rides in the snapshot alongside the captured tree.
- Restore reads it back when the descriptor doesn't pin uid:gid; falls back to
0:0 with a clear notice if missing.
- The 5-field form (with explicit uid:gid) is still supported as an override; it
wins and skips the sidecar write entirely.
Update all 4 current labels to the new 3-field form
"<container>:<container_path>:<host_subdir>" (nextcloud, bookstack, gitea,
owncloud). Engine handles both formats during the transition.
Verified with stubs: 3-field capture writes the sidecar with the discovered
33:33; restore reads it back; 5-field override correctly skips the sidecar
write. backup_files.sh parses.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Mirror the restic system-config adapters for the other two engines, each in that
engine's own convention, so system backup/restore/status/retention work on any
location regardless of engine:
- kopia: BackupSystemToLocation (--tags system:config), SystemSnapshotsJson
(filter tag system:config), RestoreSystemLatest, ForgetSystem (per-source policy
on $configs_dir + maintenance).
- borg: BackupSystemToLocation (archive system-<host>-<ts>, comment system=config;
no app is named "system" so the namespace can't collide), SystemSnapshotsJson
(--glob-archives system-<host>-*), RestoreSystemLatest, ForgetSystem (prune the
system-<host>-* glob).
No dispatcher change needed — engineBackupSystem/SystemSnapshotsJson/
RestoreSystemLatest/ForgetSystem already resolve <engine><fn> per location. All
three engines now define the full set; syntax clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Final-review gaps in the system-config backup:
1. Scheduled (cron) backups skipped it — backupScheduleEnabledApps only queued
per-app backups, so the daily schedule never refreshed the system config (and
thus the backup-location creds could go stale). Now it queues a
`libreportal backup system` task (or runs inline on terminal-only installs),
and skips the reproducible libreportal app for consistency with backupAllApps.
2. No retention on system snapshots — they bypass backupAppStart's per-app forget,
so they accumulated unbounded. Add resticForgetSystem (tag system=config,
respects append-only + the same keep-* policy) + engineForgetSystem dispatcher;
backupSystemConfig now applies retention across all locations after snapshotting.
Verified with stubs: backupSystemConfig snapshots AND prunes on every location;
engineForgetSystem pairs with resticForgetSystem; scheduled createTaskFile call
matches the existing 3-arg signature.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Make the system config a tracked backup, not just action buttons:
- engine: resticSystemSnapshotsJson (tag system=config) + engineSystemSnapshotsJson
dispatcher — query the system snapshots the way per-app status is queried.
- webui_backup_dashboard.sh: emit a "system": { latest_snapshot, latest_time }
object (latest system snapshot on the primary location), and exclude the
libreportal WebUI app from the per-app grid (it's intentionally not backed up, so
it no longer shows a perpetual "No backup yet" tile).
- backup dashboard card: a status line (dot + "Last backed up <relative>" / "No
backup yet"), populated in renderDashboard from d.system — mirrors the app tiles.
Verified: shell + JS parse; dashboard content assembles to valid JSON with the
system key; engine query defined + dispatched; frontend reads d.system into the
#backup-system-status element.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
(a) Docs: reserve tools/ scripts/ resources/ as LibrePortal folder names (apps must
not bind-mount to them); document resources/ as the home for nest-able data AND for
.sh payloads that execute on load (vs scripts/ for sourced functions); document the
backup model (what's captured vs reproducible).
(b) System-config backup so a bare-metal restore is self-sufficient — this is why
the system root is its own tree. New scripts/backup/system/backup_system.sh:
- backupSystemConfig snapshots <system>/configs (global settings, WebUI creds, and
the BACKUP-LOCATION creds — otherwise the keys to reach your own backups live only
on the box) to every enabled location. Lightweight static-dir snapshot — it does
NOT go through backupAppStart (no containers to quiesce / DBs to dump).
- restic adapter resticBackupSystemToLocation (tag system=config) + dispatcher
engineBackupSystem; restore via resticRestoreSystemLatest / engineRestoreSystemLatest
+ backupRestoreSystemConfig (restores to a STAGING dir — never auto-overwrites
live config).
- backupAllApps runs it after the app loop.
WebUI exclusion: backupAllApps skips the 'libreportal' app — its frontend + generated
JSON regenerate, and its only state (the login) is in the system config now captured
above. Nothing in its data dir warrants a snapshot.
Verified with stubs: app loop skips libreportal + invokes the system backup; the
system backup dispatches to both locations; backup/restore function names pair with
the dispatcher. NOTE: restic-only (the sole live engine adapter); end-to-end repo
round-trip still needs a live box before being relied on.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
backupLocationLocalGuard (engine-agnostic, in location_paths.sh), wired into the
dispatcher before init, readiness, and every backup write (engineInitLocation /
engineEnsureLocationReady / engineBackupApp):
- Filesystem warning: the ownership model chowns the repo to the backup user, which
needs POSIX permissions — warn (non-fatal) on FAT/exFAT/NTFS via findmnt FSTYPE.
- Mount-presence refusal: a location with CFG_BACKUP_LOC_<idx>_REQUIRE_MOUNT=true
(an external/removable disk) is refused when its path isn't on a real mount
(findmnt TARGET is '/' or unknown) — so an unplugged drive never silently fills
the system disk. Opt-in; default false leaves on-disk locations unaffected.
New REQUIRE_MOUNT field documented in the location.config template (location_add.sh)
so it surfaces on the Locations page. Verified: REQUIRE_MOUNT+unmounted refuses;
default allows; non-local no-ops.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Introduce scripts/source/paths.sh as the canonical path resolver for three
independently-relocatable roots:
LP_SYSTEM_DIR manager-owned control plane (configs/logs/install/db/ssl/ssh/migrate)
LP_CONTAINERS_DIR container-user-owned live app data
LP_BACKUPS_DIR container-user-owned backup repos (own mount-able)
Roots come from the environment when set (install bakes them; CLI/app inherit
from init.sh), else default to /libreportal-*. A transitional compat default
keeps EXISTING installs (legacy single /docker tree, by config marker) on /docker
until a deliberate reinstall, so deploying this never strands a running box.
- init.sh derives the same vars inline (self-contained for the bare /root/init.sh
reinstall case); paths.sh mirrors it for the standalone task/check processors,
which now self-locate their scripts dir and source it.
- Replace functional /docker literals with the derived vars across runtime,
install, backup, crontab, crowdsec/restic, headscale, and reinstall paths;
clean the inert '== /docker/containers/*' guard fallbacks to the variable form.
- backend: CONTAINERS_DIR now from LP_CONTAINERS_DIR (compose env, filled at
generation via a new CONTAINERS_DIR_TAG), legacy-safe default for un-recreated
containers.
- backup default path falls back to the backups root; exclude paths.sh from the
sourced-file arrays (bootstrap file, sourced explicitly).
The CLI-wrapper heredoc + root helpers still reference /docker; those get baked
in phase 3. No layout/ownership change yet (phase 2).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Bring the remaining deferred subsystems under the scoped sudoers, and drop
the one that's redundant.
Backup engines + app configs -> root-owned helpers (same pattern as
ownership/dns/ssh/socket/svc):
- scripts/system/libreportal-bininstall: install <restic|kopia> — does the
whole pkg-manager/signed-download install itself for a fixed, validated
engine name (no blanket sudo apt-get/install). restic_install/kopia_install
call it.
- scripts/system/libreportal-appcfg: {adguard-auth <user> <bcrypt>|
crowdsec-priority|owncloud-config <public> <host> <ip> <public_ip>} —
faithful ports of the AdGuard yaml / CrowdSec bouncer / ownCloud config.php
rewrites, fixed paths + validated args. adguard_auth/crowdsec_fix_priority/
owncloud_setup_config call it.
- run_privileged: runBinInstall / runAppCfg; init.sh installs + allowlists both.
Retire standalone (host-level) WireGuard — it's a duplicate of the
containerized containers/wireguard app (+ headscale mesh), its slirp4netns
speed rationale is largely moot with a better rootless net backend / typical
WAN-bound throughput, and it was the heaviest host-root subsystem (apt +
sysctl + iptables + /etc/wireguard), the worst fit for the rootless/
least-privilege direction:
- moved scripts/wireguard/ + manage_wireguard.sh + check_wireguard.sh to
scripts/unused/; dropped the install-path call, the Tools menu 'w' entry,
and the requirement check; removed the half-built libreportal-wg helper.
- generate_arrays.sh now also skips system/ (root-owned helpers, never
sourced); arrays regenerated (files_wireguard.sh pruned).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
- docker_run: in rooted mode run docker AS the manager via the docker
group (no sudo); the type=='sudo' branch was unreachable dead code
- 8 db helpers: fix 'command -v sudo sqlite3' guard to 'command -v
sqlite3' (bodies already query via runInstallOp)
- restic/kopia single-file dump: write target_file via runBackupOp tee
(as the backup user, matching the snapshot-restore path) instead of
root tee
- adguard auth: root-owned scratch via runSystem mktemp
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The borg/restic/kopia engines all dropped to the dedicated backup user
via scattered 'sudo -E -u $docker_install_user'. Centralize that into a
single runBackupOp helper so the backup subsystem has one audit point and
the scoped sudoers needs only the (dockerinstall) drop rule.
Also:
- owncloud config heredoc tees -> runSystem (container-UID file)
- webui_display_logins: fix the broken 'command -v sudo sqlite3' guard
to 'command -v sqlite3' (body already runs sqlite3 via runInstallOp)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The backup engine already drops to the backup user (sudo -E -u
$docker_install_user) and backupLocationOwner == $docker_install_user, which is
exactly what runFileOp/runFileWrite resolve to in both modes. So convert the
raw-sudo data ops (mkdir/chmod/rm/find/cat/grep/mv/chown/tee on backup repos,
location configs, keys, manifests) to runFileOp/runFileWrite — creating files
as the owner directly, no root chown. backup_verify creates its scratch as the
backup user (runFileOp mktemp) instead of chown-after. Binary installs
(kopia tar/install, borg dnf) -> runSystem. The 44 sudo -u engine drops stay
(already least-privilege; the scoped sudoers will grant them).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Model A prototype (run start.sh AS the manager, escalate only via helpers):
- check_root.sh: accept the manager user, not root-only (init.sh keeps its own
install-time root check).
- init.sh: guard the top-level root-check + installer entrypoint with
BASH_SOURCE!=$0 so it runs ONLY when init.sh is executed directly; when
start.sh sources it as the manager the entrypoint (and its root check) no
longer fires.
Also: convert bare daemon-touching 'docker' calls (no helper -> hit the
nonexistent /var/run socket in rootless) to runFileOp docker across
app_status, app_health_*, network_prune, ip_is_available, check_docker_network,
backup_db (db dumps) and crontab_check_processor. cd&&compose rooted-branches
and 'docker compose --version' checks left as-is (rooted-only / no daemon).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The apps SQLite DB ($docker_dir/$db_file) is owned by the manager user, so
read/write it AS the manager via runInstallOp instead of sudo (root). 48 call
sites across 28 scripts. In rooted this drops root->manager (correct owner);
in rootless it's the manager too (using runFileOp/dockerinstall here was the
'unable to open database' bug). The broken 'command -v sudo sqlite3' check
lines are left untouched (separate pre-existing issue).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Foundation for a scoped sudoers: route every genuine system-admin command
(systemctl/ufw/ufw-docker/nft/apt/apt-get/pacman/sysctl/useradd/usermod/
service/wg/wg-quick/cscli/loginctl) through runSystem instead of raw sudo
across 28 active scripts. runSystem is 'sudo "$@"' so this is byte-identical
in every mode (safe on live installs) — it just collects all real-root use at
one chokepoint that will define the eventual /etc/sudoers.d allowlist.
Also: revert a crowdsec advice message the sweep wrongly rewrote (the admin
types sudo, not runSystem), and give crontab_check_processor.sh the same
startup bootstrap as the task processor — it runs standalone via cron and
already used runFileOp/runFileWrite (undefined there), so it was silently
broken; now it sources the helpers + docker-type config.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
- restic_install, crowdsec_update/verify_firewall/fix_priority: pure host
ops (apt/cscli/nft/systemctl, /etc/crowdsec) -> runSystem.
- kopia_backup/borg_restore: ignore-file/target tee+chown+mkdir -> runFileOp/
runFileWrite; kept the 'sudo -E -u dockerinstall' engine calls as-is —
those already run as the unprivileged backup user (least-privilege; the
scoped sudoers will permit (dockerinstall)).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Convert the backup/restore data-plane sudo calls (mkdir/chown/rm/sqlite3/tar/
gzip|tee) to runFileOp/runFileWrite. Rooted behaviour is identical (helper runs
sudo); rootless will run them as the unprivileged install user. Pilot subsystem
for the wider de-sudo. verify.sh's /tmp scratch ops left as-is (different
ownership domain, handled separately).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
With live dumps + container-side file captures the live app dir intentionally
differs from the snapshot (raw DB dirs and private trees are excluded, replaced
by dumps/captures), so the old source-vs-restored file-count check false-failed.
The scratch restore succeeding already proves restorability (restic hash-checks
every blob); keep a non-empty sanity check instead.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The container capture preserved the app's ownership (e.g. www-data 0640), so
restic still hit permission denied on the staging copy. chown the staging tree
to the backup user after capture (modes unchanged, so the owner reads fine);
real ownership is reapplied from the descriptor on restore.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Reads files the backup user can't see from the host (container-owned, e.g.
Nextcloud's www-data data dir) by streaming them out THROUGH the container
(docker exec tar) — no host root, no host read perms, works rooted + rootless.
Extracts to staging as plain files so restic keeps full dedup + per-file
restore (not a piped tar blob); the live path is excluded from the snapshot.
Restore streams the staging copy back through a throwaway in-namespace
container that recreates the tree with the app's uid:gid.
Declared via a libreportal.backup.files compose label; Nextcloud (html, 33:33)
is the first to use it. Live capture failure falls back to stop-snapshot-start.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Live-restore of a server DB (MariaDB/Postgres) raced the engine's first-init:
it starts a throwaway temp server, runs setup, then restarts the real one. The
old ping-based readiness passed against the temp server and the load hit the
restart, failing once.
- _backupDbWaitReady now requires a real query to succeed on two consecutive
checks, so the restart breaks the streak and we only proceed once the real
server is stably up.
- The dump load is retried (idempotent — the dump drops+recreates each object)
to ride past a final init bounce.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Found while testing live backups end-to-end:
- Engine backup adapters logged to stdout, so the caller's $() snapshot-id
capture was polluted with log text — verify-after-backup then failed with
'no matching ID' on every run. Route their log lines to stderr so stdout is
only the id (restic/borg/kopia).
- 'libreportal app restore <app> --latest' (as the help advertises) and the
bare 'restore <app>' both failed: --latest was passed to restic verbatim and
unset args arrive as the literal 'empty'. Normalise both to 'latest'.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
When a location uses SSH key auth, show a key card: paste an existing private
key, or 'Generate keypair', then the card displays the public key to copy into
the remote server's authorized_keys (with Copy/Delete). Wires to the
ssh-key-set/generate/delete CLI; key mutations refresh locations.json so the
card reflects state immediately. applySshAuthVisibility toggles the card vs the
password field by auth mode. Private key only ever flows in (base64); only the
public key is ever shown.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Expose the existing location_ssh.sh key store through the backup CLI:
'backup location ssh-key-set|ssh-key-generate|ssh-key-public|ssh-key-delete <idx>'
(the WebUI runs these as tasks). The locations generator now emits
ssh_key_exists + ssh_public_key (public key only — the private key never
leaves the per-location ssh.key file), so the editor can show the key state.
Also fix the stale SSH_AUTH label (~/.ssh/id_rsa -> managed per-location key).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Adds CFG_<APP>_BACKUP_STRATEGY (default auto) so an app's backup strategy can
be overridden from its Advanced config tab, taking precedence over the global
default. Added to the 10 live-capable apps, so the dropdown's 'live' option only
appears where it actually works.
- backupResolveStrategy now checks the per-app override before the global value.
- backupAppLiveCapable / backupAppStrategyOptions expose capability + the valid
option set; predicate helpers hardened with explicit returns so they behave
identically with or without shell errexit.
- BACKUP_STRATEGY field mapping (select, advanced) renders the dropdown.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
- Add libreportal.backup.db labels to the MariaDB/Postgres apps (nextcloud,
owncloud, bookstack, mastodon, invidious) so they back up live + consistent.
- If a declared dump cannot be taken (DB down, wrong path), the backup falls
back to stop-snapshot-start for that run instead of snapshotting torn data —
a misconfiguration degrades to 'safe with downtime', never to 'unsafe'.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Adds a logical-dump path so apps with a database can be backed up with zero
downtime and full consistency, instead of stopping the container.
- backup_db.sh: dump each declared DB live (mysqldump --single-transaction /
pg_dump / sqlite3 .backup), exclude the raw data dir from the snapshot, and
replay the dump on restore (pre-start rehydrate for sqlite, post-start load
for server engines).
- Databases are declared via a 'libreportal.backup.db' compose label so the
metadata travels with the app in the snapshot.
- New 'auto' strategy (now the default): live where a DB is dumpable or the app
is marked live-safe, stop-snapshot-start otherwise. Explicit stop/pause/live
remain as overrides.
- restic/borg/kopia adapters honour an exclude list on the live path.
- Manifest records the resolved per-app strategy and dumped databases.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Automatic path mode hardcoded /docker/backups/<id>, baked into the Path Mode
dropdown label. Add a CFG_BACKUP_DEFAULT_PATH option in the Backup Engine
config ("Default Backup Location", default /docker/backups) and have
backupLocationResolvedPath build the auto path from it (<base>/<id>, trailing
slash tolerated). Defaults to the old path, so existing auto locations are
unchanged.
Path Mode's option is now just "Automatic" (no inline path); its tooltip
points at the Default Backup Location config option instead.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The backup engine is an implementation detail — LibrePortal picks a sensible
default and handles it — so it doesn't belong next to Name/Type on the
Connection tab. Add ENGINE to LOC_ADVANCED_SUFFIXES and mark it **ADVANCED**
in the location.config template + seed so it's metadata-driven.
Since the engine select now lives in the Advanced tab while SSH-auth and
path-mode stay on Connection, refreshInlineTypeFields re-applies the dynamic
behaviors (engine filtering, SSH/path visibility) against the shared
.task-details scope rather than a single panel.
Also fixed the live per-location engine label (restic -> Restic) which now
surfaces in the dropdown via the generator-emitted options.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The config generator only scanned flat per-category files, so the dynamic
CFG_BACKUP_LOC_N_* keys carried no titles/descriptions/options — the Locations
editor had to hardcode that metadata in backup-page.js. Add a pass that
descends into configs/backup/locations/<n>/location.config and emits each key
(value/title/description/options) into the config map, plus an "advanced"
flag parsed from a **ADVANCED** token in the field comment (stripped from the
user-facing description).
These keys use subcategory "backup_locations", which isn't in any category's
subcategory_order, so the generic /config page ignores them — only the custom
Locations editor consumes them. URI, SSH port, and append-only are marked
advanced. Verified: configs.json stays valid JSON and /config subcategories
are unchanged.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
- Display the restic engine as "Restic" to match BorgBackup/Kopia. The
lowercase name lived in scripts/backup/engines/restic.json (drives the
location-row engine pill, per-location engine select, and engine modal),
the hardcoded per-location dropdown options, the engine-list fallback, and
the config-option metadata. All set to "Restic".
- In each location's Engine dropdown, float the system-default engine
(CFG_BACKUP_ENGINE) to the top and tag it "(default)", mirroring the
retention-preset pattern.
Repo config metadata is the install template (add-only reconciliation), so
the live /docker/configs/backup/backup_engine label was updated in place too
for the global Configuration-tab dropdown on this install.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Application backups were driven by one crontab entry per app, each offset by
id * CFG_BACKUP_CRONTAB_APP_INTERVAL minutes. That minute offset is written
straight into cron's 0-59 minute field, so past ~20 apps it overflowed into
an invalid entry that silently never fired, and the fixed spacing could not
serialize backups that ran longer than the gap.
Replace it with a single daily entry (`libreportal backup scheduled`) that
enqueues a backup task per enabled app. The existing systemd task processor
drains them serially — no minute overflow, real serialization, and backups
are now visible/cancellable in the Tasks UI. Per-app enable is read from
CFG_<APP>_BACKUP at schedule time instead of being mirrored into crontab.
Removes the stagger machinery (timing/setup/check/remove scripts), the
now-unused cron_jobs table + insert, and the CFG_BACKUP_CRONTAB_APP_INTERVAL
config knob and its WebUI field.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
A free, open, self-hosted app platform (GNU AGPLv3): one-click app deploys,
Traefik reverse proxy with automatic SSL, rootless Docker support, gluetun
VPN routing, and a web dashboard to manage it all.
Free & open forever to self-host; optional paid hosted services fund it.
See PROMISE.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
