promise.md names Connect as a paid service for "keeping off-site backups", with
two constraints that are load-bearing rather than marketing: it never sees your
data, and every hosted service has a free equivalent in the open code. Nothing
was implemented — no endpoint, no account, no client support.
The client half turns out to be almost entirely there, because a Connect
destination is not a new kind of thing: it is a restic REST repository whose
password never leaves the machine. So `connect` resolves exactly like `rest` in
resticLocationUri — it IS one. It is a separate TYPE only so the UI can tell it
apart from a REST server someone runs themselves, which are identical on disk.
Availability is data, not code: CFG_BACKUP_CONNECT_ENDPOINT (empty) is reported
through the locations feed as connect:{available,endpoint}, and the wizard
renders from that — the option present but disabled, its panel saying what it
will be and that SFTP and S3 do the same job today. The day the service exists,
setting that one value turns it on with no release. Verified both ways.
The device code is a credential, so it goes through the secret channel as a
reference rather than travelling in the wizard payload, which is base64'd into a
world-readable task file.
Design, and what the service still has to provide, in
docs/roadmap/connect-backup-destination.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The step asked one question — pick a destination, or "not now" — while the
system underneath already had a full location model: eight backend types, per
location engine, path mode, credentials and retention, and a generated
locations.json carrying all of it. None of that was reachable during setup, so a
second destination, or even seeing where the first one points, meant finding the
Backup page afterwards.
Now it mirrors the Storage step — the choice above, the list below:
Backups Automatic — daily, on a schedule | Manual
Destinations Local disk [default] /libreportal-backups/1 [Edit]
+ Add destination
Automatic/Manual needed a setting, because there was no off switch:
crontabSetupBackupScheduler installed the entry unconditionally. CFG_BACKUP_MODE
is explicit rather than overloading "empty schedule", so it reads properly in
the config editor too, and Manual REMOVES an entry that is already installed
rather than merely declining to add one — otherwise answering Manual changes
nothing. The schedule itself is left alone, so switching back restores the time
the user picked.
Destinations are seeded from locations.json, so the default one is shown and
editable instead of being discovered later, and only entries the user actually
added or changed are submitted. A destination on the same disk as the app data
says so on the card rather than in a paragraph under the step.
Remote destinations are what the secret channel was for. The wizard payload is
base64'd into a task's command string and tasks are recorded world-readable, so
a password is POSTed to /api/setup/secret, which writes it where only the
manager can read it and returns an opaque reference; the reference travels in
the payload and setup_apply redeems it once, at the write. A reference that
cannot be redeemed leaves the password alone and says so, rather than blanking
it.
Verified in the browser on a clean install: the step renders both modes, lists
the existing destination at its resolved path, and the add dialog swaps between
local and remote fields. scripts/dev/lp-backup-setup-test covers the apply side,
including that what reaches the config is the secret and never the reference.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The WebUI config editor renders these inline comments as field help.
Several had grown into paragraphs (updater, rootless networking, MTU,
install mode, backup engine); cut them back to a single line while
keeping option lists and **ADVANCED**/**DEV** markers intact.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Follow-up to the backup-refresh throttle/dedupe, cutting the cost of the
remote pulls that do still happen.
* restic stats now runs in --mode raw-data (restic_check.sh). The default
restore-size mode walks every snapshot's tree to sum logical file sizes —
the slowest restic op — just to fill a size readout. raw-data reads the
index only and reports the repository's actual deduplicated on-disk size,
which is exactly what the dashboard already labels "deduplicated,
encrypted". raw-data omits total_file_count, so the per-location card now
shows that location's snapshot count (already loaded client-side, and more
useful for a backup repo) instead of a file count.
* engineLocationStats now shares the same per-refresh memoiser as
engineSnapshotsJson (engine_dispatch.sh). Both the locations and dashboard
generators call it per location, so repo stats went from two restic calls
per location per refresh to one. Factored the cache into _engineCachedPull.
* SSH connection reuse for SFTP locations (backup_ssh.sh): ControlMaster=auto
with a self-reaping ControlPersist master, so the several restic
subprocesses a refresh/backup spawns against one location share a single
authenticated connection instead of a fresh handshake each — the dominant
per-call cost on a high-latency link. Toggle via CFG_BACKUP_SSH_MULTIPLEX.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The "Refreshing backup data..." step on every WebUI update fired one
`restic stats` (restore-size mode — the slowest restic op) plus ~4
identical unfiltered `restic snapshots --json` pulls per enabled
location (dashboard, snapshots, app-status, migrate each pulled their
own), all over a fresh SSH connection for remote repos, on every pass
with no throttle — the slow, "frozen"-looking line users hit on poor
links.
Two fixes:
1. Dedupe. engineSnapshotsJson transparently memoises the first
unfiltered whole-repo pull per location to a shared cache dir
(LP_SNAP_CACHE_DIR, set by webui_updater around the chain), so the
four generators reuse one restic call instead of four. Filtered and
failed/empty pulls always fall through to a live call.
2. Throttle. Backups and location changes already regenerate this data
live when they happen, so the routine pass is only a drift catch-up.
Split the cheap local-only generators (engines/schema/passwords —
no remote I/O) out to always run, and gate the remote pull behind
CFG_BACKUP_DASHBOARD_REFRESH_INTERVAL (minutes, default 30, 0 =
every update). A completed backup touches a dirty marker that forces
the next pass to pull; WEBUI_UPDATER_FORCE still forces a full
refresh.
Net: N locations x 5 remote restic calls every update -> 1 call per
location, only when something actually changed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Config sections (Basic, Mail, DNS, Firewall, …) rendered as a bare name. Add a
small data-driven icon before each: config files now carry a `# @icon <emoji>`
directive in their header, the config generator surfaces it as the subcategory's
`icon`, and the config editor prepends it to the section <h3> (covers every
render path — regular/master/toggle/domains/catalogs/whitelist — since they all
drop displaySubcategory straight into the heading).
- webui_generate_configs.sh: parse `# @icon` (first 10 lines) → emit "icon".
- config-manager.js: prepend subcategoryData.icon to the section title.
- configs/**: added an @icon to all 23 base config files (backup/general/network/
security/webui). The live install's config files get the same directive + a
configs.json regen so existing installs show the icons too.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Introduce scripts/source/paths.sh as the canonical path resolver for three
independently-relocatable roots:
LP_SYSTEM_DIR manager-owned control plane (configs/logs/install/db/ssl/ssh/migrate)
LP_CONTAINERS_DIR container-user-owned live app data
LP_BACKUPS_DIR container-user-owned backup repos (own mount-able)
Roots come from the environment when set (install bakes them; CLI/app inherit
from init.sh), else default to /libreportal-*. A transitional compat default
keeps EXISTING installs (legacy single /docker tree, by config marker) on /docker
until a deliberate reinstall, so deploying this never strands a running box.
- init.sh derives the same vars inline (self-contained for the bare /root/init.sh
reinstall case); paths.sh mirrors it for the standalone task/check processors,
which now self-locate their scripts dir and source it.
- Replace functional /docker literals with the derived vars across runtime,
install, backup, crontab, crowdsec/restic, headscale, and reinstall paths;
clean the inert '== /docker/containers/*' guard fallbacks to the variable form.
- backend: CONTAINERS_DIR now from LP_CONTAINERS_DIR (compose env, filled at
generation via a new CONTAINERS_DIR_TAG), legacy-safe default for un-recreated
containers.
- backup default path falls back to the backups root; exclude paths.sh from the
sourced-file arrays (bootstrap file, sourced explicitly).
The CLI-wrapper heredoc + root helpers still reference /docker; those get baked
in phase 3. No layout/ownership change yet (phase 2).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Expose the existing location_ssh.sh key store through the backup CLI:
'backup location ssh-key-set|ssh-key-generate|ssh-key-public|ssh-key-delete <idx>'
(the WebUI runs these as tasks). The locations generator now emits
ssh_key_exists + ssh_public_key (public key only — the private key never
leaves the per-location ssh.key file), so the editor can show the key state.
Also fix the stale SSH_AUTH label (~/.ssh/id_rsa -> managed per-location key).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Adds a logical-dump path so apps with a database can be backed up with zero
downtime and full consistency, instead of stopping the container.
- backup_db.sh: dump each declared DB live (mysqldump --single-transaction /
pg_dump / sqlite3 .backup), exclude the raw data dir from the snapshot, and
replay the dump on restore (pre-start rehydrate for sqlite, post-start load
for server engines).
- Databases are declared via a 'libreportal.backup.db' compose label so the
metadata travels with the app in the snapshot.
- New 'auto' strategy (now the default): live where a DB is dumpable or the app
is marked live-safe, stop-snapshot-start otherwise. Explicit stop/pause/live
remain as overrides.
- restic/borg/kopia adapters honour an exclude list on the live path.
- Manifest records the resolved per-app strategy and dumped databases.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Automatic path mode hardcoded /docker/backups/<id>, baked into the Path Mode
dropdown label. Add a CFG_BACKUP_DEFAULT_PATH option in the Backup Engine
config ("Default Backup Location", default /docker/backups) and have
backupLocationResolvedPath build the auto path from it (<base>/<id>, trailing
slash tolerated). Defaults to the old path, so existing auto locations are
unchanged.
Path Mode's option is now just "Automatic" (no inline path); its tooltip
points at the Default Backup Location config option instead.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The backup engine is an implementation detail — LibrePortal picks a sensible
default and handles it — so it doesn't belong next to Name/Type on the
Connection tab. Add ENGINE to LOC_ADVANCED_SUFFIXES and mark it **ADVANCED**
in the location.config template + seed so it's metadata-driven.
Since the engine select now lives in the Advanced tab while SSH-auth and
path-mode stay on Connection, refreshInlineTypeFields re-applies the dynamic
behaviors (engine filtering, SSH/path visibility) against the shared
.task-details scope rather than a single panel.
Also fixed the live per-location engine label (restic -> Restic) which now
surfaces in the dropdown via the generator-emitted options.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The config generator only scanned flat per-category files, so the dynamic
CFG_BACKUP_LOC_N_* keys carried no titles/descriptions/options — the Locations
editor had to hardcode that metadata in backup-page.js. Add a pass that
descends into configs/backup/locations/<n>/location.config and emits each key
(value/title/description/options) into the config map, plus an "advanced"
flag parsed from a **ADVANCED** token in the field comment (stripped from the
user-facing description).
These keys use subcategory "backup_locations", which isn't in any category's
subcategory_order, so the generic /config page ignores them — only the custom
Locations editor consumes them. URI, SSH port, and append-only are marked
advanced. Verified: configs.json stays valid JSON and /config subcategories
are unchanged.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
- Display the restic engine as "Restic" to match BorgBackup/Kopia. The
lowercase name lived in scripts/backup/engines/restic.json (drives the
location-row engine pill, per-location engine select, and engine modal),
the hardcoded per-location dropdown options, the engine-list fallback, and
the config-option metadata. All set to "Restic".
- In each location's Engine dropdown, float the system-default engine
(CFG_BACKUP_ENGINE) to the top and tag it "(default)", mirroring the
retention-preset pattern.
Repo config metadata is the install template (add-only reconciliation), so
the live /docker/configs/backup/backup_engine label was updated in place too
for the global Configuration-tab dropdown on this install.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Drive the section title from the filename instead of the JS display-override
added earlier: rename configs/backup/backup_advanced -> backup_engine, update
the category SUBCATEGORY_ORDER and the file's header comment, and revert the
formatSubcategoryName override. The CFG_BACKUP_* keys are unchanged.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Application backups were driven by one crontab entry per app, each offset by
id * CFG_BACKUP_CRONTAB_APP_INTERVAL minutes. That minute offset is written
straight into cron's 0-59 minute field, so past ~20 apps it overflowed into
an invalid entry that silently never fired, and the fixed spacing could not
serialize backups that ran longer than the gap.
Replace it with a single daily entry (`libreportal backup scheduled`) that
enqueues a backup task per enabled app. The existing systemd task processor
drains them serially — no minute overflow, real serialization, and backups
are now visible/cancellable in the Tasks UI. Per-app enable is read from
CFG_<APP>_BACKUP at schedule time instead of being mirrored into crontab.
Removes the stagger machinery (timing/setup/check/remove scripts), the
now-unused cron_jobs table + insert, and the CFG_BACKUP_CRONTAB_APP_INTERVAL
config knob and its WebUI field.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
A free, open, self-hosted app platform (GNU AGPLv3): one-click app deploys,
Traefik reverse proxy with automatic SSL, rootless Docker support, gluetun
VPN routing, and a web dashboard to manage it all.
Free & open forever to self-host; optional paid hosted services fund it.
See PROMISE.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
