Phase 3 of docs/roadmap/storage-locations.md.
The step appears only when the candidate scan finds a filesystem
LibrePortal isn't already using, so the single-disk case — which is most
boxes — is completely unchanged. It sits before Recommended because a
location has to exist before an app can be placed on it.
Supporting two conditional steps meant the wizard could no longer treat
'position in the DOM' and 'step index' as the same number: Metrics was
advanced-only and got away with 'length minus one', but a step hidden in
the MIDDLE leaves a gap. Navigation, progress, validation and submit now
all run off _visibleSteps(), and section matching is by data-step rather
than DOM position.
Unusable candidates render greyed WITH the reason rather than being
filtered out — 'why isn't my drive listed?' is a support burden, and
'exFAT can't store file ownership' is actionable. The step is skipped
only when nothing usable was found at all.
What the wizard sends is a request, not an instruction: setupApplyConfig
feeds each path through storageAdd, so the fitness checks and the root
helper's admission rules both re-run regardless of what arrived in the
payload.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An offline trivy install crash-looped (server FATALs when it can't fetch the
vuln DB), and on rootless docker the restart storm churned the shared network's
port-forwarder until the WebUI's own published host port was torn down — the
WebUI stayed healthy INSIDE its container but was unreachable from the host, with
nothing detecting or healing it.
Three fixes, in the house self-healing style (mirrors the network-drift trio):
1. Control-plane health checker wired into the existing task-processor idle poll
(maybeRegenPoll), no new daemon. dockerHealthScan (read-only) detects daemon
down, a WebUI running-but-host-port-unreachable (the port-forward corruption),
and crash-looping containers. webuiSystemHealthCheck writes
frontend/data/system/health_status.json + self-dispatches a heal — the user
can't click a button on a dead WebUI, so the poll drives the fix. Frontend
health-notifier surfaces a topbar badge + dashboard banner + details panel.
2. Failure cap, enforced centrally by dockerHealthHeal (task-gated): stops
crash-loopers (removing the churn), restarts the WebUI to re-publish a lost
port forward, and — only if that fails — recycles the rootless daemon and
restarts the core container. Caps every app immediately, no template churn.
3. Trivy no longer crash-loops offline: the server runs in a shell retry-loop so
the container stays Up and quietly retries on a backoff instead of exiting
FATAL. Verified: container stays Up across repeated DB-download failures.
Core WebUI compose gains restart: unless-stopped so it self-recovers after a
reboot / daemon recycle instead of staying down.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
webuiRegistryCatalogScan (run by updater check, same atomic keep-prior
pattern as webuiArtifactScan) writes apps/generated/registry_catalog.json:
the type:"app"/kind:"bundle" rows of the signed index annotated with
defined/installed, browse metadata from the envelope meta, and icons
mirrored into core/icons/apps/registry/ ONLY when their bytes match the
sha256 pin in the signed index — the browser stays same-origin; a tampered
or oversized icon is skipped, never served.
webuiArtifactScan now selects type=="hotfix" so app rows never render as
pseudo-hotfixes in the Improvements tab, and counts+logs artifacts of
unrecognized type instead of surfacing them (the §8.1 forward-compat
firewall on the scan path).
Harness vs a locally served registry: 14/14 (catalog row + meta + flags,
icon pin verify + tamper skip, hotfix-only stream, unknown-type skip+log,
unreachable-registry keeps prior files).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Closes the gap behind the vpn-recreate bug: when the shared network is
recreated with a different /24, every app's stored static IP is left
outside it and adoptDockerSubnet only realigns CFG, not the apps.
- networkScanConflicts (network_conflicts.sh): read-only scan diffing each
active network_resources IP against docker's real subnet (via ipInSubnet).
Per-service routing-aware — skips gateway-routed services whose ipv4 is
commented out in the deployed compose, so gluetun apps don't false-positive.
Distinguishes 'daemon down' (benign) from 'network missing' (real).
- webuiSystemNetworkCheck (webui_system_network.sh): self-throttled generator
that writes frontend/data/system/network_status.json (modelled on
verify_status.json). Wired into webuiSystemUpdate AND run unconditionally
every ~60s from the task-processor poll (regen webui is mtime-gated and
would never fire on drift, which touches no source file).
- networkHealConflicts (network_heal.sh) + 'libreportal system network
check|heal [app]': the heal adopts docker's subnet in-process, then re-IPs
stranded apps with reset_network=ip (ports preserved), gluetun first.
Mutating path runs only through the task system (dual-mode, like update
apply); read-only check runs inline.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- CFG_HOTFIX_AUTO (security-breakage|all|off, default security-breakage) seeded in
general_terminal; reaches existing installs via the add-only config reconciler.
- webui_artifact_scan.sh (webuiArtifactScan): fetch+verify the signed index, write
artifacts_available.json ATOMICALLY (build in temp → jq-validate → one write;
keep the prior file on any failure — never emits broken JSON). Annotates each
artifact with applied (a per-id record exists) + applicable (target installed).
- artifactApplyAuto + `libreportal artifact apply-auto`: enqueue apply tasks for
the eligible signed hotfixes — only when the index is VERIFIED-signed, only
auto==true + in the severity policy + applicable + not already applied. Each
apply is its own task (visible in the log + History), never applied inline.
- `updater check` now also refreshes the index (webuiArtifactScan) and runs
artifactApplyAuto — one front door, no second phone-home.
Unit-tested 13/13: policy filtering (security-breakage / off / all), auto:false
exclusion, already-applied skip, non-installed-app skip, unsigned-index fail-closed,
and the scan transform's signed/applied/applicable fields.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The task processor is a systemd-service daemon, not a cron job — move it out
of the misleadingly-named scripts/crontab/task/ to scripts/task/.
To stop the systemd unit from baking the processor's in-tree path (the footprint
coupling that forces a reinstall on every reorg), the unit now ExecStarts the
stable wrapper: /usr/local/bin/libreportal __task-processor. start.sh intercepts
that early (after paths.sh, before the heavy load), exports install_scripts_dir,
and exec's the processor with start_script. Future moves/renames need only the
one hand-off updated + a regen — no footprint bump.
- git mv scripts/crontab/task -> scripts/task (filenames kept; cron-watchdog grep
+ function names unchanged)
- libreportal-svc: ExecStart -> stable wrapper launcher
- start.sh: __task-processor internal launcher (export install_scripts_dir; exec)
- crontab_task_processor.sh: fix self-location ../.. -> .. for the new 1-level
depth (latent bug the move would otherwise have introduced)
- regen files_*/function_manifest; add task_scripts to the app/cli aggregates
- footprint_version 3 -> 4 (root-owned svc unit changed -> needs a root reinstall)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
A peer is a named reference to another LibrePortal instance. Phase 2 only
implements kind=backup-channel (friendly label over a hostname that shows
up in a shared backup repo); direct-ssh-direct and direct-ssh-via-relay
(Connect's blind-relay) are reserved enum values for Phase 3.
DB schema (db_create_tables.sh):
CREATE TABLE peers (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT UNIQUE NOT NULL,
kind TEXT NOT NULL DEFAULT 'backup-channel',
config_json TEXT NOT NULL DEFAULT '{}',
status TEXT DEFAULT 'unknown',
last_seen TEXT,
created_at TEXT DEFAULT CURRENT_TIMESTAMP
);
+ indexes on name and kind.
config_json is kind-specific so new transports don't need a schema
migration. For backup-channel it carries {"hostname":"","loc_idx":N}.
Bash module (scripts/peer/):
peer_helpers.sh _peerDb, peerSqlEscape, peerValidateName/Kind.
peer_add.sh peerAdd <name> <kind> [k=v ...] → INSERT, refresh
generator. Rejects unimplemented kinds early so users
don't create dead-end peer records.
peer_remove.sh peerRemove <name> → DELETE.
peer_list.sh peerList → JSON array; peerGet, peerNameForHostname
(reverse-lookup for the migrate-tab overlay).
peer_check.sh peerCheckReachable, peerCheckAll. For backup-channel
'reachable' = at least one snapshot from that hostname
visible in (preferred|any enabled) location. Updates
status + last_seen so UI dots render without re-probing.
CLI (scripts/cli/commands/peer/):
libreportal peer list
libreportal peer get <name>
libreportal peer add <name> backup-channel hostname=<host> [loc_idx=<n>]
libreportal peer remove <name>
libreportal peer check [name]
Auto-routed by cli_initialize.sh's category-discovery.
WebUI data generator (scripts/webui/data/generators/peers/webui_peers.sh):
Emits data/peers/generated/peers.json with the peerList output and a
generated_at envelope. Hooked into webuiLibrePortalUpdate alongside the
backup generators.
Frontend:
- New top-level /peers route in spa.js (PeersPage class, peers-content.html).
- 'Peers' nav item in the topbar between Backups and the right-side controls.
- Add-peer modal with friendly-name + kind + hostname + preferred-location
selector (populated from the existing backup-locations data).
- Per-peer card with status dot, last-checked time, Check + Remove buttons.
- Phase 3 kinds appear in the kind dropdown as disabled options so users
can see what's coming.
Source-array wiring:
- generate_arrays.sh auto-created files_peer.sh from the new peer/ dir.
- cli_files.sh + app_files.sh include ${peer_scripts[@]} alphabetically.
- files_webui.sh auto-picked-up the new peers/ generator subfolder.
The migrate-tab friendly-name overlay (use peer names in /backup/migrate
when a peer record exists for a hostname) is intentionally deferred — it's
a 5-line frontend lookup once peers.json is loaded; cleaner to add after
Phase 3 ships its peer-detail view.
Signed-off-by: librelad <librelad@digitalangels.vip>
Phase 1 of the migration-system refresh. Surfaces Phase 0's kernel
(libreportal restore migrate ...) as a WebUI flow so users don't have
to drop to the CLI to pull an app from a peer's backups.
backend / data generator:
scripts/webui/data/generators/backup/webui_backup_migrate.sh
Walks every enabled backup location, lists every (other_host, app)
pair with snapshot count + latest id/date, and emits a single
destination summary block (installed apps, running apps, disk free)
so the frontend can compute collisions and warnings without per-row
API round-trips. Filters out our own hostname — we don't migrate to
ourselves. Output: data/backup/generated/migrate.json.
Hooked into the standard webuiLibrePortalUpdate refresh pipeline,
so 'libreportal regen webui' (and the periodic task-processor poll)
keep it fresh on their own.
frontend:
- New 'Migrate' sidebar tab on /backup, sits between Locations and
Configuration. Path-based URL: /backup/migrate.
- Per-source-host cards listing every available app, with snapshot
count + relative-time hint, collision dot when the app is already
installed here, and per-app + per-host migrate buttons.
- Confirm modal with two checkboxes matching the kernel's defaults:
[✓] Back up the destination's existing copy first (pre-migrate
backup; auto-disabled when there's nothing to back up)
[✓] Rewrite host-bound URLs to this host (URL rewrite
— uncheck only to keep source hostnames)
On confirm, runs 'libreportal restore migrate app/system …' via the
task system; opt-out checkboxes append --no-pre-backup / --keep-urls
only when the user un-ticks, matching the kernel's default-on flags.
- Empty state when no other hosts have visible backups, explaining
the shared-backup-location prerequisite.
The CLI dispatcher hooks (Phase 0) wire restore migrate app/system to
migrateApplyApp/migrateApplySystem, so the WebUI gets pre-backup safety,
URL rewrite, and structured progress (when --json-progress is set; not
needed here yet — the task system's log tail is enough for v1).
Signed-off-by: librelad <librelad@digitalangels.vip>
Move scripts/webui/data/generators/apps/webui_gluetun_providers.sh ->
containers/gluetun/scripts/gluetun_providers.sh and replace the gluetun-specific
gated call in webui_updater.sh with a generic per-app loop: an installed app may
define appWebuiRefresh_<app> (in its scripts/) for data it wants refreshed on
every WebUI update. gluetun provides appWebuiRefresh_gluetun (a thin wrapper over
webuiGenerateGluetunProviders).
- No gluetun-specific code remains in central WebUI code — it's a true drop-in.
- Install gate preserved + generalized: the loop iterates the manager-owned
install templates (listable) and tests each app's live compose directly (works
without list perm on the container-user data dir), so non-users never pay for it.
- webuiGenerateGluetunProviders keeps its name (still called by the installer and
the gluetun_refresh_providers tool); now sourced via the container scan.
- Regenerate arrays (generator drops out of files_webui).
Loop verified with stubs: only installed apps with a defined hook fire; apps
without a hook are skipped; nothing fires when nothing's installed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Add `lpRegen` (scripts/webui/webui_regen.sh) — one entry point that rebuilds the
file-derived artifacts whose sources changed, so callers don't have to know which
generator owns what. Self-heal is a cheap `find -newer` mtime compare (no watcher
/ daemon): a stage runs only when a source is newer than its artifact, or --force.
- `libreportal regen [all|webui|arrays] [--force]` CLI command (new category).
- Task processor idle tick runs a throttled `regen webui` poll, so an app dropped
in out-of-band (drag-drop / marketplace) appears on its own — no manual command,
no inotify (works on the relocatable/external-drive roots where inotify can't).
- make_release.sh guards against shipping stale source arrays (regenerate; abort
if the committed tree was out of date), killing the "forgot generate_arrays" bug
class at the build boundary.
- Document the front door in DEVELOPMENT.md.
webui scope rebuilds from containers/<app>/{*.config,tools/*.tools.json}; arrays
scope from scripts/** (a dev/build concern — a no-op on a normal install). Gate
logic verified in a sandbox (clean/config-newer/tools-newer/force/missing).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Add webui_system_metrics.sh, run each minute from webuiSystemUpdate:
- whole-server snapshot (metrics.json): CPU% + load, memory + swap,
per-mount disk + inodes, network rx/tx rate, docker summary
- capped ring buffer (metrics_history.json, 24h default) for trend charts
- per-app docker stats grouped by compose project (metrics_apps.json)
plus a short per-app history (metrics_apps_history.json) for sparklines
CPU% and network rate use stateful deltas stashed beside the JSON; all
host metrics read from /proc and docker via runFileOp, so it works rootless.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Fresh, on-demand inbound SSH-access management for the host (replaces the old
maze). scripts/ssh/host_access.sh manages the install user's authorized_keys —
add a pasted public key (validated), list, remove — and toggles sshd password
login behind a lockout guard (won't disable passwords with no key; won't drop
the last key while passwords are off; sshd -t before reload, with backup).
New 'ssh' CLI category (status/key-add/key-remove/password-auth/generate) and
a webuiGenerateSshAccess snapshot (data/ssh/access.json: user, password_auth,
authorized keys as type+fingerprint+comment — public only) wired into the
regen chain. Nothing runs automatically; only explicit admin actions change
anything. WebUI page next.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The old inbound-admin-SSH layer was effectively dead: gated on config flags
that don't exist (CFG_SSHKEY_*_ENABLED, CFG_REQUIREMENT_SSHREMOTE), its
authorized_keys installer was unwired, and its download path (sshdownload
container) was already retired. What remained reachable was either a no-op or
a lockout footgun (disable-passwords with no working key install).
Remove it whole: scripts/ssh/*, the four SSH requirement checks, the SSH tools
menu, the dead webui SSH populater, and the unused ssh DB inserts; drop their
calls from the start/requirements/menu flows. A fresh, WebUI-driven admin SSH
access feature replaces it next.
Also make generate_arrays.sh self-healing: prune files_*.sh whose source
folder no longer exists (cleared the now-stale files_ssh.sh + an orphan
files_api.sh) so removed areas don't linger in the sourced set.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
New script files are sourced from the committed files_*.sh arrays (built by
generate_arrays.sh), not a live tree scan — and quick deploys don't rerun
generate_arrays. So the schema generator added last commit was never loaded
live: webuiGenerateBackupSchema was undefined, breaking the webui_updater
backup chain at that step (skipping the passwords regen after it) and leaving
schema.json un-generated.
Regenerate the arrays so the file is registered; deploy now sources it and
'webui generate all' rebuilds schema.json on its own.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
A free, open, self-hosted app platform (GNU AGPLv3): one-click app deploys,
Traefik reverse proxy with automatic SSL, rootless Docker support, gluetun
VPN routing, and a web dashboard to manage it all.
Free & open forever to self-host; optional paid hosted services fund it.
See PROMISE.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
