87 Commits

Author SHA1 Message Date
librelad
751a4578d6 webui: trim the field tooltips added for the slot rename
Matches b562059 — the fourteen mapping entries added alongside the slot rename
were written before that landed and ran 88-100 chars against a median of 44.
2026-08-18 19:53:06 +01:00
librelad
a1541ace23 refactor(secrets): slot-number every generated config key
Makes the convention uniform: if a config key holds a generated value, its name
ends in a slot number. 42 keys across the catalog, up from the 9 database ones
done previously — admin passwords, app keys, tokens, HMAC and auth secrets,
generated usernames and database names. An app needing a second credential of a
kind now just adds _2; nothing is registered anywhere, since the tag name is
derived from the key by tags_processor_app_config_values.

Keys holding an operator-chosen value (CFG_NEXTCLOUD_ADMIN_USER=admin) keep their
names — the slot number is what marks a value as generated.

The rename would have silently cost seven keys their WebUI field mapping. The
frontend resolver matches a mapping key against a config key by equality, _suffix
or prefix_ (apps-manager.js findMatchingCFGKey), so the generic "ADMIN_PASSWORD"
entry stops matching CFG_GITEA_ADMIN_PASSWORD_1 — it neither ends with
_ADMIN_PASSWORD nor starts with ADMIN_PASSWORD_. Rather than loosen the matcher
(PORT_1 relies on its numeric suffix being part of the name), add explicit
entries. Did the same for eight keys that were already unmapped before this
change, so all 42 now render with a label and, where appropriate, masked: the
only one typed as text is Mastodon's VAPID public key, which is public by design.

Verified by simulating the resolver against every app config, and by running each
app in the catalog through fill -> hook -> templating: every secret tag
substitutes, no RANDOMIZED placeholder survives, every compose still parses.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 19:51:27 +01:00
librelad
65167463f9 fix(chat apps): tag every service so its IP actually substitutes
Installing rocketchat failed with

    invalid IPv4 address: ParseAddr("IP_DATA_2"): unable to parse IP

ipUpdateComposeTags allocates one IP per SERVICE_TAG_N annotation and fills
IP_TAG_i only where SERVICE_TAG_i exists. The four new apps tagged only their
primary service, so every sidecar — matrix's postgres, mattermost's postgres,
rocketchat's mongo, and fifteen of stoat's sixteen — kept a literal IP_DATA_n
in the deployed compose and docker refused to create the container.

Tag every service that carries an ipv4_address, index-aligned with its IP_TAG.
For stoat that also meant moving caddy from SERVICE_TAG_1 to _6 so the indices
line up with the IPs rather than the reading order.

mastodon had the same latent break (IP_TAG_2 and _3 untagged) and is fixed the
same way — it would have failed on first install for the same reason.

SERVICE_TAG carries the compose *key*, not container_name: 'libreportal app
restart <app> <service>' passes it to 'docker compose restart', which only
understands keys.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 19:48:09 +01:00
librelad
b562059251 webui: trim overlong field tooltips to one-liners
Cut the tooltips that had grown into paragraphs (backup strategy,
version, monitoring, DB/secret fields, Dashy shortcuts) down to a
single line, matching the concise style of the rest of the file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 19:43:25 +01:00
librelad
4685320353 feat(secrets): real VAPID keypair for mastodon, slot-numbered DB passwords
VAPID: the two values are the halves of one P-256 keypair, not independent
secrets — the browser verifies that a push is signed by the private key matching
the public key it subscribed with. The RANDOMIZED* generators mint each
placeholder on its own, so they produced two unrelated strings and web push could
never have worked. Generate the pair in mastodon_install_post_setup the way stoat
already does, encoded as Mastodon's webpush gem expects: unpadded URL-safe base64
of the 32-byte private scalar and the 65-byte uncompressed public point, sliced
out of the SEC1 DER. Verified by rebuilding the key from the emitted private half
and re-deriving the public point — openssl accepts it and the point matches.

Generated once and never rotated (rotation would invalidate every subscription),
but a pair of the wrong shape is replaced, so an install carrying the old
unrelated strings heals itself on next install — their public half is 42 chars
where a real point is 87.

Slots: CFG_<APP>_DB_PASSWORD -> CFG_<APP>_DB_PASSWORD_1 and likewise for
DB_ROOT_PASSWORD, across mastodon, owncloud, mattermost, matrix, nextcloud and
bookstack, so a database credential is always a numbered slot and a second one is
just _2. Renaming a key means reconciliation drops the old and adds the new
holding its placeholder, so an existing install regenerates unless the value is
carried over first — documented, including that the old file survives as
.<app>.config.bak.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 19:42:50 +01:00
librelad
5706498565 fix(secrets): move app credentials into <app>.config, fix slot collision
Five apps (mastodon, owncloud, mattermost, matrix, stoat) took their generated
secrets from the compose-side generator tags PASSWORD_TAG_<n>/RANDOM_TAG_<n>/
HEX_TAG_<n>/VAPID_TAG_<n>. Those mint a fresh secret on every templating run, so
a reinstall handed the app a new database password while its data volume kept the
one initdb was given, and the app came back up unable to open its own database.

Move them to <app>.config as RANDOMIZED* placeholders, reaching the compose via
the #LIBREPORTAL|<APP>_<KEY>_TAG| mechanism tags_processor_app_config_values
already provides. No new handler: the tag name is derived from the config key, so
this is a config line plus a tag per secret. Generation is unchanged — still
random on first install; the value is now remembered instead of re-rolled.

Also fixes two things this exposed:

- The RANDOMIZED* replacers matched unanchored. `sort -u` orders slots lexically
  (1, 10, 11, 2), so slot 1's pattern rewrote the prefix inside slot 10's
  placeholder and slots 10+ ended up holding slot 1's secret with a digit glued
  on — derivable, and invisible because the values weren't byte-identical.
  Anchoring with \b makes match order irrelevant. Verified at 20 slots across
  all four placeholder types: 64 keys, 64 distinct values, no prefix collisions.

- generateRandomPassword drew from base64 without constraining the mix; measured
  over 2000 draws, 1 in 40 contained no digit at all. Retry until the result has
  both a digit and a letter, bounded so a pathological length can't spin.

owncloud gains a fix in passing: its compose seeded the admin account from
PASSWORD_TAG_2 while the WebUI displayed CFG_OWNCLOUD_ADMIN_PASSWORD, which was
generated separately and never used. Both now read the same value.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 19:33:40 +01:00
librelad
1e9e042d41 webui: shorten UPDATE_TYPE tooltip to a one-liner
Match the concise style of the other field tooltips instead of
explaining the whole update/rollback flow.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 05:22:09 +01:00
librelad
42995eb373 fix(webui): prune icons for apps that no longer exist
Removing Focalboard from the catalogue left its icon still being served:
the sync only ever ADDS, so every app ever dropped leaves a file behind
that the portal keeps offering for something that is gone. Same shape as
the task queue that only ever appended.

webuiPruneAppIcons runs at the end of the sync and removes only icons it
can match to a missing template — anything else in the directory is left
alone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 04:14:55 +01:00
librelad
ca266e9391 feat(updater): flag apps whose image upstream has stopped rebuilding
"Up to date" answers one question — has the tag I track moved? — and an
abandoned project answers it reassuringly forever. The tag stays put, the
digest never changes, and the app reports as current while receiving no
security patches at all. Nothing in the UI could tell a healthy stable
app from a dead one.

An audit of all 34 anchor images found five in exactly that state:
speedtest (4.4y since rebuild), focalboard (2.8y — Mattermost dropped
support in 2023), pihole-unbound (2.3y), trilium (2.2y), unbound (1.8y).

The scan now records image_updated_at per app (one cheap Hub call inside
the existing registry window, cached between windows like everything
else) and emits stale_after_days from CFG_UPDATER_STALE_DAYS (365, 0
disables) so the UI and the config agree on one number.

Surfaced as an "unmaintained?" severity chip on the fleet row and a
dated explanation in the app detail. Phrased as an observation rather
than an accusation — plenty of small tools are simply finished — but it
does spell out the security consequence, because that is the part a user
cannot infer from "up to date".

Deliberately NOT a "needs action" row on the Overview board: it is not
fixable by pressing anything, and a permanently amber board teaches
people to ignore the board.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-13 01:09:03 +01:00
librelad
2c589b2a51 fix(updater): treat docker.io/ as Docker Hub, not a third-party registry
The registry helpers rejected any repo containing a dotted host segment,
which caught 'docker.io/authelia/authelia' — Docker Hub spelled out in
full. Those apps were silently skipped by tag enumeration and version
laddering. Strip the docker.io/ and index.docker.io/ prefixes before the
host check; genuinely third-party registries (ghcr.io, quay.io, lscr.io)
are still correctly skipped.

Found by auditing every app's anchor image.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-13 00:55:04 +01:00
librelad
7334557706 feat(updater): detect newer release lines, not just newer builds
The digest compare only ever asks about the tag already pinned, so it
answers "has my tag been rebuilt?" and can never answer "does a newer
version exist?". An app on v0.16 reports up to date forever while 0.17
ships. That is the gap between an app that updates and an app that is
current, and it silently affects every pinned app.

Adds tag enumeration for VERSIONED tags only (rolling tags already move
on their own): list the repo's tags, keep those sharing the current tag's
SHAPE, and pick the numerically greatest.

Shape matching is the whole safety story — v0.16 -> v#.# so it can never
"upgrade" you onto v0.16-alpine, 31-fpm-alpine onto 31-apache, or a date
tag onto a semver one. Comparison is component-wise numeric, so 0.10 > 0.9
and 1.0 > 0.99 (a string sort gets both wrong), with 10# forcing base ten
so an upstream "08" cannot be read as octal. 15 unit tests cover it.

Docker Hub only, deliberately: all three pinned apps live there, it needs
no auth, and the generic OCI tags/list wants a per-registry token dance.
Other registries stay quiet rather than guess. Throttled inside the
existing registry window and cached between windows so it cannot flicker.

Surfaced as INFORMATION, never an action: no button applies it, because a
version move can carry a data migration. `update_available` and the "up
to date" badge keep their exact meaning; the new state sits beside them
and points at the Version field.

Against the live registry: stalwart v0.16 is current, nextcloud is on
31-fpm-alpine with 34-fpm-alpine out, mastodon on v4.2.0 with v4.6.5 out.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 23:30:08 +01:00
librelad
da97daf5f7 fix(apps): move Application Version to the Advanced tab
It sets the image tag, so a wrong value stops the app starting — that
belongs with the other expert settings, not beside feature toggles.
Tooltip now explains the split it participates in: automatic updates
apply rebuilds OF this version, changing it moves between releases.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 23:19:02 +01:00
librelad
66c79f997e feat(updater): install window, honest Check-now, failed-auto surfacing
Four fixes that make the auto-updater a trustworthy background system:

* CFG_UPDATER_WINDOW (default 06:00-08:00 host time, right after the
  05:00 backup cron; HH:MM-HH:MM wraps midnight, 'always' = any time).
  Gates only the enqueue — scans keep running all day, so the Updates
  page stays current and pending updates visibly wait for the window.
  Malformed values fail closed and are rejected by the WebUI validator.

* "Check now" actually checks: an explicit `updater check` sets
  UPDATER_REGISTRY_FORCE=1. The flag existed but nothing ever set it,
  so the button silently reused the 6h digest cache and could not find
  a build the user knew had shipped. Force also overrides interval 0,
  which now means "manual-only" as documented in the roadmap.

* Registry stamp moved from /tmp to <system>/logs: the task processor
  runs under PrivateTmp, so daemon and CLI each kept a separate 6h
  clock and the daemon's reset on every service restart.

* A failed automatic attempt is no longer invisible: the scan emits
  auto_attempted_digest (the one-shot no-retry stamp), and when it
  matches the available build the UI stops promising an install that
  will never come — per-app detail explains, the fleet row gets an
  "auto failed" chip, and the Overview board counts it as needing you.

Also corrects the CFG_TIMEZONE label: it sets the containers' TZ only;
scheduled tasks follow the host clock (timedatectl), and the old
"Timezone for scheduled tasks" wording promised a knob that never
existed. The window + auto_window display state plainly WHEN updates
land, answering "how does the user know when the next update happens".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-11 21:06:27 +01:00
librelad
7fae6bc308 fix(updater): stop a callee blanking the app name mid-update
First real end-to-end auto-update on a live install failed like this:

  Automatically updating trivy (a recovery snapshot is taken first)
  Snapshotting trivy before update…
  Pulling new image(s) for …
  Update of  failed — rolling back…
  Could not roll  back automatically

The app name went empty after the snapshot. Cause: bash is dynamically
scoped, so a callee assigning an undeclared variable writes the CALLER's
local of that name — and a `while read app` loop leaves it EMPTY at EOF.
webuiBackupAppStatus's dashboard generator runs at the end of every backup
and did exactly that to updaterApplyApp's `app`.

Nothing was damaged: the pull ran against an empty name, failed before
touching the image, and the rollback was a no-op on a nonexistent app.

Fixed both ends. The generator (and three gluetun loops with the same
latent leak) now declare `local app`. updaterApplyApp/updaterRollbackApp
hold the name in `_upd_app` so they no longer depend on every callee's
hygiene, and updaterApplyAll stops leaking its own loop var.

This is exactly the untested path the roadmap flagged: "apply/revert not
yet exercised end-to-end on a live install with a pending update."

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-11 16:37:14 +01:00
librelad
cdeb2d1658 feat(updater): per-app UPDATE_TYPE, automatic by default
Adds the decision half of the app updater. Detection (P2) and the
snapshot-first apply/revert (P3) were already real, but nothing ever
pressed the button — every update waited for a click.

  CFG_<APP>_UPDATE_TYPE=auto|manual   per app, default auto (33 templates)
  CFG_UPDATER_AUTO=true|false         master switch, default true

updaterAppPolicy resolves the two the way backupResolveStrategy already
resolves backup strategy: the global switch can only make things more
manual. updaterApplyAuto runs at the end of `updater check` and enqueues
the ordinary updater_apply task for each auto app that has an update —
never applies inline, so an automatic update is the same code path, task
log, History entry and Roll back button as a manual one.

Safety: each attempt stamps its target digest under generated/auto/, so a
build that fails is rolled back and then left alone rather than retried on
every scan; in-flight updater tasks are skipped so scans can't stack.

Tracked end to end: updates.json carries each app's resolved update_type,
History entries carry trigger=manual|auto. The WebUI says whether updates
install themselves, chips only the apps that opted out, labels automatic
history, and — since an auto app's pending update needs no decision — keeps
it off the Overview board's "Needs action" view.

Also fixes artifactApplyAuto enqueueing without --detach: it runs inside
the single-threaded task processor's own poll, so following the new task in
the foreground waits for a task that cannot start until it returns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-11 16:22:04 +01:00
librelad
1de4d5d970 feat(updater): auto-check status line on the app Updates tab, drop manual Check
The per-app Updates tab pushed a manual "Check" button (header + empty state)
even though scans run automatically on CFG_UPDATER_SCAN_INTERVAL — so an app with
nothing to update read like an empty/actionable page. Replace the manual Check
with a calm status line inside the panel: "Checked automatically · last checked X
· next check ~Y", backup-schedule style. The genuine Apply/Roll back actions stay
(applying is still manual and safe). No auto-apply.

- webui_updater_scan.sh: stamp scan_interval_minutes alongside generated_at in
  updates.json so the display needs no separate config fetch (0 = auto off).
- updater-page.js: renderAutoCheckLine() + fmtRelFuture().
- app-tabbed-manager.js: drop the header/empty-state Check buttons; render the
  auto-check line; friendlier no-data copy.
- overview.css: style .updater-autocheck (green dot live / muted when off).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 22:38:05 +01:00
librelad
d44a052a3e fix(updater): don't drop up-to-date apps from updates.json
The per-app object build used `available_version:($available_version|select(.!=""))`
to omit the field when empty. But in jq, a `{key: (empty)}` makes the WHOLE
object construction emit nothing — so every app with no available update (empty
available_version) produced no object and was silently dropped from updates.json.
Only apps WITH a pending update survived; an all-up-to-date fleet showed an empty
list. (Missed in P2's sandbox test because both fixture apps had updates.)

Emit an explicit null instead: `(if $available_version=="" then null else … end)`.
Verified: an up-to-date app (trivy, local==registry digest) now emits with
available_version:null; apps with updates still carry the string.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-07-17 23:42:32 +01:00
librelad
5a53a1b4c6 feat(updater): P2 — real registry detection + per-type version display
webuiUpdaterScan now tells the truth instead of hardcoding update_available=false.

Per app it resolves three facts (see roadmap §2): the running RepoDigest (one
`docker inspect` of the anchor image), the display version (OCI label →
versioned tag → channel·shortdigest), and — throttled — the registry's current
index digest for the channel tag (`docker buildx imagetools inspect`, the same
identity as RepoDigest, verified exact). update_available = the two digests
differ. Emits type (versioned|rolling), channel, current/available digests +
versions, and a services[] array (every image line, anchor flagged).

- Registry lookups throttled separately from the scan: CFG_UPDATER_REGISTRY_
  INTERVAL (min, default 360; 0 = local-only), own /tmp stamp, reuse of the
  prior available_digest between windows so the app list still refreshes every
  scan. UPDATER_REGISTRY_FORCE=1 forces a live pull (the Check-now button).
  Registry failure (offline/rate-limited) = "unknown", never a false "changed".
- Digest-compare fully detects rolling apps' new builds; for versioned apps it
  catches rebuilds of the pinned tag (newer-version enumeration is a later
  step) — honest per type, and versioned apps are user-picked via P1b anyway.
- Fixes a P1b regression: updaterPrimaryImage now strips the trailing version
  sentinel comment (`s/ #.*//`) via _updaterCleanImageRef — without this the
  anchor ref (and the live CVE scanner's image arg) carried the comment.
- JSON built with jq for safe escaping; jq-less fallback keeps output valid.

Verified via a simulated 2-app install: navidrome → "0.62.0" (OCI label) +
detected update; rolling app → channel·shortdigest + update; throttled re-run
reuses the prior digest; all output valid JSON.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-07-17 23:30:28 +01:00
librelad
25b496df06 ux(health): rename "Repair Control Plane" → "Fix System Issues" (plainer copy)
"Control plane" is jargon. Rename the self-heal task and de-jargon all the
user-facing copy: task titles ("LibrePortal - Fix System Issues" / "System
Health Check"), the action label, the badge/banner/panel text and "Fix now"
button, the status summaries, and the heal task-log messages. Behaviour
unchanged; code comments keep the technical term where accurate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-07-17 23:11:34 +01:00
librelad
22d7bcf505 fix(updater): anchor app version to <slug>-service, not first image; revise design
The scan read the app's "current image" via `grep -m1 image:` — the first
image line. For apps that declare a companion service first this mislabels the
app: ollama lists `ollama-webui` (open-webui) before `ollama-service`
(ollama/ollama), so an app named ollama reported open-webui's version.

Add `updaterPrimaryImage`: resolve the version anchor from the app's
`<slug>-service` image (the universal primary-service naming convention,
33/33 apps; underscores→hyphens for slugs like libreportal_catalog). Falls back
to the first image line off-convention. Used by both the version and CVE loops.
Verified: only ollama changes (→ ollama/ollama:latest); nextcloud, mastodon,
jitsi, gitea, vaultwarden anchors unchanged.

Also revise docs/roadmap/app-version-updater-and-cve.md to the config-first
direction agreed this session:
- CFG_<APP>_VERSION ADOPTED (was rejected): the #LIBREPORTAL tagging system
  makes the config the source (compose tag derived from it), not a second one.
- Two version TYPES — versioned (real tag, version picker) vs rolling (floating
  channel, digest-freshness); digest is the uniform detection engine for both,
  version numbers are display enrichment. Answers "why not just compare numbers"
  (most upstreams publish none; no universal latest-version API).
- Multi-service anchor = <slug>-service, not first line; lock-step sets (jitsi)
  = one channel → several image lines; sidecars tracked-by-digest, not headline.
- Phases updated: P0 anchor (done) → P1 config-first pin → P2 detection+display.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-07-17 22:57:12 +01:00
librelad
24fbc36518 fix(health): robust crash-loop detection (RestartCount delta) + task display
Crash-loop detection keyed on `docker ps --filter status=restarting`, but a
backed-off loop sits "exited" between restarts once docker's backoff grows to
tens of seconds — so a slowed loop is missed. Detect via RestartCount CLIMBING
between scans (what a crash loop actually is), unioned with the instantaneous
restarting signal for fast loops. Baseline counts persist in
.health_restart_counts, written only by the throttled check so the heal's
re-scans don't disturb the delta.

Also give the system_health_heal / system_network_heal tasks proper display
(they fell through to the raw command + generic ⚙️): friendly titles in
formatCommandForUser, type icons (🩺 / 🌐) in getTaskTypeIcon, and action
labels in formatActionTitle — so they read as "LibrePortal - Repair Control
Plane" etc. with an icon, like install tasks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-07-17 22:53:54 +01:00
librelad
2d1e4aa98f feat(health): self-healing control-plane watchdog + crash-loop failure cap
An offline trivy install crash-looped (server FATALs when it can't fetch the
vuln DB), and on rootless docker the restart storm churned the shared network's
port-forwarder until the WebUI's own published host port was torn down — the
WebUI stayed healthy INSIDE its container but was unreachable from the host, with
nothing detecting or healing it.

Three fixes, in the house self-healing style (mirrors the network-drift trio):

1. Control-plane health checker wired into the existing task-processor idle poll
   (maybeRegenPoll), no new daemon. dockerHealthScan (read-only) detects daemon
   down, a WebUI running-but-host-port-unreachable (the port-forward corruption),
   and crash-looping containers. webuiSystemHealthCheck writes
   frontend/data/system/health_status.json + self-dispatches a heal — the user
   can't click a button on a dead WebUI, so the poll drives the fix. Frontend
   health-notifier surfaces a topbar badge + dashboard banner + details panel.

2. Failure cap, enforced centrally by dockerHealthHeal (task-gated): stops
   crash-loopers (removing the churn), restarts the WebUI to re-publish a lost
   port forward, and — only if that fails — recycles the rootless daemon and
   restarts the core container. Caps every app immediately, no template churn.

3. Trivy no longer crash-loops offline: the server runs in a shell retry-loop so
   the container stays Up and quietly retries on a backoff instead of exiting
   FATAL. Verified: container stays Up across repeated DB-download failures.
   Core WebUI compose gains restart: unless-stopped so it self-recovers after a
   reboot / daemon recycle instead of staying down.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-07-17 22:03:02 +01:00
librelad
71a02374d1 perf(backup): raw-data stats, dedupe repo stats, SSH connection reuse
Follow-up to the backup-refresh throttle/dedupe, cutting the cost of the
remote pulls that do still happen.

* restic stats now runs in --mode raw-data (restic_check.sh). The default
  restore-size mode walks every snapshot's tree to sum logical file sizes —
  the slowest restic op — just to fill a size readout. raw-data reads the
  index only and reports the repository's actual deduplicated on-disk size,
  which is exactly what the dashboard already labels "deduplicated,
  encrypted". raw-data omits total_file_count, so the per-location card now
  shows that location's snapshot count (already loaded client-side, and more
  useful for a backup repo) instead of a file count.

* engineLocationStats now shares the same per-refresh memoiser as
  engineSnapshotsJson (engine_dispatch.sh). Both the locations and dashboard
  generators call it per location, so repo stats went from two restic calls
  per location per refresh to one. Factored the cache into _engineCachedPull.

* SSH connection reuse for SFTP locations (backup_ssh.sh): ControlMaster=auto
  with a self-reaping ControlPersist master, so the several restic
  subprocesses a refresh/backup spawns against one location share a single
  authenticated connection instead of a fresh handshake each — the dominant
  per-call cost on a high-latency link. Toggle via CFG_BACKUP_SSH_MULTIPLEX.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-07-17 21:56:55 +01:00
librelad
a5cd8d625b perf(backup): throttle + dedupe the WebUI backup dashboard refresh
The "Refreshing backup data..." step on every WebUI update fired one
`restic stats` (restore-size mode — the slowest restic op) plus ~4
identical unfiltered `restic snapshots --json` pulls per enabled
location (dashboard, snapshots, app-status, migrate each pulled their
own), all over a fresh SSH connection for remote repos, on every pass
with no throttle — the slow, "frozen"-looking line users hit on poor
links.

Two fixes:

1. Dedupe. engineSnapshotsJson transparently memoises the first
   unfiltered whole-repo pull per location to a shared cache dir
   (LP_SNAP_CACHE_DIR, set by webui_updater around the chain), so the
   four generators reuse one restic call instead of four. Filtered and
   failed/empty pulls always fall through to a live call.

2. Throttle. Backups and location changes already regenerate this data
   live when they happen, so the routine pass is only a drift catch-up.
   Split the cheap local-only generators (engines/schema/passwords —
   no remote I/O) out to always run, and gate the remote pull behind
   CFG_BACKUP_DASHBOARD_REFRESH_INTERVAL (minutes, default 30, 0 =
   every update). A completed backup touches a dirty marker that forces
   the next pass to pull; WEBUI_UPDATER_FORCE still forces a full
   refresh.

Net: N locations x 5 remote restic calls every update -> 1 call per
location, only when something actually changed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-07-17 21:35:24 +01:00
librelad
abd8e0b68b feat(updater): surface Trivy CVE scanner state (installed / DB updating / ready)
Trivy runs as a server whose vulnerability DB downloads on first boot; until it
lands no scan can produce results. Previously the updater generator wrote an
empty-but-valid cves.json the moment the file was missing, so installing Trivy
painted a green "no known vulnerabilities" all-clear that was actually a lie —
the DB hadn't even downloaded, and the Updates/Security view gave no signal.

Add an honest scanner state the WebUI branches on:
- containers/trivy/scripts/trivy_scan.sh — trivyScannerState (absent |
  db_updating | ready) via `trivy version -f json`, trivyDbUpdatedAt, and
  trivyScanImageCves (per-image scan normalized to {id,severity,package,
  installed,fixed_in,url}, deduped). All degrade safely on error.
- webui_updater_scan.sh — stamp cves.json with scanner.state; only run real
  per-image scans once the DB is ready. Always rewritten so state tracks live.
- updater-page.js — Security tab shows a loading box while the DB updates, an
  install nudge when absent, and the genuine 🎉 only when ready+empty; Overview
  CVE card sub + hint reflect the state.
- overview-manager.js — fleet Security row surfaces the "building CVE database"
  pending state instead of silently omitting.
- function_manifest.sh — regenerated for the new trivy_scan.sh functions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 16:47:07 +01:00
librelad
e85fba3b09 auto: session-start commit — 4 file(s) at 2026-07-13 15:49:56 2026-07-13 15:49:56 +01:00
librelad
9b32444475 feat(config): per-file icons before the config-section names
Config sections (Basic, Mail, DNS, Firewall, …) rendered as a bare name. Add a
small data-driven icon before each: config files now carry a `# @icon <emoji>`
directive in their header, the config generator surfaces it as the subcategory's
`icon`, and the config editor prepends it to the section <h3> (covers every
render path — regular/master/toggle/domains/catalogs/whitelist — since they all
drop displaySubcategory straight into the heading).

- webui_generate_configs.sh: parse `# @icon` (first 10 lines) → emit "icon".
- config-manager.js: prepend subcategoryData.icon to the section title.
- configs/**: added an @icon to all 23 base config files (backup/general/network/
  security/webui). The live install's config files get the same directive + a
  configs.json regen so existing installs show the icons too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-07-06 22:33:08 +01:00
librelad
90a20e9e17 feat(catalog): multiple catalog sources — backend model + multi-source scan + CLI
First slice of multi-catalog ("taps") support. Today the App Center browses one
catalog (get.libreportal.org). This adds an ordered list of catalog sources and
teaches the browse scan to merge them.

- New scripts/catalog/catalog_sources.sh — the source list in its OWN file
  ($configs_dir/catalog/sources.json). Source #1 is ALWAYS the official catalog,
  synthesized live from CFG_RELEASE_BASE_URL/CHANNEL (can't be edited/removed,
  always pinned on top). Extra sources are stored as a small JSON array and are
  UNVERIFIED (trust=community). Helpers: catalogSourcesJson / catalogEnabledSources
  (priority order) / catalogSourceAdd|Remove|Toggle|List / catalogFetchCommunityIndex.
- webui_registry_scan.sh now walks catalogEnabledSources: the OFFICIAL source is
  still signature-verified (lpFetchIndexInto, unchanged trust path); third-party
  sources are fetched unverified. Apps are merged by slug into one card carrying a
  sources[] array in priority order (highest first = default). Trust/verified are
  taken from the SOURCE, never the artifact's self-claim, so a community index
  can't promote itself to "official". Icons still mirror same-origin from the
  official index only. registry_catalog.json gains top-level sources[] + per-app
  sources[]; the old source{} object + signed/serial are kept for back-compat.
- New `libreportal catalog source list|add|remove|enable|disable` + `catalog
  refresh` CLI (dynamic-routed). Mutations go through the task system (cliTaskRun
  "…" "catalog"), never a new mutating API.

Scope firewall: this governs APP BROWSE + ADD only. LibrePortal's own updates and
hotfixes still resolve from the official CFG_RELEASE_BASE_URL alone — lpFetchIndex
is untouched, so a third-party catalog can never become a system-update channel.

Next: `app add --source`, then the WebUI (domains-style block source manager +
Add-dialog source picker with the Official badge / unverified warning).

Signed-off-by: librelad <librelad@digitalangels.vip>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 19:39:21 +01:00
librelad
10846b120e perf(backup): fetch snapshots once per location, filter in jq — kill the O(apps×locations) restic storm
The backup-data refresh in the WebUI update chain shelled out to
`restic snapshots` once PER APP (dashboard, app-status) and once per
(host, app) (migrate) — every call a `sudo`+`restic` spawn that re-opens
and decrypts the repo. With N apps and L locations that's O(N·L) (app-status)
plus O(hosts·apps·L) (migrate) spawns on every update — the silent stall
after "Refreshed backup dashboard data...".

One `restic snapshots --json` already returns the whole repo, so fetch it
ONCE per location and derive the per-app / per-host / system views in jq.
Validated against a live restic 0.19 repo: the jq filter reproduces
`restic snapshots --tag <t> --host <h>` exactly (incl. empty-host = all
hosts, and newest = last), and count/latest extraction matches the old grep.

- dashboard: one primary-location fetch → all app tiles + system status
  (was N snapshot calls + 1).
- app-status: one fetch per location → filter per app (was N×L calls);
  also de-recursed into a flat app loop.
- migrate: filter the already-fetched per-location list (was a spawn per
  host×app). Fixes a latent bug too — `grep -oc` over single-line JSON
  always reported a count of 1; jq `length` is now accurate.

restic snapshot spawns per refresh drop from ~O(N·L + hosts·apps·L) to ~O(L).
`restic stats` (still run per location in locations + dashboard) is a
separate, non-multiplicative cost left untouched to preserve its semantics.

Signed-off-by: librelad <librelad@digitalangels.vip>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 18:19:58 +01:00
librelad
cc36d86e57 feat(webui/apps): registry Add opens a detail modal with a marketplace link
Replaces the extra on-card marketplace tags (the separate Official trust
badge) with a richer, more elegant flow: a registry card keeps a single
'Available' pill, and clicking Add (or the card) opens a modal outlining the
app — full description, publisher + trust, version, the add command, and a
'View full page on the marketplace ↗' link to the app's page on the source
it came from — with the actual add as the confirm. Gives a beat to review a
community app before pulling it in.

- webui_registry_scan.sh re-emits source{base,channel}; loadApps stashes it
  (window.registryCatalogSource) + carries per-app version.
- openRegistryDetails() builds the eo-modal; addRegistryApp() is now just the
  task dispatch (the modal's confirm / a fallback).
- The marketplace website supports a #<slug> deep-link (focus one app, with a
  'Show all' bar), so the modal's link lands on that app's page.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-07-04 22:04:46 +01:00
librelad
cfb3a4f755 revert: drop the dedicated Marketplace section — keep catalog merged into the grid
Reverts a4e65df. The maintainer prefers the more fluid model where registry
apps flow into the App Center grid as 'Available — Add' cards alongside
installed/local apps, rather than a separate /apps/marketplace destination.
Restores the Stage-4 merged-grid behaviour (loadApps merges
registry_catalog.json; createAppCard renders registry cards; addRegistryApp
dispatches app_add). The make_app.sh featured passthrough + the generator's
featured/source fields go with it (unused by the merged grid).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-07-04 21:50:57 +01:00
librelad
a4e65df77f feat(webui/marketplace): dedicated Marketplace section in the App Center
Splits the catalog out of the main grid into its own destination, the
WordPress 'Add Plugins' vs 'Installed Plugins' model: the grid is now
purely 'your apps' (local definitions), and the new Marketplace section is
'get more apps' (the remote signed catalog).

- New MarketplacePage (components/apps/marketplace/) mounts at
  /apps/marketplace inside the apps feature (same sub-dispatch pattern as
  /apps/overview — no new top-level component). Pinned sidebar entry with a
  live 'available to add' count badge.
- Status strip: signed/unsigned, available + catalog counts, serial, source
  base+channel, freshness, and a Refresh that re-runs the host-side registry
  scan via updater_check.
- Publisher-curated Featured shelf (meta.featured, set at publish time — no
  tracking/popularity), category chips + search, per-app detail modal
  (long description, publisher/trust/version, add command), and the chained
  Add & set-up flow: dispatch app_add, and when the definition lands, hand
  off to the app's config/install page.
- State-aware cards: Available (Add) / Added (Set up →) / Installed (Open).
- Backend: make_app.sh passes through meta.featured; webui_registry_scan.sh
  emits featured + source{base,channel} in registry_catalog.json.
- Removed the grid's registry-merge + registry card path + its CSS (moved to
  the namespaced marketplace surface); app_add task wiring + completion
  handler retained and reused.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-07-04 21:38:12 +01:00
librelad
87edd09994 feat(webui/registry): catalog scan generator + hotfix-only Improvements stream
webuiRegistryCatalogScan (run by updater check, same atomic keep-prior
pattern as webuiArtifactScan) writes apps/generated/registry_catalog.json:
the type:"app"/kind:"bundle" rows of the signed index annotated with
defined/installed, browse metadata from the envelope meta, and icons
mirrored into core/icons/apps/registry/ ONLY when their bytes match the
sha256 pin in the signed index — the browser stays same-origin; a tampered
or oversized icon is skipped, never served.

webuiArtifactScan now selects type=="hotfix" so app rows never render as
pseudo-hotfixes in the Improvements tab, and counts+logs artifacts of
unrecognized type instead of surfacing them (the §8.1 forward-compat
firewall on the scan path).

Harness vs a locally served registry: 14/14 (catalog row + meta + flags,
icon pin verify + tamper skip, hotfix-only stream, unknown-type skip+log,
unreachable-registry keeps prior files).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-07-03 21:17:26 +01:00
librelad
36a5c87397 fix(artifacts): propagate LP_INDEX_SIGSTATE to callers via lpFetchIndexInto
Every caller captured the index with var=$(lpFetchIndex), which runs the
fetch in a command-substitution subshell — the LP_INDEX_SIGSTATE global it
sets never reached the caller. On a box with real signing active the
artifactApply/apply-auto gates would therefore refuse a correctly signed
index (fail-closed, but the apply path would be dead on arrival the day
signing activates), and artifact index / the WebUI scan would report a
verified feed as UNSIGNED.

New lpFetchIndexInto <var> [cache] runs the fetch in the calling shell and
assigns via printf -v; all four call sites converted. Verified with a
source-and-mock harness against a locally served index: 10/10 (sigstate
reaches caller, serial high-water, anti-rollback refuse, staleness refuse,
id enumeration, envelope round-trip).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-07-03 20:43:59 +01:00
librelad
fa47e16cab feat(updater): automatic background scan for versions, CVEs & improvements
Replace the click-to-scan-only flow with a self-throttled auto-scan that
rides the existing task-processor idle poll (the same shape as the
network-drift check — no new daemon, unit, or endpoint):

- 'libreportal updater check auto' gates on the age of the generated
  updates.json vs CFG_UPDATER_SCAN_INTERVAL (minutes, default 30,
  0 disables); a fresh file makes the 60s tick a single stat() + return.
  Manual checks and post-update rescans reset the clock for free, and a
  missing file means the first scan runs ~a minute after install.
- Eligible signed hotfixes keep flowing through artifactApplyAuto, which
  only enqueues ordinary tasks — mutations stay on the task path.
- Open updater surfaces (standalone /updater and the fleet Overview's
  headless UpdaterPage) follow along with a 60s static-JSON re-read that
  repaints only when a generated_at stamp changed; timer released via
  dispose() on unmount, ticks skipped while hidden.
- Empty states now say the first scan happens automatically; Check now
  stays as the immediate manual override.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-06-12 22:07:42 +01:00
librelad
20f8ca2eb5 feat(network): detect + heal apps stranded off the docker subnet
Closes the gap behind the vpn-recreate bug: when the shared network is
recreated with a different /24, every app's stored static IP is left
outside it and adoptDockerSubnet only realigns CFG, not the apps.

- networkScanConflicts (network_conflicts.sh): read-only scan diffing each
  active network_resources IP against docker's real subnet (via ipInSubnet).
  Per-service routing-aware — skips gateway-routed services whose ipv4 is
  commented out in the deployed compose, so gluetun apps don't false-positive.
  Distinguishes 'daemon down' (benign) from 'network missing' (real).

- webuiSystemNetworkCheck (webui_system_network.sh): self-throttled generator
  that writes frontend/data/system/network_status.json (modelled on
  verify_status.json). Wired into webuiSystemUpdate AND run unconditionally
  every ~60s from the task-processor poll (regen webui is mtime-gated and
  would never fire on drift, which touches no source file).

- networkHealConflicts (network_heal.sh) + 'libreportal system network
  check|heal [app]': the heal adopts docker's subnet in-process, then re-IPs
  stranded apps with reset_network=ip (ports preserved), gluetun first.
  Mutating path runs only through the task system (dual-mode, like update
  apply); read-only check runs inline.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 16:03:53 +01:00
librelad
96b04392dc feat(distribution): Phase 3 — hotfix scan generator + severity-split auto-apply
- CFG_HOTFIX_AUTO (security-breakage|all|off, default security-breakage) seeded in
  general_terminal; reaches existing installs via the add-only config reconciler.
- webui_artifact_scan.sh (webuiArtifactScan): fetch+verify the signed index, write
  artifacts_available.json ATOMICALLY (build in temp → jq-validate → one write;
  keep the prior file on any failure — never emits broken JSON). Annotates each
  artifact with applied (a per-id record exists) + applicable (target installed).
- artifactApplyAuto + `libreportal artifact apply-auto`: enqueue apply tasks for
  the eligible signed hotfixes — only when the index is VERIFIED-signed, only
  auto==true + in the severity policy + applicable + not already applied. Each
  apply is its own task (visible in the log + History), never applied inline.
- `updater check` now also refreshes the index (webuiArtifactScan) and runs
  artifactApplyAuto — one front door, no second phone-home.

Unit-tested 13/13: policy filtering (security-breakage / off / all), auto:false
exclusion, already-applied skip, non-installed-app skip, unsigned-index fail-closed,
and the scan transform's signed/applied/applicable fields.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-31 20:53:54 +01:00
librelad
f49455e38e fix(de-sudo): route all confirmed container-tree writes through the privileged path
Exhaustive audit (workflow: 19 finders + adversarial per-file verify; 85 raw ->
66 unique -> 39 confirmed) found 36 direct writes into the container-owned tree
that bypass runFileOp/runFileWrite/runCfgOp (manager => EACCES in rootless) plus
3 $?-masking sites. Fixes by area:

- apps: grafana + prometheus install hooks (sudo chmod -> runFileOp chmod);
  gluetun provider etag (tee -> runFileWrite).
- webui generators: task-create (10 sites: mkdir/chown/tee/jq|tee/sed|tee ->
  runFileOp/runFileWrite); app-icons (mkdir/cp/mv); config icon cp; system
  metrics + update throttle stamps (runAsManager touch -> runFileOp touch);
  setup-lock rm; updater history seed + cp.
- task health checker: 4 log writes (tee -a -> runFileWrite -a) + 3 find -delete
  (-> runFileOp find).
- config reconcile: backup cp -> runCfgOp; live cp -> runFileWrite < tmp for
  container-owned configs (the container user can't read a manager 0600 tmp).
- peer pull: tar extract into the container tree -> runFileOp tar.
- masking: ip_find_available + folder_group(x2) — split 'local VAR=$(cmd)' so $?
  reaches the following [[ $? ]] check.

15 files, all pass bash -n; fixed idioms confirmed gone.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-31 03:50:48 +01:00
librelad
053a620e22 fix(reliability): split local result=$(cmd) so $? survives for checkSuccess
'local result=$(cmd)' resets $? to 0 (the local builtin's own exit), so the
following checkSuccess always saw success regardless of cmd's real exit — the
mechanism that masked the de-sudo write failures. Split declaration from
assignment ('local result; result=$(cmd)') across all 235 active-code sites
(84 files) so the command's exit reaches checkSuccess. No behaviour change
beyond $? now being accurate (no set -e in runtime code; multi-line
assignments transform safely).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-31 03:09:25 +01:00
librelad
6089eb0882 fix(de-sudo): route container-tree writes through the privileged path
Two more cases of the manager writing directly into the container-owned
/libreportal-containers tree (same class as the regen-poll stamp), both masked
by a '✓ Success' that printed anyway:

- Password replacers (config/password/*): used 'runInstallOp sed -i' (manager)
  on app configs copied into the container tree, so sed -i EACCES'd its temp
  file and the substitution silently failed — the adguard.config 'couldn't open
  temporary file', leaving the literal RANDOMIZEDPASSWORD placeholder. Added
  runCfgOp (picks runFileOp vs runInstallOp by the target file's location) and
  routed every $file grep/sed/awk through it: password, username, hex, vapid,
  appkey, and bcrypt.

- Updater generator (webui_updater_scan): 'runFileOp cp <manager-tmp>' can't
  read the manager's 0600 mktemp as the container user, so it fell through to a
  manager 'cp' that EACCES'd on the container-owned out_dir. Switched the three
  writes to 'runFileWrite < tmp' (manager shell reads the tmp; container user
  tees the write).

Both deploy via the normal quick path (relocatable scripts) — no footprint bump,
no reinstall.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-31 02:33:10 +01:00
librelad
f0c5f607d7 fix(webui): point app/category icon generators at /core/icons
The frontend modularization moved icons to frontend/core/icons/ and updated the
frontend JS, but the host-side generators were never updated — they wrote the
apps.json/categories 'icon' field as /icons/apps/<app>.svg and /icons/categories/
<cat>.svg, and webui_app_icons.sh / webui_config.sh synced icon files into the
non-existent frontend/icons/apps. Those served paths 404 (text/html catch-all),
so every app card fell back to default.svg (the generic box) instead of its real
logo.

Repointed to /core/icons/... (where the SVGs actually live and serve as
image/svg+xml):
- webui_config.sh: icon dir + emitted apps.json icon path
- webui_app_icons.sh: icon sync dir + comment
- webui_container_setup.sh: comment
- webui_create_app_categories.sh: 11 category icon paths

Source fix only — the live apps.json refreshes on the next host-side regen
(lpRegen). NOT touched: scripts/backup/engines/*.json '/icons/config/backup.svg'
(that SVG lives at the oddly-nested components/admin/config/icons/config/ and
serves at neither path — needs a placement decision, flagged separately).

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-30 23:48:19 +01:00
librelad
daa336449a feat(updater): backend — data generator + 'libreportal updater' CLI with DR
- scripts/webui/data/generators/updater/webui_updater_scan.sh (webuiUpdaterScan):
  writes frontend/data/updater/generated/{updates,cves,history}.json from the
  installed-apps DB (current image per app from compose). Available-version +
  CVE-scanner are clearly-marked pluggable hooks; always emits valid JSON.
- scripts/cli/commands/updater/{cli_updater_commands.sh,cli_updater_header.sh}:
  auto-dispatched as 'libreportal updater <sub>' (check/apply/apply-all/rollback).
  apply does disaster-recovery FIRST — snapshots the app via the backup engine,
  then pulls + recreates (real dockerComposeUp/compose-pull helpers), records
  history, and auto-rolls-back on failure. Standard LIBREPORTAL_TASK_EXEC
  enqueue/exec split so WebUI + CLI share locking + audit trail.

New .sh files: the array/function-manifest regen self-heals on deploy; the
check path also sources its generator on demand to cover the gap.

NOTE: host-side bash — written to the repo's conventions but not runnable in
this env; this is the surface to test (the WebUI feature is lp-shot-verified).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-30 03:13:26 +01:00
librelad
7a0477ff8b feat(system): per-folder breakdown under each app on the Storage page
Extend the app-storage generator to record every bind mount's size and
in-container path, grouped into a per-app folder list. The "Storage by
app" rows are now expandable: click an app to see where its space goes
(e.g. /var/lib/mysql vs /data), with external-drive folders flagged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-28 20:33:21 +01:00
librelad
4b39cf770b feat(system): per-app on-disk storage on the Storage page
Docker only tracks where an app's data lives (its bind mounts), not how
big a bind-mounted host dir is — so named-volume accounting reads ~0 for
LibrePortal, whose app data lives in bind mounts. Add a generator that
reads each app's mount map from `docker inspect` and `du`s the directories
(via runFileOp, so it runs as the data-owning user and isn't blocked by
rootless UID mapping). `du -x` keeps each measurement on its own
filesystem, so data on a separate disk is reported as a distinct
"external" total. The generator self-throttles to ~10 min since du is
heavier than the per-minute metrics. Surfaced as a "Storage by app"
section on the Storage page.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-28 20:06:34 +01:00
librelad
b28268a61f feat(system): "Verified" integrity check against the signed release manifest
Adds per-file integrity attestation on top of the existing signed-tarball
release flow. make_release now generates a SHA256SUMS manifest over the shipped
tree and (when a key is configured) signs it, riding both inside the release
tarball so they land in the install tree with no extra download.

lpVerifyInstall (scripts/source/verify.sh) re-hashes the install tree against
that manifest and verifies the manifest's minisign signature against the
root-owned footprint pubkey, yielding states: verified / modified / tampered /
unsigned / unverifiable / development. webuiSystemVerify writes verify_status.json
(throttled daily, force on demand, also after each update apply), surfaced as an
Integrity line + "Verify now" button on the Admin → Overview Updates card and a
row in the update details panel. `libreportal verify` exposes the same check on
the CLI.

Honest framing: this is a self-check (run by the software it verifies), so red
fires only for genuine modified/tampered states; the badge tooltip points to
out-of-band `minisign -Vm` for an independent guarantee.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-28 19:41:22 +01:00
librelad
b7d95f5e95 fix(webui): app-log removal uses runFileOp rm -f so uninstall can't hang
The uninstall branch of webuiUpdateAppLog removed the per-app WebUI log
with a bare `rm`. The log lives in the container data plane and is owned
by the container user, often without a write bit. A bare rm (run as root
via `sudo init.sh uninstall`) prompts interactively for write-protected
files — which hangs an otherwise-unattended deploy: the uninstall phase
of a `full` redeploy stopped dead at "rm: remove write-protected regular
file '.../frontend/logs/apps/<app>.log'?".

Route it through runFileOp rm -f (as the container-data owner, force) to
match the neighbouring uninstall_app.sh and the install branch's
owner-aware createTouch/runFileWrite helpers. No prompt, correct owner.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-28 13:46:16 +01:00
librelad
e72d3d278d style(backup): shorten the BACKUP_STRATEGY tooltip to one practical sentence
The advanced field was carrying a four-clause explanation walking through
"quiesce", "live dump", "stop fallback", and the live-option eligibility
rules — useful background for engineers, overwhelming as a hover tooltip
for the typical user picking from the dropdown.

New tooltip:
  "How the app is prepared before each backup. Automatic is recommended;
   pick Live only if you need zero downtime."

Two lines, plain language, points at the right default. The dropdown's
own labels (Automatic (recommended) / Stop→snapshot→start / Pause→snapshot
→unpause / Live — no downtime) carry the technical specifics; the
tooltip no longer has to.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-27 01:26:27 +01:00
librelad
3319614be4 fix(webui): route the migrate.json + peers.json writes through runFileWrite
Two webui-data generators wrote to a temp file via bare `cat > "$temp_file"`
then `runFileOp mv` to the final path. The temp file's path sits inside
$containers_dir/libreportal/frontend/data/<x>/generated/ — owned by the
dockerinstall user (the data plane). The generators run as the manager,
who can't open paths under that tree for write, so every WebUI update
hit:

    webui_backup_migrate.sh: line 125: …/migrate.json.tmp.<pid>: Permission denied
    mv: cannot stat '…/migrate.json.tmp.<pid>': No such file or directory
    webui_peers.sh: line 23: …/peers.json.tmp.<pid>: Permission denied
    mv: cannot stat '…/peers.json.tmp.<pid>': No such file or directory

Pipe the heredoc through `runFileWrite "$output_file"` instead — same
shape the 5 sibling generators in this dir (backup_app_status,
backup_locations, backup_passwords, backup_snapshots, backup_dashboard)
already use. runFileWrite routes the write via the install user that
owns the data tree, so the file lands on disk in one step (no temp +
mv dance needed). The unused `local temp_file=...` declarations dropped
out cleanly.

The trailing `runFileOp chmod 644 "$output_file"` stays — it's the only
guarantee the WebUI container (which reads these files RO) sees them as
world-readable regardless of dockerinstall's umask.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 22:53:10 +01:00
librelad
1014dd6e42 feat(peers): introduce 'Peer' as a first-class concept (Phase 2)
A peer is a named reference to another LibrePortal instance. Phase 2 only
implements kind=backup-channel (friendly label over a hostname that shows
up in a shared backup repo); direct-ssh-direct and direct-ssh-via-relay
(Connect's blind-relay) are reserved enum values for Phase 3.

DB schema (db_create_tables.sh):
  CREATE TABLE peers (
    id           INTEGER PRIMARY KEY AUTOINCREMENT,
    name         TEXT UNIQUE NOT NULL,
    kind         TEXT NOT NULL DEFAULT 'backup-channel',
    config_json  TEXT NOT NULL DEFAULT '{}',
    status       TEXT DEFAULT 'unknown',
    last_seen    TEXT,
    created_at   TEXT DEFAULT CURRENT_TIMESTAMP
  );
  + indexes on name and kind.

  config_json is kind-specific so new transports don't need a schema
  migration. For backup-channel it carries {"hostname":"","loc_idx":N}.

Bash module (scripts/peer/):
  peer_helpers.sh   _peerDb, peerSqlEscape, peerValidateName/Kind.
  peer_add.sh       peerAdd <name> <kind> [k=v ...] → INSERT, refresh
                    generator. Rejects unimplemented kinds early so users
                    don't create dead-end peer records.
  peer_remove.sh    peerRemove <name> → DELETE.
  peer_list.sh      peerList → JSON array; peerGet, peerNameForHostname
                    (reverse-lookup for the migrate-tab overlay).
  peer_check.sh     peerCheckReachable, peerCheckAll. For backup-channel
                    'reachable' = at least one snapshot from that hostname
                    visible in (preferred|any enabled) location. Updates
                    status + last_seen so UI dots render without re-probing.

CLI (scripts/cli/commands/peer/):
  libreportal peer list
  libreportal peer get <name>
  libreportal peer add <name> backup-channel hostname=<host> [loc_idx=<n>]
  libreportal peer remove <name>
  libreportal peer check [name]

  Auto-routed by cli_initialize.sh's category-discovery.

WebUI data generator (scripts/webui/data/generators/peers/webui_peers.sh):
  Emits data/peers/generated/peers.json with the peerList output and a
  generated_at envelope. Hooked into webuiLibrePortalUpdate alongside the
  backup generators.

Frontend:
  - New top-level /peers route in spa.js (PeersPage class, peers-content.html).
  - 'Peers' nav item in the topbar between Backups and the right-side controls.
  - Add-peer modal with friendly-name + kind + hostname + preferred-location
    selector (populated from the existing backup-locations data).
  - Per-peer card with status dot, last-checked time, Check + Remove buttons.
  - Phase 3 kinds appear in the kind dropdown as disabled options so users
    can see what's coming.

Source-array wiring:
  - generate_arrays.sh auto-created files_peer.sh from the new peer/ dir.
  - cli_files.sh + app_files.sh include ${peer_scripts[@]} alphabetically.
  - files_webui.sh auto-picked-up the new peers/ generator subfolder.

The migrate-tab friendly-name overlay (use peer names in /backup/migrate
when a peer record exists for a hostname) is intentionally deferred — it's
a 5-line frontend lookup once peers.json is loaded; cleaner to add after
Phase 3 ships its peer-detail view.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 17:43:56 +01:00
librelad
52e4280a67 feat(webui): add 'Migrate' tab — restore an app from another LibrePortal
Phase 1 of the migration-system refresh. Surfaces Phase 0's kernel
(libreportal restore migrate ...) as a WebUI flow so users don't have
to drop to the CLI to pull an app from a peer's backups.

backend / data generator:
  scripts/webui/data/generators/backup/webui_backup_migrate.sh
    Walks every enabled backup location, lists every (other_host, app)
    pair with snapshot count + latest id/date, and emits a single
    destination summary block (installed apps, running apps, disk free)
    so the frontend can compute collisions and warnings without per-row
    API round-trips. Filters out our own hostname — we don't migrate to
    ourselves. Output: data/backup/generated/migrate.json.
    Hooked into the standard webuiLibrePortalUpdate refresh pipeline,
    so 'libreportal regen webui' (and the periodic task-processor poll)
    keep it fresh on their own.

frontend:
  - New 'Migrate' sidebar tab on /backup, sits between Locations and
    Configuration. Path-based URL: /backup/migrate.
  - Per-source-host cards listing every available app, with snapshot
    count + relative-time hint, collision dot when the app is already
    installed here, and per-app + per-host migrate buttons.
  - Confirm modal with two checkboxes matching the kernel's defaults:
      [✓] Back up the destination's existing copy first   (pre-migrate
          backup; auto-disabled when there's nothing to back up)
      [✓] Rewrite host-bound URLs to this host             (URL rewrite
          — uncheck only to keep source hostnames)
    On confirm, runs 'libreportal restore migrate app/system …' via the
    task system; opt-out checkboxes append --no-pre-backup / --keep-urls
    only when the user un-ticks, matching the kernel's default-on flags.
  - Empty state when no other hosts have visible backups, explaining
    the shared-backup-location prerequisite.

The CLI dispatcher hooks (Phase 0) wire restore migrate app/system to
migrateApplyApp/migrateApplySystem, so the WebUI gets pre-backup safety,
URL rewrite, and structured progress (when --json-progress is set; not
needed here yet — the task system's log tail is enough for v1).

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 17:32:01 +01:00