222 Commits

Author SHA1 Message Date
librelad
5cf5b88b16 fix(webui): flatten redundant components/admin/config/icons/config/ + repair engine logos
The config-category icons sat at admin/config/icons/CONFIG/ — the inner config/
duplicates the subsystem name; they belong in the icons root. Moved all 6
(backup, features, general, network, security, webui) up to
components/admin/config/icons/ and updated the two consumers (config-manager.js
header icon, config-sidebar.js category icons).

Also fixed the backup-engine logos: scripts/backup/engines/{restic,kopia,borg}
.json pointed 'logo' at /icons/config/backup.svg — a path that 404'd on two
counts (missing the components/admin/config prefix AND the now-removed config/
nesting), so the engine-details modal logo silently hid. Repointed to the real
served path /components/admin/config/icons/backup.svg.

(Left the meaningful icon groupings alone — admin/system/icons/{cpu,os} and
apps/core/icons/vpn are vendor/OS/provider logo sets, not redundant nesting.
The backup engines borrowing an admin-config icon is a minor smell; a dedicated
backup-engine icon could replace it later if wanted.)

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-31 00:07:49 +01:00
librelad
f0c5f607d7 fix(webui): point app/category icon generators at /core/icons
The frontend modularization moved icons to frontend/core/icons/ and updated the
frontend JS, but the host-side generators were never updated — they wrote the
apps.json/categories 'icon' field as /icons/apps/<app>.svg and /icons/categories/
<cat>.svg, and webui_app_icons.sh / webui_config.sh synced icon files into the
non-existent frontend/icons/apps. Those served paths 404 (text/html catch-all),
so every app card fell back to default.svg (the generic box) instead of its real
logo.

Repointed to /core/icons/... (where the SVGs actually live and serve as
image/svg+xml):
- webui_config.sh: icon dir + emitted apps.json icon path
- webui_app_icons.sh: icon sync dir + comment
- webui_container_setup.sh: comment
- webui_create_app_categories.sh: 11 category icon paths

Source fix only — the live apps.json refreshes on the next host-side regen
(lpRegen). NOT touched: scripts/backup/engines/*.json '/icons/config/backup.svg'
(that SVG lives at the oddly-nested components/admin/config/icons/config/ and
serves at neither path — needs a placement decision, flagged separately).

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-30 23:48:19 +01:00
librelad
daa336449a feat(updater): backend — data generator + 'libreportal updater' CLI with DR
- scripts/webui/data/generators/updater/webui_updater_scan.sh (webuiUpdaterScan):
  writes frontend/data/updater/generated/{updates,cves,history}.json from the
  installed-apps DB (current image per app from compose). Available-version +
  CVE-scanner are clearly-marked pluggable hooks; always emits valid JSON.
- scripts/cli/commands/updater/{cli_updater_commands.sh,cli_updater_header.sh}:
  auto-dispatched as 'libreportal updater <sub>' (check/apply/apply-all/rollback).
  apply does disaster-recovery FIRST — snapshots the app via the backup engine,
  then pulls + recreates (real dockerComposeUp/compose-pull helpers), records
  history, and auto-rolls-back on failure. Standard LIBREPORTAL_TASK_EXEC
  enqueue/exec split so WebUI + CLI share locking + audit trail.

New .sh files: the array/function-manifest regen self-heals on deploy; the
check path also sources its generator on demand to cover the gap.

NOTE: host-side bash — written to the repo's conventions but not runnable in
this env; this is the surface to test (the WebUI feature is lp-shot-verified).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-30 03:13:26 +01:00
librelad
ad08ce2324 refactor(app-scan): auto-clean leftover folders, drop bogus wipe prompt
The empty-folder reaper only ever fired on folders with no real data
(empty, or only a regenerable .config and/or migrate.txt marker), yet
prompted 'THIS WILL WIPE ALL DATA' before each removal — a question
about data that didn't exist. Collapse the four duplicated branches into
one reason-string path, clean these leftovers automatically, and fix the
stale $app_name used in the DB-delete (it deleted the wrong row when
looping over $folder_name).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-29 14:53:41 +01:00
librelad
b1ffe9d052 chore(rootless): trim AppArmor banner text
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-29 14:50:11 +01:00
librelad
9ca5cc6c7c feat(system): full, deletable images list on the Storage page
Replaces the read-only "Largest images" top-10 table with a Tasks-style list of
ALL Docker images, with select-one / select-multiple / clear-all removal that
mirrors the Tasks page UX (row checkboxes, master select-all, a button that
morphs Clear All ↔ Delete Selected (N), an eo confirm modal).

Deletion routes through the task system, NOT a new web API: a new
`libreportal system image rm [--force] <ids>` CLI subcommand (validates each
ref, loops runFileOp docker image rm, reports a tally) is invoked via the
system_image_rm task action — same pattern as Reclaim. The web backend change
is read-only (uncap the existing /storage image list). In-use images are
skipped by default with an opt-in "force-remove" toggle (warned). The page
stays put, toasts, and refreshes on the task's completion event.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-28 21:32:29 +01:00
librelad
7a0477ff8b feat(system): per-folder breakdown under each app on the Storage page
Extend the app-storage generator to record every bind mount's size and
in-container path, grouped into a per-app folder list. The "Storage by
app" rows are now expandable: click an app to see where its space goes
(e.g. /var/lib/mysql vs /data), with external-drive folders flagged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-28 20:33:21 +01:00
librelad
4b39cf770b feat(system): per-app on-disk storage on the Storage page
Docker only tracks where an app's data lives (its bind mounts), not how
big a bind-mounted host dir is — so named-volume accounting reads ~0 for
LibrePortal, whose app data lives in bind mounts. Add a generator that
reads each app's mount map from `docker inspect` and `du`s the directories
(via runFileOp, so it runs as the data-owning user and isn't blocked by
rootless UID mapping). `du -x` keeps each measurement on its own
filesystem, so data on a separate disk is reported as a distinct
"external" total. The generator self-throttles to ~10 min since du is
heavier than the per-minute metrics. Surfaced as a "Storage by app"
section on the Storage page.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-28 20:06:34 +01:00
librelad
b28268a61f feat(system): "Verified" integrity check against the signed release manifest
Adds per-file integrity attestation on top of the existing signed-tarball
release flow. make_release now generates a SHA256SUMS manifest over the shipped
tree and (when a key is configured) signs it, riding both inside the release
tarball so they land in the install tree with no extra download.

lpVerifyInstall (scripts/source/verify.sh) re-hashes the install tree against
that manifest and verifies the manifest's minisign signature against the
root-owned footprint pubkey, yielding states: verified / modified / tampered /
unsigned / unverifiable / development. webuiSystemVerify writes verify_status.json
(throttled daily, force on demand, also after each update apply), surfaced as an
Integrity line + "Verify now" button on the Admin → Overview Updates card and a
row in the update details panel. `libreportal verify` exposes the same check on
the CLI.

Honest framing: this is a self-check (run by the software it verifies), so red
fires only for genuine modified/tampered states; the badge tooltip points to
out-of-band `minisign -Vm` for an independent guarantee.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-28 19:41:22 +01:00
librelad
49cf7e8bec ux(system): move Reclaim button top-right, make it actually free space
Three fixes from testing the storage page:

- Placement: the "Reclaim space" button moves into the page header,
  top-right (matching the metric page), instead of sitting in the body.

- It now actually reclaims: build cache needs -a to drop (docker reports
  0 B "reclaimable" without it, but it's pure cache — safe to clear), so
  the CLI uses `docker builder prune -af`. Previously the safe scope
  freed ~nothing on a box whose reclaimable was mostly cache.

- Honest "Reclaimable" number: /api/system/storage was counting the
  whole build cache AND unused tagged images, overstating what the safe
  prune frees (e.g. 340 MB shown, ~96 MB per docker, button cleared 0).
  Reclaimable now = dangling images + build cache only; stopped
  containers and volumes are never counted (the safe prune never touches
  them). Headline now matches the button's effect.

Also simplify the CLI output (drop the jargony scope notice and the
reclaimed-total greps) and re-enable the now-persistent header button
after the post-reclaim refreshes.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-28 19:06:02 +01:00
librelad
3031c6cab9 feat(system): "Reclaim space" action on the Storage page
Adds a `libreportal system reclaim` CLI command and an orange "Reclaim
space" button on /admin/config/system/storage (the v2 prune control the
page always hinted at).

Scope is deliberately SAFE: build cache + dangling (untagged) images
only (docker builder prune -f + docker image prune -f via the
rootless-aware runFileOp). It never touches volumes (app data) or
tagged/in-use images, so nothing an app relies on is removed.

Wiring mirrors system_update: a systemReclaim() action + system_reclaim
route case run the command verbatim through the task processor. The
button confirms via showConfirmation, shows a spinner, and re-reads
storage usage as the prune lands. Button styled with --status-warning to
match the Reclaimable stat it sits under, with a note clarifying scope.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-28 18:50:27 +01:00
librelad
5be49b67c6 feat(setup): dev-mode easter egg on the Experience step
Tap the Advanced card 10 times and a full-width "Dev mode activated"
strip slides in beneath the two cards — the same 10-tap pattern as the
topbar logo and services-manager unlocks, now at install time. The
choice rides the setup payload (dev_mode) so setup_apply.sh persists
CFG_DEV_MODE=true, and it's mirrored in-process via LpUi.dev so the
next surface already reflects it. 10 more taps toggles it back off.

Counting the Advanced radio's click (not the label's) sidesteps the
label->input double-fire; the radio is pointer-events:none, so each tap
reaches it exactly once. The strip is [hidden] by default (no phantom
gap in the flex column) and replays its entrance keyframes each reveal.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-28 14:12:29 +01:00
librelad
b7d95f5e95 fix(webui): app-log removal uses runFileOp rm -f so uninstall can't hang
The uninstall branch of webuiUpdateAppLog removed the per-app WebUI log
with a bare `rm`. The log lives in the container data plane and is owned
by the container user, often without a write bit. A bare rm (run as root
via `sudo init.sh uninstall`) prompts interactively for write-protected
files — which hangs an otherwise-unattended deploy: the uninstall phase
of a `full` redeploy stopped dead at "rm: remove write-protected regular
file '.../frontend/logs/apps/<app>.log'?".

Route it through runFileOp rm -f (as the container-data owner, force) to
match the neighbouring uninstall_app.sh and the install branch's
owner-aware createTouch/runFileWrite helpers. No prompt, correct owner.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-28 13:46:16 +01:00
librelad
9a92805bdb feat(ui): Beginner/Advanced experience level + linked dev mode + setup-wizard step
Adds the install-time Beginner/Advanced choice the user described, with
the linked dev-mode escape hatch and global body-class machinery that
any surface can hang advanced/dev-only DOM off.

Three-tier mental model, two flags in the data model:

  Beginner            default. nothing extra shown.
  Advanced            .lp-advanced DOM revealed; advanced wizard steps shown
  Adv+Dev             .lp-dev DOM also revealed; dev-only fields visible

Linking rule (enforced inside LpUi):
  - enabling dev auto-enables advanced (dev w/o advanced is incoherent)
  - disabling advanced auto-disables dev

Wire shape:
  CFG_INSTALL_LEVEL                  beginner | advanced (general_basic)
  CFG_DEV_MODE                       existing, unchanged behaviour
  window.LpUi.{advanced,dev}         {get(), set(), apply()}
  localStorage keys                  lp.ui.advanced, lp.ui.dev, lp.ui.seeded
  body classes                       lp-ui--advanced, lp-ui--dev
  events                             lp-ui-advanced-changed, lp-ui-dev-changed
  global CSS gates                   body:not(.lp-ui--advanced) .lp-advanced { hide }
                                     body:not(.lp-ui--dev) .lp-dev { hide }

Setup wizard:
  - New step 1 "Choose your experience" with Beginner/Advanced cards.
    Beginner is preselected so race-through gets the safe default.
  - Picking a level updates totalSteps live (4 for beginner, 5 for
    advanced) so the progress bar reflects the choice.
  - Metrics step (Prometheus + Grafana) is gated to Advanced — beginner
    never sees it, never gets asked, never installs them by accident.
  - Submit payload now carries install_level; setup-routes.js validates
    it against the enum (beginner|advanced).
  - scripts/setup/setup_apply.sh writes it to CFG_INSTALL_LEVEL via
    updateConfigOption.
  - On submit, LpUi.advanced.set is called immediately so the next
    surface (running-tasks page) is already in the right mode — no
    refresh needed.

WebUI bootstrap:
  - js/utils/lp-ui.js loads first thing in index.html (before any other
    bootstrap) so body.lp-ui--advanced is applied pre-paint — no FOUC
    of advanced content on a fresh tab.
  - On first run, seeds lp.ui.advanced from CFG_INSTALL_LEVEL.
    Subsequent loads honour the user's per-browser override.
  - Mirrors CFG_DEV_MODE → lp.ui.dev on the seed pass.

Dev-mode unlock:
  - Existing 10-click LibrePortal-logo easter egg unchanged.
  - NEW: same 10-click unlock on the Advanced toggle (in services-manager).
    Reuses the countdown-toast pattern; on the 10th click delegates to
    the topbar's _setDevMode so there's one canonical setter and the
    config_update task path stays singular.
  - TopbarComponent now exposes its instance as window.topbar so the
    toggle's tap handler can reach _setDevMode.
  - topbar._setDevMode also calls LpUi.dev.set(enabled) so the body
    class flips immediately (no reload needed to see dev-only DOM).

Convention rolled out:
  - Services tab's .service-rich panel was already gated on
    body.lp-ui--advanced.
  - .lp-advanced / .lp-dev are now first-class hide classes any
    component can tag DOM with — see style.css globals.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-27 23:31:50 +01:00
librelad
66e747e1ba fix(uninstall): always run name-based container cleanup + drop CLI hint
Two small uninstall-output tweaks.

1. dockerComposeDownRemove now ALWAYS calls dockerRemoveApp (the
   `docker ps -aqf name=…` → stop + rm sweep) as a fallback, even when
   the compose-down step is skipped because the app dir is missing.
   Before, a partial prior uninstall (compose file gone but containers
   still running) produced "App directory not found. Skipping container
   shutdown." and then proceeded as if the uninstall were complete —
   leaving the actual containers running. The name-based sweep also
   runs after a successful compose-down to catch anything compose
   wouldn't pick up (renamed services, orphans from earlier failures).

   While here: the OS_TYPE gate (only Ubuntu/Debian) is gone too —
   `docker compose down` works on any OS with docker, and gating it
   meant Arch/etc. users got NO compose teardown at all.

2. The step-2 header "Keeping Docker images (pass --delete-images to
   remove)" trimmed to just "Keeping Docker images". The `isNotice`
   line below already explains the reuse-on-reinstall behaviour; the
   CLI-flag hint reads as noise in the WebUI task log where users
   can't act on it anyway. CLI users can still pass --delete-images
   (cli_app_commands.sh wires it as before) or tick the WebUI's
   "Also delete docker image" checkbox.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-27 15:41:19 +01:00
librelad
cb055b4b1f fix(uninstall): wipe container sub-UID app data via root helper
dockerDeleteData (uninstall) and the wipe-before-restore step in
restoreAppStart both did `runFileOp rm -rf $containers_dir$app_name`,
which runs as $CFG_DOCKER_INSTALL_USER (dockerinstall, uid 1002 on
rootless). That user owns app-template files but CANNOT remove
container sub-UID dirs created by the daemon's userns mapping —
postgres data at uid 232070, nextcloud html at uid 33, etc. The rm
therefore silently failed with

  rm: cannot remove '/libreportal-containers/invidious/postgresdata':
    Permission denied

while still reporting "<app> successfully uninstalled" — leaving the
sub-UID directory tree on disk to confuse the next install and leak
storage.

Fix: route the wipe through a new `app-data-remove` action in the
root-owned libreportal-ownership helper. Root can rm sub-UID files
unconditionally. The helper validates the app name (alphanumeric +
. _ -, no traversal), refuses the WebUI's own slot (libreportal), and
is idempotent when the dir is already gone.

Two callers updated:
- scripts/docker/app/uninstall/delete_data.sh
- scripts/restore/restore_app_start.sh

The helper itself ships root-owned at /usr/local/lib/libreportal/, so a
fresh install or release upgrade is needed to pick up the new action.
Bumped init.sh footprint_version 2 → 3 so the runtime updater
prompts a root re-install on the next release.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-27 15:32:44 +01:00
librelad
9817d6945a fix(config): silence app-data Permission-denied chatter from config reconcile
checkApplicationsConfigFilesMissingVariables does a `find $containers_dir
-maxdepth 2 -type f -name '*.config'` to enumerate every app's live
config. runFileOp drops privileges to $CFG_DOCKER_INSTALL_USER
(dockerinstall), which is intentionally the *manager* for the rootless
data plane — but doesn't own the per-app container sub-UID dirs (e.g.
invidious/postgresdata uid 232070, nextcloud/html uid 33).

At maxdepth 2 find doesn't actually need to descend into those dirs to
satisfy the name filter, but it tries to anyway and emits chatter like

  find: '.../invidious/postgresdata': Permission denied

every time the function runs (config-reconcile path on install / app
start / restart). Cosmetic only — the actual .config files are at the
right depth and ARE found — but it shows up in the live CLI output
during installs.

2>/dev/null on the find. The function's purpose is purely to enumerate
LibrePortal-managed .config files; sub-UID data dirs are by design
unreachable to the manager and there's no signal in that error.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-27 15:14:08 +01:00
librelad
3294ca4e41 fix(boot): source run_privileged.sh before checkConfigFilesMissingFiles
load_sources.sh calls checkConfigFilesMissingFiles() after init.sh +
variables.sh but BEFORE initilize_files.sh sources the function
manifest. checkConfigFilesMissingFiles uses runInstallOp (in
docker/command/run_privileged.sh) to copy missing config templates —
under LP_LAZY=1 that's an autoload stub that only exists once the
manifest is sourced. So when any template is genuinely missing, the
copy call hits "runInstallOp: command not found" and the file silently
never gets copied.

Symptom on a fresh CLI invocation (foreground or processor subprocess
inheriting LIBREPORTAL_TASK_EXEC=1) where a new config category was
added:
  config_check_missing.sh: line 33: runInstallOp: command not found
  ✓ Success 1 config files were missing and have been added to the
    configs folder.    ← false success: the count incremented but the
                         copy itself didn't happen

Fix: source run_privileged.sh directly in load_sources.sh just before
the missing-files check. The file is pure function definitions (runAsManager
/ runFileOp / runFileWrite / runInstallOp / runInstallWrite), no side
effects, ~150 lines — safe to source unconditionally and idempotent
with the eager/lazy load that happens later. Adds <1ms to every CLI
invocation; saves silent failures on the rare path that calls it.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-27 14:58:43 +01:00
librelad
f252d7680b fix(cli): camelCase task fields match WebUI shape (createdAt, not created_at)
WebUI-created tasks emit camelCase initial fields (createdAt, startedAt,
completedAt, heartbeatAt, exitCode, errorMessage) per
tasks-manager.js / task-manager.js conventions, with createdAt in
ISO-UTC-with-ms (`2026-05-27T13:01:26.345Z`). The processor then layers
snake_case status fields (started_at, heartbeat_at, …) on top as the
task runs.

The CLI's cliTaskRun was writing snake_case only — `created_at` with
local-tz offset. The task panel's renderer reads `task.createdAt`
directly (no alias), so CLI-queued tasks showed blank Created/Started
columns until the processor wrote its own snake_case overlay
(which doesn't include createdAt at all). Visible symptom: dates
"broken" on CLI-queued tasks.

Now the initial JSON cliTaskRun writes matches what the WebUI's
"Install" button writes:

  {
    id, command, status: queued,
    createdAt: "<ISO-UTC-with-ms>",
    startedAt: null, completedAt: null, heartbeatAt: null,
    exitCode: null, errorMessage: null,
    type, app
  }

Processor side is unchanged (still adds snake_case overlay on
status transitions — that's how WebUI tasks already work). No JSON
shape change for in-flight tasks.

ALSO (out-of-repo): /home/user/Documents/Scripts/update.sh now restarts
the systemd `libreportal.service` task processor after the docker
`libreportal-service` container restart. Same reason — both pre-load
code at startup, both need a restart to pick up changes. Without this,
deploys silently kept a stale processor running old code while the
disk reflected the new code; the install task-routing recursion I just
saw was a direct consequence.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-27 14:51:13 +01:00
librelad
8a3bf505c3 refactor(config): disperse Features section into category Advanced groups
The Features section was a grab-bag of ~27 toggles, most of which are
either category-specific (firewall, SSL, Docker network, SSH hardening)
or install-time choices that brick the box if flipped on a live
install (the WebUI / config / CLI / Docker requirements). One page
made auditing easier but flattened the risk hierarchy.

Reorganised so each toggle lives where it conceptually belongs, and
the dangerous install-time set is double-gated:

  network_docker     (Advanced)  DOCKER_NETWORK, DOCKER_NETWORK_PRUNE,
                                  DOCKER_SWITCHER
  network_firewall   (Advanced)  UFW, UFWD, WHITELIST_PORT_UPDATER  [new]
  network_domains    (field-Adv) SSLCERTS
  security_ssh       (Advanced)  SSHKEY_DOWNLOADER, SSH_DISABLE_PASSWORDS,
                                  BCRYPT_SAVE, GLUETUN_FOR_ALL          [new]
  general_terminal   (Advanced)  CRONTAB, CONFIGS_CHECK,
                                  CONFIGS_AUTO_UPDATE, CONFIGS_AUTO_DELETE,
                                  MISSING_IPS, CONTINUE_PROMPT,
                                  SUGGEST_INSTALLS, SUGGEST_METRICS
  general_install    (Adv+DEV)   CONFIG, COMMAND, WEBUI, WEBUI_SERVICE,
                                  DATABASE, PASSWORDS, DOCKER_CE,
                                  DOCKER_COMPOSE

The install-time eight are marked **ADVANCED** **DEV** — invisible
unless Developer Mode is on AND "Show Advanced Options" is expanded.
Each field's description was updated to note "Disabling on an existing
install will brick the system" / "install-time choice only" so a user
who does get to the toggle understands the gun before pulling the
trigger.

Other cleanup that fell out:
- Removed `configs/features/` directory entirely.
- Added the two new subcategories to SUBCATEGORY_ORDER in
  network/.category and security/.category.
- Dropped the `category === 'features'` Danger Zone header special-case
  in config-manager.js and its .danger-zone-section--header-only CSS
  variant (sole user).
- Trimmed an obsolete "Edit the features config" notice in
  check_requirements.sh.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-27 14:39:58 +01:00
librelad
3f582120ba feat(cli): route all long-running app + update commands through tasks
Extends the install-routing spike (e5273a4) to every long-running CLI
command, so CLI and WebUI now share one execution path everywhere:

  app install      ← already done
  app uninstall
  app start / stop / restart / up / down / reload
  app backup
  app restore
  update apply
  backup app create   (matches `app backup` — same end target)

Each handler now has the same shape:
  if [[ "$LIBREPORTAL_TASK_EXEC" == "1" ]]; then
      <inline call>            # processor's recursive invocation
  else
      cliTaskRun "<cmd>" <type> <app>   # user invocation: enqueue + follow
  fi

Processor change — crontab_task_processor.sh:
  Adds `export LIBREPORTAL_TASK_EXEC=1` next to LIBREPORTAL_NONINTERACTIVE.
  Universal bypass: every task command the processor runs (CLI-queued OR
  pre-existing WebUI-queued like `libreportal app install adguard`)
  inherits the env var, so the inline branch fires and we never
  re-enqueue. This also lets us drop the env-var prefix the install spike
  was baking into the command string (e5273a4) — cleaner task files +
  one place to think about the bypass.

`backup app schedule` (the cron-driven path that already enqueues via
createTaskFile in backup_app_schedule.sh) is left alone — different
entry point, different runtime context, already correctly task-routed.

Why route the fast ones too (start/stop/restart/up/down):
  Consistency beats the ~1s task-roundtrip latency for a CLI button.
  Locking now serialises a CLI `app stop foo` against a WebUI restart of
  the same app; the audit trail covers every state change. Cheap to
  revert any individually if the latency turns out to bother someone.

Validated live earlier with `libreportal app install dashy` — task file
written, processor dispatched, follower streamed install live, exit 0
propagated. Same machinery now powers the other 9 handlers.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-27 14:38:14 +01:00
librelad
e5273a482d feat(cli): route app install through the task processor + live follower
Spike — closes the gap where the CLI install bypassed the very task system
the WebUI uses. Now both surfaces hit the same path:

  user types `libreportal app install dashy`
    → CLI enqueues a task file in $TASK_DIR (identical shape to the
      WebUI's createTaskFile)
    → pokes $TASK_DIR/.queue.fifo so the processor dispatches in <100ms
      instead of waiting up to IDLE_POLL_SECS
    → CLI tails the task log + polls .status, exits with the task's
      exit_code on terminal state
    → Ctrl-C detaches the follower without killing the task — the
      WebUI's tasks panel keeps showing it

Bypass: the recursive command in the task file is prefixed
`LIBREPORTAL_TASK_EXEC=1 libreportal app install <name>`. The install
branch in cli_app_commands.sh honours that env var by running inline,
which is what the processor's eval invocation hits. No processor
changes — the bypass travels with the task.

Wins:
  - one log file per install, shared by CLI + WebUI (audit trail + replay)
  - locking serialises CLI + WebUI installs (no more two-frontend race)
  - WebUI's "current task" indicator now reflects CLI work too
  - free `--detach` for fire-and-forget queueing

New: scripts/cli/task/cli_task_run.sh
  cliTaskRun <cmd> [type] [app] [--detach]
    Enqueues + follows; --detach prints the task id and exits 0.
  cliTaskFollow <task_id>
    `tail -F` the log + jq-poll the status; returns the task's exit_code.
    Designed to be reused for `libreportal task log <id>` reattach later.

Trade-off: ~200-500ms latency before the first byte (write task file,
processor wakes, opens log, follower starts tailing). Negligible for
install/update/backup — fast commands (list/status/config get) still
run inline. The current branch only changes `app install`; uninstall +
update + backup can be moved on the same pattern once this lands clean.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-27 14:29:30 +01:00
librelad
4d7027258d feat(app): Wave B + C — collapse 28 per-app installers onto generic driver
Finishes the installApp refactor started in d941f59 (Wave A). Every app
whose <app>.sh was either pure boilerplate (Wave B) or boilerplate +
small custom logic (Wave C) now routes through the generic driver in
scripts/app/install/app_install.sh; bespoke logic moved to declarative
hooks in containers/<app>/scripts/<app>_install_hooks.sh.

Net: ~4,000 lines of duplicated 10-step sequence gone. From 31 per-app
.sh files (pre-Wave-A) down to 2 intentional keepers.

DELETED outright (pure boilerplate — driver replaces them identically):
  jellyfin, mastodon, focalboard, ipinfo, speedtest, dashy, invidious,
  nextcloud, ollama, vaultwarden, pihole

DELETED + hook-extracted (small bespoke step preserved in a hook):
  bookstack, moneyapp, owncloud, trilium, searxng, gitea, headscale,
  unbound, prometheus, grafana, gluetun, wireguard, jitsimeet, authelia,
  traefik, adguard, onlyoffice

KEPT (intentional special cases):
  crowdsec      — host-app pattern (no docker compose, runs as apt+
                   systemd via installCrowdsecHost; uninstall/stop/
                   restart hooks already live in this file and are
                   invoked by dockerUninstall/Stop/RestartApp directly).
  libreportal   — WebUI bootstrap. Pre-compose image build + post-install
                   webuiLibrePortalUpdate + bootstrap-time suppression of
                   menuShowFinalMessages don't fit the generic flow.

Driver change — scripts/app/install/app_install.sh:
  Moved monitoringToggleAppConfig "$app_name" "docker-compose.yml" from
  the post-start integrations block into the install body at post-compose
  (right after dockerComposeSetupFile, before docker-compose up). The
  toggle edits the compose file on disk — running it after start meant
  the container had already been brought up with the unmodified compose,
  so the metrics endpoint wouldn't reflect CFG_<APP>_MONITORING until
  the next restart. Matches the original ordering in every per-app .sh
  that used to call it inline.

Hook surface (declare-f-gated, silent no-op when absent):
  <slug>_install_pre              before any install work
  <slug>_install_post_setup       after dockerConfigSetupToContainer
  <slug>_install_post_compose     after dockerComposeSetupFile (+ the
                                  shared monitoring toggle on the compose)
  <slug>_install_post_start       after dockerComposeUpdateAndStartApp
  <slug>_install_message_data     echoes extra argv for menuShowFinalMessages
  <slug>_install_post             very last thing, after the final message
  + the existing _uninstall_pre/_post, _stop_post, _restart_post

Notable extractions:
  bookstack  — _install_post_start: probe :PORT_1/login until 200/302,
               then `bookstack:create-admin` inside the container with
               CFG_BOOKSTACK_ADMIN_{EMAIL,PASSWORD}; falls back to the
               seeded admin@admin.com on timeout.
  adguard    — _install_post_start drives the wizard's HTTP API
               (POST /control/install/configure) so the admin doesn't
               click through five pages, then pins the admin bind back
               to 0.0.0.0:3000 (matches the compose mapping) and health
               checks. _install_message_data echoes user/password to
               menuShowFinalMessages.
  authelia   — _install_pre requirements; _install_post_compose copies
               configuration.yml + users_database.yml, substitutes
               theme/domain/host, generates JWT/session/storage secrets,
               toggles monitoring on configuration.yml; _install_post_start
               argon2-hashes the admin password via the container, writes
               users_database.yml, restarts; _install_post echoes creds.
  traefik    — _install_pre prompts for the LE email if CFG_TRAEFIK_EMAIL
               is unset; _install_post_compose copies static + dynamic
               configs, wires CFG_TRAEFIK_DASHBOARD_ACCESS (local-only /
               domain-only / public), toggles monitoring on traefik.yml,
               then traefikUpdateWhitelist + traefikSetupLoginCredentials.
  wireguard  — _install_pre host-conflict guard (/etc/wireguard/params);
               _install_post_compose persists CFG_WIREGUARD_SUBNET,
               resolves WG_HOST (domain+traefik → host_setup, else IP),
               runs runAppCfg wireguard-ip-forward; _install_post_start
               restarts after wg-easy installs its iptables rules.
  jitsimeet  — _install_post_setup downloads the tagged release zip from
               GitHub; _install_post_compose mass-edits the .env and runs
               gen-passwords.sh; _install_post_start rewrites nginx
               default site to usedport1/2 + restart.
  prometheus — _install_post_compose seeds prometheus.yml under
               $containers_dir/prometheus/prometheus/; _install_post_start
               sets 0777 on storage dirs so the container TSDB can write
               regardless of host UID mapping.
  grafana    — _install_pre requirements; _install_post_start 0777 on
               grafana_storage.
  gluetun    — _install_post_start refreshes the provider snapshot,
               reattaches every routed app (the netns container ID is
               stale after gluetun gets recreated), then prompts to
               onboard any existing apps.
  + the smaller bookstack-shape extractions for owncloud (version scrape),
    trilium / searxng (wait-for-first-boot-config), gitea (Prometheus
    bearer token sync), headscale / unbound (config copy), moneyapp
    (Auth.js AUTH_URL), onlyoffice (compose-resolved user/pass into the
    final message).

Manifest + arrays regenerated. Verified end-to-end:
  - bash -n on every hook file + the driver: clean
  - Each hook file sources cleanly in a subshell, exposes only the
    intended functions, flagged lazy-loadable (not eager)
  - Smoke-stubbed install run for jellyfin (pure), nextcloud (pure),
    bookstack (hooked), crowdsec (kept): correct dispatch in all cases —
    deleted apps route to installApp, kept apps still hit their real
    function

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-27 13:26:49 +01:00
librelad
d941f59388 feat(app): generic installApp driver + dispatcher fallback (Wave A)
The 31 containers/<app>/<app>.sh files each defined install<App>() with
the SAME 10-step sequence — ~4,000 lines of duplicated boilerplate.
Replaces all that with one generic driver + hook surface.

scripts/app/install/app_install.sh:
  installApp <slug> [config_variables]
    — Dispatches on $<slug> (c/u/s/r/i) the same way the per-app .sh
      files did. Same convention; dockerInstallApp's existing
      `declare $app=i` callsite needs no change.
    — Runs the standard sequence: dockerConfigSetupToContainer →
      dockerComposeSetupFile → optional .env copy → fixPermissions →
      dockerComposeUpdateAndStartApp → standard post-install steps
      (appUpdateSpecifics, setupHeadscale, databaseInstallApp,
      webuiContainerSetup, monitoring registration) → final message.
    — Hooks (all declare-f-gated, silent no-op when absent):
        <slug>_install_pre / _post_setup / _post_compose / _post_start
        <slug>_install_message_data   (echoes extra args for menu)
        <slug>_install_post
        <slug>_uninstall_pre / _post
        <slug>_stop_post
        <slug>_restart_post
      Hooks live in containers/<app>/tools/<app>_tools.sh (auto-sourced
      per the modular-per-app-tools convention).

function_install_app.sh:
  When no install<App>() function exists, fall through to
  `installApp <app_name>` instead of erroring. So an app with no .sh
  at all becomes a zero-byte addition — drop in <app>.config +
  docker-compose.yml + <app>.svg, done.

containers/linkding/linkding.sh:
  Deleted (canary). Linkding's body was 100% standard sequence;
  fallback handles it identically. Smoke-tested with stubbed helpers
  — dispatcher fires, generic runs full flow, monitoring integration
  + final-message hook plumbing all intact.

Wave B (next): delete the .sh for every other 'pure-boilerplate' app
(~15 candidates per the survey). Wave C: extract custom logic from
the 7 fat apps into hooks before deleting their .sh.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-27 01:43:08 +01:00
librelad
e72d3d278d style(backup): shorten the BACKUP_STRATEGY tooltip to one practical sentence
The advanced field was carrying a four-clause explanation walking through
"quiesce", "live dump", "stop fallback", and the live-option eligibility
rules — useful background for engineers, overwhelming as a hover tooltip
for the typical user picking from the dropdown.

New tooltip:
  "How the app is prepared before each backup. Automatic is recommended;
   pick Live only if you need zero downtime."

Two lines, plain language, points at the right default. The dropdown's
own labels (Automatic (recommended) / Stop→snapshot→start / Pause→snapshot
→unpause / Live — no downtime) carry the technical specifics; the
tooltip no longer has to.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-27 01:26:27 +01:00
librelad
56d2f8105c fix(config): updateConfigOption uses the right de-sudo helper for the file's tree
Symptom (reported on adguard install):
  sed: couldn't open temporary file /libreportal-containers/adguard/sedZaGX2H:
       Permission denied
  ✗ Error Updated CFG_ADGUARD_BACKUP to true
  ! Notice Non-interactive mode: aborting on error.

Root cause: updateConfigOption ran a raw `sed -i` regardless of which tree
the config file sits in. Works fine for /libreportal-system/configs/*
(manager-owned) but breaks on /libreportal-containers/<app>/<app>.config
(dockerinstall-owned in rootless mode) — sed -i writes its temp file next
to the target, inheriting the directory's perms, and the manager can't
write inside dockerinstall dirs. EVERY app installer that mutates a
CFG_<APP>_* value (the autogenerated random password, BACKUP toggle,
PORT override, etc.) goes through this function, so this was a latent
ticking bomb across all containers.

Fix: pick the helper based on path —
  -  under $containers_dir  → runFileOp  (escalates to
    dockerinstall in rootless, runs as manager in rooted)
  - otherwise                            → runInstallOp (always manager)
Read paths (grep / source) stay unwrapped — both dirs are world-readable;
only the write needs the privilege swap.

Net: no more 'Permission denied' on app installs; the de-sudo pattern is
now respected end-to-end for CFG writes.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-27 01:16:02 +01:00
librelad
061fa5e391 fix(lazy-load): count both braces per line so embedded awk doesn't strand the depth tracker
`webuiSystemUpdate` was calling `webuiSystemMetrics` and getting
"command not found": the lazy-load manifest was missing it (and
`webuiSystemApps`), even though both are defined in
webui_system_metrics.sh.

Cause: the manifest scanner's awk-based depth tracker is line-based.
The first function in that file, `_metricsReadCpu`, contains an
embedded awk script:

    _metricsReadCpu() {
        awk '/^cpu /{        <-- the open brace runs to end-of-line, depth += 1
            ...
        }' /proc/stat        <-- the close brace is mid-line, NOT counted
    }                        <-- depth-- but we started one too high

The increment rule fired for the `/^cpu /{` line (open brace at EOL),
the matching close on the `}' /proc/stat` line was not counted because
the existing depth>0 branch only decremented when stripped was EXACTLY
`}`. Result: depth ended at 1 instead of 0 after _metricsReadCpu, and
every subsequent funcname() header was treated as "still inside a
function" and silently dropped. Same pattern across the ~6 files the
lazy-loader memory called out as "false-positive eager".

Fix: in the depth>0 branch, count open AND close braces per line
symmetrically and apply the net delta. Drops the redundant "exactly
`}`" special-case rule, clamps depth at 0 as a sanity backstop. False
positives from braces inside string literals could under/over-count
locally, but the clamp + the next legitimate `funcname()` header at
depth-0 re-anchors the tracker.

Result on a full regen:
  - 858 → 876 functions indexed (+18 previously-stranded)
  - webuiSystemMetrics + webuiSystemApps now correctly autoloaded
  - eager-file count: 9 → 11 (two files that genuinely have both
    function defs and top-level side effects are now correctly seen
    both as eager AND get their functions indexed — net win on
    every axis)

Verified live: the WebUI updater that was failing with "command not
found" now prints "Updated system information..." cleanly.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-27 00:10:35 +01:00
librelad
ea59d5b268 feat(config): reconcileConfigFile now syncs comments from template (preserves user values)
Existing installs were locked out of template-side description /
options / marker changes because reconciliation kept the user's whole
`CFG_KEY=value     # comment` line verbatim. So new metadata like the
**DEV** marker I'm adding for the developer-mode feature wouldn't take
effect on already-deployed boxes — only fresh installs.

Updated reconcileConfigFile to split each line into value-part and
comment-part. User value is still sacred; the comment (title,
description, [options], **ADVANCED**/**DEV** markers) now comes from
the template. Field renames, label tweaks, marker additions/removals
shipped in a release reach existing installs on the next CLI
invocation (which runs the reconciler).

Specifically unblocks: the developer-mode WebUI feature (CFG_DEV_MODE
field gets added by the existing add-only path; CFG_INSTALL_MODE and
CFG_RELEASE_CHANNEL now pick up their new **DEV** markers and the
'Release - Stable' / 'Bleeding Edge' labels).

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 23:53:21 +01:00
librelad
52e0227bb6 chore(cleanup): retire appGenerate — dead-on-arrival app-skeleton wizard
`libreportal app generate <name>` (and the menu's "g. Generate App" entry)
was broken three independent ways and incompatible with the per-app
architecture the project actually uses now:

  1. Copies from $install_containers_dir/template/ which doesn't exist —
     the only template/ in the tree was in scripts/unused/OLD_CONTAINERS/
     and was never installed into the live tree. cp -r would just fail.

  2. Every sed call used BSD/macOS syntax `sed -i '' -e …`. On Linux
     (every distro this targets) the empty '' becomes a positional file
     argument, so the substitutions never ran. 8 calls, all broken.

  3. Even if it had run, the produced skeleton would have been a
     pre-modular-tools / pre-per-port-subdomain app shape: no tools/,
     no scripts/ subdir, HOST_NAME=test in the .config. Every active
     containers/<app>/ today carries the modular layout the rest of the
     framework expects.

Plus the recent cleanups (the prompt loop fix in 9ffc8e4, the per-port
subdomain refactor in 2e4f420) had been peeling pieces off it without
the root question — does the function still belong? — getting asked.

Delete the whole surface:
  - scripts/app/app_generate.sh (157 lines, the function body)
  - scripts/unused/OLD_CONTAINERS/template/ (the never-installed source
    files appGenerate would have copied — stale enough to still carry
    HOST_NAME=test, CFG_<X>_HOST_NAME, and 248 lines of compose template)
  - menu entry "g. Generate App" + its dispatch in menu_main.sh
  - "generate" case branch in cli_app_commands.sh
  - `libreportal app generate` line in cli_app_header.sh
  - The corresponding entries auto-drop from files_app.sh +
    function_manifest.sh via regen.

New apps are added the way the catalog already grew — by hand-crafting
containers/<app>/{<app>.sh, <app>.config, docker-compose.yml,
tools/<app>.tools.json, scripts/<app>_*.sh}. Copying an existing app's
folder + renaming is the closest thing to a "generator" and it's a one-
command operation.

Net: -556 lines, no behaviour lost (the function never worked).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 23:48:35 +01:00
librelad
9ffc8e4924 chore(cleanup): retire two stale leftovers from the lazy-load + subdomain work
initilize_files.sh's load-comment said "Phase 6 will tackle this with a
precompiled cache file" — but Phase 6 deliberately shipped a different
(simpler) win, and Phase 7 was reconsidered and rejected (the ~30 ms
saving doesn't justify the invalidation complexity across every CFG
write site). Rewrite the comment to describe current behaviour: the two
config scans run because bash can't lazy-load variables, and the
precompile was looked at and dropped.

app_generate.sh had a `read -p "" host_name` inside a `while true` with
no break — anyone who actually ran `libreportal app generate` would
have hung at the hostname prompt forever. The value was then fed into
a `sed 's/HOST_NAME=test/HOST_NAME='"$host_name"'/g'` against the new
app's .config file, but the post-2e4f420 template no longer carries
HOST_NAME=test (per-port subdomains in the PORT row drive routing now).
So the prompt was infinite-looping to gather a value that fed a no-op
sed. Drop both — the function loses no real capability since the
subdomain field is set per-port via the standard PORT row.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 23:42:12 +01:00
librelad
4063283db1 feat(rootless): proper AppArmor profile for pasta network driver
The Debian-shipped passt AppArmor profile (/etc/apparmor.d/usr.bin.passt)
denies the accesses pasta needs to plumb rootlesskit's netns:
  - ptrace_read on the rootlesskit child to enter its user namespace
  - read /run/user/<uid>/dockerd-rootless/netns (the netns file)
  - read /proc/<pid>/net/{tcp,tcp6,udp,udp6} for implicit port forwarding

Without these the rootless docker daemon fails with:
  pasta failed with exit code 1:
  Couldn't open user namespace /proc/<pid>/ns/user: Permission denied

scripts/docker/install/rootless/rootless_apparmor.sh:
  New installRootlessApparmorForPasta() — idempotent fixup.
  1. Adds `include if exists <local/usr.bin.passt>` to the main profile
     (one line; re-adding is a no-op via grep).
  2. Writes /etc/apparmor.d/local/usr.bin.passt with the four rules
     pasta needs. The /local/ pattern is the standard Debian AppArmor
     hook for site-managed overrides — survives `apt upgrade passt`
     because it's outside the package's managed paths.
  3. Reloads via apparmor_parser -r.

Called from installDockerRootless after the override.conf write, gated
on $rootless_net == pasta. slirp4netns installs skip it.

This box was already manually patched while debugging the pasta swap —
the installer-side change makes it idempotent across reinstalls and
applies the same fix on any other host that installs rootless docker
with pasta as the net driver.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 23:13:04 +01:00
librelad
3319614be4 fix(webui): route the migrate.json + peers.json writes through runFileWrite
Two webui-data generators wrote to a temp file via bare `cat > "$temp_file"`
then `runFileOp mv` to the final path. The temp file's path sits inside
$containers_dir/libreportal/frontend/data/<x>/generated/ — owned by the
dockerinstall user (the data plane). The generators run as the manager,
who can't open paths under that tree for write, so every WebUI update
hit:

    webui_backup_migrate.sh: line 125: …/migrate.json.tmp.<pid>: Permission denied
    mv: cannot stat '…/migrate.json.tmp.<pid>': No such file or directory
    webui_peers.sh: line 23: …/peers.json.tmp.<pid>: Permission denied
    mv: cannot stat '…/peers.json.tmp.<pid>': No such file or directory

Pipe the heredoc through `runFileWrite "$output_file"` instead — same
shape the 5 sibling generators in this dir (backup_app_status,
backup_locations, backup_passwords, backup_snapshots, backup_dashboard)
already use. runFileWrite routes the write via the install user that
owns the data tree, so the file lands on disk in one step (no temp +
mv dance needed). The unused `local temp_file=...` declarations dropped
out cleanly.

The trailing `runFileOp chmod 644 "$output_file"` stays — it's the only
guarantee the WebUI container (which reads these files RO) sees them as
world-readable regardless of dockerinstall's umask.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 22:53:10 +01:00
librelad
7513a62fde feat(crowdsec): migrate host-install to a dedicated libreportal-crowdsec helper
CrowdSec's host-side install (the agent + nftables bouncer the LibrePortal
Traefik plugin talks to) had stayed on blanket sudo throughout the rootless +
de-sudo hardening: `sudo apt-get install crowdsec`, `curl | sudo bash`,
`sudo sed -i /etc/crowdsec/config.yaml`, `sudo touch + sudo chmod /var/log/
crowdsec*.log`, `echo $key | sudo tee /etc/crowdsec/traefik_bouncer.key`,
plus `sudo cscli capi register / console enroll / bouncers add`. None of
those are in the scoped LP_HELPERS / LP_SYSTEM sudoers grant the manager
now holds, so any user who enabled crowdsec would have hit hard sudo
failures on every privileged step.

Follow the libreportal-appcfg / libreportal-bininstall pattern: one new
root-owned helper at /usr/local/lib/libreportal/libreportal-crowdsec
that does every privileged op behind a fixed action vocabulary with strict
argument validation. The manager calls in via runCrowdsec — the scoped
sudoers grants exactly one binary, the same trust boundary the other
helpers rely on.

Actions:
  install               apt repo + agent + firewall-bouncer + enable +
                        crowdsecurity/{linux,sshd} collections + reload
                        (idempotent — skips parts already in place)
  services <verb>       enable | disable | restart
  capi <verb>           register | unregister | status
  console <verb>        enroll <token> | disenroll | status
                        token format strictly validated
  bouncer-traefik-init  cscli register + write the manager-owned key file
                        atomically (returns EXISTS or GENERATED:<key>)
  bouncer-priority      bouncer yaml nftables priority → -100
                        (moved from libreportal-appcfg; one helper for
                        every crowdsec root op)
  bind-lapi             flip listen_uri to 0.0.0.0:8080 in config.yaml
  prometheus <on…|off>  flip the prometheus block (validated addr/port)
  touch-host-logs       create + chmod 0644 /var/log/crowdsec*.log so the
                        libreportal container can tail them

Wired in via:
  - new sudoers Cmnd_Alias entry for the helper in LP_HELPERS
  - new helper baked alongside the others by initRootHelpers
    (replaces __SYSTEM_DIR__ / __CONTAINERS_DIR__ / __MANAGER__ at
    install, with safe runtime fallbacks if unbaked)
  - new runCrowdsec dispatch in scripts/docker/command/run_privileged.sh

containers/crowdsec/scripts/crowdsec_install_host.sh now drives the whole
flow through runCrowdsec — every `sudo …` is gone, the compose-toggle sed
uses runFileOp, and the security_crowdsec CFG mirror uses runInstallOp
(configs/ is manager-owned). Net: install script shrinks ~80 lines while
gaining a single auditable trust boundary. crowdsec_fix_priority.sh swung
over to runCrowdsec bouncer-priority too — the appcfg crowdsec_priority
action drops out cleanly.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 22:05:39 +01:00
librelad
18e692ffbb fix(backup): reset HOME when dropping to the backup user in runBackupOp
runBackupOp dropped privileges to $docker_install_user with `sudo -E`,
which preserves the CALLER's environment — including HOME. The caller is
the manager (libreportal), so restic-running-as-dockerinstall ended up
with HOME=/home/libreportal and tried to mkdir
`/home/libreportal/.cache/restic` for its cache. dockerinstall can't
write into libreportal's home, so every backup ran with:

    unable to open cache: mkdir /home/libreportal/.cache/restic: permission denied

twice (once in backup, once in the verify-via-scratch-restore step), with
restic falling back to a no-cache run that's a few × slower than it
should be.

Add `-H` (sudo's "reset HOME to target user's home"). Now restic sees
HOME=/home/dockerinstall, creates ~/.cache/restic there (dockerinstall
owns its own home, no help needed), and the warning is gone. Confirmed
live: a `backup app create linkding` round-trip is silent on cache, and
the dir lands at /home/dockerinstall/.cache/restic, mode 0700, correctly
owned.

All restic/borg/kopia calls funnel through runBackupOp, so this single
character fix covers every backup-tool invocation.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 21:56:43 +01:00
librelad
3d09105ed5 fix(update): define backup_install_dir + correct gitCleanInstallBackups find
Three small bugs in the legacy git-update flow that all hung off the
same never-set variable:

1. backup_install_dir was referenced in 4 files (reset_git_backup,
   install_git_backup, use_git_backup, config_git_check) but DEFINED
   nowhere — never has been, in any branch or tag. Resolved to "", so
   "$backup_install_dir/$backupFolder" became "/backup_<ts>" (filesystem
   root, perm denied). Add it to libreportalDerivePaths beside the other
   roots, point it at $backup_dir/install (a sibling of restic's per-
   location subdirs at $backup_dir/<idx>), and add it to initFolders so
   it exists on first install.

2. gitCleanInstallBackups' find expression was
       find ... -mindepth 1 -type f ! -name '*.zip' -o -type d ! -name '*.zip' -exec rm -rf {} +
   `-o` binds looser than the implicit -a, so the -exec only applied to
   the second clause. That meant: every non-.zip DIR anywhere under the
   tree got deleted; every non-.zip FILE got matched and ignored. Even
   once $backup_install_dir resolved correctly the cleanup would've
   wrecked unrelated dirs.
   Collapsed to `-mindepth 1 -maxdepth 1 ! -name '*.zip' -exec rm -rf {} +`
   — direct children of $backup_install_dir, kill everything that isn't
   a zip, let -rf take care of the dirs. Synthetic-tree smoke test
   confirms only the .zip files survive.

3. use_git_backup.sh had a typo'd doubled var:
       copyFolders "$backup_install_dir$backup_install_dir/$backup_file_without_zip/" ...
   Reduced to the single $backup_install_dir/$backup_file_without_zip/.

All three only fire in the manual-update path (libreportal update apply
under git/local install mode); the install-blocking path is unaffected.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 21:47:12 +01:00
librelad
4fc155acfa chore(cleanup): delete the orphan tagsProcessorStandardReplacements
Tree-wide audit (working tree + deployed install + every local/remote ref
+ every reachable commit + unreachable objects via git fsck) found zero
external callers. Existed dead since v0.1.0 — never wired in.

The function set DOMAINSUBNAME, TIMEZONE, DOCKER_NETWORK (all duplicates
of fills that happen elsewhere) plus the two unique-to-it CONFIGS_DIR_TAG
+ CONTAINERS_DIR_TAG. Those two are already wired directly into the
standard tag-fill block in dockerConfigSetupFileWithData (commit 521f08b),
so dropping the source file leaves no behavioural gap.

Also tighten the comment that explained why we inlined the two tags —
don't reference the function we're deleting in the same change. Describe
the current behaviour, not the history (per repo convention).

Regenerated the auto arrays + function_manifest.sh: the 3 stale entries
referencing this function drop out cleanly. files_cli.sh / files_config.sh
/ files_source.sh also rebuilt — no net content change beyond dropping
this one path.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 21:41:24 +01:00
librelad
c6d92bbc58 perf(backup): drop sudo overhead from sourceBackupLocations (Phase 6)
The location_loader's find ran through runFileOp (sudo -u libreportal),
which forks a sudo shell purely for the find call. configs/backup/
locations/ is already manager-owned (not in the dockerinstall-owned
containers tree), so the sudo step adds ~50 ms of process-creation
overhead for zero security benefit.

Plain `find` now, with a tiny [[ ! -r || ! -x ]] guard that falls
back to runFileOp if someone relocates the dir to a non-traversable
location. Same observable behaviour, ~50 ms saved per CLI invocation.

This is the simpler half of Phase 6 — the libreportal_configs scan
itself was already plain-find (only ~11 ms total for 22 files). The
remaining costly scan is app_configs against /libreportal-containers/,
which legitimately needs sudo because dockerinstall owns that tree.
Precompiling its content is possible but adds invalidation complexity
(updateConfigOption writes happen at runtime) — deferred for now;
better return on simpler interventions.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 21:39:22 +01:00
librelad
f07ec0e358 fix(lazy-load): exclude function_manifest.sh from the early find-loop
The early loop in sourceInitilize() sources every .sh under source/files/
recursively — including the new arrays/function_manifest.sh, which now
carries ~860 autoload stub definitions (~50 ms parse cost). Even in
eager mode where lazy infrastructure is never touched, every invocation
was paying that cost up front.

The manifest is only needed in lazy mode, where it's sourced explicitly
at the top of the lazy branch. Excluding it from the early loop:
  - Eager mode: drops the ~50 ms regression introduced by Phase 5.
  - Lazy mode: unchanged — the explicit source still runs.

This brings eager back to the pre-Phase-5 baseline and lets the lazy
container-stub gain (skipping sourceScanFiles containers, ~70 ms) show
through as a real saving.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 21:32:52 +01:00
librelad
77342c8047 feat(lazy-load): extend manifest to containers/ + skip container scan (Phase 5)
Containers used to be eager-loaded via `sourceScanFiles "containers"`
even under LP_LAZY=1 — sourcing all ~160 installer functions up front.
Phase 5 brings them into the autoload-stub mechanism.

generate_function_manifest.sh now scans BOTH scripts/ AND containers/
(maxdepth 3, matching sourceScanFiles' existing prune), with a per-entry
root selector so stub emission uses the right base directory:

  scripts/peer/peer_add.sh    →  source "${install_scripts_dir}peer/peer_add.sh"
  containers/linkding/linkding.sh →  source "${install_containers_dir}linkding/linkding.sh"

New manifest exports:
  LP_FN_MAP             funcname → relpath        (existing)
  LP_FN_ROOT            funcname → scripts|containers   NEW
  LP_EAGER_FILES        "<root>:<relpath>" entries     NEW format
  ~860 autoload stubs   (was ~700; +160 from containers)

Loader changes (initilize_files.sh):
  - Parses LP_EAGER_FILES entries as `root:path`, dispatches to the
    right install_*_dir. Pre-Phase-5 entries without a colon default to
    scripts (backwards-compatible).
  - sourceScanFiles "containers" is skipped when LP_LAZY=1 AND
    LP_FN_MAP is loaded (manifest-driven autoload covers it).
    Eager mode and lazy-with-missing-manifest both still run the scan.

Measurement target: ~70 ms saved on top of Phase 4. Verified separately
in the commit message of the next deploy.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 21:30:36 +01:00
librelad
dba63873f5 feat(lazy-load): flip CLI to LP_LAZY=1 by default (Phase 4)
scripts/source/load_sources.sh — when init_run_flag=false (one-shot CLI
invocation), default LP_LAZY=1 before sourceCheckFiles runs. Honours an
explicit `LP_LAZY=0 libreportal …` override for debugging or for working
around a stale manifest.

Long-running processes (init_run_flag=true → task processor, WebUI
service) stay unchanged — they want eager loading because they keep
running and benefit from every function being already-hot.

Measured on this box, 3 runs each:
  EAGER (default before this commit):
    real 0.91s / 1.12s / 0.94s   avg ~0.99s wall
    user 0.40s / 0.41s / 0.42s   avg ~0.41s CPU
  LAZY (new default for CLI):
    real 0.63s / 0.65s / 0.66s   avg ~0.65s wall
    user 0.26s / 0.27s / 0.26s   avg ~0.26s CPU

Wall: ~340ms saved per invocation (34%).
User: ~150ms saved per invocation (37%).

Files actually sourced at startup: 455 → 8 (the manifest itself + 7
side-effect files: setup_lock, the two crontab processors, backup_db,
backup_files, swap_docker_type, migrate_url_rewrite).

Safety nets:
- Missing manifest auto-falls-back to eager loading (init.sh check
  in the lazy branch sets LP_LAZY=0 if function_manifest.sh is absent).
- Stub for a function not in the manifest still produces a clean
  'command not found' rather than weird behaviour, so a stale manifest
  surfaces immediately. `libreportal regen arrays` regenerates both
  files_*.sh and function_manifest.sh.

Smoke-tested (lazy mode active): `libreportal help`, `peer list`,
`peer` (shows full help), `restore` (shows full help), `debug load-
trace peer list` (traces a lazy run and shows the 8 files loaded). All
output identical to eager mode.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 20:56:25 +01:00
librelad
3b410fe6d9 fix(lazy-load): skip manifest itself + files_*.sh arrays from scan
The previous run had 32 eager files; 24 of them were the auto-generated
files_*.sh arrays (only useful to the eager loader) + the manifest
itself (which the lazy loader sources explicitly). Eager-sourcing them
under lazy mode was pure overhead — ~55ms on the manifest alone (it was
being parsed twice, once via the explicit lazy-loader source and once
via the LP_EAGER_FILES loop).

Down to 8 eager files (the genuinely-side-effecting ones: setup_lock.sh,
the two crontab task processors, backup_db.sh, backup_files.sh,
docker swap_docker_type.sh, migrate_url_rewrite.sh, cli_debug_commands.sh).

The files_*.sh arrays are still sourced by the eager loader's existing
path — that's unchanged. Lazy mode just doesn't need them because it
never iterates files_libreportal_app[@] / files_libreportal_cli[@].

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 20:54:38 +01:00
librelad
c68254ad70 feat(lazy-load): dual loader with LP_LAZY=1 opt-in (Phase 3)
scripts/source/loading/initilize_files.sh gains an LP_LAZY=1 branch:
  - Sources scripts/source/files/arrays/function_manifest.sh once. The
    manifest defines LP_FN_MAP, LP_EAGER_FILES, AND ~700 autoload stubs
    (precompiled by the generator — one parse cost vs evaluating 700
    snippets at startup).
  - Eager-sources every file listed in LP_EAGER_FILES (top-level side
    effects: variable assignments, source calls, bare commands). These
    can't safely be deferred — they'd skip the side effect, not just the
    function definition.
  - Skips the bulk loop that sources every files_to_source[@] entry.

Default behaviour (LP_LAZY unset or 0) is byte-identical to the previous
loader — every file gets eager-sourced up front. Long-running processes
(WebUI service, task processor) leave LP_LAZY unset because their first
call to anything wants the function already hot.

Each autoload stub looks like:
  funcname() {
    source "${install_scripts_dir}path/to/file.sh"
    funcname "$@"
  }

First call sources the real file, which redefines the function with the
real body; the stub's trailing `funcname "$@"` then calls the freshly-
defined real implementation. Sourcing the file also redefines stubs for
any sibling functions the same file declares, so they don't re-source.

Safety nets:
- Missing manifest → fall back to eager loading (`export LP_LAZY=0`).
  No regression risk if someone enables LP_LAZY=1 on a stale install
  whose regen never ran.
- LP_LOAD_TRACE=1 still works in lazy mode — it records the manifest
  parse + each eager file (tagged LAZY-manifest / LAZY-EAGER) so Phase 4
  can measure the actual saving.

No automatic flip yet — this commit only adds the path. Phase 4 will set
LP_LAZY=1 by default for the CLI entrypoint (and re-measure with the
trace tool from Phase 1).

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 20:51:24 +01:00
librelad
7a66801ead feat(lazy-load): function manifest generator + lpRegen wiring (Phase 2)
scripts/source/files/generate_function_manifest.sh — scans every .sh in
scripts/ (skip-list matches generate_arrays.sh, plus excludes peer_shell.sh
which is a standalone forced-command target) and emits
scripts/source/files/arrays/function_manifest.sh:

  declare -gA LP_FN_MAP=(
      [acquireSingletonLock]="crontab/task/crontab_task_processor.sh"
      [adoptDockerSubnet]="checks/requirements/check_docker_network.sh"
      ...                                  # 698 entries
  )
  LP_EAGER_FILES=(
      "backup/db/backup_db.sh"
      "source/files/arrays/files_app.sh"
      ...                                  # 32 entries (~7% of files)
  )

The lazy loader (Phase 3) consumes LP_FN_MAP to install autoload stubs of
the form `name() { source "$LP_FN_MAP[name]"; name "$@"; }`. First call
sources the real file, which redefines the stub with the real body;
subsequent calls hit the real one. LP_EAGER_FILES enumerates files with
top-level side effects (variable assignments, source calls, bare commands
outside any function) — those MUST always source so the side effects fire.

Heuristic correctness, in order of importance:
- Function header detection requires EMPTY parens (`name()`), not just
  `name(` — otherwise lines like `if (...)`, `for (...)`, `while (...)`
  in embedded awk/perl get misread as bash function defs.
- Handles three function styles: `name() {` (same line), `name()\n{`
  (LibrePortal convention), and one-liners `name() { body; }`.
- Tracks { } balance for inside-function depth, with the safe fallback that
  ambiguous cases get marked eager (false positive = no behaviour change;
  false negative would skip a needed source).
- Files containing embedded awk/perl with their own { } blocks (about 6 of
  them: cli_debug_commands.sh, crontab_task_processor.sh, backup_db.sh,
  backup_files.sh, etc.) get false-positive flagged eager — acceptable
  because they just stay eager-loaded, matching today's behaviour.
- Collisions report to stderr (last-write wins, same as eager-load
  semantics); no collisions found in the current tree.

Wiring:
- lpRegenArrays (`libreportal regen arrays`) now also runs the manifest
  generator when the existing arrays need regen, keeping the two in sync.
- update.sh's quick-deploy regen step does the same after copying files
  to the live install. Best-effort: failures don't abort because lazy
  loading is opt-in (LP_LAZY=1) in Phase 3 and not the default yet.

Scanned: 454 files, indexed 698 function definitions, 32 eager (9 real,
23 auto-generated arrays + the manifest itself). 0 name collisions.

No behaviour change in this commit — the manifest is just data the loader
in Phase 3 will use. The default eager loading path is untouched.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 20:47:54 +01:00
librelad
a4d3b78cdb feat(debug): LP_LOAD_TRACE + 'libreportal debug load-trace' (lazy-load Phase 1)
First step toward an autoload-style lazy loader for the 499-file source
tree (current cold load ~1s wall / 340ms user-time per CLI invocation,
mostly spent sourcing files the command never calls). This commit only
measures — no behaviour change unless LP_LOAD_TRACE=1.

LP_LOAD_TRACE=1 instrumentation (scripts/source/loading/initilize_files.sh):
  Wraps each  in the main file-list loop with EPOCHREALTIME
  before/after, writes `<elapsed_ms>\t<file_relpath>` to
  $LP_LOAD_TRACE_FILE (default /tmp/libreportal-load-trace.<pid>.log).
  Zero overhead when the env var is unset (one [[ test per file).

libreportal debug load-trace [cmd...]:
  New `debug` CLI category. Spawns a child `libreportal <args>` (default
  'help') with LP_LOAD_TRACE=1, then awk-aggregates the trace: wall vs
  cumulative source time, file count, top-15 hottest files. The diff
  between wall and cumulative-source = bash startup + dispatch + the
  command's own work.

Used in the next phases to (a) validate that the lazy loader actually
delivers the speedup we expect and (b) flag any single file that hogs
disproportionate time (rare `heredoc | sed | base64` style work at
source time would show up here as a >10ms entry).

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 20:33:22 +01:00
librelad
dc77ddaa4c feat(linkding): add full per-app tools (5 user-management actions)
Linkding has shipped without any Tools-tab actions since v0.1.0 — the only
artifact was scripts/menu/tools/manage_linkding.sh, a dead legacy CLI menu
referencing an `appLinkdingSetupUser` function that was never defined.
Build the real thing, mirroring bookstack's pattern (manifest + thin tool
wrappers + auth_adapter that drives the app's native admin shell):

  containers/linkding/tools/linkding.tools.json    — manifest, 5 tools
  containers/linkding/tools/linkding_<id>.sh       — one wrapper per tool
  containers/linkding/scripts/linkding_auth.sh     — Django shell driver

Tools (all category=users, so the WebUI's custom user-list panel and its
row-level 🔑 / 👑 / 🗑 buttons light up):

  reset_password   — set_password on an existing user (random if blank)
  create_account   — create_user / create_superuser
  list_users       — emits EZ_USER\t<username>\t<username>\t<role> rows
                     (linkding is username-primary, so username goes into
                     both display slots — keeps the panel click-through
                     identifier consistent with the other tools' fields)
  delete_user      — delete by username (destructive, confirm gated)
  set_admin        — toggle is_superuser + is_staff

Implementation runs entirely inside the linkding-service container via
`runFileOp docker exec ... python manage.py shell -c "<code>"`, reading
inputs through `-e` env vars so quoting stays safe. Django's default
get_user_model() User is used directly — passwords hash exactly the way
the web UI does, admin flags map to the same fields the UI reads.

Also drop the dead legacy stub (scripts/menu/tools/manage_linkding.sh)
and regenerate files_menu.sh so the source-scan no longer pulls it in.
Nothing referenced linkdingToolsMenu — verified by tree-wide grep.

Verified live on dev-ai (Debian 12, linkding installed, Django 5 + sqlite):

  $ libreportal app tool linkding create_account 'username=alice|password=…|admin=true'
  ✓ Linkding user created — Username: alice — Password: …

  $ libreportal app tool linkding list_users ''
  EZ_USER  alice  alice  admin

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 20:23:56 +01:00
librelad
82f64eb5c0 feat(migrate): app-specific hooks + peer friendly-name overlay (Phase 4)
Polish pass for the migration system. Two concrete additions; the live-mirror
and full drift-verify ideas from the original plan are intentionally
deferred — both need real-world test data to land correctly, and the kernel
already exposes everything they'd need.

Per-app migrate hooks (scripts/migrate/migrate_hooks.sh):
  Apps can declare two optional functions in their tools.sh (already
  auto-sourced per [[libreportal-modular-app-tools]]):

    <app>_migrate_pre()   — runs before stop+wipe
    <app>_migrate_post()  — runs after restart, before the user sees it

  Each receives:
    $1 = source identifier (peer name or backup-tag hostname)
    $2 = transport ("restic" | "direct-ssh")

  migrateRunHook() is now called from both migration apply paths:
    - migrate_apply.sh (restic-mediated, shared backup channel)
    - peer_pull.sh    (direct-SSH, peer-shell stream)

  Use cases: rotate federation keys after a Mastodon move, regenerate
  OIDC client secrets, drop SaaS-style locks, fix hostname-baked configs
  the URL-rewrite layer doesn't cover.

  Hooks are optional — apps without them inherit the standard flow.
  Failed hooks emit a non-fatal notice (the rest of the migrate still
  reaches 'done') so a single bad hook can't strand an otherwise-working
  app in stopped state.

Peer friendly-name overlay (Migrate tab):
  Was deferred from Phase 2 because it required Phase 3's UI to feel
  cohesive. BackupPage.refreshAll() now also fetches peers.json and builds
  a hostname → peer-name lookup. renderMigrate() shows
      'homelab (host: homelab.lan)'
  for any backup-channel peer that matches the source host, and falls back
  to the bare hostname when no peer is defined. Same data, friendlier UI.

Skipped (genuinely deferred, not just out of time):
  - Live mirror / warm-standby (continuous one-way sync). Needs a scheduler
    + drift-state to track. Right place for it is a separate feature on top
    of the existing kernel rather than bolted onto migrate.
  - Drift-verify ("what would change if I migrated?"). Cheap to write but
    needs a real cross-host pair to validate against — adding it untested
    would just be theatre.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 18:00:26 +01:00
librelad
3fe2c0660a feat(peers): direct peer SSH — pairing + peer-shell + pull (Phase 3)
End-to-end direct-ssh-direct: two LibrePortal instances exchange pairing
tokens, each authorizes the other to call a locked-down peer-shell dispatcher
via SSH forced-command, then either side can pull live app data from the
other without needing a shared backup repo.

Push and Connect-via-relay are deferred — push is symmetric to pull (same
forced-command, opposite verb), and the relay variant waits for Connect to
actually exist (config_json + kind enum already future-proofed in Phase 2).

Key generation (peer_key.sh):
  One ed25519 keypair per install at ~<manager>/.ssh/libreportal-peer{,.pub}.
  Generated lazily on the first peer-related call. Used as our outbound
  SSH identity AND as the pubkey other instances authorize.

Forced-command dispatcher (peer_shell.sh):
  Standalone script, deployed by peerInstallShell() to
  ~<manager>/.local/bin/peer-shell. authorized_keys entries look like:
    command="~/.local/bin/peer-shell <peer-name>",no-pty,no-port-forwarding,
    no-X11-forwarding,no-agent-forwarding,no-user-rc ssh-ed25519 AAAA… peer:<name>
  sshd hands us $SSH_ORIGINAL_COMMAND; we parse, whitelist the verb, and
  refuse anything else. Verbs:
    ping        Liveness probe (JSON ok:true).
    list-apps   JSON {peer, apps:[{slug, size_kb}]}.
    stream-app  tar of containers_dir/<slug> to stdout (slug strictly
                validated — lowercase alnum+dash; rejects path traversal).
  Audit log appended to ~/.local/state/libreportal/peer-shell.log. Excluded
  from the generated source arrays (would crash any sourcing shell on empty
  SSH_ORIGINAL_COMMAND); generate_arrays.sh skip-list extended.

Pairing token (peer_pairing.sh):
  Format: lp-peer|v1|<name>|<user>|<host>|<port>|<base64-pubkey>|<fingerprint>
  Pipe-delimited because the SHA256 fingerprint and base64 pubkey both
  contain ':'. peerPairingParse decodes + re-derives the fingerprint from
  the actual key, refusing tokens with mismatched fingerprints (catches
  truncation / tampering). peerPairingAccept:
    1. Installs peer-shell (peerInstallShell).
    2. Appends to authorized_keys with the lockdown options above.
    3. Inserts a peers row (kind=direct-ssh-direct, config carries host,
       port, user, fingerprint).
  Symmetric — user runs accept on BOTH sides with the other's token to
  enable bidirectional calls.

Outbound SSH (peer_remote.sh):
  peerExec <name> <verb> [args] — looks up the peer's connection config and
  ssh's in with the right key, BatchMode + ConnectTimeout + accept-new for
  the host key. peerPing wraps it and updates peers.status + last_seen.

Pull-an-app (peer_pull.sh):
  peerPullApp <peer> <app> [--no-pre-backup] [--keep-urls]
    1. peerPing (refuse if unreachable).
    2. migratePreBackupDestination (reuses the Phase 0 safety wrapper —
       same restic-tagged pre-migrate snapshot as the backup-channel flow).
    3. Stop + wipe destination's app folder.
    4. peerExec stream-app | tar -x (pipefail; bails on partial transfers).
    5. migrateApplyUrlRewrite + dockerComposeUpdateAndStartApp install
       (URL repointing, idempotent install path).
    6. dockerComposeUp + post-restore hooks.
  Identical Stage-2..6 to migrateApplyApp; only the data source differs
  (tar-over-SSH instead of restic-restore).

CLI (cli_peer_commands.sh + header):
  libreportal peer token                — emit this host's pairing token
  libreportal peer pair <token> [name]  — accept a token (override name)
  libreportal peer apps <peer>          — live peer-shell list-apps
  libreportal peer pull <peer> <app> [--no-pre-backup] [--keep-urls]

WebUI (/peers):
  Header gains 'Show my token' and 'Pair with token' buttons (both open
  modals around the matching CLI verbs). Token modal warns the user that
  the token is credentials. Pair modal accepts a free-form override name.
  Direct-SSH peer cards gain a 'List apps' button that opens an inline
  drawer showing the peer's live app inventory (via peer apps) with per-
  app 'Pull' buttons. Pull modal has the same two safety toggles as the
  Migrate tab (pre-backup ON, URL rewrite ON by default).
  Backup-channel manual-add modal kept; direct-SSH must use the token flow.

Smoke-tested:
  - All 16 peer-subsystem functions register without crashing the shell.
  - peer-shell ping ⇒ {ok:true}; unknown-verb refused; path-traversal slug
    refused; valid-slug streams.
  - Token emit→parse round-trip preserves every field; garbage rejected
    with not-a-token; v99 rejected with unsupported-version.
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 17:56:57 +01:00
librelad
763092a278 fix(wireguard): move /etc IP-forward edit into libreportal-appcfg
The standalone WireGuard install used to flip net.ipv4.ip_forward by
appending+uncommenting `/etc/sysctl/99-custom.conf` via blanket sudo
(sudo tee, sudo sed, sudo sysctl -p). Two problems with that on a
de-sudoed manager:
  - The path is non-standard. The conventional location is
    /etc/sysctl.d/*.conf (drop-ins, loaded by sysctl --system) — the
    old file may not even exist, leaving forwarding silently off.
  - `sudo tee /etc` and `sudo sed -i /etc` are not in LP_SYSTEM. The
    manager has lost the broad sudo it once had, so this would now
    fail outright on every wireguard install.

Add a `wireguard-ip-forward` action to libreportal-appcfg that:
  - writes /etc/sysctl.d/99-libreportal-wireguard.conf (a drop-in we
    own and rewrite idempotently), and
  - reloads via `sysctl --system` (with a `sysctl -p <dropin>` fallback).

containers/wireguard/wireguard.sh now calls `runAppCfg wireguard-ip-forward`
through the existing helper-dispatch path — the whole edit runs as root
in one validated step, no `sudo` in the per-app script.

Same de-sudo pattern as adguard-auth / crowdsec-priority / owncloud-config
already use.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 17:48:43 +01:00
librelad
1014dd6e42 feat(peers): introduce 'Peer' as a first-class concept (Phase 2)
A peer is a named reference to another LibrePortal instance. Phase 2 only
implements kind=backup-channel (friendly label over a hostname that shows
up in a shared backup repo); direct-ssh-direct and direct-ssh-via-relay
(Connect's blind-relay) are reserved enum values for Phase 3.

DB schema (db_create_tables.sh):
  CREATE TABLE peers (
    id           INTEGER PRIMARY KEY AUTOINCREMENT,
    name         TEXT UNIQUE NOT NULL,
    kind         TEXT NOT NULL DEFAULT 'backup-channel',
    config_json  TEXT NOT NULL DEFAULT '{}',
    status       TEXT DEFAULT 'unknown',
    last_seen    TEXT,
    created_at   TEXT DEFAULT CURRENT_TIMESTAMP
  );
  + indexes on name and kind.

  config_json is kind-specific so new transports don't need a schema
  migration. For backup-channel it carries {"hostname":"","loc_idx":N}.

Bash module (scripts/peer/):
  peer_helpers.sh   _peerDb, peerSqlEscape, peerValidateName/Kind.
  peer_add.sh       peerAdd <name> <kind> [k=v ...] → INSERT, refresh
                    generator. Rejects unimplemented kinds early so users
                    don't create dead-end peer records.
  peer_remove.sh    peerRemove <name> → DELETE.
  peer_list.sh      peerList → JSON array; peerGet, peerNameForHostname
                    (reverse-lookup for the migrate-tab overlay).
  peer_check.sh     peerCheckReachable, peerCheckAll. For backup-channel
                    'reachable' = at least one snapshot from that hostname
                    visible in (preferred|any enabled) location. Updates
                    status + last_seen so UI dots render without re-probing.

CLI (scripts/cli/commands/peer/):
  libreportal peer list
  libreportal peer get <name>
  libreportal peer add <name> backup-channel hostname=<host> [loc_idx=<n>]
  libreportal peer remove <name>
  libreportal peer check [name]

  Auto-routed by cli_initialize.sh's category-discovery.

WebUI data generator (scripts/webui/data/generators/peers/webui_peers.sh):
  Emits data/peers/generated/peers.json with the peerList output and a
  generated_at envelope. Hooked into webuiLibrePortalUpdate alongside the
  backup generators.

Frontend:
  - New top-level /peers route in spa.js (PeersPage class, peers-content.html).
  - 'Peers' nav item in the topbar between Backups and the right-side controls.
  - Add-peer modal with friendly-name + kind + hostname + preferred-location
    selector (populated from the existing backup-locations data).
  - Per-peer card with status dot, last-checked time, Check + Remove buttons.
  - Phase 3 kinds appear in the kind dropdown as disabled options so users
    can see what's coming.

Source-array wiring:
  - generate_arrays.sh auto-created files_peer.sh from the new peer/ dir.
  - cli_files.sh + app_files.sh include ${peer_scripts[@]} alphabetically.
  - files_webui.sh auto-picked-up the new peers/ generator subfolder.

The migrate-tab friendly-name overlay (use peer names in /backup/migrate
when a peer record exists for a hostname) is intentionally deferred — it's
a 5-line frontend lookup once peers.json is loaded; cleaner to add after
Phase 3 ships its peer-detail view.

Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 17:43:56 +01:00
librelad
1f930cca74 fix(install): route the early .env tag substitutions through runFileOp
configFileSetupData runs as the manager (libreportal user) during install,
but writes into /libreportal-containers/<app>/, which is owned by the
container user (dockerinstall) under the three-root layout. The six bare
`sed -i` calls in this function were missing the `runFileOp` wrapper that
every other in-tree sed-on-app-files call already uses (e.g. setup_dns.sh's
WG_DEFAULT_DNS edits), so on first run `sed -i` failed to create its temp
file in the live dir:

    sed: couldn't open temporary file /libreportal-containers/linkding/sedaCaUNU: Permission denied
    ✗ Error Updated DOMAINSUBNAMEHERE with: bookmark.
    ! Notice Non-interactive mode: aborting on error.

…which aborted the install at step 3 of every per-app config setup.

Replace `result=$(sed -i ...)` with `result=$(runFileOp sed -i ...)` so each
substitution runs as the owner of the target file (via the bin-install
helper). All six call sites use the same pattern — done as a single
`replace_all` over the unique prefix.

Tags fixed: DOMAINSUBNAMEHERE, APPADDRESSHERE, DOMAINSUBNAME_DATA,
TIMEZONE_DATA, EMAILHERE, HOSTIPHERE.

Verified live on a fresh install: `libreportal app install linkding` now
completes cleanly through all 10 install steps and lands the container.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
2026-05-26 17:36:51 +01:00