LibrePortal/docs/DEVELOPMENT.md

# LibrePortal — Development & Releases

How to run a development copy, cut stable/edge releases, and test them before they
go out. For installing/using LibrePortal, see [USER.md](USER.md).

## Mental model (read this first)

**Install modes** — `CFG_INSTALL_MODE` decides where the code comes from:

| Mode | Source | Use |
|---|---|---|
| `release` (default) | a checksum-verified `.tar.gz` over HTTPS | end users / stable |
| `git` | `git clone` of the repo | contributors tracking a branch |
| `local` | a copy of a local folder | hacking on the code on the box |

**Three roots** (each relocatable at install, then fixed):
`<system>` (manager-owned: configs/db/logs/install) · `<containers>` (container
user: app data) · `<backups>` (container user: repos). Defaults `/libreportal-*`.

**Two users:** the **manager** (`sudo_user_name`, default `libreportal`) owns the
control plane and runs the runtime; the **container user** (`CFG_DOCKER_INSTALL_USER`,
default `dockerinstall`) owns app data + runs rootless Docker. Genuine-root actions
go through fixed, root-owned helpers in `/usr/local/lib/libreportal/` (paths +
manager name are *baked* into them at install — never read from runtime config).

**Key files:**
- `init.sh` — the installer (self-contained; creates users/folders/helpers, bakes things).
- `install.sh` — the thin bootstrap (download+verify+extract a release, then run init.sh).
- `scripts/source/paths.sh` — resolves the three roots + manager user.
- `scripts/source/fetch.sh` — `lpFetchRelease` / `lpVersionGt` (runtime fetch + version compare).
- `scripts/release/make_release.sh` — builds release artifacts.
- `VERSION` — the single source of the version number.

## Run a development copy

From a clone of the repo, on a **throwaway Debian/Ubuntu host** (install is
destructive — it creates system users and dirs):

**Local mode** (install from the working tree — best for hacking):
```bash
sudo ./init.sh --random-password --local init
# custom locations work in dev too:
sudo ./init.sh --random-password --local --system-dir=/srv/lp --manager-user=lpadmin init
```

**Git mode** (track a branch):
```bash
# via the bootstrap:
sudo ./install.sh --git-url=https://example.com/you/LibrePortal.git \
                  --git-user=USER --git-token=TOKEN
# or directly:  ./init.sh init <password> <git_user> <git_token> <git_url> true git
```

**Iterating:** re-run the installer to redeploy after changes (local mode re-copies
the tree). To wipe and start over:
```bash
sudo ./init.sh uninstall                    # removes the three roots + users + footprint
sudo ./init.sh --skip-docker-images uninstall   # keep the Docker layer for a fast reinstall
```

> This repo's CI/hook setup may auto-deploy on commit (commit on a branch →
> auto-merge → redeploy). That's environment-specific; the commands above are the
> portable way to stand up and refresh a dev box.

## Cut a release (stable or edge)

1. **Bump the version** in `VERSION` (semver, e.g. `0.2.0` → `0.3.0`). Commit it.
2. **Build the artifact** (uses `git archive`, so it ships only committed files and
   honours `.gitattributes export-ignore` — `scripts/unused`, `site`, `docs`,
   `.claude`, the release tooling, etc. never ship):
   ```bash
   scripts/release/make_release.sh stable        # or: edge   [git-ref]
   ```
   Produces, under `dist/<channel>/`:
   - `libreportal-<version>.tar.gz` — the release
   - `libreportal-<version>.tar.gz.sha256` — its checksum
   - `latest.json` — `{ version, channel, url, sha256, notes }` (the channel pointer)
3. **Publish** by serving `dist/<channel>/*` at `https://get.libreportal.org/<channel>/…`.
   The host (the `getlibreportal` + `weblibreportal` apps) lives in the separate
   **LibrePortal-Infra** repo, which overlays onto an install and picks these up via
   its `publish.sh`. `latest.json` is what makes a version "the latest".

**Channels:** `stable` is the default users get; `edge` is for early/testing
builds. Same tooling, different `<channel>`. To promote an edge build to stable,
rebuild with `make_release.sh stable` at that ref (or copy its artifacts into the
`stable/` path and update `stable/latest.json`).

## Test a release locally before publishing

No hosting needed — serve `dist/` and point an install at it:

```bash
# build, then serve the artifacts
scripts/release/make_release.sh stable
( cd dist && python3 -m http.server 8000 )

# on a throwaway host, install from your local server:
LP_RELEASE_BASE_URL=http://<your-ip>:8000 \
  sudo ./install.sh --channel=stable --system-dir=/libreportal-system
```

`LP_RELEASE_BASE_URL` overrides the release host everywhere (installer, updater,
recovery). Quick non-destructive checks:

```bash
# fetch + verify + stage only (no install):
LP_RELEASE_BASE_URL=http://<ip>:8000 ./install.sh --dry-run
# tamper with the tarball and confirm it's refused:
echo x >> dist/stable/libreportal-*.tar.gz
LP_RELEASE_BASE_URL=http://<ip>:8000 ./install.sh --dry-run   # => CHECKSUM MISMATCH
```

## How updates work (so you can reason about them)

In `release` mode the WebUI badge + `libreportal update apply` compare the local
`VERSION` against the channel's `latest.json` (`lpVersionGt`); if newer, they
`lpFetchRelease` the new tarball (verified) and redeploy. Because the install tree
is *code only* (configs/logs/backups live in the other roots), the update just
replaces it — no backup/restore dance. `git`/`local` modes keep their existing
git-based update path.

**The footprint exception (important).** `update apply` runs as the *manager*, and
the manager is deliberately forbidden from rewriting the **root-owned footprint**
(the helpers in `/usr/local/lib/libreportal/`, the CLI wrapper, the uninstall
launcher, the systemd unit, the sudoers) — that immutability *is* the de-sudo
security boundary. So a manager-run update can refresh **code/apps/WebUI**, but not
those. To track when an update touches them, `init.sh` carries a `footprint_version`
integer, baked at install into `/usr/local/lib/libreportal/.footprint_version` and
published in `latest.json`. When the channel's `footprint_version` exceeds the
installed one, the updater **refuses the WebUI apply** and the badge flags
`footprint_update_needed` — the user re-runs the installer as root
(`curl … install.sh | sudo bash`), which fetches *and* re-bakes the footprint
atomically. (Re-running the installer is idempotent.)

➡️ **BUMP `footprint_version` in `init.sh` whenever you change anything in
`scripts/system/*`, the CLI wrapper, the uninstall launcher, the systemd unit, or
the sudoers.** Forgetting it means those root components silently stay stale until
the next full reinstall.

## Signing releases (minisign)

The sha256 only proves a download is intact — a *compromised host* could swap the
tarball **and** its checksum. A minisign signature proves the release is genuinely
ours: the host can't forge it without the offline secret key. It ships **inactive**
(a `REPLACE_ME` placeholder), so installs work today; once you set a real key,
verification becomes **required** for release installs + updates.

**One-time setup:**
```bash
minisign -G -p libreportal.pub -s ~/.minisign/libreportal.key   # generate the keypair
# 1. keep ~/.minisign/libreportal.key OFFLINE (this is the thing to protect)
# 2. paste the PUBLIC key (the RW… line) into BOTH:
#      - libreportal.pub   (ships + installed to the root footprint, used by updates)
#      - install.sh         LP_MINISIGN_PUBKEY=…  (the standalone bootstrap)
# 3. bump footprint_version in init.sh (the footprint's public key changed)
```

**Signing a build:** point `make_release` at the secret key on the release machine:
```bash
LP_MINISIGN_SECKEY=~/.minisign/libreportal.key scripts/release/make_release.sh stable
```
It emits `libreportal-<ver>.tar.gz.minisig` alongside the tarball. `install.sh` and
the updater (`lpFetchRelease`) download `.minisig`, verify it against the public key,
and **refuse** on a bad/missing signature. `--no-verify-signature` on `install.sh`
is a dev-only escape hatch.

> Rotating the key later = repeat steps 2–3 and re-bump `footprint_version` (the
> root-owned public key is part of the footprint).

## Conventions

- **Versioning:** semver in `VERSION`. Bump before building; `latest.json` carries it.
- **An app is a self-contained drop-in.** Everything for an app lives under
  `containers/<app>/`:
  - **Tools tab actions:** declare them in `containers/<app>/tools/<app>.tools.json`
    (`{ "tools": [ … ] }`) and put each function beside it at
    `containers/<app>/tools/<app>_<tool_id>.sh` (function `app<App><PascalToolId>`).
  - **Logic helpers** (anything that isn't a Tools-tab action — e.g. the auth
    adapter, post-install fixups): `containers/<app>/scripts/<app>_*.sh`.
  - **Static data / container payloads:** `containers/<app>/resources/` — it's
    **pruned** by the container scan (never sourced), so it may nest freely AND is
    the home for any `.sh` that *executes on load* (a payload run inside a
    container, e.g. headscale's `tailscale.sh`). Rule: a sourced function file →
    `scripts/`; a script that runs when invoked → `resources/`.
  - The container scan **live-sources** every `.sh` under `containers/<app>/`
    (`tools/` and `scripts/`), and `webui_tools.sh` **auto-merges** the
    `.tools.json`. So dropping the folder onto an install is all it takes — no
    central edits, no array regen.
  - **Reserved folder names:** `tools/`, `scripts/`, `resources/` are
    LibrePortal's. An app's compose must **not** bind-mount to `./tools`,
    `./scripts`, or `./resources` — name app mount dirs anything else (`./data`,
    `./config`, …) or nest them under `resources/`. (The scan is `maxdepth 3`, so
    one sourced subfolder level is the cap — keep `scripts/` flat; name files
    `<app>_<purpose>.sh` instead of nesting.)
- **Per-app hook conventions.** Every per-app extension is a conventionally-named
  function the central code dispatches via `declare -F` — define it, the
  framework picks it up; don't, you're a clean no-op. All hooks live in
  `containers/<app>/scripts/<app>_*.sh` unless noted. The hooks the framework
  currently dispatches:

  | Hook function | When it runs | Notes |
  |---|---|---|
  | `app<App><PascalToolId>` | a Tools-tab button is clicked | declared in `tools/<app>.tools.json`; function in `tools/<app>_<tool_id>.sh`. Dispatched by `dockerAppRunTool`. |
  | `appUpdateSpecifics_<app>($app)` | end of every install / update | post-install fixups, side-effects. May set `shouldrestart=true` to request a restart. |
  | `appSetupComposeTags_<app>($file)` | during compose templating | fill computed (non-CFG) compose tags. CFG_`<APP>`_* tags are filled generically — only needed for tags that require computation. |
  | `appWebuiRefresh_<app>` | every `webuiLibrePortalUpdate` while the app is installed | data that needs ongoing refresh (e.g. gluetun's provider list snapshot). |
  | `appTraefikSkipsDefaultMiddleware_<app>` | router setup | **marker only** — function presence means "opt out of `default@file` middleware." |
  | `appTraefikExtraMiddlewares_<app>` | router setup | echo extra middleware entries (one per line) to append to this app's chain. |
  | `appNetworkApplyMode_<provider>($file)` | templating, when an app sets `CFG_<APP>_NETWORK=<provider>` | switch the routed app's compose to the provider's networking (e.g. toggle the `GLUETUN_OFF`/`ON` regions). The `<provider>` is the provider's app slug; the hook lives in `containers/<provider>/scripts/`. |
  | `appNetworkRegisterPorts_<provider>` | templating + uninstall, for any provider with a hook | refresh the provider's forwarded-port wiring (so a routed app's ports appear / removed apps drop out). Self-skip when the provider isn't installed. |

  None of these need to be wired anywhere — the container scan live-sources
  `containers/<app>/scripts/*.sh` (depth 3, only `resources/` pruned), and the
  central dispatchers find the hook by name. So new apps adopting any of these
  is just "create the file + define the function."
- **What gets backed up** (so you know what's safe to regenerate vs. preserve):
  per-app backups capture only the live data dir `<containers>/<app>/` — the
  deployed compose, the live `<app>.config` (settings + secrets), and the mounted
  data (DBs dumped logically; raw DB dirs excluded). The **system config**
  (`<system>/configs` — global settings, WebUI creds, **backup-location creds**)
  is captured by a separate system-config snapshot. **Not** backed up (reproducible
  from the release): `scripts/`, `tools/`, install templates, and the `libreportal`
  WebUI app itself (its frontend + generated JSON regenerate; its only state, the
  login, lives in the system config). Restore = reinstall the code, then restore
  the system config + each app's data dir. The system snapshot runs automatically
  in `backup all` and is also on-demand via `libreportal backup system` /
  `restore system` (restore lands in a staging dir — never overwrites live config)
  and the WebUI **backup dashboard → "System config" card**, which shows its
  last-backup status (tag `system=config`, surfaced as `system` in the dashboard
  JSON) the same way per-app tiles do. The `libreportal` app is excluded from the
  per-app grid since it isn't backed up.
- **New runtime script?** Add it under `scripts/<area>/…` and run
  `scripts/source/files/generate_arrays.sh run` (or `libreportal regen arrays`) so
  it's sourced (build/standalone tooling under `scripts/release` and
  `scripts/system` is intentionally excluded). `make_release` refuses to build if
  the committed arrays are stale, so a release can't ship a mismatch.
- **Regenerating generated data:** `libreportal regen [all|webui|arrays] [--force]`
  is the one front door — it rebuilds only what's stale (a `find -newer` check, no
  watcher). The task processor runs `regen webui` on its idle tick, so a dropped-in
  app (drag-drop / marketplace) appears on its own without a manual command.
- **Don't** make the OS footprint (`/etc/*`, `/usr/local/*`) relocatable — it's
  fixed by design for the privilege model.