899e04bcd3
Add `lpRegen` (scripts/webui/webui_regen.sh) — one entry point that rebuilds the
file-derived artifacts whose sources changed, so callers don't have to know which
generator owns what. Self-heal is a cheap `find -newer` mtime compare (no watcher
/ daemon): a stage runs only when a source is newer than its artifact, or --force.
- `libreportal regen [all|webui|arrays] [--force]` CLI command (new category).
- Task processor idle tick runs a throttled `regen webui` poll, so an app dropped
in out-of-band (drag-drop / marketplace) appears on its own — no manual command,
no inotify (works on the relocatable/external-drive roots where inotify can't).
- make_release.sh guards against shipping stale source arrays (regenerate; abort
if the committed tree was out of date), killing the "forgot generate_arrays" bug
class at the build boundary.
- Document the front door in DEVELOPMENT.md.
webui scope rebuilds from containers/<app>/{*.config,tools/*.tools.json}; arrays
scope from scripts/** (a dev/build concern — a no-op on a normal install). Gate
logic verified in a sandbox (clean/config-newer/tools-newer/force/missing).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
195 lines
9.7 KiB
Markdown
195 lines
9.7 KiB
Markdown
# LibrePortal — Development & Releases
|
||
|
||
How to run a development copy, cut stable/edge releases, and test them before they
|
||
go out. For installing/using LibrePortal, see [USER.md](USER.md).
|
||
|
||
## Mental model (read this first)
|
||
|
||
**Install modes** — `CFG_INSTALL_MODE` decides where the code comes from:
|
||
|
||
| Mode | Source | Use |
|
||
|---|---|---|
|
||
| `release` (default) | a checksum-verified `.tar.gz` over HTTPS | end users / stable |
|
||
| `git` | `git clone` of the repo | contributors tracking a branch |
|
||
| `local` | a copy of a local folder | hacking on the code on the box |
|
||
|
||
**Three roots** (each relocatable at install, then fixed):
|
||
`<system>` (manager-owned: configs/db/logs/install) · `<containers>` (container
|
||
user: app data) · `<backups>` (container user: repos). Defaults `/libreportal-*`.
|
||
|
||
**Two users:** the **manager** (`sudo_user_name`, default `libreportal`) owns the
|
||
control plane and runs the runtime; the **container user** (`CFG_DOCKER_INSTALL_USER`,
|
||
default `dockerinstall`) owns app data + runs rootless Docker. Genuine-root actions
|
||
go through fixed, root-owned helpers in `/usr/local/lib/libreportal/` (paths +
|
||
manager name are *baked* into them at install — never read from runtime config).
|
||
|
||
**Key files:**
|
||
- `init.sh` — the installer (self-contained; creates users/folders/helpers, bakes things).
|
||
- `install.sh` — the thin bootstrap (download+verify+extract a release, then run init.sh).
|
||
- `scripts/source/paths.sh` — resolves the three roots + manager user.
|
||
- `scripts/source/fetch.sh` — `lpFetchRelease` / `lpVersionGt` (runtime fetch + version compare).
|
||
- `scripts/release/make_release.sh` — builds release artifacts.
|
||
- `VERSION` — the single source of the version number.
|
||
|
||
## Run a development copy
|
||
|
||
From a clone of the repo, on a **throwaway Debian/Ubuntu host** (install is
|
||
destructive — it creates system users and dirs):
|
||
|
||
**Local mode** (install from the working tree — best for hacking):
|
||
```bash
|
||
sudo ./init.sh --random-password --local init
|
||
# custom locations work in dev too:
|
||
sudo ./init.sh --random-password --local --system-dir=/srv/lp --manager-user=lpadmin init
|
||
```
|
||
|
||
**Git mode** (track a branch):
|
||
```bash
|
||
# via the bootstrap:
|
||
sudo ./install.sh --git-url=https://example.com/you/LibrePortal.git \
|
||
--git-user=USER --git-token=TOKEN
|
||
# or directly: ./init.sh init <password> <git_user> <git_token> <git_url> true git
|
||
```
|
||
|
||
**Iterating:** re-run the installer to redeploy after changes (local mode re-copies
|
||
the tree). To wipe and start over:
|
||
```bash
|
||
sudo ./init.sh uninstall # removes the three roots + users + footprint
|
||
sudo ./init.sh --skip-docker-images uninstall # keep the Docker layer for a fast reinstall
|
||
```
|
||
|
||
> This repo's CI/hook setup may auto-deploy on commit (commit on a branch →
|
||
> auto-merge → redeploy). That's environment-specific; the commands above are the
|
||
> portable way to stand up and refresh a dev box.
|
||
|
||
## Cut a release (stable or edge)
|
||
|
||
1. **Bump the version** in `VERSION` (semver, e.g. `0.2.0` → `0.3.0`). Commit it.
|
||
2. **Build the artifact** (uses `git archive`, so it ships only committed files and
|
||
honours `.gitattributes export-ignore` — `scripts/unused`, `site`, `docs`,
|
||
`.claude`, the release tooling, etc. never ship):
|
||
```bash
|
||
scripts/release/make_release.sh stable # or: edge [git-ref]
|
||
```
|
||
Produces, under `dist/<channel>/`:
|
||
- `libreportal-<version>.tar.gz` — the release
|
||
- `libreportal-<version>.tar.gz.sha256` — its checksum
|
||
- `latest.json` — `{ version, channel, url, sha256, notes }` (the channel pointer)
|
||
3. **Publish** by serving `dist/<channel>/*` at `https://get.libreportal.org/<channel>/…`.
|
||
The host (the `getlibreportal` + `weblibreportal` apps) lives in the separate
|
||
**LibrePortal-Infra** repo, which overlays onto an install and picks these up via
|
||
its `publish.sh`. `latest.json` is what makes a version "the latest".
|
||
|
||
**Channels:** `stable` is the default users get; `edge` is for early/testing
|
||
builds. Same tooling, different `<channel>`. To promote an edge build to stable,
|
||
rebuild with `make_release.sh stable` at that ref (or copy its artifacts into the
|
||
`stable/` path and update `stable/latest.json`).
|
||
|
||
## Test a release locally before publishing
|
||
|
||
No hosting needed — serve `dist/` and point an install at it:
|
||
|
||
```bash
|
||
# build, then serve the artifacts
|
||
scripts/release/make_release.sh stable
|
||
( cd dist && python3 -m http.server 8000 )
|
||
|
||
# on a throwaway host, install from your local server:
|
||
LP_RELEASE_BASE_URL=http://<your-ip>:8000 \
|
||
sudo ./install.sh --channel=stable --system-dir=/libreportal-system
|
||
```
|
||
|
||
`LP_RELEASE_BASE_URL` overrides the release host everywhere (installer, updater,
|
||
recovery). Quick non-destructive checks:
|
||
|
||
```bash
|
||
# fetch + verify + stage only (no install):
|
||
LP_RELEASE_BASE_URL=http://<ip>:8000 ./install.sh --dry-run
|
||
# tamper with the tarball and confirm it's refused:
|
||
echo x >> dist/stable/libreportal-*.tar.gz
|
||
LP_RELEASE_BASE_URL=http://<ip>:8000 ./install.sh --dry-run # => CHECKSUM MISMATCH
|
||
```
|
||
|
||
## How updates work (so you can reason about them)
|
||
|
||
In `release` mode the WebUI badge + `libreportal update apply` compare the local
|
||
`VERSION` against the channel's `latest.json` (`lpVersionGt`); if newer, they
|
||
`lpFetchRelease` the new tarball (verified) and redeploy. Because the install tree
|
||
is *code only* (configs/logs/backups live in the other roots), the update just
|
||
replaces it — no backup/restore dance. `git`/`local` modes keep their existing
|
||
git-based update path.
|
||
|
||
**The footprint exception (important).** `update apply` runs as the *manager*, and
|
||
the manager is deliberately forbidden from rewriting the **root-owned footprint**
|
||
(the helpers in `/usr/local/lib/libreportal/`, the CLI wrapper, the uninstall
|
||
launcher, the systemd unit, the sudoers) — that immutability *is* the de-sudo
|
||
security boundary. So a manager-run update can refresh **code/apps/WebUI**, but not
|
||
those. To track when an update touches them, `init.sh` carries a `footprint_version`
|
||
integer, baked at install into `/usr/local/lib/libreportal/.footprint_version` and
|
||
published in `latest.json`. When the channel's `footprint_version` exceeds the
|
||
installed one, the updater **refuses the WebUI apply** and the badge flags
|
||
`footprint_update_needed` — the user re-runs the installer as root
|
||
(`curl … install.sh | sudo bash`), which fetches *and* re-bakes the footprint
|
||
atomically. (Re-running the installer is idempotent.)
|
||
|
||
➡️ **BUMP `footprint_version` in `init.sh` whenever you change anything in
|
||
`scripts/system/*`, the CLI wrapper, the uninstall launcher, the systemd unit, or
|
||
the sudoers.** Forgetting it means those root components silently stay stale until
|
||
the next full reinstall.
|
||
|
||
## Signing releases (minisign)
|
||
|
||
The sha256 only proves a download is intact — a *compromised host* could swap the
|
||
tarball **and** its checksum. A minisign signature proves the release is genuinely
|
||
ours: the host can't forge it without the offline secret key. It ships **inactive**
|
||
(a `REPLACE_ME` placeholder), so installs work today; once you set a real key,
|
||
verification becomes **required** for release installs + updates.
|
||
|
||
**One-time setup:**
|
||
```bash
|
||
minisign -G -p libreportal.pub -s ~/.minisign/libreportal.key # generate the keypair
|
||
# 1. keep ~/.minisign/libreportal.key OFFLINE (this is the thing to protect)
|
||
# 2. paste the PUBLIC key (the RW… line) into BOTH:
|
||
# - libreportal.pub (ships + installed to the root footprint, used by updates)
|
||
# - install.sh LP_MINISIGN_PUBKEY=… (the standalone bootstrap)
|
||
# 3. bump footprint_version in init.sh (the footprint's public key changed)
|
||
```
|
||
|
||
**Signing a build:** point `make_release` at the secret key on the release machine:
|
||
```bash
|
||
LP_MINISIGN_SECKEY=~/.minisign/libreportal.key scripts/release/make_release.sh stable
|
||
```
|
||
It emits `libreportal-<ver>.tar.gz.minisig` alongside the tarball. `install.sh` and
|
||
the updater (`lpFetchRelease`) download `.minisig`, verify it against the public key,
|
||
and **refuse** on a bad/missing signature. `--no-verify-signature` on `install.sh`
|
||
is a dev-only escape hatch.
|
||
|
||
> Rotating the key later = repeat steps 2–3 and re-bump `footprint_version` (the
|
||
> root-owned public key is part of the footprint).
|
||
|
||
## Conventions
|
||
|
||
- **Versioning:** semver in `VERSION`. Bump before building; `latest.json` carries it.
|
||
- **An app is a self-contained drop-in.** Everything for an app lives under
|
||
`containers/<app>/`:
|
||
- **Tools tab actions:** declare them in `containers/<app>/tools/<app>.tools.json`
|
||
(`{ "tools": [ … ] }`) and put each function beside it at
|
||
`containers/<app>/tools/<app>_<tool_id>.sh` (function `app<App><PascalToolId>`).
|
||
- **Logic helpers** (anything that isn't a Tools-tab action — e.g. the auth
|
||
adapter, post-install fixups): `containers/<app>/scripts/<app>_*.sh`.
|
||
- The container scan **live-sources** every `.sh` under `containers/<app>/`
|
||
(`tools/` and `scripts/`), and `webui_tools.sh` **auto-merges** the
|
||
`.tools.json`. So dropping the folder onto an install is all it takes — no
|
||
central edits, no array regen.
|
||
- **New runtime script?** Add it under `scripts/<area>/…` and run
|
||
`scripts/source/files/generate_arrays.sh run` (or `libreportal regen arrays`) so
|
||
it's sourced (build/standalone tooling under `scripts/release` and
|
||
`scripts/system` is intentionally excluded). `make_release` refuses to build if
|
||
the committed arrays are stale, so a release can't ship a mismatch.
|
||
- **Regenerating generated data:** `libreportal regen [all|webui|arrays] [--force]`
|
||
is the one front door — it rebuilds only what's stale (a `find -newer` check, no
|
||
watcher). The task processor runs `regen webui` on its idle tick, so a dropped-in
|
||
app (drag-drop / marketplace) appears on its own without a manual command.
|
||
- **Don't** make the OS footprint (`/etc/*`, `/usr/local/*`) relocatable — it's
|
||
fixed by design for the privilege model.
|