diff --git a/docs/roadmap/updates-and-distribution.md b/docs/roadmap/updates-and-distribution.md index 730dc6f..b745c92 100644 --- a/docs/roadmap/updates-and-distribution.md +++ b/docs/roadmap/updates-and-distribution.md @@ -1,10 +1,13 @@ # LibrePortal — Updates, Improvements & Distribution (Roadmap / Vision) -**Status:** Discussion / vision — *not committed decisions yet* · **Audience:** us, future-self · **Scope:** the updater feature, "hotfixes", and how third-party themes/apps/components get distributed · **Origin:** brainstorm 2026-05-30/31 +**Status:** §0–§7 are the brainstorm (vision). **§8 is the committed format spec** and the open forks (§6) are resolved there. · **Audience:** us, future-self · **Scope:** the updater feature, "hotfixes", and how third-party themes/apps/components get distributed · **Origin:** brainstorm 2026-05-30/31 → format decided & Phase 1 built 2026-05-31 -> This is a thinking doc, not a spec. It captures where a design conversation -> landed so we don't lose it. Actionable items are `TODO` checkboxes; the open -> forks at the bottom are genuinely undecided. Nothing here is built. +> Sections 0–7 below are the original thinking doc — kept verbatim so the +> reasoning isn't lost. **The conclusion of that brainstorm is §8: the concrete +> artifact format**, designed so apps/themes/components slot into the same pipe a +> hotfix uses. Phase 1 of it (the signed-fetch+verify read primitive) is already +> built — see §8.7. The forks in §6 are no longer open; §8.5 records how each was +> resolved. --- @@ -67,10 +70,11 @@ is reversible through the same task → snapshot → apply path. The safety net bold default. `TODO` (when prioritized): -- [ ] Define the declarative hotfix schema (the allowed operations + checksum preconditions). -- [ ] Decide auto-apply policy (uniform vs severity-split). -- [ ] Surface applied/available hotfixes as a stream in the updater + History audit trail. -- [ ] Sign + publish the hotfix manifest on the same channel as the version check. +- [x] Define the declarative hotfix schema (the allowed operations + checksum preconditions). → **§8.2** +- [x] Decide auto-apply policy (uniform vs severity-split). → **§8.5 fork 2** (severity-split) +- [x] Fetch + verify the signed manifest on the same channel as the version check. → **§8.7 Phase 1 (built)** +- [ ] Apply pipeline for the ops (snapshot → apply → verify → rollback → History). → §8.7 Phase 2 +- [ ] Surface applied/available hotfixes as a stream in the updater + History audit trail. → §8.7 Phase 3 --- @@ -173,7 +177,11 @@ Flag only — not on the table. --- -## 6. Open forks (genuinely undecided — decide before any of this becomes a plan) +## 6. Open forks (RESOLVED — see §8.5) + +> These were the genuinely-undecided questions. They are now decided; §8.5 holds +> the resolutions and the reasoning. Kept here for the record. + 1. **Hotfix scope** — config/compose tweaks only, or can a hotfix patch app files / our own WebUI code too? (Sets the entire risk profile.) 2. **Auto-apply policy** — uniformly on-by-default, or split by severity (security auto, tweaks surface-and-suggest)? @@ -189,3 +197,193 @@ Flag only — not on the table. Add them as they resurface.)* - [ ] _…_ + +--- + +# Part II — The format (committed spec) + +## 8. The artifact format + +This is the concrete shape the brainstorm landed on. It was stress-tested by a +four-lens design pass (marketplace-first, security-first, simplicity/reuse-first, +ops-ux-first) that **converged** on the same model — which is why it's promoted +from "vision" to "spec". The whole thing is **one verb over a type-tagged +envelope**; a hotfix is the first artifact type, and apps/themes/components are +*new envelope rows*, not new features. + +### 8.0 Three layers (each already half-built) + +| Layer | What it is | Reuses | +|---|---|---| +| **INDEX** | A static, team-signed JSON catalog at `$base/$channel/index.json` (+ `.minisig`), in the **same release tree as `latest.json`**. A list of artifact ENVELOPES. | `fetch.sh` downloaders, the footprint signing key, the existing update-check phone-home | +| **ENVELOPE** | One artifact entry. **Fixed** metadata for every type; the *only* type-specific part is `payload`, a tagged union keyed by `payload.kind`. | — (new, but tiny + frozen) | +| **PIPELINE** | The verb: fetch → verify(sha256+sig) → snapshot → apply → verify → auto-rollback → History. | `lpFetchRelease`/`lpVerifyMinisig`, `updaterApplyApp` (snapshot/rollback/History), the task system | + +The envelope **never changes** as new types arrive. Only two fields carry the +type information: `type` and `payload.kind`. That is the whole marketplace seam. + +### 8.1 The INDEX + ENVELOPE (example) + +`get.libreportal.org/stable/index.json` (signed by `index.json.minisig`): + +```jsonc +{ + "schema": 1, + "index_serial": 17, // monotonic; anti-rollback (TUF-lite) + "valid_until": 1750000000, // epoch; a stale feed is REFUSED (anti-withholding) + "generated_at": "2026-05-31T12:00:00Z", + "artifacts": [ + { + "id": "hf-vaultwarden-signup-env-2026-05", // stable, unique + "type": "hotfix", // hotfix | app | theme | component + "version": 1, // bump to re-issue/supersede + "publisher": { "name": "LibrePortal", "trust": "official" }, // official|community|custom + "severity": "breakage", // security|breakage|compat|tweak + "auto": true, // see §8.5 fork 2 (severity-split default) + "title": "Fix Vaultwarden signup after upstream env rename", + "why": "Upstream renamed SIGNUPS_ALLOWED; logins break until the new key is set.", + "applies_when": { // gates; missing = always + "app": "vaultwarden", "min_lp": "1.0.0", "max_lp": null, + "max_footprint": 4 + }, + "payload": { + "kind": "ops", // ops (hotfix) | bundle (app/theme/component) + "url": "stable/payloads/hf-vaultwarden-signup-env-2026-05.json", + "sha256": "…", "sig": "stable/payloads/hf-…json.minisig" + } + } + ] +} +``` + +**Fixed fields, identical for every type:** `id, type, version, publisher{name,trust}, +severity, auto, title, why, applies_when, payload{kind,url,sha256,sig}`. An app entry +is byte-for-byte this shape with `type:"app"`, `payload.kind:"bundle"`, and a tarball +payload. A theme is `type:"theme"`, `kind:"bundle"`. Nothing in the envelope moves. + +**Forward-compat firewall:** an installed box that doesn't recognise a `type` or a +`payload.kind` **skips + logs** it (never errors). So the registry can publish new +types the day a newer client understands them, without breaking older installs. + +### 8.2 The op vocabulary (`payload.kind:"ops"` — the hotfix body) + +A **bounded, closed, declarative** set. **There is no `run-script` op, ever** — that +is the supply-chain contract from §1. The payload file is `{ "schema":1, "ops":[ … ] }`. +The applier is a hardcoded dispatch `case`; an **unknown op name aborts the whole +artifact** (fail-closed, never a partial apply). Every op: + +1. is **precondition-guarded** (checksum / `expect_current`) — it refuses on local drift + rather than clobbering, +2. is **reversible** — reverse is the snapshot restore the pipeline already takes, so + even a buggy op can't make rollback wrong, +3. writes **only through the existing privilege funnels** — `runInstallOp`/`runFileOp` + by tree (never raw `sudo`); `set-config-key` rides `updateConfigOption`, which already + routes the write correctly per the de-sudo split. + +| op | args | apply (existing fn) | reverse | precondition | +|---|---|---|---|---| +| `set-config-key` | `key,value` | `updateConfigOption KEY VALUE` | restore snapshot | `key` matches `^CFG_[A-Z0-9_]+$`; opt. `expect_current` | +| `add-compose-label` / `remove-compose-label` | `app,service,label` | edit `containers//docker-compose.yml` via `runFileOp` | inverse op / snapshot | service exists | +| `set-compose-image` | `app,service,image` | rewrite the `image:` line | restore prior image | current image == `expect_current` | +| `ensure-env` | `app,service,key,value` | upsert env entry | restore / remove | — | +| `patch-file-if-checksum-matches` | `path,expect_sha256,content_ref` | write new content **iff** current sha256 matches | restore snapshot | **hard** sha256 match; path-allowlisted to `containers//` + `configs/` | + +`set-compose-image` + `patch-file-if-checksum-matches` are the upstream-breakage killers +(§1). The checksum lock turns "patch a file" from an arbitrary write into a drift-safe, +conflict-detecting, reversible transform. + +### 8.3 The PIPELINE (the verb) — `libreportal artifact apply ` + +A generalization of `updaterApplyApp`, run **only as a task** (`cliTaskRun "libreportal +artifact apply " "artifact_apply" "" ""`; the processor re-invokes with +`LIBREPORTAL_TASK_EXEC=1`). Seven steps — **six are type-agnostic; only step 4 dispatches +on `payload.kind`**: + +0. **RESOLVE** (read-only) — `lpFetchIndex` (cached), find the envelope by id, check + `applies_when` + `lpVersionGt` + `max_footprint <= lpInstalledFootprintVersion` + (reuse `fetch.sh`'s exact footprint guard). Gate fails → History `skipped` + reason. +1. **FETCH** — `_lpDownload "$base/$channel/$payload.url"`. +2. **VERIFY** — `_lpSha256` == `payload.sha256`, then `lpVerifyMinisig` against the + per-artifact `payload.sig`. (Two-tier: footprint key signs the index; the index + pins each payload's hash + sig.) +3. **SNAPSHOT** — `libreportal backup app ` (the Backup engine) — the reversibility + anchor that makes auto-apply defensible. +4. **APPLY** — `kind:"ops"` → the §8.2 interpreter; `kind:"bundle"` → drop+scan/regen + (Phase 4). **Only this step knows the type.** +5. **VERIFY** — app healthy / container up (reuse the updater's post-check). +6. **AUTO-ROLLBACK on failure** — `updaterRollbackApp auto` (restore the snapshot). +7. **HISTORY** — `updaterRecordHistory` (extended with `artifact_id`, `serial`) → the + existing History tab. **Nothing silent.** + +### 8.4 The marketplace seam + +**Unchanged forever** (built once, reused): the index file + location + bash-native +parser; the whole envelope shape; pipeline steps 0–3,5–7; the two-tier trust chain; +mutations-via-tasks; the `valid_until`/`index_serial` guarantees. Adding apps/themes/ +components is **purely additive**: + +- a new `type` value becomes "handled" in step 4's dispatch (old boxes skip+log — §8.1 firewall); +- those types use `payload.kind:"bundle"` (a signed tarball) + one new bundle handler; +- a **custom source ("tap")** is just a second `(base_url, pubkey)` pair appended to a + list — zero envelope change, the registry opens without us hosting or gatekeeping. + +This is exactly the §3 "registry, not marketplace" shape, now expressed in the format. + +### 8.5 Fork resolutions (was §6) + +1. **Hotfix scope** → **config/compose ops + checksum-pinned file patches; NO code + execution.** `patch-file-if-checksum-matches` is allowlisted to `containers//` + + `configs/` and is drift-safe + reversible. **Our own install tree (WebUI/CLI code) is + off-limits to hotfixes** — it already has a signed, whole-tree-verified delivery channel + (releases + `SHA256SUMS` + `verify.sh`); letting a hotfix mutate it would open a second, + finer-grained code-injection surface that bypasses the whole-tree signature. Code fixes + ride an edge/out-of-band release. The killer use case (upstream breakage) is 100% + config/compose, so this loses nothing real. +2. **Auto-apply policy** → **severity-split, declarative in the envelope** (`severity` + + `auto`). `security`/`breakage` → auto-apply ON by default (defensible because the + snapshot/auto-rollback safety net exists); `compat`/`tweak` → surface + one-click, auto + only under an opt-in "auto-improve". +3. **Hotfix locality** → **both.** `applies_when.app` makes an artifact app-scoped (it also + surfaces on that app's page); a null app is system-wide. One field, both behaviours. +4. **Third-party — yet?** → **first-party only now.** The index ships with `trust:"official"` + entries; `community`/`custom` tiers just start appearing later (and gate the riskiest ops). + The "tap" mechanism is designed-in but unbuilt until there's real demand (§4 sequencing). +5. **App catalog entry point** → **curated Browse-&-Add** (first-party definitions as the + seed catalog), with bring-your-own-compose remaining the advanced/“custom source” path. + +### 8.6 Trust & transparency (the non-negotiables, in the format) + +- **Two-tier signatures** anchored on the **root-owned footprint key** (`/usr/local/lib/ + libreportal/libreportal.pub`) — the manager can't swap it, so it can't bless a forgery. +- **`valid_until`** — a signed feed that simply *stops advancing* is the silent-withholding + / targeting attack; a stale index is **refused**, not treated as "no updates". Same spirit + as the [warrant canary](../../) (freshness = signal). +- **`index_serial`** — monotonic; a lower serial than we've accepted is a rollback attack → + refused. The high-water mark is recorded locally and never lowered by a refused fetch. +- **Public + identical for everyone** — one signed feed; a targeted hotfix to a single + victim is impossible to send without it being publicly visible. +- **Nothing silent** — every apply lands in **History** with what / why / revert. + +### 8.7 Build phases & status + +- ✅ **Phase 1 — the signed-fetch + verify read primitive (BUILT 2026-05-31).** + - `lpVerifyMinisig` factored out of `lpFetchRelease` (`scripts/source/fetch.sh`) — the + single trust anchor now shared by releases *and* the index; `lpFetchRelease` refactored + to use it (no behaviour change). + - `scripts/source/artifacts.sh`: `lpFetchIndex` (download → **verify-before-parse** → + `valid_until` freshness → `index_serial` anti-rollback high-water → emit verified JSON), + plus parsing accessors (jq when present, grep fallback; the trust core is jq-free). + - `libreportal artifact index` (`scripts/cli/commands/artifact/`) — read-only front door + that fetches + verifies + lists. Runs directly (no mutation), like `updater check`. + - Self-tested: trust core fails closed (real key + no minisign → refuse), happy path, + stale-refused, rollback-refused, signature-refused, jq + grep parsing — 12/12. +- ⬜ **Phase 2 — the ops applier + apply verb.** `artifactApply`/`artifactApplyOps` with + the §8.2 vocabulary, per-payload sig check, snapshot → apply → verify → auto-rollback → + `updaterRecordHistory` (extend `history.json` with `artifact_id`/`serial`), wired as the + `artifact_apply` task. Makes the Vaultwarden killer use case real, first-party. *(next)* +- ⬜ **Phase 3 — WebUI surfacing.** A `webui_artifact_scan.sh` generator (clone of the + updater scan) writes `data/updater/generated/artifacts_available.json`; a "Hotfixes" + section in the Updates page reads it (graceful-absent). Hook the index fetch into the + existing update-check call site — **no second phone-home**. +- ⬜ **Phase 4 — marketplace types.** `payload.kind:"bundle"` handler (drop + scan/regen) + + `type:"app"|"theme"|"component"` in step 4; later, the "tap" (custom source) UX. diff --git a/scripts/cli/commands/artifact/cli_artifact_commands.sh b/scripts/cli/commands/artifact/cli_artifact_commands.sh new file mode 100644 index 0000000..e029377 --- /dev/null +++ b/scripts/cli/commands/artifact/cli_artifact_commands.sh @@ -0,0 +1,77 @@ +#!/bin/bash + +# Artifact command handler — `libreportal artifact ` +# --------------------------------------------------------------------------- +# Dispatched automatically by cli_initialize.sh (category -> cliHandleArtifactCommands). +# +# This is PHASE 1 of the unified distribution primitive: the READ side. It fetches +# and verifies the team-signed artifact index (hotfixes today; apps/themes/ +# components later — all the same envelope) and lists what's available. It makes +# NO changes to the system, so — like `updater check` — it runs directly rather +# than through the task system. The state-changing `apply`/`rollback` verbs (which +# DO route through tasks → snapshot → declarative ops → rollback → History) arrive +# in Phase 2. See docs/roadmap/updates-and-distribution.md. + +cliHandleArtifactCommands() +{ + local sub="$initial_command2" + + # Lazy-loader gap: ensure the read primitives are defined. These are new + # files; the array/manifest regen self-heals them on deploy, but this covers + # the window before that (mirrors cli_updater_commands.sh sourcing its + # generator). artifacts.sh leans on fetch.sh helpers, so load both. + if ! declare -F lpFetchIndex >/dev/null 2>&1; then + source "$install_scripts_dir/source/fetch.sh" 2>/dev/null + source "$install_scripts_dir/source/artifacts.sh" 2>/dev/null + fi + + case "$sub" in + ""|"index"|"list") + artifactListIndex + ;; + *) + cliShowArtifactHelp + ;; + esac +} + +# Fetch + verify the signed index and print a human summary. Read-only. +artifactListIndex() +{ + isHeader "Artifact index ($(lpReleaseChannel))" + + local json + if ! json="$(lpFetchIndex)"; then + isError "Could not fetch or verify the artifact index from $(lpArtifactIndexUrl)." + isNotice "Nothing is published yet, or the channel is unreachable. (This is expected before the first index ships.)" + return 1 + fi + + local serial generated_at + serial="$(_lpJsonNum "$json" index_serial)" + generated_at="$(lpIndexTop generated_at "$json")" + isNotice "Signed + verified. serial=${serial:-?} generated=${generated_at:-?}" + + local ids; ids="$(lpIndexArtifactIds "$json")" + if [[ -z "$ids" ]]; then + isSuccessful "0 artifacts available — the index is empty (nothing to apply)." + return 0 + fi + + local n=0 id obj title type sev + while IFS= read -r id; do + [[ -z "$id" ]] && continue + n=$((n + 1)) + obj="$(lpArtifactById "$json" "$id")" + if [[ -n "$obj" ]]; then + title="$(_lpJsonStr "$obj" title)" + type="$(_lpJsonStr "$obj" type)" + sev="$(_lpJsonStr "$obj" severity)" + echo " • [${type:-?}/${sev:-info}] $id — ${title:-}" + else + echo " • $id" + fi + done <<< "$ids" + + isSuccessful "$n artifact(s) available." +} diff --git a/scripts/cli/commands/artifact/cli_artifact_header.sh b/scripts/cli/commands/artifact/cli_artifact_header.sh new file mode 100644 index 0000000..2e827d9 --- /dev/null +++ b/scripts/cli/commands/artifact/cli_artifact_header.sh @@ -0,0 +1,20 @@ +#!/bin/bash + +# Artifact (distribution primitive) Commands Header +# Shows available `libreportal artifact` subcommands. + +cliShowArtifactHelp() +{ + echo "" + echo "Available Artifact Commands:" + echo "" + echo " libreportal artifact index - Fetch + verify the signed artifact index and list what's available" + echo "" + echo "An 'artifact' is anything LibrePortal pulls from the outside and applies" + echo "reversibly — a hotfix today; apps / themes / components later. They share" + echo "one team-signed catalog (index.json) on the same channel as the version" + echo "check. This read side verifies the catalog against the root-owned signing" + echo "key; the apply pipeline (snapshot → declarative ops → rollback → History)" + echo "lands in a later phase. See docs/roadmap/updates-and-distribution.md." + echo "" +} diff --git a/scripts/source/artifacts.sh b/scripts/source/artifacts.sh new file mode 100644 index 0000000..a8b1ffc --- /dev/null +++ b/scripts/source/artifacts.sh @@ -0,0 +1,115 @@ +#!/bin/bash +# +# LibrePortal artifact-index helpers — the READ side of the unified distribution +# primitive (see docs/roadmap/updates-and-distribution.md). +# +# An "artifact" is anything LibrePortal pulls from the outside and applies +# reversibly: a HOTFIX today; apps / themes / components later. They all share +# ONE team-signed catalog — the INDEX — published in the SAME release tree as +# latest.json: $base/$channel/index.json (+ index.json.minisig). +# +# This file is PHASE 1 of that primitive: fetch + verify + parse the index. It +# performs NO mutation. The apply pipeline (snapshot → declarative ops → verify → +# auto-rollback → History) is Phase 2 (scripts/cli/commands/artifact). Keeping +# the read side here means the trust core is testable on its own and the WebUI +# scan can surface "available artifacts" before any apply machinery exists. +# +# Trust chain — fail-closed at every step once the footprint key is real: +# footprint pubkey --signs--> index.json --lists--> per-artifact {sha256, sig} +# Verification reuses lpVerifyMinisig (fetch.sh) — the EXACT anchor the release +# fetch uses — so the manager can't bless a forged catalog any more than a forged +# release. Two transparency guarantees, both jq-free so the trust core never +# depends on jq being present: +# valid_until — refuse a stale/withheld feed. A signed feed that simply stops +# advancing is the silent-withholding / targeting attack the +# warrant-canary model exists to defeat; treat a frozen feed as +# a signal, not as "no updates". +# index_serial — monotonic counter; refuse a serial below the highest we have +# already accepted (a rollback that re-introduces a pulled or +# again-vulnerable entry). + +# The index sits next to latest.json on the same channel; reuse those resolvers +# (lpReleaseBaseUrl/lpReleaseChannel live in fetch.sh). +lpArtifactIndexUrl() { echo "$(lpReleaseBaseUrl)/$(lpReleaseChannel)/index.json"; } + +# Runtime-owned high-water mark for index_serial (the anti-rollback anchor). It +# lives alongside the other generated updater data so it ships/clears with that +# state; the dir is in the container tree, so writes go through the container +# funnel. Reads are fine as any user (world-readable). +lpArtifactSerialFile() { echo "${containers_dir%/}/libreportal/frontend/data/updater/generated/.index_serial"; } +lpArtifactLastSerial() { local v; v=$(cat "$(lpArtifactSerialFile)" 2>/dev/null | tr -dc '0-9'); echo "${v:-0}"; } +lpArtifactRecordSerial() { + local serial="$1" f; f="$(lpArtifactSerialFile)" + [[ "$serial" =~ ^[0-9]+$ ]] || return 0 + runFileOp mkdir -p "$(dirname "$f")" 2>/dev/null || true + printf '%s\n' "$serial" | runFileWrite "$f" +} + +# Fetch + verify the signed artifact index. +# $1 (optional): also cache the verified JSON to this path (for the WebUI scan). +# Echoes the verified JSON to stdout on success. Returns non-zero (printing +# nothing usable) on ANY download / signature / freshness / rollback failure — +# callers MUST NOT proceed on a non-zero return (fail-closed). +lpFetchIndex() { + local cache="${1:-}" base channel tmp idx sig json valid_until nowts serial last + base="$(lpReleaseBaseUrl)"; channel="$(lpReleaseChannel)" + [[ -n "$(_lpFetchTool)" ]] || { isError "lpFetchIndex: need curl or wget"; return 1; } + + tmp="$(mktemp -d)"; idx="$tmp/index.json"; sig="$tmp/index.json.minisig" + if ! _lpDownload "$base/$channel/index.json" "$idx"; then + isError "lpFetchIndex: could not download the artifact index"; rm -rf "$tmp"; return 1 + fi + + # Signature FIRST — never parse an unverified document to make trust + # decisions. Fetch the .minisig best-effort; lpVerifyMinisig decides whether + # a missing/invalid signature is fatal (it is, once the key is real). + _lpDownload "$base/$channel/index.json.minisig" "$sig" 2>/dev/null || true + if ! lpVerifyMinisig "$idx" "$sig" >/dev/null; then rm -rf "$tmp"; return 1; fi + + json="$(cat "$idx")" + + # Freshness — refuse a signed-but-stale feed. + valid_until="$(_lpJsonNum "$json" valid_until)" + if [[ -n "$valid_until" ]]; then + nowts="$(date +%s 2>/dev/null)" + if [[ -n "$nowts" ]] && (( valid_until < nowts )); then + isError "lpFetchIndex: artifact index is stale (valid_until elapsed) — refusing"; rm -rf "$tmp"; return 1 + fi + fi + + # Anti-rollback — serial must not go backwards from the highest accepted. + serial="$(_lpJsonNum "$json" index_serial)" + last="$(lpArtifactLastSerial)" + if [[ -n "$serial" ]] && (( serial < last )); then + isError "lpFetchIndex: index_serial $serial below last-seen $last (rollback) — refusing"; rm -rf "$tmp"; return 1 + fi + [[ -n "$serial" ]] && lpArtifactRecordSerial "$serial" + + [[ -n "$cache" ]] && printf '%s' "$json" | runFileWrite "$cache" + printf '%s' "$json" + rm -rf "$tmp" + return 0 +} + +# --- Parsing accessors ------------------------------------------------------- +# The trust-critical fields (index_serial / valid_until / signature) are read +# jq-free above so the security core has no jq dependency. Enumerating the +# artifacts ARRAY for display is best-effort: jq when present (the runtime path +# has it — updaterRecordHistory already relies on it), with a flat grep fallback. + +lpIndexTop() { _lpJsonStr "$2" "$1"; } # lpIndexTop -> top-level scalar + +lpIndexArtifactIds() { # echo one artifact id per line + local json="$1" + if command -v jq >/dev/null 2>&1; then + printf '%s' "$json" | jq -r '.artifacts[]?.id // empty' 2>/dev/null + return 0 + fi + printf '%s' "$json" | grep -oE '"id"[[:space:]]*:[[:space:]]*"[^"]*"' | sed -E 's/.*"([^"]*)"$/\1/' +} + +lpArtifactById() { # lpArtifactById -> the artifact object (jq only) + local json="$1" id="$2" + command -v jq >/dev/null 2>&1 || return 1 + printf '%s' "$json" | jq -ce --arg id "$id" '.artifacts[]? | select(.id==$id)' 2>/dev/null +} diff --git a/scripts/source/fetch.sh b/scripts/source/fetch.sh index 3a64c56..3644e87 100644 --- a/scripts/source/fetch.sh +++ b/scripts/source/fetch.sh @@ -31,6 +31,35 @@ _lpSha256() { if command -v sha256sum >/dev/null 2>&1; then sha256sum "$1" | cut _lpJsonStr() { printf '%s' "$1" | grep -oE "\"$2\"[[:space:]]*:[[:space:]]*\"[^\"]*\"" | head -1 | sed -E 's/.*"([^"]*)"$/\1/'; } _lpJsonNum() { printf '%s' "$1" | grep -oE "\"$2\"[[:space:]]*:[[:space:]]*[0-9]+" | head -1 | grep -oE '[0-9]+$'; } +# Verify a downloaded file against a detached minisig using the ROOT-OWNED +# footprint public key (/usr/local/lib/libreportal/libreportal.pub). The key is +# in the footprint so the manager can't swap it to bless a forgery. Trust posture +# matches lpVerifyInstall (verify.sh): once the key is REAL (not the REPLACE_ME +# placeholder) a valid signature is MANDATORY; an unactivated/dev build skips it. +# This is the single trust anchor shared by the release fetch (lpFetchRelease) +# and the artifact-index fetch (lpFetchIndex, source/artifacts.sh). +# Args: +# Returns 0 = OK to proceed (signature verified, OR signing not activated); +# 1 = REFUSE (key real but minisign missing / sig missing / sig invalid). +# Echoes "verified" or "unsigned" for the caller's audit line. +lpVerifyMinisig() { + local file="$1" sig="$2" + local pub="/usr/local/lib/libreportal/libreportal.pub" + if [[ ! -f "$pub" ]] || grep -q REPLACE_ME "$pub" 2>/dev/null; then + echo unsigned; return 0 + fi + if ! command -v minisign >/dev/null 2>&1; then + isError "lpVerifyMinisig: minisign required to verify '$file' but not installed"; return 1 + fi + if [[ ! -f "$sig" ]]; then + isError "lpVerifyMinisig: signature for '$file' missing — refusing"; return 1 + fi + if ! minisign -Vm "$file" -p "$pub" -x "$sig" >/dev/null 2>&1; then + isError "lpVerifyMinisig: SIGNATURE INVALID for '$file' — refusing"; return 1 + fi + echo verified; return 0 +} + # Root-owned-footprint version: the one INSTALLED on this box (marker written by # init.sh) vs the one the channel's latest release ships (manifest). When the # latter is greater, a plain manager-run update can't apply it (it can't rewrite @@ -73,22 +102,15 @@ lpFetchRelease() { got="$(_lpSha256 "$tar")" if [[ "$got" != "$want_sha" ]]; then isError "lpFetchRelease: CHECKSUM MISMATCH ($tarname) — refusing"; rm -rf "$tmp"; return 1; fi - # Signature: once the root-owned public key is real (not the REPLACE_ME - # placeholder), a valid minisign signature is REQUIRED. The key lives in the - # footprint so the manager can't swap it to accept a forged update. - local pub="/usr/local/lib/libreportal/libreportal.pub" - if [[ -f "$pub" ]] && ! grep -q REPLACE_ME "$pub" 2>/dev/null; then - if ! command -v minisign >/dev/null 2>&1; then - isError "lpFetchRelease: minisign required to verify the release but not installed"; rm -rf "$tmp"; return 1 - fi - if ! _lpDownload "$base/$channel/$tarname.minisig" "$tar.minisig"; then - isError "lpFetchRelease: release signature (.minisig) missing — refusing"; rm -rf "$tmp"; return 1 - fi - if ! minisign -Vm "$tar" -p "$pub" -x "$tar.minisig" >/dev/null 2>&1; then - isError "lpFetchRelease: SIGNATURE INVALID ($tarname) — refusing"; rm -rf "$tmp"; return 1 - fi - isNotice "Release signature verified." - fi + # Signature: verify against the root-owned footprint key (mandatory once the + # key is real; skipped only for an unsigned/dev build). Shared trust anchor + # with the artifact-index path — see lpVerifyMinisig above. Fetch the .minisig + # first (best-effort) so the verifier can find it; the verifier itself decides + # whether a missing/invalid signature is fatal. + _lpDownload "$base/$channel/$tarname.minisig" "$tar.minisig" 2>/dev/null || true + local sigstate + if ! sigstate="$(lpVerifyMinisig "$tar" "$tar.minisig")"; then rm -rf "$tmp"; return 1; fi + [[ "$sigstate" == "verified" ]] && isNotice "Release signature verified." # Replace the install tree (code only; configs/logs are in the system tree). runInstallOp rm -rf "$script_dir" diff --git a/scripts/source/files/arrays/files_cli.sh b/scripts/source/files/arrays/files_cli.sh index dd3edb3..229b4d9 100755 --- a/scripts/source/files/arrays/files_cli.sh +++ b/scripts/source/files/arrays/files_cli.sh @@ -10,6 +10,8 @@ cli_scripts=( "cli/commands/app/cli_app_header.sh" "cli/commands/app/cli_app_restore.sh" "cli/commands/app/cli_app_tool_list.sh" + "cli/commands/artifact/cli_artifact_commands.sh" + "cli/commands/artifact/cli_artifact_header.sh" "cli/commands/backup/cli_backup_commands.sh" "cli/commands/backup/cli_backup_header.sh" "cli/commands/config/cli_config_commands.sh" diff --git a/scripts/source/files/arrays/files_source.sh b/scripts/source/files/arrays/files_source.sh index 1c2f150..2bfce85 100755 --- a/scripts/source/files/arrays/files_source.sh +++ b/scripts/source/files/arrays/files_source.sh @@ -4,6 +4,7 @@ # Do not edit manually - run './scripts/source/files/generate_arrays.sh run' to regenerate source_scripts=( + "source/artifacts.sh" "source/fetch.sh" "source/files/arrays/files_app.sh" "source/files/arrays/files_backup.sh" diff --git a/scripts/source/files/arrays/function_manifest.sh b/scripts/source/files/arrays/function_manifest.sh index 0def572..73756a7 100644 --- a/scripts/source/files/arrays/function_manifest.sh +++ b/scripts/source/files/arrays/function_manifest.sh @@ -101,6 +101,7 @@ declare -gA LP_FN_MAP=( [appUpdateSpecifics_nextcloud]="nextcloud/scripts/nextcloud_update_specifics.sh" [appUpdateSpecifics_pihole]="pihole/scripts/pihole_update_specifics.sh" [appWebuiRefresh_gluetun]="gluetun/scripts/gluetun_providers.sh" + [artifactListIndex]="cli/commands/artifact/cli_artifact_commands.sh" [atomicWriteWebUI]="webui/data/utils/webui_atomic_write.sh" [authAdapter_adguard_setPassword]="adguard/scripts/adguard_auth.sh" [authAdapter_bookstack_createUser]="bookstack/scripts/bookstack_auth.sh" @@ -249,6 +250,7 @@ declare -gA LP_FN_MAP=( [cliDebugLoadTrace]="cli/commands/debug/cli_debug_commands.sh" [cliFirewallHeader]="cli/commands/firewall/cli_firewall_header.sh" [cliHandleAppCommands]="cli/commands/app/cli_app_commands.sh" + [cliHandleArtifactCommands]="cli/commands/artifact/cli_artifact_commands.sh" [cliHandleBackupCommands]="cli/commands/backup/cli_backup_commands.sh" [cliHandleConfigCommands]="cli/commands/config/cli_config_commands.sh" [cliHandleDebugCommands]="cli/commands/debug/cli_debug_commands.sh" @@ -272,6 +274,7 @@ declare -gA LP_FN_MAP=( [cliInitialize]="cli/cli_initialize.sh" [cliRunVerify]="cli/commands/verify/cli_verify_commands.sh" [cliShowAppHelp]="cli/commands/app/cli_app_header.sh" + [cliShowArtifactHelp]="cli/commands/artifact/cli_artifact_header.sh" [cliShowBackupHelp]="cli/commands/backup/cli_backup_header.sh" [cliShowConfigHelp]="cli/commands/config/cli_config_header.sh" [cliShowDebugHelp]="cli/commands/debug/cli_debug_header.sh" @@ -553,10 +556,18 @@ declare -gA LP_FN_MAP=( [logDebug]="task/crontab_task_processor.sh" [logError]="task/crontab_task_processor.sh" [logInfo]="task/crontab_task_processor.sh" + [lpArtifactById]="source/artifacts.sh" + [lpArtifactIndexUrl]="source/artifacts.sh" + [lpArtifactLastSerial]="source/artifacts.sh" + [lpArtifactRecordSerial]="source/artifacts.sh" + [lpArtifactSerialFile]="source/artifacts.sh" [_lpDownload]="source/fetch.sh" + [lpFetchIndex]="source/artifacts.sh" [lpFetchRelease]="source/fetch.sh" [lpFetchSource]="source/fetch.sh" [_lpFetchTool]="source/fetch.sh" + [lpIndexArtifactIds]="source/artifacts.sh" + [lpIndexTop]="source/artifacts.sh" [lpInstalledFootprintVersion]="source/fetch.sh" [_lpJsonNum]="source/fetch.sh" [_lpJsonStr]="source/fetch.sh" @@ -570,6 +581,7 @@ declare -gA LP_FN_MAP=( [lpReleaseLatestVersion]="source/fetch.sh" [_lpSha256]="source/fetch.sh" [lpVerifyInstall]="source/verify.sh" + [lpVerifyMinisig]="source/fetch.sh" [lpVerifyPubKeyPath]="source/verify.sh" [lpVersionGt]="source/fetch.sh" [mainLoop]="task/crontab_task_processor.sh" @@ -1008,6 +1020,7 @@ declare -gA LP_FN_ROOT=( [appUpdateSpecifics_nextcloud]="containers" [appUpdateSpecifics_pihole]="containers" [appWebuiRefresh_gluetun]="containers" + [artifactListIndex]="scripts" [atomicWriteWebUI]="scripts" [authAdapter_adguard_setPassword]="containers" [authAdapter_bookstack_createUser]="containers" @@ -1156,6 +1169,7 @@ declare -gA LP_FN_ROOT=( [cliDebugLoadTrace]="scripts" [cliFirewallHeader]="scripts" [cliHandleAppCommands]="scripts" + [cliHandleArtifactCommands]="scripts" [cliHandleBackupCommands]="scripts" [cliHandleConfigCommands]="scripts" [cliHandleDebugCommands]="scripts" @@ -1179,6 +1193,7 @@ declare -gA LP_FN_ROOT=( [cliInitialize]="scripts" [cliRunVerify]="scripts" [cliShowAppHelp]="scripts" + [cliShowArtifactHelp]="scripts" [cliShowBackupHelp]="scripts" [cliShowConfigHelp]="scripts" [cliShowDebugHelp]="scripts" @@ -1460,10 +1475,18 @@ declare -gA LP_FN_ROOT=( [logDebug]="scripts" [logError]="scripts" [logInfo]="scripts" + [lpArtifactById]="scripts" + [lpArtifactIndexUrl]="scripts" + [lpArtifactLastSerial]="scripts" + [lpArtifactRecordSerial]="scripts" + [lpArtifactSerialFile]="scripts" [_lpDownload]="scripts" + [lpFetchIndex]="scripts" [lpFetchRelease]="scripts" [lpFetchSource]="scripts" [_lpFetchTool]="scripts" + [lpIndexArtifactIds]="scripts" + [lpIndexTop]="scripts" [lpInstalledFootprintVersion]="scripts" [_lpJsonNum]="scripts" [_lpJsonStr]="scripts" @@ -1477,6 +1500,7 @@ declare -gA LP_FN_ROOT=( [lpReleaseLatestVersion]="scripts" [_lpSha256]="scripts" [lpVerifyInstall]="scripts" + [lpVerifyMinisig]="scripts" [lpVerifyPubKeyPath]="scripts" [lpVersionGt]="scripts" [mainLoop]="scripts" @@ -1935,6 +1959,7 @@ appUpdateSpecifics_libreportal() { source "${install_containers_dir}libreportal/ appUpdateSpecifics_nextcloud() { source "${install_containers_dir}nextcloud/scripts/nextcloud_update_specifics.sh"; appUpdateSpecifics_nextcloud "$@"; } appUpdateSpecifics_pihole() { source "${install_containers_dir}pihole/scripts/pihole_update_specifics.sh"; appUpdateSpecifics_pihole "$@"; } appWebuiRefresh_gluetun() { source "${install_containers_dir}gluetun/scripts/gluetun_providers.sh"; appWebuiRefresh_gluetun "$@"; } +artifactListIndex() { source "${install_scripts_dir}cli/commands/artifact/cli_artifact_commands.sh"; artifactListIndex "$@"; } atomicWriteWebUI() { source "${install_scripts_dir}webui/data/utils/webui_atomic_write.sh"; atomicWriteWebUI "$@"; } authAdapter_adguard_setPassword() { source "${install_containers_dir}adguard/scripts/adguard_auth.sh"; authAdapter_adguard_setPassword "$@"; } authAdapter_bookstack_createUser() { source "${install_containers_dir}bookstack/scripts/bookstack_auth.sh"; authAdapter_bookstack_createUser "$@"; } @@ -2083,6 +2108,7 @@ cliAppToolList() { source "${install_scripts_dir}cli/commands/app/cli_app_tool_l cliDebugLoadTrace() { source "${install_scripts_dir}cli/commands/debug/cli_debug_commands.sh"; cliDebugLoadTrace "$@"; } cliFirewallHeader() { source "${install_scripts_dir}cli/commands/firewall/cli_firewall_header.sh"; cliFirewallHeader "$@"; } cliHandleAppCommands() { source "${install_scripts_dir}cli/commands/app/cli_app_commands.sh"; cliHandleAppCommands "$@"; } +cliHandleArtifactCommands() { source "${install_scripts_dir}cli/commands/artifact/cli_artifact_commands.sh"; cliHandleArtifactCommands "$@"; } cliHandleBackupCommands() { source "${install_scripts_dir}cli/commands/backup/cli_backup_commands.sh"; cliHandleBackupCommands "$@"; } cliHandleConfigCommands() { source "${install_scripts_dir}cli/commands/config/cli_config_commands.sh"; cliHandleConfigCommands "$@"; } cliHandleDebugCommands() { source "${install_scripts_dir}cli/commands/debug/cli_debug_commands.sh"; cliHandleDebugCommands "$@"; } @@ -2106,6 +2132,7 @@ cliHandleWebuiCommands() { source "${install_scripts_dir}cli/commands/webui/cli_ cliInitialize() { source "${install_scripts_dir}cli/cli_initialize.sh"; cliInitialize "$@"; } cliRunVerify() { source "${install_scripts_dir}cli/commands/verify/cli_verify_commands.sh"; cliRunVerify "$@"; } cliShowAppHelp() { source "${install_scripts_dir}cli/commands/app/cli_app_header.sh"; cliShowAppHelp "$@"; } +cliShowArtifactHelp() { source "${install_scripts_dir}cli/commands/artifact/cli_artifact_header.sh"; cliShowArtifactHelp "$@"; } cliShowBackupHelp() { source "${install_scripts_dir}cli/commands/backup/cli_backup_header.sh"; cliShowBackupHelp "$@"; } cliShowConfigHelp() { source "${install_scripts_dir}cli/commands/config/cli_config_header.sh"; cliShowConfigHelp "$@"; } cliShowDebugHelp() { source "${install_scripts_dir}cli/commands/debug/cli_debug_header.sh"; cliShowDebugHelp "$@"; } @@ -2387,10 +2414,18 @@ locationRemove() { source "${install_scripts_dir}backup/locations/location_remov logDebug() { source "${install_scripts_dir}task/crontab_task_processor.sh"; logDebug "$@"; } logError() { source "${install_scripts_dir}task/crontab_task_processor.sh"; logError "$@"; } logInfo() { source "${install_scripts_dir}task/crontab_task_processor.sh"; logInfo "$@"; } +lpArtifactById() { source "${install_scripts_dir}source/artifacts.sh"; lpArtifactById "$@"; } +lpArtifactIndexUrl() { source "${install_scripts_dir}source/artifacts.sh"; lpArtifactIndexUrl "$@"; } +lpArtifactLastSerial() { source "${install_scripts_dir}source/artifacts.sh"; lpArtifactLastSerial "$@"; } +lpArtifactRecordSerial() { source "${install_scripts_dir}source/artifacts.sh"; lpArtifactRecordSerial "$@"; } +lpArtifactSerialFile() { source "${install_scripts_dir}source/artifacts.sh"; lpArtifactSerialFile "$@"; } _lpDownload() { source "${install_scripts_dir}source/fetch.sh"; _lpDownload "$@"; } +lpFetchIndex() { source "${install_scripts_dir}source/artifacts.sh"; lpFetchIndex "$@"; } lpFetchRelease() { source "${install_scripts_dir}source/fetch.sh"; lpFetchRelease "$@"; } lpFetchSource() { source "${install_scripts_dir}source/fetch.sh"; lpFetchSource "$@"; } _lpFetchTool() { source "${install_scripts_dir}source/fetch.sh"; _lpFetchTool "$@"; } +lpIndexArtifactIds() { source "${install_scripts_dir}source/artifacts.sh"; lpIndexArtifactIds "$@"; } +lpIndexTop() { source "${install_scripts_dir}source/artifacts.sh"; lpIndexTop "$@"; } lpInstalledFootprintVersion() { source "${install_scripts_dir}source/fetch.sh"; lpInstalledFootprintVersion "$@"; } _lpJsonNum() { source "${install_scripts_dir}source/fetch.sh"; _lpJsonNum "$@"; } _lpJsonStr() { source "${install_scripts_dir}source/fetch.sh"; _lpJsonStr "$@"; } @@ -2404,6 +2439,7 @@ lpReleaseLatestFootprint() { source "${install_scripts_dir}source/fetch.sh"; lpR lpReleaseLatestVersion() { source "${install_scripts_dir}source/fetch.sh"; lpReleaseLatestVersion "$@"; } _lpSha256() { source "${install_scripts_dir}source/fetch.sh"; _lpSha256 "$@"; } lpVerifyInstall() { source "${install_scripts_dir}source/verify.sh"; lpVerifyInstall "$@"; } +lpVerifyMinisig() { source "${install_scripts_dir}source/fetch.sh"; lpVerifyMinisig "$@"; } lpVerifyPubKeyPath() { source "${install_scripts_dir}source/verify.sh"; lpVerifyPubKeyPath "$@"; } lpVersionGt() { source "${install_scripts_dir}source/fetch.sh"; lpVersionGt "$@"; } mainLoop() { source "${install_scripts_dir}task/crontab_task_processor.sh"; mainLoop "$@"; }