docs(roadmap): design for per-app version updater + CVE scanning
Fills the gap updates-and-distribution.md always deferred: how versions are actually detected, pinned, and reverted, and how cves.json gets real data. Core decisions: floating tag stays the channel while the live compose pins the digest (making detection possible and rollback honest), and CVE scanning runs as a LibrePortal-orchestrated ephemeral Trivy container (no host binary, no always-on scanner app). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: librelad <librelad@digitalangels.vip>
This commit is contained in:
parent
0a3b681211
commit
6bf76d1774
112
docs/roadmap/app-version-updater-and-cve.md
Normal file
112
docs/roadmap/app-version-updater-and-cve.md
Normal file
@ -0,0 +1,112 @@
|
||||
# LibrePortal — Per-App Version Updater & CVE Scanning (Roadmap / Design)
|
||||
|
||||
**Status:** Design — **not built.** Agreed direction 2026-07-14; fills the two deliberately-unwired hooks in `webui_updater_scan.sh`. · **Audience:** us, future-self · **Scope:** real "update available" detection, per-app version identity (pin/track/revert), and the vulnerability scanner behind `/apps/overview/updates` · **Origin:** "the updates system is currently not set up" conversation 2026-07-14. Companion to [updates-and-distribution.md](updates-and-distribution.md) — that doc owns the *signed artifact/hotfix pipe*; this one owns the *generators* it always treated as pre-existing.
|
||||
|
||||
---
|
||||
|
||||
## 0. Where we actually are (so we don't redesign what exists)
|
||||
|
||||
Most of the updater is **already built and working**. What's missing is narrow:
|
||||
|
||||
| Piece | Status |
|
||||
|---|---|
|
||||
| `updater apply` — snapshot → `compose pull` → `up`, auto-rollback on failure | ✅ real (`cli_updater_commands.sh`) |
|
||||
| `history.json` (fail-closed from→to audit trail) | ✅ real |
|
||||
| Signed hotfix channel (fetch/verify/apply/undo, `CFG_HOTFIX_AUTO`) | ✅ real |
|
||||
| Auto-scan scheduling (`updater check auto`, 30-min throttle) | ✅ real |
|
||||
| WebUI — fleet Updates tab, per-app Updates tab, Security/Recovery/History renderers, Update/Roll-back buttons wired to tasks | ✅ real, waiting on data |
|
||||
| **"Update available" detection** | ❌ stub — `update_available` is hard-coded `false` (`webui_updater_scan.sh:55-58` is a marked hook) |
|
||||
| **CVE data** (`cves.json`) | ❌ stub — written empty once, then never rewritten (the `[ ! -f ]` guard at `:82-89` must go) |
|
||||
| Recovery fields (`dr_ready`, `last_snapshot*`) | ❌ never emitted; frontend infers from History |
|
||||
|
||||
So this is **not** a new tab or a new subsystem. The surfaces the feature needs already exist — the fleet **Updates** tab and the **per-app Updates tab** (the "version tab per app" — it's already there, beside Backups). The job is to make the two generators tell the truth, and to fix one real correctness hole described next.
|
||||
|
||||
## 1. The core problem: `latest` is a channel, not a version
|
||||
|
||||
Across `containers/*/docker-compose.yml` there are 49 active `image:` lines: **21 on `:latest`, 11 untagged (implicit latest), 17 on pinned or floating-stable tags**. Zero are variable-driven. Only nextcloud (`31-fpm-alpine`) and mastodon (`v4.2.0`) are meaningfully pinned among primary apps.
|
||||
|
||||
Floating tags break both halves of what we want:
|
||||
|
||||
- **Detection is textually impossible.** "Is `vaultwarden/server:latest` newer than `vaultwarden/server:latest`?" — the compose file can't answer; the string never changes.
|
||||
- **Revert is currently a lie.** `updaterRollbackApp` restores the *data* snapshot and re-runs `compose up` — but the local image is already the *new* latest, so you get **new code on old data**. History records `from → to`, and for a floating tag both are the same string.
|
||||
|
||||
### The one idea
|
||||
|
||||
> **Keep the floating tag as the *channel*; pin the *digest* in the live compose.**
|
||||
> `image: vaultwarden/server:latest@sha256:ab12…`
|
||||
|
||||
Compose supports `tag@digest` natively. The tag stays as the human-readable statement of *what we track* ("latest", "stable", "31-fpm-alpine"); the digest pins *exactly which build runs*. Then:
|
||||
|
||||
- **Detect:** resolve the channel tag at the registry (`docker manifest inspect`, no pull) and compare digests → a truthful `update_available`, even for `latest`.
|
||||
- **Update:** snapshot → rewrite the pin to the new digest → pull → up. Nothing moves unless the user (or a future policy) says so.
|
||||
- **Revert:** restore the data snapshot **and** rewrite the pin back to the previous digest. Now rollback means what the button says.
|
||||
|
||||
Why not just pin semver tags in all ~45 templates instead? Because upstreams are inconsistent (some publish semver, some only `latest`, linuxserver uses their own scheme), it's a permanent manual maintenance burden, and it still wouldn't tell us when a *pinned* tag's build is republished. Digest pinning is universal, automatic, and template-free — templates stay exactly as they are; pinning happens on the **live** copy at install time, which is already the mutable source of truth (template→live copy-then-edit is the established model, and the artifact system's `set-compose-image` op is a working, drift-guarded, undoable sed for precisely this line).
|
||||
|
||||
### Multi-service apps
|
||||
|
||||
Nextcloud is 4 images, jitsi is 4. The scan currently reads only the first `image:` line. Design: **pin and track every `image:` line, keyed by service**. `updates.json` keeps its per-app top-level fields (primary service — first entry — so the existing UI works unchanged) and gains a `services[]` array; the UI can later aggregate ("1 of 4 services has an update"). An app "has an update" if *any* service does; apply updates all outdated services in one snapshot.
|
||||
|
||||
## 2. Version identity — what we record and show
|
||||
|
||||
- **Source of truth:** the pin in the live compose file. No parallel `CFG_<APP>_VERSION` variable — that would be a second source of truth that drifts, and the `_TAG` namespace already means template placeholders here (a real footgun).
|
||||
- **Human-readable version:** digests are unreadable, so enrich for display: `docker image inspect` the local image for the OCI label `org.opencontainers.image.version` (most linuxserver/ghcr images carry it). Display order: label → tag (if not latest/untagged) → short digest (`sha256:ab12…` → `ab12cde`). `available_version` starts life as "new build of *latest*" plus short digest; label enrichment for the *remote* side is best-effort later (needs a config-blob fetch — not worth blocking on).
|
||||
- **`updates.json` additions** (existing fields keep their meaning): per app `channel` (the tag), `pinned_digest`, `available_digest`, `services[] { service, image, channel, pinned_digest, available_digest, update_available, version }`, plus the recovery fields the frontend already knows how to render (`dr_ready`, `last_snapshot`, `last_snapshot_version`, `last_snapshot_at`) sourced from the backup data that already exists.
|
||||
- **History** entries record full pinned refs (`repo:tag@sha256:…`) in `from`/`to`, which is what makes revert-from-history possible.
|
||||
|
||||
## 3. Detection mechanics
|
||||
|
||||
- `docker manifest inspect <repo:tag>` (rootless, via `dockerCommandRunInstallUser`) returns the registry digest without pulling; compare against the pinned digest. Local `RepoDigests` and the registry manifest-list digest are the same identity, so the comparison is exact.
|
||||
- **Throttle separately from the scan.** The 30-min `updater check auto` stat-gate stays cheap; registry lookups get their own stamp + knob, mirroring the established `/tmp` stamp idiom: `CFG_UPDATER_REGISTRY_INTERVAL` (minutes, **default 360**, `0` = manual-only). ~30 Docker Hub images once per 6 h sits comfortably under anonymous rate limits; add small jitter so fleets don't thundering-herd.
|
||||
- **Failure = stale, not error.** A registry timeout keeps the previous verdict and stamps `last_checked`; never flip an app to "unscanned" because the network blinked (per-registry short-circuit after the first timeout in a run).
|
||||
- **Reconcile existing installs:** on first scan, any installed app whose compose line is unpinned gets pinned to its *currently running* image's digest (`RepoDigests`) — zero behaviour change, purely recording reality. New installs pin right after the first pull. Apps using `build:` instead of `image:` are out of scope (scan already ignores them).
|
||||
|
||||
## 4. Apply, revert, and the per-app surface
|
||||
|
||||
`updaterApplyApp` becomes: resolve new digest (from `updates.json`) → **snapshot (unchanged, fail-closed)** → rewrite pin(s) via the factored-out first-class helper shared with the artifact op's sed → `compose pull` → `up` → history `ok` with digest-refs → rescan. On failure: auto-rollback now also **rewrites the pins back** before restoring the snapshot — that closes the "new code on old data" hole.
|
||||
|
||||
`updaterRollbackApp` (the user-facing Roll back button): read the last `update/ok` history entry's `from` refs → rewrite pins → restore data snapshot → `up` → history `rolled-back`. This is the "allow users to revert back" requirement, and it only works because of the pins.
|
||||
|
||||
**UI:** no new navigation. The per-app **Updates** tab gains a Version section (already scaffolded in `renderAppDetail`): channel · running build · available build · Update / Roll back. The fleet tab's rows finally show real `current → available`. Everything routes through the existing `updater_apply` / `updater_rollback` tasks — mutations stay task-only, no new API endpoints.
|
||||
|
||||
## 5. CVE scanning — decision
|
||||
|
||||
Three candidate shapes were on the table:
|
||||
|
||||
| Shape | Verdict |
|
||||
|---|---|
|
||||
| **A. Always-on scanner app container** (trivy server / a "security app" in the catalog) | ❌ Resident RAM for a periodic job; permanent docker-socket exposure; duplicates our UI with its own. |
|
||||
| **B. Scanner binary on the host** | ❌ Grows the host footprint we've deliberately kept lean (rootless + de-sudo); another thing to install/update outside the app model. |
|
||||
| **C. LibrePortal-orchestrated, ephemeral scanner container** — the scan task does `docker run --rm aquasec/trivy:<pinned> image --format json <image>` per installed image | ✅ **Chosen.** No resident process, no host binary, socket exposed only for the seconds a scan runs, rides the existing task/throttle machinery, and the scanner itself is version-pinned like any other image. |
|
||||
|
||||
So: **system-managed orchestration, containerized execution.** Not its own catalog app, not a host package.
|
||||
|
||||
**Scanner: Trivy** (over grype — both fine, trivy has the larger ecosystem, single pinned OCI image, clean JSON, Apache-2.0). Details:
|
||||
|
||||
- Vulnerability DB cached in a named volume (`libreportal-trivy-cache`, ~600 MB on disk); the DB refresh is the only network traffic.
|
||||
- Image access via the rootless docker socket mounted read-only into the ephemeral container (same daemon that owns the images), run as `dockerinstall` like every other docker op.
|
||||
- **Privacy posture (worth stating in user-facing copy):** nothing about your images or apps ever leaves the box — Trivy matches locally against a downloaded DB; the only outbound call is the DB fetch from ghcr. Default **on** (it's a security feature and the traffic is one public DB pull), with a single honest switch: `CFG_UPDATER_CVE_SCAN=on|off` (`off` for air-gapped boxes) and `CFG_UPDATER_CVE_INTERVAL` (minutes, **default 1440** — daily; scans are the expensive step, results don't change hourly). Both live in `configs/webui/webui_updater` next to the existing knobs.
|
||||
- Output maps straight onto the schema the frontend already renders: per app `cves[] { id, severity, package, fixed_in, url }` + global `totals`. Dedupe per image (shared base layers repeat findings), scan each distinct image once per run. **Drop the `[ ! -f ]` guard** so re-scans overwrite `cves.json`.
|
||||
- **Not alarmist** (house rule): the Security tab lists everything, but the per-app chip/badge only fires for **critical/high with a fix available** — "your box has 400 unfixable medium CVEs" red badges are noise, not signal.
|
||||
- **Tie-in with updates:** a CVE whose `fixed_in` is satisfied by the available build marks that update as a *security update* — the Security filter chip and severity sort already exist in the UI, they just start meaning something.
|
||||
|
||||
## 6. Auto-update policy (deliberately later)
|
||||
|
||||
Once detection + pinned apply are trustworthy, add `CFG_UPDATER_AUTO=off|security|all` (default **off**): `security` auto-enqueues `updater_apply` only for security updates (mirrors `CFG_HOTFIX_AUTO`'s severity-split precedent, and like `artifactApplyAuto` it only *enqueues tasks*). Not part of the initial build — auto-updating before the revert story is proven live would be backwards.
|
||||
|
||||
## 7. Build phases (each independently shippable)
|
||||
|
||||
1. **P1 — Pin foundation.** Factor the compose image-line rewrite into a shared helper (artifact op + updater both use it, all `image:` lines not just the first). Pin-on-install + reconcile-on-first-scan. No UI change yet.
|
||||
2. **P2 — Real detection.** Registry digest compare in `webuiUpdaterScan` behind the new interval knob; `updates.json` gains services/digests; fleet + per-app tabs light up with truthful data. Emit recovery fields while in there.
|
||||
3. **P3 — Pinned apply/revert.** `updaterApplyApp`/`updaterRollbackApp` rewrite pins as in §4; history carries digest refs; failure path restores pins. *After this, the Update and Roll back buttons are honest.*
|
||||
4. **P4 — CVE scanner.** Ephemeral trivy runs, `cves.json` for real, guard dropped, totals + severity chips live, security-update tie-in.
|
||||
5. **P5 — Polish.** Per-app critical-CVE / update chip on the app header (pattern exists for improvements), remote version-label enrichment, "N services" aggregation in rows.
|
||||
6. **P6 — Auto-update policy** (§6), only after P3 has soaked on a real install.
|
||||
|
||||
## 8. Rejected alternatives (for the record)
|
||||
|
||||
- **`CFG_<APP>_VERSION` config vars** — second source of truth vs the compose file, collides with the `_TAG` placeholder namespace, and doesn't solve floating `latest` by itself.
|
||||
- **Pin semver tags across all templates** — permanent curation burden, inconsistent upstreams, still blind to republished tags.
|
||||
- **Watchtower-style auto-pull of latest** — maximum freshness, zero visibility, no revert; the exact opposite of "monitor, update deliberately, roll back."
|
||||
- **Docker Scout / hosted scanners** — requires accounts / sends data off-box; against the ethos.
|
||||
- **A new top-level "Versions" area** — unnecessary; the fleet Updates tab + per-app Updates tab already are that surface, they just need real data.
|
||||
@ -88,8 +88,8 @@ it a reason to be its own thing. Rename the concept from **"App Updater"** to
|
||||
**"Updates & Improvements"** — the single front door for *everything that changes your
|
||||
install from the outside*:
|
||||
|
||||
- **App updates** (version bumps)
|
||||
- **Security** (CVEs — the urgent stuff)
|
||||
- **App updates** (version bumps — the detection/pinning/revert design is [app-version-updater-and-cve.md](app-version-updater-and-cve.md))
|
||||
- **Security** (CVEs — the urgent stuff; scanner design in the same doc)
|
||||
- **Hotfixes** (curated small improvements — §1)
|
||||
- **Recovery** (the safety net that makes all of it safe to apply)
|
||||
- **History** (audit trail of everything applied)
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user