The Storage step was a technical dump: every check's full sentence
concatenated onto the card, so the fstab line the user is meant to act on
was buried in prose nobody reads.
The card now shows plain facts and at most two short flags — "Low on
space · Won't be mounted after a reboot" — with everything else behind a
Details button. The modal carries the technical spec (device, UUID, mount
options, removable), every check with its full explanation, and the
fstab offer.
That needed the shell to stop joining checks into one string: the
generator emits a record per check, plus the fstab line as its own field,
so neither the card nor the modal has to parse anything back out of the
other.
The screenshot caught a bug this restructure introduced: summaries keyed
on check id alone, so a PASSING check printed the failure wording next to
a green tick — "This drive's format can't store file ownership" above
"Filesystem: ext4". Now severity-aware.
On writing /etc/fstab — §1 ruled it out and §6.3 now records why that
reverses. The warning is useless to the audience this is for: "add this
line to fstab" assumes SSH, root, an editor, and knowing what fstab is,
and the likely outcome is a reboot where nothing starts. What makes it
defensible is nofail + x-systemd.device-timeout, which mean a missing
device can never block boot — without that pair it would stay a non-goal,
because the failure being risked (an unbootable machine) is worse than
the one being fixed.
Enforced in the root helper: UUID never /dev/sdX, append inside a marked
block, refuse a target or UUID already described, refuse the root
filesystem, require a live mount, timestamped backup, and
`findmnt --verify` before the file is installed — a file that doesn't
parse never reaches /etc. Opt-in only.
Verified against a real filesystem: entry added and verifies, the
persistence warning then disappears on the next scan, and duplicate /
root-fs / non-mountpoint / relative are each refused with the reason.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The phasing table now reflects reality rather than intent: 0, 1, 2 and 4
are done, 5 and 6 are partial (the migrate "unknown location" prompt and
the Disks WebUI page are the gaps), 3 is untouched.
§10.8 records why the app->location index detonated, because the mistake
is easy to repeat and the blast radius was the whole machine. The file's
stated requirements — manager-owned, on the system disk — are both true
of configs/, which is why it looked like the right home. The property it
violated is that configs/ is SOURCED, so a TSV of "<slug> <path>" is a
script, and the row for the app named `libreportal` made it a
self-invoking one.
The generalisable rule is in there too: ask what a directory's contract
is, not just who owns it. configs/** is executed; containers/<app>/**.config
is executed. Both are load-bearing conventions a plain data file silently
breaks.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Resolves open question 1. The device becomes the organising concept and
the two registries become roles on it, so a user with one drive holding
both app data and snapshots stops seeing "bigdisk" twice meaning two
different things.
It extends a page that already exists rather than adding one:
/admin/system/storage is currently the Docker disk breakdown, and
webui_system_metrics.sh already builds a disks array from df. The view is
that array enriched with lsblk -J -e7 (model, transport, rotational,
removable — with -e7 dropping the ~15 snap loop devices a desktop box
carries) and joined against both registries.
The union is the part that matters more than the enrichment: a registered
drive that is currently unplugged does not appear in lsblk at all, and
that is exactly when someone opens this page. So rows come from the
registry first and attached hardware second — a missing device still
renders, marked not attached, naming the apps stranded on it. Rows key on
fs_uuid, since /dev/sdb1 reorders across reboots and would scramble the
table.
This also gives every standing badge from §6/§6.1/§6.2 a natural home,
because each is a property of the device rather than of either registry,
and it turns "type the same path into two different pages" into actions
on the row you are already looking at.
Two limitations written down rather than discovered later: ZFS datasets
report distinct st_dev but share a pool, so grouping by device
over-reports free space (btrfs subvolumes are fine); and the page stays
read-only about the system — it can show the fstab line to add, never
add it.
Phase 6 now builds this instead of a separate Storage page, which with
disks as the frame would have reintroduced the split this resolves.
Also folds in §10.5 — the dashboard gauge becomes a summary linking here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Resolves open question 2. One big disk holding both the app data and its
snapshots is what most people actually have, and two facts make it cheap:
reconcile() already chowns CONTAINERS_DIR and BACKUPS_DIR to the same
container user, so there is no permission negotiation; and sibling dirs
on one filesystem don't nest, so §3 admits them today unchanged.
The registries stay separate — that was always about trust and lifecycle,
not hardware — so §1's non-goal is reworded rather than dropped.
One hard rule survives: same drive yes, nested never. A storage location
containing a backup repo is a recursive-inclusion trap, and §3's nesting
refusal already covers both directions. What needs work is the error —
pointing storage at /mnt/bigdisk when /mnt/bigdisk/backups exists fails
the empty-dir rule, and the message must suggest a subdirectory rather
than saying "not empty", because that is the likely first attempt.
Shared fate gets the §6.1 treatment: durable badge on both locations and
a line in the backup summary, stated accurately rather than moralised —
a same-drive backup still covers accidental deletion, bad updates and
ransomware; what it doesn't survive is the disk dying. Also names the
compounding case, since nobody pictures it: a shared drive that is also
removable takes the apps and the restore path away at the same moment.
Checks: free space becomes per-device (two locations on one filesystem
draw from one pool, so a growing repo can starve the apps), plus a new
shared-device check that warns and never refuses.
Notes that this makes the naming collision worse and suggests the
resolution — a Disks view with one row per device showing which roles
LibrePortal has on it, registries separate underneath.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reboot-persistence and hot-plug become loud durable warnings instead of
refusals, and the --allow-transient flag goes away — there is nothing
left for it to unlock.
The reasoning, recorded in §6.1 so it isn't relitigated: "not in fstab"
and "removable" describe a supported setup, not a broken one — the media
library on a USB disk is a reason to want this feature at all. Refusing
there fails an honest user on their first reasonable attempt and teaches
them to hunt for an override rather than read the warning. We can afford
to allow it because the dangerous moment is start-up, not registration,
and start-up is already gated: no marker, no location, appDir fails and
dockerComposeUp refuses. Blocking at registration buys nothing the
runtime doesn't already enforce.
So the obligation moves from prevent to inform, and "loud" has to mean
durable rather than a toast: the consequence in plain words plus a
ready-to-copy fstab line at registration, a standing badge on the
location for as long as it holds, and a start-up refusal that names the
cause instead of failing generically.
Fitness checks now refuse only when a location cannot work at all
(checks 1-5), never for merely needing care. Wizard cards get three
distinct states, with warned fully selectable and explicitly not
second-class.
Marked the §10.1 gate load-bearing: it is now the only thing between an
unmounted disk and an app rebuilding itself empty, so anything that
relaxes it has to revisit §6.1 as well.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds the three surfaces the plan was missing, and the check suite that
gates them.
§6 splits validation in two, which is the point: admission (§3) is a
security question answered by the root helper and it refuses; fitness is
"will app data actually work here", needs no privilege, and therefore
can be run speculatively against a disk the user hasn't chosen — which is
what lets the wizard grade candidates before anything is committed. Ten
checks, of which two are worth calling out: a real chown probe, because
NFS root_squash reports a healthy nfs4 and then can't chown; and reboot
persistence, because hand-mount → register → install Nextcloud → reboot
is §10.1's disaster on day one. Checks recur on the crontab rather than
only gating at add time — a location that passed in March can be full,
read-only or unplugged by June.
§7 adds the setup-wizard Storage step. It follows the Metrics precedent
for a conditional step but with a better trigger: only appear if the box
has somewhere else to put things, so the one-disk case is untouched.
Candidate detection has to filter hard — this dev box has ~15 snap loop
devices to drop before the real answer shows. Refused candidates render
greyed with the reason rather than vanishing.
§5.2 puts the resolved path in the field's comment for hand-recovery,
written only on change (install, move, or a regen that finds it stale) so
it isn't churn on a user-editable file in the container-owned tree. It's
a breadcrumb: nothing reads it to decide anything, and the WebUI panel
composes its description from appDir so the editor is right even when
the comment is stale.
Phasing regrouped to 7 phases; §10.4 folded into §6; open question 5
resolved (build the flag — unattended installs have no wizard).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Pinning an app to the primary location isn't a fact about its storage,
it's a statement about whether the field is editable — so it belongs in
the comment, where the renderer's other display metadata already lives.
webui_generate_configs.sh already parses **ADVANCED** out of a field's
comment; **READONLY** parses the same way and needs no new key, no
engine-side list, and no special case in appDir or the migrate path. It
also has uses beyond this feature: crowdsec.config:72 warns that editing
a field does nothing, next to an editable input.
Also records why CFG_<APP>_STORAGE holds a location name rather than a
path, since it's the kind of decision that gets relitigated: mount paths
move (relabel, /media/<user>/…, a different session) and a name means
re-registering once instead of rewriting every app config; and migrate's
central question — "this host has no such location, where should it go?"
— is a registry lookup, not a string comparison. The config stores intent,
the manifest stores fact. Unlike backup locations there's no case for a
custom-path override: root only touches registered roots, so a path
outside the registry is unusable rather than merely discouraged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three refinements that make the plan smaller rather than bigger, plus
an honest accounting of what stays awkward.
The .libreportal-storage marker now does three jobs instead of one:
admission, mount detection and migrate provenance. Marker-lives-on-the-
drive replaces the findmnt/REQUIRE_MOUNT/fs_uuid guard family with a
single file read, and it covers cases those miss — a LUKS volume that
was never unlocked, a dropped network mount. It also unblocks adopting
a drive that already holds app data, which strict emptiness forbade;
that relaxation is free, because planting a marker requires write
access you'd have to already have.
appDir becomes the one place availability is checked, so an absent
drive fails centrally instead of needing a guard at 200 callers. And
the sweep verifies itself: retire containers_dir to a sentinel path so
survivors fail loudly on something that cannot exist.
Pinning infra apps becomes CFG_<APP>_STORAGE_PINNED, mirroring
MULTI_INSTANCE, rather than a hardcoded list in the engine.
New §9 records the four things that don't get a clean answer: the
TOCTOU window and the parent-writability restriction that closes it,
two registries that will look like one, a 90-file diff that only
process makes safe, and large moves simply being slow.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Proposal for turning the single containers root into a registry of
locations, with each app choosing one — so bulk data can live on a
second disk or an external drive while the control plane stays put.
Records what the codebase already gives us for free (relative compose
volumes, self-describing app dirs, the backup-locations pattern), the
~200 call sites that must move behind an appDir() resolver, and the
one constraint that shapes everything: the root helpers bake their
paths at install precisely so the manager cannot redirect a root
chown, so the location registry has to be root-owned with an
empty-directory admission rule.
Also notes a latent bug found while checking: restore builds its
include path from the LOCAL containers root, so migrating between
hosts with different --containers-dir values silently restores nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four fixes that make the auto-updater a trustworthy background system:
* CFG_UPDATER_WINDOW (default 06:00-08:00 host time, right after the
05:00 backup cron; HH:MM-HH:MM wraps midnight, 'always' = any time).
Gates only the enqueue — scans keep running all day, so the Updates
page stays current and pending updates visibly wait for the window.
Malformed values fail closed and are rejected by the WebUI validator.
* "Check now" actually checks: an explicit `updater check` sets
UPDATER_REGISTRY_FORCE=1. The flag existed but nothing ever set it,
so the button silently reused the 6h digest cache and could not find
a build the user knew had shipped. Force also overrides interval 0,
which now means "manual-only" as documented in the roadmap.
* Registry stamp moved from /tmp to <system>/logs: the task processor
runs under PrivateTmp, so daemon and CLI each kept a separate 6h
clock and the daemon's reset on every service restart.
* A failed automatic attempt is no longer invisible: the scan emits
auto_attempted_digest (the one-shot no-retry stamp), and when it
matches the available build the UI stops promising an install that
will never come — per-app detail explains, the fleet row gets an
"auto failed" chip, and the Overview board counts it as needing you.
Also corrects the CFG_TIMEZONE label: it sets the containers' TZ only;
scheduled tasks follow the host clock (timedatectl), and the old
"Timezone for scheduled tasks" wording promised a knob that never
existed. The window + auto_window display state plainly WHEN updates
land, answering "how does the user know when the next update happens".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds the decision half of the app updater. Detection (P2) and the
snapshot-first apply/revert (P3) were already real, but nothing ever
pressed the button — every update waited for a click.
CFG_<APP>_UPDATE_TYPE=auto|manual per app, default auto (33 templates)
CFG_UPDATER_AUTO=true|false master switch, default true
updaterAppPolicy resolves the two the way backupResolveStrategy already
resolves backup strategy: the global switch can only make things more
manual. updaterApplyAuto runs at the end of `updater check` and enqueues
the ordinary updater_apply task for each auto app that has an update —
never applies inline, so an automatic update is the same code path, task
log, History entry and Roll back button as a manual one.
Safety: each attempt stamps its target digest under generated/auto/, so a
build that fails is rolled back and then left alone rather than retried on
every scan; in-flight updater tasks are skipped so scans can't stack.
Tracked end to end: updates.json carries each app's resolved update_type,
History entries carry trigger=manual|auto. The WebUI says whether updates
install themselves, chips only the apps that opted out, labels automatic
history, and — since an auto app's pending update needs no decision — keeps
it off the Overview board's "Needs action" view.
Also fixes artifactApplyAuto enqueueing without --detach: it runs inside
the single-threaded task processor's own poll, so following the new task in
the foreground waits for a task that cannot start until it returns.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
P0 anchor, P1 config-first pin, P2 registry detection + display, and P3 honest
pinned apply/revert are all built and deployed. Records the detection nuance
(digest-compare fully covers rolling apps; versioned newer-version enumeration
is a P5 add) and the remaining deferred work.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The scan read the app's "current image" via `grep -m1 image:` — the first
image line. For apps that declare a companion service first this mislabels the
app: ollama lists `ollama-webui` (open-webui) before `ollama-service`
(ollama/ollama), so an app named ollama reported open-webui's version.
Add `updaterPrimaryImage`: resolve the version anchor from the app's
`<slug>-service` image (the universal primary-service naming convention,
33/33 apps; underscores→hyphens for slugs like libreportal_catalog). Falls back
to the first image line off-convention. Used by both the version and CVE loops.
Verified: only ollama changes (→ ollama/ollama:latest); nextcloud, mastodon,
jitsi, gitea, vaultwarden anchors unchanged.
Also revise docs/roadmap/app-version-updater-and-cve.md to the config-first
direction agreed this session:
- CFG_<APP>_VERSION ADOPTED (was rejected): the #LIBREPORTAL tagging system
makes the config the source (compose tag derived from it), not a second one.
- Two version TYPES — versioned (real tag, version picker) vs rolling (floating
channel, digest-freshness); digest is the uniform detection engine for both,
version numbers are display enrichment. Answers "why not just compare numbers"
(most upstreams publish none; no universal latest-version API).
- Multi-service anchor = <slug>-service, not first line; lock-step sets (jitsi)
= one channel → several image lines; sidecars tracked-by-digest, not headline.
- Phases updated: P0 anchor (done) → P1 config-first pin → P2 detection+display.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
New containers/trivy app (aquasec/trivy in server mode, private API port,
docker socket + cache volume) in the security,recommended categories, plus
placement in the setup wizard recommended step, the server-side install
tier after crowdsec, and the CLI first-install prompt. The updater's CVE
scan (design doc P4) will gate on this app being installed; §5 of the
design doc updated to record the app-based decision.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Fills the gap updates-and-distribution.md always deferred: how versions
are actually detected, pinned, and reverted, and how cves.json gets real
data. Core decisions: floating tag stays the channel while the live
compose pins the digest (making detection possible and rollback honest),
and CVE scanning runs as a LibrePortal-orchestrated ephemeral Trivy
container (no host binary, no always-on scanner app).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Roadmap: Phase 6 entry (app bundles + Browse-&-Add, built 2026-07-03),
the app-bundle format addendum (meta object, tarball contract, collision
policy, trust-then-quarantine ordering), deferred bullet trimmed to what
actually remains (community quarantine, taps, theme/component, canary
countersign) with a pointer to marketplace-website.md for the website
layer. development.md gains the 'Publish an app to the marketplace
catalog' section (make_app.sh flow + the app add loop).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The marketplace system ships as containers/libreportal-marketplace (a normal
app definition, hidden behind CFG_DEV_MODE via a new CFG_<APP>_DEV_ONLY
convention): it serves the signed catalog tree plus a client-rendered browse
UI over the same index.json every box verifies. The official instance is our
own install of this app; self-hosting a marketplace = installing it.
Community submissions stay PR -> review -> sign (phase 2).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Design note for a per-app Files tab scoped to LibrePortal-managed files
(not system files): four file buckets (hidden/view-only/editable/lever),
the advanced/dev mode as the single escalation lever (not per-file flags),
and the hard rule that the flag is UX-only while the locked-down task CLI
stays the security boundary (jail + secret allowlist).
Includes the live UID-access spike: the manager owns and can write the
config tree (/libreportal-system/configs) directly, but the container tree
(/libreportal-containers/<app>) is dockerinstall-owned — readable, not
writable — so config edits need no helper while compose-class edits do.
webui_logins is manager-readable, so secret-hiding must live in the CLI
allowlist, not in perms.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Update §8.7 + the banner + §1 TODOs to reflect that Phases 2–5 shipped today
(apply/revert pipeline, severity-split auto-apply, the WebUI Improvements stream
+ per-app chip, and make_hotfix.sh). Only the registry/marketplace stays
deferred (demand-gated by design).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The four-lens design panel finished (marketplace-first ranked top) and
confirmed the format; graft in the strongest refinements it surfaced so
the spec is genuinely "done":
- Publishers MAP trust anchor: `publisher` is now a key into an index-root
`publishers` map ({display, role, key}) the team-signed index vouches
for, not an inline {name,trust}. An artifact's claimed trust is honored
only if the publisher's role permits AND its sig verifies against that
key — so a community key can never self-certify as official. This is the
load-bearing trust mechanism for the marketplace seam.
- Two-tier reversibility: a per-op `undo` array (precise revert) plus the
snapshot (dirty-op fallback).
- All-or-nothing dry-precheck-all before any snapshot; unknown op rejects
the whole artifact at validation.
- Canonical-bytes signing rule (sign the exact artifact bytes, never
re-serialize on the box) + warrant-canary countersigning index_serial.
- Op vocabulary grown to the full set (set-data-file as the bridge to
bundles; set/unset-compose-env; ensure-compose-up/restart-service).
- Envelope gains version/supersedes/reversible + richer applies_when
(image_match/requires/conflicts).
- CFG_HOTFIX_AUTO + staged rollout / randomized delay / recall-via-supersedes.
- Flag the VERIFIED existing bug: updaterRecordHistory silently skips the
audit entry when jq is absent (cli_updater_commands.sh:154-168) — Phase 2
must make it fail-closed; "nothing silent" depends on it.
- Phases re-sequenced (P2 heart, P3 auto-apply, P4 WebUI, P5 make_hotfix.sh,
deferred registry).
Spec-only change — no code; the Phase 1 read primitive is unaffected (it's
a generic verified fetch; publisher/envelope internals are Phase 2).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Build the read side of the unified distribution primitive from
docs/roadmap/updates-and-distribution.md: one team-signed catalog
(index.json) on the same channel as latest.json, listing type-tagged
artifact envelopes. A hotfix is the first artifact type; apps/themes/
components are future envelope rows through the SAME pipe — the
marketplace seam is just the `type` + `payload.kind` fields.
Phase 1 is fetch + verify + parse only (NO mutation; the snapshot →
ops → rollback → History apply verb is Phase 2):
- Factor `lpVerifyMinisig` out of `lpFetchRelease` (scripts/source/
fetch.sh) — one trust anchor (the root-owned footprint key) now
shared by releases and the index; refactor `lpFetchRelease` to use
it (behaviour-preserving, still fail-closed).
- scripts/source/artifacts.sh: `lpFetchIndex` — download →
verify-before-parse → `valid_until` freshness (anti-withholding) →
`index_serial` monotonic high-water (anti-rollback, TUF-lite) → emit
verified JSON. Trust core is jq-free; parsing accessors prefer jq
with a grep fallback.
- `libreportal artifact index` (scripts/cli/commands/artifact/) —
read-only front door that fetches, verifies and lists. Runs directly
like `updater check` (no task; no mutation).
- Regenerate the source arrays + lazy-load function manifest for the
new files.
Doc: promote the format from vision to spec (§8) — 3 layers
(INDEX/ENVELOPE/PIPELINE), the bounded declarative op vocabulary (no
run-script, ever), the apply pipeline mapped onto existing functions,
the marketplace seam, and resolutions for all five open forks.
Self-tested 12/12: trust core fails closed (real key + no minisign →
refuse), happy path, stale-refused, rollback-refused, signature-refused,
jq + grep parsing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Sort docs/ into guide/ contributing/ architecture/ roadmap/ and rename
to consistent kebab-case (USER->guide/install-and-use, FOOTPRINT->
architecture/system-footprint, frontend-modularization->architecture/
webui-architecture, etc.). Add a docs/README.md index and a docs/
CONTRIBUTING.md pointer so the forge still surfaces the contributing
guide. Fix every reference (README, init.sh comments, frontend code
comments, and the USER<->DEVELOPMENT cross-links). History preserved
via git mv. Root stays README.md + CLAUDE.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
