Moving the results under the Folder field kept the placement but threw away the
presentation: the card became a line of prose. Path, snapshot count and age
read better as a card, and that is what the found list already used.
One renderer now serves both. The same fact should not look like two different
things depending on whether it came from the scan or from pressing Check — only
whether it is still a choice differs, so a pickable one is a <button> and a
confirmed one is a <div> with a quiet accent edge and no hover. A card that
looks clickable but is not is worse than one that does not.
The geometry moved off the button-only selector for the same reason; the
confirmed card is a div and was collapsing without it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The results were rendered above the whole form, which made them read as a
separate step rather than as an answer about the Folder input. They now sit
under that field, inside its box, alongside the verdict.
What is shown depends on how many were found, because those are different
situations:
one it is the answer, not a choice — filled in, with its verdict. No
card, because the card and the verdict said the same thing twice
several listed as buttons, most snapshots first; the field stays empty,
since this is genuinely ambiguous and guessing would be worse
none says so — "we looked and there is nothing here" is information
And a bug: the single result filled in only the PLACEHOLDER, so the field was
still empty and pressing Check replied "give a full path, starting with /"
about the very backup shown directly above it. It fills the value in now, and
shows the verdict straight from the scan rather than making the user press
Check to be told what is already on screen. Never overwrites something already
typed — the user's own answer outranks anything we guessed.
_adoptSingleResult moved into renderFoundBackups: "show what we found, and if
there is exactly one, take it" is one behaviour, and splitting it meant the
test could only reach half of it.
Not done, deliberately: listing individual snapshots before the password. The
count is a directory listing, but each snapshot's identity — host, tags,
contents, when they are from — is in the encrypted object. Unlocked, all that
could be shown is a column of hex IDs and file timestamps, which is not
something anyone can choose between. After unlocking, the Contents step already
lists the settings snapshot and one entry per app, which is how people think
about it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Backup step opened with an empty box and /mnt/usb/libreportal-backups as
the placeholder — a path nobody has, presented as the shape of the answer.
Someone rebuilding a server was being asked to recall from memory the one thing
they came here because they had lost.
Two additions, and the point of both is that neither needs the repository
password. A restic repository keeps one file per snapshot under snapshots/, so
"is there a backup here, and how many" is a directory listing. Nothing is
decrypted — reading what is IN those snapshots is the next step, and that does
need the password.
restore scan looks where a backup actually is: this install's own backups root
(the disk often survives), every location the install already knows about, and
one level under each non-OS mount, a just-plugged-in drive being the other half
of "the system drive died". Bounded to named shapes and maxdepth 1, never a
filesystem walk — a scan nobody waits for is a scan nobody uses. Results are
buttons, most snapshots first, each showing its count and the age of its
newest snapshot; clicking one fills the path in.
restore verify <path> answers the same for a typed path. Its most useful answer
is the near-miss: pointing at the folder that CONTAINS the repositories rather
than at one of them, which it names and offers as a button rather than
explaining the distinction in prose.
A repository is recognised by config plus the snapshots, keys and data
directories together. config alone would match any folder with a file of that
name, and offering a stray directory as someone's backup is worse than finding
nothing.
The placeholder now comes from this machine — the first repository found, or
the install's own backups root — since a placeholder's job is to show the shape
of the answer and only a real one does that. The backups root is in the storage
feed for it.
The found entries are buttons and had to own their geometry: .setup-app-card
carries no layout, it is a bare wrapper elsewhere, so a <button> wearing it
collapsed to one cramped line.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The advanced Storage step offered /mnt/disk/apps/libreportal-system as the
default home for LibrePortal's own tree. A registered location's path is where
APP DATA goes and is usually a subdirectory of the drive, so deriving anything
else from it nests that thing inside the app data — LibrePortal's own files
buried under it, on a path that reads as a mistake because it is one.
Both defaults now come off the location's mount point, which meant adding
"mount" to each entry in the storage feed; only the system block carried one.
LibrePortal /mnt/disk/apps/libreportal-system
-> /mnt/disk/libreportal-system
New apps, unregistered drive /mnt/disk
-> /mnt/disk/libreportal-apps
New apps, registered location unchanged — it exists and may hold data, and
proposing a different directory on the same
drive would strand it
Names follow the layout the rest of the product uses (libreportal-system,
libreportal-containers, libreportal-backups) rather than a bare "apps", so a
drive shared with anything else stays legible.
collectStorage() no longer registers the drive picked for LibrePortal. A
storage location is somewhere app data lives; the system tree is not app data
and relocate creates that directory itself as root. Picking a drive there was
producing a location nobody asked for, on a mount chosen for something else.
Also in this change, from the Backup step:
- The backend-specific fields are boxed under their own heading with a note,
so choosing SFTP reveals "the SFTP part" rather than three more loose rows.
- Fields had no vertical spacing. .setup-step gives its DIRECT children a
16px gap, which is where every other step's fields get theirs; these sit a
level deeper inside a .setup-section and inherited none of it, so each
input ran into the next field's label.
- Two field icons carried U+FE0F. Those codepoints have a text form and the
selector only requests the emoji one, so they sat on a different baseline
to the plain emoji beside them — the box measured perfectly centred while
the glyph did not look it.
- ?mode=restore&type=sftp makes the restore branch reachable by URL. Getting
there previously took a click and a change event, so every screenshot and
test had to drive the page before it could look at it.
Two test bugs fixed while doing it: a duplicate `const visible` in one scope
(a parse error, so the whole eval silently returned nothing), and a stub that
covered the POST but not the poll, leaving a 60s loop running that kept the
page from ever going network-idle.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two things about the restore source step were wrong.
The progress bar said "Source" while the heading said "Where is your backup?",
which reads as two different steps. Every other step's section title is its step
name — Storage, Backups, Import — so this one is "Backup" in both places, with
the friendly question moved to the hint where the rest of the wizard puts it.
Contents and Rebuild got the same treatment.
And the fields used the Storage step's label-left rows. That layout suits a
column of dropdowns; a form of typed values in the middle of a wizard that
looks nothing like the rest of it just reads as unfinished. They now use the
same shape as Identity: a label with a tooltip, then an icon beside the input.
Every field has both, including the password.
On automating the relocate: it cannot be a WebUI action, and the reason is the
thing the privilege model rests on. Root helpers have their paths baked at
install so the manager cannot redirect a privileged operation by editing
something it owns; relocating re-bakes those paths, so a helper that did it
from a caller-supplied path would hand the manager the whole trust boundary.
Narrowing to "registry targets only" does not help either — the manager can add
to that registry by design. libreportal-relocate says this at the top and is
deliberately outside the manager's sudoers.
What was fixable is the part that actually annoyed — being handed a command
with no idea whether it worked:
- Copy button, with a fallback that selects the text and names the keys,
because clipboard access needs a secure context and a LAN install on
http:// is not one.
- The pending move persists to localStorage, not session state: it happens in
a terminal minutes or days later, after the tab is gone.
- A watcher on every page shows the outstanding command, polls for the move
landing, and reloads to the homepage — whatever route you were on belonged
to the old install.
"Landed" means the host REPORTS its system dir as the target, which is why
system_dir is now in storage.json. Deliberately not "the server restarted": an
ordinary container restart is indistinguishable, and announcing a relocation
that never happened is worse than saying nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A repository holds one system=config snapshot and one snapshot per app. They
are separate because they are used separately: the settings tree is small,
changes rarely and is meaningless per-app, while app data is large, changes at
its own rate, and has to be restorable, movable and ageable on its own — which
is what the per-app tag buys.
The step listed "Apps" and "Domains" as peer sections, which hid that entirely.
It read as though a backup held three kinds of thing, and gave no clue that the
domains come OUT of the system snapshot rather than being a third kind.
Now: a Settings section (one snapshot, dated, saying plainly that it carries
every backup repository with its credentials and is restored first because it
is what makes the others reachable) with the domains nested under it and each
one's DNS verdict; then an App data section, one snapshot each, dated and
sized. A repository with app data and no settings snapshot says so — the
consequence, that repositories and logins do not come back, is not something to
find out afterwards.
restoreInspect emits that shape now, assembled with jq against the discover
JSON rather than by hand-rolled string concatenation, and carries each
snapshot's date: a backup's age is what people actually judge it by.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Third occurrence of the same defect, in a third directory. config-adopt gave a
file the destination did not already have 0640 — "these can hold secrets, so
default closed" — but the config tree is 0755 and the backup account reads all
of it. One 0640 file makes restic report permission denied, write an INCOMPLETE
snapshot and exit 3, so the entire system-config backup is reported failed, and
a first-run restore has nothing to restore from.
A new file now inherits from a sibling in the same directory, matching whatever
that tree's convention is rather than having a mode picked for it.
Since this has now happened three times for three unrelated reasons, the test
asserts the class rather than the instances: no file anywhere under configs/
may be unreadable by the backup account. Mutation-checked — chmod 0640 on any
one config makes it fail.
Also repaired the files the earlier buggy adoption runs clamped on this
install, and fixed lp-storage-step-test's hardcoded step index, which the new
Start step had shifted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The wizard's first question is now "is this a new server, or a replacement for
one?", which §2 of the roadmap described and nothing implemented. Start asks,
and the answer selects one of two disjoint step sets:
new Start > Experience > Identity > Domains > Storage > Backups
> Import > Recommended > (Metrics)
restore Start > Source > Contents > Rebuild
Disjoint deliberately. A restore is never asked for an install name, domains or
an app list — the backup answers all three, and asking invites someone to type
an answer that is about to be written over. The test asserts non-overlap in
both directions, not just that the restore steps appear.
Source collects the repository the way the Backup page does, minus everything
that only means something for a place you write TO: no retention, no schedule,
no enable toggle. The password leaves through the one-shot secret:<ref> channel
and is cleared from the DOM, and the test asserts the value never appears in
the payload — that payload reaches a task command line, and tasks are recorded
world-readable.
Contents is the reconciliation, rendered: apps with sizes, and each domain with
a verdict, checked through the same /api/setup/dns-check the Domains step uses
rather than adding a second way to ask. Plus the offer to leave the strays out
until DNS is repointed.
Rebuild runs `restore rebuild`: settings first (they carry every other
repository's credentials), then domains, then apps with no explicit list so
bulk discovers and re-preflights them itself.
Inserting Start shifted every step index by one. validateStep was a chain of
idx === 1 … idx === 6, carrying a comment that already explained which earlier
insertions had moved them — it is keyed on the step name now.
lp-storage-step-test had the same pin and did not survive: it called
validateStep(3) for Storage, which had become Domains, and reported that
nothing blocked. That reads exactly like validation being broken. Tests look
their step up by name now too.
Also: locationRemove's fix means a failed connect can finally clean up after
itself, so a wrong password no longer leaves a dead destination behind on every
retry.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
restore inspect answers "what would a restore from here bring?" without writing
anything: hosts, apps with sizes, and the domains — read out of the
system-config snapshot with engineDumpFile, the same way the preflight reads an
app manifest. Knowing a backup hands you six domains of which four point
elsewhere, before committing, is the difference between a rebuild and a
surprise.
restore connect is the WebUI entry point: creates the location from a base64
payload, redeems the repository password from the single-use secret channel,
inspects. Deliberately does not engineInitLocation — every other path that
creates a location initialises it because it is about to write there; this one
reads a repository that already exists. This is what unblocks the constraint
app_portable.sh records: a .lpapp could live in the WebUI because nothing
secret crosses from browser to host, and the repository restore could not. The
secret:<ref> channel is that missing piece.
A wrong password is the ordinary case and the user retries, so a failed connect
removes the location it just made. Otherwise every attempt left another
half-configured destination behind.
Three things found by using it:
- locationRemove never worked. It unlinked as the container user, but
configs/ is manager-owned, so it was always denied — and the result was
never checked, so isSuccessful printed anyway and a "removed" location came
back on the next listing. Now runInstallOp, and the directory is checked.
- webuiSecretSweep had no callers. An abandoned flow left its repository
password on disk forever. The sweep now runs in /api/setup/secret before
each write, tied to the one event guaranteed to happen.
- Adoption took the WebUI down. config-adopt chowned every adopted file to
manager:manager 0640, and webui_logins is bind-mounted into the container,
which then could not read its own credentials: exit 137 with no log line.
It also clamped every parent directory it passed through, closing
configs/webui and configs/backup to the container user.
The fix is a principle, not a special case: a restore replaces the CONTENT
of a config file and nothing else. The live install already knows who may
read each one. Adoption preserves the destination's ownership and mode,
defaults closed only for a file that did not exist, and never
re-permissions a directory it passes through.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The installer's restore path printed "Settings restored" and had never restored
a setting. backupRestoreSystemConfig only STAGES — right in general, since
overwriting a running control plane's config should not be automatic — but
nothing ever adopted the staged tree. The backup locations, domains and logins
landed in $restore_dir/system-config and stayed there.
So adoption is its own step now (`restore adopt`), allowed only on a machine
with nothing on it yet. backup/locations/ is adopted as a subtree, since the
index is part of the path and that directory is the whole point: it holds every
repository and its credentials, which is what makes "one password unlocks the
rest" true. Deliberately NOT adopted: the container account and its generated
password, port allocations, docker/rootless wiring, and storage/locations —
those describe the old machine, and a registry of drives this box does not have
would make every placement lookup resolve to a phantom.
The guard failed in the shape this project keeps hitting. It globbed the
containers directory, but the manager can traverse that without listing it, so
the glob returned a literal '*', the loop skipped it, and the function returned
"first run" on a machine with three apps. It adopted over a live install in
testing. It now asks the container user for the listing and fails closed: an
unreadable directory means "in use", never "empty".
Two config modes were inverted, found because a restore cannot restore from a
snapshot that was never taken:
- storage location configs were 0640 and hold no secrets. The backup runs as
the container user, could not read them, and restic wrote an INCOMPLETE
snapshot and exited 3 — so EVERY system-config backup failed once a second
storage location existed. Now 0644, with the test asserting they stay
secret-free so that mode remains defensible.
- backup location configs were 0644 and hold the repository password; nobody
could read them. They cannot simply be tightened, because the backup has to
read the credentials it uses — so the directory carries the restriction
(config-secure, manager:container 0750) and the file stays readable to the
two accounts that belong.
Fixing that surfaced a third: config-adopt clamped existing parent directories
to manager:manager 0750, closing configs/backup to the container user and
breaking the credential read the directory fix had just preserved.
restore domains reports which restored domains point here, and the installer
offers to drop the strays. Three verdicts, not two: setupCheckDomainPointsHere
falls back to hostname -I, and comparing a public A record to a private 10.x
address would condemn every correctly-pointed domain on a LAN-only box, which
is the deployment this product targets. Unverifiable is never offered for
deletion.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Follow-up to fd0a0fd, which put a "Custom path…" entry in the drive dropdown.
That was the wrong shape: the dropdown answers "which disk", and an entry
meaning "actually, let me type a directory" sitting in the same list makes
typing one look like one of the normal answers. Picking a disk is the whole
question for most people.
The dropdown now offers drives and nothing else. Exact paths are a section
under it, advanced only — the same reason Metrics is advanced-only, since the
directory under a chosen drive is operator detail and the beginner path
deliberately does not get a wall of that. Beginners get the drive's default,
which is what they would have typed anyway.
Each field is prefilled from the selected drive and follows it when the drive
changes, so a path belonging to the old drive is never left behind. The
LibrePortal row appears only for a non-primary drive: relocating it onto the
drive it already sits on is not a move.
storageSystemChoice stays a drive, and the new storageSystemTarget holds the
relocate path. collectStorage() registers what the dropdowns point at, and a
system directory is not an app-data location — registering .../libreportal-system
as one would be wrong. Asserted directly.
The validator skips an untouched default: that is whatever the install already
uses, and second-guessing it would reject a legitimate layout.
lp-storage-custom-test -> lp-storage-step-test, and it no longer waits for a
drive to happen to be unplugged: it injects an unmounted candidate and
re-renders, so the offline assertions run everywhere rather than only on a
machine where ambient state obliges. That injection had its own trap worth
recording — renderStorage() rebuilds the selects, so a reference held across it
points at a detached node and setting .value on it succeeds while changing
nothing. Three assertions passed against a control no longer in the page.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A registered drive that is unplugged rendered through the same path as any
other candidate — a "needs care" badge, "free of" with no numbers on either
side, an empty meter. To a first-time installer that reads as two broken disks
the scan turned up, with nothing tying the card back to a drive they registered
and later unplugged. Say "not connected", name the path, and draw no meter: a
meter with nothing in it is a claim about free space nobody measured. The same
locations are withheld from the dropdowns, since the wizard cannot stat a
directory on a drive that is absent.
Both dropdowns now end in "Custom path…", for a NAS mount or an LVM volume the
disk heuristics never rank as a candidate. Validation goes through
validateStep(3) rather than a disabled button: the apply side already refuses a
relative or system path, but its refusal is to fall back to the system disk,
and that is indistinguishable from having chosen the system disk on purpose.
A typed path is not a registered location, so setup_apply registers it via
storageAdd — which is what keeps the empty-directory admission rule and the
fitness checks in play — named after its basename, so it reads as "nas" rather
than "location-3" in the placement menus.
libreportal-storage: accept the name the listing prints. remove matched id and
path only, so `remove location-3` failed against a row displayed as
location-3. Root-owned helper changed, so footprint_version 10 -> 11.
Expose window.setupWizard: the instance was local to a promise in the
orchestrator and unreachable from the console or a test.
lp-storage-custom-test drives the step in a browser. Two holes it found in the
tests themselves, both the shape it exists to catch — a check whose failure
mode is to not run:
- It counted the cards that say "not connected" and asserted over those.
Turn the feature off and the count is zero, every() over an empty list is
true, and the block passed having checked nothing. The expectation now
comes from the feed.
- Both browser tests exited 0 whenever the page returned nothing. Under sudo,
where chromium will not start, they reported PASS having asserted nothing.
They now probe with `lp-shot --url` and curl: if the WebUI answers HTTP the
browser is the only thing that can have broken, and that is a failure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The report added last commit never fired. It reads
/usr/local/lib/libreportal/storage.roots to list which locations still hold app
data — and that directory is removed earlier in the same function, so it found
an empty registry and printed nothing.
Which is precisely the silence it was written to prevent, and worse than not
having it: indistinguishable from "there was nothing left". Caught on a clean
teardown that left an app on each of two test disks and said so about neither.
Snapshot the registry before the removal and report from that.
scripts/dev/lp-uninstall-report-test runs the real capture and report fragments
from init.sh with the deletion between them, so what it guards is the ORDER
rather than the wording. Verified by moving the capture back after the delete.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The dialog test caught the new option immediately, which is what it is for.
Updated for five backends, plus checks specific to Connect: it must be offered,
disabled, labelled so the reason is visible rather than just greyed out, carry
no credential field while unusable, and point at the free equivalent that works
today.
connect is left out of the per-type field-swap loop on purpose — a disabled
option cannot be selected, which is the behaviour we want and is asserted
directly instead.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported: the dropdowns in Add destination don't work. They rendered correctly,
reported as enhanced, and did nothing when clicked.
custom-select portals its popup into <body> at z-index 1200, chosen — as
forms.css says in as many words — to clear eo-modal at 1100. The wizard raises
its modal to 10000, because at 1100 a modal opened from inside the wizard
rendered behind the wizard itself. That fix silently broke the other invariant:
the popup then opened behind the dialog that owns it. Raise the popup with it,
scoped to the wizard so nothing else's stacking moves.
The test already asserted the select was enhanced, which was true and useless —
the control was enhanced, it just could not be reached. So it now hit-tests:
open the popup and ask what is actually on top at its own centre, then click an
option and check the value, the button label and the field group all follow.
Verified by removing the rule again: two checks fail.
That is the second time this pair has bitten (the modal itself did the same
thing earlier), so the rule and the reason now sit together in one comment.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported after looking at the step: the add button unstyled, the dialog missing
the fields a backup location actually has, and its dropdowns not working. Three
real faults, and one reason all three shipped.
* "+ Add destination" carried class .setup-add-domain, which I invented. The
real one is .setup-domain-add, so no rule matched and it rendered as a bare
browser button in the middle of a styled form.
* The dialog asked for name / type / host / user / path / password. A backup
location has SSH port and auth method (key or password — key is the default
and needs nothing typed), S3 access and secret keys, B2 account id and key,
and a path mode. It now asks for what each backend needs, with the wording
taken from the location config so the wizard and the Backup page describe
the same thing the same way.
* .setup-field styled input[type=text] and [type=email] but not [type=password]
or [type=number], so a credential field and the SSH port rendered unstyled
even inside a correct container.
Only the credentials go through the secret channel — SSH password, S3 secret
key, B2 account key. The rest is ordinary configuration and travels as itself.
The reason all three shipped is that I checked the step by querying the DOM and
never looked at it. Structural checks cannot see an unstyled control, and a
dialog is behind a click so a screenshot cannot reach it either. So:
lp-shot --eval <route> <js> run JS in the page and print the result
LP_SHOT_EVAL=<js> run JS before a capture — open a dialog, then shoot
and scripts/dev/lp-backup-dialog-test drives the whole thing in a real browser:
opens it, swaps every backend and asserts only that backend's fields show,
toggles SSH auth and asserts the password field follows, submits, and asserts
the credential is not left in the DOM.
Its styling check needed two attempts, which is the point of mutation-testing
it: "is the background transparent" passes for an unstyled button, because a
native button is grey rather than transparent. It now compares the control
against a bare <button> in the same parent, so "no rule matched" is what fails.
Verified: reintroducing the wrong class fails the test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The step asked one question — pick a destination, or "not now" — while the
system underneath already had a full location model: eight backend types, per
location engine, path mode, credentials and retention, and a generated
locations.json carrying all of it. None of that was reachable during setup, so a
second destination, or even seeing where the first one points, meant finding the
Backup page afterwards.
Now it mirrors the Storage step — the choice above, the list below:
Backups Automatic — daily, on a schedule | Manual
Destinations Local disk [default] /libreportal-backups/1 [Edit]
+ Add destination
Automatic/Manual needed a setting, because there was no off switch:
crontabSetupBackupScheduler installed the entry unconditionally. CFG_BACKUP_MODE
is explicit rather than overloading "empty schedule", so it reads properly in
the config editor too, and Manual REMOVES an entry that is already installed
rather than merely declining to add one — otherwise answering Manual changes
nothing. The schedule itself is left alone, so switching back restores the time
the user picked.
Destinations are seeded from locations.json, so the default one is shown and
editable instead of being discovered later, and only entries the user actually
added or changed are submitted. A destination on the same disk as the app data
says so on the card rather than in a paragraph under the step.
Remote destinations are what the secret channel was for. The wizard payload is
base64'd into a task's command string and tasks are recorded world-readable, so
a password is POSTed to /api/setup/secret, which writes it where only the
manager can read it and returns an opaque reference; the reference travels in
the payload and setup_apply redeems it once, at the write. A reference that
cannot be redeemed leaves the password alone and says so, rather than blanking
it.
Verified in the browser on a clean install: the step renders both modes, lists
the existing destination at its resolved path, and the add dialog swaps between
local and remote fields. scripts/dev/lp-backup-setup-test covers the apply side,
including that what reaches the config is the secret and never the reference.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A password typed in the WebUI has to reach the host, and both existing routes
leak it. As part of a task's command string it lands in
frontend/data/tasks/*.json — 0644, inside a world-readable directory — and is
visible in `ps` while the task runs; as a plain file there it is either
world-readable at 0644 or unreadable by the manager at 0640. Verified still true
on a clean install. A backup repository password sent that way is the key to
every backup the user has, readable by any local account.
libreportal-ownership gains `secret-dir`: the mirror of _webui_bind_access.
That one makes manager-owned config readable by the container; this makes a
container-written file readable by the MANAGER. The directory is
<container>:<manager> mode 2730 — setgid so each file inherits the manager's
group, the container writes it 0640, and 0730 leaves the directory unlistable
because the manager is handed a filename rather than going looking. Group rwx
is what lets it unlink after reading.
The WebUI then sends a REFERENCE ("secret:<id>") wherever it used to send the
value, and configUpdateBatch redeems it at the last moment before the write.
That is the single point every config write from the WebUI passes through, so
this covers every password field rather than only the backup ones — which is
what docs/roadmap/first-run-restore.md §4.1 asked for. A reference that cannot
be redeemed leaves the field unchanged rather than blanking it.
Verified on a live install: the container drops a secret, the manager applies it
by reference, the file is unlinked, `nobody` can neither read nor list it, and a
second redemption of the same reference fails.
footprint_version 9 -> 10 (root-owned helper changed).
Also fixes a block of constructor initialisations I spliced into the middle of
renderStorageChoices in aa44e0b: on a single-drive box — the case in the
screenshot that prompted this — rendering the Storage step silently reset
backupDest and cleared the import selections.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
dockerComposeUp and dockerComposeDown derive compose_file from $compose_setup,
which setupBasicScanVariables reads from CFG_<APP>_COMPOSE_FILE — a variable
that is only set once the app's config has been sourced, and is not always. A
restore wipes and re-creates the app folder around those calls.
setupBasicScanVariables already handles that, falling back to the standard file.
Neither compose function did: with compose_setup empty, neither branch ran,
compose_file stayed UNSET, and the guard
[ ! -f "$(appDir "$app")/$compose_file" ]
then tested the app DIRECTORY, which is never a regular file. So the app was
reported as having no compose file and quietly not started.
Found restoring an app on a clean install: the restore ran to completion and
reported success, having neither stopped nor started the app —
---- 3. Shutting down container(s) for restoration
! Notice Unable to find the compose file to docker compose down this application.
---- 10. Starting up the linkding docker service(s)
! Notice Unable to find the compose file to docker compose up this application.
— while docker-compose.yml sat in the app directory the whole time. Verified:
with CFG_LINKDING_COMPOSE_FILE unset, dockerComposeUp now starts the app.
scripts/dev/lp-compose-file-test covers the empty case, the normal one, and a
genuinely missing file so the guard still fires when it should.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The resolver already supported per-app placement — CFG_<APP>_STORAGE names a
location and appDir sends data, compose and config there — and 37 of 39 app
templates ship the field. What was missing was choosing AT INSTALL TIME. The
only routes were editing a config by hand before installing, or installing onto
the default disk and then `app move`ing it, which copies the data twice.
libreportal app install <app> --storage=<location>
and the App Center's existing storage dropdown, which travels inside
config_variables. Both resolve to one answer in storageChoiceFor, so there is a
single code path.
Ordering is the whole difficulty, and getting it wrong is quiet. installApp
copies the app template into appDir(), sources it, and later applies the form
overrides. The choice has to be live before the copy (or the directory is
created on the wrong disk), written into the config before the source (or the
template's "default" wins and every later appDir in that process returns the
primary root), and folded into config_variables (or the override pass writes
"default" back). Miss any one and the directory and its config disagree — which
resolves correctly only until something sources the config.
Refuses an unknown or unmounted location, an existing directory, and an app
whose template marks the field **READONLY** (fixed to the primary root because
other apps reach it by literal path — storageMoveApp already refuses to move
those, and installing one elsewhere is the same violation from the other end).
Three shipped bugs found making this work:
* updateConfigOption chose its write helper by comparing the path against
$containers_dir — the PRIMARY root only — so an app on any other registered
location took the manager branch and `sed -i` failed with exactly the
permission error the comment above that code describes. `app move` writes
the new location with `|| true`, so it reported a successful move while
leaving the config naming the old disk.
* storageLocationName resolved a location's name only from an in-scope
CFG_STORAGE_LOC_<id>_NAME, falling back to the bare id. That name is the
value CFG_<APP>_STORAGE is set to, so the generated dropdown offered
"location-1" as both label and value — a choice that does not resolve. Read
it from the location's config when the variable is not in scope.
* storageSyncAllAppComments was written for "the regen path" and never wired
into one. Every CFG_<APP>_STORAGE option list was frozen at install time, so
adding a drive did not make it selectable anywhere. Called from the storage
generator now, which runs exactly when those lists go stale — and extended
to app TEMPLATES, since an app not installed yet is precisely the one whose
install form needs to show which drives exist.
Verified on a live install with three locations: linkding and authelia on disk1,
ipinfo on disk2, fourteen on the default root, each config naming its own drive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
initPickRoots can only offer MOUNTED filesystems, so on a box whose second drive
is new — unmounted, often unformatted — it returned in silence and the "where
should LibrePortal keep things?" prompt never appeared. The users most likely to
want a separate disk were the ones told nothing.
Add a notice listing what is attached and how to use it, deliberately only a
notice: mounting or formatting someone's disk is not something an installer
should do unasked, and `storage add` does it properly later with the
empty-directory admission rule and the fitness checks behind it.
Two things it has to get right, and both bit during development:
* "has no mount point" is not "is free" — the disk holding root has no mount
point of its own, its partition does, so the naive check offered the user
the disk they booted from. Walk the parent chain and mark holders in use.
* lsblk -r renders an empty mount point as a run of spaces, which `read`
collapses, shifting every later column left; an LVM member then parsed as a
mount point and was offered as free space. Use -P.
scripts/dev/lp-installer-disks-test runs the real function against a stubbed
lsblk that honours the output flag it is passed — without that the stub answers
-P to everything and cannot tell the two parsers apart, which is the bug being
guarded against. Verified both regressions fail it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The wizard's Storage step builds its first entry from primaryRoot() — the
app-data root — and labelled it "System disk". On a default install those are
the same drive and the name is honest. Installed with --containers-dir on its
own disk they are not, and the step then showed the DATA drive's size under the
system disk's name while the actual system disk never appeared in the list.
Seen on a matrix case-2 install (apps on a 29.4G test disk, system on a 912G
root): "System disk — 26.7G free of 29.4G".
The generator now reports whether that root is really on the OS disk
(is_os_disk, by st_dev against /), and the wizard labels it from that: "System
disk" when they coincide, otherwise the mount point. The "system" badge stays —
it marks the default location, which is still what it is.
Also add lp-shot --token / --cookie-js. A screenshot answers "does it render";
"does this wizard step work" needs clicking, which needs a real browser, which
needs the session lp-shot already knows how to mint from the stored jwtSecret.
This bug was found that way.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The three roots reach running code three ways — the CLI wrapper exports them,
the task-processor unit carries them as Environment=, and anything those start
inherits them. An @reboot crontab entry is started by none of the three: it runs
a script by absolute path, so paths.sh fell through to
: "${LP_CONTAINERS_DIR:=/libreportal-containers}"
That entry is crontab_boot_app_reconcile.sh, which brings every installed app up
at boot. On a relocated install it therefore reconciled against the DEFAULT
root, and that does not fail — docker creates the bind-mount directories it does
not find, so every app comes back empty while the real data sits untouched on
the other disk. Nothing logs an error; the only symptom is opening an app and
finding it blank (storage-locations §10.1 calls this the top data-integrity
risk).
Verified on a case-2 install (apps on /mnt/lptest2): a bare environment resolved
containers_dir to /libreportal-containers/ where the real root was
/mnt/lptest2/libreportal-containers.
Recover them from the systemd unit, which is the authoritative baked record —
init.sh already reads it back the same way, libreportal-relocate rewrites it,
and it is root-owned, so this is not the manager reading a config it can edit.
An explicit environment still wins; with no unit the defaults are unchanged.
scripts/dev/lp-paths-roots-test pins all four cases; verified it fails when the
recovery is removed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The three roots are independently relocatable, and the failures that matter are
the ones where only ONE of them moves: paths are baked into root-owned helpers,
the systemd unit and the CLI wrapper at install time, so anything that resolves
a root at runtime instead works on a default install and points at the wrong
disk on a relocated one. Testing "all default" or "all moved" misses that.
scripts/dev/lp-testdisk loopback ext4 disks — a real superblock, its own
st_dev and free space, thrown away between runs
scripts/dev/lp-install-matrix installs across the four root combinations and
checks each landed on the intended DEVICE, that
the helpers were baked (no __PLACEHOLDER__ left)
and that the WebUI answers
First thing the harness turned up: lp-shot hardcoded /libreportal-containers for
both the compose file it reads the published port from and the .auth.json it
signs a session with. On an install whose app data is on another disk it fell
back to a default port and a missing auth file — which looks exactly like a
WebUI that failed to boot. It now reads the baked LP_CONTAINERS_DIR back out of
the CLI wrapper.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
grafana restored and then died with "attempt to write a readonly database",
repeatedly. Its database is recorded in the snapshot as 231543:1002 and landed
as 1002:1002 — the owner was lost, so grafana, running as 231543, could not
write it at mode 0640.
Restore runs as the backup user with no CAP_CHOWN, so it reinstates ownership
inside a user namespace. The prefix was
unshare --map-root-user --map-users=SUB:SUB:N --map-groups=SUB:SUB:N
and unshare accepts ONE range per option, so the backup user's own GID was never
mapped — while app data is written as <container-uid>:<backup-user>. The group
half of every such chown referred to an unmapped id, lchown returned EINVAL, and
the file kept the restoring user's ownership. restic reports those as "ignoring
error ..." and still exits 0, so nothing failed: 1626 of one 13-app restore's
2086 failed chowns were grafana's, under a restore that reported success.
restic-userns-exec uses newuidmap/newgidmap, which write the multi-range maps
unshare cannot express:
uid: 0 <- caller inner root, or caps are dropped at exec
SUB.. <- SUB.. identity, so restic can name the stored uid
gid: caller <- caller identity: the group half of app-data chowns
SUB.. <- SUB.. identity
The caller's own UID is deliberately not identity-mapped — that slot is spent on
inner root — and a file stored as <caller>:<caller> lands owned by the caller
anyway, because that is who inner root is outside. So the one case this cannot
map is the one case needing no mapping. `unshare --map-auto --map-current-user`
is not a shortcut: it maps the subuid range to low inner ids while restic needs
identity. Tested.
Measured live, restoring grafana: failed chowns 1626 -> 12 (the 12 being the
caller's own files, correct), grafana.db back to 231543:1002, grafana up and
writing. Falls back to running the command plainly when there is no subuid
range, no newuidmap, or the namespace will not start.
scripts/dev/lp-userns-ownership-test pins all three ownership cases; verified
the old prefix fails it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
portFindNextAvailablePort consulted LibrePortal's own network_resources table
plus a hardcoded list (8080, 6060) and CFG_RESERVED_PORTS_EXTRA — while the
comment above it claimed a picked port "can never collide with a host service at
compose-up time". It can: the list only covers what someone thought to write
down. Ask the kernel instead, via ss, read once per allocation rather than per
candidate. No ss => empty set => exactly the old behaviour.
Found while restoring 13 apps onto a desktop, though not the cause there:
stoat's livekit publishes a FIXED udp range (50000-50100, which it advertises to
clients and so cannot be re-rolled), and kdeconnectd held 50016. That collision
needs its own answer; this fixes the randomly-allocated ports, which had the
same exposure with no reason to.
Also make the bulk restore stop reporting a half-running app as a clean success.
continue-on-error lets a failed compose-up log and carry on, so restoreAppStart
returns 0 either way — which is how that run printed "13 apps restored" while
four of stoat's containers had exited 101. checkSuccess already appends every
failure to error_report.log, so watch it grow across each app and name the ones
that were noisy.
scripts/dev/lp-port-host-test binds a real socket and asserts the allocator
refuses that port; verified it fails when the check is removed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
scripts/dev/lp-cli-argv-test builds stubs from the real invocation line in
init.sh and the real LP_CLI_ARGS line in start.sh, then pushes thirteen app
names through them — so editing either file is what makes it fail. Verified
against both regressions: dropping "$@" from the wrapper, and reading "$@"
instead of "${@:10}" in start.sh.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
storageSnapshotSourcePath resolved a snapshot's source path with
engineSnapshotsJson "$idx" "$snapshot_id"
but that function's second parameter is an app TAG filter. So it ran
`restic snapshots --tag app=<snapshot-id>`, matched nothing, and returned 1 —
every time, for every snapshot, since the file was written.
Nothing broke loudly, because both callers have a fallback:
* storageRestoreAppTo fell through to "restoring in place", reinstating the
exact cross-root bug the file exists to fix — restoring onto a host whose
containers root differs from the source's matched no include path and
restored nothing, silently
* the first-run preflight never read a manifest, so every app reported size
"?" and its fit and location checks passed unconditionally. Thirteen green
ticks that had checked nothing.
Add engineSnapshotPaths: restic answers it with a positional snapshot id, kopia
by filtering its list. borg has no adapter on purpose — it rebuilds its listing
from archive metadata that carries no paths — so a missing adapter is a quiet
"no" and those callers keep their in-place fallback.
Add scripts/dev/lp-preflight-test, which pins the cases that must say NO: an
app too big for the disk, one this version no longer ships, one whose storage
location is gone, and a resolver that reaches for the app-tag filter again.
Verified against both historical bugs — reintroducing either fails the test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Per-app placement worked but had no default: a box with a big second disk
meant setting CFG_<APP>_STORAGE on every app individually. CFG_STORAGE_DEFAULT
fixes that, and the wizard asks for it in one line.
CFG_<APP>_STORAGE now has three states rather than two, and the third is
the point:
<name> this app goes there, whatever the default says
primary this app goes on the install-time root, explicitly
default no opinion — follow CFG_STORAGE_DEFAULT
Templates ship "default", so the setting reaches every app without
touching 37 configs, while an app that was deliberately placed keeps its
placement. "primary" is new, and needed: without it there was no way to
say "keep this one on the system disk" once the global default moved.
A default naming a location that has since been removed falls back to the
primary root rather than refusing — a disk that got unregistered must not
make apps un-installable.
The wizard asks only once a second drive is ticked; with nothing ticked
there is one possible answer and a control would be furniture. It sets a
default, not a placement, and the value stored is the location NAME, so it
survives the disk being remounted elsewhere.
scripts/dev/lp-storage-default-test covers all three states plus the
removed-location fallback.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Follow-up to 928e244, which stopped configs/ subdirectories being sourced
without a .category marker. That closed the hole; this removes the thing
that fell into it.
storageIndexFile pointed at configs/storage/app_locations. The file's
requirements are only "manager-owned" and "not on a removable disk" —
configs/ satisfies both, which is why I put it there, and it was still
wrong: that tree carries a third property the file violates. sourceScanFiles
SOURCES what it finds under configs/, and sourcing means executing.
The index is a TSV of "<slug><TAB><root>", which bash reads as a command
and its argument. Harmless while no slug matched a real executable. The
row for the app named `libreportal` armed it, because that IS the CLI on
PATH: sourcing ran `libreportal /libreportal-containers`, which re-entered
the scan, which sourced the file again — one process pair per level until
the host OOMed and took the desktop session with it.
It now lives at $system_dir/storage/app_locations, with a one-shot
migration so an install that already has an index keeps knowing where its
apps live rather than silently forgetting. libreportal-ownership
reconciles the new directory, and scan_files.sh gained a note that
configs/storage/ carries no .category on purpose.
scripts/dev/lp-configs-guard-test covers both ends: the index never lands
in configs/, a legacy one migrates, and a file of the exact detonating
shape placed in an unmarked configs/ subdirectory is not executed while a
marked category still loads.
Also wires sourceStorageLocations into the config scan beside
sourceBackupLocations — per-location configs sit at depth 3, below the
generic scan, and need their own walker like the backup ones do.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Phase 0 of docs/roadmap/storage-locations.md — the resolver layer. No
behaviour change yet: with no registry present, every function here
returns exactly what the old single-root code did, which is what makes
the ~200-site sweep that follows safe to land incrementally.
primaryRoot / webuiDir the install-time root, and the one tree that
never moves
storageRoots every registered root, primary first
storageRootAvailable marker present == drive mounted
pathIsContainerData replaces the `== "$containers_dir"*` idiom
that picks manager vs container-user elevation
appDir / appDirSlash THE resolver, memoised
storageLocationPath/Name name <-> path, via the root-owned registry
storageIndexGet/Set app -> location cache
appDir resolves discovery-first: whichever root actually holds
<slug>/<slug>.config wins, so a hand-move or half-finished migration
self-heals rather than corrupting.
The index exists because of a bug the unit test caught immediately.
Discovery cannot see an unmounted disk, so an installed app on an
unplugged drive looked identical to a brand-new app — and the fallback
handed back the PRIMARY root. Docker would then have created the bind
mounts there and booted the app empty on the wrong disk, which is the
precise failure the availability design exists to prevent. The index is
manager-owned (deliberately not on the removable disk: it must be
readable exactly when that disk is absent), consulted only when the scan
comes up empty, and rewritten by every successful scan so the disk stays
authoritative whenever it is actually present.
Availability is gated in appDir alone rather than at each caller: every
site reaches it by construction. It returns non-zero AND prints an
unusable sentinel path, so the many callers that will never check $?
still fail loudly on something harmless.
scripts/dev/lp-storage-test covers all of it against a throwaway tree.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three visible faults on the Tasks page, one shared root.
Upgrade tasks rendered as their raw command — "libreportal updater
upgrade rocketchat 8.7.1" beside properly named neighbours. The title
table had rows for updater check/apply/apply-all/rollback and none for
upgrade, because the upgrade command is assembled in task-actions.js
rather than task-commands.js — and lp-task-names, the guard built to
catch exactly this, only read task-commands.js. It certified 16 commands
and reported that as the whole surface; the surface was 29. The guard
now reads both dispatch sites (JS ${expr} interpolations become sample
placeholders; commented-out prose mentioning commands in backticks is
skipped, or it reports fictional commands), and all 29 pass.
A removed instance's tasks outlive it, and its slug rendered as a tech
identifier: "Bookstack_uitest - Remove Instance". getAppDisplayName
cannot help — it capitalises as its own fallback, so unknown is
indistinguishable from known-and-plain. The formatter now does the same
membership test the helper uses internally: slug absent from
window.apps, prefix before the underscore present -> render the way
live instances are shown, "Bookstack · uitest".
Same story for the icon: bookstack_uitest.svg is deleted with the
instance, and onerror="display:none" left a bare gap in the row. Now a
fallback chain — the TYPE's icon (which survives), then the LibrePortal
logo. Verified live: the dead instance's rows show bookstack.svg with
the fallback marker set, everything else keeps its own icon.
Verified in a real browser session — full render, zero console errors.
The 'add' verb also joins the app-action map so "Add Application" is
deliberate wording rather than the blind "<Verb> Application" compose.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Task titles come from one table whose final fallback returns the raw command
string, so a dispatched command with no matching row does not error — it just
renders as "libreportal instance remove bookstack_work" beside properly named
neighbours. That silence is why this kept being fixed and kept coming back.
The guard reads BOTH files as source — the command templates from
task-commands.js and the pattern table from tasks-format.js — so it fails on a
command added without a name rather than leaving it to be noticed in the UI.
Two checks, both from source rather than guessed from rendered text:
1. Nothing falls through: a title equal to its command, or still starting with
"libreportal ", means the raw fallback was reached.
2. Every `libreportal app <verb>` verb has an actionMap entry. Without one the
generic branch composes "<Verb> Application", which is how "Up Application"
and "Down Application" shipped.
The second check reads the actionMap keys instead of pattern-matching the title,
which a first attempt did and which was wrong: "Reload Application" is both a
correct hand-written label and what the generic branch emits, so the rendered
text cannot distinguish them and the heuristic failed a title that was fine.
Verified by breaking it deliberately in both directions — adding a command with
no pattern, and deleting an actionMap verb. Each is caught, named, and pointed at
the file to edit; both files were restored byte-identical afterwards.
Lives in scripts/dev, which .gitattributes marks export-ignore, so it never ships
in a release tarball. Needs a node and borrows the running container's when the
host has none, the same constraint lp-shot works around for chromium.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The WebUI screenshot helper CLAUDE.md already tells agents to use only
ever existed on the maintainer's box. Vendoring it means it survives a
machine rebuild and the setup steps are written down.
It does NOT ship: make_release.sh builds with `git archive`, which honours
export-ignore, so scripts/dev joins scripts/release and docs on that list.
Verified — the staged tarball has 1666 files and none under scripts/dev.
Keeping it out of releases is deliberate, not incidental. lp-shot signs
itself a session from the jwtSecret in frontend/.auth.json, which is fine
on a box where you already own that file, and has no business sitting in
a user's install where it would read as a backdoor.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
librelad