The resolver already supported per-app placement — CFG_<APP>_STORAGE names a
location and appDir sends data, compose and config there — and 37 of 39 app
templates ship the field. What was missing was choosing AT INSTALL TIME. The
only routes were editing a config by hand before installing, or installing onto
the default disk and then `app move`ing it, which copies the data twice.
libreportal app install <app> --storage=<location>
and the App Center's existing storage dropdown, which travels inside
config_variables. Both resolve to one answer in storageChoiceFor, so there is a
single code path.
Ordering is the whole difficulty, and getting it wrong is quiet. installApp
copies the app template into appDir(), sources it, and later applies the form
overrides. The choice has to be live before the copy (or the directory is
created on the wrong disk), written into the config before the source (or the
template's "default" wins and every later appDir in that process returns the
primary root), and folded into config_variables (or the override pass writes
"default" back). Miss any one and the directory and its config disagree — which
resolves correctly only until something sources the config.
Refuses an unknown or unmounted location, an existing directory, and an app
whose template marks the field **READONLY** (fixed to the primary root because
other apps reach it by literal path — storageMoveApp already refuses to move
those, and installing one elsewhere is the same violation from the other end).
Three shipped bugs found making this work:
* updateConfigOption chose its write helper by comparing the path against
$containers_dir — the PRIMARY root only — so an app on any other registered
location took the manager branch and `sed -i` failed with exactly the
permission error the comment above that code describes. `app move` writes
the new location with `|| true`, so it reported a successful move while
leaving the config naming the old disk.
* storageLocationName resolved a location's name only from an in-scope
CFG_STORAGE_LOC_<id>_NAME, falling back to the bare id. That name is the
value CFG_<APP>_STORAGE is set to, so the generated dropdown offered
"location-1" as both label and value — a choice that does not resolve. Read
it from the location's config when the variable is not in scope.
* storageSyncAllAppComments was written for "the regen path" and never wired
into one. Every CFG_<APP>_STORAGE option list was frozen at install time, so
adding a drive did not make it selectable anywhere. Called from the storage
generator now, which runs exactly when those lists go stale — and extended
to app TEMPLATES, since an app not installed yet is precisely the one whose
install form needs to show which drives exist.
Verified on a live install with three locations: linkding and authelia on disk1,
ipinfo on disk2, fourteen on the default root, each config naming its own drive.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The main sweep — ~260 call sites across ~100 files move from string
concatenation on a single root to appDir/storageAppDirs/storageAppConfigs.
On a single-root install the resolved paths are identical, so this is a
no-op until a location is registered.
Enumerators were the interesting half. `for d in "$containers_dir"/*/`
appears in the menus, the registry/artifact scanners and the DNS setup —
and a shell glob cannot list a rootless 751 tree at all, which is the
same bug config_find_file.sh already documents in a comment. Routing them
through storageAppDirs (which enumerates as the owning user) fixes that
alongside the multi-root work.
Three places needed judgement rather than substitution:
db_app_scan.sh deletes database rows and port allocations for apps whose
folder is missing, and reaps "empty" app dirs. With a storage location
unmounted, every app on it looks exactly like that. Each of those
branches now gates on appStorageAvailable first — an app on an unplugged
drive is skipped with a notice, never deleted.
instance_create.sh rewrites cloned hooks so an instance touches its own
directory instead of the base app's. Its sed matched ${containers_dir}<type>,
which this sweep just replaced with $(appDir <type>) — so it would have
silently stopped redirecting, and an instance would have written to the
original's files (the adguard auth adapter case its own comment warns
about). Now matches both appDir forms, verified against bare, quoted,
unrelated-app, legacy and prose cases.
peer_shell/peer_pull streamed and extracted relative to the primary root.
Both now use the app's own root, and peer_shell keeps a single-root
fallback since it runs as a restricted SSH shell with no LibrePortal env.
Also fixes a pre-existing bug found on the way: webui_app_config.sh
tested "$containers_dir/frontend/data/last_update", one level short of the
real tree under the libreportal app dir, so the WebUI refresh trigger
after a config update has never once fired.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two mechanical sweeps, no behaviour change on a single-root install.
The 14 `[[ "$p" == "$containers_dir"* ]]` prefix tests that decide
manager-vs-container-user elevation become pathIsContainerData, so a file
on a second storage root is no longer misclassified as manager-owned —
which would have written it with the wrong owner and failed later, far
from the cause. The 65 references to the WebUI's own tree become
webuiDir(), which is pinned to the primary root by design.
Two traps found while doing it:
run_privileged.sh is sourced directly by init.sh without paths.sh, so it
needs a fallback. Defining one named pathIsContainerData was wrong:
generate_function_manifest.sh indexes top-level definitions, and the
resulting autoload stub would have shadowed the real multi-root
implementation with the primary-only fallback — silently classifying
every file on a second disk as manager-owned, which is exactly the bug
this sweep exists to prevent. Renamed to _runCfgIsContainerPath, which
delegates when the real one is loaded.
setup_lock.sh built its path in a top-level assignment, so it was
evaluated at source time and needed the file flagged eager. Made it a
function instead: the path resolves on call, and the file drops off
LP_EAGER_FILES entirely.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The port editor used to serialise ten of the twelve columns, so saving
any port on an app silently discarded that app's Traefik subdomain and
the router fell back to the app-name default. Navidrome lost "music" and
Speedtest lost "speedtest" exactly that way, and nothing reported it —
the app kept working, on the wrong hostname.
The writer is fixed, but an install already carrying the damage would
keep it forever: reconcile preserves the user's value, and a truncated
row IS the user's value as far as it can tell. It now tops such a row up
from the template, appending ONLY the columns the live row does not
reach. Everything the live row states wins — including a deliberately
blanked column — so clearing a subdomain is not undone, and a live row
longer than its template is left alone.
Two faults of my own, caught while testing it end to end:
The notice was printed on stdout. This function returns its result
through a command substitution, so the notice text was captured INTO the
config value and written to navidrome's descriptor. It goes to stderr.
The width in the message was measured after the merge, so it reported
the post-merge count as the "before". Captured up front instead.
Verified against a live install: truncating navidrome's row to 9 columns
and running `config check` restores it to 11 with "music" intact, a
second run changes nothing, and no port on the box is left Traefik-
managed without a subdomain. Unit-checked that it declines to act on a
complete row, a non-port key, a row longer than its template, and never
overwrites a live value; quoting style is preserved either way.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The compose rewrite assumed each app had exactly <type>-service and <type>_db.
That holds for Bookstack and almost nothing else: cloning Nextcloud left -db,
-redis and -web pointing at the ORIGINAL app's containers, and Matrix, Ollama,
Mastodon, Owncloud, Gitea, Jitsi, Invidious, Rocketchat and Mattermost all had
the same hole. Docker refuses a duplicate container name and two Traefik
routers sharing a name fight over the host, so those clones could not have
worked.
Service identities are now discovered from the compose itself — its
SERVICE_TAG_<n> markers plus its container_name values — and each is renamed.
Verified across all 38 shipped apps: 15 are fixed, 21 produce byte-identical
output to the old rule (Bookstack among them, so the running instances are
unaffected), and 2 are refused.
Details worth knowing:
- Separators compare as equivalent, so the app dir libreportal_catalog matches
its libreportal-catalog-* services instead of being wrongly refused.
- Tokens are substituted longest-first through placeholders. \b has to end a
token because per-port routers are named <service>-<portname>
(traefik.http.routers.adguard-service-webui), which also means a short name
could otherwise match inside a longer one — ordering is what prevents that.
- Commented-out lines are not harvested. Several templates park an optional
sidecar behind # (adguard-exporter, pihole-exporter, wireguard-exporter);
renaming those also mangled the image name in the same block, leaving a trap
for anyone uncommenting it. Commented image: lines are skipped too.
- image: lines are genuinely excluded now. The old comment claimed service
tokens "never appear in an image path", but libreportal builds a local image
named after its own service and the old rule rewrote that reference.
An app with a service carrying no <type> prefix (stoat's api/database/minio,
prometheus's node-exporter/cadvisor) cannot be made unique mechanically, and
rewriting a bare word like minio would corrupt image: minio/minio. Those are
refused with an explanation and the partial clone is removed, rather than
handed back as an instance that silently fights the base app.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
crowdsec_install_host.sh wrote CFG_CROWDSEC_TRAEFIK_LAPI_KEY into
${configs_dir}security/security_crowdsec, but no such template ships in
configs/security/ (only security_logins and security_ssh), so
checkConfigFilesMissingFiles never created it, the -f guard always failed, and
the key was never mirrored — every install logged "Live config not present yet"
and the setting stayed empty. The key is declared in crowdsec.config, so point
the write there.
Switched the hand-rolled sed for updateConfigOption, which escapes the value,
routes the write through the user owning the containers tree, and re-sources so
the key is live in the same run. The old sed used | as its delimiter and would
have corrupted the file on a key containing one; verified the new path
round-trips a key with + / and | intact.
updateConfigOption logged "Updated <key> to <value>", and checkSuccess both
prints its message and appends it to the docker log — so mirroring the bouncer
key would have written it to disk in plaintext, as every admin password the auth
adapters persist already was. Credential-looking keys now log the name only;
everything else still logs its value, which is what makes that log useful.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The portal was found crash-looping with EACCES on /app/webui_logins.
configs/webui/* are bind-mounted into the container, which reads them
through its GROUP (files 0640, group = container owner). Two paths reset
that group to the manager and never gave it back:
* init.sh setupConfigsFromRepo — `chown -R manager:manager` over the
whole configs tree on every install/redeploy (the documented local-
mode deploy), and
* the runtime config reconcile — rewriting a live config replaces the
file as the manager, so ANY release that merely adds a key to a
webui_* config would break the portal.
Neither breaks anything immediately: the running container holds its
open files, so the failure only appears at the next restart, long after
the change that caused it. That is exactly how it surfaced here — a
deploy in the evening, a dead WebUI later.
init.sh gains restoreWebuiBindAccess (prefers the root ownership helper,
inline chown fallback for the first install, no-op when the container
user does not exist yet) called right after the chown; the reconcile
calls the existing reconcileWebuiDirOwnership when it is in scope.
Verified by reproducing the break (chown -R manager over configs), then
running the fixed deploy and force-recreating the container: group is
restored to the container owner and the portal serves on 3179.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The config reconcile pass printed one 'Reconciled config: <name> (backup:
.<name>.bak)' line per changed file. Drop the per-file message entirely:
the intro notice and the two per-section '...completed.' confirmations are
enough, and the backup mention added noise. The hidden .<file>.bak sibling
is still written for safety — it's just no longer announced.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Reconcile backups now land as .<file>.bak instead of <file>.bak, so they
no longer clutter the configs folder. The .bak suffix is kept, so every
existing walker/sourcing exclusion still applies.
Also exclude dotfiles and *.bak from findConfigFileForOption: it walked
the configs tree with no backup exclusion, so depending on directory
order a 'config update' could resolve a key to the backup file and write
the user's change there — silently lost.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Move the WebUI-updater settings out of general_terminal into their own
advanced webui-category file (webui_logs precedent): new
configs/webui/webui_updater holds CFG_UPDATER_SCAN_INTERVAL and the
migrated CFG_HOTFIX_AUTO, listed in webui/.category.
The move only reaches existing installs if the config convergence
machinery works, and three pieces of it silently didn't:
- checkConfigFilesMissingFiles walked a stale hardcoded category list
('general features network' — features doesn't exist; webui/backup/
security never healed). Derive the categories from the template tree
instead, and heal .category metadata too: copy it when absent and
merge missing SUBCATEGORY_ORDER entries when present, so healed files
actually appear in the WebUI Config editor. core_categories removed.
- Option reconciliation never touched ANY nested config file: configs_dir
carries a trailing slash, so rel stripping missed ('configs//'), the
template lookup failed, and reconcileConfigFile early-returned for
every file. Strip the slash before matching.
- reconcileConfigFile's AUTO_DELETE=false branch read a never-populated
live_line array, losing the dropped keys it promised to keep. Populate
it alongside live_value.
Also exclude *.bak from config sourcing (reconciliation writes <file>.bak
next to live configs — now that it runs, sourcing backups would resurrect
deleted keys), and add 'libreportal config check' as a non-interactive
front door to the converge pass (was only reachable via install flows and
the interactive menu).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Symptom (reported on adguard install):
sed: couldn't open temporary file /libreportal-containers/adguard/sedZaGX2H:
Permission denied
✗ Error Updated CFG_ADGUARD_BACKUP to true
! Notice Non-interactive mode: aborting on error.
Root cause: updateConfigOption ran a raw `sed -i` regardless of which tree
the config file sits in. Works fine for /libreportal-system/configs/*
(manager-owned) but breaks on /libreportal-containers/<app>/<app>.config
(dockerinstall-owned in rootless mode) — sed -i writes its temp file next
to the target, inheriting the directory's perms, and the manager can't
write inside dockerinstall dirs. EVERY app installer that mutates a
CFG_<APP>_* value (the autogenerated random password, BACKUP toggle,
PORT override, etc.) goes through this function, so this was a latent
ticking bomb across all containers.
Fix: pick the helper based on path —
- under $containers_dir → runFileOp (escalates to
dockerinstall in rootless, runs as manager in rooted)
- otherwise → runInstallOp (always manager)
Read paths (grep / source) stay unwrapped — both dirs are world-readable;
only the write needs the privilege swap.
Net: no more 'Permission denied' on app installs; the de-sudo pattern is
now respected end-to-end for CFG writes.
Signed-off-by: librelad <librelad@digitalangels.vip>
Existing installs were locked out of template-side description /
options / marker changes because reconciliation kept the user's whole
`CFG_KEY=value # comment` line verbatim. So new metadata like the
**DEV** marker I'm adding for the developer-mode feature wouldn't take
effect on already-deployed boxes — only fresh installs.
Updated reconcileConfigFile to split each line into value-part and
comment-part. User value is still sacred; the comment (title,
description, [options], **ADVANCED**/**DEV** markers) now comes from
the template. Field renames, label tweaks, marker additions/removals
shipped in a release reach existing installs on the next CLI
invocation (which runs the reconciler).
Specifically unblocks: the developer-mode WebUI feature (CFG_DEV_MODE
field gets added by the existing add-only path; CFG_INSTALL_MODE and
CFG_RELEASE_CHANNEL now pick up their new **DEV** markers and the
'Release - Stable' / 'Bleeding Edge' labels).
Signed-off-by: librelad <librelad@digitalangels.vip>
configFileSetupData runs as the manager (libreportal user) during install,
but writes into /libreportal-containers/<app>/, which is owned by the
container user (dockerinstall) under the three-root layout. The six bare
`sed -i` calls in this function were missing the `runFileOp` wrapper that
every other in-tree sed-on-app-files call already uses (e.g. setup_dns.sh's
WG_DEFAULT_DNS edits), so on first run `sed -i` failed to create its temp
file in the live dir:
sed: couldn't open temporary file /libreportal-containers/linkding/sedaCaUNU: Permission denied
✗ Error Updated DOMAINSUBNAMEHERE with: bookmark.
! Notice Non-interactive mode: aborting on error.
…which aborted the install at step 3 of every per-app config setup.
Replace `result=$(sed -i ...)` with `result=$(runFileOp sed -i ...)` so each
substitution runs as the owner of the target file (via the bin-install
helper). All six call sites use the same pattern — done as a single
`replace_all` over the unique prefix.
Tags fixed: DOMAINSUBNAMEHERE, APPADDRESSHERE, DOMAINSUBNAME_DATA,
TIMEZONE_DATA, EMAILHERE, HOSTIPHERE.
Verified live on a fresh install: `libreportal app install linkding` now
completes cleanly through all 10 install steps and lands the container.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Application backups were driven by one crontab entry per app, each offset by
id * CFG_BACKUP_CRONTAB_APP_INTERVAL minutes. That minute offset is written
straight into cron's 0-59 minute field, so past ~20 apps it overflowed into
an invalid entry that silently never fired, and the fixed spacing could not
serialize backups that ran longer than the gap.
Replace it with a single daily entry (`libreportal backup scheduled`) that
enqueues a backup task per enabled app. The existing systemd task processor
drains them serially — no minute overflow, real serialization, and backups
are now visible/cancellable in the Tasks UI. Per-app enable is read from
CFG_<APP>_BACKUP at schedule time instead of being mirrored into crontab.
Removes the stagger machinery (timing/setup/check/remove scripts), the
now-unused cron_jobs table + insert, and the CFG_BACKUP_CRONTAB_APP_INTERVAL
config knob and its WebUI field.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Propagate the ✓ Success / ✗ Error / ! Notice / ❯ Question glyphs (from markers.sh) through the rest of the pipeline: swap the inlined helpers in init.sh and generate_arrays.sh, and replace raw echo -e "${RED}ERROR:${NC}" calls with the isX helpers in config_check_missing.sh, check_success.sh, initilize_files.sh, and reset_git.sh.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Replaces the slow, interactive per-variable scan with a deterministic
reconcile: each live config is rebuilt from its (freshly-cloned) template —
keeping the user's existing values, adding new template keys
(CFG_REQUIREMENT_CONFIGS_AUTO_UPDATE), and dropping keys the template no
longer defines (new CFG_REQUIREMENT_CONFIGS_AUTO_DELETE, default true).
Structure/order/comments follow the template; non-interactive; atomic with a
.bak; refuses to act on a missing/empty template so a broken clone can't wipe
a config. Applies to both general and per-app configs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
A free, open, self-hosted app platform (GNU AGPLv3): one-click app deploys,
Traefik reverse proxy with automatic SSL, rootless Docker support, gluetun
VPN routing, and a web dashboard to manage it all.
Free & open forever to self-host; optional paid hosted services fund it.
See PROMISE.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
