Move the WebUI-updater settings out of general_terminal into their own
advanced webui-category file (webui_logs precedent): new
configs/webui/webui_updater holds CFG_UPDATER_SCAN_INTERVAL and the
migrated CFG_HOTFIX_AUTO, listed in webui/.category.
The move only reaches existing installs if the config convergence
machinery works, and three pieces of it silently didn't:
- checkConfigFilesMissingFiles walked a stale hardcoded category list
('general features network' — features doesn't exist; webui/backup/
security never healed). Derive the categories from the template tree
instead, and heal .category metadata too: copy it when absent and
merge missing SUBCATEGORY_ORDER entries when present, so healed files
actually appear in the WebUI Config editor. core_categories removed.
- Option reconciliation never touched ANY nested config file: configs_dir
carries a trailing slash, so rel stripping missed ('configs//'), the
template lookup failed, and reconcileConfigFile early-returned for
every file. Strip the slash before matching.
- reconcileConfigFile's AUTO_DELETE=false branch read a never-populated
live_line array, losing the dropped keys it promised to keep. Populate
it alongside live_value.
Also exclude *.bak from config sourcing (reconciliation writes <file>.bak
next to live configs — now that it runs, sourcing backups would resurrect
deleted keys), and add 'libreportal config check' as a non-interactive
front door to the converge pass (was only reachable via install flows and
the interactive menu).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
initilize_files.sh's load-comment said "Phase 6 will tackle this with a
precompiled cache file" — but Phase 6 deliberately shipped a different
(simpler) win, and Phase 7 was reconsidered and rejected (the ~30 ms
saving doesn't justify the invalidation complexity across every CFG
write site). Rewrite the comment to describe current behaviour: the two
config scans run because bash can't lazy-load variables, and the
precompile was looked at and dropped.
app_generate.sh had a `read -p "" host_name` inside a `while true` with
no break — anyone who actually ran `libreportal app generate` would
have hung at the hostname prompt forever. The value was then fed into
a `sed 's/HOST_NAME=test/HOST_NAME='"$host_name"'/g'` against the new
app's .config file, but the post-2e4f420 template no longer carries
HOST_NAME=test (per-port subdomains in the PORT row drive routing now).
So the prompt was infinite-looping to gather a value that fed a no-op
sed. Drop both — the function loses no real capability since the
subdomain field is set per-port via the standard PORT row.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
The early loop in sourceInitilize() sources every .sh under source/files/
recursively — including the new arrays/function_manifest.sh, which now
carries ~860 autoload stub definitions (~50 ms parse cost). Even in
eager mode where lazy infrastructure is never touched, every invocation
was paying that cost up front.
The manifest is only needed in lazy mode, where it's sourced explicitly
at the top of the lazy branch. Excluding it from the early loop:
- Eager mode: drops the ~50 ms regression introduced by Phase 5.
- Lazy mode: unchanged — the explicit source still runs.
This brings eager back to the pre-Phase-5 baseline and lets the lazy
container-stub gain (skipping sourceScanFiles containers, ~70 ms) show
through as a real saving.
Signed-off-by: librelad <librelad@digitalangels.vip>
Containers used to be eager-loaded via `sourceScanFiles "containers"`
even under LP_LAZY=1 — sourcing all ~160 installer functions up front.
Phase 5 brings them into the autoload-stub mechanism.
generate_function_manifest.sh now scans BOTH scripts/ AND containers/
(maxdepth 3, matching sourceScanFiles' existing prune), with a per-entry
root selector so stub emission uses the right base directory:
scripts/peer/peer_add.sh → source "${install_scripts_dir}peer/peer_add.sh"
containers/linkding/linkding.sh → source "${install_containers_dir}linkding/linkding.sh"
New manifest exports:
LP_FN_MAP funcname → relpath (existing)
LP_FN_ROOT funcname → scripts|containers NEW
LP_EAGER_FILES "<root>:<relpath>" entries NEW format
~860 autoload stubs (was ~700; +160 from containers)
Loader changes (initilize_files.sh):
- Parses LP_EAGER_FILES entries as `root:path`, dispatches to the
right install_*_dir. Pre-Phase-5 entries without a colon default to
scripts (backwards-compatible).
- sourceScanFiles "containers" is skipped when LP_LAZY=1 AND
LP_FN_MAP is loaded (manifest-driven autoload covers it).
Eager mode and lazy-with-missing-manifest both still run the scan.
Measurement target: ~70 ms saved on top of Phase 4. Verified separately
in the commit message of the next deploy.
Signed-off-by: librelad <librelad@digitalangels.vip>
scripts/source/loading/initilize_files.sh gains an LP_LAZY=1 branch:
- Sources scripts/source/files/arrays/function_manifest.sh once. The
manifest defines LP_FN_MAP, LP_EAGER_FILES, AND ~700 autoload stubs
(precompiled by the generator — one parse cost vs evaluating 700
snippets at startup).
- Eager-sources every file listed in LP_EAGER_FILES (top-level side
effects: variable assignments, source calls, bare commands). These
can't safely be deferred — they'd skip the side effect, not just the
function definition.
- Skips the bulk loop that sources every files_to_source[@] entry.
Default behaviour (LP_LAZY unset or 0) is byte-identical to the previous
loader — every file gets eager-sourced up front. Long-running processes
(WebUI service, task processor) leave LP_LAZY unset because their first
call to anything wants the function already hot.
Each autoload stub looks like:
funcname() {
source "${install_scripts_dir}path/to/file.sh"
funcname "$@"
}
First call sources the real file, which redefines the function with the
real body; the stub's trailing `funcname "$@"` then calls the freshly-
defined real implementation. Sourcing the file also redefines stubs for
any sibling functions the same file declares, so they don't re-source.
Safety nets:
- Missing manifest → fall back to eager loading (`export LP_LAZY=0`).
No regression risk if someone enables LP_LAZY=1 on a stale install
whose regen never ran.
- LP_LOAD_TRACE=1 still works in lazy mode — it records the manifest
parse + each eager file (tagged LAZY-manifest / LAZY-EAGER) so Phase 4
can measure the actual saving.
No automatic flip yet — this commit only adds the path. Phase 4 will set
LP_LAZY=1 by default for the CLI entrypoint (and re-measure with the
trace tool from Phase 1).
Signed-off-by: librelad <librelad@digitalangels.vip>
First step toward an autoload-style lazy loader for the 499-file source
tree (current cold load ~1s wall / 340ms user-time per CLI invocation,
mostly spent sourcing files the command never calls). This commit only
measures — no behaviour change unless LP_LOAD_TRACE=1.
LP_LOAD_TRACE=1 instrumentation (scripts/source/loading/initilize_files.sh):
Wraps each in the main file-list loop with EPOCHREALTIME
before/after, writes `<elapsed_ms>\t<file_relpath>` to
$LP_LOAD_TRACE_FILE (default /tmp/libreportal-load-trace.<pid>.log).
Zero overhead when the env var is unset (one [[ test per file).
libreportal debug load-trace [cmd...]:
New `debug` CLI category. Spawns a child `libreportal <args>` (default
'help') with LP_LOAD_TRACE=1, then awk-aggregates the trace: wall vs
cumulative source time, file count, top-15 hottest files. The diff
between wall and cumulative-source = bash startup + dispatch + the
command's own work.
Used in the next phases to (a) validate that the lazy loader actually
delivers the speedup we expect and (b) flag any single file that hogs
disproportionate time (rare `heredoc | sed | base64` style work at
source time would show up here as a >10ms entry).
Signed-off-by: librelad <librelad@digitalangels.vip>
Triage of a broken fresh install:
1. init.sh → all root setup → completeInitMessage hands off to
`libreportal run install` as the manager.
2. start.sh sources load_sources.sh, which calls sourceCheckFiles "run".
3. sourceCheckFiles "run" calls checkUpdates — its only path to startLoad on
a non-local mode is via the git/release recovery branches.
4. git fails (the deployed install dir has no .git), lpFetchRelease fails (no
reachable release manifest), none of the recovery branches converge on
startLoad, and the install silently exits with WebUI + service unset.
Fix: completeInitMessage exports LIBREPORTAL_INITIAL_INSTALL=1, and the
sourceCheckFiles "run" branch calls startLoad directly when that's set — same
endpoint the local-mode branch hits. We just installed the latest code from
this tree; checking for updates on the first run was nonsensical and the
recovery gauntlet would only break things.
Confirmed by re-running uninstall + install: the install now reaches the
Pre-Installation / database / WebUI build / crontab / WebUI compose-up steps
and produces a working WebUI. (A separate compose-tag bug surfaced next —
fixed in the follow-up commit.)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Reverts the 2>/dev/null band-aids and fixes the root cause. The
manager-run install boot scans app configs under /docker/containers AS
the container user (runFileOp). But init.sh's initFolders creates that
dir manager-owned, and the handover to the container user happened later
(start_preinstall), AFTER the boot scans — so the scans ran as the
container user against a dir it didn't own yet: "find:
'/docker/containers/': Permission denied" (cosmetic; the dir is empty
that early, but it's the wrong ownership at the wrong time).
Add initContainerLayer() to init.sh's root phase (after initGIT +
initUpdateConfigs, before the manager-run handoff): rootless-only, it
creates the docker-install user if missing and chowns /docker/containers
to it (751). The later rootless setup is now idempotent — it finds the
user existing and just (re)asserts its password + daemon config (moved
updateDockerInstallPassword out of the create-only branch). Rooted is
unaffected (containers stay manager-owned, which the manager reads).
Result: by the time the boot scans run, /docker/containers is owned by
the user doing the scanning — no permission error, nothing suppressed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Early in an install the docker-type config isn't loaded yet, so runFileOp
falls back to the manager, which can't list the container-owned (751)
/docker/containers/ dir. Two best-effort scans then leaked
"find: '/docker/containers/': Permission denied" to the install output
(x3 per run): scan_files.sh's app_configs scan and the application config
reconcile. No app configs exist that early on a fresh install, so the
empty result is correct — just suppress the find stderr (the -print0
output still flows). Cosmetic only; doesn't change what's enumerated once
the config is loaded.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
scan_files used 'sudo find' to enumerate config files to source. Under the
scoped sudoers that's denied, so NO configs got sourced -> CFG_DOCKER_INSTALL_TYPE
ended up empty -> runFileOp/runFileWrite fell back to the manager branch and
every container-path write failed. Root cause of the 'sudo: a password is
required' + 'tee: Permission denied' storm when running under the scoped grant.
- configs/ scan (manager-owned): plain find
- app_configs scan (/docker/containers, docker-install-owned, not list-readable
by the manager): runFileOp find (enumerate as that user; manager still sources
each .config, which is o+r). 'containers' install templates stay plain find.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
Propagate the ✓ Success / ✗ Error / ! Notice / ❯ Question glyphs (from markers.sh) through the rest of the pipeline: swap the inlined helpers in init.sh and generate_arrays.sh, and replace raw echo -e "${RED}ERROR:${NC}" calls with the isX helpers in config_check_missing.sh, check_success.sh, initilize_files.sh, and reset_git.sh.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
A free, open, self-hosted app platform (GNU AGPLv3): one-click app deploys,
Traefik reverse proxy with automatic SSL, rootless Docker support, gluetun
VPN routing, and a web dashboard to manage it all.
Free & open forever to self-host; optional paid hosted services fund it.
See PROMISE.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
