The previous run had 32 eager files; 24 of them were the auto-generated
files_*.sh arrays (only useful to the eager loader) + the manifest
itself (which the lazy loader sources explicitly). Eager-sourcing them
under lazy mode was pure overhead — ~55ms on the manifest alone (it was
being parsed twice, once via the explicit lazy-loader source and once
via the LP_EAGER_FILES loop).
Down to 8 eager files (the genuinely-side-effecting ones: setup_lock.sh,
the two crontab task processors, backup_db.sh, backup_files.sh,
docker swap_docker_type.sh, migrate_url_rewrite.sh, cli_debug_commands.sh).
The files_*.sh arrays are still sourced by the eager loader's existing
path — that's unchanged. Lazy mode just doesn't need them because it
never iterates files_libreportal_app[@] / files_libreportal_cli[@].
Signed-off-by: librelad <librelad@digitalangels.vip>
scripts/source/loading/initilize_files.sh gains an LP_LAZY=1 branch:
- Sources scripts/source/files/arrays/function_manifest.sh once. The
manifest defines LP_FN_MAP, LP_EAGER_FILES, AND ~700 autoload stubs
(precompiled by the generator — one parse cost vs evaluating 700
snippets at startup).
- Eager-sources every file listed in LP_EAGER_FILES (top-level side
effects: variable assignments, source calls, bare commands). These
can't safely be deferred — they'd skip the side effect, not just the
function definition.
- Skips the bulk loop that sources every files_to_source[@] entry.
Default behaviour (LP_LAZY unset or 0) is byte-identical to the previous
loader — every file gets eager-sourced up front. Long-running processes
(WebUI service, task processor) leave LP_LAZY unset because their first
call to anything wants the function already hot.
Each autoload stub looks like:
funcname() {
source "${install_scripts_dir}path/to/file.sh"
funcname "$@"
}
First call sources the real file, which redefines the function with the
real body; the stub's trailing `funcname "$@"` then calls the freshly-
defined real implementation. Sourcing the file also redefines stubs for
any sibling functions the same file declares, so they don't re-source.
Safety nets:
- Missing manifest → fall back to eager loading (`export LP_LAZY=0`).
No regression risk if someone enables LP_LAZY=1 on a stale install
whose regen never ran.
- LP_LOAD_TRACE=1 still works in lazy mode — it records the manifest
parse + each eager file (tagged LAZY-manifest / LAZY-EAGER) so Phase 4
can measure the actual saving.
No automatic flip yet — this commit only adds the path. Phase 4 will set
LP_LAZY=1 by default for the CLI entrypoint (and re-measure with the
trace tool from Phase 1).
Signed-off-by: librelad <librelad@digitalangels.vip>
scripts/source/files/generate_function_manifest.sh — scans every .sh in
scripts/ (skip-list matches generate_arrays.sh, plus excludes peer_shell.sh
which is a standalone forced-command target) and emits
scripts/source/files/arrays/function_manifest.sh:
declare -gA LP_FN_MAP=(
[acquireSingletonLock]="crontab/task/crontab_task_processor.sh"
[adoptDockerSubnet]="checks/requirements/check_docker_network.sh"
... # 698 entries
)
LP_EAGER_FILES=(
"backup/db/backup_db.sh"
"source/files/arrays/files_app.sh"
... # 32 entries (~7% of files)
)
The lazy loader (Phase 3) consumes LP_FN_MAP to install autoload stubs of
the form `name() { source "$LP_FN_MAP[name]"; name "$@"; }`. First call
sources the real file, which redefines the stub with the real body;
subsequent calls hit the real one. LP_EAGER_FILES enumerates files with
top-level side effects (variable assignments, source calls, bare commands
outside any function) — those MUST always source so the side effects fire.
Heuristic correctness, in order of importance:
- Function header detection requires EMPTY parens (`name()`), not just
`name(` — otherwise lines like `if (...)`, `for (...)`, `while (...)`
in embedded awk/perl get misread as bash function defs.
- Handles three function styles: `name() {` (same line), `name()\n{`
(LibrePortal convention), and one-liners `name() { body; }`.
- Tracks { } balance for inside-function depth, with the safe fallback that
ambiguous cases get marked eager (false positive = no behaviour change;
false negative would skip a needed source).
- Files containing embedded awk/perl with their own { } blocks (about 6 of
them: cli_debug_commands.sh, crontab_task_processor.sh, backup_db.sh,
backup_files.sh, etc.) get false-positive flagged eager — acceptable
because they just stay eager-loaded, matching today's behaviour.
- Collisions report to stderr (last-write wins, same as eager-load
semantics); no collisions found in the current tree.
Wiring:
- lpRegenArrays (`libreportal regen arrays`) now also runs the manifest
generator when the existing arrays need regen, keeping the two in sync.
- update.sh's quick-deploy regen step does the same after copying files
to the live install. Best-effort: failures don't abort because lazy
loading is opt-in (LP_LAZY=1) in Phase 3 and not the default yet.
Scanned: 454 files, indexed 698 function definitions, 32 eager (9 real,
23 auto-generated arrays + the manifest itself). 0 name collisions.
No behaviour change in this commit — the manifest is just data the loader
in Phase 3 will use. The default eager loading path is untouched.
Signed-off-by: librelad <librelad@digitalangels.vip>
