The main sweep — ~260 call sites across ~100 files move from string
concatenation on a single root to appDir/storageAppDirs/storageAppConfigs.
On a single-root install the resolved paths are identical, so this is a
no-op until a location is registered.
Enumerators were the interesting half. `for d in "$containers_dir"/*/`
appears in the menus, the registry/artifact scanners and the DNS setup —
and a shell glob cannot list a rootless 751 tree at all, which is the
same bug config_find_file.sh already documents in a comment. Routing them
through storageAppDirs (which enumerates as the owning user) fixes that
alongside the multi-root work.
Three places needed judgement rather than substitution:
db_app_scan.sh deletes database rows and port allocations for apps whose
folder is missing, and reaps "empty" app dirs. With a storage location
unmounted, every app on it looks exactly like that. Each of those
branches now gates on appStorageAvailable first — an app on an unplugged
drive is skipped with a notice, never deleted.
instance_create.sh rewrites cloned hooks so an instance touches its own
directory instead of the base app's. Its sed matched ${containers_dir}<type>,
which this sweep just replaced with $(appDir <type>) — so it would have
silently stopped redirecting, and an instance would have written to the
original's files (the adguard auth adapter case its own comment warns
about). Now matches both appDir forms, verified against bare, quoted,
unrelated-app, legacy and prose cases.
peer_shell/peer_pull streamed and extracted relative to the primary root.
Both now use the app's own root, and peer_shell keeps a single-root
fallback since it runs as a restricted SSH shell with no LibrePortal env.
Also fixes a pre-existing bug found on the way: webui_app_config.sh
tested "$containers_dir/frontend/data/last_update", one level short of the
real tree under the libreportal app dir, so the WebUI refresh trigger
after a config update has never once fired.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fixing _instanceRewriteTools does nothing for an instance already on disk, and
a clone from the old code is broken in ways that never announce themselves:
every Tools action answers "App '<slug>' has no tool '<id>'" because
dockerAppRunTool wants app<Ucfirst><Pascal>; `authPersistCfg <type>` writes the
instance's new admin credential into the BASE app's config; and the clone
defines the base app's adapter and tool names while its bodies exec against the
instance's container, so the loader keeps whichever it sourced last and the
base app's user tools can end up administering the instance — decided by
nothing but find(1) order. Seen on a live install: the generated manifest
resolved [appBookstackListUsers] to bookstack_test's copy.
libreportal instance repair [slug] [--dry-run]
Rewrites the template dir only — no container is touched, nothing reinstalled,
so it does not route through the task system the way create/remove do.
Idempotent by construction. Two of the three renames match their own output
(appMattermost_teest… still starts with appMattermost), and a clone from the
old code is only PARTLY wrong — its suffix hooks were always correct and end at
the slug with no trailing underscore, which the infix rule would otherwise read
as type + id + () and append the id twice
(appSetupComposeTags_nextcloud_family_family). Three sentinels park the
already-correct spellings before the rewrite and restore them after, so a
healthy instance is a no-op and an interrupted run can just be re-run.
Verified against fixtures built with the old rule set for all five
multi-instance apps that ship tools: after repair each tree is byte-identical
to a fresh clone from the fixed cloner, a second pass reports "already
correct", and --dry-run leaves checksums untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four defects that all reduce to "a function an instance defines is invisible
to the code that dispatches it". Reported as `mattermost_teest has no upgrade
verifier`, for an app whose verifier was on disk the whole time.
* generate_function_manifest.sh shipped 0664. lpRegenArrays invokes it as an
executable, so it died rc=126 on every call and `|| true` swallowed it — the
manifest was never rebuilt on any live system, only laid down at deploy.
Its sibling generate_arrays.sh is 0775, which is why the files_*.sh arrays
looked current while the manifest was byte-identical to the shipped copy.
* lpRegenArrays now runs both generators through bash rather than depending on
the exec bit, reports a manifest failure instead of hiding it, and treats a
new containers/<app> dir as stale — the one event on a live box that adds
functions was the one the scripts/-only mtime check could not see.
* updaterHasVerifier consults the disk before answering no. The CLI runs
LP_LAZY=1, where the container scan is skipped and every function must come
from the build-time manifest, so an app created after the build reads as
having no verifier. GATE 1 then refuses an upgrade that is fully verifiable,
and updaterUpgradeAuto's `|| continue` drops the app in silence for good.
Self-healing regardless of manifest staleness, which matters because a
self-update restores the shipped manifest and drops instance entries again.
* _instanceRewriteTools gains three renames. authAdapter_<type>_<method>() was
caught by neither the prefix rule (no word boundary before _<type>) nor the
suffix rule (needs () right after the type), so the clone defined the base
app's adapter name while pointing at its own container — every instance user
tool answered "does not implement", and which definition survived came down
to find(1) order. Bare-app arguments to authAdapterCall/authPersistCfg went
unrewritten too, so an instance's password reset wrote the credential into
the base app's config. And dockerAppRunTool wants app<Ucfirst><Pascal>, which
no rule produced, so every tool on every instance was unreachable. The infix
rename runs before the suffix rename: the reverse order appends the id half
twice (appSetupComposeTags_nextcloud_work_work).
Verified on a live install: the upgrade ladder now plans mattermost_teest
11.9 -> 11.10, and `regen arrays --force` indexes the instance hooks.
Also carries in-flight instance-removal regen work from a concurrent session
on the same worktree (_lpRegenOrphanedApp, instanceRemove's WebUI refresh).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both now clear every guard: services are <type>-prefixed, and their host ports
became random in the previous commit.
Enabling them surfaced a real bug that would have made vaultwarden instances
fail to start, found by dry-running the clone path before trusting the flag.
Eight apps define an app-specific compose-tags hook named with the app as a
SUFFIX — appSetupComposeTags_vaultwarden — and docker_config_setup_data.sh
dispatches it as appSetupComposeTags_${app_name}. The tools rewrite only
renamed the <type>_ PREFIX form, so a clone kept the base name: it defined a
function nobody calls (colliding with the base app's), its ADMIN_TOKEN and
SIGNUPS_ALLOWED tags were never filled, and the pre-start guard would have
refused to launch the instance. Now renamed, anchored on the () of a definition
so only real function names are touched.
The same hooks pass tag NAMES as strings ("VAULTWARDEN_ADMIN_TOKEN_1_TAG"),
invisible to the lowercase renames, while the cloned compose had already moved
to <SLUG>_..._TAG. Those are rewritten too, mirroring compose rule 4. Verified:
the tags the cloned hook sets now match the cloned compose exactly.
Also affects matrix, nextcloud, speedtest, pihole, gluetun and wireguard, which
ship the same hook shape — latent for those, since none are enabled.
WebUI: the instance bar on app details rendered nothing at all for apps without
instance support, which reads as "this build has no instance feature" and sends
people hunting for a setting that isn't missing. It now states the reason where
the pills would be, and names the blocking ports when it can — the port rows
are in the config the frontend already holds, so it mirrors
_instanceCheckPortsInstanceable (skipping disabled and random rows). The other
blocker lives in the compose, which the frontend never sees, so that case is
left unexplained rather than guessed at.
Bookstack's rewritten compose and tool tree remain byte-identical to the
running instances.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both pinned an arbitrary host port — vaultwarden 8201:80, searxng 8083:8080 —
which was the only thing blocking them from being instanced. Neither number is
meaningful the way pihole's 53 or stalwart's 25 are, so both become
random:<internal> and portAllocate assigns each install (and each future
instance) its own. The ports appeared nowhere else: no hook, no compose, no
docs. Neither app is installed on the maintainer's box, so nothing to migrate.
Both now clear every instance guard. Of the eight apps the port guard caught,
that leaves six, all genuinely one-per-host.
Also made compose rewrite rules 2 and 3 skip commented lines, for the same
reason rule 1 already does. Spotted while verifying the above: vaultwarden
parks an optional exporter behind #, and rule 2 rewrote the container_name
inside that dead block while the service key above it kept the old name,
leaving it internally inconsistent. Harmless — rule 2 is anchored on
container_name: so it could never reach the image line — but there is no reason
to touch a commented block at all. Bookstack's rewritten identities remain
byte-identical to the running instance.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Audit of the per-app install hooks for singleton assumptions. The naming work
so far made identities unique, but a second copy still has to bind its own
ports, and `8201:80` is the same 8201 for every instance — the second container
simply fails at compose-up. `random:<internal>` is what makes an app
instanceable, since portAllocate then hands each instance its own host port.
Seven apps are caught: pihole (53 tcp+udp), stalwart (25/465/587/993), unbound
(5335 tcp+udp), traefik (443), searxng (8083), vaultwarden (8201), stoat
(7881). The message distinguishes the two cases, because they need opposite
fixes: an arbitrary pin like vaultwarden's 8201 should just become random,
while a DNS server on 53 or a mail server on 25 is genuinely one-per-host and
should never be instanced.
Runs before anything is cloned — this is a property of the app, not of the
instance. Bookstack is unaffected (all its ports are already random).
The rest of the hook audit found nothing further to fix:
- No hook writes to another app's config or deployed directory. The three that
reference ${containers_dir}traefik / headscale only test [[ -d ]] to detect
whether those are installed.
- Only two hooks read a foreign CFG_ namespace, and both are system-wide
settings (CFG_DOCKER_INSTALL_TYPE, CFG_ENABLE_VIDEO), not another app's.
- No app declares a fixed container IP; all come from IP_TAG allocation.
- Host-level writes are limited to wireguard's sysctl IPv4-forwarding drop-in
(global and idempotent) and its /etc/wireguard/params conflict probe. Traefik
writes only under $containers_dir$app_name. Stalwart's /etc/stalwart path is
inside its container.
Not mechanically checkable, so left as maintainer judgement: gluetun is a
network provider other apps join via network_mode container:gluetun-service,
and it plus wireguard hold NET_ADMIN and /dev/net/tun. Both are one-per-host
for reasons no guard can see.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Audit of per-app hooks/tools found 19 of 33 apps whose helpers would have
operated on the BASE app after cloning. Two general causes, both fixed by
rewriting classes rather than patching apps:
- Container references escaped the rewrite whenever a flag sat between the
docker verb and the target (`docker exec -u git gitea-service …`), since the
old rule only matched a name immediately after the verb — and the hyphenated
form missed the `<type>_` rule too. Hook trees now get the same discovered
identity rename the compose does, reading names from the TYPE's compose since
the clone has already been rewritten by then. Safe to apply broadly: the
compose pass runs first and aborts for any app whose identities aren't
<type>-prefixed, so a bare word like stoat's `api` never reaches it.
- Hooks that build the deployed path as "${containers_dir}<type>/..." instead
of "$containers_dir$app_name/..." read and WROTE the base app's files —
adguard's auth adapter edits AdGuardHome.yaml, so an instance would have
rewritten the original's config. The trailing slash is optional in the match:
dashy tests [[ -d "${containers_dir}dashy" ]] and gluetun cds into it, both
ending at the quote. Only the first path component is touched, so
${containers_dir}prometheus/prometheus/... keeps its inner segment.
Re-audit: all 33 apps with hook trees are clean. Stoat still leaks, but it is
refused at the compose stage and never reaches this code.
Volumes audited too, and need no changes: no app uses named volumes, so the
./relative bind mounts every app uses resolve inside each instance's own
deployed dir. The absolute sources that exist are host or in-container paths
correctly shared read-only (/etc/localtime, /sys, /etc/ssl/certs). Jitsi's
${CONFIG} is set per-app by its own hook to $containers_dir$app_name/... and so
follows the slug.
Bookstack's rewritten tool tree is byte-identical to the live instance's across
all 8 files, so the running instances are unaffected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The compose rewrite assumed each app had exactly <type>-service and <type>_db.
That holds for Bookstack and almost nothing else: cloning Nextcloud left -db,
-redis and -web pointing at the ORIGINAL app's containers, and Matrix, Ollama,
Mastodon, Owncloud, Gitea, Jitsi, Invidious, Rocketchat and Mattermost all had
the same hole. Docker refuses a duplicate container name and two Traefik
routers sharing a name fight over the host, so those clones could not have
worked.
Service identities are now discovered from the compose itself — its
SERVICE_TAG_<n> markers plus its container_name values — and each is renamed.
Verified across all 38 shipped apps: 15 are fixed, 21 produce byte-identical
output to the old rule (Bookstack among them, so the running instances are
unaffected), and 2 are refused.
Details worth knowing:
- Separators compare as equivalent, so the app dir libreportal_catalog matches
its libreportal-catalog-* services instead of being wrongly refused.
- Tokens are substituted longest-first through placeholders. \b has to end a
token because per-port routers are named <service>-<portname>
(traefik.http.routers.adguard-service-webui), which also means a short name
could otherwise match inside a longer one — ordering is what prevents that.
- Commented-out lines are not harvested. Several templates park an optional
sidecar behind # (adguard-exporter, pihole-exporter, wireguard-exporter);
renaming those also mangled the image name in the same block, leaving a trap
for anyone uncommenting it. Commented image: lines are skipped too.
- image: lines are genuinely excluded now. The old comment claimed service
tokens "never appear in an image path", but libreportal builds a local image
named after its own service and the old rule rewrote that reference.
An app with a service carrying no <type> prefix (stoat's api/database/minio,
prometheus's node-exporter/cadvisor) cannot be made unique mechanically, and
rewriting a bare word like minio would corrupt image: minio/minio. Those are
refused with an explanation and the partial clone is removed, rather than
handed back as an instance that silently fights the base app.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An instance's isolation never needed a domain — its own slug, dir, secrets,
IP and randomly-allocated host port already make two copies independent. But
the routing layer assumed one, so a LAN-only box got a broken instance rather
than a port-served one. Four fixes:
- instanceCreate now rewrites the parent-service column of the cloned config's
PORT_ rows to match the service names it stamps into the compose. That value
is stored as network_resources.parent_service and joined against the
compose-derived service names, so an instance left carrying the TYPE's
service name matched nothing: it rendered in the WebUI with no port, no URL
and no login row despite being up and reachable.
- `instance create --local` (plus a LAN-only toggle in the modal) forces every
port to access=private, traefik=false, for a second copy that should stay
off the domain even when one is configured.
- initializeAppVariables forces the traefik column false when no CFG_DOMAIN_n
is set. Previously a traefik=true port with an empty domain stamped
Host(`app.`) — a trailing-dot host matching nothing — and dragged APP_URL to
https://app. with it, breaking every app that builds its links from APP_URL.
host_setup is blanked for the same reason. The published host port is
untouched; access type, not the traefik flag, gates allocation.
- APP_URL's direct host-port branch now prefers a new $local_ip_v4 (the source
IP for the default route) over $public_ip_v4, which is the WAN address from
an external resolver. LibrePortal never forwards ports, so the WAN address
was unreachable for exactly the LAN/VPN clients that branch serves.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Instance install (bugs found by running one end to end):
- The cloned compose kept the TYPE's tag namespace
(#LIBREPORTAL|BOOKSTACK_APP_KEY_1_TAG|...) while the config had been
re-namespaced to CFG_<SLUG>_*, so tagsProcessorAppConfigValues matched
nothing, the placeholders survived and the pre-start guard refused to
launch. Rewrite the tag names and *_DATA tokens too — narrowly, so an
app whose compose sets a real env var named after itself is untouched.
- Tools/hooks kept uppercase CFG_<TYPE>_ reads, so an instance
provisioned itself from the type's config and ignored its own values.
- Cloned hooks were never loaded: both loaders run at startup, before the
instance dir exists, so _appCallHook's `declare -F` found nothing and
every <slug>_install_* hook silently no-opped — for bookstack that is
the readiness probe and the admin bootstrap. Source the instance's own
scripts in-process, then regen arrays + manifest for later runs.
- bookstack's hook hardcoded the container name after `docker exec -e ...`
flags, where the rewriter can't see it, so an instance's admin bootstrap
ran against the BASE app's container — including a tinker DELETE of a
user. Target "$app_name" instead, and teach the rewriter the
container="<type>" assignment form used by auth adapters.
network_resources uniqueness:
UNIQUE(resource_type, resource_value) is right for 'ip' and 'port' but the
port-tag writer stores descriptive rows in the same table with INSERT OR
REPLACE, so every install DELETED the matching row from whichever app held
it. traefik_managed and url_accessible are booleans, so the whole table
could only ever hold one row of each. Observed live: installing a second
bookstack took all four traefik_managed/url_accessible rows from stoat and
bookstack, and removing that instance took the stolen rows with it.
Replace it with a partial unique index scoped to ip/port, and migrate
existing databases in place (SQLite can't drop a constraint, so the table
is rebuilt inside a transaction). The migration is invoked from
portUpdateComposeTags, not just databaseCreateTables — the latter only
runs from startPreInstall, which a working install never re-runs.
Verified: two bookstacks now hold port_tag_internal=80, traefik_managed
and url_accessible simultaneously; duplicate host ports and IPs are still
rejected; instance installs, serves HTTP 200, provisions its own admin in
its own database, and removes cleanly with no orphan rows.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
instanceCreate re-namespaced the config KEYS (CFG_<TYPE>_* ->
CFG_<SLUG>_*) but left APP_NAME's VALUE at the type. installApp resolves
the app it operates on from CFG_<SLUG>_APP_NAME, so `instance create
bookstack fwtest` ran the entire install pipeline against the BASE app:
"Install bookstack", the base deployed dir, compose down/up on the
already-running base container, base DB row re-stamped — and the instance
never installed at all. Only the template dir was left behind.
Every base app ships APP_NAME == its own slug; stamp instances the same.
Instance install still does not complete after this: the cloned compose
keeps the type's tag namespace (#LIBREPORTAL|BOOKSTACK_APP_KEY_1_TAG|...)
while the config now defines CFG_<SLUG>_*, so the placeholders never
substitute and the pre-start guard refuses to launch. Fixing that is a
separate change to _instanceRewriteCompose.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Lets a *multi-instance-capable* app run as several fully isolated instances
on one box (e.g. two Bookstack/WordPress sites, or a "family" + "work"
Nextcloud) — distinct data, DB, subdomain, backups and update cadence.
Design: an instance is just another app. It gets its own slug (<type>_<id>),
its own CFG_<SLUG>_* namespace, deployed dir, DB row, IP/port allocation and
host, so the entire existing pipeline (scan, install, services, routing,
updater, backups) treats it like any app with zero changes. All
instance-specific rewriting is confined to a clone of the type's template;
the shipped template and the core engine are untouched.
Gating: opt-in per app via CFG_<TYPE>_MULTI_INSTANCE=true. Only Bookstack
carries it for now (the validated reference). The other 31 apps are
unaffected — the feature is invisible unless the flag is present.
- scripts/instance/instance_create.sh — clone + re-namespace config, rewrite
compose identity (container_name / Traefik routers / backup labels) and
per-app tools, set a hostname-safe subdomain (PORT field 10), then hand off
to dockerInstallApp. Plus instanceList / instanceRemove.
- libreportal instance create|remove|list — new CLI category; mutations route
through the task system (no new mutating API endpoint).
- WebUI: "instance of <type>" badge + a "New instance" card action on capable
apps, and a create modal (name + domain# + subdomain, live host preview)
that dispatches the standard task. Capability/instance-of read straight off
the already-exposed app config.
Known follow-ups (documented): flip the flag on more apps after a compose
identity check (Nextcloud next); per-app tools are best-effort isolated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: librelad <librelad@digitalangels.vip>
librelad