Optimization proposal¶
The repo works, but has accumulated real complexity: duplicated device data across three sources, 13 provisioning scripts that overlap Ansible roles, in-progress config-directory migrations, and a couple of latent inconsistencies found while writing this documentation set. This is a prioritized list of what to fix, in order of value vs. effort. Nothing here has been implemented yet except where noted.
1. Single device source of truth (highest value)¶
Problem: config/devices.yml is the real source of truth for
Terraform, but Ansible has its own, manually-maintained inventory
(ansible/inventory/production/hosts.yml, hosts-lite.yml). Nothing keeps
them in sync automatically — scripts/generate-ansible-inventory.sh exists
to bridge them, but has to be run and reviewed by hand. This is the same
class of problem the wiki's devices.md generator (scripts/generate_device_docs.py,
added alongside this doc) solves for documentation: read devices.yml
once, generate everything else.
Proposal: Extend generate-ansible-inventory.sh (or replace it with a
small Python script sharing the YAML-parsing logic with
generate_device_docs.py) so it's the only writer of
ansible/inventory/production/hosts*.yml, and add a CI check that fails if
the committed inventory doesn't match what the generator would produce from
config/devices.yml. This removes an entire class of drift bugs (a device
added to one file and not the other).
Effort: medium. The tricky part is preserving per-host overrides that
currently live directly in hosts.yml (e.g. host_vars/-style entries) —
those need an escape hatch that survives regeneration.
2. terraform/main.tf production-prefix inconsistency¶
Problem: every environment's device-ID local uses
"${local.devices_config.environments.<env>.prefix}-${serial}" — except
production, which hardcodes "rpi-prod-${serial}" instead of reading
environments.production.prefix (terraform/main.tf:21-24 and :61-63).
Currently harmless because the two values are identical, but renaming the
prefix key in devices.yml for production alone would silently not
change production device IDs, while doing the same for staging/dev would
work as expected — a foot-gun for whoever touches this next without reading
the code closely.
Proposal: make production consistent with the other five blocks (two one-line changes). Low effort, should just be done — filed here rather than fixed inline while writing docs because it's a behavior change to production-affecting Terraform and deserves its own reviewed PR, not a drive-by edit.
3. Script vs. Ansible-role overlap¶
Problem: 13 shell scripts (~3600 lines) in scripts/ overlap
responsibilities with Ansible roles/playbooks — e.g.
provision-mqtt-certs.sh vs. the mqtt_client role, reinstall-nodered.sh
vs. ansible/playbooks/reinstall-nodered.yml. Two parallel provisioning
paths for the same outcome means fixes have to be applied twice (or get
applied once and drift). scripts/batch-provision.sh also references
./rpi-init.sh, which doesn't exist in scripts/ — a real broken
reference, not just theoretical duplication.
Proposal: audit each script against the Ansible role/playbook that
covers the same ground; keep the script only where it does something
Ansible genuinely can't (one-time bootstrap before Ansible can even connect,
interactive credential prompts), and either delete or thin the rest down to
a wrapper that calls the playbook. Start with batch-provision.sh since
it's already broken.
Effort: medium-high — needs care per script, not a bulk deletion.
4. config/ directory migration debt¶
Problem: two group_vars/all.yml files exist —
ansible/group_vars/all.yml (vault-encrypted) and
ansible/inventory/production/group_vars/all.yml (plaintext, the one that
actually loads for these playbooks — see the security-quick-wins PR, which
found ansible/group_vars/all/ isn't even on Ansible's search path here).
Plus a dedicated scripts/migrate-config-directories.sh +
validate-migration.sh + taskfiles/migration.yml, all pointing at an
in-progress reorganization.
Proposal: finish the migration (whatever its original end-state was —
check SCRIPTS-CLEANUP.md in the archive for context), confirm the
migration scripts are no longer needed, and delete them along with the
now-redundant group_vars/all.yml duplicate. ansible/group_vars/all/nodered.yml
specifically is currently dead configuration — it defines
node_red_project_encryption_key etc., but nothing loads that directory
for these playbooks; it happens to be harmless right now only because its
values match the role defaults in roles/node_red/defaults/main.yml
exactly. Worth resolving explicitly (either wire it up or remove it) so a
future edit to that file doesn't silently do nothing.
5. node_red_project_encryption_key in plaintext git¶
Problem: ansible/group_vars/all/nodered.yml:14 (or wherever it ends
up after item 4 above) has node_red_project_encryption_key: "tagaitechteam"
committed in plaintext. It has to be identical across the whole fleet to
decrypt flows_cred.json, which is exactly why it's tempting to just leave
it in a group_vars file — but that means it's sitting in git history
un-encrypted.
Proposal: move it into the existing Ansible Vault
(ansible/inventory/production/group_vars/vault.yml). Changing the value
itself requires re-entering every Node-RED credential fleet-wide, so do
that as a separate, deliberate, coordinated action — not bundled with the
vault migration.
6. Git history still contains the leaked MQTT password¶
Problem: docs/All connections.yaml (removed from the tree in the
security-quick-wins PR) still contains the plaintext MQTT password in git
history. Removing it from history means git filter-repo (or
git filter-branch) followed by a force-push.
Proposal: rotate the MQTT password in EMQX first — that's the real fix, and it's independent of whether history gets rewritten. History purge is a separate, explicitly owner-approved action given the force-push and the fact that every existing clone/fork would need to be re-cloned afterward. Not something to do as a side effect of a docs cleanup.
7. GitHub Pages / Cloudflare Access bypass — resolved by switching to Cloudflare Pages¶
Original problem: the docs wiki was originally planned on GitHub Pages,
whose *.github.io origin is reachable directly, bypassing Cloudflare
Access on docs.tagai.uk.
What actually happened: GitHub Pages turned out to be blocked
regardless — this repo is private and our GitHub org plan doesn't include
private-repo GitHub Pages. Switched to Cloudflare Pages (direct upload from
GitHub Actions) instead, which has no such plan gate. This closes the
*.github.io bypass specifically, but introduces a residual, not-yet-verified
caveat: Cloudflare Pages projects also get a default *.pages.dev fallback
domain, and whether that's actually covered by Cloudflare Access hasn't
been confirmed — Cloudflare's dashboard-only "protect preview deployments"
toggle isn't exposed via Terraform and, per Cloudflare's own docs, may only
cover per-deployment preview URLs rather than the project's general
.pages.dev domain. See ../security-access.md for
the current state — check Settings → Access on the
gaisro-technika-docs Pages project in the dashboard to close this out.
Suggested order¶
1 (device source of truth) and 2 (prefix bug) first — they're the
foundation everything else in the fleet's tooling builds on, and 2 is a
five-minute fix once someone's looking at main.tf anyway. 5 and 6
(secrets) can happen in parallel with those since they don't depend on
anything. 3 (script consolidation) and 4 (config migration cleanup) are
the biggest effort and lowest urgency — good candidates for whenever
there's a quiet week, not urgent enough to block anything else.