Investigated a resource-safety abort during Odoo restore testing:
available RAM was hovering ~1.3-1.5GB even under normal conditions on
this host (k3s + Jenkins + Docker monitoring/wazuh stack + dev tooling
baseline), not a leak from any single process. 1536MB was tripping on
ordinary single-app backups; 1024MB still leaves real headroom above
what a pg_dump/mysqldump/tar step actually needs.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017avLHFqkiti3g62Anq9sVA
The pod running app.py had no kubectl binary and only a read-only SA
(management-platform-viewer-sa) — backup-k8s-apps.sh/restore-k8s-apps.sh
could never actually run from inside it. Adds:
- management-platform-backup-role (ClusterRole, bound via RoleBinding only
in n8n/odoo/mautic/nextcloud/erpnext — not cluster-wide): get/list/watch
on pods/deployments/configmaps/ingresses/PVCs, pods/exec create, pods
create+delete (needed for restore's populate-before-scale-up loader pod),
secrets get/list/update/patch, deployments/scale update/patch only (no
write on full Deployment/Service/ConfigMap/Ingress specs).
- management-platform-sa, replacing management-platform-viewer-sa as the
pod's identity — inherits the old viewer-role's read-only bindings too
(retargeted in management-platform-viewer-rbac.yaml) so the existing
cluster-view page keeps working under one SA.
- k3s binary hostPath-mounted read-only into the pod (same pattern as the
Jenkins agent's docker-cli container), so `k3s kubectl` is available
where no separate kubectl binary exists.
Both scripts updated to fall back to k3s kubectl when no kubectl is on
PATH, and to use /proc/meminfo instead of `free` for the resource-safety
check (not present in the pod's minimal image). Also fixes a real issue
found while live-testing this from inside the pod: the pod's pre-existing
/root hostPath mount also exposes the host's own admin ~/.kube/config,
which k3s kubectl was silently preferring over the scoped SA token —
forcing --kubeconfig=/dev/null in both scripts closes that.
Verified end-to-end from inside the actual management-platform pod:
manifests/secret/DB-dump/PVC-data backup for n8n succeeds under the new
SA's scoped permissions, and a kube-system access attempt is correctly
rejected (Forbidden) once the kubeconfig leak is closed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
backup-myapps.sh/restore-myapps.sh only ever knew about Docker volumes,
which no longer exist for n8n/odoo/mautic/nextcloud/frappe now that
they're all running in k3s — nightly backups have been silently hollow
for these apps since the migration. These new scripts capture/restore
each app's k8s manifests, Secret, DB (pg_dump/mysqldump via kubectl exec),
and app-data PVC contents (kubectl exec tar, never raw filesystem access),
reusing the existing local+VM+R2 3-2-1 storage/retention logic unchanged.
Tested standalone: full backup of all 5 apps produces verified non-empty
manifests/secrets/DB dumps/PVC data; restore-in-place tested end-to-end
against n8n (DB row counts and PVC data matched exactly pre/post, app
verified healthy over HTTPS).
Not wired into management-platform yet (app.py stays untouched) — restore
script also doesn't yet handle a fresh/DR cluster where the PVCs and their
manifests don't already exist (same-cluster restore only for now).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
