Investigated a resource-safety abort during Odoo restore testing:
available RAM was hovering ~1.3-1.5GB even under normal conditions on
this host (k3s + Jenkins + Docker monitoring/wazuh stack + dev tooling
baseline), not a leak from any single process. 1536MB was tripping on
ordinary single-app backups; 1024MB still leaves real headroom above
what a pg_dump/mysqldump/tar step actually needs.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017avLHFqkiti3g62Anq9sVA
18 straight clone failures today (early EOF / invalid index-pack) even with
http.postBuffer/http.version tuning applied — Gitea's HTTP endpoint itself is
degraded right now, not something fixable client-side. Sidesteps it entirely:
the docker-agent pod template now hostPath-mounts /root/CloudOps (read-only,
see k3s-cloud-init.groovy) and Checkout just `cp -r`s from it, no network
fetch at all. Build/SonarQube paths revert to ./platform/ (no more repo/
subdirectory, since we're not cloning into one).
Real tradeoff, accepted deliberately: builds now reflect whatever's on the
VPS's disk at build time, not necessarily what's pushed to Gitea. Depends on
/root/CloudOps being kept current via the normal git pull workflow.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The Git plugin's checkout([...]) step doesn't expose http.postBuffer/
http.version tuning, which is what's reliably fixed every stuck
push/clone today (VPS and both laptops) — meanwhile checkout was failing
~10 times in a row, consistently dying at 84-95% during the shallow clone
(early EOF / invalid index-pack). Bypasses the plugin entirely: a plain
`git clone` with those settings applied, wrapped in its own 8-attempt
bash retry loop, using the existing dd13f593-06b1-4a5a-9477-2df0d186b490
credential (confirmed via credentials.xml to be a UsernamePasswordCredentialsImpl,
compatible with the usernamePassword binding as-is).
Code now lands in a repo/ subdirectory instead of the workspace root, so
every stage referencing a repo-relative path was updated to match:
- Build Docker Image: ./platform/ -> ./repo/platform/
- SonarQube Analysis: -Dsonar.sources=platform -> -Dsonar.sources=repo/platform
Deploy to Kubernetes needed no changes — it only references the already-built
image tag and hardcoded kubectl args, no filesystem paths into the checkout.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
api_backup_run and restore_start were still hardcoded to the Docker-era
backup-myapps.sh/restore-myapps.sh, which target Docker volumes that no
longer exist for the 5 apps now running in k3s (n8n, odoo, mautic,
nextcloud, frappe/erpnext) — every backup/restore triggered from the UI
has been silently hollow for these apps since the migration.
- api_backup_run: /root/backup-myapps.sh -> /root/CloudOps/backup/backup-k8s-apps.sh
- restore_start: switched from the old image-relative path (which resolved
to /app/restore-myapps.sh, baked into the Docker image from platform/)
to an absolute /root/CloudOps/backup/restore-k8s-apps.sh path — the new
backup/ folder is a sibling of platform/, not part of the Docker build
context, so it's only reachable via the pod's existing /root hostPath
mount, the same way api_backup_run already reaches its script.
--apps flag shape and the job-log streaming contract are unchanged, so no
frontend/template changes needed for the core flow. modules/backups.py's
three hardcoded myapps-backup-* references (get_local_backups, get_vm_backups
x2, delete_backup's filename regex) now also recognize the new
myapps-k8s-backup-* prefix, so both backup lineages stay visible/manageable
in the UI.
Depends on the previous commit (scoped ServiceAccount + k3s kubectl access
for the pod) to actually function when triggered from the UI.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The pod running app.py had no kubectl binary and only a read-only SA
(management-platform-viewer-sa) — backup-k8s-apps.sh/restore-k8s-apps.sh
could never actually run from inside it. Adds:
- management-platform-backup-role (ClusterRole, bound via RoleBinding only
in n8n/odoo/mautic/nextcloud/erpnext — not cluster-wide): get/list/watch
on pods/deployments/configmaps/ingresses/PVCs, pods/exec create, pods
create+delete (needed for restore's populate-before-scale-up loader pod),
secrets get/list/update/patch, deployments/scale update/patch only (no
write on full Deployment/Service/ConfigMap/Ingress specs).
- management-platform-sa, replacing management-platform-viewer-sa as the
pod's identity — inherits the old viewer-role's read-only bindings too
(retargeted in management-platform-viewer-rbac.yaml) so the existing
cluster-view page keeps working under one SA.
- k3s binary hostPath-mounted read-only into the pod (same pattern as the
Jenkins agent's docker-cli container), so `k3s kubectl` is available
where no separate kubectl binary exists.
Both scripts updated to fall back to k3s kubectl when no kubectl is on
PATH, and to use /proc/meminfo instead of `free` for the resource-safety
check (not present in the pod's minimal image). Also fixes a real issue
found while live-testing this from inside the pod: the pod's pre-existing
/root hostPath mount also exposes the host's own admin ~/.kube/config,
which k3s kubectl was silently preferring over the scoped SA token —
forcing --kubeconfig=/dev/null in both scripts closes that.
Verified end-to-end from inside the actual management-platform pod:
manifests/secret/DB-dump/PVC-data backup for n8n succeeds under the new
SA's scoped permissions, and a kube-system access attempt is correctly
rejected (Forbidden) once the kubeconfig leak is closed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
backup-myapps.sh/restore-myapps.sh only ever knew about Docker volumes,
which no longer exist for n8n/odoo/mautic/nextcloud/frappe now that
they're all running in k3s — nightly backups have been silently hollow
for these apps since the migration. These new scripts capture/restore
each app's k8s manifests, Secret, DB (pg_dump/mysqldump via kubectl exec),
and app-data PVC contents (kubectl exec tar, never raw filesystem access),
reusing the existing local+VM+R2 3-2-1 storage/retention logic unchanged.
Tested standalone: full backup of all 5 apps produces verified non-empty
manifests/secrets/DB dumps/PVC data; restore-in-place tested end-to-end
against n8n (DB row counts and PVC data matched exactly pre/post, app
verified healthy over HTTPS).
Not wired into management-platform yet (app.py stays untouched) — restore
script also doesn't yet handle a fresh/DR cluster where the PVCs and their
manifests don't already exist (same-cluster restore only for now).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
New modules/kubernetes.py lists pods and deployments across the 8 app
namespaces via the in-cluster management-platform-viewer-sa (read-only,
no secrets/exec/log), with metrics-server usage as a best-effort extra
that degrades silently when unreachable. New /cluster page and
/api/cluster endpoint, nav entry, and templates render pods/deployments
grouped by namespace using the existing card/badge design system.
Additive only — no existing Docker routes or views touched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
ServiceAccount management-platform-viewer-sa (in management-platform ns),
ClusterRole management-platform-viewer-role (get/list/watch on pods,
services, deployments.apps only - no secrets, no write verbs), and a
RoleBinding per app namespace (n8n, odoo, mautic, erpnext, nextcloud,
jenkins-agents, jenkins, management-platform). Already applied to the
live cluster and verified via kubectl auth can-i; committing here for
source-of-truth tracking.
Source-of-truth manifest for the management-platform Deployment, with
serviceAccountName: management-platform-viewer-sa added so the pod can
authenticate for the read-only k8s cluster-view feature. Not yet applied
to the cluster.
