Give management-platform a scoped ServiceAccount for k8s backup/restore

The pod running app.py had no kubectl binary and only a read-only SA
(management-platform-viewer-sa) — backup-k8s-apps.sh/restore-k8s-apps.sh
could never actually run from inside it. Adds:

- management-platform-backup-role (ClusterRole, bound via RoleBinding only
  in n8n/odoo/mautic/nextcloud/erpnext — not cluster-wide): get/list/watch
  on pods/deployments/configmaps/ingresses/PVCs, pods/exec create, pods
  create+delete (needed for restore's populate-before-scale-up loader pod),
  secrets get/list/update/patch, deployments/scale update/patch only (no
  write on full Deployment/Service/ConfigMap/Ingress specs).
- management-platform-sa, replacing management-platform-viewer-sa as the
  pod's identity — inherits the old viewer-role's read-only bindings too
  (retargeted in management-platform-viewer-rbac.yaml) so the existing
  cluster-view page keeps working under one SA.
- k3s binary hostPath-mounted read-only into the pod (same pattern as the
  Jenkins agent's docker-cli container), so `k3s kubectl` is available
  where no separate kubectl binary exists.

Both scripts updated to fall back to k3s kubectl when no kubectl is on
PATH, and to use /proc/meminfo instead of `free` for the resource-safety
check (not present in the pod's minimal image). Also fixes a real issue
found while live-testing this from inside the pod: the pod's pre-existing
/root hostPath mount also exposes the host's own admin ~/.kube/config,
which k3s kubectl was silently preferring over the scoped SA token —
forcing --kubeconfig=/dev/null in both scripts closes that.

Verified end-to-end from inside the actual management-platform pod:
manifests/secret/DB-dump/PVC-data backup for n8n succeeds under the new
SA's scoped permissions, and a kube-system access attempt is correctly
rejected (Forbidden) once the kubeconfig leak is closed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
root
2026-08-20 11:30:03 +02:00
parent 8255527254
commit f3f08c3ef6
5 changed files with 222 additions and 25 deletions

View File

@@ -26,6 +26,26 @@
set -uo pipefail
# When run inside the management-platform pod, only the static k3s binary is
# hostPath-mounted (no separate kubectl binary) — fall back to `k3s kubectl`.
# IMPORTANT: this pod also has a pre-existing rw hostPath mount of /root (for
# unrelated file-browsing features), which means the host's own root-owned
# ~/.kube/config (full cluster-admin) is silently visible inside the pod too.
# `--kubeconfig=/dev/null` is REQUIRED here to stop k3s kubectl from picking
# that up as a base config — without it, every command below silently runs
# as cluster-admin instead of the scoped management-platform-sa token,
# defeating the RBAC in management-platform-backup-rbac.yaml entirely.
if ! command -v kubectl &>/dev/null && command -v k3s &>/dev/null; then
kubectl() {
k3s kubectl \
--kubeconfig=/dev/null \
--server=https://kubernetes.default.svc \
--certificate-authority=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt \
--token="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
"$@"
}
fi
# --------------------------------------------------
# Parse --apps
# --------------------------------------------------
@@ -132,7 +152,9 @@ log_status() {
# --------------------------------------------------
check_resources() {
local avail_mb avail_gb
avail_mb=$(free -m | awk '/^Mem:/{print $7}')
# /proc/meminfo instead of `free` — `free` isn't present in the
# management-platform pod's minimal image, /proc/meminfo always is.
avail_mb=$(awk '/^MemAvailable:/{printf "%d", $2/1024}' /proc/meminfo)
avail_gb=$(df -BG / | awk 'NR==2{gsub("G","",$4); print $4}')
if [ "$avail_mb" -lt 1536 ]; then
echo " ❌ RAM available ${avail_mb}MB < 1536MB threshold — aborting."

View File

@@ -36,6 +36,26 @@ set -uo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR"
# When run inside the management-platform pod, only the static k3s binary is
# hostPath-mounted (no separate kubectl binary) — fall back to `k3s kubectl`.
# IMPORTANT: this pod also has a pre-existing rw hostPath mount of /root (for
# unrelated file-browsing features), which means the host's own root-owned
# ~/.kube/config (full cluster-admin) is silently visible inside the pod too.
# `--kubeconfig=/dev/null` is REQUIRED here to stop k3s kubectl from picking
# that up as a base config — without it, every command below silently runs
# as cluster-admin instead of the scoped management-platform-sa token,
# defeating the RBAC in management-platform-backup-rbac.yaml entirely.
if ! command -v kubectl &>/dev/null && command -v k3s &>/dev/null; then
kubectl() {
k3s kubectl \
--kubeconfig=/dev/null \
--server=https://kubernetes.default.svc \
--certificate-authority=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt \
--token="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
"$@"
}
fi
APPS_FILTER="all"
VALID_APPS="frappe odoo nextcloud mautic n8n"
@@ -98,7 +118,9 @@ echo "========================================="
check_resources() {
local avail_mb avail_gb
avail_mb=$(free -m | awk '/^Mem:/{print $7}')
# /proc/meminfo instead of `free` — `free` isn't present in the
# management-platform pod's minimal image, /proc/meminfo always is.
avail_mb=$(awk '/^MemAvailable:/{printf "%d", $2/1024}' /proc/meminfo)
avail_gb=$(df -BG / | awk 'NR==2{gsub("G","",$4); print $4}')
if [ "$avail_mb" -lt 1536 ] || [ "$avail_gb" -lt 10 ]; then
echo " ❌ Resource safety threshold hit (RAM=${avail_mb}MB, Disk=${avail_gb}GB) — aborting."
@@ -139,15 +161,21 @@ for app in $ALL_APPS; do
app_ok=true
# ---- 1. Secret + manifests (apply — idempotent) ----
# ---- 1. Secret (apply — idempotent update, no create: every Secret
# restored here already exists in a same-cluster restore) ----
if [ -f "$APP_DIR/secret.yaml" ]; then
echo -n " 🔑 Applying Secret ... "
kubectl apply -f "$APP_DIR/secret.yaml" &>/dev/null && echo "✅" || { echo "⚠️ FAILED"; app_ok=false; }
fi
if [ -f "$APP_DIR/manifests.yaml" ]; then
echo -n " 📄 Applying manifests ... "
kubectl apply -f "$APP_DIR/manifests.yaml" &>/dev/null && echo "✅" || { echo "⚠️ FAILED"; app_ok=false; }
fi
# NOTE: manifests.yaml (Deployment/Service/ConfigMap/Ingress specs) is
# captured by backup-k8s-apps.sh but deliberately NOT re-applied here.
# Same-cluster restore-in-place never needs to reconcile those specs —
# only Secret values, PVC data, and DB content actually change — and
# the pod's RBAC (management-platform-backup-role) intentionally grants
# no write on Deployments/Services/ConfigMaps/Ingresses beyond the
# deployments/scale subresource used in step 2/4 below. manifests.yaml
# is kept in every backup purely as a captured reference for a future
# fresh-cluster/DR restore path, which is explicitly out of scope today.
# ---- 2. Scale app down ----
prior_replicas=$(kubectl get deployment "$app_deploy" -n "$ns" -o jsonpath='{.spec.replicas}' 2>/dev/null || echo 1)

View File

@@ -22,7 +22,7 @@ spec:
app.kubernetes.io/managed-by: navitrends-pfe-migration
owner: ameni-boukattaya
spec:
serviceAccountName: management-platform-viewer-sa
serviceAccountName: management-platform-sa
containers:
- name: management-platform
image: management-platform:latest
@@ -47,6 +47,9 @@ spec:
- mountPath: /app/config.py
name: config-py
readOnly: true
- mountPath: /usr/local/bin/k3s
name: k3s-bin
readOnly: true
volumes:
- name: docker-sock
hostPath:
@@ -68,3 +71,7 @@ spec:
hostPath:
path: /root/management-platform/config.py
type: File
- name: k3s-bin
hostPath:
path: /usr/local/bin/k3s
type: File

View File

@@ -0,0 +1,149 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: management-platform-backup-role
labels:
owner: ameni-boukattaya
app.kubernetes.io/managed-by: navitrends-pfe-migration
rules:
# Read + resolve exec targets (deploy/<name> shorthand needs get on the
# Deployment plus get/list on the Pods it selects).
- apiGroups: [""]
resources: ["pods", "persistentvolumeclaims"]
verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
resources: ["deployments"]
verbs: ["get", "list", "watch"]
# Read-only manifest capture for backup (`kubectl get deployment,service,
# configmap,ingress -o yaml`). Services and Deployments are already
# readable via management-platform-viewer-role (this SA also holds that
# binding — see management-platform-viewer-rbac.yaml); configmaps and
# ingresses are not, so they're granted explicitly here. Read-only, no
# write — matches "no write beyond scale" for everything but Secrets.
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["get", "list", "watch"]
- apiGroups: ["networking.k8s.io"]
resources: ["ingresses"]
verbs: ["get", "list", "watch"]
# Run pg_dump/mysqldump/psql/mysql/tar inside existing DB and app pods,
# and inside the short-lived loader pod restore creates for PVC repopulation.
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create"]
# Create/delete ONLY the short-lived alpine loader pod restore uses to
# repopulate a PVC after the app Deployment is scaled to 0 (no running
# app pod exists at that point to exec into — see restore-k8s-apps.sh
# step 3). Does not grant any write on the app/DB Deployments' own pods.
- apiGroups: [""]
resources: ["pods"]
verbs: ["create", "delete"]
# Scale only — deliberately NOT "update"/"patch" on the deployments
# resource itself, so this role cannot change a Deployment's image,
# env, mounts, etc. Same-cluster restore never reconciles Deployment/
# Service/ConfigMap/Ingress specs; those are captured in each backup's
# manifests.yaml for reference but are not re-applied by restore-k8s-apps.sh.
- apiGroups: ["apps"]
resources: ["deployments/scale"]
verbs: ["get", "update", "patch"]
# Read DB credentials for pg_dump/mysqldump/psql/mysql (backup + restore),
# and roll a Secret's values back to what's in the backup during restore
# (kubectl apply -f secret.yaml). No create/delete — every Secret this
# role touches already exists; restore only ever updates in place.
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "list", "update", "patch"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: management-platform-sa
namespace: management-platform
labels:
owner: ameni-boukattaya
app.kubernetes.io/managed-by: navitrends-pfe-migration
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: management-platform-backup-binding
namespace: n8n
labels:
owner: ameni-boukattaya
app.kubernetes.io/managed-by: navitrends-pfe-migration
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: management-platform-backup-role
subjects:
- kind: ServiceAccount
name: management-platform-sa
namespace: management-platform
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: management-platform-backup-binding
namespace: odoo
labels:
owner: ameni-boukattaya
app.kubernetes.io/managed-by: navitrends-pfe-migration
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: management-platform-backup-role
subjects:
- kind: ServiceAccount
name: management-platform-sa
namespace: management-platform
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: management-platform-backup-binding
namespace: mautic
labels:
owner: ameni-boukattaya
app.kubernetes.io/managed-by: navitrends-pfe-migration
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: management-platform-backup-role
subjects:
- kind: ServiceAccount
name: management-platform-sa
namespace: management-platform
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: management-platform-backup-binding
namespace: nextcloud
labels:
owner: ameni-boukattaya
app.kubernetes.io/managed-by: navitrends-pfe-migration
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: management-platform-backup-role
subjects:
- kind: ServiceAccount
name: management-platform-sa
namespace: management-platform
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: management-platform-backup-binding
namespace: erpnext
labels:
owner: ameni-boukattaya
app.kubernetes.io/managed-by: navitrends-pfe-migration
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: management-platform-backup-role
subjects:
- kind: ServiceAccount
name: management-platform-sa
namespace: management-platform

View File

@@ -1,12 +1,3 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: management-platform-viewer-sa
namespace: management-platform
labels:
owner: ameni-boukattaya
app.kubernetes.io/managed-by: navitrends-pfe-migration
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
@@ -36,7 +27,7 @@ roleRef:
name: management-platform-viewer-role
subjects:
- kind: ServiceAccount
name: management-platform-viewer-sa
name: management-platform-sa
namespace: management-platform
---
apiVersion: rbac.authorization.k8s.io/v1
@@ -53,7 +44,7 @@ roleRef:
name: management-platform-viewer-role
subjects:
- kind: ServiceAccount
name: management-platform-viewer-sa
name: management-platform-sa
namespace: management-platform
---
apiVersion: rbac.authorization.k8s.io/v1
@@ -70,7 +61,7 @@ roleRef:
name: management-platform-viewer-role
subjects:
- kind: ServiceAccount
name: management-platform-viewer-sa
name: management-platform-sa
namespace: management-platform
---
apiVersion: rbac.authorization.k8s.io/v1
@@ -87,7 +78,7 @@ roleRef:
name: management-platform-viewer-role
subjects:
- kind: ServiceAccount
name: management-platform-viewer-sa
name: management-platform-sa
namespace: management-platform
---
apiVersion: rbac.authorization.k8s.io/v1
@@ -104,7 +95,7 @@ roleRef:
name: management-platform-viewer-role
subjects:
- kind: ServiceAccount
name: management-platform-viewer-sa
name: management-platform-sa
namespace: management-platform
---
apiVersion: rbac.authorization.k8s.io/v1
@@ -121,7 +112,7 @@ roleRef:
name: management-platform-viewer-role
subjects:
- kind: ServiceAccount
name: management-platform-viewer-sa
name: management-platform-sa
namespace: management-platform
---
apiVersion: rbac.authorization.k8s.io/v1
@@ -138,7 +129,7 @@ roleRef:
name: management-platform-viewer-role
subjects:
- kind: ServiceAccount
name: management-platform-viewer-sa
name: management-platform-sa
namespace: management-platform
---
apiVersion: rbac.authorization.k8s.io/v1
@@ -155,5 +146,5 @@ roleRef:
name: management-platform-viewer-role
subjects:
- kind: ServiceAccount
name: management-platform-viewer-sa
name: management-platform-sa
namespace: management-platform