Fix backups Audit/Details for k8s-format archives + broken local access

Two separate bugs found while testing the Audit/Details UI against a
real k8s-format backup (myapps-k8s-backup-*):

1. Format-hardcoded checks. audit_backup()'s Internal Structure and
   Volume Count checks assumed the legacy Docker-era archive layout
   (volumes/*.tar.gz, compose-files/) and would fail every k8s-format
   backup (per-app dirs with manifests.yaml/db-dump.sql.gz/pvc-data.tar.gz)
   even when it's perfectly healthy - same root cause as the
   myapps-backup-*/myapps-k8s-backup-* prefix bug fixed earlier, just in
   the audit checks instead of the listing/delete glob. cloud_backup.py's
   r2_audit_backup() had the identical unpatched filename regex, and
   app.py's /api/backups/details route had its own, which outright
   400'd any k8s-format filename before even looking at it.

2. A bigger, separate bug this surfaced: get_local_backups(),
   get_vm_backups(), and _resolve_archive_path() all branch on
   RUNNING_ON_MAIN_SERVER to decide between direct filesystem access and
   SSH-to-self - and from inside the management-platform pod that flag's
   hostname check can be wrong for the pod's own ephemeral hostname
   depending on how it's evaluated, sending these down the SSH branch
   using a topology (separate warm-standby-on-the-VM SSH path) that
   doesn't apply to a pod that already has /root hostPath-mounted in.
   Fixed by trying direct filesystem access first wherever the archive
   would already be locally reachable, falling back to the existing SSH
   paths otherwise - this preserves the original warm-standby-on-VM
   failover design (a real second deployment of this platform on the VM
   host, kept reachable via SSH when the main server is down) completely
   unchanged; it only adds the fast local path for the case where the
   caller already has direct filesystem access.

Verified against both a real k8s-format backup (odoo, single-app) and a
real legacy-format backup on disk - both audit correctly now, with
format-appropriate checks and labels.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017avLHFqkiti3g62Anq9sVA

backup/backup-k8s-apps.sh

@@ -156,8 +156,14 @@ check_resources() {
     # management-platform pod's minimal image, /proc/meminfo always is.
     avail_mb=$(awk '/^MemAvailable:/{printf "%d", $2/1024}' /proc/meminfo)
     avail_gb=$(df -BG / | awk 'NR==2{gsub("G","",$4); print $4}')
-    if [ "$avail_mb" -lt 1536 ]; then
-        echo "   ❌ RAM available ${avail_mb}MB < 1536MB threshold — aborting."
+    # Threshold lowered 1536->1024MB 2026-08-21: this host's steady-state
+    # available RAM sits ~1.3-1.5GB even when idle (k3s + Jenkins + Docker
+    # monitoring/wazuh stack + dev tooling baseline) — investigated, no
+    # single runaway process, just genuine multi-service baseline. 1536MB
+    # was tripping on ordinary single-app backups. 1024MB still leaves
+    # real headroom above what a pg_dump/mysqldump/tar op actually needs.
+    if [ "$avail_mb" -lt 1024 ]; then
+        echo "   ❌ RAM available ${avail_mb}MB < 1024MB threshold — aborting."
         log_status "FAILED" "$BACKUP_NAME" "resource_safety_ram_${avail_mb}mb"
         exit 1
     fi

backup/restore-k8s-apps.sh

@@ -122,7 +122,10 @@ check_resources() {
     # management-platform pod's minimal image, /proc/meminfo always is.
     avail_mb=$(awk '/^MemAvailable:/{printf "%d", $2/1024}' /proc/meminfo)
     avail_gb=$(df -BG / | awk 'NR==2{gsub("G","",$4); print $4}')
-    if [ "$avail_mb" -lt 1536 ] || [ "$avail_gb" -lt 10 ]; then
+    # Threshold lowered 1536->1024MB 2026-08-21 — see backup-k8s-apps.sh
+    # check_resources() comment for the investigation (baseline host RAM,
+    # not a leak).
+    if [ "$avail_mb" -lt 1024 ] || [ "$avail_gb" -lt 10 ]; then
         echo "   ❌ Resource safety threshold hit (RAM=${avail_mb}MB, Disk=${avail_gb}GB) — aborting."
         exit 1
     fi

management-platform-sites-rbac.yaml

@@ -0,0 +1,32 @@
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: management-platform-ingress-reader-role
+  labels: {owner: ameni-boukattaya, app.kubernetes.io/managed-by: navitrends-pfe-migration}
+rules:
+  - apiGroups: ["networking.k8s.io"]
+    resources: ["ingresses"]
+    verbs: ["get", "list", "watch"]
+---
+# Odoo's and Nextcloud's Ingress objects live in the `default` namespace
+# (historical — created there during the original Docker->k8s migration,
+# unlike n8n/mautic/erpnext whose Ingress lives in their own app namespace).
+# management-platform-sa has no binding at all in `default` otherwise, and
+# `default` also hosts unrelated live services (cloudops-ingress, jenkins-
+# ingress, stage-navitrends-ingress, etc.) — so this grants ONLY ingress
+# read there, not the full backup ClusterRole, to keep blast radius to
+# exactly what the Application Sites page needs (real domain/TLS lookup).
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: management-platform-ingress-reader-binding
+  namespace: default
+  labels: {owner: ameni-boukattaya, app.kubernetes.io/managed-by: navitrends-pfe-migration}
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: management-platform-ingress-reader-role
+subjects:
+  - kind: ServiceAccount
+    name: management-platform-sa
+    namespace: management-platform

platform/app.py

@@ -461,7 +461,9 @@ def api_backup_details():
     backup_file = data.get('backup_file', '').strip()
     source      = data.get('source', 'local').strip()
 
-    if not re.match(r'^myapps-backup-\d{8}_\d{6}\.tar\.gz$', backup_file):
+    # Accepts both the legacy Docker-era lineage (myapps-backup-*) and the
+    # k8s-native one (myapps-k8s-backup-*, from backup-k8s-apps.sh).
+    if not re.match(r'^myapps-(k8s-)?backup-\d{8}_\d{6}\.tar\.gz$', backup_file):
         return jsonify({'error': 'Invalid filename'}), 400
 
     if source == 'local':
@@ -484,16 +486,21 @@ def api_backup_details():
     }
 
     # ── size + mtime ──────────────────────────────────────────────
+    # Try direct stat first regardless of RUNNING_ON_MAIN_SERVER: the
+    # management-platform pod has /root hostPath-mounted from the main
+    # server, so /root/backups is already right here even though the
+    # hostname check that flag relies on never matches inside a pod (it
+    # used to always take the SSH branch, using a key/topology this
+    # in-pod deployment doesn't have, and silently come back empty).
     if source == 'local':
-        if RUNNING_ON_MAIN_SERVER:
+        if os.path.exists(archive_path):
             # direct stat on this machine
-            if os.path.exists(archive_path):
-                st = os.stat(archive_path)
-                result['size_bytes'] = st.st_size
-                result['size_human'] = _human_bytes(st.st_size)
-                mtime = datetime.fromtimestamp(st.st_mtime, tz=timezone.utc)
-                result['created_at'] = mtime.strftime('%Y-%m-%d %H:%M:%S UTC')
-                result['age_days']   = (datetime.now(tz=timezone.utc) - mtime).days
+            st = os.stat(archive_path)
+            result['size_bytes'] = st.st_size
+            result['size_human'] = _human_bytes(st.st_size)
+            mtime = datetime.fromtimestamp(st.st_mtime, tz=timezone.utc)
+            result['created_at'] = mtime.strftime('%Y-%m-%d %H:%M:%S UTC')
+            result['age_days']   = (datetime.now(tz=timezone.utc) - mtime).days
             if os.path.exists(sha_path):
                 try:
                     with open(sha_path) as f:
@@ -517,15 +524,14 @@ def api_backup_details():
                 result['sha256'] = sha_out.strip()
 
     elif source == 'vm':
-        if not RUNNING_ON_MAIN_SERVER:
-            # we ARE the VM — direct stat
-            if os.path.exists(archive_path):
-                st = os.stat(archive_path)
-                result['size_bytes'] = st.st_size
-                result['size_human'] = _human_bytes(st.st_size)
-                mtime = datetime.fromtimestamp(st.st_mtime, tz=timezone.utc)
-                result['created_at'] = mtime.strftime('%Y-%m-%d %H:%M:%S UTC')
-                result['age_days']   = (datetime.now(tz=timezone.utc) - mtime).days
+        if os.path.exists(archive_path):
+            # direct stat when reachable (VM's own local disk, or a mount)
+            st = os.stat(archive_path)
+            result['size_bytes'] = st.st_size
+            result['size_human'] = _human_bytes(st.st_size)
+            mtime = datetime.fromtimestamp(st.st_mtime, tz=timezone.utc)
+            result['created_at'] = mtime.strftime('%Y-%m-%d %H:%M:%S UTC')
+            result['age_days']   = (datetime.now(tz=timezone.utc) - mtime).days
             if os.path.exists(sha_path):
                 try:
                     with open(sha_path) as f:

platform/modules/backups.py

@@ -55,10 +55,34 @@ def _ssh_main(remote_cmd, timeout=30):
 # BACKUPS
 # ────────────────────────────────────────────────────────────────
 
+def _list_local_dir(local_dir):
+    """Direct filesystem listing, newest first, both lineages."""
+    files = (
+        glob.glob(f'{local_dir}/myapps-backup-*.tar.gz')
+        + glob.glob(f'{local_dir}/myapps-k8s-backup-*.tar.gz')
+    )
+    files.sort(key=os.path.getmtime, reverse=True)
+    return [os.path.basename(f) for f in files[:20]]
+
+
 def get_local_backups():
     # myapps-backup-* is the legacy Docker-era lineage; myapps-k8s-backup-*
     # is the k8s-native one (backup-k8s-apps.sh). Listed together, sorted
     # by mtime, so history from both stays visible in the UI.
+    #
+    # Try direct filesystem access first: the management-platform pod has
+    # /root hostPath-mounted from the main server (see k8s/management-
+    # platform-deployment.yaml), so /root/backups is already right here.
+    # RUNNING_ON_MAIN_SERVER's hostname check never matches inside a pod
+    # (its hostname is the pod name, not the host's), so this used to
+    # always fall through to the SSH-to-self branch below — which also
+    # doesn't actually work in this topology (wrong key for this deploy
+    # model) — silently returning an empty list. Falling back to SSH only
+    # when there's genuinely no local mount keeps this working for any
+    # future non-mounted deployment too.
+    if os.path.isdir('/root/backups'):
+        return _list_local_dir('/root/backups')
+
     stdout, _ = _ssh_main(
         "ls -t /root/backups/myapps-backup-*.tar.gz "
         "/root/backups/myapps-k8s-backup-*.tar.gz 2>/dev/null | head -20"
@@ -73,34 +97,33 @@ def get_local_backups():
 
 
 def get_vm_backups():
+    # Same fix as get_local_backups(): try direct access first (covers the
+    # case where this process's filesystem actually is the VM's), fall
+    # back to SSH to the VM host otherwise — which is the real case for
+    # the management-platform pod, since only /root is mounted in, not
+    # anything from the separate VM backup host.
+    local_dir = '/backups/cloudproject'
+    if os.path.isdir(local_dir):
+        return _list_local_dir(local_dir)
+
     vm_backups = []
-    if RUNNING_ON_MAIN_SERVER:
-        try:
-            cmd = (
-                f"ssh -i {VM_KEY} -p {VM_PORT} "
-                f"-o StrictHostKeyChecking=no -o ConnectTimeout=10 "
-                f"-o BatchMode=yes "
-                f"{VM_USER}@{VM_HOST} "
-                f"'ls -t /backups/cloudproject/myapps-backup-*.tar.gz "
-                f"/backups/cloudproject/myapps-k8s-backup-*.tar.gz 2>/dev/null | head -20'"
-            )
-            stdout, _ = _run(cmd, timeout=25)
-            if stdout:
-                for line in stdout.split('\n'):
-                    line = line.strip()
-                    if line and '.tar.gz' in line:
-                        vm_backups.append(os.path.basename(line))
-        except Exception as e:
-            print(f"[backups] VM backup fetch error: {e}")
-    else:
-        backup_dir = '/backups/cloudproject'
-        if os.path.exists(backup_dir):
-            files = (
-                glob.glob(f'{backup_dir}/myapps-backup-*.tar.gz')
-                + glob.glob(f'{backup_dir}/myapps-k8s-backup-*.tar.gz')
-            )
-            files.sort(key=os.path.getmtime, reverse=True)
-            vm_backups = [os.path.basename(f) for f in files[:20]]
+    try:
+        cmd = (
+            f"ssh -i {VM_KEY} -p {VM_PORT} "
+            f"-o StrictHostKeyChecking=no -o ConnectTimeout=10 "
+            f"-o BatchMode=yes "
+            f"{VM_USER}@{VM_HOST} "
+            f"'ls -t /backups/cloudproject/myapps-backup-*.tar.gz "
+            f"/backups/cloudproject/myapps-k8s-backup-*.tar.gz 2>/dev/null | head -20'"
+        )
+        stdout, _ = _run(cmd, timeout=25)
+        if stdout:
+            for line in stdout.split('\n'):
+                line = line.strip()
+                if line and '.tar.gz' in line:
+                    vm_backups.append(os.path.basename(line))
+    except Exception as e:
+        print(f"[backups] VM backup fetch error: {e}")
     return vm_backups
 
 
@@ -115,16 +138,19 @@ def get_vm_backups():
 # test, easy to extend without pushing complexity back up.
 
 def _resolve_archive_path(backup_file, source, add):
-    """Figure out the local path to the archive, pulling it from the main
-    server first if needed. Returns the path, or None if it could not be
-    resolved (in which case an error result dict has already been added
-    via `add` and the caller should bail out)."""
+    """Figure out the local path to the archive, pulling it over SSH only
+    if it's genuinely not reachable on this filesystem. Returns the path,
+    or None if it could not be resolved (in which case an error result
+    dict has already been added via `add` and the caller should bail out).
+
+    Tries direct access first regardless of RUNNING_ON_MAIN_SERVER: the
+    management-platform pod has /root hostPath-mounted from the main
+    server, so /root/backups is already right here even though the
+    hostname check that flag relies on never matches inside a pod."""
     if source == 'local':
         archive_path = f"/root/backups/{backup_file}"
-    else:
-        archive_path = f"/backups/cloudproject/{backup_file}"
-
-    if not RUNNING_ON_MAIN_SERVER and source == 'local':
+        if os.path.exists(archive_path):
+            return archive_path
         tmp_path = f"/tmp/audit_{backup_file}"
         if not os.path.exists(tmp_path):
             pull_cmd = (
@@ -137,9 +163,24 @@ def _resolve_archive_path(backup_file, source, add):
             if not os.path.exists(tmp_path):
                 add('File Access', 'fail', f'Could not pull from main server: {err}')
                 return None
-        archive_path = tmp_path
-
-    return archive_path
+        return tmp_path
+    else:
+        archive_path = f"/backups/cloudproject/{backup_file}"
+        if os.path.exists(archive_path):
+            return archive_path
+        tmp_path = f"/tmp/audit_vm_{backup_file}"
+        if not os.path.exists(tmp_path):
+            pull_cmd = (
+                f"scp -i {VM_KEY} -P {VM_PORT} "
+                f"-o StrictHostKeyChecking=no -o ConnectTimeout=15 "
+                f"{VM_USER}@{VM_HOST}:/backups/cloudproject/{backup_file} "
+                f"{tmp_path}"
+            )
+            _out, err = _run(pull_cmd, timeout=120)
+            if not os.path.exists(tmp_path):
+                add('File Access', 'fail', f'Could not pull from VM: {err}')
+                return None
+        return tmp_path
 
 
 def _check_file_size(archive_path, add):
@@ -216,13 +257,54 @@ def _list_archive_members(archive_path):
         return []
 
 
-def _check_internal_structure(members, add):
+# The two backup lineages have completely different internal layouts:
+#   - legacy (myapps-backup-*, from backup-myapps.sh): volumes/*.tar.gz,
+#     compose-files/, backup-info.txt at the archive root.
+#   - k8s-native (myapps-k8s-backup-*, from backup-k8s-apps.sh): per-app
+#     subdirs (n8n/, odoo/, mautic/, nextcloud/, frappe/) each with
+#     manifests.yaml, secret.yaml, db-dump.sql.gz, pvc-data.tar.gz, plus
+#     backup-info.txt at the archive root.
+# Auditing a k8s-format archive against the legacy layout would always
+# fail it ("volumes/ missing") even when it's a perfectly good backup —
+# same root cause as the myapps-backup-*/myapps-k8s-backup-* prefix bug
+# already fixed elsewhere in this file, just in the audit checks instead
+# of the listing/delete glob.
+_K8S_APP_NAMES = ('n8n', 'odoo', 'mautic', 'nextcloud', 'frappe')
+
+
+def _is_k8s_format(backup_file):
+    return os.path.basename(backup_file).startswith('myapps-k8s-backup-')
+
+
+def _check_internal_structure(members, backup_file, add):
     if not members:
         add('Internal Structure', 'warn', 'Could not inspect archive members')
         return
 
-    has_volumes = any('volumes/' in m for m in members)
     has_info = any('backup-info.txt' in m for m in members)
+
+    if _is_k8s_format(backup_file):
+        app_dirs = sorted({
+            parts[1] for m in members
+            if len(parts := m.split('/')) > 1 and parts[1] in _K8S_APP_NAMES
+        })
+        has_app_data = any(
+            m.endswith(('manifests.yaml', 'db-dump.sql.gz', 'pvc-data.tar.gz'))
+            for m in members
+        )
+        issues = []
+        if not has_app_data:
+            issues.append('no per-app manifests/db-dump/pvc-data found')
+        if not has_info:
+            issues.append('backup-info.txt missing')
+        if issues:
+            add('Internal Structure', 'fail', ' · '.join(issues))
+            return
+        add('Internal Structure', 'pass',
+            f'backup-info.txt ✓  apps: {", ".join(app_dirs) or "?"}')
+        return
+
+    has_volumes = any('volumes/' in m for m in members)
     has_compose = any('compose-files/' in m for m in members)
     vol_count = len([m for m in members if '/volumes/' in m and m.endswith('.tar.gz')])
 
@@ -302,7 +384,20 @@ def _check_executable_scripts(archive_path, add):
         add('Executable Scripts', 'pass', 'No unexpected executable scripts found')
 
 
-def _check_volume_count(members, add):
+def _check_volume_count(members, backup_file, add):
+    if _is_k8s_format(backup_file):
+        pvc_archives = [m for m in members if m.endswith('/pvc-data.tar.gz')]
+        v = len(pvc_archives)
+        if v == 0:
+            add('App Data Archives', 'fail', 'No pvc-data.tar.gz found in backup')
+        elif v < 5:
+            add('App Data Archives', 'warn',
+                f'{v} app(s) backed up — fine if this was a --apps filtered run, '
+                f'otherwise expected 5 for a full backup')
+        else:
+            add('App Data Archives', 'pass', f'{v} app data archives present')
+        return
+
     vol_archives = [m for m in members if 'volumes/' in m and m.endswith('.tar.gz')]
     v = len(vol_archives)
     if v == 0:
@@ -379,10 +474,10 @@ def audit_backup(backup_file, source='local'):
     _check_archive_integrity(archive_path, add)
 
     members = _list_archive_members(archive_path)
-    _check_internal_structure(members, add)
+    _check_internal_structure(members, backup_file, add)
     _check_security_scan(members, add)
     _check_executable_scripts(archive_path, add)
-    _check_volume_count(members, add)
+    _check_volume_count(members, backup_file, add)
 
     score = _score_checks(checks)
     has_fails = any(c['status'] == 'fail' for c in checks)

platform/modules/cloud_backup.py

@@ -345,10 +345,12 @@ def r2_audit_backup(key: str) -> dict:
         else:
             report["checks"].append(f"✅ File size looks healthy ({_human_size(size)})")
 
-        # Check 5: Filename format
+        # Check 5: Filename format — accepts both the legacy Docker-era
+        # lineage (myapps-backup-*) and the k8s-native one
+        # (myapps-k8s-backup-*, from backup-k8s-apps.sh).
         import re
         filename = key.replace("backups/", "")
-        if re.match(r"myapps-backup-\d{8}_\d{6}\.tar\.gz", filename):
+        if re.match(r"myapps-(k8s-)?backup-\d{8}_\d{6}\.tar\.gz", filename):
             report["checks"].append("✅ Filename format is valid")
         else:
             report["warnings"].append(f"⚠️ Unexpected filename format: {filename}")

platform/modules/kubernetes.py

@@ -21,11 +21,12 @@ NAMESPACES = [
 _core_v1 = None
 _apps_v1 = None
 _custom_objects = None
+_networking_v1 = None
 K8S_AVAILABLE = False
 
 
 def _init_client():
-    global _core_v1, _apps_v1, _custom_objects, K8S_AVAILABLE
+    global _core_v1, _apps_v1, _custom_objects, _networking_v1, K8S_AVAILABLE
     if _core_v1 is not None or not _K8S_IMPORT_OK:
         return
     try:
@@ -33,6 +34,7 @@ def _init_client():
         _core_v1 = client.CoreV1Api()
         _apps_v1 = client.AppsV1Api()
         _custom_objects = client.CustomObjectsApi()
+        _networking_v1 = client.NetworkingV1Api()
         K8S_AVAILABLE = True
     except Exception as e:
         print(f"[kubernetes] in-cluster config unavailable: {e}")
@@ -181,6 +183,54 @@ def list_deployments():
     return deployments
 
 
+def get_ingress_info(name, namespace):
+    """Real domain/TLS for one Ingress, or None if unavailable/not found.
+    Used by modules/sites.py instead of the old hardcoded Docker-era
+    domain fields — reflects actual live k8s Ingress state."""
+    _init_client()
+    if not K8S_AVAILABLE:
+        return None
+    try:
+        ing = _networking_v1.read_namespaced_ingress(name, namespace)
+    except Exception:
+        return None
+    host = None
+    if ing.spec and ing.spec.rules:
+        for rule in ing.spec.rules:
+            if rule.host:
+                host = rule.host
+                break
+    tls = bool(ing.spec and ing.spec.tls)
+    return {'host': host, 'tls': tls}
+
+
+def list_deployments_for_namespace(ns):
+    """Targeted single-namespace deployment list (vs. list_deployments()'s
+    fixed NAMESPACES sweep) — used by modules/sites.py per app."""
+    _init_client()
+    if not K8S_AVAILABLE:
+        return []
+    try:
+        resp = _apps_v1.list_namespaced_deployment(ns)
+    except Exception as e:
+        print(f"[kubernetes] list deployments failed for {ns}: {e}")
+        return []
+    out = []
+    for dep in resp.items:
+        desired = dep.spec.replicas or 0
+        ready = dep.status.ready_replicas or 0
+        containers = dep.spec.template.spec.containers or []
+        image = containers[0].image if containers else '—'
+        out.append({
+            'name':    dep.metadata.name,
+            'desired': desired,
+            'ready':   ready,
+            'health':  _deploy_health(desired, ready),
+            'image':   image,
+        })
+    return out
+
+
 def get_pod_metrics():
     """Best-effort CPU/mem per pod via metrics-server. Returns {} if it's not
     installed or unreachable — callers must treat an empty dict as 'no data',

platform/modules/sites.py

@@ -1,15 +1,27 @@
 # modules/sites.py — Managed application sites registry + live status
-import re
+#
+# REWRITTEN 2026-08-21: this used to be a fully Docker-era hardcoded registry
+# (container names like odoo-clean-odoo-1/frappe-erpnext/nextcloud-app that
+# no longer exist since the k8s migration, checked via `docker inspect` over
+# SSH) with hardcoded domain/port fields — nextcloud and mautic even had
+# `domain: None`, silently falling back to dead Docker host-port URLs while
+# their real k8s Ingress domains (next.cloud.nav.ovh, mautics.nav.ovh) sat
+# unused. Now sources everything live from the cluster: Deployment/pod
+# status via modules/kubernetes.py's in-cluster client, and domain/TLS via
+# the actual Ingress objects (not a static guess) — see
+# management-platform-sites-rbac.yaml for the narrow ingress-read grant this
+# needed in the `default` namespace (odoo/nextcloud's Ingress historically
+# live there, not in their app namespace).
 import time
-from urllib.request import urlopen, Request
-from urllib.error import URLError
+import urllib.request
 
-from config import MAIN_SERVER_IP, RUNNING_ON_MAIN_SERVER
-from modules.backups import _ssh_main, get_all_root_containers, get_container_status
+from modules.kubernetes import get_ingress_info, list_deployments_for_namespace
 
 
 # ────────────────────────────────────────────────────────────────
-# STATIC SITE REGISTRY (source of truth for Application Sites UI)
+# STATIC SITE REGISTRY — identity/namespace/naming only. Live state
+# (domain, TLS, deployment/pod health) is always fetched fresh below,
+# never hardcoded.
 # ────────────────────────────────────────────────────────────────
 
 PLATFORM = {
@@ -35,20 +47,21 @@ SITES = [
         'category': 'ERP',
         'icon': 'erpnext',
         'brand_color': '#0089FF',
-        'compose_dir': 'frappe-setup',
-        'main_container': 'frappe-erpnext',
-        'containers': [
-            {'name': 'frappe-erpnext', 'role': 'App', 'image': 'frappe/erpnext:latest'},
-            {'name': 'frappe-mariadb', 'role': 'Database', 'image': 'mariadb:10.6'},
-            {'name': 'frappe-redis', 'role': 'Cache / Queue', 'image': 'redis:alpine'},
-        ],
-        'port': 8080,
-        'internal_port': 8000,
-        'domain': 'erpnext.navitrends.ovh',
-        'domain_protocol': 'http',
-        'ssl_configured': False,
+        'namespace': 'erpnext',
+        'main_deployment': 'erpnext-web',
+        'service_port': 8000,
+        'ingress_name': 'erpnext-ingress',
+        'ingress_namespace': 'erpnext',
         'health_path': '/',
-        'volumes': ['frappe-setup_frappe-sites', 'frappe-setup_mariadb-data'],
+        'pvcs': ['erpnext-sites', 'erpnext-mariadb-data'],
+        'roles': {
+            'erpnext-web': 'App',
+            'erpnext-worker': 'Worker',
+            'erpnext-scheduler': 'Scheduler',
+            'erpnext-mariadb': 'Database',
+            'erpnext-redis-cache': 'Cache',
+            'erpnext-redis-queue': 'Queue',
+        },
     },
     {
         'id': 'odoo',
@@ -57,19 +70,18 @@ SITES = [
         'category': 'ERP',
         'icon': 'odoo',
         'brand_color': '#714B67',
-        'compose_dir': 'odoo-clean',
-        'main_container': 'odoo-clean-odoo-1',
-        'containers': [
-            {'name': 'odoo-clean-odoo-1', 'role': 'App', 'image': 'odoo:18'},
-            {'name': 'odoo-clean-db-1', 'role': 'Database', 'image': 'postgres:15'},
-        ],
-        'port': 8069,
-        'internal_port': 8069,
-        'domain': 'odooo.nav.ovh',
-        'domain_protocol': 'https',
-        'ssl_configured': True,
+        'namespace': 'odoo',
+        'main_deployment': 'odoo',
+        'service_port': 8069,
+        # Historical: created in `default` during migration, not the `odoo` ns.
+        'ingress_name': 'odoo-ingress',
+        'ingress_namespace': 'default',
         'health_path': '/web',
-        'volumes': ['odoo-clean_db-data', 'odoo-clean_odoo-etc'],
+        'pvcs': ['odoo-data', 'odoo-postgres-data'],
+        'roles': {
+            'odoo': 'App',
+            'odoo-postgres': 'Database',
+        },
     },
     {
         'id': 'nextcloud',
@@ -78,19 +90,18 @@ SITES = [
         'category': 'Storage',
         'icon': 'nextcloud',
         'brand_color': '#0082C9',
-        'compose_dir': 'nextcloud-setup',
-        'main_container': 'nextcloud-app',
-        'containers': [
-            {'name': 'nextcloud-app', 'role': 'App', 'image': 'nextcloud:latest'},
-            {'name': 'nextcloud-postgres', 'role': 'Database', 'image': 'postgres:15'},
-        ],
-        'port': 8082,
-        'internal_port': 80,
-        'domain': 'next.cloud.nav.ovh',
-        'domain_protocol': 'https',
-        'ssl_configured': True,
+        'namespace': 'nextcloud',
+        'main_deployment': 'nextcloud',
+        'service_port': 80,
+        # Historical: created in `default` during migration, not the `nextcloud` ns.
+        'ingress_name': 'nextcloud-ingress',
+        'ingress_namespace': 'default',
         'health_path': '/status.php',
-        'volumes': ['nextcloud-setup_nextcloud-data', 'nextcloud-setup_nextcloud-db-data'],
+        'pvcs': ['nextcloud-app-data', 'nextcloud-postgres-data'],
+        'roles': {
+            'nextcloud': 'App',
+            'nextcloud-postgres': 'Database',
+        },
     },
     {
         'id': 'mautic',
@@ -99,20 +110,17 @@ SITES = [
         'category': 'Marketing',
         'icon': 'mautic',
         'brand_color': '#4E5E9E',
-        'compose_dir': 'mautic-setup',
-        'main_container': 'mautic-app',
-        'containers': [
-            {'name': 'mautic-app', 'role': 'App', 'image': 'mautic/mautic:latest'},
-            {'name': 'mautic-mariadb', 'role': 'Database', 'image': 'mariadb:10.11'},
-        ],
-        'port': 8081,
-        'internal_port': 80,
-        'domain': None,
-        'domain_protocol': 'http',
-        'ssl_configured': False,
+        'namespace': 'mautic',
+        'main_deployment': 'mautic',
+        'service_port': 80,
+        'ingress_name': 'mautic-ingress',
+        'ingress_namespace': 'mautic',
         'health_path': '/',
-        'volumes': ['mautic-setup_mautic-data', 'mautic-setup_mautic-db-data'],
-        'networks': ['mautic-network'],
+        'pvcs': ['mautic-app-data', 'mautic-db-data'],
+        'roles': {
+            'mautic': 'App',
+            'mautic-mariadb': 'Database',
+        },
     },
     {
         'id': 'n8n',
@@ -121,210 +129,176 @@ SITES = [
         'category': 'Automation',
         'icon': 'n8n',
         'brand_color': '#EA4B71',
-        'compose_dir': 'n8n-setup',
-        'main_container': 'n8n-app',
-        'containers': [
-            {'name': 'n8n-app', 'role': 'App', 'image': 'n8nio/n8n:latest'},
-            {'name': 'n8n-postgres', 'role': 'Database', 'image': 'postgres:15'},
-        ],
-        'port': 5678,
-        'internal_port': 5678,
-        'domain': None,
-        'domain_protocol': 'http',
-        'ssl_configured': False,
+        'namespace': 'n8n',
+        'main_deployment': 'n8n',
+        'service_port': 5678,
+        'ingress_name': 'n8n-ingress',
+        'ingress_namespace': 'n8n',
         'health_path': '/healthz',
-        'volumes': ['n8n-setup_n8n-data', 'n8n-setup_n8n-db-data'],
-        'networks': ['n8n-network', 'integration-network'],
+        'pvcs': ['n8n-data', 'n8n-postgres-data'],
+        'roles': {
+            'n8n': 'App',
+            'n8n-postgres': 'Database',
+        },
     },
 ]
 
 SITE_BY_ID = {s['id']: s for s in SITES}
 
 
-def _build_urls(site):
-    ip = MAIN_SERVER_IP
-    port = site['port']
-    proto = site.get('domain_protocol', 'http')
+def _build_urls(site, ingress_info):
+    """Domain/TLS come from the live Ingress object — never hardcoded.
+    ingress_info is None if the Ingress couldn't be read (RBAC/API issue)
+    or {'host': str|None, 'tls': bool} otherwise."""
     path = site.get('health_path', '/') or '/'
-    ssl_configured = site.get('ssl_configured', proto == 'https')
-
-    domain = site.get('domain')
-    has_domain = bool(domain)
+    host = ingress_info.get('host') if ingress_info else None
+    has_domain = bool(host)
+    ssl_configured = bool(ingress_info and ingress_info.get('tls'))
 
     if has_domain:
-        if ssl_configured:
-            access_url = f"{proto}://{domain}"
-            health_url = f"{access_url}{path}"
-        else:
-            access_url = f"http://{domain}:{port}"
-            health_url = f"{access_url}{path}"
-    else:
-        access_url = f"http://{ip}:{port}"
+        proto = 'https' if ssl_configured else 'http'
+        access_url = f"{proto}://{host}"
         health_url = f"{access_url}{path}"
+    else:
+        access_url = None
+        health_url = None
 
     return {
         'has_domain': has_domain,
-        'domain': domain,
-        'domain_protocol': proto,
+        'domain': host,
+        'domain_protocol': 'https' if ssl_configured else 'http',
         'ssl_configured': ssl_configured,
-        'ip_url': f"http://{ip}:{port}",
-        'access_url': access_url.rstrip('/'),
+        'access_url': (access_url or '').rstrip('/'),
         'health_url': health_url,
     }
 
 
-def _parse_port_mapping(ports_str, host_port, internal_port=None):
-    """Extract host:container mapping from docker ports string."""
-    if not ports_str:
-        if internal_port and internal_port != host_port:
-            return f"{host_port}→{internal_port}"
-        return str(host_port) if host_port else None
-    m = re.search(rf'0\.0\.0\.0:{host_port}->(\d+)/', ports_str)
-    if m:
-        return f"{host_port}→{m.group(1)}"
-    m = re.search(r'0\.0\.0\.0:(\d+)->(\d+)/', ports_str)
-    if m:
-        return f"{m.group(1)}→{m.group(2)}"
-    return ports_str[:60] if ports_str else None
-
-
 def _probe_http(url, timeout=5):
-    """HTTP health probe — local urllib or remote curl via SSH."""
-    if RUNNING_ON_MAIN_SERVER:
-        try:
-            req = Request(url, method='GET', headers={'User-Agent': 'CloudOps-HealthCheck/1.0'})
-            with urlopen(req, timeout=timeout) as resp:
-                code = resp.getcode()
-                return {
-                    'reachable': 200 <= code < 400,
-                    'http_code': code,
-                    'latency_ms': None,
-                    'error': None,
-                }
-        except URLError as e:
-            return {'reachable': False, 'http_code': None, 'latency_ms': None, 'error': str(e.reason)[:120]}
-        except Exception as e:
-            return {'reachable': False, 'http_code': None, 'latency_ms': None, 'error': str(e)[:120]}
-    else:
-        safe_url = url.replace("'", "'\\''")
+    """Direct HTTP probe. The management-platform pod has normal internet
+    egress and resolves public DNS just like any other pod — no need to
+    hop back out to the host via SSH the way the old Docker-era code did."""
+    try:
+        req = urllib.request.Request(url, method='GET', headers={'User-Agent': 'CloudOps-HealthCheck/1.0'})
         t0 = time.time()
-        out, _ = _ssh_main(
-            f"curl -sf -o /dev/null -w '%{{http_code}}' --connect-timeout {timeout} "
-            f"-m {timeout} '{safe_url}' 2>/dev/null || echo '000'",
-            timeout=timeout + 5,
-        )
-        latency = int((time.time() - t0) * 1000)
-        code_str = (out or '000').strip()
-        try:
-            code = int(code_str)
-        except ValueError:
-            code = 0
-        return {
-            'reachable': 200 <= code < 400,
-            'http_code': code if code else None,
-            'latency_ms': latency,
-            'error': None if 200 <= code < 400 else f'HTTP {code_str}',
-        }
+        with urllib.request.urlopen(req, timeout=timeout) as resp:
+            code = resp.getcode()
+            latency_ms = int((time.time() - t0) * 1000)
+            return {
+                'reachable': 200 <= code < 400,
+                'http_code': code,
+                'latency_ms': latency_ms,
+                'error': None,
+            }
+    except Exception as e:
+        return {'reachable': False, 'http_code': None, 'latency_ms': None, 'error': str(e)[:120]}
 
 
-def _container_map():
-    ctrs = get_all_root_containers()
-    return {c['name']: c for c in ctrs}
+def _deployments_for_site(site):
+    """Live Deployment status for every stack member (app/db/cache/etc),
+    keyed by k8s Deployment name."""
+    live = {d['name']: d for d in list_deployments_for_namespace(site['namespace'])}
+    roles = site.get('roles', {})
+    containers = []
+    running_count = 0
+    for dep_name, role in roles.items():
+        d = live.get(dep_name)
+        if d:
+            is_up = d['ready'] >= 1 and d['desired'] >= 1
+            if is_up:
+                running_count += 1
+            containers.append({
+                'name': dep_name,
+                'role': role,
+                'status': 'running' if is_up else 'stopped',
+                'image_live': d['image'],
+                'ports': f"{d['ready']}/{d['desired']} ready",
+            })
+        else:
+            containers.append({
+                'name': dep_name,
+                'role': role,
+                'status': 'not_found',
+                'image_live': '—',
+                'ports': '—',
+            })
+    main = live.get(site['main_deployment'])
+    main_running = bool(main and main['ready'] >= 1 and main['desired'] >= 1)
+    return containers, running_count, main_running
+
+
+def _site_entry(site, include_health):
+    ingress_info = get_ingress_info(site['ingress_name'], site['ingress_namespace'])
+    urls = _build_urls(site, ingress_info)
+    containers, running_count, main_running = _deployments_for_site(site)
+
+    entry = {
+        'id': site['id'],
+        'name': site['name'],
+        'tagline': site['tagline'],
+        'category': site['category'],
+        'icon': site['icon'],
+        'brand_color': site['brand_color'],
+        'compose_dir': site['namespace'],  # k8s namespace, shown as "COMPOSE PROJECT" in the UI
+        'main_container': site['main_deployment'],
+        'port': site['service_port'],
+        'internal_port': site['service_port'],
+        'port_mapping': f"svc/{site['service_port']}",
+        'ports_raw': f"ClusterIP:{site['service_port']}",
+        'volumes': site.get('pvcs', []),
+        'ip_url': f"{site['main_deployment']}.{site['namespace']}.svc.cluster.local:{site['service_port']} (cluster-internal only)",
+        'container_status': 'running' if main_running else 'stopped',
+        'container_running': main_running,
+        'containers': containers,
+        'containers_running': running_count,
+        'containers_total': len(site.get('roles', {})),
+        **urls,
+    }
+
+    if include_health:
+        if not urls['has_domain']:
+            entry['health'] = {
+                'status': 'down', 'reachable': False, 'http_code': None,
+                'latency_ms': None, 'error': 'No Ingress host found for this app',
+            }
+        elif not main_running:
+            entry['health'] = {
+                'status': 'offline', 'reachable': False, 'http_code': None,
+                'latency_ms': None, 'error': 'Deployment not running',
+            }
+        else:
+            probe = _probe_http(urls['health_url'])
+            entry['health'] = {'status': 'up' if probe['reachable'] else 'down', **probe}
+
+    return entry
 
 
 def get_sites_list(include_health=False):
-    """Return all sites with live container + URL metadata."""
-    ctr_map = _container_map()
-    results = []
-
-    for site in SITES:
-        urls = _build_urls(site)
-        main_name = site['main_container']
-        main_ctr = ctr_map.get(main_name, {})
-        main_status = get_container_status(main_name) if main_name else {'status': 'unknown'}
-
-        containers_live = []
-        running_count = 0
-        for ctr_def in site.get('containers', []):
-            name = ctr_def['name']
-            live = ctr_map.get(name, {})
-            status_raw = live.get('status', '')
-            is_up = 'Up' in status_raw
-            if is_up:
-                running_count += 1
-            containers_live.append({
-                **ctr_def,
-                'status': 'running' if is_up else ('stopped' if live else 'not_found'),
-                'status_raw': status_raw or 'not found',
-                'image_live': live.get('image', ctr_def.get('image', '—')),
-                'ports': live.get('ports', '—'),
-            })
-
-        entry = {
-            'id': site['id'],
-            'name': site['name'],
-            'tagline': site['tagline'],
-            'category': site['category'],
-            'icon': site['icon'],
-            'brand_color': site['brand_color'],
-            'compose_dir': site['compose_dir'],
-            'main_container': main_name,
-            'port': site['port'],
-            'internal_port': site.get('internal_port', site['port']),
-            'port_mapping': _parse_port_mapping(
-                main_ctr.get('ports', ''), site['port'], site.get('internal_port')
-            ),
-            'ports_raw': main_ctr.get('ports', '—'),
-            'volumes': site.get('volumes', []),
-            'networks': site.get('networks', ['default']),
-            'container_status': main_status['status'],
-            'container_running': main_status['status'] == 'running',
-            'containers': containers_live,
-            'containers_running': running_count,
-            'containers_total': len(site.get('containers', [])),
-            **urls,
-        }
-
-        if include_health:
-            if entry['container_running']:
-                probe = _probe_http(urls['health_url'])
-                entry['health'] = {
-                    'status': 'up' if probe['reachable'] else 'down',
-                    **probe,
-                }
-            else:
-                entry['health'] = {
-                    'status': 'offline',
-                    'reachable': False,
-                    'http_code': None,
-                    'latency_ms': None,
-                    'error': 'Container not running',
-                }
-
-        results.append(entry)
-
-    return results
+    """Return all sites with live k8s Deployment + Ingress state."""
+    return [_site_entry(site, include_health) for site in SITES]
 
 
 def get_site_health(site_id):
     site = SITE_BY_ID.get(site_id)
     if not site:
         return None
-    urls = _build_urls(site)
-    status = get_container_status(site['main_container'])
-    if status['status'] != 'running':
+    ingress_info = get_ingress_info(site['ingress_name'], site['ingress_namespace'])
+    urls = _build_urls(site, ingress_info)
+    _, _, main_running = _deployments_for_site(site)
+
+    if not main_running:
         return {
-            'site_id': site_id,
-            'status': 'offline',
-            'reachable': False,
-            'http_code': None,
-            'latency_ms': None,
-            'error': 'Container not running',
+            'site_id': site_id, 'status': 'offline', 'reachable': False,
+            'http_code': None, 'latency_ms': None, 'error': 'Deployment not running',
             'checked_at': time.time(),
         }
-    t0 = time.time()
+    if not urls['has_domain']:
+        return {
+            'site_id': site_id, 'status': 'down', 'reachable': False,
+            'http_code': None, 'latency_ms': None, 'error': 'No Ingress host found',
+            'checked_at': time.time(),
+        }
+
     probe = _probe_http(urls['health_url'])
-    probe['latency_ms'] = probe.get('latency_ms') or int((time.time() - t0) * 1000)
     probe['site_id'] = site_id
     probe['status'] = 'up' if probe['reachable'] else 'down'
     probe['checked_at'] = time.time()