Kubernetes Multi-Tenancy with Namespaces and Network Policies: A Practical Guide for GitOps Teams
Kubernetes Multi-Tenancy with Namespaces and Network Policies: A Practical Guide for GitOps Teams
Namespaces are only the first layer of tenant isolation. This guide shows how to combine RBAC, Pod Security Admission, quotas, NetworkPolicies, and Flux service-account impersonation so teams can share a cluster without sharing blast radius.
TL;DR
Kubernetes multi-tenancy works only when you treat namespaces as one control in a larger isolation stack. A shared cluster needs per-tenant namespaces, namespace-scoped RBAC, Pod Security Admission labels, ResourceQuota and LimitRange defaults, and default-deny NetworkPolicies with explicit exceptions. If you use Flux CD, you also need controller lockdown and service-account impersonation so one tenant's GitOps objects cannot reach across namespaces. The practical goal is not perfect isolation from a namespace alone, but predictable blast-radius control for teams that share a cluster.
Namespace-Per-Tenant Is a Starting Point, Not a Complete Isolation Model
The original draft was directionally right: namespaces and NetworkPolicies matter, and Flux CD needs extra care in a shared cluster. The missing part is that namespace isolation by itself is not the same thing as tenant isolation.
The official Kubernetes multi-tenancy guidance is explicit about the trade-off. You can share a cluster by separating workloads into namespaces, or you can give tenants stronger isolation with separate clusters or virtual control planes. A namespace-based model is practical when teams trust the platform team and mostly need blast-radius reduction, policy separation, and self-service. It is the wrong model when tenants need hard isolation from cluster-scoped APIs, infrastructure admins, or each other's control planes.
For most internal platform teams, the safer framing is this:
A tenant boundary in Kubernetes is a stack, not a single object.
Namespace: resource scoping and organizationRBAC: who can read or mutate namespaced resourcesPod Security Admission: what pods are allowed to runResourceQuotaandLimitRange: how much a tenant can consume by defaultNetworkPolicy: which pods can talk to which podsFlux impersonation and controller lockdown: how GitOps automation stays inside the tenant boundary
If one of those layers is missing, the cluster may still look multi-tenant in diagrams while behaving like a shared free-for-all in production.
1. Build a Namespace Boundary the API Server Can Enforce
Namespaces are still the right first move because most application resources are namespaced. They also give you a natural unit for quotas, RBAC, and policy labels. The Kubernetes docs note that namespaces are intended for environments with many users spread across multiple teams or projects, and the platform automatically applies the immutable label kubernetes.io/metadata.name to each namespace. That label is useful later in NetworkPolicy selectors because it gives you a stable namespace identity without inventing another naming convention.
What namespaces do not isolate:
- cluster-scoped resources such as CRDs, nodes, and storage classes
- overly broad
ClusterRoleBindingpermissions - network paths when the CNI is not enforcing
NetworkPolicy - unsafe pod settings if you do not apply Pod Security Admission or equivalent policy controls
Start with a tenant namespace that carries security labels from day one:
apiVersion: v1
kind: Namespace
metadata:
name: tenant-a-prod
labels:
tenant.platform.example/name: tenant-a
pod-security.kubernetes.io/enforce: restricted
pod-security.kubernetes.io/audit: restricted
pod-security.kubernetes.io/warn: restricted
Then attach quota and default sizing controls to the same namespace. This prevents the classic shared-cluster failure mode where one noisy tenant consumes all schedulable capacity because requests and limits were never standardized.
apiVersion: v1
kind: ResourceQuota
metadata:
name: tenant-a-quota
namespace: tenant-a-prod
spec:
hard:
requests.cpu: "8"
requests.memory: 16Gi
limits.cpu: "16"
limits.memory: 32Gi
pods: "100"
persistentvolumeclaims: "20"
---
apiVersion: v1
kind: LimitRange
metadata:
name: tenant-a-defaults
namespace: tenant-a-prod
spec:
limits:
- type: Container
defaultRequest:
cpu: 100m
memory: 128Mi
default:
cpu: 500m
memory: 512Mi
That is the minimum baseline for a tenant namespace. Without it, multi-tenancy becomes an honor system.
2. NetworkPolicy Works Best When You Start From Default Deny
Kubernetes NetworkPolicy rules are often explained too casually. Two details matter in practice:
- Policies are additive, not first-match.
- For traffic between two pods to be allowed, the effective egress rules on the source and ingress rules on the destination both need to allow that path.
That means tenant isolation should start with deny-by-default policy and then add only the flows a tenant actually needs. The Kubernetes multi-tenancy guide also warns against broad selectors that accidentally include many namespaces. In particular, an empty namespaceSelector: {} effectively means "all namespaces," which is the opposite of tenant isolation.
Begin with a namespace-wide deny:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
namespace: tenant-a-prod
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
Then add the minimum safe exceptions. Most tenants need at least same-namespace traffic and DNS:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-same-namespace-and-dns
namespace: tenant-a-prod
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
ingress:
- from:
- podSelector: {}
egress:
- to:
- podSelector: {}
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
If tenants must consume a shared ingress or observability namespace, make that exception explicit instead of falling back to an open namespace selector:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-from-shared-ingress
namespace: tenant-a-prod
spec:
podSelector:
matchLabels:
app.kubernetes.io/part-of: tenant-a
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: ingress-shared
Two operational caveats are easy to miss:
NetworkPolicyonly works if your CNI plugin implements it.- DNS pod labels vary by distribution, so the
k8s-app: kube-dnsselector may need to be adjusted for your cluster.
That is why production multi-tenancy needs policy tests in CI or conformance checks after every cluster upgrade. A YAML file that applied successfully is not proof that isolation still works.
3. RBAC Defines Whether a Tenant Can Stay Inside Its Namespace
A namespace is only meaningful if the identities acting inside it are namespace-scoped too. The Kubernetes RBAC guidance is clear about minimizing privileged tokens and avoiding broad bindings. In a multi-tenant cluster, the common failure is not malicious behavior. It is accidental over-permissioning through convenience roles or inherited cluster-admin access.
For tenant automation, prefer a dedicated service account with a custom Role instead of handing out a namespace-wide admin pattern by default:
apiVersion: v1
kind: ServiceAccount
metadata:
name: tenant-reconciler
namespace: tenant-a-prod
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: tenant-reconciler
namespace: tenant-a-prod
rules:
- apiGroups: [""]
resources: ["configmaps", "secrets", "services", "serviceaccounts"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["apps"]
resources: ["deployments", "statefulsets", "daemonsets", "replicasets"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["batch"]
resources: ["jobs", "cronjobs"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["networking.k8s.io"]
resources: ["ingresses", "networkpolicies"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: tenant-reconciler
namespace: tenant-a-prod
subjects:
- kind: ServiceAccount
name: tenant-reconciler
namespace: tenant-a-prod
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: tenant-reconciler
This pattern keeps the tenant's automation powerful enough to do useful work while still stopping it at the namespace boundary.
4. Flux CD Can Preserve or Break Tenant Isolation
This is the part the original post only hinted at. In a shared cluster, GitOps can either reinforce tenancy or punch straight through it.
Flux provides the primitives to keep tenant reconciliations inside a namespace, but they only help if you configure them intentionally:
Kustomization.spec.serviceAccountNamelets a reconciliation run as a chosen service account.HelmRelease.spec.serviceAccountNamedoes the same for Helm-based workloads.- Flux security best practices recommend lockdown flags such as
--no-cross-namespace-refs=trueand--no-remote-bases=truefor shared clusters. - Platform admins can also enforce a default impersonation account with
--default-service-accountso tenants do not silently reconcile with broader privileges than intended.
The tenant-facing object should reference the namespace-scoped service account you created earlier:
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: tenant-a-apps
namespace: tenant-a-prod
spec:
interval: 10m
path: ./clusters/prod/tenant-a
prune: true
serviceAccountName: tenant-reconciler
sourceRef:
kind: GitRepository
name: tenant-a-config
At the platform layer, the controller install should be patched with shared-cluster safeguards. The exact bootstrap mechanism differs by install method, but the intent should be unambiguous:
apiVersion: apps/v1
kind: Deployment
metadata:
name: kustomize-controller
namespace: flux-system
spec:
template:
spec:
containers:
- name: manager
args:
- --no-cross-namespace-refs=true
- --no-remote-bases=true
- --default-service-account=default
If you skip this layer, you have not really built multi-tenant GitOps. You have built shared automation with tenant-flavored namespaces.
5. Why This Layered Model Holds Up Better Than Namespace-Only Guidance
The reason platform teams struggle with Kubernetes multi-tenancy is that they keep asking one primitive to do every job. Namespaces are asked to provide identity isolation, network isolation, workload hardening, resource fairness, and GitOps authorization all at once. They cannot.
The layered model works because each control addresses a different escape path:
| Layer | Protects Against | Common Failure | | --- | --- | --- | | Namespace | accidental object overlap | everything lands in default | | RBAC | API misuse across tenant boundaries | broad ClusterRoleBinding | | Pod Security Admission | privileged or unsafe pod specs | tenants can request risky pod settings | | ResourceQuota and LimitRange | noisy-neighbor resource exhaustion | requests and limits omitted | | NetworkPolicy | lateral traffic between tenants | flat east-west connectivity | | Flux impersonation | GitOps reconciling with excessive privileges | controller runs effectively as cluster admin |
That is also why the Kubernetes docs present separate clusters or virtual control planes as valid alternatives. When a tenant needs cluster-scoped API control, materially different compliance policy, or strong distrust boundaries, namespace-based sharing stops being the pragmatic option.
What To Do Next
If you already run a shared cluster, the fastest hardening sequence is usually:
- Create one namespace per tenant and label each namespace with Pod Security Admission policy.
- Add
ResourceQuotaandLimitRangeobjects before opening self-service onboarding. - Enforce default-deny ingress and egress, then add explicit allow rules for DNS and approved shared services.
- Replace cluster-wide tenant automation with namespace-scoped service accounts and
RoleBinding. - Patch Flux controllers for shared-cluster lockdown and require
serviceAccountNamein tenant reconciliations. - Revisit whether high-trust and low-trust tenants should really share the same cluster at all.
That sequence is more useful than debating whether namespaces are "enough." The real question is whether the whole isolation stack is coherent.
Frequently Asked Questions
Q: Are namespaces enough for Kubernetes multi-tenancy? No. Namespaces scope namespaced resources, but shared-cluster safety also depends on RBAC, Pod Security Admission, resource controls, and network isolation. Without those layers, tenants still share too much blast radius.
Q: What is the safest default NetworkPolicy posture for tenants? Default-deny ingress and egress, then add narrow allow rules for same-namespace traffic, DNS, ingress, and approved platform services. That aligns with how Kubernetes evaluates policies and reduces accidental cross-tenant reachability.
Q: How should Flux CD be configured in a multi-tenant cluster? Run tenant reconciliations with namespace-scoped service accounts and enable controller safeguards that block cross-namespace references and remote bases where appropriate. In shared clusters, GitOps permissions should be at least as constrained as a human operator's permissions inside that namespace.
Q: When should I stop sharing a cluster and move to a stronger boundary? Move to separate clusters or virtual control planes when tenants require hard isolation, cluster-scoped customization, materially different compliance controls, or stronger distrust assumptions. Namespace-based sharing is an efficiency choice, not a universal security boundary.
Comments
Post a Comment