Implementing Kubernetes Pod Security Standards Without Breaking Production

Implementing Kubernetes Pod Security Standards Without Breaking Production

Most Pod Security Standards rollouts fail for a simple reason: teams jump straight to restricted without measuring what their workloads actually do. This guide shows how to move from audit to enforce with namespace labels, compliant pod specs, and clear exception boundaries.

TL;DR

Kubernetes Pod Security Standards are effective only when you treat them as an incremental enforcement program rather than a single label change. The modern path is Pod Security Admission, which became stable in Kubernetes v1.25 after PodSecurityPolicy was removed. A production rollout should start with warn and audit, pin policy versions, and then enforce baseline or restricted per namespace. The practical work is in fixing pod specs: run as non-root, drop Linux capabilities, disable privilege escalation, and set seccomp explicitly so application teams can pass admission without last-minute outages.

Official Kubernetes logo used to illustrate a Pod Security Standards rollout.

Pod Security Standards Fail When They Are Treated Like a Checkbox

Kubernetes absolutely needs a repeatable way to stop unsafe pods, but Pod Security Standards are not old PodSecurityPolicy objects with new names. They are predefined security profiles enforced by Pod Security Admission, and the rollout strategy matters as much as the labels.

That distinction matters because the migration path changed years ago. The Kubernetes project deprecated PodSecurityPolicy in v1.21, removed it in v1.25, and graduated Pod Security Admission to stable in the same v1.25 release. If you are still writing PSP manifests, you are automating a feature that no longer exists in current Kubernetes releases.

For platform teams, the practical goal is straightforward:

  • use namespace labels to declare the security posture you want
  • use warn and audit before enforce
  • fix workload specs so they pass admission for the right reasons
  • keep privileged exceptions isolated to explicitly chosen namespaces

That is how you reduce blast radius without turning your next deployment into a support incident.

1. What Pod Security Standards Actually Enforce

Pod Security Standards define three profiles:

  • privileged: the least restrictive option, intended only for workloads that genuinely need elevated access
  • baseline: blocks clearly unsafe settings while remaining compatible with many existing applications
  • restricted: applies the strongest built-in hardening and is the right target for most ordinary application namespaces

Pod Security Admission applies those standards at the namespace boundary using labels. Each namespace can independently define:

  • enforce: reject violating pods
  • audit: add audit annotations for violating pods
  • warn: return user-facing warnings for violating pods and pod templates

The most important design choice is that workload resources are warned and audited early, but enforce applies to the resulting pods. That means developers can see failures coming before the ReplicaSet or Job controller starts creating blocked pods in production.

A safe namespace baseline looks like this:

apiVersion: v1
kind: Namespace
metadata:
  name: payments-prod
  labels:
    pod-security.kubernetes.io/enforce: baseline
    pod-security.kubernetes.io/enforce-version: v1.35
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/audit-version: v1.35
    pod-security.kubernetes.io/warn: restricted
    pod-security.kubernetes.io/warn-version: v1.35

Two details are easy to miss:

  • version pinning matters because policy rules evolve with Kubernetes releases
  • baseline for enforce plus restricted for warn and audit is often the cleanest first production step

That gives teams time to fix workloads while still preventing the most dangerous pod configurations immediately.

2. Roll Out PSS Like a Migration, Not a Big-Bang Security Switch

The official Kubernetes guidance recommends a multi-mode strategy for a reason. When you add or change an enforce label, the admission plugin checks existing pods in the namespace against the new policy and returns warnings. That means you can preview the operational impact before you create a deployment outage.

Start by evaluating namespaces with server-side dry run:

kubectl label --dry-run=server --overwrite ns payments-prod \
  pod-security.kubernetes.io/enforce=baseline \
  pod-security.kubernetes.io/enforce-version=v1.35 \
  pod-security.kubernetes.io/audit=restricted \
  pod-security.kubernetes.io/audit-version=v1.35 \
  pod-security.kubernetes.io/warn=restricted \
  pod-security.kubernetes.io/warn-version=v1.35

If you are early in the program, apply audit and warn to every namespace before you enforce anything:

kubectl label --overwrite ns --all \
  pod-security.kubernetes.io/audit=baseline \
  pod-security.kubernetes.io/audit-version=v1.35 \
  pod-security.kubernetes.io/warn=baseline \
  pod-security.kubernetes.io/warn-version=v1.35

That immediately exposes where your estate stands:

  • legacy images that still assume root
  • controllers that request host namespaces or extra capabilities
  • workloads that never set seccomp or explicit security context fields

The rollout sequence that usually works is:

  1. Label all namespaces with warn and audit.
  2. Move low-risk application namespaces to enforce=baseline.
  3. Fix application specs until they pass restricted.
  4. Reserve privileged only for tightly controlled namespaces such as special infrastructure or security tooling.

The critical idea is not "make every namespace restricted today." It is "make unlabeled namespaces disappear, and make each exception intentional."

3. Most restricted Violations Are Workload Design Problems

A PSS rollout does not succeed because platform engineers add labels. It succeeds because application manifests stop depending on privileged defaults.

For Linux workloads, a restricted-friendly deployment usually needs:

  • runAsNonRoot: true
  • allowPrivilegeEscalation: false
  • capabilities.drop: ["ALL"]
  • seccompProfile.type: RuntimeDefault
  • no hostNetwork, hostPID, or hostPath shortcuts

Here is a deployment shape that aligns with the built-in policy:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payments-api
  namespace: payments-prod
spec:
  replicas: 3
  selector:
    matchLabels:
      app: payments-api
  template:
    metadata:
      labels:
        app: payments-api
    spec:
      securityContext:
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: api
          image: ghcr.io/acme/payments-api:v1.8.2
          ports:
            - containerPort: 8080
          securityContext:
            runAsNonRoot: true
            runAsUser: 10001
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop:
                - ALL
          volumeMounts:
            - name: tmp
              mountPath: /tmp
      volumes:
        - name: tmp
          emptyDir: {}

This example highlights a pattern teams often miss: readOnlyRootFilesystem: true is not enough by itself. The application also needs writable paths for scratch space, caches, or temporary files. If the image still writes under /var/tmp or /app/logs, admission may pass while the container still crashes at runtime.

That is why the hard part of PSS is image hygiene:

  • remove assumptions that PID 1 runs as root
  • stop binding to privileged ports unless the image and service design require it
  • move temporary writes to mounted volumes
  • explicitly configure seccomp instead of relying on a cluster default that may not exist everywhere

The security context docs are clear here: these fields affect process identity, filesystem behavior, Linux capabilities, and seccomp. A label-only rollout that ignores those runtime behaviors is not a security program. It is a false sense of completion.

4. Namespace Labels Are the Portable Path, but Cluster Defaults Still Matter

If you control the API server configuration, Pod Security Admission also supports cluster-wide defaults and exemptions through the admission controller configuration. That is useful when you want a consistent baseline or when certain namespaces must be exempted for valid operational reasons.

Example:

apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
  - name: PodSecurity
    configuration:
      apiVersion: pod-security.admission.config.k8s.io/v1
      kind: PodSecurityConfiguration
      defaults:
        enforce: "baseline"
        enforce-version: "latest"
        audit: "restricted"
        audit-version: "latest"
        warn: "restricted"
        warn-version: "latest"
      exemptions:
        namespaces:
          - kube-system
          - cni-system
        runtimeClasses: []
        usernames: []

This is powerful, but it needs restraint.

The admission docs explicitly warn that exemptions must be enumerated and that exempting controller identities is dangerous. If you exempt broad service accounts or controller users, you can accidentally exempt any workload those controllers create. For most organizations, the safer rule is:

  • exempt namespaces only when there is a documented platform reason
  • keep privileged infrastructure workloads separate from application namespaces
  • review exemptions during every cluster upgrade and every platform onboarding

On platforms where you do not manage the API server directly, namespace labels remain the most portable and predictable mechanism. That is usually enough for application namespaces, provided you keep the exceptions small and visible.

Why Kubernetes Built It This Way

Pod Security Standards are intentionally opinionated and intentionally limited. They do not try to express every policy your organization might want. Instead, they cover a broadly useful hardening baseline that the Kubernetes project can maintain centrally as the API evolves.

That trade-off is good engineering.

It means platform teams get:

  • a built-in control with no custom admission code
  • a clear migration away from the failed complexity of PodSecurityPolicy
  • predictable semantics across namespaces and clusters

It also means you still need other policy engines for organization-specific rules such as allowed registries, mandatory labels, or custom tenancy controls. The official PSS documentation explicitly points to third-party alternatives such as Kyverno, OPA Gatekeeper, and Kubewarden for those cases.

The right model is not "PSS versus policy engines." It is:

  • use PSS for baseline pod hardening
  • use broader policy tooling where you need custom rules

That separation keeps the default security path simple while leaving room for stricter organizational controls.

Frequently Asked Questions

Q: Is baseline enough for production namespaces? baseline is a good first enforcement step, but many ordinary application namespaces should eventually target restricted. The answer depends on whether the workload truly needs elevated Linux behavior or is merely carrying old image assumptions.

Q: Why did my deployment show warnings even though the pods were not blocked? That usually means warn or audit is set to a stricter level than enforce. This is intentional and useful because it lets teams see future violations before the namespace starts rejecting pods.

Q: Can I use PSS on Windows workloads? Yes, but some controls are OS-specific. The Pod Security Standards documentation notes that several Linux-specific restrictions changed in v1.25 to account for .spec.os.name, so mixed-OS clusters should read the current policy details carefully.

Q: What is the fastest way to clean up a namespace before moving to restricted? Inventory the workloads that still run as root, depend on privilege escalation, or omit seccomp and capability settings. Those are usually the highest-yield fixes before you tighten enforcement.

Resources

Comments

Popular posts from this blog

Bootstrapping Kubernetes Clusters with Terraform and Argo CD: A Durable Two-Layer Approach

Argo CD Auto-Sync and Health Checks: An Operator's Guide to Safe GitOps Reconciliation

Kubernetes Multi-Tenancy with Namespaces and Network Policies: A Practical Guide for GitOps Teams