Operators vs Helm: What Platform Teams Should Use When
Operators vs Helm: What Platform Teams Should Use When
Helm and Operators solve different Kubernetes lifecycle problems. This deep dive gives platform teams a practical decision model, failure modes, migration path, and working YAML examples.
TL;DR
Platform teams should choose Helm when the job is packaging, parameterizing, installing, upgrading, and rolling back mostly static Kubernetes resources. Choose an Operator when the system needs a controller that watches live cluster state, reconciles drift, writes status, manages finalizers, or encodes domain-specific upgrade logic. The strongest platforms often use both: Helm to install controllers and default resources, Operators to run ongoing lifecycle automation. The decision boundary is not complexity alone. It is whether ownership must continue after the initial apply.
The Real Decision Is Runtime Ownership
The usual Operators vs Helm debate gets framed as "simple versus complex." That is useful for first-pass triage, but it is not precise enough for a platform team that has to support dozens of services, shared clusters, stateful dependencies, and multiple application teams.
A better question is: who owns the application after the manifest reaches the Kubernetes API server?
Helm owns packaging and release operations. It renders templates, sends manifests to Kubernetes, stores release history, supports upgrades and rollbacks, and gives teams a repeatable way to parameterize installs. Once Helm exits, Kubernetes controllers continue managing the built-in resources, but Helm itself is no longer running a domain-specific loop for your application.
An Operator owns a live control loop. Kubernetes describes controllers as loops that watch cluster state and move current state closer to desired state. The Operator pattern combines custom resources with custom controllers, which lets a team encode application-specific operational knowledge directly into the Kubernetes API.
That distinction matters more than chart size. A 2,000-line Helm chart can still be the right tool if the workload lifecycle is mostly "render, apply, wait, roll back." A small Operator can be the wrong tool if it only wraps static YAML with a permanent controller that nobody will maintain.
Use this boundary:
| Requirement | Prefer Helm | Prefer Operator |
|---|---|---|
| Repeatable install with environment-specific values | Yes | Sometimes |
| Release history and manual rollback | Yes | Usually no |
| Continuous drift correction beyond built-in Kubernetes controllers | No | Yes |
| Custom status conditions exposed through the Kubernetes API | No | Yes |
| External resource cleanup before delete | Hook at best | Yes, with finalizers |
| Stateful upgrade sequencing | Limited | Yes |
| CRD schema ownership and version conversion | Usually separate | Yes |
| Low runtime footprint | Yes | No |
| Domain-specific automation | No | Yes |
The practical answer for most platform teams is not "Operators replace Helm." It is "Helm installs things; Operators operate things."
What Helm Actually Owns
Helm is a client-side release tool for Kubernetes applications. A chart defines templates, default values, dependencies, and metadata. helm install or helm upgrade renders those templates with a values set, then submits the generated Kubernetes objects. The Helm docs describe upgrades as least-invasive changes and rollbacks as a way to return a release to an earlier revision using release history.
That makes Helm excellent for platform contracts where the desired shape is declarative and predictable:
# charts/api-gateway/Chart.yaml
apiVersion: v2
name: api-gateway
description: Shared ingress gateway for application teams
type: application
version: 1.8.0
appVersion: "2.4.1"
# charts/api-gateway/values.yaml
replicaCount: 3
image:
repository: ghcr.io/example/api-gateway
tag: "2.4.1"
service:
type: ClusterIP
port: 8080
podDisruptionBudget:
enabled: true
minAvailable: 2
resources:
requests:
cpu: 250m
memory: 256Mi
limits:
memory: 512Mi
helm upgrade --install api-gateway ./charts/api-gateway \
--namespace platform-system \
--create-namespace \
--values environments/prod/api-gateway.yaml \
--wait \
--timeout 10m
This model is easy to audit. You can diff rendered manifests, sign chart artifacts, pin chart versions, and standardize values files per environment. GitOps tools can run Helm template rendering as part of the desired state pipeline. Application teams can consume a common chart while platform engineers keep a smaller number of chart APIs stable.
Helm is also intentionally bounded. Hooks can run Jobs at lifecycle points such as pre-install, post-install, pre-upgrade, post-upgrade, pre-delete, and pre-rollback, according to the Helm chart hooks documentation. Hooks are useful for migrations, smoke tests, or one-time coordination. They are not a replacement for a controller.
apiVersion: batch/v1
kind: Job
metadata:
name: api-gateway-schema-check
annotations:
"helm.sh/hook": pre-upgrade
"helm.sh/hook-weight": "-10"
"helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
spec:
template:
spec:
restartPolicy: Never
containers:
- name: check
image: ghcr.io/example/schema-check:1.2.0
args:
- "--target-version=2.4.1"
The failure mode is that hooks are tied to Helm operations, not to ongoing cluster reality. If a cloud load balancer drifts, a backup schedule falls behind, a replica gets stuck during leader election, or a shard needs rebalancing after a node failure, Helm will not wake up and fix it. Helm can only act when something invokes Helm.
That is the lifecycle trade-off. Helm gives platform teams strong packaging, release, and rollback mechanics with low operational overhead. It does not give them a domain-specific runtime brain.
What an Operator Actually Owns
An Operator starts by extending the Kubernetes API. A CustomResourceDefinition creates a new resource type. A custom resource stores desired state. A controller watches that desired state and reconciles it against the real cluster and, often, external systems.
The Kubernetes custom resources documentation draws a clear line: a custom resource alone stores structured data, but a custom resource plus a custom controller creates a true declarative API. That is the reason Operators exist.
Here is a deliberately small CRD for a managed Tempo-like tracing stack:
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: tracingstacks.platform.example.com
spec:
group: platform.example.com
scope: Namespaced
names:
plural: tracingstacks
singular: tracingstack
kind: TracingStack
shortNames:
- tstack
versions:
- name: v1alpha1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
required: ["spec"]
properties:
spec:
type: object
required: ["storage", "ingestion"]
properties:
storage:
type: object
required: ["bucketName", "retentionDays"]
properties:
bucketName:
type: string
retentionDays:
type: integer
minimum: 1
maximum: 365
ingestion:
type: object
properties:
replicas:
type: integer
minimum: 1
maximum: 20
status:
type: object
properties:
observedGeneration:
type: integer
phase:
type: string
conditions:
type: array
items:
type: object
required: ["type", "status"]
properties:
type:
type: string
status:
type: string
reason:
type: string
message:
type: string
subresources:
status: {}
Now application teams consume a platform API instead of editing 900 lines of chart values:
apiVersion: platform.example.com/v1alpha1
kind: TracingStack
metadata:
name: checkout-traces
namespace: checkout
spec:
storage:
bucketName: checkout-prod-traces
retentionDays: 30
ingestion:
replicas: 3
The controller can now run a reconciliation loop:
func (r *TracingStackReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
stack := &platformv1alpha1.TracingStack{}
if err := r.Get(ctx, req.NamespacedName, stack); err != nil {
return ctrl.Result{}, client.IgnoreNotFound(err)
}
if !stack.ObjectMeta.DeletionTimestamp.IsZero() {
return r.finalize(ctx, stack)
}
if controllerutil.AddFinalizer(stack, "platform.example.com/tracing-finalizer") {
return ctrl.Result{}, r.Update(ctx, stack)
}
desired := buildStatefulSet(stack)
if err := controllerutil.SetControllerReference(stack, desired, r.Scheme); err != nil {
return ctrl.Result{}, err
}
if err := r.applyStatefulSet(ctx, desired); err != nil {
return r.setCondition(ctx, stack, "Ready", "False", "ApplyFailed", err.Error())
}
if err := r.ensureBucketPolicy(ctx, stack); err != nil {
return r.setCondition(ctx, stack, "Ready", "False", "StoragePolicyFailed", err.Error())
}
stack.Status.ObservedGeneration = stack.Generation
stack.Status.Phase = "Ready"
return ctrl.Result{}, r.Status().Update(ctx, stack)
}
That code path shows why Operators are powerful and expensive. The controller must be idempotent. It must handle partial failure. It must update status without fighting spec writers. It must add finalizers only when it can remove them. It must avoid hot loops, permission gaps, invalid owner references, and unsafe adoption of resources it does not own.
The payoff is that the platform API can encode behavior Helm cannot express cleanly:
- Wait for an object store bucket policy before marking a tracing stack ready.
- Rotate credentials when an external secret version changes.
- Coordinate StatefulSet partitioned rollouts after checking quorum.
- Update
.status.conditionswith user-facing reasons. - Clean up cloud resources before deleting the custom resource.
- Convert custom resources between API versions during an upgrade.
The cost is a production service. A controller needs logs, metrics, alerts, RBAC review, leader election, release notes, compatibility testing, and a maintenance owner. Treating an Operator as "just YAML with Go" is how platform teams create another critical service without an SLO.
CRDs Are the Sharp Edge Between the Two
CRDs are where Helm and Operators often meet. Helm can install CRDs, but the Helm CRD best practices call out important caveats: the CRD declaration must exist before custom resources of that kind can be used, CRDs in the crds directory are installed by default, and Helm does not support every CRD lifecycle action the same way it handles ordinary templated resources.
That affects chart design. A chart that includes both a CRD and instances of that CRD can fail validation because the API server discovery state changes as the CRD is registered. The safe patterns are:
- Install CRDs in a dedicated platform chart, then install custom resources separately.
- Use Helm to install the Operator and its CRDs, but let application teams apply custom resources through GitOps.
- Keep CRDs out of tenant charts unless the platform team accepts responsibility for API compatibility.
A dedicated CRD chart can be intentionally boring:
charts/tracing-operator-crds/
Chart.yaml
crds/
platform.example.com_tracingstacks.yaml
Then the Operator chart installs runtime components:
charts/tracing-operator/
templates/
deployment.yaml
role.yaml
rolebinding.yaml
serviceaccount.yaml
metrics-service.yaml
Finally, tenants create TracingStack resources in their own repos:
kubectl apply -f teams/checkout/tracingstack.yaml
kubectl wait tracingstack checkout-traces \
--for=jsonpath='{.status.phase}'=Ready \
--namespace checkout \
--timeout 15m
This split gives platform teams a stable API boundary. CRD schema and controller code move on a platform release cadence. Tenant desired state moves on an application release cadence.
Versioning deserves the same discipline. Kubernetes documents CRD versioning and conversion so API authors can serve multiple versions, migrate stored versions, and use conversion webhooks when schema changes require custom logic. That is a real API lifecycle. If your platform team is not ready to maintain compatibility rules, defaulting to a Helm values schema may be safer than exposing a custom Kubernetes API too early.
Failure Modes Platform Teams Should Design For
Helm and Operators fail differently. This is why "which is better" is the wrong question. The safer tool is the one whose failure mode your team can observe and repair.
Helm Failure Modes
Rendered YAML is valid but semantically wrong. Kubernetes accepts the objects, but the app fails because a value generated a bad env var, resource request, or probe path. Use helm template, schema validation, chart tests, and admission policy to catch this earlier.
helm lint ./charts/api-gateway
helm template api-gateway ./charts/api-gateway \
--values environments/prod/api-gateway.yaml \
--namespace platform-system > rendered.yaml
kubectl diff --server-side -f rendered.yaml
Upgrade waits for the wrong readiness signal. Helm's --wait can wait for Kubernetes readiness objects, but it cannot know whether your application completed a domain-specific process unless you model that as a Kubernetes object or a hook Job.
Hooks become hidden control flow. A pre-upgrade Job can block a release, but if hook cleanup policies are wrong, old Jobs and logs can disappear before an incident review. Keep hook Jobs small, observable, and safe to rerun.
CRDs break release expectations. Helm CRD handling is intentionally conservative because deleting or upgrading CRDs can delete or corrupt user data if done casually. Treat CRDs as platform APIs, not as ordinary chart internals.
Release ownership conflicts with GitOps. If one system runs helm upgrade and another applies rendered manifests, both can fight over annotations, labels, and field ownership. Pick one owner for each resource path.
Operator Failure Modes
The reconcile loop is not idempotent. A controller may create duplicate cloud resources, restart workloads unnecessarily, or overwrite manual remediation. Every external side effect needs a stable key, a read-before-write path, and retry-safe behavior.
Status lies or goes stale. A custom resource with Ready=True but broken child resources is worse than no status. Use observedGeneration so readers know whether status reflects the latest spec.
status:
observedGeneration: 14
phase: Degraded
conditions:
- type: Ready
status: "False"
reason: StoragePolicyFailed
message: "Bucket policy update was denied by the cloud API"
Finalizers block deletion. Kubernetes finalizers tell the API server to keep an object in a terminating state until cleanup finishes. The finalizers documentation explains that the controller must remove its finalizer when cleanup is complete. If the controller is gone, misconfigured, or missing permissions, deletion can hang.
Owner references are invalid. Kubernetes garbage collection uses owner references to clean up dependents. Cross-namespace owner references are restricted, and invalid owner references can prevent expected cleanup. For namespaced tenant APIs that create cluster-scoped resources, design cleanup explicitly instead of assuming owner references can model every relationship.
The Operator becomes a privileged bottleneck. A controller that watches all namespaces and provisions infrastructure may need broad RBAC. That makes code review, audit logging, and release discipline more important than they are for a static chart.
A Practical Selection Model
Use Helm when the platform contract is a parameterized deployment unit:
- The desired resources are mostly built-in Kubernetes types.
- Upgrade sequencing fits Kubernetes rollout mechanics plus simple Jobs.
- Tenants need different values, not different behavior.
- Rollback to a previous rendered release is meaningful.
- The platform team wants low runtime overhead.
Use an Operator when the platform contract is an API with behavior:
- The resource has lifecycle phases that matter to users.
- The system must reconcile continuously after install.
- External infrastructure must be created, adopted, checked, or cleaned up.
- The application needs domain-specific upgrade orchestration.
- Status conditions are part of the support model.
- You need custom deletion semantics through finalizers.
Use both when the platform needs a packaged controller:
# Platform-owned cluster bootstrap
helm upgrade --install tracing-operator-crds ./charts/tracing-operator-crds \
--namespace platform-system
helm upgrade --install tracing-operator ./charts/tracing-operator \
--namespace platform-system \
--wait
# Tenant-owned desired state
kubectl apply -f teams/checkout/tracingstack.yaml
This hybrid is common because it respects the strengths of each tool. Helm distributes and upgrades the Operator deployment. The Operator reconciles application instances.
The key is not to blur responsibility. If Helm owns the Operator deployment and the Operator owns child StatefulSets, do not also expose chart values that let tenant teams mutate those child StatefulSets directly. They should mutate the custom resource spec. The controller should translate that API into owned resources.
Migration Guidance: From Chart to Operator Without a Rewrite Cliff
Most platform teams do not need to jump directly from a mature chart to a full Operator. A staged migration is safer.
Stage 1: Stabilize the Helm Contract
Before writing controller code, make the chart boring:
- Add a JSON schema for values.
- Remove global values that create surprising cross-template behavior.
- Standardize labels and annotations.
- Split CRDs from tenant resources.
- Add chart tests for rendered objects.
- Document supported upgrade paths.
# values.schema.json excerpt
{
"$schema": "https://json-schema.org/schema#",
"type": "object",
"properties": {
"replicaCount": {
"type": "integer",
"minimum": 1,
"maximum": 20
},
"storage": {
"type": "object",
"required": ["bucketName"],
"properties": {
"bucketName": { "type": "string", "minLength": 3 }
}
}
},
"required": ["replicaCount", "storage"]
}
This step often solves the original problem. If the chart becomes understandable and supportable, do not build an Operator just to look more "Kubernetes native."
Stage 2: Define the Platform API
If the chart still leaks too much implementation detail, design a CRD around user intent. A good custom resource is not a one-to-one copy of values.yaml. It hides internal layout and exposes the decisions tenants should actually own.
Bad CRD shape:
spec:
statefulSetTemplate:
spec:
template:
spec:
containers:
- name: ingester
args: ["--internal-flag=true"]
Better CRD shape:
spec:
ingestion:
replicas: 3
profile: balanced
storage:
retentionDays: 30
The second form gives the platform team room to change internal Deployments, StatefulSets, probes, sidecars, and flags without breaking every tenant.
Stage 3: Let the Operator Adopt, Then Act
The first controller should do less than you think:
- Watch the custom resource.
- Render the same child resources the Helm chart rendered.
- Set owner references where valid.
- Write status conditions.
- Avoid destructive changes until adoption is explicit.
Only after that loop is stable should you move riskier behavior into the controller: backup orchestration, partitioned upgrades, cloud cleanup, certificate rotation, shard rebalancing, or data migration.
Stage 4: Keep Helm as the Installer
Even after the Operator owns the runtime lifecycle, Helm can remain the distribution mechanism for the controller itself. That keeps operational responsibilities clean:
- Platform release: Helm upgrades controller deployment and CRDs.
- Tenant release: GitOps applies custom resources.
- Runtime automation: Operator reconciles desired state and status.
This staged path avoids the two worst migrations: a giant flag day where every tenant changes at once, and a half-migration where Helm and the Operator both mutate the same child resources.
Why Platform Teams Should Design Around Control Loops
Kubernetes is already built from controllers. Deployments reconcile ReplicaSets. Jobs reconcile Pods. Garbage collection reconciles owner and dependent relationships. When you build an Operator, you are adding another controller to that ecosystem, so it must behave like a good Kubernetes citizen.
That means the Operator should:
- Treat
.specas user-owned desired state. - Treat
.statusas controller-owned observed state. - Use conditions with stable
typeandreasonvalues. - Use finalizers only for cleanup that cannot be handled by owner references.
- Make reconcile operations idempotent and retry-safe.
- Prefer server-side ownership clarity over broad patching.
- Publish metrics for reconcile duration, errors, queue depth, and external API failures.
- Include runbooks for stuck finalizers and degraded conditions.
Helm charts need a different discipline:
- Keep chart values small and typed.
- Use helper templates for labels, names, and selectors.
- Avoid hooks unless the operation is truly tied to a release lifecycle event.
- Render manifests in CI with production-like values.
- Keep CRDs in a separate chart or a clearly owned platform layer.
- Pin chart versions and document supported upgrade paths.
- Use
helm history,helm get values, and Git history during incident review.
These are not competing disciplines. They are separate layers of a platform.
Decision Examples
Ingress controller: Use Helm. Most teams need a repeatable install, a small set of values, and upgrades that map to a Deployment or DaemonSet rollout. Use an Operator only if the controller manages custom external infrastructure or tenant-facing APIs beyond normal Kubernetes resources.
PostgreSQL cluster: Prefer an Operator. Backups, restore, failover, replica promotion, major version upgrades, PVC handling, and status conditions need a runtime control loop. Helm can install the database Operator.
Shared observability stack: Use both. Helm can install Prometheus, Grafana, Tempo, or an observability Operator. Operators are useful when each team needs a custom resource that provisions a scoped stack, writes readiness status, and handles storage or tenant lifecycle.
Internal stateless service template: Use Helm. If the service is a Deployment, Service, HPA, PodDisruptionBudget, and a few ConfigMaps, a well-designed chart or GitOps generator is usually enough.
Cloud resource provisioning from Kubernetes: Prefer an Operator or a dedicated control plane integration. Helm hooks can create resources, but they are poor at continuous drift correction and safe cleanup.
What to Measure Before Standardizing
Do not standardize on Operators vs Helm by preference. Measure operational load:
- Mean time to diagnose a failed install or reconcile.
- Number of chart values exposed to tenants.
- Number of manual post-install steps in runbooks.
- Number of incidents caused by drift after install.
- Percentage of upgrades that require custom sequencing.
- Number of stuck finalizers or orphaned resources.
- Controller reconcile error rate and latency.
- Time to roll back a bad release.
If most incidents happen during install or upgrade, improve Helm chart validation, values schemas, and release pipelines first. If most incidents happen days after install because the system needs ongoing decisions, a controller may be the missing abstraction.
Frequently Asked Questions
Q: When should a platform team use a Kubernetes Operator instead of Helm? A: Use an Operator when the system needs ongoing runtime ownership after installation. Good signals include custom status, finalizers, external resource cleanup, failover, backup orchestration, drift correction, or upgrade sequencing that depends on live application state.
Q: Can Helm and Operators be used together? A: Yes. The common pattern is Helm for installing the Operator, RBAC, metrics service, and CRDs, then custom resources for the application instances. This keeps packaging and runtime reconciliation in separate layers.
Q: Why are CRDs hard to manage in Helm charts? A: CRDs change the Kubernetes API surface, so they must be registered before custom resources can be validated. Helm supports CRDs in a chart's crds directory, but its documentation calls out caveats around dry runs, upgrades, and deletion because CRDs can hold user data.
Q: Are Operators always better for complex applications? A: No. Some complex applications still have a static Kubernetes shape and can be managed well with Helm. Operators are justified when complexity is behavioral, not just when there is a lot of YAML.
Q: What is the safest migration path from Helm to an Operator? A: First stabilize the Helm chart and values schema. Then introduce a small CRD that expresses user intent, build a controller that adopts existing resources safely, and move lifecycle automation into reconciliation only after status and ownership boundaries are reliable.
Related Internal Guides
- Environment Promotion Strategies for GitOps Pipelines
- Argo CD Auto-Sync and Health Checks
- Kubernetes Multi-Tenancy with Namespaces and Network Policies
Comments
Post a Comment