Private Amazon EKS Clusters and Ingress Patterns: A Network Design Deep Dive

Private Amazon EKS Clusters and Ingress Patterns: A Network Design Deep Dive

Private EKS clusters fail when teams treat the private endpoint as the whole design. This deep dive covers API access, VPC endpoints, internal ALB/NLB ingress, private DNS, and operational failure modes.

TL;DR

Private Amazon EKS clusters are not just EKS clusters with the public endpoint disabled. They are VPC-bound operating environments where API access, node bootstrap, image pulls, IAM credential exchange, load balancer reconciliation, DNS, and operator access must all work without assuming public internet reachability. The reliable pattern is to separate control-plane access from workload ingress, keep nodes in private subnets, choose public, restricted-public-plus-private, or private-only API endpoints deliberately, expose applications through internal ALB/NLB or connected networks, and pre-build VPC endpoints, private DNS, and break-glass access before cutover.

Generated architecture diagram for a private Amazon EKS cluster with private API access, VPC endpoints, Route 53 private DNS, and internal ALB and NLB ingress.
A private EKS design has four separate paths: operator access to the Kubernetes API, node and pod access to AWS services, private DNS resolution, and application ingress.

Private EKS Is a Network Architecture, Not a Checkbox

The most common private Amazon EKS design mistake is collapsing four different traffic paths into one phrase: "make the cluster private." That phrase hides the hard parts. The Kubernetes API endpoint, node bootstrap, controller access to AWS APIs, image pulls, pod identity, DNS, and application ingress do not become private in the same way, and they do not fail in the same way.

Amazon EKS gives you endpoint access controls for the Kubernetes API server. The AWS Load Balancer Controller gives you Kubernetes-native reconciliation for Application Load Balancers and Network Load Balancers. Route 53 private hosted zones and Resolver endpoints give you private names. VPC endpoints and PrivateLink give nodes and pods private paths to AWS service APIs. Those pieces are related, but they are not interchangeable.

The better mental model is this:

  • Control-plane access answers: "How do operators, CI jobs, nodes, and Fargate pods reach the Kubernetes API?"
  • Workload egress answers: "How do nodes, controllers, and pods call AWS APIs without public internet access?"
  • Application ingress answers: "How do clients reach services running inside the cluster?"
  • DNS answers: "Which resolver returns the private address, from which network, for which name?"

The original post correctly emphasized that private EKS clusters need deliberate ingress design. The deeper correction is that ingress is only one lane. A private-only EKS API can be perfectly configured while the AWS Load Balancer Controller cannot call Elastic Load Balancing, pods cannot call STS, nodes cannot pull images from ECR, and on-premises clients cannot resolve the private application hostname.

Key design metrics to track before production cutover:

  • 3 endpoint modes for the Kubernetes API: public only, public plus private, and private only.
  • At least 2 Availability Zones for typical ALB/NLB-backed ingress, unless you are handling a documented single-subnet edge case.
  • 10 common AWS service dependencies to review in private clusters: EKS, EKS Auth, EC2, ECR API, ECR Docker, S3, STS, Elastic Load Balancing, CloudWatch Logs, and SSM.
  • 4 failure domains to test independently: API access, image pull, AWS API calls from controllers and pods, and DNS resolution from every client network.

AWS documents the EKS cluster endpoint modes, private cluster requirements, and private subnet guidance. Treat those documents as separate checklists, not as one generic "private cluster" feature.

1. Decide Who Can Reach the Kubernetes API

The EKS Kubernetes API endpoint is the administrative entry point used by kubectl, controllers, CI/CD, node bootstrap paths, and managed integrations. It is not the same thing as application ingress. Disabling public endpoint access does not expose an app privately; it only changes who can speak to the control plane.

EKS supports three useful endpoint postures:

API endpoint modeWhat it meansGood fitMain risk
Public onlyThe default for new clusters. Kubernetes API requests use the public endpoint, protected by IAM, Kubernetes RBAC, and optional public CIDR restrictions.Early development, temporary migration, clusters where private connectivity is not ready.Nodes and automation can depend on NAT or internet-reachable paths. Public CIDR mistakes are high impact.
Public and privateRequests from inside the VPC use the private endpoint; approved external networks can still use the public endpoint.Transitional production posture. CI and operators move private first while emergency public access remains CIDR-restricted.Hybrid nodes and clients can resolve differently. Public and private controls are separate.
Private onlyKubernetes API traffic must originate from the VPC or a connected network.Regulated or internal-only environments with VPN, Direct Connect, Transit Gateway, SSM, or in-VPC automation already working.You can lock yourself out if RBAC, IAM, DNS, or network access is not ready before disabling public access.

An eksctl cluster config can make the endpoint choice explicit:

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: private-platform
  region: us-east-1
  version: "1.30"

vpc:
  clusterEndpoints:
    publicAccess: false
    privateAccess: true
  subnets:
    private:
      us-east-1a:
        id: subnet-0aaa1111
      us-east-1b:
        id: subnet-0bbb2222
      us-east-1c:
        id: subnet-0ccc3333

managedNodeGroups:
  - name: system-private
    privateNetworking: true
    instanceTypes: ["m7i.large"]
    desiredCapacity: 3

The equivalent AWS CLI update for an existing cluster is blunt and should be run only after testing the private path:

aws eks update-cluster-config \
  --region us-east-1 \
  --name private-platform \
  --resources-vpc-config endpointPublicAccess=false,endpointPrivateAccess=true

For a safer migration, start with private access enabled and public access restricted to known CIDRs:

aws eks update-cluster-config \
  --region us-east-1 \
  --name private-platform \
  --resources-vpc-config \
  endpointPrivateAccess=true,endpointPublicAccess=true,publicAccessCidrs=203.0.113.10/32

That posture lets in-VPC automation, nodes, and operators validate the private endpoint before the public endpoint disappears.

What Changes When Private Access Is Enabled

When private endpoint access is enabled, EKS creates and manages a Route 53 private hosted zone for the cluster endpoint and associates it with the cluster VPC. That hosted zone is not visible as a normal hosted zone in your Route 53 account. EKS also requires the VPC DNS attributes enableDnsHostnames and enableDnsSupport to be true and the DHCP options to include AmazonProvidedDNS.

Check the VPC before blaming Kubernetes:

aws ec2 describe-vpcs \
  --vpc-ids vpc-0123456789abcdef0 \
  --query 'Vpcs[0].{VpcId:VpcId,CidrBlock:CidrBlock}'

aws ec2 describe-vpc-attribute \
  --vpc-id vpc-0123456789abcdef0 \
  --attribute enableDnsSupport

aws ec2 describe-vpc-attribute \
  --vpc-id vpc-0123456789abcdef0 \
  --attribute enableDnsHostnames

If a private-only cluster cannot be reached, test DNS first from inside the VPC:

aws eks describe-cluster \
  --region us-east-1 \
  --name private-platform \
  --query 'cluster.endpoint' \
  --output text

dig +short EXAMPLE.gr7.us-east-1.eks.amazonaws.com
curl -vk https://EXAMPLE.gr7.us-east-1.eks.amazonaws.com/readyz

A DNS answer that does not resolve to private addresses, a timeout to port 443, and an RBAC denial are three different failures. Fix them in that order: name resolution, network reachability, then identity and Kubernetes authorization.

2. Keep Nodes Private, But Do Not Starve Them

Private worker nodes are the normal EKS production shape: nodes run in private subnets, application load balancers live in the appropriate public or private subnets, and outbound dependency access is explicitly designed. For private-only clusters, the stricter version is that nodes and pods have no direct outbound internet path and must use VPC endpoints, internal mirrors, or controlled egress.

The trap is assuming "private subnets" means "nothing needs outbound connectivity." Nodes still need to:

  • register with the Kubernetes API
  • pull images from a registry
  • call EC2 APIs through the VPC CNI and controllers
  • exchange credentials through STS or EKS Auth, depending on IAM model
  • send logs and metrics if observability is expected
  • let controllers create, tag, and update AWS resources

AWS's private cluster guidance calls out that nodes need private endpoint access to register and that pod identity and IRSA have separate AWS API requirements. For self-managed nodes without internet access, bootstrap configuration may need the cluster API endpoint and certificate authority passed explicitly so the bootstrap path does not have to call the EKS API from inside the isolated VPC.

For Amazon Linux 2023-style node configuration, the shape is:

---
apiVersion: node.eks.aws/v1alpha1
kind: NodeConfig
spec:
  cluster:
    name: private-platform
    apiServerEndpoint: https://EXAMPLE.gr7.us-east-1.eks.amazonaws.com
    certificateAuthority: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0t...
    cidr: 10.100.0.0/16

Do not copy Outposts-only bootstrap flags into a standard regional EKS design. Flags such as --enable-local-outpost and --cluster-id belong to EKS local clusters on AWS Outposts, not to ordinary private EKS clusters running in a Region. For standard private EKS, the decision is usually managed node group versus self-managed node bootstrap, private endpoint reachability, DNS, and AWS API access.

Baseline VPC Endpoints for Private EKS

The exact endpoint set depends on what runs in the cluster, but the baseline is predictable:

REGION=us-east-1
VPC_ID=vpc-0123456789abcdef0
SUBNET_IDS="subnet-0aaa1111 subnet-0bbb2222 subnet-0ccc3333"
ENDPOINT_SG=sg-0123456789abcdef0

for service in \
  ec2 \
  ecr.api \
  ecr.dkr \
  eks \
  eks-auth \
  elasticloadbalancing \
  logs \
  ssm \
  ssmmessages \
  ec2messages \
  sts
do
  aws ec2 create-vpc-endpoint \
    --region "$REGION" \
    --vpc-id "$VPC_ID" \
    --vpc-endpoint-type Interface \
    --service-name "com.amazonaws.${REGION}.${service}" \
    --subnet-ids $SUBNET_IDS \
    --security-group-ids "$ENDPOINT_SG" \
    --private-dns-enabled
done

aws ec2 create-vpc-endpoint \
  --region "$REGION" \
  --vpc-id "$VPC_ID" \
  --vpc-endpoint-type Gateway \
  --service-name "com.amazonaws.${REGION}.s3" \
  --route-table-ids rtb-0aaa1111 rtb-0bbb2222

Security groups on interface endpoints matter. If endpoint ENIs do not allow port 443 from node and pod CIDRs, private DNS will resolve cleanly and the connection will still fail. That is one of the more misleading private-cluster failure modes because the application sees an HTTPS timeout, not a clean AWS API error.

STS Must Be Regional

IRSA and many AWS SDK credential flows call AWS STS. In private VPC designs, create the STS interface endpoint and make workloads use regional STS endpoints. Older SDK defaults or explicit sts.amazonaws.com calls can bypass the regional endpoint strategy.

For Kubernetes workloads, make the regional behavior explicit:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payments-api
spec:
  template:
    spec:
      serviceAccountName: payments-api
      containers:
        - name: app
          image: 111122223333.dkr.ecr.us-east-1.amazonaws.com/payments-api:2026-06-13
          env:
            - name: AWS_REGION
              value: us-east-1
            - name: AWS_STS_REGIONAL_ENDPOINTS
              value: regional

Do the same check for controllers, not only application pods. The AWS Load Balancer Controller needs Elastic Load Balancing and EC2 API access. ExternalDNS may need Route 53. Cluster Autoscaler or Karpenter needs its own service dependencies. Observability agents may need CloudWatch Logs, X-Ray, OTLP collectors, or vendor-private connectivity.

3. Design Operator Access Before You Disable Public Access

A private-only cluster is only operable if humans and automation can reach it privately. There are four common patterns.

PatternHow it worksStrengthWatch out for
VPN or Direct ConnectEngineers or CI reach the VPC through private routed connectivity, often through Transit Gateway.Natural for enterprises with existing network controls.DNS forwarding and route propagation must be tested from every source network.
SSM Session ManagerOperators start a session to an EC2 instance in the VPC and run kubectl there.Avoids inbound SSH from the internet and works well for break-glass access.Requires SSM, SSMMessages, EC2Messages, IAM, and endpoint security groups to work.
Bastion hostA hardened EC2 instance in a controlled subnet runs kubectl or acts as a jump point.Simple to understand and easy to isolate.SSH exposure, patching, key management, and audit logs must be handled deliberately.
In-VPC automation runnerCI/CD agents run in private subnets and deploy directly.Best long-term production path.You need a separate plan for emergency human access when CI is down.

A minimal SSM-oriented break-glass host can be deployed without public SSH:

Resources:
  EksOpsInstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      Roles:
        - !Ref EksOpsRole

  EksOpsHost:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: ami-0123456789abcdef0
      InstanceType: t3.small
      IamInstanceProfile: !Ref EksOpsInstanceProfile
      SubnetId: subnet-0aaa1111
      SecurityGroupIds:
        - sg-ops-host
      Tags:
        - Key: Name
          Value: private-eks-ops-host

The instance role then needs IAM permission to describe the cluster and an EKS access entry or Kubernetes RBAC mapping that authorizes the required groups. The cluster security group must allow inbound TCP 443 to the private endpoint from the ops host security group or connected network CIDR.

Validate the access path with a command sequence you can run during an incident:

aws ssm start-session --target i-0123456789abcdef0

aws eks update-kubeconfig \
  --region us-east-1 \
  --name private-platform

kubectl auth can-i list pods -A
kubectl get --raw='/readyz?verbose'

If that sequence is not tested before public access is removed, the private cluster is not production-ready.

4. Choose Ingress by Protocol and Client Location

Ingress for a private EKS cluster is not automatically private. It depends on load balancer scheme, subnet selection, security groups, DNS, and the network path from clients to the load balancer.

Use this decision rule:

  • Choose an internal ALB for HTTP/HTTPS applications that need host routing, path routing, redirects, TLS termination, health checks, and pod-level targets.
  • Choose an internal NLB for TCP, TLS pass-through, gRPC over TCP where L7 ALB features are not desired, static IP patterns, PrivateLink producer services, or protocol preservation.
  • Choose no load balancer for pure in-cluster service-to-service traffic; use ClusterIP, service mesh, or Gateway API inside the cluster boundary.
  • Choose public ALB/NLB only when the application is intentionally internet-facing, even if the EKS API endpoint is private.

The AWS Load Balancer Controller provisions ALBs for Kubernetes Ingress resources and NLBs for Services of type LoadBalancer. It uses subnet tags and annotations to decide where to place load balancers and how to configure target groups. AWS best-practice guidance also notes that ELB provisioning for ingress requires subnet planning across Availability Zones.

Internal ALB for HTTP Ingress

For private application ingress, the ALB scheme must be internal and the DNS name should be private. This example routes directly to pod IPs through the AWS VPC CNI:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: platform-api
  namespace: platform
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internal
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/subnets: subnet-0aaa1111,subnet-0bbb2222,subnet-0ccc3333
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:111122223333:certificate/abcde-12345
    alb.ingress.kubernetes.io/healthcheck-path: /healthz
    alb.ingress.kubernetes.io/success-codes: "200-399"
    alb.ingress.kubernetes.io/load-balancer-attributes: routing.http.drop_invalid_header_fields.enabled=true
spec:
  rules:
    - host: api.platform.internal.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: platform-api
                port:
                  number: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: platform-api
  namespace: platform
spec:
  type: ClusterIP
  selector:
    app.kubernetes.io/name: platform-api
  ports:
    - name: http
      port: 8080
      targetPort: 8080

target-type: ip sends traffic directly to pod IPs. That avoids NodePort routing and can reduce unnecessary hops, but it depends on pod IPs being routable in the VPC through the AWS VPC CNI. target-type: instance routes to node NodePorts and can be useful for older designs, but it adds another network layer to reason about.

Internal NLB for TCP and TLS Pass-Through

For TCP services, expose a Kubernetes Service through an internal NLB:

apiVersion: v1
kind: Service
metadata:
  name: broker
  namespace: messaging
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-scheme: internal
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol: TCP
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
    service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags: Environment=prod,Owner=platform
spec:
  type: LoadBalancer
  loadBalancerClass: service.k8s.aws/nlb
  selector:
    app.kubernetes.io/name: broker
  ports:
    - name: tls
      port: 9093
      targetPort: 9093
      protocol: TCP

The current AWS Load Balancer Controller NLB docs describe internal NLBs as the secure default for recent controller versions and require an explicit annotation for internet-facing NLBs. In production, do not rely on defaults alone. Put the scheme in the manifest so review tools and humans can see the intended exposure.

Be cautious about changing load balancer type or target type on an existing Service. Controller docs warn that changing certain reconciliation annotations on an existing Service can cause misconfigured or leaked AWS resources. For material changes, create a new Service, move DNS, then delete the old Service after target health and traffic are confirmed.

5. DNS Is the Contract Between Private Networks

Private ingress is not useful if only the cluster can resolve the name. DNS needs to be designed from each client location:

  • pods inside the EKS cluster
  • EC2 instances in the same VPC
  • other VPCs connected by Transit Gateway or peering
  • on-premises networks connected through VPN or Direct Connect
  • CI/CD runners
  • emergency operator workstations

Route 53 private hosted zones answer DNS queries only for associated VPCs or hybrid networks that resolve through Route 53 Resolver inbound endpoints. This is different from public DNS, and it is also different from the EKS-managed private hosted zone used for the cluster API endpoint.

A practical pattern is to use a private hosted zone for internal application names:

aws route53 create-hosted-zone \
  --name internal.example.com \
  --vpc VPCRegion=us-east-1,VPCId=vpc-0123456789abcdef0 \
  --caller-reference private-eks-internal-20260613 \
  --hosted-zone-config Comment="Private names for EKS internal ingress",PrivateZone=true

Then map the internal ALB name to an application record:

{
  "Comment": "Alias platform API to the internal ALB",
  "Changes": [
    {
      "Action": "UPSERT",
      "ResourceRecordSet": {
        "Name": "api.platform.internal.example.com.",
        "Type": "A",
        "AliasTarget": {
          "HostedZoneId": "Z35SXDOTRQ7X7K",
          "DNSName": "internal-k8s-platform-api-123456789.us-east-1.elb.amazonaws.com.",
          "EvaluateTargetHealth": true
        }
      }
    }
  ]
}

If you automate records with ExternalDNS, scope it tightly:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: external-dns
  namespace: platform-dns
spec:
  template:
    spec:
      serviceAccountName: external-dns
      containers:
        - name: external-dns
          image: registry.k8s.io/external-dns/external-dns:v0.15.0
          args:
            - --source=ingress
            - --domain-filter=internal.example.com
            - --provider=aws
            - --registry=txt
            - --txt-owner-id=private-platform
            - --aws-zone-type=private

For hybrid DNS, do not forward private queries to the VPC CIDR plus two resolver address from on-premises DNS servers. AWS Route 53 Resolver guidance recommends inbound Resolver endpoints for on-premises or connected networks. In practice, that means your corporate DNS forwards internal.example.com to the inbound Resolver endpoint IPs, and Route 53 answers using the private hosted zone associated with the VPC.

Private DNS failure is often mistaken for ingress failure. Test the name from every place that will use it:

dig api.platform.internal.example.com
curl -vk https://api.platform.internal.example.com/healthz

kubectl run dns-test \
  --rm -it \
  --image=public.ecr.aws/amazonlinux/amazonlinux:2023 \
  --restart=Never \
  -- bash -lc 'dnf -y install bind-utils curl && dig api.platform.internal.example.com && curl -sk https://api.platform.internal.example.com/healthz'

If DNS works in a pod but fails from the corporate network, fix Resolver forwarding. If DNS works from the corporate network but curl times out, inspect routes, security groups, NACLs, and load balancer target health. If curl reaches the load balancer and returns a 503, inspect Kubernetes endpoints and target group registration.

6. Operational Failure Modes Worth Testing

Private EKS designs usually fail at integration boundaries. A test plan should include failure injection and clean diagnostic commands, not just a successful deployment.

SymptomLikely causeFast checkFix
kubectl times outNo route to private endpoint, cluster security group missing 443, or DNS not returning private IPs.dig <cluster-endpoint> and curl -vk <endpoint>/readyz from the same network.Fix VPC DNS, routes, VPN/TGW propagation, or cluster security group ingress.
Nodes never joinPrivate endpoint disabled, bootstrap cannot discover endpoint, node role unauthorized, or VPC endpoints missing.journalctl -u kubelet, node bootstrap logs, aws eks describe-cluster from the node path.Enable private endpoint access, pass bootstrap endpoint/CA where needed, fix node IAM and endpoint access.
Pods cannot assume rolesSTS endpoint missing, SDK calls global STS, trust policy or service account annotation wrong.Inspect pod env, AWS SDK logs, VPC Flow Logs to STS, aws sts get-caller-identity from a debug pod.Add STS endpoint, force regional STS, fix IRSA or Pod Identity binding.
Image pulls hangECR API, ECR Docker, or S3 endpoint path missing; endpoint security group blocks 443.kubectl describe pod, container runtime logs, VPC endpoint metrics.Add ecr.api, ecr.dkr, and S3 gateway endpoint; allow node and pod CIDRs to endpoint SG.
AWS Load Balancer Controller cannot reconcileMissing EC2 or Elastic Load Balancing endpoint, IAM policy issue, subnet tags missing.Controller logs and events on Ingress or Service.Add endpoints, correct IAM, tag subnets, verify controller flags for private clusters.
ALB exists but targets are unhealthyPod readiness path mismatch, security group rule missing, wrong target type, pod not listening.Target group health description and kubectl get endpoints.Fix health check annotations, pod readiness, backend SG rules, or target type.
Internal DNS works only in one VPCPrivate hosted zone not associated with all required VPCs or Resolver forwarding is incomplete.aws route53 list-hosted-zones-by-vpc and resolver query logs.Associate the zone, use Route 53 Profiles where appropriate, or configure inbound/outbound Resolver endpoints.
Private-only cluster cannot be administered during incidentNo tested SSM/bastion/VPN/CI path or missing RBAC for break-glass principal.Attempt the documented break-glass runbook quarterly.Build and test an emergency access path before disabling public endpoint access.

A private cluster readiness script should verify each boundary:

set -euo pipefail

CLUSTER=private-platform
REGION=us-east-1
APP_HOST=api.platform.internal.example.com

aws eks describe-cluster --region "$REGION" --name "$CLUSTER" \
  --query 'cluster.resourcesVpcConfig.{private:endpointPrivateAccess,public:endpointPublicAccess,cidrs:publicAccessCidrs}'

kubectl get nodes -o wide
kubectl -n kube-system logs deploy/aws-load-balancer-controller --tail=80
kubectl get ingress -A
kubectl get service -A --field-selector spec.type=LoadBalancer

dig +short "$APP_HOST"
curl -skI "https://${APP_HOST}/healthz"

Run that from an operator path, not only from your laptop on the public internet.

Why This Design Works

The private EKS architecture that survives production incidents is intentionally boring: private endpoint access for the control plane, private nodes, explicit AWS service endpoints, internal load balancers for internal applications, Route 53 private hosted zones for application names, and a tested operator path through VPN, Direct Connect, SSM, or in-VPC automation.

The important choice is separation. Do not use the EKS API endpoint posture to reason about application exposure. Do not use a private ALB to prove that nodes can pull images. Do not use successful kubectl get pods from a bastion to prove that on-premises clients can resolve private DNS. Each path has its own DNS, route table, security group, IAM, and service dependency story.

That separation also makes trade-offs clearer:

  • If public API access remains enabled, narrow it with CIDR allow lists and move automation to the private endpoint.
  • If the cluster is private-only, keep a tested private operator path and an RBAC-approved break-glass role.
  • If workloads have no internet egress, pre-provision the AWS service endpoints and internal image sources they need.
  • If ingress is internal, pair internal load balancer scheme with private DNS and connected-network routing.
  • If hybrid clients need private names, use Route 53 Resolver endpoints instead of ad hoc DNS forwarding to VPC resolver addresses.

The result is not merely "more secure." It is more diagnosable. When a private Amazon EKS cluster fails, you can locate the failure in one lane: Kubernetes API access, AWS service egress, application ingress, or DNS.

Implementation Checklist

Use this as the pre-cutover review for private Amazon EKS clusters:

  • Enable EKS private endpoint access and decide whether public endpoint access is disabled or CIDR-restricted.
  • Confirm VPC DNS support, DNS hostnames, and DHCP options are compatible with EKS private endpoint resolution.
  • Place nodes in private subnets across at least two Availability Zones, and tag subnets for the AWS Load Balancer Controller.
  • Create interface and gateway endpoints for the AWS services used by nodes, controllers, and workloads.
  • Configure endpoint security groups to allow HTTPS from node and pod CIDRs or security groups.
  • Force regional STS behavior for workloads that use IRSA or SDK credential calls.
  • Install the AWS Load Balancer Controller with private-cluster-aware flags where needed, and verify it can call EC2 and Elastic Load Balancing APIs.
  • Use alb.ingress.kubernetes.io/scheme: internal for private HTTP ingress and service.beta.kubernetes.io/aws-load-balancer-scheme: internal for private NLB services.
  • Create Route 53 private hosted zones for internal application names and associate them with every VPC that needs resolution.
  • Use Route 53 Resolver inbound endpoints for on-premises or hybrid DNS integration.
  • Test operator access through VPN, Direct Connect, SSM, bastion, Cloud9, CloudShell VPC environments, or private CI runners before disabling public endpoint access.
  • Document a break-glass sequence that proves DNS, TCP 443, IAM, and Kubernetes RBAC.

Frequently Asked Questions

Q: How do I access a private Amazon EKS cluster? A: Use a network path that originates inside the cluster VPC or a connected network. Common choices are VPN, Direct Connect, Transit Gateway, SSM Session Manager to an in-VPC host, a hardened bastion host, Cloud9 in the VPC, CloudShell VPC environments where supported, or CI runners deployed in private subnets.

Q: Is private-only API endpoint access always better than restricted public endpoint access? A: Not automatically. Private-only access is stronger when your private operator and automation paths are reliable, but it is risky if CI/CD, DNS, RBAC, and emergency access are not ready. A restricted public endpoint plus private access is often the safer migration state.

Q: Should private EKS applications use an internal ALB or internal NLB? A: Use an internal ALB when HTTP routing features matter: host rules, path rules, redirects, TLS termination, health checks, and direct pod IP targets. Use an internal NLB for TCP, TLS pass-through, static IP-oriented clients, PrivateLink producer services, or protocols that should not be interpreted at layer 7.

Q: What DNS setup is required for private EKS ingress? A: Use Route 53 private hosted zones for application records and associate the zones with the VPCs that need them. For on-premises or hybrid clients, forward private zones to Route 53 Resolver inbound endpoint IPs instead of forwarding directly to the VPC resolver address.

Q: Why can nodes join a private EKS cluster while workloads still fail? A: Node registration proves only that nodes can reach the Kubernetes API and are authorized to join. Workloads may still fail because ECR, S3, STS, EKS Auth, Elastic Load Balancing, CloudWatch Logs, Route 53, or third-party dependency paths are missing or blocked by endpoint security groups.

Related Internal Guides

Resources

Comments

Popular posts from this blog

Argo CD Auto-Sync and Health Checks: An Operator's Guide to Safe GitOps Reconciliation

Bootstrapping Kubernetes Clusters with Terraform and Argo CD: A Durable Two-Layer Approach

Kubernetes Multi-Tenancy with Namespaces and Network Policies: A Practical Guide for GitOps Teams