Lesson 03 — Amazon EKS Networking

Learning Objectives

By the end of this lesson, you will be able to:

Explain the Amazon EKS networking architecture
Understand how Kubernetes and Amazon VPC networking work together
Explain how the Amazon VPC CNI assigns IP addresses to Pods
Understand Pod-to-Pod, Pod-to-Service and Pod-to-external communication
Design VPCs and subnets for Amazon EKS
Explain Kubernetes Service types
Design secure ingress and egress architectures
Apply Kubernetes Network Policies
Understand Security Groups for Pods
Plan for IPv4 address consumption and IP exhaustion
Understand custom networking, prefix delegation and IPv6 options
Design enterprise Amazon EKS networking controls
Monitor and troubleshoot Amazon EKS network traffic

Why This Matters

Networking is one of the most important and complex parts of Amazon EKS.

Every application running in Kubernetes depends on networking for:

Pod communication
Service discovery
Internet access
AWS service access
Database access
Load balancing
DNS resolution
Monitoring
Security inspection

A poorly designed Amazon EKS network can result in:

Pod IP exhaustion
Unrestricted lateral movement
Exposed applications
Uncontrolled internet access
Database exposure
Failed service communication
DNS outages
Inconsistent security enforcement
Difficult incident investigation
Unexpected network costs

Application Availability

+

Security

+

Scalability

+

Performance

=

Effective Amazon EKS Networking

For a Cloud Security Engineer, Amazon EKS networking must be understood at both the AWS and Kubernetes layers.

Amazon EKS Networking Overview

Amazon EKS networking combines several technologies.

Amazon EKS Networking

├── Amazon VPC
├── VPC Subnets
├── Route Tables
├── Security Groups
├── Network ACLs
├── Elastic Network Interfaces
├── Amazon VPC CNI
├── Kubernetes Pod Networking
├── Kubernetes Services
├── Ingress Controllers
├── Network Policies
├── DNS
└── AWS Load Balancers

These components work together to deliver application connectivity.

Amazon EKS Network Layers

Amazon EKS networking can be divided into four major layers.

Layer 1 — AWS Network

VPC, Subnets, Routes, Security Groups and Network ACLs

↓

Layer 2 — Node Network

EC2 Worker Nodes and Elastic Network Interfaces

↓

Layer 3 — Kubernetes Network

Pods, Services, Network Policies and DNS

↓

Layer 4 — Application Network

Ingress, Egress, APIs, Databases and External Services

A security issue at any layer can affect the entire application.

Kubernetes Networking Requirements

Kubernetes networking is built around several fundamental expectations:

Every Pod receives an IP address.
Pods can communicate with other Pods.
Nodes can communicate with Pods.
Services provide stable access to groups of Pods.
Applications should not depend on a Pod retaining the same IP address.
Network security should be explicitly enforced.

Pods are temporary resources.

Their IP addresses may change when they are:

Restarted
Rescheduled
Recreated
Moved to another node

Applications therefore normally connect through Kubernetes Services rather than directly to Pod IP addresses.

High-Level Amazon EKS Network Architecture

Internet or Corporate Network

↓

Amazon Route 53

↓

AWS WAF

↓

Application Load Balancer

↓

Kubernetes Ingress

↓

Kubernetes Service

↓

Application Pods

↓

Internal Services or Databases

The exact architecture depends on whether the application is:

Public
Internal
Hybrid
Multi-region
Regulated
Service-to-service only

Amazon VPC

An Amazon VPC is the primary AWS network boundary for an EKS cluster.

The VPC provides:

IP address ranges
Subnets
Routing
Security Groups
Network ACLs
Internet connectivity
Private connectivity
VPC endpoints
Traffic logging

Amazon VPC

├── Public Subnets
├── Private Application Subnets
└── Restricted Data Subnets

The VPC should be designed before the EKS cluster is created.

Recommended VPC Architecture

Amazon VPC

├── Availability Zone A
│   ├── Public Subnet
│   ├── Private EKS Subnet
│   └── Restricted Data Subnet
│
├── Availability Zone B
│   ├── Public Subnet
│   ├── Private EKS Subnet
│   └── Restricted Data Subnet
│
└── Availability Zone C
    ├── Public Subnet
    ├── Private EKS Subnet
    └── Restricted Data Subnet

Production worker nodes should generally run in private subnets.

Public Subnets

Public subnets have routes to an Internet Gateway.

They may contain:

Internet-facing load balancers
NAT Gateways
Controlled ingress components

They should not normally contain:

Production worker nodes
Sensitive databases
Internal-only applications

Private Subnets

Private subnets do not provide direct inbound internet access.

They commonly contain:

Amazon EKS worker nodes
Application Pods
Internal load balancers
Platform services
Security tooling

Outbound access may be provided through:

NAT Gateways
Egress proxies
Central firewalls
VPC endpoints

Restricted Data Subnets

Restricted data subnets may contain:

Amazon RDS
Amazon Aurora
Internal data services
Sensitive application components

Access should be limited to approved application workloads.

Application Pod

↓

Approved Security Group or Network Path

↓

Database

Route Tables

Route tables determine where network traffic is sent.

Examples include:

Public Subnet

0.0.0.0/0

↓

Internet Gateway

Private Subnet

0.0.0.0/0

↓

NAT Gateway or Central Firewall

Incorrect routes can cause:

Internet exposure
Failed AWS API access
Broken application traffic
Security inspection bypass
Asymmetric routing

Network Access Control Lists

Network ACLs operate at the subnet level.

They are:

Stateless
Applied to inbound and outbound traffic
Evaluated using ordered rules

Network ACLs can provide additional subnet-level controls, but they should not be treated as the primary workload-level security mechanism.

Network ACL designs must account for:

Return traffic
Ephemeral ports
Load balancer health checks
Cross-subnet communication

Security Groups

Security Groups operate as stateful virtual firewalls.

They can be associated with:

Worker-node network interfaces
Load balancers
Pod network interfaces in supported configurations
Databases
Other AWS services

Security Groups control:

Inbound traffic
Outbound traffic
Protocols
Ports
Source and destination networks
References to other Security Groups

EKS Cluster Security Group

Amazon EKS creates or uses a cluster Security Group for communication between the managed control plane and cluster resources.

The cluster Security Group supports communication required for:

Kubernetes API connectivity
Node registration
Cluster operations
Managed service integration

Changes to cluster Security Group rules should be carefully tested.

Overly restrictive rules may prevent:

Nodes from joining the cluster
Pods from communicating correctly
Control-plane operations
Managed add-on functionality

Worker-Node Security Groups

Worker-node Security Groups commonly control:

Control-plane-to-node traffic
Node-to-node communication
Load-balancer-to-node communication
Monitoring traffic
Outbound connectivity
Administrative access where permitted

Avoid unrestricted rules such as:

Inbound:

All Traffic

Source:

0.0.0.0/0

Every rule should have a documented purpose.

Amazon VPC CNI

The Amazon VPC Container Network Interface plugin provides Pod networking for Amazon EKS.

It is commonly deployed as the aws-node DaemonSet.

Worker Node

↓

Amazon VPC CNI

↓

Elastic Network Interfaces

↓

VPC IP Addresses

↓

Pods

The plugin integrates Kubernetes Pod networking directly with the Amazon VPC.

Native VPC Pod Addressing

With the Amazon VPC CNI, Pods receive IP addresses from the VPC address space.

Amazon VPC CIDR

10.0.0.0/16

↓

Private Subnet

10.0.1.0/24

↓

Worker Node ENI

↓

Pod IP

10.0.1.25

This allows Pods to communicate with VPC resources using native routing.

Benefits of Native VPC Networking

Benefits include:

Native VPC routing
Direct Pod-to-Pod communication
Integration with AWS networking services
VPC Flow Logs visibility
Security Groups for Pods
High network performance
Easier communication with AWS services

Amazon VPC CNI Components

The Amazon VPC CNI includes components responsible for:

Managing Elastic Network Interfaces
Assigning IP addresses to Pods
Maintaining available IP pools
Configuring Pod networking
Supporting network-policy enforcement
Supporting advanced networking features

On EC2-based nodes, the CNI runs on each node.

Elastic Network Interfaces

An Elastic Network Interface is a virtual network interface attached to an EC2 instance.

An ENI can have:

A primary private IP address
Secondary private IP addresses
Security Groups
A subnet association
A MAC address

EC2 Worker Node

├── Primary ENI
│   ├── Node IP
│   ├── Pod IP
│   └── Pod IP
│
└── Additional ENI
    ├── Pod IP
    └── Pod IP

Pod IP Allocation

When a Pod is created:

Pod Scheduled

↓

kubelet Requests Pod Networking

↓

Amazon VPC CNI Allocates IP

↓

Network Namespace Configured

↓

Pod Starts

When the Pod is removed, the IP address can be returned to the available pool.

Pod Density

The number of Pods that can run on a node may depend on:

EC2 instance type
Number of supported ENIs
Number of available IP addresses
VPC CNI configuration
Prefix delegation
Subnet capacity
Operating-system limits

Pod density must be included in capacity planning.

IP Address Consumption

With VPC-native Pod networking, Pods consume addresses from VPC subnets.

Example:

30 Worker Nodes

×

40 Pods per Node

=

1,200 Potential Pod IP Addresses

Additional addresses are required for:

Worker nodes
Load balancers
Scaling events
Platform services
Failover capacity
AWS-reserved subnet addresses

IP Exhaustion

IP exhaustion occurs when a subnet no longer has enough addresses for new resources.

Symptoms may include:

Pods remaining in Pending
Pod sandbox creation failures
CNI allocation errors
Node scaling failures
Load balancer provisioning failures
Application capacity problems

Monitoring Available IP Addresses

Monitor:

Available subnet IP addresses
ENI allocation
Pod growth
Node growth
Failed CNI operations
Maximum Pod capacity
Scaling trends

IP monitoring should generate alerts before exhaustion occurs.

IP Capacity Planning

Ask:

How many worker nodes will exist?
How many Pods will run per node?
What is the maximum expected scale?
How much failover capacity is required?
Are load balancers using the same subnets?
Are subnets shared with other services?
Will additional Regions or clusters be added?
Is IPv6 feasible?

Prefix Delegation

Prefix delegation allows the Amazon VPC CNI to allocate groups of IP addresses to supported node network interfaces.

Worker Node ENI

↓

Delegated IP Prefix

↓

Multiple Pod IP Addresses

Potential benefits include:

Increased Pod density
Faster IP allocation
More efficient IP management
Reduced ENI pressure

Prefix delegation still requires sufficient subnet capacity.

Prefix Delegation Security Considerations

Before enabling prefix delegation:

Validate instance compatibility.
Test maximum Pod configuration.
Monitor subnet consumption.
Review node bootstrap settings.
Confirm CNI compatibility.
Test scaling and recovery.

Custom Networking

Custom networking allows Pod traffic to use network interfaces associated with alternate subnets.

Worker Node

Primary Subnet

↓

Node Primary ENI

Pod Network

↓

Secondary Subnet

↓

Pod ENI and Pod IP

Custom networking may be used to:

Separate node and Pod IP address ranges
Use secondary VPC CIDRs
Increase available Pod address capacity
Apply different subnet designs
Support specific routing requirements

Custom Networking Considerations

Custom networking introduces additional complexity.

Consider:

Subnet capacity
Route tables
Availability Zone alignment
Security Groups
Network troubleshooting
Hybrid connectivity
Network-policy compatibility
Operational ownership

It should be implemented only with a clear architecture and support model.

IPv4 Addressing

IPv4 remains common in enterprise EKS environments.

Challenges include:

Limited address space
Overlapping enterprise networks
Pod IP consumption
Multi-cluster growth
Hybrid connectivity conflicts

Organisations should plan address ranges across:

Accounts
Regions
VPCs
Clusters
Corporate networks
Partner networks

IPv6 for Amazon EKS

IPv6 can provide significantly more address capacity for Pods.

Dual-Stack VPC

↓

IPv6-Enabled EKS Cluster

↓

IPv6 Pod Addresses

Potential benefits include:

Reduced IPv4 exhaustion
Large address capacity
Easier cluster scaling

IPv6 Design Considerations

Evaluate:

Application compatibility
AWS service connectivity
On-premises support
Security tooling
Network Policies
Load balancers
DNS
Logging
SIEM parsing
Firewall rules
Incident-response capability

IPv6 must receive the same security attention as IPv4.

Pod-to-Pod Communication

Pods may communicate:

On the same node
Across different nodes
Across Availability Zones
Across namespaces

Pod A

↓

VPC Network

↓

Pod B

Without Network Policies or other controls, Pod communication may be broadly permitted.

Same-Node Pod Communication

Pod A

↓

Node Networking

↓

Pod B

Traffic may remain within the worker node depending on the network path and configuration.

Cross-Node Pod Communication

Pod A

↓

Worker Node A

↓

Amazon VPC

↓

Worker Node B

↓

Pod B

Native VPC Pod IP addresses allow this communication to be routed through the VPC.

Cross-AZ Pod Communication

Pods running in different Availability Zones can communicate through VPC routing.

Consider:

Latency
Inter-AZ data-transfer cost
Application architecture
Availability requirements
Traffic volume
Failure behaviour

Applications should balance resilience against unnecessary cross-zone traffic.

Kubernetes Services

Pod IP addresses are temporary.

Kubernetes Services provide stable access to groups of Pods.

Client

↓

Service

↓

Selected Pods

A Service uses labels and selectors to identify backend Pods.

Service Example

apiVersion: v1
kind: Service
metadata:
  name: payment-api
  namespace: payments

spec:
  selector:
    app: payment-api

  ports:
    - name: https
      port: 443
      targetPort: 8443

  type: ClusterIP

Service Types

Service Type	Purpose
ClusterIP	Internal cluster communication
NodePort	Exposes a port on worker nodes
LoadBalancer	Provisions or integrates with a load balancer
ExternalName	Maps a Service to an external DNS name

ClusterIP Service

ClusterIP is the default Service type.

Frontend Pod

↓

ClusterIP Service

↓

Backend Pods

ClusterIP Services are intended for internal communication.

They are not directly exposed to the internet.

NodePort Service

NodePort exposes the Service using a port on each worker node.

Client

↓

Worker Node IP and NodePort

↓

Service

↓

Pods

NodePort may be used behind load balancers, but direct external exposure should be carefully controlled.

Risks include:

Broad node exposure
Difficult firewall management
Expanded attack surface

LoadBalancer Service

A Service of type LoadBalancer can provision an AWS load balancer through the configured AWS integration.

Client

↓

AWS Load Balancer

↓

Kubernetes Service

↓

Pods

It may create:

Internet-facing load balancing
Internal load balancing

The selected scheme should be explicitly controlled.

Internal Load Balancers

Internal load balancers are suitable for:

Internal applications
Corporate services
Private APIs
Service-to-service communication
Hybrid network access

Corporate User

↓

VPN or Direct Connect

↓

Internal Load Balancer

↓

Kubernetes Service

Internet-Facing Load Balancers

Internet-facing load balancers are suitable for approved public applications.

Security controls should include:

TLS
AWS WAF
Restricted listeners
Approved certificates
Authentication
Access logging
Rate limiting
Threat monitoring

Ingress

Ingress provides HTTP and HTTPS routing to Kubernetes Services.

Client

↓

Ingress Load Balancer

↓

Ingress Rules

├── /payments → Payment Service
├── /accounts → Account Service
└── /support → Support Service

Ingress allows several Services to share one load-balancing architecture.

Ingress Resource Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: customer-platform
  namespace: production

spec:
  rules:
    - host: portal.example.com

      http:
        paths:
          - path: /payments
            pathType: Prefix

            backend:
              service:
                name: payment-api

                port:
                  number: 443

Ingress Controller

An Ingress resource requires an Ingress controller.

Common options include:

AWS Load Balancer Controller
NGINX Ingress Controller
Service-mesh ingress gateways
Enterprise API gateway integrations

The controller watches Kubernetes resources and configures the traffic-routing infrastructure.

AWS Load Balancer Controller

The AWS Load Balancer Controller can integrate Kubernetes resources with AWS load-balancing services.

It may manage:

Application Load Balancers
Network Load Balancers
Target groups
Listener rules
Security Group integration

The controller should use a dedicated workload identity with least-privilege AWS permissions.

Secure Public Ingress Architecture

Internet

↓

Amazon Route 53

↓

AWS Shield Protections

↓

AWS WAF

↓

Application Load Balancer

↓

Ingress Routing

↓

Kubernetes Service

↓

Application Pods

Ingress Security Controls

Implement:

TLS-only listeners
Approved TLS policies
Certificate lifecycle management
AWS WAF rules
Request-size limits
Rate limiting
Authentication
Restricted administrative paths
Load-balancer access logs
Security monitoring
Approved hostnames
Admission policies

Ingress Risks

Common ingress risks include:

Unapproved public exposure
Plaintext HTTP
Wildcard hostnames
Weak TLS settings
Missing WAF protection
Incorrect backend routing
Shared load-balancer risks
Unrestricted administrative endpoints
Unmonitored access logs

Egress

Egress is outbound communication from Pods.

Pods may need to access:

AWS APIs
Container registries
External APIs
Partner systems
Package repositories
Databases
Corporate services
DNS

Default Egress Risk

Without egress controls, a compromised Pod may:

Exfiltrate data
Download malware
Contact command-and-control infrastructure
Scan external systems
Abuse cloud credentials
Access unauthorised SaaS services

Enterprise Egress Architecture

Application Pod

↓

Kubernetes Egress Policy

↓

Internal Egress Proxy or Firewall

↓

NAT Gateway

↓

Approved External Destination

Egress Control Options

Use a combination of:

Kubernetes Network Policies
Security Groups
Central firewalls
Egress proxies
NAT Gateways
DNS filtering
VPC endpoints
Route controls
Service-mesh policies

No single layer provides complete egress governance.

NAT Gateways

NAT Gateways allow resources in private subnets to initiate internet connections.

Private Pod or Node

↓

Private Route Table

↓

NAT Gateway

↓

Internet Gateway

↓

Internet

Consider:

High-availability design
One NAT Gateway per Availability Zone where required
Data-processing cost
Central firewall requirements
Route consistency
Failure scenarios

Centralised Egress Inspection

Some enterprises route outbound traffic through:

AWS Network Firewall
Transit Gateway
Central egress VPC
Secure web proxy
Third-party firewall appliances

EKS VPC

↓

Transit Gateway

↓

Inspection VPC

↓

Firewall

↓

Internet

This supports central control but adds:

Routing complexity
Latency
Cost
Availability dependencies
Troubleshooting requirements

VPC Endpoints

VPC endpoints enable private connectivity to supported AWS services.

Potential services used by EKS workloads include:

Amazon ECR
Amazon S3
AWS STS
CloudWatch
AWS Secrets Manager
Systems Manager
AWS KMS

VPC Endpoint Architecture

Application Pod

↓

Private VPC Routing

↓

VPC Endpoint

↓

AWS Service

Benefits may include:

Reduced internet dependency
Improved traffic control
Reduced NAT usage
Private AWS API access
Endpoint-policy enforcement

VPC Endpoint Policies

Endpoint policies can restrict:

Which principals can use the endpoint
Which AWS resources can be accessed
Which actions are permitted

VPC endpoint policies should complement IAM policies rather than replace them.

Service Discovery

Kubernetes uses DNS-based service discovery.

Example Service:

payment-api

Within the same namespace:

payment-api

Across namespaces:

payment-api.payments

Fully qualified:

payment-api.payments.svc.cluster.local

CoreDNS

CoreDNS provides DNS resolution inside the Kubernetes cluster.

Application Pod

↓

CoreDNS

↓

Kubernetes Service Discovery

or

External DNS Resolution

CoreDNS is a critical platform component.

CoreDNS Security and Availability

Protect CoreDNS through:

Multiple replicas
Resource requests and limits
PodDisruptionBudgets
Multi-AZ placement
Restricted RBAC
Configuration governance
Monitoring
Controlled upgrades
DNS query visibility where required

DNS Security Risks

DNS may be used for:

Command-and-control communication
Data exfiltration
Domain-generation algorithms
Discovery of internal services
Access to malicious domains

Monitor for:

Unusual query volume
Long or encoded domain names
Newly registered domains
Repeated failed lookups
Unexpected external resolvers

Kubernetes Network Policies

Network Policies control traffic between Pods and network destinations.

They can control:

Ingress
Egress
Pod-to-Pod communication
Namespace communication
Selected IP ranges

Pod

↓

Network Policy

↓

Allowed or Denied Network Flow

Network Policy Requirements

A NetworkPolicy resource requires a networking implementation that supports policy enforcement.

Creating a NetworkPolicy object without compatible enforcement does not automatically secure traffic.

The platform team should verify:

CNI support
Policy-controller health
Policy coverage
Logging
Enforcement behaviour
Upgrade compatibility

Default-Allow Behaviour

In many Kubernetes environments, workloads can communicate broadly until policies isolate them.

No Network Policies

↓

Broad Pod Communication

A secure design should move toward default-deny.

Default-Deny Ingress Policy

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
  namespace: production

spec:
  podSelector: {}

  policyTypes:
    - Ingress

This isolates selected Pods from ingress traffic unless another policy explicitly permits it.

Default-Deny Egress Policy

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-egress
  namespace: production

spec:
  podSelector: {}

  policyTypes:
    - Egress

Default-Deny Ingress and Egress

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: production

spec:
  podSelector: {}

  policyTypes:
    - Ingress
    - Egress

After applying default-deny, required flows must be explicitly allowed.

Allow Frontend to API

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-frontend-to-api
  namespace: production

spec:
  podSelector:
    matchLabels:
      app: payment-api

  policyTypes:
    - Ingress

  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: frontend

      ports:
        - protocol: TCP
          port: 8443

Allow Traffic from a Namespace

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-monitoring
  namespace: production

spec:
  podSelector:
    matchLabels:
      monitoring: enabled

  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              purpose: monitoring

Namespace labels used for security decisions must be protected from unauthorised modification.

Allow DNS Egress

After enabling default-deny egress, workloads may require explicit DNS access.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns
  namespace: production

spec:
  podSelector: {}

  policyTypes:
    - Egress

  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system

          podSelector:
            matchLabels:
              k8s-app: kube-dns

      ports:
        - protocol: UDP
          port: 53

        - protocol: TCP
          port: 53

DNS labels and implementation details should be validated in the target cluster.

Network Policy Selection

Network Policies select Pods using labels.

Example:

podSelector:
  matchLabels:
    app: payment-api

Incorrect or missing labels may result in:

Unprotected workloads
Broken communication
Unexpected policy scope

Label governance is therefore a security requirement.

Network Policy Limitations

Traditional Kubernetes Network Policies may not fully address:

Application-layer identity
Domain-name-based rules
TLS inspection
Advanced Layer 7 filtering
Cross-cluster policy
Every type of node traffic
All host-network workloads
External firewall requirements

Additional controls may be required.

Network Policy Rollout Strategy

Map Required Traffic

↓

Create Policies

↓

Test in Development

↓

Monitor Denied or Failed Flows

↓

Deploy to Staging

↓

Validate Application Behaviour

↓

Enforce in Production

Avoid applying default-deny to production without understanding required dependencies.

Security Groups for Pods

Security Groups for Pods allow supported Pods to use dedicated Amazon EC2 Security Groups.

Application Pod

↓

Pod-Specific Security Group

↓

Approved AWS Resource

This can provide VPC-level access control for individual workloads.

Security Groups for Pods Use Cases

Use cases include:

Restricting database access
Isolating sensitive workloads
Controlling access to internal services
Applying application-specific VPC rules
Supporting regulated workload boundaries

Example Security Group Flow

Payment Pod Security Group

↓

TCP 5432 Allowed

↓

Payment Database Security Group

Other Pods without the approved Security Group should not receive the same access.

Network Policies and Security Groups for Pods

These controls are complementary.

Network Policies	Security Groups for Pods
Kubernetes-native	AWS VPC-native
Select Pods using labels	Associate AWS Security Groups
Control in-cluster flows	Control VPC-level flows
Managed through Kubernetes	Integrated with EC2 networking
Useful for namespace segmentation	Useful for AWS resource access

A defence-in-depth design may use both.

Security Group Policy Concept

A SecurityGroupPolicy can select Pods that require particular Security Groups.

Conceptual example:

apiVersion: vpcresources.k8s.aws/v1beta1
kind: SecurityGroupPolicy
metadata:
  name: payment-database-access
  namespace: payments

spec:
  podSelector:
    matchLabels:
      app: payment-api

  securityGroups:
    groupIds:
      - sg-example

Production identifiers should be supplied through approved deployment processes.

Security Groups for Pods Considerations

Review:

Supported compute configurations
Instance compatibility
VPC CNI configuration
Trunk and branch interface capacity
Pod startup behaviour
Network-policy interaction
Source network address translation behaviour
Operational monitoring
Failure scenarios

East-West Traffic

East-west traffic is traffic between workloads and services inside the environment.

Frontend

↓

API

↓

Internal Service

↓

Database

Threats include:

Lateral movement
Unauthorised service access
Credential abuse
Data discovery
Service impersonation

East-West Security Controls

Use:

Network Policies
Security Groups for Pods
Application authentication
Service identities
mTLS
Namespace boundaries
Dedicated node groups
Runtime monitoring

Internal traffic should not automatically be trusted.

North-South Traffic

North-south traffic enters or leaves the Kubernetes environment.

External User

↓

Ingress

↓

Application

or:

Application

↓

Egress

↓

External Service

Controls include:

Load balancers
AWS WAF
Security Groups
Firewalls
Proxies
NAT
Network Policies
TLS
Authentication

Service Mesh

A service mesh can provide additional controls such as:

Mutual TLS
Workload identity
Traffic encryption
Service-to-service authorisation
Traffic telemetry
Retry and timeout management

Service A Proxy

⇄ mTLS ⇄

Service B Proxy

A service mesh adds capability but also introduces:

Operational complexity
Certificate management
Resource overhead
Upgrade requirements
Troubleshooting challenges

It should be adopted based on clear requirements.

TLS and Encryption in Transit

Encrypt sensitive network traffic using TLS.

Encryption may be required:

From client to load balancer
From load balancer to application
Between internal services
Between Pods and databases
Across hybrid network connections
Across Regions

TLS Termination Options

TLS may terminate at:

Application Load Balancer
Ingress controller
Service-mesh gateway
Application Pod

End-to-End TLS

Client

↓

TLS

↓

Load Balancer

↓

TLS

↓

Application Pod

End-to-end TLS reduces plaintext exposure within the environment.

Hybrid Connectivity

Amazon EKS workloads may communicate with on-premises systems through:

AWS Site-to-Site VPN
AWS Direct Connect
Transit Gateway
Private WAN services
Enterprise firewalls

Amazon EKS VPC

↓

Transit Gateway

↓

Direct Connect

↓

Corporate Data Centre

Hybrid Network Challenges

Consider:

Overlapping CIDR ranges
DNS forwarding
Routing
Firewall rules
Latency
Bandwidth
Asymmetric routing
Encryption
Monitoring
Incident-response ownership

Overlapping IP Ranges

Overlapping CIDRs can prevent routing between:

VPCs
Corporate networks
Partner networks
Acquired companies
Multi-region environments

Enterprise IP address management should be centralised.

Multi-VPC Connectivity

Multiple EKS clusters may communicate through:

VPC peering
Transit Gateway
PrivateLink
Service APIs
Multi-cluster networking solutions
Service-mesh gateways

The design should avoid creating unrestricted network connectivity between every cluster.

AWS PrivateLink

AWS PrivateLink can expose services privately across VPCs and AWS accounts.

Consumer VPC

↓

Interface Endpoint

↓

PrivateLink

↓

Provider Service

Benefits include:

Private service access
Reduced broad network connectivity
Cross-account service exposure
Smaller network trust boundaries

Multi-Cluster Networking

Multi-cluster communication may be required for:

Shared services
Disaster recovery
Regional failover
Platform APIs
Central monitoring

Security requirements include:

Strong service authentication
TLS
Explicit routing
Least-privilege access
Segmented connectivity
Central monitoring

Fargate Networking

Amazon EKS Fargate Pods receive VPC networking without customer-managed worker nodes.

Design considerations include:

Pod execution roles
Subnet selection
Security Groups
Private connectivity
Load balancer integration
Logging
DNS
Network Policies
Platform-agent compatibility

EKS Auto Mode Networking

Where an organisation adopts EKS Auto Mode, AWS automates more of the cluster infrastructure lifecycle.

Cloud Security Engineers must still govern:

VPC and subnet design
Ingress exposure
Egress paths
Kubernetes Services
Network Policies
DNS security
Workload identity
Monitoring
Compliance requirements

Automation changes operational ownership but does not remove security accountability.

Network Observability

Network observability should provide visibility into:

Pod communication
Service traffic
Load balancer traffic
Internet ingress
Internet egress
DNS queries
Rejected traffic
VPC routes
Security Group decisions
Network-policy enforcement

VPC Flow Logs

VPC Flow Logs capture metadata about IP traffic.

They can support:

Traffic analysis
Incident investigation
Rejected-connection analysis
Unexpected egress detection
Security monitoring
Compliance evidence

VPC Network Interfaces

↓

VPC Flow Logs

↓

Central Log Repository

↓

SIEM

VPC Flow Log Fields

Flow records may help identify:

Source address
Destination address
Source port
Destination port
Protocol
Accepted or rejected action
Network interface
Traffic volume
Time range

They do not capture complete application payloads.

Load Balancer Logs

Load-balancer logs may provide:

Client addresses
Requested paths
Response codes
Processing times
TLS information
Backend targets
User agents

These logs should be centralised and protected.

DNS Logs

DNS logs can support:

Malware detection
Threat hunting
Data-exfiltration investigation
Dependency analysis
Application troubleshooting

DNS visibility should be integrated with security monitoring where appropriate.

Prometheus Network Metrics

Prometheus may collect:

Network throughput
Packet errors
Interface statistics
Connection counts
DNS performance
CNI metrics
Network-policy metrics
Load-balancer controller metrics

Network Monitoring Architecture

Amazon EKS

├── VPC CNI Metrics
├── VPC Flow Logs
├── Load Balancer Logs
├── DNS Logs
├── Network Policy Events
├── Application Metrics
└── Runtime Alerts

↓

CloudWatch and Prometheus

↓

Central SIEM and Grafana

↓

SOC and Platform Teams

Important Network Alerts

Create alerts for:

Subnet IP capacity below threshold
Repeated CNI allocation failures
Unexpected public load balancers
Broad Security Group rules
Network-policy controller failures
High rejected-traffic volume
Unexpected internet egress
DNS anomalies
Load-balancer health-check failures
Cross-account network changes
Unapproved route-table changes

Network Troubleshooting Workflow

Application Cannot Connect

↓

Verify DNS

↓

Verify Kubernetes Service

↓

Verify Endpoints

↓

Verify Network Policy

↓

Verify Pod Security Group

↓

Verify Node Security Group

↓

Verify Route Table

↓

Verify Network ACL

↓

Verify Firewall or Proxy

↓

Review Flow Logs

Troubleshooting should move systematically across each network layer.

Troubleshooting Pod Connectivity

Useful commands include:

kubectl get pods -A -o wide

kubectl get services -A

kubectl get endpoints -A

kubectl get endpointslices -A

kubectl get networkpolicies -A

kubectl describe pod <pod-name> \
  -n <namespace>

Verify CoreDNS

kubectl get pods \
  -n kube-system \
  -l k8s-app=kube-dns

kubectl get service \
  -n kube-system \
  kube-dns

Test DNS from a Pod

kubectl exec \
  -n production \
  <pod-name> \
  -- nslookup payment-api

Test TCP Connectivity

kubectl exec \
  -n production \
  <pod-name> \
  -- curl -v https://payment-api:443

Use troubleshooting tools only through approved images and procedures.

Avoid installing unapproved tools directly into production containers.

Ephemeral Debug Containers

Where supported and approved, ephemeral containers can assist troubleshooting without rebuilding an application image.

Security controls should restrict:

Who can create debug containers
Which images may be used
Privileged settings
Production use
Audit logging

Debug capability can introduce privilege-escalation risk.

Troubleshooting Service Connectivity

Check:

Does the Service exist?

↓

Does the selector match Pods?

↓

Are EndpointSlices populated?

↓

Is the target port correct?

↓

Is the Pod listening?

↓

Does a Network Policy allow traffic?

↓

Do AWS network controls allow traffic?

Troubleshooting Pod IP Allocation

Check:

Available subnet IP addresses
VPC CNI Pod status
CNI logs
ENI capacity
Node maximum Pod settings
IAM permissions
Prefix delegation settings
Custom networking configuration

Example:

kubectl get pods \
  -n kube-system \
  -l k8s-app=aws-node

Protecting the VPC CNI

The Amazon VPC CNI is a critical cluster component.

Protect it through:

Restricted RBAC
Dedicated workload identity
Approved version management
GitOps or managed add-on governance
Configuration monitoring
Resource monitoring
Change alerts
Controlled upgrades

A compromised networking component may affect every Pod on the node.

VPC CNI IAM Permissions

The CNI requires permissions to manage networking resources.

Use a dedicated role and least-privilege configuration supported by the chosen EKS identity approach.

Avoid unnecessarily relying on the worker-node role for all CNI operations.

Network Policy Governance

Enterprise policy should define:

Who owns Network Policies
Who approves connectivity
Required default-deny controls
How exceptions are managed
How changes are tested
How traffic is documented
How policy coverage is measured
How policy failures are investigated

Application Connectivity Register

Maintain a record such as:

Source	Destination	Port	Protocol	Business Purpose	Owner
Frontend	Payment API	443	TCP	Customer transactions	Payments Team
Payment API	Database	5432	TCP	Transaction storage	Payments Team
All application Pods	CoreDNS	53	UDP/TCP	DNS resolution	Platform Team
Monitoring	Application metrics	9090	TCP	Metrics collection	Operations

This record can be translated into Network Policies and firewall rules.

Network Policy as Code

Store Network Policies in Git.

network-policies/

├── default-deny/
├── platform-services/
├── application-flows/
├── monitoring/
├── ingress/
├── egress/
├── exceptions/
└── tests/

Benefits include:

Review
Version control
Automated testing
Audit history
Rollback
Multi-cluster consistency

Admission Policies for Networking

Admission policies can enforce requirements such as:

Every namespace must contain a default-deny policy.
Only approved namespaces may create LoadBalancer Services.
Public load balancers require approval labels.
Host networking is prohibited.
Host ports are restricted.
Ingress resources must use approved classes.
TLS must be configured.
Only approved annotations may be used.
External IP addresses are restricted.

Restrict LoadBalancer Services

Uncontrolled LoadBalancer Services can cause:

Public exposure
Security Group creation
Increased cost
Shadow infrastructure
Compliance violations

Use admission control to restrict which teams and namespaces may create them.

Restrict External IPs

Kubernetes Service configurations that reference external IPs should be reviewed carefully.

Risks include:

Traffic redirection
Unapproved exposure
Routing confusion
Security-control bypass

Host Network Risk

A Pod using:

hostNetwork: true

shares the node network namespace.

Risks include:

Reduced network isolation
Access to node-local services
Port conflicts
Bypass of some Pod-network controls
Increased node exposure

Host networking should be restricted to approved platform components.

Enterprise Network Segmentation Model

Internet Zone

↓

Public Load Balancer

↓

Ingress Zone

↓

Application Namespace

↓

Internal Service Namespace

↓

Restricted Data Zone

Controls should exist between every trust zone.

Trust Zones

Example trust zones include:

Zone	Example Resources
Public Edge	Internet-facing load balancers
Ingress	Ingress controllers
Application	Business application Pods
Platform	DNS, GitOps and monitoring
Security	Policy and runtime tools
Data	Databases and sensitive services
Management	Administrative and automation access

Zero Trust Networking

Zero Trust assumes that network location alone does not establish trust.

Apply:

Explicit authentication
Explicit authorisation
Least-privilege connectivity
Encryption
Continuous monitoring
Workload identity
Segmentation

Internal Network

Does Not Automatically Mean

Trusted Network

Enterprise Amazon EKS Network Architecture

Corporate Users and Internet Clients

↓

Route 53

↓

AWS WAF

↓

Public or Internal Load Balancer

↓

Approved Ingress Controller

↓

Kubernetes Services

↓

Application Pods

↓

Network Policies

↓

Security Groups for Pods

↓

Private AWS Services Through VPC Endpoints

or

↓

Controlled Egress Firewall and NAT

↓

Approved External Services

Common Amazon EKS Networking Risks

Risk	Potential Impact	Primary Controls
Pod IP exhaustion	Scaling failure	Capacity planning and monitoring
Public worker nodes	Increased exposure	Private subnets
Open Security Groups	Unauthorised access	Least-privilege rules
Missing Network Policies	Lateral movement	Default-deny policies
Unrestricted egress	Data exfiltration	Egress controls
Public API endpoint	Control-plane exposure	Private or restricted endpoint
Unapproved load balancer	Public application exposure	Admission policy
DNS compromise	Traffic redirection	CoreDNS protection
Broad node IAM role	AWS privilege abuse	Pod-level identity
CNI failure	Pod networking outage	Monitoring and HA
Overlapping CIDRs	Connectivity failure	Central IP management
Missing flow logs	Limited investigation	VPC Flow Logs

Common Design Mistakes

Small Subnets

Problem: Pod growth consumes every available IP address.

Response:

Estimate maximum node and Pod capacity.
Reserve growth capacity.
Consider secondary CIDRs, prefix delegation or IPv6.

Public Worker Nodes

Problem: Worker nodes have unnecessary direct internet exposure.

Response:

Run nodes in private subnets.

No Default-Deny Policy

Problem: Compromised Pods can move laterally.

Response:

Implement default-deny and explicitly allow required traffic.

Unrestricted Egress

Problem: Workloads can access any internet destination.

Response:

Use Network Policies, firewalls, proxies, DNS controls and VPC endpoints.

One Shared Node Security Group

Problem: Every Pod effectively receives the same broad VPC access.

Response:

Use dedicated node groups or Security Groups for Pods where justified.

Uncontrolled LoadBalancer Services

Problem: Teams create publicly reachable infrastructure without review.

Response:

Apply RBAC and admission policies.

No DNS Monitoring

Problem: Malicious DNS activity remains undetected.

Response:

Collect and analyse DNS telemetry.

Missing VPC Flow Logs

Problem: Network investigations lack traffic metadata.

Response:

Enable centralised flow logging.

Direct Production Changes

Problem: Network configuration drifts from the approved design.

Response:

Use Infrastructure as Code and GitOps.

Enterprise Implementation Strategy

Phase 1 — Discover Requirements

Identify applications.
Document traffic flows.
Classify data.
Identify public and private services.
Define hybrid connectivity.
Estimate scale.
Identify compliance requirements.

Phase 2 — Design the VPC

Select non-overlapping CIDRs.
Use multiple Availability Zones.
Define public and private subnets.
Plan restricted data subnets.
Configure route tables.
Plan NAT and egress inspection.
Define VPC endpoints.

Phase 3 — Plan Pod Addressing

Estimate node count.
Estimate Pods per node.
Reserve failover capacity.
Monitor subnet availability.
Evaluate prefix delegation.
Evaluate secondary CIDRs.
Evaluate IPv6.

Phase 4 — Secure Control-Plane Connectivity

Select private or restricted API access.
Define administrative network paths.
Configure Security Groups.
Integrate identity federation.
Monitor API access.

Phase 5 — Design Ingress

Classify public and internal applications.
Select approved ingress controllers.
Configure AWS WAF where required.
Require TLS.
Restrict load-balancer creation.
Enable access logging.

Phase 6 — Design Egress

Document approved destinations.
Deploy VPC endpoints.
Configure NAT or firewalls.
Implement egress Network Policies.
Monitor DNS and outbound traffic.
Alert on unusual destinations.

Phase 7 — Implement Segmentation

Deploy default-deny Network Policies.
Allow required application flows.
Separate trust zones.
Use Security Groups for Pods where appropriate.
Restrict host networking.
Protect namespace labels.

Phase 8 — Implement Observability

Enable VPC Flow Logs.
Enable load-balancer logging.
Monitor CoreDNS.
Collect CNI metrics.
Monitor policy enforcement.
Integrate events with the SIEM.

Phase 9 — Test the Architecture

Test:

Pod-to-Pod communication
Service discovery
Ingress
Egress
DNS
Database connectivity
Network-policy enforcement
Availability Zone failure
NAT failure
Subnet capacity
Load-balancer health

Phase 10 — Govern Continuously

Review network rules.
Detect public exposure.
Track IP capacity.
Review Network Policy coverage.
Review LoadBalancer Services.
Monitor route changes.
Review egress destinations.
Test incident-response procedures.

Enterprise Best Practices

As a Cloud Security Engineer:

Design EKS networking before creating the cluster.
Use non-overlapping VPC address ranges.
Deploy worker nodes in private subnets.
Use multiple Availability Zones.
Plan Pod IP capacity for peak growth and failover.
Monitor subnet address availability.
Use prefix delegation or custom networking only after validating requirements.
Evaluate IPv6 as part of long-term address planning.
Prefer private EKS API access for sensitive environments.
Restrict Security Groups to required ports and sources.
Apply default-deny Network Policies.
Explicitly allow required application flows.
Control both ingress and egress.
Use Security Groups for Pods for suitable VPC-level isolation.
Protect CoreDNS and monitor DNS activity.
Restrict public LoadBalancer Services.
Require TLS for application traffic.
Use VPC endpoints for private AWS service access.
Centralise VPC Flow Logs and load-balancer logs.
Protect the Amazon VPC CNI and its IAM permissions.
Store network configuration and policies in Git.
Use admission control to enforce networking standards.
Regularly test network isolation and failure scenarios.

Real-World Scenario

A financial services company runs a payment-processing platform on Amazon EKS.

The environment contains:

Public customer APIs
Internal payment services
Fraud-detection workloads
Restricted databases
Central monitoring tools
Connectivity to an on-premises settlement system

During a security assessment, the organisation discovers:

Worker nodes running in public subnets
No default-deny Network Policies
Unrestricted outbound internet access
Multiple unapproved public LoadBalancer Services
Broad database Security Group rules
Subnets approaching IP exhaustion
Missing DNS monitoring
No central VPC Flow Logs

The Cloud Security and Platform teams redesign the network.

They implement:

Private worker-node subnets across three Availability Zones.
Public subnets limited to internet-facing load balancers and NAT Gateways.
Restricted data subnets for databases.
A private EKS API endpoint accessed through approved corporate connectivity.
Larger Pod subnet address ranges.
Prefix delegation after capacity and compatibility testing.
Default-deny ingress and egress Network Policies.
Explicit policies for frontend, API, monitoring, DNS and database traffic.
Security Groups for Pods for the payment API.
Database rules that trust only the payment workload Security Group.
AWS WAF and TLS for customer-facing ingress.
Admission policies restricting public LoadBalancer Services.
VPC endpoints for ECR, S3, STS, CloudWatch and Secrets Manager.
Central outbound traffic inspection.
VPC Flow Logs, load-balancer logs and DNS telemetry.
SIEM alerts for unusual egress and rejected traffic.
Automated subnet-capacity monitoring.
GitOps management for Network Policies and ingress resources.

The redesigned architecture reduces lateral movement, improves visibility, prevents uncontrolled exposure and supports future application growth.

Key Takeaways

Amazon EKS networking combines Kubernetes networking with Amazon VPC networking.
The Amazon VPC CNI assigns VPC IP addresses to Pods.
Pod growth directly affects subnet IP consumption.
Production worker nodes should generally run in private subnets.
VPC address space must be planned for nodes, Pods, load balancers and scaling.
Kubernetes Services provide stable connectivity to temporary Pods.
Ingress controls incoming application traffic.
Egress controls reduce data-exfiltration and malware risks.
Default-deny Network Policies reduce lateral movement.
Network Policies and Security Groups for Pods provide complementary controls.
VPC endpoints provide private access to supported AWS services.
CoreDNS is a critical security and availability component.
VPC Flow Logs and load-balancer logs support monitoring and incident response.
Network configuration should be automated, version-controlled and continuously reviewed.
Zero Trust requires explicit identity, authorisation, encryption and monitoring rather than trusting internal network location.

Knowledge Check

1. What is the role of the Amazon VPC CNI in an EKS cluster?

Answer: The Amazon VPC CNI manages Pod networking by creating or using network interfaces, assigning VPC IP addresses to Pods and configuring the required network connectivity on worker nodes.

2. Why can Amazon EKS clusters experience subnet IP exhaustion?

Answer: Pods using VPC-native networking consume IP addresses from VPC subnets. As the number of worker nodes and Pods grows, the available subnet address capacity may be exhausted.

3. What is the purpose of a default-deny Network Policy?

Answer: A default-deny policy isolates selected Pods by blocking ingress, egress or both until required traffic is explicitly permitted by another policy.

4. How do Network Policies and Security Groups for Pods differ?

Answer: Network Policies provide Kubernetes-native traffic controls based primarily on Pod and namespace selection, while Security Groups for Pods provide AWS VPC-level controls using Amazon EC2 Security Groups assigned to selected Pods.

5. Why should outbound traffic from Pods be controlled?

Answer: Egress controls reduce the risk of data exfiltration, malware downloads, command-and-control communication, credential abuse and access to unauthorised external services.

What’s Next?

In the next lesson, we will explore Amazon EKS Identity and Access Management, including AWS IAM authentication, EKS access entries, Kubernetes RBAC, workload identities, least privilege and enterprise access-governance patterns.

➡️ Next Lesson: Lesson 04 — Amazon EKS Identity and Access Management