Lab 03 — Explore Kubernetes Architecture and Cluster Components

Mission Information

Item	Details
Lab ID	K8S-FND-LAB-03
Difficulty	Beginner to Intermediate
Estimated Time	90–120 minutes
Environment	Local Kubernetes cluster
Platform	Docker Desktop and kind
Cost	Free
Primary Role	Kubernetes Security Engineer
Module	Module 01 — Kubernetes Fundamentals for Security Engineers
Previous Lab	Lab 02 — Deploy Your First Secure Application

Mission Scenario

CloudNova Technologies has created a local Kubernetes environment and deployed its first security-hardened application.

Before the organisation expands the platform, the Cloud Security team must understand how the cluster operates internally.

The security team needs visibility into:

• The Kubernetes Control Plane
• Worker Node components
• Kubernetes system Pods
• API Server communication
• etcd storage
• Pod scheduling
• Container execution
• Service networking
• DNS resolution
• Cluster-wide resources
• Kubernetes security boundaries
• The overall cluster attack surface

You have been assigned to perform a structured architecture assessment.

Your mission is to inspect the cluster, identify its components, trace how workloads are created, analyse internal networking, and document the security importance of each architectural layer.

Learning Objectives

By completing this lab, you will be able to:

• Identify Kubernetes Control Plane components
• Inspect Worker Node components
• Understand how kind represents Kubernetes nodes
• Review static Pods and system workloads
• Locate the API Server, Scheduler, Controller Manager and etcd
• Inspect kubelet and kube-proxy behaviour
• Trace a Deployment from API request to running container
• Review Pod IP addresses and Service endpoints
• Test Kubernetes DNS resolution
• Identify Namespace-scoped and cluster-scoped resources
• Review node labels, taints and conditions
• Examine Kubernetes API resources
• Identify critical Kubernetes security assets
• Document the Kubernetes attack surface
• Produce an architecture assessment report

Architecture

You will inspect the following environment:

Windows Workstation
        │
        ▼
kubectl Client
        │
        ▼
Kubernetes API Server
        │
        ├── Authentication
        ├── Authorisation
        └── Admission Control
                │
                ▼
        Kubernetes Control Plane
        ├── kube-apiserver
        ├── etcd
        ├── kube-scheduler
        └── kube-controller-manager
                │
                ▼
        Kubernetes Worker Node
        ├── kubelet
        ├── containerd
        ├── kube-proxy
        └── Application Pods
                │
                ▼
        Services, DNS and Cluster Networking

Lab Outcomes

At the end of this lab, you should have:

• A verified Kubernetes cluster
• A dedicated architecture lab Namespace
• A running demonstration workload
• Evidence of all major Control Plane components
• Evidence of Worker Node configuration
• A traced workload deployment lifecycle
• A tested Service and DNS configuration
• A list of Namespace-scoped and cluster-scoped resources
• A Kubernetes attack-surface assessment
• An architecture assessment report

Prerequisites

Before starting, ensure that you have:

• Completed Lab 01
• Completed Lab 02
• Docker Desktop installed and running
• kubectl installed
• kind installed
• Visual Studio Code installed
• PowerShell or Git Bash available
• A running kind Kubernetes cluster
• Basic knowledge of Pods, Deployments, Services and Namespaces

Tools Used

Tool	Purpose
Docker Desktop	Runs kind Kubernetes nodes
kind	Provides the local Kubernetes cluster
kubectl	Communicates with the Kubernetes API
PowerShell	Executes Windows commands
Git Bash	Provides a Unix-style terminal
Visual Studio Code	Creates and reviews manifests
curl	Tests application and Service connectivity

Lab Safety

This lab is designed for a local learning environment.

Do not perform the inspection commands against a production cluster unless you are authorised.

Some cluster inspection commands expose:

• Internal IP addresses
• Component configuration
• Node information
• Kubernetes versions
• Runtime details
• Service Account names
• Cluster endpoints

Treat this information as security-sensitive in enterprise environments.

Task 01 — Verify the Kubernetes Cluster

Step 1 — Confirm Docker Is Running

docker version

Both the Docker client and server should respond.

Step 2 — List kind Clusters

kind get clusters

Expected cluster:

cloudnova-security-lab

Step 3 — Verify the Current Context

kubectl config current-context

Expected result:

kind-cloudnova-security-lab

Step 4 — Verify Cluster Connectivity

kubectl cluster-info

Expected output should include:

Kubernetes control plane is running at ...
CoreDNS is running at ...

Step 5 — Verify the Nodes

kubectl get nodes -o wide

Expected nodes:

cloudnova-security-lab-control-plane
cloudnova-security-lab-worker

Both nodes should show:

Ready

Task 02 — Recreate the Cluster if Required

When the cluster from the previous labs no longer exists, recreate it using the configuration from Lab 01.

Create or reuse:

kind-cluster.yaml

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4

name: cloudnova-security-lab

nodes:
  - role: control-plane

  - role: worker

PowerShell

kind create cluster --config .\kind-cluster.yaml

Git Bash

kind create cluster --config kind-cluster.yaml

Verify:

kubectl get nodes

Task 03 — Create the Lab Workspace

PowerShell

New-Item -ItemType Directory -Path C:\GoHackersCloud-Labs\kubernetes\lab-03 -Force
Set-Location C:\GoHackersCloud-Labs\kubernetes\lab-03

Git Bash

mkdir -p /c/GoHackersCloud-Labs/kubernetes/lab-03
cd /c/GoHackersCloud-Labs/kubernetes/lab-03

Verify the current location.

PowerShell

Get-Location

Git Bash

pwd

Task 04 — Create the Architecture Lab Namespace

Create:

01-namespace.yaml

Add:

apiVersion: v1
kind: Namespace
metadata:
  name: cloudnova-architecture-lab
  labels:
    environment: development
    owner: cloud-security-team
    purpose: architecture-assessment
    business-unit: cloudnova-technologies

Apply the Namespace

PowerShell

kubectl apply -f .\01-namespace.yaml

Git Bash

kubectl apply -f 01-namespace.yaml

Verify the Namespace

kubectl get namespace cloudnova-architecture-lab --show-labels

Task 05 — Identify the Kubernetes Nodes

Step 1 — List the Nodes

kubectl get nodes

Review:

• Node name
• Status
• Role
• Age
• Kubernetes version

Step 2 — Display Extended Information

kubectl get nodes -o wide

Record:

• Internal IP
• Operating system
• Kernel version
• Container runtime
• Kubernetes version

Step 3 — Display Node Labels

kubectl get nodes --show-labels

Node labels are used for:

• Scheduling
• Workload placement
• Topology awareness
• Policy decisions
• Node selection

Step 4 — Display Labels in a Readable Format

kubectl label node cloudnova-security-lab-control-plane --list

kubectl label node cloudnova-security-lab-worker --list

Identify labels related to:

• Hostname
• Operating system
• Architecture
• Node role
• Kubernetes topology

Task 06 — Inspect the Control Plane Node

Run:

kubectl describe node cloudnova-security-lab-control-plane

Review the following sections:

Roles
Labels
Annotations
Taints
Conditions
Capacity
Allocatable
System Info
Non-terminated Pods
Events

Security Review

Record the following:

• Is the node marked as a Control Plane?
• Does it have a NoSchedule taint?
• Which system Pods run on the node?
• What container runtime is installed?
• What Kubernetes version is running?
• Is the node reporting any pressure conditions?

Expected Control Plane Taint

You may observe:

node-role.kubernetes.io/control-plane:NoSchedule

This helps prevent ordinary application workloads from being scheduled on the Control Plane.

Task 07 — Inspect the Worker Node

Run:

kubectl describe node cloudnova-security-lab-worker

Review:

• Labels
• Taints
• Conditions
• Capacity
• Allocatable resources
• System information
• Running Pods
• Events

Compare the Nodes

Complete the following table:

Component	Control Plane Node	Worker Node
Node role
Taints
System Pods
Application Pods
Container runtime
Kubernetes version
CPU capacity
Memory capacity

Task 08 — Inspect the kind Node Containers

kind runs each Kubernetes node as a Docker container.

Step 1 — List Running Containers

docker ps

Expected containers:

cloudnova-security-lab-control-plane
cloudnova-security-lab-worker

Step 2 — Display Container Names Only

PowerShell

docker ps --format "{{.Names}}"

Git Bash

docker ps --format '{{.Names}}'

Step 3 — Inspect the Control Plane Container

docker inspect cloudnova-security-lab-control-plane

Review:

• Container image
• Container state
• Network settings
• Mounted filesystems
• Runtime configuration

Step 4 — Inspect the Worker Container

docker inspect cloudnova-security-lab-worker

Architecture Observation

Physical Workstation
        │
        ▼
Docker Desktop
        │
        ├── Control Plane Container
        └── Worker Node Container
                │
                ▼
        Kubernetes Pods

This architecture is suitable for local learning but differs from a production environment, where nodes are commonly virtual machines or managed cloud compute instances.

Task 09 — Inspect Kubernetes Namespaces

Run:

kubectl get namespaces

Expected Namespaces include:

default
kube-node-lease
kube-public
kube-system
local-path-storage
cloudnova-architecture-lab

Review Namespace Purpose

Namespace	Purpose
default	Default location for unspecified workloads
kube-system	Kubernetes system components
kube-public	Publicly readable cluster information
kube-node-lease	Node heartbeat and lease objects
local-path-storage	kind local storage provisioner
cloudnova-architecture-lab	Lab workloads

Security Observation

The kube-system Namespace contains security-critical components and should have highly restricted administrative access.

Task 10 — Inspect the Control Plane Components

List the system Pods.

kubectl get pods -n kube-system -o wide

Locate the following:

• kube-apiserver
• etcd
• kube-scheduler
• kube-controller-manager

PowerShell Filters

kubectl get pods -n kube-system | findstr kube-apiserver
kubectl get pods -n kube-system | findstr etcd
kubectl get pods -n kube-system | findstr kube-scheduler
kubectl get pods -n kube-system | findstr kube-controller-manager

Git Bash Filters

kubectl get pods -n kube-system | grep kube-apiserver
kubectl get pods -n kube-system | grep etcd
kubectl get pods -n kube-system | grep kube-scheduler
kubectl get pods -n kube-system | grep kube-controller-manager

Record Component Placement

Complete:

Component	Node	Security Importance
kube-apiserver
etcd
kube-scheduler
kube-controller-manager

Task 11 — Inspect the API Server

Step 1 — Locate the API Server Pod

kubectl get pods -n kube-system -l component=kube-apiserver

Step 2 — Describe the API Server Pod

Copy the Pod name and run:

kubectl describe pod <API-SERVER-POD> -n kube-system

Review:

• Image
• Command
• Arguments
• Host network settings
• Mounted volumes
• Liveness probe
• Startup probe
• Node placement

Step 3 — View the API Server Manifest

Because the API Server runs as a static Pod in kind, inspect its manifest from the Control Plane container.

docker exec cloudnova-security-lab-control-plane cat /etc/kubernetes/manifests/kube-apiserver.yaml

Review arguments related to:

• Authorization mode
• Admission plugins
• Audit configuration
• etcd connection
• Service Account configuration
• TLS certificates
• API Server port

Security Importance

The API Server is the primary interface to the Kubernetes cluster.

User or Workload
        │
        ▼
Authentication
        │
        ▼
Authorisation
        │
        ▼
Admission Control
        │
        ▼
Kubernetes API Server

A compromised API Server can provide an attacker with control over the entire cluster.

Task 12 — Inspect etcd

Step 1 — Locate the etcd Pod

kubectl get pods -n kube-system -l component=etcd

Step 2 — Describe etcd

kubectl describe pod <ETCD-POD> -n kube-system

Review:

• Image
• Command
• Listening ports
• Mounted certificates
• Data directory
• Node placement

Step 3 — View the Static Pod Manifest

docker exec cloudnova-security-lab-control-plane cat /etc/kubernetes/manifests/etcd.yaml

Security Importance

etcd stores Kubernetes cluster state.

Examples include:

• Deployments
• Pods
• Services
• Secrets
• ConfigMaps
• RBAC objects
• Namespace configuration

Kubernetes API Server
        │
        ▼
etcd
        │
        └── Cluster State

If etcd is compromised, an attacker may gain access to sensitive cluster data.

Task 13 — Inspect the Kubernetes Scheduler

Step 1 — Locate the Scheduler

kubectl get pods -n kube-system -l component=kube-scheduler

Step 2 — Describe the Scheduler Pod

kubectl describe pod <SCHEDULER-POD> -n kube-system

Step 3 — View the Scheduler Manifest

docker exec cloudnova-security-lab-control-plane cat /etc/kubernetes/manifests/kube-scheduler.yaml

Scheduler Role

The Scheduler decides where new Pods should run.

It evaluates:

• Resource requirements
• Node availability
• Node selectors
• Affinity rules
• Taints and tolerations
• Scheduling policies

Unscheduled Pod
        │
        ▼
kube-scheduler
        │
        ▼
Selected Worker Node

Task 14 — Inspect the Controller Manager

Step 1 — Locate the Controller Manager

kubectl get pods -n kube-system -l component=kube-controller-manager

Step 2 — Describe the Pod

kubectl describe pod <CONTROLLER-MANAGER-POD> -n kube-system

Step 3 — View the Static Pod Manifest

docker exec cloudnova-security-lab-control-plane cat /etc/kubernetes/manifests/kube-controller-manager.yaml

Controller Manager Role

The Controller Manager continuously compares the actual cluster state with the desired state.

Examples:

Desired Replicas: 3
Actual Replicas: 2
        │
        ▼
Controller Manager Detects Difference
        │
        ▼
Replacement Pod Created

Task 15 — Inspect Static Pod Manifests

List the Control Plane static Pod manifests.

docker exec cloudnova-security-lab-control-plane ls -la /etc/kubernetes/manifests

Expected files:

etcd.yaml
kube-apiserver.yaml
kube-controller-manager.yaml
kube-scheduler.yaml

Architecture Observation

The kubelet monitors this directory.

When a manifest appears inside it, the kubelet creates the corresponding static Pod.

/etc/kubernetes/manifests
        │
        ▼
kubelet
        │
        ▼
Static Pod

Security Risk

An attacker who can modify this directory may be able to create or alter privileged Control Plane workloads.

Task 16 — Inspect kubelet

The kubelet runs as a node process rather than as a normal Kubernetes Pod.

Step 1 — Check kubelet on the Control Plane

docker exec cloudnova-security-lab-control-plane systemctl status kubelet

Step 2 — Check kubelet on the Worker Node

docker exec cloudnova-security-lab-worker systemctl status kubelet

If the full status output is difficult to read, use:

docker exec cloudnova-security-lab-worker systemctl is-active kubelet

Expected result:

active

Step 3 — Review kubelet Process Information

docker exec cloudnova-security-lab-worker ps aux

PowerShell Filter

docker exec cloudnova-security-lab-worker ps aux | findstr kubelet

Git Bash Filter

docker exec cloudnova-security-lab-worker ps aux | grep kubelet

Security Importance

The kubelet:

• Registers the node
• Starts and stops Pods
• Communicates with the container runtime
• Mounts volumes
• Reports node and Pod status
• Executes health checks

Compromise of the kubelet may allow an attacker to control workloads on that node.

Task 17 — Inspect the Container Runtime

Modern Kubernetes commonly uses containerd.

Step 1 — Confirm the Runtime from Kubernetes

kubectl get nodes -o wide

Look for:

CONTAINER-RUNTIME

Step 2 — Inspect containerd on the Worker Node

docker exec cloudnova-security-lab-worker systemctl is-active containerd

Expected result:

active

Step 3 — Review Runtime Version

docker exec cloudnova-security-lab-worker containerd --version

Step 4 — List Kubernetes Containers

docker exec cloudnova-security-lab-worker crictl ps

This displays containers managed through the Kubernetes Container Runtime Interface.

Security Importance

The runtime creates and executes containers.

A vulnerable or misconfigured runtime may increase the risk of:

• Container escape
• Privileged workload abuse
• Host filesystem access
• Runtime socket abuse
• Node compromise

Task 18 — Inspect kube-proxy

Step 1 — Locate kube-proxy Pods

kubectl get pods -n kube-system -l k8s-app=kube-proxy -o wide

You should see one kube-proxy Pod per node.

Step 2 — Inspect the DaemonSet

kubectl get daemonset kube-proxy -n kube-system

Step 3 — Describe kube-proxy

kubectl describe daemonset kube-proxy -n kube-system

kube-proxy Role

kube-proxy helps implement Kubernetes Service networking.

Client Pod
        │
        ▼
Service ClusterIP
        │
        ▼
kube-proxy Rules
        │
        ▼
Application Pod

Task 19 — Inspect CoreDNS

Step 1 — Locate CoreDNS

kubectl get pods -n kube-system -l k8s-app=kube-dns -o wide

Step 2 — Inspect the CoreDNS Deployment

kubectl describe deployment coredns -n kube-system

Step 3 — Review the CoreDNS ConfigMap

kubectl get configmap coredns -n kube-system -o yaml

CoreDNS Role

CoreDNS provides internal Kubernetes DNS resolution.

Example:

cloudnova-architecture-service
        │
        ▼
cloudnova-architecture-service.cloudnova-architecture-lab.svc.cluster.local

Task 20 — Create the Demonstration Deployment

Create:

02-architecture-deployment.yaml

Add:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: cloudnova-architecture-web
  namespace: cloudnova-architecture-lab
  labels:
    app: cloudnova-architecture-web
spec:
  replicas: 2

  selector:
    matchLabels:
      app: cloudnova-architecture-web

  template:
    metadata:
      labels:
        app: cloudnova-architecture-web
        environment: development

    spec:
      containers:
        - name: nginx
          image: nginx:stable-alpine

          ports:
            - name: http
              containerPort: 80
              protocol: TCP

          resources:
            requests:
              cpu: 50m
              memory: 32Mi

            limits:
              cpu: 200m
              memory: 128Mi

Apply the Deployment

PowerShell

kubectl apply -f .\02-architecture-deployment.yaml

Git Bash

kubectl apply -f 02-architecture-deployment.yaml

Watch the Rollout

kubectl rollout status deployment/cloudnova-architecture-web -n cloudnova-architecture-lab

Verify the Pods

kubectl get pods -n cloudnova-architecture-lab -o wide

Task 21 — Trace the Deployment Hierarchy

Run:

kubectl get deployments -n cloudnova-architecture-lab

kubectl get replicasets -n cloudnova-architecture-lab

kubectl get pods -n cloudnova-architecture-lab

The object hierarchy is:

Deployment
        │
        ▼
ReplicaSet
        │
        ├── Pod 1
        └── Pod 2

Inspect Owner References

Copy one ReplicaSet name.

kubectl get replicaset <REPLICASET-NAME> -n cloudnova-architecture-lab -o yaml

Locate:

ownerReferences:

Copy one Pod name.

kubectl get pod <POD-NAME> -n cloudnova-architecture-lab -o yaml

Locate:

ownerReferences:

Architecture Observation

The owner references prove the relationship between:

• Deployment
• ReplicaSet
• Pod

Task 22 — Trace the Workload Creation Process

When you applied the Deployment manifest, the following process occurred:

kubectl apply
        │
        ▼
API Server
        │
        ├── Authentication
        ├── Authorisation
        └── Admission
                │
                ▼
Deployment Stored in etcd
        │
        ▼
Deployment Controller
        │
        ▼
ReplicaSet Created
        │
        ▼
Pod Objects Created
        │
        ▼
Scheduler Selects Node
        │
        ▼
kubelet Receives Pod Specification
        │
        ▼
containerd Creates Container
        │
        ▼
Pod Becomes Running

Review Creation Events

kubectl get events -n cloudnova-architecture-lab --sort-by=.metadata.creationTimestamp

Look for events such as:

Scheduled
Pulling
Pulled
Created
Started
ScalingReplicaSet

These events provide evidence of the workload lifecycle.

Task 23 — Inspect Pod Scheduling

Step 1 — Display Pod Placement

kubectl get pods -n cloudnova-architecture-lab -o wide

Identify the node running each Pod.

Step 2 — Inspect a Pod

kubectl describe pod <POD-NAME> -n cloudnova-architecture-lab

Review:

• Node
• Pod IP
• Container ID
• Image
• Conditions
• Events

Step 3 — Extract the Assigned Node

kubectl get pod <POD-NAME> -n cloudnova-architecture-lab -o jsonpath="{.spec.nodeName}"

Step 4 — Extract the Pod IP

kubectl get pod <POD-NAME> -n cloudnova-architecture-lab -o jsonpath="{.status.podIP}"

Task 24 — Inspect the Container from the Node

Identify the Worker Node running the Pod.

Then list the containers on the Worker Node:

docker exec cloudnova-security-lab-worker crictl ps

Locate the NGINX container.

Display Pod Sandbox Information

docker exec cloudnova-security-lab-worker crictl pods

This shows the Pod sandboxes managed by the runtime.

Architecture Observation

Kubernetes Pod Object
        │
        ▼
kubelet
        │
        ▼
Container Runtime
        │
        ▼
Pod Sandbox
        │
        ▼
Application Container

Task 25 — Create a Kubernetes Service

Create:

03-service.yaml

Add:

apiVersion: v1
kind: Service
metadata:
  name: cloudnova-architecture-service
  namespace: cloudnova-architecture-lab
  labels:
    app: cloudnova-architecture-web
spec:
  type: ClusterIP

  selector:
    app: cloudnova-architecture-web

  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: http

Apply the Service

PowerShell

kubectl apply -f .\03-service.yaml

Git Bash

kubectl apply -f 03-service.yaml

Verify the Service

kubectl get service -n cloudnova-architecture-lab

Inspect the Service

kubectl describe service cloudnova-architecture-service -n cloudnova-architecture-lab

Task 26 — Inspect Service Endpoints

Run:

kubectl get endpoints -n cloudnova-architecture-lab

For newer Kubernetes versions, also run:

kubectl get endpointslices -n cloudnova-architecture-lab

Compare Pod IPs and Endpoints

kubectl get pods -n cloudnova-architecture-lab -o wide

The Service endpoints should match the Pod IP addresses.

Service ClusterIP
        │
        ├── Pod IP 1
        └── Pod IP 2

Task 27 — Create a DNS Test Pod

Create:

04-dns-test-pod.yaml

Add:

apiVersion: v1
kind: Pod
metadata:
  name: dns-test-client
  namespace: cloudnova-architecture-lab
  labels:
    app: dns-test-client
spec:
  containers:
    - name: dns-client
      image: busybox:1.36
      command:
        - sleep
        - "3600"

      resources:
        requests:
          cpu: 10m
          memory: 16Mi
        limits:
          cpu: 50m
          memory: 32Mi

Apply the Pod

PowerShell

kubectl apply -f .\04-dns-test-pod.yaml

Git Bash

kubectl apply -f 04-dns-test-pod.yaml

Wait for the Pod

kubectl wait --for=condition=Ready pod/dns-test-client -n cloudnova-architecture-lab --timeout=90s

Task 28 — Test Kubernetes DNS

Step 1 — Resolve the Short Service Name

kubectl exec dns-test-client -n cloudnova-architecture-lab -- nslookup cloudnova-architecture-service

Step 2 — Resolve the Namespace-Qualified Name

kubectl exec dns-test-client -n cloudnova-architecture-lab -- nslookup cloudnova-architecture-service.cloudnova-architecture-lab

Step 3 — Resolve the Fully Qualified Domain Name

kubectl exec dns-test-client -n cloudnova-architecture-lab -- nslookup cloudnova-architecture-service.cloudnova-architecture-lab.svc.cluster.local

Expected result:

The Service name resolves to the Service ClusterIP.

DNS Naming Structure

service-name.namespace.svc.cluster.local

For this lab:

cloudnova-architecture-service.cloudnova-architecture-lab.svc.cluster.local

Task 29 — Test Service Connectivity

Run:

kubectl exec dns-test-client -n cloudnova-architecture-lab -- wget -qO- http://cloudnova-architecture-service

Expected result:

The NGINX HTML page should be returned.

Test the Fully Qualified Name

kubectl exec dns-test-client -n cloudnova-architecture-lab -- wget -qO- http://cloudnova-architecture-service.cloudnova-architecture-lab.svc.cluster.local

Network Flow

DNS Test Pod
        │
        ▼
CoreDNS
        │
        ▼
Service ClusterIP
        │
        ▼
kube-proxy Rules
        │
        ▼
NGINX Pod

Task 30 — Inspect Pod Network Configuration

Step 1 — Display Pod IP Addresses

kubectl get pods -n cloudnova-architecture-lab -o wide

Step 2 — Display Service IP Addresses

kubectl get services -n cloudnova-architecture-lab -o wide

Step 3 — Inspect Pod Network Interfaces

kubectl exec dns-test-client -n cloudnova-architecture-lab -- ip addr

When the ip command is unavailable in the image, use:

kubectl exec dns-test-client -n cloudnova-architecture-lab -- hostname -i

Step 4 — Review DNS Configuration

kubectl exec dns-test-client -n cloudnova-architecture-lab -- cat /etc/resolv.conf

Review:

• Nameserver
• Search domains
• DNS options

Expected search domains may include:

cloudnova-architecture-lab.svc.cluster.local
svc.cluster.local
cluster.local

Task 31 — Inspect Cluster Services

List all Services:

kubectl get services --all-namespaces

Identify:

• Kubernetes API Service
• CoreDNS Service
• Local storage services
• Lab Service

Inspect the Kubernetes API Service

kubectl get service kubernetes -n default

kubectl describe service kubernetes -n default

Security Importance

The internal kubernetes Service provides access to the Kubernetes API Server.

Applications should not receive API access unless required.

Task 32 — Inspect Kubernetes API Resources

Run:

kubectl api-resources

This command displays resources recognised by the Kubernetes API Server.

Review:

• Resource name
• Short name
• API group
• Namespace scope
• Kind

Display Namespace-Scoped Resources

kubectl api-resources --namespaced=true

Examples include:

• Pods
• Deployments
• Services
• ConfigMaps
• Secrets
• Roles
• RoleBindings

Display Cluster-Scoped Resources

kubectl api-resources --namespaced=false

Examples include:

• Nodes
• Namespaces
• ClusterRoles
• ClusterRoleBindings
• PersistentVolumes
• CustomResourceDefinitions

Security Observation

Cluster-scoped permissions generally create greater risk because they can affect the entire Kubernetes environment.

Task 33 — Review Kubernetes API Versions

Run:

kubectl api-versions

Review the available API groups.

Examples may include:

apps/v1
batch/v1
networking.k8s.io/v1
rbac.authorization.k8s.io/v1
storage.k8s.io/v1

Security Importance

Deprecated API versions can create:

• Upgrade risks
• Compatibility problems
• Policy gaps
• Unsupported configurations

Enterprise teams should monitor and remove deprecated API usage.

Task 34 — Review Cluster Configuration

Run:

kubectl config view

Review:

• Cluster name
• Server endpoint
• Context
• User
• Certificate configuration

Do not share complete kubeconfig files publicly.

Display the Current Context

kubectl config current-context

Display Available Contexts

kubectl config get-contexts

Security Risk

A kubeconfig may contain:

• Client certificates
• Authentication tokens
• Cluster endpoints
• User identities
• Administrative credentials

Protect kubeconfig files as sensitive assets.

Task 35 — Inspect Authentication Identity

Run:

kubectl auth whoami

When supported, this displays the current Kubernetes identity.

You may see a user associated with the local kind administrator context.

Review Current Permissions

kubectl auth can-i --list

Test Administrative Access

kubectl auth can-i create namespaces

kubectl auth can-i get secrets --all-namespaces

In a local kind cluster, the current identity commonly has broad administrative access.

Security Observation

Broad permissions are acceptable for a dedicated local lab administrator.

They would be inappropriate for most enterprise developers and application users.

Task 36 — Review Cluster Roles and Bindings

Step 1 — List ClusterRoles

kubectl get clusterroles

Step 2 — List ClusterRoleBindings

kubectl get clusterrolebindings

Step 3 — Inspect the cluster-admin Role

kubectl describe clusterrole cluster-admin

Step 4 — Locate cluster-admin Bindings

PowerShell

kubectl get clusterrolebindings | findstr cluster-admin

Git Bash

kubectl get clusterrolebindings | grep cluster-admin

Security Risk

The cluster-admin role provides nearly unrestricted access.

It should be assigned only to a small number of authorised administrators.

Task 37 — Review Service Accounts

List Service Accounts in all Namespaces.

kubectl get serviceaccounts --all-namespaces

Review:

• Default Service Accounts
• System Service Accounts
• Application Service Accounts

Inspect the Lab Default Service Account

kubectl describe serviceaccount default -n cloudnova-architecture-lab

Security Observation

Every Namespace receives a default Service Account.

Applications should use dedicated Service Accounts when they require Kubernetes API permissions.

Task 38 — Review Secrets and ConfigMaps

List ConfigMaps:

kubectl get configmaps --all-namespaces

List Secrets:

kubectl get secrets --all-namespaces

Do not display Secret contents unless there is a valid authorised requirement.

Security Importance

ConfigMaps may contain:

• Application configuration
• URLs
• Logging settings
• Feature flags

Secrets may contain:

• Tokens
• Passwords
• Certificates
• Registry credentials

Excessive access to Secrets can lead to credential compromise.

Task 39 — Review the Node Lease Mechanism

Run:

kubectl get leases -n kube-node-lease

Each node should have a corresponding Lease object.

Inspect one:

kubectl describe lease cloudnova-security-lab-worker -n kube-node-lease

Lease Purpose

Node leases provide lightweight heartbeat information.

Worker Node
        │
        ▼
Lease Updated
        │
        ▼
Control Plane Confirms Node Health

Task 40 — Review Kubernetes Component Logs

API Server Logs

kubectl logs -n kube-system <API-SERVER-POD>

Scheduler Logs

kubectl logs -n kube-system <SCHEDULER-POD>

Controller Manager Logs

kubectl logs -n kube-system <CONTROLLER-MANAGER-POD>

kube-proxy Logs

First list kube-proxy Pods:

kubectl get pods -n kube-system -l k8s-app=kube-proxy

Then:

kubectl logs -n kube-system <KUBE-PROXY-POD>

Security Monitoring Value

Component logs can help detect:

• Authentication failures
• Authorisation failures
• Scheduling problems
• Controller errors
• Network issues
• Configuration mistakes
• Suspicious API activity

Task 41 — Identify Critical Security Assets

Create a list containing at least the following assets:

Asset	Why It Is Critical
kube-apiserver	Controls access to Kubernetes resources
etcd	Stores cluster state and sensitive data
kubelet	Controls workloads on each node
container runtime	Executes application containers
kubeconfig	May contain administrative credentials
Service Account tokens	Provide workload API identity
ClusterRoleBindings	Can grant cluster-wide permissions
Secrets	Store credentials and sensitive information
Worker Nodes	Host application workloads
Control Plane certificates	Protect component identity and communication

Task 42 — Document the Kubernetes Attack Surface

Create:

kubernetes-attack-surface.md

Use:

# CloudNova Kubernetes Attack Surface

## External Access Points

- Kubernetes API endpoint
- Exposed application Services
- Container registry access
- Administrative workstations

## Control Plane Components

### kube-apiserver

- Purpose:
- Sensitive configuration:
- Possible threats:
- Recommended controls:

### etcd

- Purpose:
- Sensitive data:
- Possible threats:
- Recommended controls:

### kube-scheduler

- Purpose:
- Possible threats:
- Recommended controls:

### kube-controller-manager

- Purpose:
- Possible threats:
- Recommended controls:

## Worker Node Components

### kubelet

- Purpose:
- Possible threats:
- Recommended controls:

### Container Runtime

- Purpose:
- Possible threats:
- Recommended controls:

### kube-proxy

- Purpose:
- Possible threats:
- Recommended controls:

## Workload Layer

- Privileged containers
- Vulnerable images
- Excessive Service Account permissions
- Exposed Secrets
- Missing resource limits
- Missing network restrictions
- Writable filesystems
- Host-mounted volumes

## Identity Layer

- Administrative kubeconfig files
- ClusterRoleBindings
- Default Service Accounts
- Long-lived tokens
- Excessive RBAC permissions

## Network Layer

- Public API Server
- Public Services
- Missing Network Policies
- Unencrypted communication
- Unrestricted east-west traffic

## Storage Layer

- Unencrypted etcd
- Unencrypted persistent storage
- Exposed snapshots
- Excessive volume permissions
- Missing backups

## Primary Risks

1.
2.
3.
4.
5.

## Recommended Security Priorities

1.
2.
3.
4.
5.

Task 43 — Perform an Architecture Security Assessment

Review each layer.

Control Plane Assessment

Check:

• Control Plane components are identifiable
• Static Pod manifests are protected
• API Server access is controlled
• etcd is treated as sensitive
• Administrative access is limited

Worker Node Assessment

Check:

• kubelet is active
• containerd is active
• application Pods run on Worker Nodes
• node access is restricted
• runtime configuration is monitored

Workload Assessment

Check:

• Workloads use dedicated Namespaces
• Resource requests and limits exist
• Service exposure is controlled
• Service Accounts are reviewed
• Security contexts are planned

Network Assessment

Check:

• Service type is understood
• Pod IPs are internal
• DNS resolution works
• ClusterIP is not directly public
• Network Policies are recommended

Identity Assessment

Check:

• Current administrator permissions are understood
• ClusterRoleBindings are reviewed
• Service Accounts are visible
• kubeconfig is protected
• Least privilege is recommended

Task 44 — Collect Lab Evidence

Capture screenshots or command output for the following.

Evidence 01 — Cluster Information

kubectl cluster-info

Evidence 02 — Nodes

kubectl get nodes -o wide

Evidence 03 — Node Labels

kubectl get nodes --show-labels

Evidence 04 — Control Plane Components

kubectl get pods -n kube-system -o wide

Evidence 05 — Static Pod Manifests

docker exec cloudnova-security-lab-control-plane ls -la /etc/kubernetes/manifests

Evidence 06 — kubelet

docker exec cloudnova-security-lab-worker systemctl is-active kubelet

Evidence 07 — containerd

docker exec cloudnova-security-lab-worker containerd --version

Evidence 08 — Deployment Hierarchy

kubectl get deployments,replicasets,pods -n cloudnova-architecture-lab

Evidence 09 — Pod Placement

kubectl get pods -n cloudnova-architecture-lab -o wide

Evidence 10 — Service and Endpoints

kubectl get service,endpoints,endpointslices -n cloudnova-architecture-lab

Evidence 11 — DNS Resolution

kubectl exec dns-test-client -n cloudnova-architecture-lab -- nslookup cloudnova-architecture-service

Evidence 12 — Service Connectivity

kubectl exec dns-test-client -n cloudnova-architecture-lab -- wget -qO- http://cloudnova-architecture-service

Evidence 13 — API Resources

kubectl api-resources

Evidence 14 — Current Permissions

kubectl auth can-i --list

Evidence 15 — Cluster-Scoped Resources

kubectl api-resources --namespaced=false

Task 45 — Create the Architecture Assessment Report

Create:

kubernetes-architecture-assessment.md

Use:

# CloudNova Kubernetes Architecture Assessment

## Assessment Information

- Lab ID: K8S-FND-LAB-03
- Cluster:
- Cluster type:
- Kubernetes context:
- Assessment date:
- Assessor:

## Cluster Overview

- Number of Control Plane nodes:
- Number of Worker Nodes:
- Kubernetes version:
- Container runtime:
- Cluster network:
- DNS service:

## Control Plane Components

### API Server

- Pod name:
- Node:
- Primary function:
- Security importance:
- Key risks:

### etcd

- Pod name:
- Node:
- Primary function:
- Security importance:
- Key risks:

### Scheduler

- Pod name:
- Node:
- Primary function:
- Security importance:
- Key risks:

### Controller Manager

- Pod name:
- Node:
- Primary function:
- Security importance:
- Key risks:

## Worker Node Components

### kubelet

- Status:
- Primary function:
- Security importance:
- Key risks:

### Container Runtime

- Runtime:
- Version:
- Primary function:
- Security importance:
- Key risks:

### kube-proxy

- Deployment type:
- Primary function:
- Security importance:
- Key risks:

## Workload Lifecycle

Document the flow from:

1. kubectl request
2. API Server
3. etcd
4. Controller Manager
5. Scheduler
6. kubelet
7. Container runtime
8. Running Pod

## Networking Assessment

- Pod network:
- Service type:
- Service ClusterIP:
- Endpoint count:
- DNS resolution result:
- Internal connectivity result:
- Network Policy status:

## Identity Assessment

- Current Kubernetes identity:
- Current permission level:
- cluster-admin access:
- Service Accounts reviewed:
- Least-privilege concerns:

## Critical Security Assets

1.
2.
3.
4.
5.

## Key Security Findings

### Finding 1

- Description:
- Impact:
- Likelihood:
- Severity:
- Recommendation:

### Finding 2

- Description:
- Impact:
- Likelihood:
- Severity:
- Recommendation:

### Finding 3

- Description:
- Impact:
- Likelihood:
- Severity:
- Recommendation:

## Overall Architecture Assessment

State whether the architecture is suitable for:

- Local learning:
- Development:
- Testing:
- Production:

## Recommended Next Actions

1.
2.
3.
4.
5.

Suggested Assessment Conclusion

The cluster is suitable for local learning and controlled development testing.

The architecture demonstrates the major Kubernetes Control Plane, Worker Node, workload and networking components. However, additional controls would be required before production use, including stronger identity governance, network segmentation, audit logging, secrets protection, image assurance, runtime monitoring, node hardening and disaster recovery.

Task 46 — Review All Lab Resources

Run:

kubectl get all -n cloudnova-architecture-lab

Review:

• Deployment
• ReplicaSet
• Application Pods
• DNS test Pod
• Service

Review the Namespace:

kubectl get namespace cloudnova-architecture-lab --show-labels

Review Service endpoints:

kubectl get endpoints,endpointslices -n cloudnova-architecture-lab

Task 47 — Clean Up the Lab

Step 1 — Delete the DNS Test Pod

PowerShell

kubectl delete -f .\04-dns-test-pod.yaml

Git Bash

kubectl delete -f 04-dns-test-pod.yaml

Step 2 — Delete the Service

PowerShell

kubectl delete -f .\03-service.yaml

Git Bash

kubectl delete -f 03-service.yaml

Step 3 — Delete the Deployment

PowerShell

kubectl delete -f .\02-architecture-deployment.yaml

Git Bash

kubectl delete -f 02-architecture-deployment.yaml

Step 4 — Delete the Namespace

PowerShell

kubectl delete -f .\01-namespace.yaml

Git Bash

kubectl delete -f 01-namespace.yaml

Step 5 — Verify Cleanup

kubectl get namespace cloudnova-architecture-lab

Expected result:

NotFound

Optional Cluster Cleanup

Keep the cluster for the next lab.

When the cluster is no longer required:

kind delete cluster --name cloudnova-security-lab

Troubleshooting Guide

Issue 01 — Control Plane Pods Are Not Listed

Check:

kubectl get pods -n kube-system

Confirm that the current context points to the kind cluster.

kubectl config current-context

Issue 02 — Static Pod Manifest Command Fails

Confirm that the kind Control Plane container exists.

docker ps

Use the exact container name:

cloudnova-security-lab-control-plane

Issue 03 — systemctl Command Fails

Try:

docker exec cloudnova-security-lab-worker ps aux

Then filter for kubelet or containerd.

PowerShell

docker exec cloudnova-security-lab-worker ps aux | findstr kubelet

Git Bash

docker exec cloudnova-security-lab-worker ps aux | grep kubelet

Issue 04 — crictl Command Is Unavailable

Confirm you are running the command inside the kind node:

docker exec cloudnova-security-lab-worker crictl ps

Do not run crictl directly from Windows unless it has been installed separately.

Issue 05 — DNS Test Pod Is Pending

Check:

kubectl describe pod dns-test-client -n cloudnova-architecture-lab

Review:

• Scheduling
• Image pull status
• Node availability
• Resource availability

Issue 06 — DNS Resolution Fails

Check CoreDNS:

kubectl get pods -n kube-system -l k8s-app=kube-dns

Check the Service:

kubectl get service cloudnova-architecture-service -n cloudnova-architecture-lab

Check endpoints:

kubectl get endpoints -n cloudnova-architecture-lab

Issue 07 — Service Has No Endpoints

Verify that the Service selector matches the Pod label.

kubectl get pods -n cloudnova-architecture-lab --show-labels

kubectl describe service cloudnova-architecture-service -n cloudnova-architecture-lab

Expected matching label:

app=cloudnova-architecture-web

Issue 08 — Application Connectivity Fails

Check:

kubectl get pods -n cloudnova-architecture-lab
kubectl get service -n cloudnova-architecture-lab
kubectl get endpoints -n cloudnova-architecture-lab

Review application logs:

kubectl logs deployment/cloudnova-architecture-web -n cloudnova-architecture-lab

Issue 09 — kubectl auth whoami Is Unsupported

Some Kubernetes client versions may not support the command.

Use:

kubectl config view --minify

Then review the current context and user.

Issue 10 — Component Logs Are Difficult to Read

Limit the output:

kubectl logs -n kube-system <POD-NAME> --tail=50

Display recent logs:

kubectl logs -n kube-system <POD-NAME> --since=10m

Validation Checklist

Confirm that you completed the following:

• Verified the Kubernetes cluster
• Created the Lab 03 workspace
• Created the architecture lab Namespace
• Inspected the Control Plane node
• Inspected the Worker Node
• Compared node labels and taints
• Inspected kind node containers
• Reviewed all Kubernetes Namespaces
• Located the API Server
• Located etcd
• Located the Scheduler
• Located the Controller Manager
• Inspected static Pod manifests
• Verified kubelet
• Verified containerd
• Inspected kube-proxy
• Inspected CoreDNS
• Deployed the demonstration application
• Traced Deployment, ReplicaSet and Pod ownership
• Reviewed workload creation events
• Identified Pod scheduling and node placement
• Inspected runtime containers
• Created a ClusterIP Service
• Compared Pod IPs with Service endpoints
• Created a DNS test Pod
• Tested short-name DNS resolution
• Tested fully qualified DNS resolution
• Tested Service connectivity
• Reviewed Pod DNS configuration
• Inspected Kubernetes API resources
• Compared Namespace-scoped and cluster-scoped resources
• Reviewed kubeconfig information
• Reviewed current permissions
• Reviewed ClusterRoles and ClusterRoleBindings
• Reviewed Service Accounts
• Reviewed Secrets and ConfigMaps
• Reviewed node Leases
• Reviewed Kubernetes component logs
• Documented critical security assets
• Completed the attack-surface assessment
• Completed the architecture assessment report
• Collected the required evidence
• Cleaned up the lab resources

Knowledge Check

Question 1

Which Kubernetes component receives API requests from users and workloads?

• A. kube-proxy
• B. kube-apiserver
• C. CoreDNS
• D. containerd

Answer: B

Question 2

Which component stores Kubernetes cluster state?

• A. etcd
• B. kubelet
• C. Scheduler
• D. Ingress Controller

Answer: A

Question 3

Which component selects a node for an unscheduled Pod?

• A. kube-controller-manager
• B. kube-proxy
• C. kube-scheduler
• D. CoreDNS

Answer: C

Question 4

Which component ensures that the actual cluster state moves towards the desired state?

• A. Controller Manager
• B. Docker Desktop
• C. Service
• D. ConfigMap

Answer: A

Question 5

Which component runs on each Kubernetes node and manages Pod execution?

• A. kubelet
• B. etcd
• C. API Server
• D. ClusterRole

Answer: A

Question 6

Which component creates and executes containers?

• A. CoreDNS
• B. container runtime
• C. Namespace
• D. Service Account

Answer: B

Question 7

What is the purpose of kube-proxy?

• A. Store Kubernetes Secrets
• B. Implement Service networking rules
• C. Schedule Pods
• D. Authenticate users

Answer: B

Question 8

What does CoreDNS provide?

• A. Container image scanning
• B. Kubernetes internal DNS resolution
• C. Storage encryption
• D. RBAC authorisation

Answer: B

Question 9

Which resource provides a stable virtual IP address for a group of Pods?

• A. Deployment
• B. Service
• C. Secret
• D. Node

Answer: B

Question 10

Why is etcd considered a critical security asset?

• A. It stores cluster state and potentially sensitive data.
• B. It only stores application HTML.
• C. It creates Docker images.
• D. It provides internet connectivity.

Answer: A

Question 11

Which resource type is cluster-scoped?

• A. Pod
• B. ConfigMap
• C. Node
• D. Service

Answer: C

Question 12

Why should kubeconfig files be protected?

• A. They may contain cluster endpoints and administrative credentials.
• B. They contain only public documentation.
• C. They are required only for Docker images.
• D. They cannot be used to access Kubernetes.

Answer: A

Skills Developed

By completing this lab, you practised:

• Kubernetes architecture analysis
• Control Plane inspection
• Worker Node inspection
• Static Pod analysis
• API Server configuration review
• etcd security analysis
• Scheduler inspection
• Controller Manager inspection
• kubelet verification
• Container runtime inspection
• kube-proxy analysis
• CoreDNS inspection
• Workload lifecycle tracing
• Deployment ownership analysis
• Pod scheduling analysis
• Service and endpoint inspection
• Kubernetes DNS testing
• Internal connectivity testing
• API resource discovery
• Cluster-scoped resource analysis
• kubeconfig review
• RBAC visibility review
• Security asset identification
• Attack-surface documentation
• Architecture assessment reporting
• Evidence collection

Lab Summary

In this lab, you explored the internal architecture of the CloudNova Technologies Kubernetes cluster.

You identified and inspected:

• The Kubernetes API Server
• etcd
• The Scheduler
• The Controller Manager
• The kubelet
• The container runtime
• kube-proxy
• CoreDNS
• Control Plane and Worker Nodes
• Deployments, ReplicaSets and Pods
• Services and endpoints
• Kubernetes DNS
• Namespace-scoped and cluster-scoped resources
• Service Accounts and cluster permissions

You traced the complete workload lifecycle:

kubectl
   │
   ▼
API Server
   │
   ▼
etcd
   │
   ▼
Controller Manager
   │
   ▼
Scheduler
   │
   ▼
kubelet
   │
   ▼
containerd
   │
   ▼
Running Application Pod

You also documented the Kubernetes attack surface and identified the Control Plane, nodes, credentials, Service Accounts, RBAC objects, Secrets and runtime components as critical security assets.

This architectural understanding is essential because Kubernetes security cannot be implemented effectively without first understanding how the platform processes requests, stores state, schedules workloads, runs containers and routes network traffic.

What’s Next?

In the next lab, you will work with Kubernetes Namespaces, ConfigMaps, Secrets and storage resources to understand how applications organise configuration and persistent data.

➡️ Next Lab: Lab 04 — Manage Kubernetes Namespaces, ConfigMaps, Secrets and Storage