Skip to content
GoHackersCloud Academy

Runbook 01 — Kubernetes Logging & Monitoring Health Assessment

Runbook Overview

Section titled “Runbook Overview”

Logging and monitoring are the foundation of Kubernetes Security Operations.

Without complete visibility, organizations cannot:

  • Detect attacks
  • Investigate incidents
  • Meet compliance requirements
  • Troubleshoot production issues
  • Perform digital forensics
  • Measure operational health

This runbook provides a structured assessment that Cloud Security Engineers, Platform Engineers and SOC Analysts can use to evaluate the health of Kubernetes logging and monitoring capabilities across Amazon EKS environments.

The assessment aligns with enterprise operational practices used during:

  • Security Health Checks
  • Production Readiness Reviews
  • Compliance Audits
  • SOC Maturity Assessments
  • Cloud Security Assessments

Assessment Information

Section titled “Assessment Information”

Assessment Objective

Section titled “Assessment Objective”

Evaluate the effectiveness, completeness and operational readiness of Kubernetes logging and monitoring across Amazon EKS.

Estimated Time

Section titled “Estimated Time”

2–3 Hours

Assessment Type

Section titled “Assessment Type”
  • Operational Assessment
  • Security Assessment
  • Infrastructure Review
  • Compliance Validation
  • SOC Readiness Assessment

Assessment Scope

Section titled “Assessment Scope”

Review the following components:

  • Kubernetes Audit Logging
  • Amazon CloudWatch Logs
  • Prometheus
  • Grafana
  • Falco
  • AWS CloudTrail
  • Amazon GuardDuty
  • AWS Security Hub
  • Alerting
  • SIEM Integration
  • Operational Dashboards
  • Monitoring Governance

Business Scenario

Section titled “Business Scenario”

A financial organisation operates:

  • 420 Amazon EKS clusters
  • 9 AWS Accounts
  • Multiple AWS Regions
  • 24×7 Security Operations Centre

Senior management requests an enterprise-wide health assessment after an external audit identifies inconsistent monitoring configurations across several production clusters.

You are assigned to assess the monitoring platform and identify operational risks before the next compliance audit.

Enterprise Assessment Architecture

Section titled “Enterprise Assessment Architecture”
Amazon EKS


↓


Audit Logs


↓


Falco


↓


Prometheus


↓


Grafana


↓


CloudWatch


↓


CloudTrail


↓


GuardDuty


↓


Security Hub


↓


SIEM


↓


SOC

Assessment Checklist

Section titled “Assessment Checklist”
Component Status Notes
Audit Logging Enabled ☐
CloudWatch Log Groups ☐
Falco Running ☐
Prometheus Healthy ☐
Grafana Accessible ☐
Alertmanager Configured ☐
CloudTrail Enabled ☐
GuardDuty Enabled ☐
Security Hub Enabled ☐
SIEM Receiving Logs ☐
Alerting Operational ☐
Dashboards Reviewed ☐

Phase 1 — Kubernetes Audit Logging Review

Section titled “Phase 1 — Kubernetes Audit Logging Review”

Objective

Section titled “Objective”

Confirm that Kubernetes API activity is being recorded.

Verification

Section titled “Verification”

Describe the cluster.

Terminal window
aws eks describe-cluster \
--name production-cluster \
--query "cluster.logging"

Verify:

  • Audit Logs enabled
  • API Logs enabled
  • Authenticator Logs enabled
  • Scheduler Logs enabled
  • Controller Manager Logs enabled

Health Checks

Section titled “Health Checks”
Check Pass Fail
Audit Logs Enabled ☐ ☐
API Logs Enabled ☐ ☐
Log Delivery Working ☐ ☐
Log Retention Configured ☐ ☐

Phase 2 — CloudWatch Assessment

Section titled “Phase 2 — CloudWatch Assessment”

Verify Log Groups.

Terminal window
aws logs describe-log-groups

Review:

  • Log retention
  • Encryption
  • Naming standards
  • Access permissions

Health Checks

Section titled “Health Checks”
Check Pass Fail
Log Groups Present ☐ ☐
KMS Encryption Enabled ☐ ☐
Retention Policy Applied ☐ ☐
IAM Access Restricted ☐ ☐

Phase 3 — Falco Runtime Assessment

Section titled “Phase 3 — Falco Runtime Assessment”

Verify Falco.

Terminal window
kubectl get pods -n falco

Verify DaemonSet.

Terminal window
kubectl get daemonset -n falco

Review:

  • Running Pods
  • Failed Pods
  • Runtime alerts
  • Rule configuration

Health Checks

Section titled “Health Checks”
Check Pass Fail
DaemonSet Running ☐ ☐
Alerts Generated ☐ ☐
Rules Updated ☐ ☐
Runtime Coverage Complete ☐ ☐

Phase 4 — Prometheus Assessment

Section titled “Phase 4 — Prometheus Assessment”

Verify monitoring components.

Terminal window
kubectl get pods -n monitoring

Review:

  • Prometheus
  • Node Exporter
  • kube-state-metrics
  • Alertmanager

Health Checks

Section titled “Health Checks”
Check Pass Fail
Prometheus Running ☐ ☐
Targets Healthy ☐ ☐
Metrics Collected ☐ ☐
Storage Healthy ☐ ☐

Phase 5 — Grafana Assessment

Section titled “Phase 5 — Grafana Assessment”

Access Grafana.

Review dashboards.

Confirm:

  • Cluster Health
  • Node Health
  • Namespace Metrics
  • API Server Metrics
  • Pod Health

Health Checks

Section titled “Health Checks”
Dashboard Healthy
Cluster Overview ☐
Node Dashboard ☐
Namespace Dashboard ☐
Kubernetes API ☐
Resource Utilisation ☐

Phase 6 — AWS Security Services

Section titled “Phase 6 — AWS Security Services”

Verify CloudTrail.

Terminal window
aws cloudtrail describe-trails

Verify GuardDuty.

Terminal window
aws guardduty list-detectors

Verify Security Hub.

Terminal window
aws securityhub get-enabled-standards

Health Checks

Section titled “Health Checks”
Service Enabled
CloudTrail ☐
GuardDuty ☐
Inspector ☐
Security Hub ☐

Phase 7 — SIEM Assessment

Section titled “Phase 7 — SIEM Assessment”

Review:

  • Log forwarding
  • Event correlation
  • Detection rules
  • Alert routing

Confirm telemetry sources.

Source Received
Audit Logs ☐
Falco ☐
CloudTrail ☐
GuardDuty ☐
Inspector ☐
Security Hub ☐

Phase 8 — Alerting Assessment

Section titled “Phase 8 — Alerting Assessment”

Generate a test event.

Example:

Terminal window
kubectl create namespace monitoring-test

Confirm:

  • Audit Log generated
  • Alert received
  • SIEM correlation completed
  • Notification delivered

Alert Validation

Section titled “Alert Validation”
Validation Pass
Alert Generated ☐
Alert Routed ☐
Alert Investigated ☐
Evidence Preserved ☐

Phase 9 — Dashboard Assessment

Section titled “Phase 9 — Dashboard Assessment”

Review operational dashboards.

Evaluate:

  • CPU utilisation
  • Memory utilisation
  • Pod restarts
  • Node availability
  • Namespace health
  • Runtime alerts
  • Audit events

Questions:

  • Are dashboards useful?
  • Are dashboards current?
  • Are alerts actionable?
  • Are KPIs visible?

Phase 10 — Operational Readiness Review

Section titled “Phase 10 — Operational Readiness Review”

Evaluate the following areas.

Area Rating (1–5)
Logging Coverage
Monitoring Coverage
Runtime Detection
Dashboard Quality
Alert Accuracy
SIEM Integration
SOC Readiness
Incident Detection
Automation
Operational Maturity

Risk Assessment

Section titled “Risk Assessment”

Document identified risks.

Risk Severity Recommendation
Missing Audit Logs High Enable Audit Logging
Missing Runtime Detection High Deploy Falco
No SIEM Integration Medium Centralize telemetry
Weak Alerting Medium Improve detection rules
Dashboard Gaps Low Enhance Grafana dashboards

Enterprise Findings Summary

Section titled “Enterprise Findings Summary”

Summarize the assessment.

Category Status
Logging
Monitoring
Detection
Investigation
Alerting
Automation
Governance
Compliance

Overall Assessment:

  • ☐ Excellent
  • ☐ Good
  • ☐ Requires Improvement
  • ☐ Critical Issues Identified

Remediation Plan

Section titled “Remediation Plan”

Prioritize corrective actions.

Priority Action Owner Target Date
High Enable missing Audit Logs Platform Team
High Deploy Falco on all nodes Security Team
High Integrate SIEM SOC Team
Medium Improve dashboards Platform Team
Medium Tune detection rules Detection Engineering
Low Review retention policies Compliance Team

Enterprise Best Practices

Section titled “Enterprise Best Practices”

As a Cloud Security Engineer:

  • Enable Audit Logging on every production cluster.
  • Monitor Kubernetes continuously using Prometheus and Grafana.
  • Deploy Falco across every worker node.
  • Forward all security telemetry to a centralized SIEM.
  • Regularly validate alerting and detection workflows.
  • Encrypt all log storage using AWS KMS.
  • Restrict access to monitoring platforms using IAM and RBAC.
  • Review dashboards daily as part of SOC operations.
  • Test monitoring capabilities during incident response exercises.
  • Conduct quarterly health assessments to maintain operational readiness.

Real-World Scenario

Section titled “Real-World Scenario”

A multinational healthcare provider experienced delayed detection of a ransomware attack because several production EKS clusters had disabled Audit Logging and incomplete runtime monitoring.

During the post-incident review, engineers discovered:

  • Missing Kubernetes Audit Logs
  • Falco deployed on only half of the worker nodes
  • Inconsistent Prometheus metric collection
  • Security Hub findings were not forwarded to the SIEM

Following a comprehensive health assessment, the organisation:

  • Standardized logging across every cluster.
  • Enabled runtime monitoring on all nodes.
  • Centralized telemetry into the enterprise SIEM.
  • Implemented automated health checks for monitoring components.
  • Reduced Mean Time to Detect (MTTD) by more than 60%.

Assessment Summary

Section titled “Assessment Summary”

Congratulations!

You have completed an enterprise Kubernetes Logging & Monitoring Health Assessment.

During this runbook you evaluated:

  • Kubernetes Audit Logging
  • CloudWatch Logs
  • Falco Runtime Security
  • Prometheus Monitoring
  • Grafana Dashboards
  • CloudTrail
  • GuardDuty
  • Security Hub
  • SIEM Integration
  • Alerting
  • SOC Readiness

This assessment provides a repeatable framework for evaluating the operational maturity of enterprise Kubernetes monitoring environments.

Key Takeaways

Section titled “Key Takeaways”
  • Effective monitoring depends on complete telemetry collection.
  • Runtime detection, logging and metrics must work together.
  • Regular health assessments identify monitoring gaps before they become security incidents.
  • Centralized monitoring improves detection, investigation and compliance.
  • Continuous assessment is essential for maintaining a mature Kubernetes security programme.

What’s Next?

Section titled “What’s Next?”

The next runbook focuses on a broader enterprise review of logging and monitoring architecture, governance and operational maturity across multiple Kubernetes clusters.

➡️ Next Runbook: Runbook 02 — Enterprise Logging & Monitoring Assessment