Runbook 01 — Security Automation Design and Safety Review

Runbook Information

Runbook Type: Security Automation Governance
Difficulty: Intermediate → Advanced
Primary Audience: SOC Analysts / Security Engineers / Cloud Security Engineers / Security Automation Engineers / Incident Responders
Security Domains: Security Automation / SOAR / SOC / Cloud Security / Vulnerability Management
Execution Model: Design Review Before Deployment
Primary Goal: Automate safely without creating unnecessary operational risk

Purpose

Security automation can reduce:

MANUAL WORK

ALERT FATIGUE

REPETITIVE ANALYSIS

RESPONSE DELAYS

CONFIGURATION DRIFT

But poorly designed automation can create:

OUTAGES

ACCOUNT LOCKOUTS

DATA LOSS

MASS FALSE POSITIVES

SECURITY CONTROL DISRUPTION

UNCONTROLLED CLOUD CHANGES

This runbook provides a repeatable decision-making process for evaluating a proposed security automation before it is deployed.

Core Principle

The most important question is not:

CAN THIS BE AUTOMATED?

It is:

SHOULD THIS BE AUTOMATED?

Automation Safety Model

SECURITY TASK
      ↓
UNDERSTAND MANUAL PROCESS
      ↓
DEFINE AUTOMATION SCOPE
      ↓
IDENTIFY DATA SOURCES
      ↓
IDENTIFY REQUIRED PRIVILEGES
      ↓
ANALYZE FAILURE MODES
      ↓
ASSESS BLAST RADIUS
      ↓
DEFINE HUMAN APPROVAL
      ↓
DEFINE ROLLBACK
      ↓
TEST SAFELY
      ↓
MONITOR
      ↓
APPROVE / REJECT

01 — Start with the Security Problem

Before writing code, document:

WHAT PROBLEM
ARE WE TRYING TO SOLVE?

Example:

SOC analysts spend 15 minutes manually enriching
every authentication alert with identity, asset,
and vulnerability context.

A poor problem statement:

We want more automation.

A better statement:

We want to reduce repetitive alert-enrichment work
while keeping investigation and incident decisions
with the analyst.

02 — Document the Current Manual Process

Write down every existing step.

Example:

ALERT RECEIVED
      ↓
ANALYST OPENS SIEM
      ↓
LOOKS UP USER
      ↓
LOOKS UP ASSET
      ↓
CHECKS VULNERABILITY STATUS
      ↓
CHECKS IOC CONTEXT
      ↓
ASSIGNS PRIORITY
      ↓
DOCUMENTS FINDING

Do not automate a process you do not understand.

03 — Identify the Automation Candidate

Determine exactly which steps are:

REPETITIVE

DETERMINISTIC

LOW-RISK

WELL-DEFINED

Good candidates:

DATA COLLECTION

FORMAT CONVERSION

DEDUPLICATION

NORMALIZATION

ENRICHMENT

REPORT GENERATION

READ-ONLY SECURITY CHECKS

04 — Identify Poor Automation Candidates

Exercise caution with:

ACCOUNT DISABLEMENT

ENDPOINT ISOLATION

RESOURCE DELETION

FIREWALL CHANGES

CREDENTIAL ROTATION

PRODUCTION CONFIGURATION CHANGES

These may be appropriate in mature environments, but they require much stronger governance.

05 — Classify the Automation

Use one of four categories:

Level	Automation Type	Example
Level 1	Read-only	Collect security data
Level 2	Recommendation	Prioritize alerts
Level 3	Controlled write	Create ticket
Level 4	High-impact response	Disable account

06 — Preferred Progression

Whenever possible, evolve automation gradually:

READ
  ↓
ANALYZE
  ↓
RECOMMEND
  ↓
APPROVED WRITE
  ↓
CONTROLLED RESPONSE

Do not jump directly from:

MANUAL

to:

FULLY AUTONOMOUS RESPONSE

07 — Define Success Criteria

Document measurable outcomes.

Example:

Reduce average alert enrichment time
from 10 minutes to under 2 minutes.

Other examples:

Reduce duplicate alerts by 25%

Generate daily cloud configuration report

Improve asset-owner mapping coverage

Reduce manual vulnerability triage effort

08 — Define What Automation Will Not Do

This is equally important.

Example:

THE AUTOMATION WILL:

Collect context

Normalize alerts

Calculate training priority

Generate analyst recommendations

but:

THE AUTOMATION WILL NOT:

Disable users

Isolate endpoints

Block source IPs

Close alerts automatically

09 — Identify Inputs

Document all data sources.

Example:

SIEM API

CMDB

IDENTITY PROVIDER

VULNERABILITY PLATFORM

THREAT INTELLIGENCE

CLOUD SECURITY PLATFORM

10 — Identify Output

Document what will be created.

Examples:

CSV REPORT

JSON RESULT

SIEM ENRICHMENT

TICKET

CASE CANDIDATE

DASHBOARD DATA

11 — Build the Data Flow

Example:

SIEM ALERT
   ↓
IDENTITY API
   ↓
CMDB
   ↓
VULNERABILITY API
   ↓
THREAT INTELLIGENCE
   ↓
NORMALIZATION
   ↓
RISK SCORE
   ↓
ANALYST QUEUE

12 — Identify Trust Boundaries

Mark where data crosses:

SYSTEM BOUNDARIES

NETWORK BOUNDARIES

CLOUD BOUNDARIES

ORGANIZATIONAL BOUNDARIES

Example:

SOC PLATFORM
     ↓
API GATEWAY
     ↓
EXTERNAL THREAT INTEL SERVICE

13 — Ask Whether Each Input Can Be Trusted

For every source ask:

CAN DATA BE MISSING?

CAN DATA BE STALE?

CAN DATA BE WRONG?

CAN DATA BE MALFORMED?

CAN DATA BE DUPLICATED?

14 — Input Validation Requirement

Every automation should follow:

INPUT
 ↓
VALIDATE
 ↓
NORMALIZE
 ↓
PROCESS

Never:

INPUT
 ↓
TRUST
 ↓
ACTION

15 — Define Required Fields

Example alert requirements:

alert_id

timestamp

severity

source

asset

If required fields are missing, decide whether to:

REJECT

QUARANTINE

MARK UNKNOWN

16 — Unknown Is Not False

This is a critical design principle.

MFA_ENABLED = FALSE

is different from:

MFA_STATUS = UNKNOWN

Likewise:

NO IOC MATCH

is different from:

IOC LOOKUP FAILED

17 — Define Normalization Rules

Examples:

HOSTNAMES
→ UPPERCASE

USERNAME
→ LOWERCASE

SEVERITY
→ critical/high/medium/low

TIMESTAMPS
→ UTC ISO 8601

18 — Record Data Provenance

For every enrichment field, ideally know:

SOURCE

COLLECTION TIME

CONFIDENCE

LAST UPDATED

19 — Data Freshness Review

Ask:

HOW OLD CAN THIS DATA BE
BEFORE IT BECOMES UNRELIABLE?

Example:

Asset inventory:
24 hours

IOC reputation:
6 hours

User privilege status:
1 hour

These are examples only.

Use the organization’s requirements.

20 — Identify Required Permissions

List every permission the automation requires.

Example:

READ SIEM ALERTS

READ USER DIRECTORY

READ ASSET INVENTORY

READ VULNERABILITY DATA

21 — Apply Least Privilege

The automation should receive:

ONLY THE ACCESS
IT NEEDS

Example:

READ-ONLY SECURITY AUDITOR

should not receive:

GLOBAL ADMINISTRATOR

22 — Separate Read and Write Credentials

Recommended architecture:

TRIAGE WORKFLOW
      ↓
READ-ONLY CREDENTIAL

RESPONSE WORKFLOW
      ↓
CONTROLLED WRITE CREDENTIAL

23 — Why Separate Credentials?

If the triage process is compromised:

READ-ONLY ACCESS

creates a smaller:

BLAST RADIUS

than:

ADMIN ACCESS

24 — Credential Type Review

Prefer when available:

WORKLOAD IDENTITY

MANAGED IDENTITY

SHORT-LIVED TOKENS

FEDERATED CREDENTIALS

over:

LONG-LIVED STATIC KEYS

25 — Secret Storage

Never place production secrets in:

SOURCE CODE

GIT

CONFIG FILES

LOGS

REPORTS

CHAT MESSAGES

Use approved:

SECRET MANAGEMENT

26 — Define Credential Rotation

Document:

WHO OWNS THE CREDENTIAL?

HOW IS IT ROTATED?

HOW IS IT REVOKED?

WHEN DOES IT EXPIRE?

27 — Identify Read Operations

Examples:

GET ALERTS

LIST USERS

DESCRIBE INSTANCES

READ CLOUD CONFIGURATION

QUERY VULNERABILITIES

These generally have lower operational risk.

28 — Identify Write Operations

Examples:

CREATE TICKET

UPDATE CASE

ADD TAG

ADD COMMENT

These are still changes and require validation.

29 — Identify High-Impact Operations

Examples:

DISABLE USER

REVOKE SESSION

ISOLATE ENDPOINT

BLOCK NETWORK ACCESS

DELETE RESOURCE

ROTATE PRODUCTION SECRET

30 — Assign Impact Classification

Use:

LOW

MEDIUM

HIGH

CRITICAL

Example:

Action	Impact
Generate report	Low
Create ticket	Low
Change alert status	Medium
Disable account	High
Delete cloud resource	Critical

31 — Identify Blast Radius

Ask:

IF THIS AUTOMATION IS WRONG,
HOW MANY PEOPLE OR SYSTEMS
CAN IT AFFECT?

32 — Blast Radius Examples

Low:

ONE REPORT FILE

Medium:

ONE SECURITY TICKET

High:

ONE PRODUCTION USER

Critical:

ALL CORPORATE ACCOUNTS

ALL CLOUD NETWORK RULES

33 — Reduce Blast Radius

Techniques include:

LIMIT SCOPE

LIMIT BATCH SIZE

LIMIT PERMISSIONS

USE ALLOWLISTS

USE CANARY EXECUTION

USE HUMAN APPROVAL

34 — Batch Size Control

Instead of:

PROCESS 10,000 CHANGES

consider:

PROCESS 10

VERIFY

CONTINUE

35 — Canary Execution

Use:

SMALL REPRESENTATIVE SCOPE

before large rollout.

Example:

ONE TEST ACCOUNT
     ↓
ONE DEVELOPMENT ACCOUNT
     ↓
SMALL PRODUCTION GROUP
     ↓
LARGER DEPLOYMENT

36 — Allowlist Design

For write actions, constrain scope.

Example:

APPROVED TEST USERS

APPROVED LAB ASSETS

APPROVED CLOUD PROJECT

37 — Denylist Is Not Enough

Do not rely only on:

DO NOT TOUCH THESE 5 SYSTEMS

Prefer:

ONLY TOUCH THESE
APPROVED SYSTEMS

38 — Failure Mode Analysis

For every automation ask:

WHAT CAN FAIL?

39 — Common Failure Modes

Examples:

API UNAVAILABLE

INVALID JSON

AUTHENTICATION FAILURE

PERMISSION DENIED

DATABASE FAILURE

RATE LIMITING

NETWORK TIMEOUT

STALE DATA

DUPLICATE EVENT

WRONG ASSET MATCH

WRONG USER MATCH

BAD SCORING RULE

PARTIAL SUCCESS

40 — Create Failure Table

Example:

Failure	Result	Safe Behavior
IOC API unavailable	Missing enrichment	Mark unknown
CMDB unavailable	No asset context	Continue with warning
Database unavailable	Cannot persist	Preserve raw data
401 from API	Authentication failure	Stop/review
429 from API	Rate limited	Backoff

41 — Fail Closed vs Fail Open

Decide deliberately.

For example:

REPORT GENERATOR FAILS

may allow:

DATA PIPELINE TO CONTINUE

but:

APPROVAL CHECK FAILS

should generally prevent:

HIGH-IMPACT ACTION

42 — Fail-Safe Principle

For security automation:

UNCERTAINTY
    ↓
MARK UNKNOWN
    ↓
HUMAN REVIEW

is often safer than:

UNCERTAINTY
    ↓
ASSUME SAFE

43 — Design Timeout Behavior

Every network request should have:

TIMEOUT

Avoid:

REQUEST
 ↓
WAIT FOREVER

44 — Retry Review

Retries are appropriate for:

TRANSIENT FAILURE

429

5XX

TEMPORARY NETWORK ERROR

45 — Do Not Retry Everything

Do not repeatedly retry:

401

403

INVALID REQUEST

BAD DATA

46 — Backoff Strategy

Example:

ATTEMPT 1

WAIT 1 SECOND

ATTEMPT 2

WAIT 2 SECONDS

ATTEMPT 3

WAIT 4 SECONDS

STOP

47 — Retry Limit

Always define:

MAXIMUM RETRIES

to avoid runaway workflows.

48 — Rate-Limit Review

Document:

API REQUEST LIMITS

EXPECTED REQUEST VOLUME

RETRY-AFTER SUPPORT

CACHE STRATEGY

49 — Pagination Review

Ask:

CAN THE API RETURN
MULTIPLE PAGES?

If yes:

PAGE 1
 ↓
PAGE 2
 ↓
...
 ↓
END

must be handled safely.

50 — Prevent Infinite Pagination

Set controls such as:

MAX PAGES

MAX RECORDS

EXPECTED TOTAL

51 — Idempotency Review

Ask:

WHAT HAPPENS IF THIS
RUNS TWICE?

52 — Good Idempotency

Example:

RUN 1
→ CREATE ONE CASE

RUN 2
→ DETECT EXISTING CASE
→ DO NOT CREATE ANOTHER

53 — Bad Idempotency

ONE ALERT
+
THREE RETRIES
=
THREE TICKETS

54 — Use Stable Identifiers

Examples:

ALERT ID

EVENT ID

FINDING ID

CASE ID

IDEMPOTENCY KEY

55 — Partial Failure Review

Example:

100 ASSETS

95 SUCCEEDED

5 FAILED

Do not report:

SUCCESS

without highlighting:

PARTIAL FAILURE

56 — Coverage Metrics

Track:

EXPECTED

COLLECTED

FAILED

UNKNOWN

57 — Example

Expected Cloud Accounts: 50

Successfully Assessed: 47

Failed: 3

Coverage: 94%

Do not describe this as:

100% CLOUD VISIBILITY

58 — Human Approval Review

For every write action ask:

DOES THIS REQUIRE
HUMAN APPROVAL?

59 — Approval Usually Recommended For

ACCOUNT DISABLEMENT

SESSION REVOCATION

ENDPOINT ISOLATION

FIREWALL CHANGE

PRODUCTION RESOURCE CHANGE

SECURITY POLICY CHANGE

60 — Approval Workflow

AUTOMATION DETECTS CONDITION
          ↓
AUTOMATION COLLECTS EVIDENCE
          ↓
AUTOMATION PROPOSES ACTION
          ↓
ANALYST REVIEWS
          ↓
APPROVE / REJECT
          ↓
ACTION
          ↓
VERIFY

61 — Approval Evidence

An approver should see:

WHAT HAPPENED?

WHY ACTION IS RECOMMENDED?

WHICH RESOURCE?

WHAT IMPACT?

WHAT ROLLBACK EXISTS?

62 — Avoid Approval Fatigue

Do not create:

500 APPROVAL REQUESTS
PER HOUR

If approval is required that often, reconsider:

SCOPE

DETECTION QUALITY

AUTOMATION DESIGN

63 — Dry Run

Every potentially impactful automation should support:

DRY RUN

when practical.

64 — Dry-Run Output

Example:

PROPOSED ACTION:

Disable account:
training-user-01

Reason:
Approved lab test

NO CHANGE PERFORMED

65 — Preview Before Write

Architecture:

INPUT
 ↓
ANALYSIS
 ↓
PROPOSED CHANGE
 ↓
PREVIEW
 ↓
APPROVAL
 ↓
EXECUTE

66 — Rollback Planning

Before automation changes something, ask:

HOW DO WE UNDO IT?

67 — Rollback Examples

Account disablement:

REENABLE ACCOUNT

Firewall change:

RESTORE PRIOR RULE

Configuration change:

RESTORE PREVIOUS CONFIG

68 — Store Previous State

Before changing configuration, capture:

CURRENT VALUE

so it can be restored.

69 — Rollback Limitations

Some actions are difficult or impossible to reverse.

Example:

DELETE DATA

Therefore:

IRREVERSIBLE ACTION
=
VERY HIGH AUTOMATION RISK

70 — Prefer Reversible Actions

Where possible prefer:

QUARANTINE

DISABLE

TAG

MARK

CREATE TICKET

over:

DELETE

71 — Verification

Do not assume:

API RETURNED 200
=
DESIRED RESULT ACHIEVED

Verify the resulting state.

72 — Verification Flow

ACTION
 ↓
API RESPONSE
 ↓
READ CURRENT STATE
 ↓
EXPECTED?
 ↓
SUCCESS

73 — Example

Automation requests:

ENABLE SECURITY LOGGING

Then verify:

IS LOGGING ACTUALLY ENABLED?

74 — Define Success State

Every write action should define:

EXPECTED STATE AFTER ACTION

75 — Define Failure State

Also define:

WHAT COUNTS AS FAILURE?

76 — Testing Strategy

Never test new security automation first against:

FULL PRODUCTION

77 — Testing Levels

UNIT TEST

SYNTHETIC DATA TEST

LOCAL LAB

INTEGRATION TEST

STAGING

CANARY

CONTROLLED PRODUCTION

78 — Unit Testing

Test individual functions such as:

NORMALIZATION

SCORING

VALIDATION

FILTERING

DEDUPLICATION

79 — Negative Testing

Test bad data:

EMPTY FIELD

INVALID JSON

BAD TIMESTAMP

UNKNOWN USER

UNKNOWN ASSET

INVALID SCORE

80 — Failure Testing

Simulate:

API OFFLINE

DATABASE DOWN

TIMEOUT

429

500

401

403

81 — Scale Testing

Test with:

10 RECORDS

1,000 RECORDS

100,000 RECORDS

when relevant.

82 — Concurrency Review

If processing in parallel ask:

CAN TWO WORKERS
CHANGE THE SAME RESOURCE?

83 — Race Conditions

Example:

WORKER 1
CREATES CASE

WORKER 2
CREATES SAME CASE

Use:

LOCKING

IDEMPOTENCY

DATABASE CONSTRAINTS

as appropriate.

84 — Test Permissions

Run automation using:

ACTUAL INTENDED SERVICE IDENTITY

not an administrator account.

85 — Permission Failure Is Valuable

If the automation fails because it lacks access:

DO NOT IMMEDIATELY
GRANT ADMIN

Determine the exact missing permission.

86 — Logging Requirements

Your automation should log:

START

END

INPUT COUNT

OUTPUT COUNT

FAILURES

RETRIES

APPROVAL

ACTION

VERIFICATION

87 — Do Not Log Secrets

Never log:

PASSWORDS

API TOKENS

PRIVATE KEYS

SESSION TOKENS

AUTHORIZATION HEADERS

88 — Structured Logging

Prefer fields such as:

{
  "event": "automation_run",
  "status": "success",
  "records_processed": 125
}

89 — Correlation ID

Generate a:

RUN ID

or:

REQUEST ID

so all related logs can be traced.

90 — Audit Trail

Important automation actions should record:

WHO / WHAT INITIATED

WHEN

WHY

INPUT

DECISION

APPROVAL

ACTION

RESULT

91 — Monitoring

Automation itself must be monitored.

Track:

RUN SUCCESS

FAILURE RATE

PROCESSING TIME

API ERRORS

DATABASE ERRORS

QUEUE DEPTH

RETRY RATE

92 — Automation Health

A broken security automation can silently create:

VISIBILITY GAPS

Therefore monitor it like:

SECURITY INFRASTRUCTURE

93 — Alert on Automation Failure

Examples:

NO DATA COLLECTED

PIPELINE FAILED

API AUTHENTICATION FAILURE

OUTPUT NOT GENERATED

94 — Avoid Alert Loops

Do not create:

AUTOMATION FAILURE ALERT
      ↓
AUTOMATION PROCESSES ITS OWN ALERT
      ↓
ANOTHER ALERT

Design clear exclusions.

95 — Performance Baseline

Document normal values.

Example:

NORMAL RUN:
2 MINUTES

NORMAL RECORD COUNT:
5,000–7,000

Large deviations can indicate:

SOURCE FAILURE

PIPELINE BUG

DATA SURGE

96 — Change Management

Treat automation changes like other security engineering changes.

Review:

CODE

PERMISSIONS

POLICIES

RISK WEIGHTS

API ENDPOINTS

OUTPUT BEHAVIOR

97 — Version Control

Use Git for:

CODE

CONFIGURATION

POLICY

DOCUMENTATION

98 — Never Commit Secrets

Use:

.gitignore

SECRET SCANNING

PRE-COMMIT CHECKS

where appropriate.

99 — Code Review

At least one reviewer should understand:

SECURITY IMPACT

FAILURE BEHAVIOR

PRIVILEGE

ROLLBACK

not only syntax.

100 — Risk Model Review

If the automation assigns risk scores, document:

INPUT FACTORS

WEIGHTS

THRESHOLDS

MODEL VERSION

101 — Explainability Requirement

Avoid:

RISK SCORE = 93

REASON = UNKNOWN

Prefer:

HIGH ALERT
+40

PRIVILEGED USER
+15

CRITICAL ASSET
+15

IOC CONTEXT
+10

102 — Avoid Double Counting

Check whether:

VENDOR SEVERITY

already includes:

ASSET CRITICALITY

before adding criticality again.

103 — Suppression Review

If automation suppresses events, every suppression should have:

OWNER

JUSTIFICATION

SCOPE

EXPIRATION

REVIEW DATE

104 — Avoid Permanent Suppression

Do not create:

IGNORE FOREVER

for dynamic security environments.

105 — Exception Management

If a control is intentionally violated:

FINDING
 ↓
BUSINESS JUSTIFICATION
 ↓
RISK REVIEW
 ↓
APPROVAL
 ↓
EXPIRATION
 ↓
REVIEW

106 — Data Retention

Document retention for:

RAW INPUT

NORMALIZED DATA

LOGS

REPORTS

DATABASE

AUDIT TRAIL

107 — Privacy Review

Security automation may process:

USERNAMES

IP ADDRESSES

DEVICE ACTIVITY

IDENTITY DATA

INVESTIGATION NOTES

Use approved privacy and retention requirements.

108 — Data Minimization

Collect only:

WHAT THE AUTOMATION NEEDS

not every field available from an API.

109 — Threat Model the Automation

Ask:

WHAT IF INPUT IS MALICIOUS?

WHAT IF API DATA IS WRONG?

WHAT IF CONFIGURATION IS MODIFIED?

WHAT IF AN ATTACKER GAINS THE SERVICE TOKEN?

WHAT IF REPORTS ARE STOLEN?

110 — Threat Model Components

Review:

INPUTS

API CLIENTS

SERVICE IDENTITY

DATABASE

QUEUE

REPORTS

LOGS

ADMIN INTERFACE

111 — Automation as a High-Value Target

Security automation may contain:

CENTRAL SECURITY DATA

POWERFUL API ACCESS

CLOUD VISIBILITY

IDENTITY INFORMATION

Protect it accordingly.

112 — Configuration Integrity

Protect:

SCORING RULES

ALLOWLISTS

SUPPRESSIONS

APPROVAL POLICY

AUTOMATION SETTINGS

113 — Change Detection

Consider monitoring changes to:

AUTOMATION CODE

POLICY FILES

SERVICE PERMISSIONS

114 — Emergency Stop

High-impact automation should have a:

KILL SWITCH

or rapid disable mechanism.

115 — Kill Switch Requirements

Operators should know:

HOW TO STOP IT

WHO CAN STOP IT

WHAT HAPPENS TO QUEUED ACTIONS

HOW TO RESTART SAFELY

116 — Queue Control

If the automation uses queues, consider:

PAUSE

DRAIN

RETRY

DEAD-LETTER QUEUE

117 — Dead-Letter Queue

Failed work items can be moved to:

REVIEW QUEUE

instead of being:

LOST

or retried forever.

118 — Recovery Planning

Document how to recover from:

DATABASE CORRUPTION

API CREDENTIAL FAILURE

SERVICE CRASH

BAD DEPLOYMENT

NETWORK FAILURE

119 — Runbook Requirement

Every production automation should have operational documentation covering:

START

STOP

HEALTH CHECK

FAILURE RECOVERY

ROLLBACK

ESCALATION

120 — Ownership

Every automation must have:

TECHNICAL OWNER

BUSINESS / SECURITY OWNER

121 — Why Ownership Matters

Without ownership:

WHO FIXES IT?

WHO APPROVES CHANGES?

WHO REVIEWS FAILURES?

WHO DECIDES RETIREMENT?

122 — Support Model

Define:

BUSINESS HOURS?

24x7?

ON-CALL?

BEST EFFORT?

123 — Dependency Ownership

Document dependencies such as:

SIEM

CLOUD API

CMDB

DATABASE

IDENTITY PROVIDER

and who owns them.

124 — Decommissioning Plan

Automation should not exist forever without review.

Ask periodically:

IS THIS STILL NEEDED?

IS THE DATA SOURCE STILL VALID?

IS THE PROCESS STILL CORRECT?

125 — Review Cadence

Possible review areas:

PERMISSIONS

RISK MODEL

DEPENDENCIES

SUPPRESSIONS

FAILURE RATES

BUSINESS NEED

126 — Automation Maturity Levels

LEVEL 1
MANUAL

LEVEL 2
SCRIPTED

LEVEL 3
SCHEDULED

LEVEL 4
INTEGRATED

LEVEL 5
ORCHESTRATED

LEVEL 6
CONTROLLED RESPONSE

127 — Level 1: Manual

Analyst performs:

EVERY STEP

Useful when the process is:

NEW

RARE

HIGH-RISK

128 — Level 2: Scripted

A script assists:

ON DEMAND

Example:

GENERATE HOST SECURITY REPORT

129 — Level 3: Scheduled

Automation runs:

DAILY

WEEKLY

HOURLY

Example:

DAILY CLOUD POSTURE REPORT

130 — Level 4: Integrated

Multiple systems communicate via:

APIS

WEBHOOKS

DATABASES

131 — Level 5: Orchestrated

The workflow combines:

SIEM

IDENTITY

CMDB

THREAT INTEL

VULNERABILITY

CASE MANAGEMENT

132 — Level 6: Controlled Response

Automation may execute approved response actions with:

STRICT GOVERNANCE

VERIFICATION

AUDIT

ROLLBACK

APPROVAL

133 — Safety Review Questionnaire

Before approving an automation, answer:

WHAT PROBLEM DOES IT SOLVE?

WHAT DOES IT READ?

WHAT DOES IT CHANGE?

WHAT ACCESS DOES IT REQUIRE?

WHAT CAN FAIL?

WHAT IS THE BLAST RADIUS?

WHAT HAPPENS WITH BAD DATA?

WHAT HAPPENS WITH MISSING DATA?

IS IT IDEMPOTENT?

IS THERE A DRY RUN?

IS THERE HUMAN APPROVAL?

IS THERE ROLLBACK?

HOW IS SUCCESS VERIFIED?

HOW IS IT MONITORED?

WHO OWNS IT?

134 — Automation Risk Matrix

Score each area:

Area	Low	Medium	High
Privilege	Read-only	Limited write	Admin
Blast radius	One record	One system	Enterprise
Reversibility	Easy	Moderate	Difficult
Human approval	Always	Conditional	None
Data quality	Strong	Variable	Unknown

135 — High-Risk Indicators

Escalate design review when automation has:

ADMINISTRATIVE CREDENTIALS

PRODUCTION WRITE ACCESS

LARGE BLAST RADIUS

IRREVERSIBLE ACTIONS

NO HUMAN APPROVAL

WEAK INPUT VALIDATION

NO ROLLBACK

NO MONITORING

136 — Approval Outcome

At the end of review choose:

APPROVED

APPROVED WITH CONDITIONS

REQUIRES REDESIGN

REJECTED

137 — Example: Safe Alert Enrichment

Proposed automation:

READ SIEM ALERT

QUERY CMDB

QUERY IDENTITY

QUERY VULNERABILITY SYSTEM

GENERATE CONTEXT

Assessment:

READ-ONLY

LOW BLAST RADIUS

REVERSIBLE

NO PRODUCTION CHANGE

Likely result:

APPROVE AFTER TESTING

138 — Example: Automatic Account Disablement

Proposed:

IF RISK SCORE > 80
DISABLE USER

Risk:

FALSE POSITIVE

BUSINESS LOCKOUT

SERVICE ACCOUNT IMPACT

RISK MODEL ERROR

IDENTITY MATCHING ERROR

Safer design:

HIGH SCORE
      ↓
COLLECT EVIDENCE
      ↓
CREATE RESPONSE RECOMMENDATION
      ↓
ANALYST APPROVAL
      ↓
CONTROLLED DISABLEMENT
      ↓
VERIFY

139 — Example: Cloud Storage Finding

Proposed:

IF PUBLIC STORAGE FOUND
DELETE STORAGE

Reject this design.

Possible safer workflow:

PUBLIC STORAGE DETECTED
        ↓
CHECK OWNER
        ↓
CHECK BUSINESS PURPOSE
        ↓
CHECK DATA CLASSIFICATION
        ↓
CREATE REVIEW TICKET
        ↓
APPROVED CHANGE

140 — Example: Ticket Automation

Proposed:

CREATE VULNERABILITY TICKET
FOR P1 FINDING

Risks:

DUPLICATE TICKETS

BAD OWNER MAPPING

BAD PRIORITY

Controls:

IDEMPOTENCY

OWNER VALIDATION

PREVIEW

AUDIT LOG

141 — Example: Firewall Automation

High-risk scenario:

IOC MATCH
   ↓
BLOCK IP

Risks:

SHARED INFRASTRUCTURE

CDN

PROXY

FALSE POSITIVE

BUSINESS OUTAGE

Safer approach:

IOC MATCH
   ↓
CORRELATE
   ↓
VALIDATE
   ↓
RECOMMEND BLOCK
   ↓
APPROVAL

142 — Design Review Template

Create:

automation-design-review.md

143 — Section 01 — Automation Overview

Document:

NAME:

OWNER:

PURPOSE:

SECURITY DOMAIN:

ENVIRONMENT:

AUTOMATION LEVEL:

144 — Section 02 — Current Manual Process

Document:

CURRENT STEPS:

CURRENT TIME:

CURRENT ERROR RATE:

CURRENT PAIN POINTS:

145 — Section 03 — Proposed Workflow

Document:

TRIGGER:

INPUTS:

PROCESSING:

OUTPUT:

WRITE ACTIONS:

146 — Section 04 — Data Sources

For each:

SOURCE:

OWNER:

AUTHENTICATION:

FRESHNESS:

FAILURE BEHAVIOR:

147 — Section 05 — Permissions

Document:

SERVICE IDENTITY:

READ PERMISSIONS:

WRITE PERMISSIONS:

ADMIN PERMISSIONS:

JUSTIFICATION:

148 — Section 06 — Failure Modes

Use:

FAILURE:

IMPACT:

DETECTION:

RECOVERY:

149 — Section 07 — Blast Radius

Document:

MAX USERS AFFECTED:

MAX SYSTEMS AFFECTED:

MAX CLOUD RESOURCES AFFECTED:

DATA LOSS POSSIBILITY:

150 — Section 08 — Approval

Document:

APPROVAL REQUIRED?

APPROVER:

ACTION PREVIEW AVAILABLE?

APPROVAL EXPIRY:

151 — Section 09 — Rollback

Document:

ROLLBACK AVAILABLE?

PREVIOUS STATE CAPTURED?

ROLLBACK TESTED?

ROLLBACK OWNER:

152 — Section 10 — Verification

Document:

SUCCESS CONDITION:

VERIFICATION METHOD:

FAILURE CONDITION:

153 — Section 11 — Testing

Document:

UNIT TEST:

SYNTHETIC TEST:

INTEGRATION TEST:

STAGING TEST:

CANARY TEST:

154 — Section 12 — Monitoring

Document:

HEALTH CHECK:

ERROR ALERTING:

METRICS:

LOGGING:

ON-CALL:

155 — Section 13 — Governance

Document:

CODE REVIEW:

POLICY VERSION:

CHANGE CONTROL:

REVIEW CADENCE:

156 — Section 14 — Decision

Use:

APPROVED

APPROVED WITH CONDITIONS

REDESIGN REQUIRED

REJECTED

157 — Pre-Production Checklist

Before production deployment verify:

158 — Production Readiness Gate

Do not approve high-impact automation if any of these are missing:

KNOWN OWNER

LEAST-PRIVILEGE DESIGN

FAILURE HANDLING

VERIFICATION

MONITORING

ROLLBACK OR ACCEPTED IRREVERSIBILITY

APPROVAL MODEL

159 — Review After Deployment

The design review is not the end.

Monitor:

ERROR RATE

FALSE POSITIVES

OPERATOR OVERRIDES

RETRY RATE

FAILED ACTIONS

BUSINESS IMPACT

160 — First 24 Hours

After deployment:

MONITOR CLOSELY

REVIEW LOGS

CHECK OUTPUT

VERIFY PERMISSIONS

CHECK ERROR RATE

161 — First Week

Review:

AUTOMATION VALUE

FAILURES

BAD DATA

ANALYST FEEDBACK

FALSE POSITIVES

PERFORMANCE

162 — Continuous Improvement

Automation lifecycle:

DESIGN
 ↓
BUILD
 ↓
TEST
 ↓
DEPLOY
 ↓
MONITOR
 ↓
LEARN
 ↓
IMPROVE

163 — When to Roll Back

Consider rollback if:

UNEXPECTED PRODUCTION IMPACT

HIGH FAILURE RATE

BAD DATA CORRELATION

INCORRECT ACTIONS

PERMISSION ISSUE

UNCONTROLLED RETRY LOOP

164 — When to Disable Automation

Disable rather than continue when:

SAFETY CONTROLS FAIL

APPROVAL SYSTEM FAILS

DATA QUALITY IS UNTRUSTWORTHY

BLAST RADIUS IS UNCLEAR

165 — Security Automation Decision Tree

IS THE PROCESS UNDERSTOOD?
      ↓
     YES
      ↓
IS THE TASK REPEATABLE?
      ↓
     YES
      ↓
IS DATA RELIABLE ENOUGH?
      ↓
     YES
      ↓
IS THE IMPACT LOW?
    ↙     ↘
  YES     NO
   ↓       ↓
AUTOMATE  ADD APPROVAL
          ↓
      CAN IT BE
      ROLLED BACK?
       ↙      ↘
     YES      NO
      ↓        ↓
   CONTROL   STRONGER
   DESIGN    GOVERNANCE

166 — Key Decision Principle

HIGHER IMPACT
=
STRONGER CONTROLS

167 — Safety Equation

AUTOMATION VALUE
        +
RELIABLE DATA
        +
LEAST PRIVILEGE
        +
TESTING
        +
MONITORING
        +
APPROVAL
        +
ROLLBACK
        =
SAFER SECURITY AUTOMATION

168 — Final Mental Model

Whenever someone says:

LET'S AUTOMATE THIS

ask:

WHAT EXACTLY?
      ↓
WHY?
      ↓
WHAT DATA?
      ↓
WHAT ACCESS?
      ↓
WHAT CAN GO WRONG?
      ↓
HOW MANY THINGS
CAN IT AFFECT?
      ↓
CAN WE DRY RUN?
      ↓
WHO APPROVES?
      ↓
CAN WE ROLL BACK?
      ↓
HOW DO WE VERIFY?
      ↓
HOW DO WE MONITOR?
      ↓
WHO OWNS IT?

Runbook Outcome

At the end of this runbook, you should have one of four clear outcomes:

APPROVED FOR IMPLEMENTATION

or:

APPROVED WITH SAFETY CONDITIONS

or:

REDESIGN REQUIRED

or:

AUTOMATION SHOULD NOT PROCEED

What’s Next?

➡️ Runbook 02 — Security Data Ingestion and Normalization

The next runbook defines the operational methodology for safely bringing security data from multiple sources into one automation pipeline.

You will cover:

SOURCE IDENTIFICATION
      ↓
COLLECTION
      ↓
SCHEMA VALIDATION
      ↓
DATA QUALITY
      ↓
NORMALIZATION
      ↓
DEDUPLICATION
      ↓
TIMESTAMP HANDLING
      ↓
IDENTITY / ASSET MATCHING
      ↓
PROVENANCE
      ↓
STORAGE
      ↓
PIPELINE HEALTH

The goal will be to establish one repeatable process for handling security data before it reaches analytics, correlation, risk scoring, or response automation.