Runbook 03 — Enterprise AI Threat Model Review
Creating an AI threat model is only the first step.
Before an enterprise AI system moves into production, the threat model must be reviewed to determine whether:
- The architecture is represented correctly
- Important assets have been identified
- Sensitive data flows are understood
- Trust boundaries are complete
- Relevant AI threats have been considered
- RAG and AI agent risks are addressed
- Attack paths are realistic
- Security controls actually break those attack paths
- Residual risks have accountable owners
This runbook provides a repeatable methodology for performing that review.
The central question is:
Does this threat model give us enough confidence that the important security risks of this AI system are understood and appropriately controlled?
Runbook Information
Section titled “Runbook Information”| Item | Details |
|---|---|
| Runbook | Runbook 03 — Enterprise AI Threat Model Review |
| Learning Path | AI Security Engineer |
| Module | 03 — AI Threat Modeling |
| Type | Enterprise Security Architecture Review |
| Primary Use | AI Threat Model Validation |
| Audience | AI Security Engineers, Security Architects, AppSec, Cloud Security, GRC |
| Output | Review Findings + Risk Decisions + Approval Recommendation |
Purpose
Section titled “Purpose”Use this runbook to independently review an AI threat model before:
- Production deployment
- Security architecture approval
- Adding sensitive enterprise data
- Adding RAG sources
- Enabling AI agents
- Adding privileged tools
- Changing foundation models
- Introducing external AI providers
- Expanding an AI pilot
- Major architecture changes
Review Philosophy
Section titled “Review Philosophy”A threat model should not be approved because:
The Diagram Looks Completeor:
OWASP Risks Were ListedThe review should determine whether the model explains:
What Are We Protecting? ↓Who or What Could Threaten It? ↓Where Could They Enter? ↓Which Trust Boundary Could Be Crossed? ↓What Attack Path Could Result? ↓What Would the Business Impact Be? ↓Which Security Control Stops It?Review Outcomes
Section titled “Review Outcomes”The final review should result in one of four outcomes:
Approved
Approved With Conditions
Rework Required
Not AcceptableReview Workflow
Section titled “Review Workflow”Use the following sequence:
01 Review Scope ↓02 Validate Business Context ↓03 Validate Architecture ↓04 Validate Assets ↓05 Validate Data Flows ↓06 Validate Trust Boundaries ↓07 Validate Threat Actors ↓08 Validate Threat Coverage ↓09 Review RAG Threats ↓10 Review Agent Threats ↓11 Review Attack Paths ↓12 Review Risk Ratings ↓13 Validate Security Controls ↓14 Review Residual Risk ↓15 Validate Ownership ↓16 Make Review DecisionPhase 01 — Confirm Review Scope
Section titled “Phase 01 — Confirm Review Scope”Before reviewing the threat model, establish exactly what system is being assessed.
Document:
Application:
Business Owner:
Technical Owner:
Security Owner:
Environment:
Architecture Version:
Threat Model Version:
Review Date:
Reviewer:Confirm whether the review includes:
-
LLM application
-
Foundation model
-
Model APIs
-
RAG
-
Vector database
-
Embedding model
-
AI memory
-
AI agents
-
Agent tools
-
Enterprise APIs
-
Cloud infrastructure
-
External AI providers
-
Training or fine-tuning pipelines
Scope Validation
Section titled “Scope Validation”Compare:
Production Architecture vsThreat Model ArchitectureEvery security-relevant production component should appear in the threat model.
Review Question
Section titled “Review Question”Ask:
Is anything capable of storing data, processing data, making decisions or performing actions missing from the threat model?
Phase 02 — Validate Business Context
Section titled “Phase 02 — Validate Business Context”The reviewer should understand what the AI system actually does.
Document:
Business Purpose:
Intended Users:
Information Processed:
Decisions Supported:
Actions Performed:
Critical Business Processes:
Potential Business Impact:Business Context Questions
Section titled “Business Context Questions”Ask:
Why does the organization need this AI system?
Who will use it?
What information will it process?
What decisions will it influence?
Can it perform actions?
Can those actions affect production?
What happens if the AI is wrong?
What happens if the AI is manipulated?Risk Context
Section titled “Risk Context”A chatbot answering public FAQs does not require the same threat model as:
Security Analyst ↓AI SOC Agent ↓Cloud Security Tools ↓Production EnvironmentThe threat model must reflect actual business impact.
Phase 03 — Validate Architecture
Section titled “Phase 03 — Validate Architecture”Compare the threat-model diagram with the implementation.
A representative architecture might include:
User │ ▼ Identity Provider │ ▼ AI Application │ ▼ LLM ┌─────────────┼─────────────┐ │ │ │ ▼ ▼ ▼ RAG Memory AI Agent │ │ ▼ ▼ Vector DB Tool Layer │ │ ▼ ▼ Enterprise Knowledge Enterprise SystemsArchitecture Review Checklist
Section titled “Architecture Review Checklist”Verify the diagram includes:
-
Users
-
Authentication
-
Application
-
LLM
-
Model provider
-
RAG
-
Vector database
-
Embedding service
-
Memory
-
Agents
-
Tools
-
External APIs
-
Enterprise systems
-
Logging
-
Administrative interfaces
Common Finding
Section titled “Common Finding”Finding:Threat Model Does Not Represent Production Architecture
Impact:Security-relevant components may not have been assessed.
Recommendation:Update the threat model to match the deployed architecturebefore completing security approval.Phase 04 — Validate Asset Identification
Section titled “Phase 04 — Validate Asset Identification”Review the asset register.
Potential AI assets include:
Enterprise Data
Customer Data
Employee Data
Source Code
Security Runbooks
Credentials
API Keys
Agent Identities
Models
Datasets
Vector Databases
Prompts
AI Memory
Production InfrastructureAsset Review Questions
Section titled “Asset Review Questions”For every major component ask:
What valuable information exists here?
What would happen if it were disclosed?
What would happen if it were modified?
What would happen if it became unavailable?Asset Register
Section titled “Asset Register”| Asset | Classification | Owner | Security Objective |
|---|---|---|---|
| HR Documents | Confidential | HR | Confidentiality |
| Security Runbooks | Confidential | Security | C + I |
| Model Registry | Restricted | AI Platform | Integrity |
| Agent Identity | Restricted | Platform | C + I |
| Production Cloud | Critical | Cloud | C + I + A |
Phase 05 — Validate Data Classification
Section titled “Phase 05 — Validate Data Classification”Review whether important information has been classified.
Typical classifications:
Public
Internal
Confidential
RestrictedCheck:
-
Prompts
-
Responses
-
RAG documents
-
Embeddings
-
Vector metadata
-
Memory
-
Logs
-
Training datasets
-
Fine-tuning datasets
Important Review Question
Section titled “Important Review Question”Ask:
Does the threat model understand where sensitive information enters, travels, persists and leaves the system?
Phase 06 — Validate Data Flows
Section titled “Phase 06 — Validate Data Flows”Review all major flows.
Example:
User Prompt ↓Application ↓LLM ProviderAnother:
Enterprise Document ↓Ingestion ↓Embedding ↓Vector DatabaseAnother:
LLM ↓Agent ↓Cloud APIFor every flow document:
Source
Destination
Data
Classification
Authentication
Authorization
Encryption
LoggingExternal Data Flows
Section titled “External Data Flows”Pay particular attention to:
Enterprise ↓External AI ProviderDetermine:
-
What leaves the organization?
-
Why?
-
What classification?
-
Is it retained?
-
Can it be used for training?
-
Is the provider approved?
Phase 07 — Validate Trust Boundaries
Section titled “Phase 07 — Validate Trust Boundaries”Trust boundaries are one of the most important parts of the review.
Look for boundaries such as:
Internet ↓AI ApplicationUser ↓LLMExternal Content ↓RAGRAG ↓LLM ContextLLM ↓AgentAgent ↓Production SystemTenant A ↓Shared AI Infrastructure ↓Tenant BTrust Boundary Review Questions
Section titled “Trust Boundary Review Questions”Ask:
What crosses this boundary?
Why is the source trusted?
How is identity established?
How is authorization enforced?
What happens if the input is malicious?
What telemetry exists?Common Finding
Section titled “Common Finding”Finding:LLM-to-Agent Boundary Not Identified
Risk:Model-generated instructions may directly influenceprivileged enterprise actions without sufficientindependent validation.Phase 08 — Validate Threat Actors
Section titled “Phase 08 — Validate Threat Actors”Review whether realistic threat actors have been considered.
Include where applicable:
-
Anonymous external attacker
-
Authenticated malicious user
-
Compromised employee
-
Insider
-
Compromised administrator
-
Compromised service account
-
Supply-chain attacker
-
Malicious content author
-
Compromised external provider
AI introduces another important source:
Malicious ContentExamples:
Document
Website
Email
API Response
RAG SourceThese can influence AI behavior even when the attacker never directly interacts with the model.
Phase 09 — Validate Threat Coverage
Section titled “Phase 09 — Validate Threat Coverage”Review whether the team used appropriate threat references.
Useful approaches include:
STRIDE+AI-Specific Threat Scenarios+OWASP GenAI Risks+MITRE ATLASThe objective is not to copy every possible threat.
The objective is to avoid obvious gaps.
STRIDE Coverage
Section titled “STRIDE Coverage”Review:
Spoofing
Tampering
Repudiation
Information Disclosure
Denial of Service
Elevation of Privilegeagainst major components.
AI-Specific Coverage
Section titled “AI-Specific Coverage”Review whether the threat model considers:
Prompt Injection
Indirect Prompt Injection
Sensitive Information Exposure
RAG Poisoning
Model Poisoning
Excessive Agency
Insecure Output Handling
Model Supply Chain
Vector Database Security
Cross-Tenant Exposure
Agent Privilege EscalationPhase 10 — Review Prompt Injection Threats
Section titled “Phase 10 — Review Prompt Injection Threats”Do not accept a threat model that says only:
Threat:Prompt Injection
Mitigation:System PromptThe reviewer should ask:
What happens after Prompt Injection succeeds?
Model:
Prompt Injection ↓Model Manipulated ↓What Can It Reach?Possible consequences:
Sensitive Context
RAG
Memory
Agent Tools
Enterprise APIsStronger Threat Modeling
Section titled “Stronger Threat Modeling”Threat:Prompt Injection manipulates AI agent.
Attack Path:User→ Prompt Injection→ LLM→ Agent Tool→ Production Cloud
Control:Independent Tool Authorization+Least Privileged Agent Identity+Approval for State-Changing ActionsPhase 11 — Review RAG Threats
Section titled “Phase 11 — Review RAG Threats”If RAG exists, verify coverage of:
-
Unauthorized retrieval
-
Lost source permissions
-
Cross-tenant retrieval
-
RAG poisoning
-
Indirect Prompt Injection
-
Metadata exposure
-
Vector database compromise
-
Stale permissions
-
Deleted content
-
Untrusted ingestion
RAG Threat Path
Section titled “RAG Threat Path”Compromised User ↓Knowledge Source ↓Poisoned Document ↓RAG ↓LLMIf agents exist:
Poisoned Document ↓RAG ↓LLM ↓Agent ↓Enterprise ToolThis second path deserves significantly more attention.
RAG Review Questions
Section titled “RAG Review Questions”Ask:
Who can add knowledge?
Who can modify it?
Are permissions preserved?
Is retrieval authorization-aware?
Can tenants cross boundaries?
Can external content enter RAG?
Can poisoned content influence agents?
Can compromised content be removed?Phase 12 — Review AI Agent Threats
Section titled “Phase 12 — Review AI Agent Threats”Agents require deeper scrutiny because they convert:
AI Outputinto:
Enterprise ActionReview:
-
Tool inventory
-
Agent identity
-
Agent permissions
-
Tool authorization
-
Tool parameters
-
Human approval
-
Tool chaining
-
Agent memory
-
Multi-agent communication
-
Emergency shutdown
Excessive Agency Review
Section titled “Excessive Agency Review”Evaluate:
Functionality×Privilege×Autonomy×ImpactA read-only knowledge assistant and an autonomous cloud administrator are not equivalent risks.
Agent Review Question
Section titled “Agent Review Question”Ask:
What is the worst thing this agent could do if the LLM were completely manipulated?
Then determine which independent control prevents it.
Phase 13 — Review Model and Supply-Chain Threats
Section titled “Phase 13 — Review Model and Supply-Chain Threats”Verify the threat model includes:
Foundation Model
Embedding Model
Libraries
AI Frameworks
Containers
Datasets
Model Registry
External APIsPotential threats include:
-
Model replacement
-
Malicious model
-
Compromised dependency
-
Poisoned dataset
-
Unauthorized registry access
-
Unapproved model version
-
External provider compromise
Model Integrity Path
Section titled “Model Integrity Path”Attacker ↓Model Registry ↓Modified Model ↓Production AISecurity controls may include:
Restricted Registry Access
Artifact Integrity
Version Control
Approval
Deployment LoggingPhase 14 — Review Identity Threats
Section titled “Phase 14 — Review Identity Threats”Verify the threat model considers:
User Identity
Service Identity
Agent Identity
Ingestion Identity
Retrieval Identity
Deployment IdentityCheck for:
-
Shared identities
-
Excessive permissions
-
Credential theft
-
Privilege escalation
-
Weak service authentication
Phase 15 — Review Multi-Tenant Threats
Section titled “Phase 15 — Review Multi-Tenant Threats”If the AI system serves multiple customers, departments or business units, validate:
Tenant Identity
Tenant Data
Tenant Vector Namespace
Tenant Memory
Tenant Tool ScopeAttack path:
Tenant A User ↓Weak Tenant Filter ↓Shared Vector Store ↓Tenant B DataCross-tenant exposure should generally receive high review priority.
Phase 16 — Review Attack Paths
Section titled “Phase 16 — Review Attack Paths”This is one of the most important phases.
Individual threats should be connected into realistic paths.
Example 1 — RAG Exposure
Section titled “Example 1 — RAG Exposure”Employee ↓Weak Retrieval Authorization ↓Restricted HR Document ↓LLM Context ↓Sensitive DisclosureExample 2 — Agent Compromise
Section titled “Example 2 — Agent Compromise”Prompt Injection ↓LLM Manipulation ↓Overprivileged Agent ↓Cloud Tool ↓Production ModificationExample 3 — Indirect Injection
Section titled “Example 3 — Indirect Injection”Attacker ↓Malicious Website ↓AI Agent Reads Website ↓Indirect Prompt Injection ↓Agent Tool ↓Enterprise ActionExample 4 — Supply Chain
Section titled “Example 4 — Supply Chain”Compromised Registry ↓Modified Model ↓Production Deployment ↓AI Integrity FailureAttack Path Review Template
Section titled “Attack Path Review Template”Attack Path ID:
Threat Actor:
Entry Point:
Asset:
Precondition:
Trust Boundary:
Weakness:
Privilege Required:
Security Controls:
Business Impact:
Risk Rating:Phase 17 — Validate Risk Ratings
Section titled “Phase 17 — Validate Risk Ratings”Do not accept severity simply because:
It Is an AI VulnerabilityReview:
Likelihood
Exploitability
Required Access
Data Sensitivity
Privilege
Autonomy
Blast Radius
Persistence
Business ImpactRisk Example
Section titled “Risk Example”Model follows unintended instruction.
No data exposed.
No tools available.
No authorization bypass.Potential rating:
Low / InformationalCompare with:
Indirect Prompt Injection ↓Privileged Autonomous Agent ↓Production Infrastructure ModificationPotential rating:
CriticalPhase 18 — Validate Security Controls
Section titled “Phase 18 — Validate Security Controls”For every High or Critical threat, ask:
Which control actually breaks this attack path?
Example:
Threat:Unauthorized RAG Access
Control:Authorization Before RetrievalAnother:
Threat:Prompt Injection → Privileged Tool
Controls:Least Privilege+Independent Tool Authorization+Human ApprovalControl Categories
Section titled “Control Categories”Classify controls as:
Preventive
Section titled “Preventive”Examples:
Authorization
Least Privilege
Network Isolation
Input ValidationDetective
Section titled “Detective”Examples:
Security Logging
SIEM Detection
Model Change Monitoring
Agent Activity MonitoringResponsive
Section titled “Responsive”Examples:
Agent Kill Switch
Credential Revocation
RAG Source Removal
Model RollbackPhase 19 — Validate Control Independence
Section titled “Phase 19 — Validate Control Independence”AI security controls should not all depend on the same model.
Weak:
LLM ↓Decides Whether Action Is Safe ↓Executes ActionBetter:
LLM ↓Requests Action ↓Deterministic Authorization ↓Policy ↓Approval ↓ExecutionCritical Review Principle
Section titled “Critical Review Principle”A security control protecting a critical enterprise boundary should not depend only on the same LLM that an attacker may manipulate.
Phase 20 — Review Defense in Depth
Section titled “Phase 20 — Review Defense in Depth”Ask:
If Prompt Filtering Fails, What Stops the Attack?
If the Model Fails, What Stops the Attack?
If RAG Is Poisoned, What Stops the Attack?
If the Agent Requests a Dangerous Tool, What Stops the Attack?A strong architecture has multiple independent layers.
Phase 21 — Review Monitoring Coverage
Section titled “Phase 21 — Review Monitoring Coverage”Threat models should include detection where prevention cannot provide complete assurance.
Verify monitoring for:
Repeated Prompt Abuse
Restricted Retrieval Attempts
Cross-Tenant Access
Unexpected Agent Tools
Privileged Agent Actions
RAG Source Changes
Model Changes
Bulk Vector ExportPhase 22 — Review Incident Response
Section titled “Phase 22 — Review Incident Response”Ask whether identified threats can actually be contained.
Verify teams can:
Disable AI Application
Disable Agent
Revoke Agent Credentials
Disable Tool
Remove RAG Source
Delete Poisoned Vectors
Rollback Model
Restore DatasetThreat-to-Response Mapping
Section titled “Threat-to-Response Mapping”Example:
Threat:RAG Poisoning
Detection:Unexpected Source Modification
Containment:Disable Source
Eradication:Remove Poisoned Content
Recovery:Rebuild Index
Validation:Retest RetrievalPhase 23 — Review Residual Risk
Section titled “Phase 23 — Review Residual Risk”Some risk will remain after controls.
Document:
Threat:
Original Risk:
Controls:
Residual Risk:
Risk Owner:
Decision:Possible decisions:
Accept
Mitigate
Transfer
AvoidResidual Risk Example
Section titled “Residual Risk Example”Threat:Indirect Prompt Injection
Original Risk:High
Controls:Read-only agentIndependent tool authorizationApproved source filteringMonitoring
Residual Risk:Low
Decision:AcceptPhase 24 — Validate Risk Ownership
Section titled “Phase 24 — Validate Risk Ownership”Every unresolved material risk should have:
Owner
Remediation
Target Date
DecisionAvoid:
High Risk
Owner:UnknownPhase 25 — Validate Threat-to-Control Traceability
Section titled “Phase 25 — Validate Threat-to-Control Traceability”Create a matrix.
| Threat | Asset | Risk | Control | Owner | Status |
|---|---|---|---|---|---|
| Unauthorized RAG Access | HR Data | High | Retrieval Authorization | AI Platform | Open |
| Agent Privilege Escalation | Cloud | Critical | Least Privilege | Cloud Security | Open |
| Model Replacement | Model | High | Registry IAM | ML Platform | Implemented |
| Sensitive Prompt Logging | User Data | Medium | Log Redaction | Operations | Partial |
This gives the reviewer visibility into whether identified risks have real controls.
Phase 26 — Review Security Requirements
Section titled “Phase 26 — Review Security Requirements”Threat modeling should produce engineering requirements.
Examples:
SEC-AI-001
RAG retrieval MUST enforce document-levelauthorization using trusted user identity.SEC-AI-002
AI agents MUST use dedicatedleast-privileged workload identities.SEC-AI-003
State-changing agent actions MUST pass throughindependent authorization before execution.SEC-AI-004
Production RAG sources MUST maintaindocument provenance.SEC-AI-005
Production model changes MUST beauthenticated, authorized and logged.Phase 27 — Validate Testing Requirements
Section titled “Phase 27 — Validate Testing Requirements”Important threats should produce security test cases.
Example:
Threat:Cross-Tenant RAG Exposure
Security Requirement:Tenant-aware retrieval
Test:Tenant A attempts semantic retrievalof Tenant B synthetic data.
Expected:DENYThis creates:
Threat ↓Security Requirement ↓Security Control ↓Security Test ↓EvidenceThat is a mature threat-modeling process.
Phase 28 — Identify Threat Model Gaps
Section titled “Phase 28 — Identify Threat Model Gaps”Common gaps include:
Missing Trust Boundary
Missing External Provider
Missing Agent Tool
Missing RAG Source
Missing Tenant Boundary
Missing Service Identity
Missing Data Flow
Missing Attack Path
Missing Detection
Missing Incident ResponseDocument each gap as a review finding.
Review Finding Template
Section titled “Review Finding Template”Finding ID:
Title:
Severity:
Threat Model Section:
Observation:
Security Impact:
Recommendation:
Owner:
Target Date:
Status:Example Review Finding
Section titled “Example Review Finding”Finding:AI Agent Tool Boundary Missing From Threat Model
Severity:High
Observation:The architecture includes an AI agent capable ofcalling cloud administration APIs, but the threatmodel ends at the LLM.
Security Impact:Prompt Injection and agent privilege escalationpaths have not been evaluated.
Recommendation:Extend the threat model through the agent, tool,workload identity and target cloud resources.
Owner:AI Platform TeamPhase 29 — Review Completeness Matrix
Section titled “Phase 29 — Review Completeness Matrix”Use:
| Area | Status |
|---|---|
| Business Context | Pass |
| Architecture | Pass |
| Assets | Pass |
| Data Classification | Partial |
| Trust Boundaries | Pass |
| Threat Actors | Pass |
| STRIDE | Pass |
| OWASP GenAI | Pass |
| MITRE ATLAS | Partial |
| RAG | Pass |
| AI Agents | Fail |
| Supply Chain | Partial |
| Attack Paths | Fail |
| Security Controls | Partial |
| Monitoring | Pass |
| Incident Response | Partial |
Statuses:
Pass
Partial
Fail
Not Applicable
Not ReviewedPhase 30 — Determine Review Outcome
Section titled “Phase 30 — Determine Review Outcome”Approved
Section titled “Approved”Use when:
Architecture accurately represented
Important threats identified
Critical attack paths analyzed
Security controls defined
No unacceptable residual riskApproved With Conditions
Section titled “Approved With Conditions”Use when:
Core threat model is acceptable
Minor gaps remain
Owners assigned
Remediation dates agreedRework Required
Section titled “Rework Required”Use when:
Important architecture missing
Threat coverage incomplete
Attack paths missing
Controls not mappedNot Acceptable
Section titled “Not Acceptable”Use when:
Critical risks misunderstood
Critical attack paths ignored
No security controls
No risk ownership
Architecture substantially inaccuratePhase 31 — Review Decision Template
Section titled “Phase 31 — Review Decision Template”Threat Model:
Application:
Version:
Reviewer:
Review Date:
Overall Assessment:
[ ] Approved
[ ] Approved With Conditions
[ ] Rework Required
[ ] Not Acceptable
Critical Findings:
High Findings:
Required Actions:
Risk Owners:
Required Retesting:
Next Review:Phase 32 — Reassessment Triggers
Section titled “Phase 32 — Reassessment Triggers”The threat model should be reviewed again when:
Foundation Model Changes
New RAG Source Added
Sensitive Data Added
New AI Agent Added
New Tool Added
Agent Permission Changed
External Provider Changed
Authentication Changed
Tenant Architecture Changed
Major Security Incident OccursEnterprise Review Checklist
Section titled “Enterprise Review Checklist”-
Application identified.
-
Architecture version recorded.
-
Threat-model version recorded.
-
Production scope confirmed.
Business
Section titled “Business”-
Business use case understood.
-
Users identified.
-
AI decisions identified.
-
Business impact understood.
Architecture
Section titled “Architecture”-
Production architecture represented.
-
Models identified.
-
RAG identified.
-
Agents identified.
-
Tools identified.
-
External providers identified.
Assets
Section titled “Assets”-
Critical assets identified.
-
Sensitive data identified.
-
Credentials identified.
-
Models and datasets identified.
-
Data classifications defined.
-
Data flows documented.
-
External processing identified.
-
Storage locations identified.
Trust Boundaries
Section titled “Trust Boundaries”-
User boundary identified.
-
External content boundary identified.
-
Model boundary identified.
-
RAG boundary identified.
-
Agent boundary identified.
-
Tenant boundary identified where applicable.
Threat Actors
Section titled “Threat Actors”-
External attacker considered.
-
Authenticated malicious user considered.
-
Insider considered.
-
Compromised identity considered.
-
Supply-chain attacker considered.
-
Malicious content considered.
AI Threats
Section titled “AI Threats”-
Prompt Injection considered.
-
Indirect Prompt Injection considered.
-
Sensitive information exposure considered.
-
RAG poisoning considered.
-
Excessive agency considered.
-
Insecure output handling considered.
-
Model supply-chain threats considered.
-
Source trust modeled.
-
Ingestion modeled.
-
Authorization modeled.
-
Vector database modeled.
-
Tenant isolation modeled.
-
Poisoning modeled.
Agents
Section titled “Agents”-
Tools inventoried.
-
Agent identity modeled.
-
Permissions modeled.
-
Autonomy modeled.
-
Tool authorization modeled.
-
Approval modeled.
-
Kill switch considered.
Supply Chain
Section titled “Supply Chain”-
Model provenance considered.
-
Model registry considered.
-
Dependencies considered.
-
Dataset integrity considered.
Attack Paths
Section titled “Attack Paths”-
Entry points identified.
-
Preconditions identified.
-
Trust-boundary crossings identified.
-
Privileges identified.
-
Business impact identified.
-
Controls identified.
-
Risk ratings justified.
-
Business impact considered.
-
Residual risk documented.
-
Risk owners assigned.
Controls
Section titled “Controls”-
Preventive controls identified.
-
Detective controls identified.
-
Responsive controls identified.
-
Critical controls independent of LLM behavior.
Monitoring
Section titled “Monitoring”-
Important threat scenarios monitored.
-
Agent activity visible.
-
RAG changes visible.
-
Model changes visible.
Incident Response
Section titled “Incident Response”-
AI can be contained.
-
Agent credentials can be revoked.
-
Poisoned RAG content can be removed.
-
Model can be rolled back.
-
Recovery is possible.
Governance
Section titled “Governance”-
Threat-to-control mapping exists.
-
Security requirements created.
-
Test requirements created.
-
Review outcome documented.
-
Reassessment triggers defined.
Enterprise Threat Model Review Deliverables
Section titled “Enterprise Threat Model Review Deliverables”The review should produce:
01 Review Summary
02 Threat Model Completeness Matrix
03 Architecture Gaps
04 Missing Threats
05 Attack Path Findings
06 Control Gaps
07 Risk Register
08 Security Requirements
09 Required Security Tests
10 Review DecisionRecommended Review Report Structure
Section titled “Recommended Review Report Structure”Enterprise AI Threat Model Review
1. Executive Summary
2. Review Decision
3. Scope
4. Business Context
5. Architecture Validation
6. Asset Review
7. Data Flow Review
8. Trust Boundary Review
9. Threat Actor Review
10. STRIDE Coverage
11. OWASP GenAI Coverage
12. MITRE ATLAS Coverage
13. RAG Threat Review
14. AI Agent Threat Review
15. Supply-Chain Threat Review
16. Attack Path Analysis
17. Risk Rating Review
18. Security Control Review
19. Monitoring & Response
20. Residual Risk
21. Review Findings
22. Required Actions
23. Final RecommendationExecutive Review Questions
Section titled “Executive Review Questions”Before approving the threat model, the reviewer should be able to answer:
Do we understand what this AI system does?
Do we know what we are protecting?
Do we understand where sensitive data flows?
Have all important trust boundaries been identified?
Have realistic attackers been considered?
Have AI-specific threats been considered?
Have RAG threats been modeled?
Have AI agent threats been modeled?
Have supply-chain threats been modeled?
Can we identify realistic end-to-end attack paths?
Do security controls break those paths?
Are critical controls independent of the LLM?
Can attacks be detected?
Can the AI system be contained?
Are residual risks understood?
Does every material risk have an owner?If several of these questions cannot be answered, the threat model is not ready for approval.
Common Review Mistakes
Section titled “Common Review Mistakes”Mistake 1 — Reviewing the Diagram Only
Section titled “Mistake 1 — Reviewing the Diagram Only”Threat modeling is about attack paths and security decisions, not diagram quality.
Mistake 2 — Treating OWASP as the Threat Model
Section titled “Mistake 2 — Treating OWASP as the Threat Model”A vulnerability list does not replace architecture-specific analysis.
Mistake 3 — Ignoring Traditional Security
Section titled “Mistake 3 — Ignoring Traditional Security”AI systems still require:
Authentication
Authorization
Network Security
Secrets Management
Logging
Least PrivilegeMistake 4 — Ending at the LLM
Section titled “Mistake 4 — Ending at the LLM”Modern AI architectures frequently continue into:
LLM ↓Agent ↓Tool ↓Enterprise SystemThreat modeling must continue through the full path.
Mistake 5 — Ignoring Indirect Attackers
Section titled “Mistake 5 — Ignoring Indirect Attackers”Attackers may influence AI through documents, websites or other content.
Mistake 6 — Treating Every Prompt Injection as Critical
Section titled “Mistake 6 — Treating Every Prompt Injection as Critical”Severity depends on what the manipulated model can reach.
Mistake 7 — Missing Attack Chains
Section titled “Mistake 7 — Missing Attack Chains”Several Medium weaknesses may combine into one Critical attack path.
Mistake 8 — Trusting the LLM as a Security Control
Section titled “Mistake 8 — Trusting the LLM as a Security Control”Critical authorization should remain deterministic.
Mistake 9 — No Threat-to-Control Mapping
Section titled “Mistake 9 — No Threat-to-Control Mapping”Every material threat should have a corresponding security requirement or risk decision.
Mistake 10 — Never Reviewing the Threat Model Again
Section titled “Mistake 10 — Never Reviewing the Threat Model Again”AI architectures change quickly.
AI Security Engineer Perspective
Section titled “AI Security Engineer Perspective”When reviewing an AI threat model, think like both:
Attacker+Security ArchitectAsk:
What did the original threat model miss?
What would I attack first?
Which trust boundary gives me the greatest advantage?
Which data is most valuable?
Which identity has the greatest privilege?
Can untrusted content reach the LLM?
Can the LLM influence tools?
Can those tools affect production?
Which control stops me?
What happens if that control fails?
Can security teams detect me?
Can they contain the system?This is the mindset expected from an enterprise AI Security Engineer.
Interview Perspective
Section titled “Interview Perspective”You may be asked:
How would you review an AI threat model?
A strong answer is:
I would first validate that the threat model matches the actual business use case and production architecture. I would review assets, sensitive data flows, trust boundaries and threat actors, then validate coverage using traditional methods such as STRIDE together with AI-specific references such as OWASP GenAI and MITRE ATLAS. I would pay particular attention to RAG, AI agents, model supply chain and end-to-end attack paths. Finally, I would verify that material threats map to independent security controls, test requirements and accountable risk owners.
Another question may be:
What is the biggest mistake when reviewing an AI threat model?
A strong answer is:
Stopping at the LLM. In enterprise systems, the highest-impact risks often exist after the model, particularly where model output influences RAG data access, agents, APIs or privileged enterprise tools.
Another question may be:
How do you evaluate Prompt Injection during threat modeling?
A strong answer is:
I assume Prompt Injection may succeed and then determine what security boundary the manipulated model could reach. If authorization, least privilege and tool controls independently prevent sensitive data access or privileged actions, the architecture has meaningful defense in depth.
Another question may be:
How do you know when an AI threat model is complete?
A strong answer is:
It is sufficiently complete when the architecture and important assets are represented, significant trust boundaries and threat actors are understood, realistic attack paths have been analyzed, material risks map to security controls and test requirements, residual risks have owners, and the model can be updated as the system changes.
Key Takeaways
Section titled “Key Takeaways”An enterprise threat-model review validates:
Business Context+Architecture+Assets+Data+Trust Boundaries+Threat Actors+AI Threats+RAG+Agents+Supply Chain+Attack Paths+Controls+Residual RiskThe review should move from:
We Created a Threat Modelto:
We Have Validated That the Threat ModelRepresents the Real Enterprise AI Systemand finally:
We Understand the Important Attack Pathsand Know Which Security Controls Break ThemMost importantly:
A useful AI threat model does not attempt to predict every possible attack. It identifies the attack paths that matter most to the business and ensures the architecture has controls capable of stopping or containing them.
Module 03 — Labs and Runbooks Complete
Section titled “Module 03 — Labs and Runbooks Complete”With this runbook, the practical structure for Module 03 — AI Threat Modeling becomes:
Lab 01 — Build an AI Threat Model
Lab 02 — Threat Model an Enterprise RAG Application
Lab 03 — Threat Model an AI Agent
Lab 04 — Enterprise AI Attack Path AnalysisRunbooks
Section titled “Runbooks”Runbook 01 — AI Threat Modeling Methodology
Runbook 02 — AI Architecture Threat Review
Runbook 03 — Enterprise AI Threat Model ReviewTogether they teach the learner to:
Build ↓Analyze ↓Review ↓Validatean enterprise AI threat model.
What’s Next?
Section titled “What’s Next?”You have now completed the threat-modeling methodology.
The learner should next move from:
What Could Go Wrong?to:
How Do We Engineer the AI PlatformSo These Attack Paths Cannot Succeed?That naturally prepares the learner for the next security-engineering module in the AI Security Engineer learning path.