Skip to content

Lab 02 — Identify AI Attack Surfaces

Before you can perform threat modeling or security testing, you need to understand where an attacker can interact with, influence or compromise an AI system.

That is the purpose of an attack surface assessment.

In the previous lab, you explored an enterprise AI architecture and identified:

  • Users
  • Identity
  • AI application
  • LLM
  • RAG
  • Vector database
  • Enterprise knowledge
  • AI agents
  • Tools
  • External providers
  • Security controls

In this lab, you will take that same architecture and ask:

Where can an attacker enter, influence, manipulate or abuse the system?

You will build a practical:

AI Attack Surface Map

covering:

User Input
+
Application APIs
+
LLM Context
+
RAG
+
Vector Database
+
Enterprise Data
+
Memory
+
Agents
+
Tools
+
Models
+
Datasets
+
External Services
Item Details
Lab Lab 02 — Identify AI Attack Surfaces
Learning Path AI Security Engineer
Module 01 — AI Security Foundations
Difficulty Beginner
Estimated Time 60–75 Minutes
Lab Type Security Architecture Analysis
Primary Skill AI Attack Surface Identification
Output Enterprise AI Attack Surface Map

You continue your work with the fictional organization:

Northstar Enterprises

The company is building:

Northstar Enterprise AI Assistant

The assistant helps employees:

  • Search internal policies

  • Find IT procedures

  • Review security runbooks

  • Access approved enterprise knowledge

  • Ask questions about cloud operations

  • Receive security recommendations

The architecture currently looks like:

Employees
Identity Provider
AI Application
┌───────────┼───────────┐
│ │ │
▼ ▼ ▼
LLM RAG AI Memory
Vector Database
Enterprise Knowledge
AI Agent
Tool Layer
Enterprise Systems

The security team has asked:

Where are the attack surfaces in this architecture?

Your task is to answer that question systematically.

Your mission is to:

Identify every meaningful attack surface in the enterprise AI architecture and document how an attacker could interact with or influence each one.

You are not exploiting the system yet.

You are identifying:

Where Could an Attack Begin?
What Can an Attacker Control?
What Can They Influence?
Which Assets Are Reachable?
Which Security Boundary Could Fail?

By completing this lab, you will learn how to:

  • Explain what an attack surface is.

  • Identify external and internal attack surfaces.

  • Identify user-controlled AI inputs.

  • Map prompt attack surfaces.

  • Identify API attack surfaces.

  • Identify RAG attack surfaces.

  • Identify vector database attack surfaces.

  • Identify memory attack surfaces.

  • Identify AI agent attack surfaces.

  • Identify tool and integration attack surfaces.

  • Identify model supply-chain attack surfaces.

  • Identify dataset attack surfaces.

  • Identify external-provider attack surfaces.

  • Build an AI attack-surface register.

  • Prioritize high-risk attack surfaces.

An attack surface is any place where an attacker may:

Interact
Influence
Modify
Access
Abuse
or
Compromise

a system.

In traditional applications, attack surfaces may include:

  • Web forms

  • APIs

  • Authentication

  • Databases

  • Admin interfaces

  • Network services

AI systems add new ones:

  • Prompts

  • RAG documents

  • Vector search

  • AI memory

  • Agent tools

  • Model registries

  • Training data

  • Embedding pipelines

These are different concepts.

An attack surface is:

A Place That Can Be Attacked

A vulnerability is:

A Weakness That Can Be Exploited

Example:

User Prompt

is an attack surface.

Prompt Injection

may be a vulnerability or attack technique against that surface.

Example:

Attack Surface:
RAG Knowledge Source
Threat:
Poisoned Document
Potential Impact:
AI Provides Manipulated Guidance

This distinction is important.

For every component ask:

1. Who can interact with it?
2. What can they control?
3. What data does it process?
4. What privilege does it have?
5. What systems does it connect to?
6. What happens if it is compromised?

Use the architecture from Lab 01.

User
Identity
AI Application
LLM
├── RAG
│ ↓
│ Vector Database
├── Memory
└── AI Agent
Tools
Enterprise Systems

Now mark every location where:

External Input Enters

or:

Control Changes

or:

Sensitive Data Is Accessed

These are likely attack surfaces.

Phase 2 — Identify External Attack Surfaces

Section titled “Phase 2 — Identify External Attack Surfaces”

Start with components reachable by users or external systems.

Possible external attack surfaces include:

Web Interface
Mobile Interface
API
Authentication Endpoint
File Upload
Prompt Input
Webhook
External Data Source

Create a list.

Example:

Surface Exposed To
AI Chat Interface Employees
AI API Internal Applications
Login Endpoint Employees
Document Upload Approved Users
External Search Internet

The most obvious AI attack surface is:

User Prompt

Architecture:

User
Prompt
LLM

The user controls:

Natural Language Input

Potential security concerns include:

  • Prompt Injection

  • Jailbreaking

  • Role manipulation

  • Context manipulation

  • Excessive input

  • Sensitive data submission

Attack Surface:
User Prompt
Actor:
Authenticated User
Controlled Input:
Natural Language
Assets Reachable:
LLM Context
RAG
Agent Tools
Potential Threats:
Prompt Injection
Jailbreak
Data Exfiltration Attempts

AI applications often maintain:

Conversation History

This creates another attack surface.

Ask:

Can previous messages influence future behavior?
Can another user access them?
Can sensitive data remain in context?
Can malicious instructions persist?

Potential risks include:

Cross-Session Leakage
Persistent Manipulation
Sensitive Data Retention

Some applications support persistent memory.

Architecture:

User
AI Memory
Future Conversations

This attack surface is important because information may persist.

Ask:

Who can write to memory?
Who can read from memory?
Can users modify stored behavior?
Can one user's memory affect another?

Potential threats:

  • Memory poisoning

  • Cross-user leakage

  • Persistent Prompt Injection

  • Sensitive information retention

Identity is still a traditional attack surface.

Architecture:

User
Identity Provider
AI Application

Potential threats include:

  • Credential theft

  • Weak authentication

  • Session hijacking

  • Token misuse

  • MFA bypass

  • Compromised accounts

A compromised account may now gain access to:

Enterprise AI
+
RAG
+
Sensitive Knowledge
+
Agent Tools

This can increase the value of account compromise.

Authorization is not always visible as a component, but it is still an important attack surface.

Ask:

Can users influence their role through prompts?
Does RAG preserve user permissions?
Can the agent perform actions the user cannot?
Can one tenant access another?

Potential threats:

Privilege Escalation
Unauthorized Retrieval
Agent Privilege Expansion
Cross-Tenant Access

The AI application itself may expose:

Web Endpoints
APIs
Admin Interfaces
Prompt Templates
Session Management
Model Configuration

Traditional application risks still apply.

Examples:

  • Broken access control

  • API authorization failures

  • Injection

  • Sensitive configuration exposure

  • Misconfiguration

AI security does not replace application security.

If the application uses a hosted LLM:

AI Application
LLM API

the integration becomes an attack surface.

Review:

API Credentials
Endpoint Configuration
Model Selection
Request Parameters
Rate Limits
Logging

Potential threats:

  • Stolen API key

  • Unauthorized model use

  • Data exposure

  • Excessive cost

  • Misconfigured provider access

System instructions influence model behavior.

Architecture:

System Prompt
+
User Prompt
LLM

Potential concerns:

  • Prompt exposure

  • Hidden sensitive information

  • User attempts to override instructions

  • Excessive reliance on prompt controls

System prompts should guide behavior.

They should not contain:

Passwords
API Keys
Private Tokens

RAG adds new interaction points.

Architecture:

User Query
Retriever
Vector Database

Potential threats include:

  • Unauthorized retrieval

  • Broad semantic queries

  • Cross-tenant retrieval

  • Metadata exposure

  • Retrieval abuse

Attack Surface:
RAG Query
Actor:
Authenticated User
Controlled Input:
Semantic Search Query
Assets Reachable:
Enterprise Documents
Potential Threats:
Unauthorized Retrieval
Sensitive Data Exposure
Cross-Tenant Leakage

Phase 12 — RAG Knowledge Source Attack Surface

Section titled “Phase 12 — RAG Knowledge Source Attack Surface”

The documents themselves may become attacker-controlled.

Examples:

Internal Wiki
Uploaded PDF
Ticket
Email
Website
User-Generated Content

Potential attack:

Attacker
Malicious Document
RAG
LLM

Potential threats:

  • RAG poisoning

  • Indirect Prompt Injection

  • Misinformation

  • Data integrity attacks

The ingestion pipeline is a major attack surface.

Architecture:

Document
Parser
Chunking
Embedding
Vector Store

Ask:

Who can submit documents?
Which sources are approved?
Can content be modified?
Are permissions preserved?
Can malicious files be processed?

Potential threats include:

  • Untrusted ingestion

  • Permission stripping

  • Document poisoning

  • Malicious file processing

Phase 14 — Document Parser Attack Surface

Section titled “Phase 14 — Document Parser Attack Surface”

RAG systems may process:

  • PDF

  • DOCX

  • HTML

  • CSV

  • Text files

The document parser is therefore an attack surface.

Traditional concerns still apply:

Malicious File
Parser Vulnerability
Resource Exhaustion

AI pipelines still depend on normal software.

Chunking can create security weaknesses.

Example:

Protected Document
Chunking
Permissions Lost

Potential risk:

Restricted Chunk
Retrieved by Unauthorized User

Attack surface question:

Does derived content preserve the security context of the original document?

Phase 16 — Embedding Service Attack Surface

Section titled “Phase 16 — Embedding Service Attack Surface”

Architecture:

Enterprise Document
Embedding Service

If externally hosted, sensitive data may cross organizational boundaries.

Review:

  • Provider

  • Data sent

  • Authentication

  • Retention

  • Logging

  • Approved usage

Phase 17 — Vector Database Attack Surface

Section titled “Phase 17 — Vector Database Attack Surface”

The vector database is a major security surface.

Potential attack paths:

Unauthorized Direct Access
Cross-Tenant Search
Vector Modification
Metadata Exposure
Bulk Export
Collection Deletion

Review:

Authentication
Authorization
Network Exposure
Write Access
Tenant Isolation
Backups

Who can:

Insert
Modify
Delete

vectors?

If too many identities can write:

Attacker
Vector Store
Poisoned Knowledge

This can create integrity risk.

Phase 19 — Vector Metadata Attack Surface

Section titled “Phase 19 — Vector Metadata Attack Surface”

Metadata may contain:

Document Name
Department
Tenant
Project
Classification

Example:

Executive-Acquisition-Plan.pdf

Even without document content, the file name itself may be sensitive.

RAG and AI applications may cache responses or retrieved content.

Architecture:

User A
Sensitive Retrieval
Cache

Potential issue:

User B
Same Query
Receives Cached Result

Ask:

Is the cache user-aware?
Tenant-aware?
Permission-aware?

AI agents significantly increase attack surface.

Architecture:

LLM
AI Agent
Tool

The agent may:

Read
Create
Modify
Delete
Send
Execute

Potential threats include:

  • Excessive agency

  • Tool misuse

  • Privilege escalation

  • Unauthorized actions

  • Agent manipulation

Phase 22 — Agent Identity Attack Surface

Section titled “Phase 22 — Agent Identity Attack Surface”

Agents need credentials.

Potential examples:

Service Account
API Token
Cloud Role
Managed Identity

If compromised or overprivileged:

Agent Identity
Enterprise Access

becomes a major risk.

Ask:

What permissions does the agent have?
What resources can it access?
How is it revoked?

Phase 23 — Tool Interface Attack Surface

Section titled “Phase 23 — Tool Interface Attack Surface”

Each agent tool is another interface.

Examples:

read_alert
search_logs
disable_user
modify_firewall
send_email

For each tool ask:

Can the model call it?
Who authorizes it?
Which parameters can be supplied?
What resources can be targeted?

Phase 24 — Tool Parameter Attack Surface

Section titled “Phase 24 — Tool Parameter Attack Surface”

Even an approved tool may be dangerous.

Example:

query_logs(account_id)

Allowed:

Development Account

Attacker attempts:

Production Account

The tool interface must validate:

Action
+
Target
+
User Authorization

Agents may combine tools.

Example:

Tool 1:
Read Sensitive Data
Tool 2:
Send External Message

Combined:

Read Data
Send Data
Potential Exfiltration

This attack surface may not be obvious when tools are reviewed individually.

Phase 26 — Shell and Code Execution Attack Surface

Section titled “Phase 26 — Shell and Code Execution Attack Surface”

An agent may expose powerful tools such as:

run_code
execute_shell

These create high-risk surfaces.

Ask:

Is arbitrary execution required?
Is it sandboxed?
Does it have credentials?
Can it access the network?
Can it access production files?

AI systems may connect to:

  • Search APIs

  • SaaS platforms

  • Cloud APIs

  • Ticketing systems

  • Email

  • Databases

Each integration adds:

Credentials
+
Data Flow
+
External Dependency

and therefore more attack surface.

Phase 28 — Model Supply-Chain Attack Surface

Section titled “Phase 28 — Model Supply-Chain Attack Surface”

AI systems depend on models.

Architecture:

Model Repository
Model
AI Application

Potential threats:

  • Malicious model

  • Model tampering

  • Unapproved model

  • Compromised repository

  • Model replacement

Phase 29 — Model Registry Attack Surface

Section titled “Phase 29 — Model Registry Attack Surface”

Ask:

Who can upload models?
Who can replace models?
Who can promote to production?
Are changes logged?

A compromised registry can affect the entire AI application.

Phase 30 — Framework and Package Attack Surface

Section titled “Phase 30 — Framework and Package Attack Surface”

AI systems depend on:

Python Packages
AI Frameworks
Agent Frameworks
Vector Clients

Traditional supply-chain threats apply:

  • Vulnerable dependencies

  • Malicious packages

  • Dependency confusion

  • Typosquatting

Training and fine-tuning datasets are attack surfaces.

Architecture:

Dataset
Training / Fine-Tuning
Model

Potential threats:

  • Data poisoning

  • Unauthorized modification

  • Sensitive data inclusion

  • Dataset tampering

Some systems collect feedback:

Helpful / Not Helpful
Ratings
Corrections
User Examples

If feedback influences future model training:

User
Feedback
Training Data

this becomes another input surface.

Phase 33 — Model Configuration Attack Surface

Section titled “Phase 33 — Model Configuration Attack Surface”

Model behavior may depend on:

Temperature
System Prompt
Tool Configuration
Safety Settings
Model Routing

Unauthorized changes may alter system behavior.

Phase 34 — Administrative Interface Attack Surface

Section titled “Phase 34 — Administrative Interface Attack Surface”

AI platforms may expose administrative functionality for:

Models
Prompts
RAG Sources
Agents
Tools
Permissions

Admin interfaces are high-value attack surfaces.

Review:

Authentication
MFA
Authorization
Logging
Change Approval

Logging is needed for security.

But logs themselves may contain:

Prompts
Responses
Sensitive Data
RAG Documents
Tool Output

Potential threats:

  • Sensitive log exposure

  • Excessive retention

  • Unauthorized log access

Phase 36 — Monitoring Infrastructure Attack Surface

Section titled “Phase 36 — Monitoring Infrastructure Attack Surface”

If security teams rely on monitoring:

AI Logs
SIEM

ask:

Can attackers disable logging?
Can logs be modified?
Are important events missing?

Integrity of security telemetry matters.

Phase 37 — External Provider Attack Surface

Section titled “Phase 37 — External Provider Attack Surface”

External services may include:

LLM Provider
Embedding Provider
Vector SaaS
External Search
AI API

Review:

Authentication
Data Transfer
Availability
Third-Party Risk
Provider Compromise

Map which AI components are network reachable.

Example:

Internet
Corporate Network
AI Application
Vector Database
Model Server
Agent Tools

Create a simple network exposure table.

Component Internet Internal Private
AI App Yes Yes
Vector DB No Yes Yes
Model Server No Yes
Admin Console No Yes

Phase 39 — Build the Attack Surface Register

Section titled “Phase 39 — Build the Attack Surface Register”

Now document everything you found.

Use:

ID Attack Surface Actor Controlled Input Asset Risk
AS-01 User Prompt Employee Text LLM Prompt Injection
AS-02 RAG Source Content Author Document Knowledge Poisoning
AS-03 Vector DB Service Identity Queries Enterprise Data Unauthorized Retrieval
AS-04 AI Agent User / LLM Tool Request Enterprise System Excessive Agency
AS-05 Model Registry Developer Model Artifact Production AI Model Tampering

Group them into:

Input Surfaces
Data Surfaces
Model Surfaces
Agent Surfaces
Infrastructure Surfaces
Supply-Chain Surfaces
Administrative Surfaces

Examples:

Prompt
File Upload
API Input
External Document

Examples:

RAG Sources
Vector DB
Memory
Cache
Logs

Examples:

LLM
System Prompt
Embedding Model
Model Registry

Examples:

Tools
Agent Identity
Tool Parameters
Tool Chaining

Examples:

Network
Cloud
Container
Runtime
Admin Console

Examples:

Models
Packages
Frameworks
Datasets

Not every attack surface deserves equal attention.

Prioritize based on:

Exposure
+
Data Sensitivity
+
Privilege
+
Autonomy
+
Business Impact
Public FAQ Prompt

No sensitive data or tools.

Internal RAG Search

Access to internal documentation.

Security RAG

Access to confidential security information.

AI Agent
Production Administrator Tool

High privilege and high impact.

Phase 42 — Build an Attack Surface Heat Map

Section titled “Phase 42 — Build an Attack Surface Heat Map”

Create:

Surface Exposure Privilege Data Priority
User Prompt High Low Internal Medium
RAG Medium Medium Confidential High
Vector DB Low High Confidential High
AI Agent Medium High Production Critical
Model Registry Low High Critical High

Phase 43 — Identify Highest-Risk Surfaces

Section titled “Phase 43 — Identify Highest-Risk Surfaces”

For Northstar, likely high-value areas include:

RAG Authorization
Vector Database
AI Agent
Agent Credentials
Tool Execution
Model Registry
Sensitive Logging

These should receive deeper testing later in the learning path.

Phase 44 — Attack Surface Mapping Exercise

Section titled “Phase 44 — Attack Surface Mapping Exercise”

Take this architecture:

User
AI Assistant
LLM
├── RAG → Vector DB → Enterprise Docs
└── Agent → Tool → Cloud

Identify at least:

10 Attack Surfaces

Recommended examples:

01 User Prompt
02 Session
03 System Prompt
04 RAG Query
05 Knowledge Source
06 Ingestion Pipeline
07 Vector Database
08 AI Agent
09 Agent Identity
10 Tool Interface
11 Model API
12 Logging

Architecture:

Internet
AI Agent
Web Search
External Websites
AI Agent
Cloud Tool

Identify the critical attack surface.

A strong answer is:

External Web Content

because:

Attacker-Controlled Website
Indirect Prompt Injection
Agent
Cloud Tool

creates a potentially serious attack path.

Architecture:

Employee
AI Assistant
Shared Vector Database
HR
Security
Finance

Identify the primary security concern.

Answer:

Retrieval Authorization

because the AI interface must not allow employees to bypass departmental access boundaries.

Architecture:

AI Agent
Administrator API Key
Production Cloud

Primary concern:

Agent Identity
+
Excessive Privilege

This creates a large blast radius.

Phase 48 — Create Attack Surface Questions

Section titled “Phase 48 — Create Attack Surface Questions”

For every architecture review, ask:

What Can Users Control?
What Can External Content Control?
What Can Developers Change?
Where Is Sensitive Data Stored?
Where Is Sensitive Data Retrieved?
Which Components Can Write?
Which Components Can Execute?
Which Identities Are Privileged?
Which Components Are Internet-Facing?
Which External Providers Are Trusted?

Create the following artifacts.

1 — Enterprise AI Attack Surface Diagram

Section titled “1 — Enterprise AI Attack Surface Diagram”

Take your architecture and mark:

Inputs
External Interfaces
Data Stores
Privileged Components
External Providers

Include:

Surface
Actor
Controlled Input
Asset
Potential Threat
Priority

Example:

Component Main Attack Surface
User Prompt
Identity Authentication
LLM Prompt Context
RAG Retrieval
Vector DB Search / Write
Memory Persistent Context
Agent Tool Selection
Tool Parameters
Model Registry Model Publishing
Logs Sensitive Data

Identify your top five.

Example:

1. AI Agent Tools
2. RAG Authorization
3. Vector Database
4. Agent Identity
5. Model Registry

For every high-risk surface, create at least three questions.

What tools can the agent use?
What identity does the agent use?
Can the agent perform write actions?
Can users retrieve unauthorized documents?
Are source permissions preserved?
Can untrusted documents enter the index?

At the end of the lab, you should be able to take an architecture and transform:

AI Components

into:

AI Attack Surfaces

For example:

RAG

becomes:

RAG Query
Knowledge Source
Ingestion
Chunking
Embedding
Vector Search
Authorization

and:

AI Agent

becomes:

Agent Identity
Tool Selection
Tool Parameters
Tool Chaining
Autonomy
Approval

This is the security mindset you are developing.

  • AI architecture reviewed.

  • External interfaces identified.

  • Internal components identified.

  • User prompts identified.

  • File uploads identified.

  • API inputs identified.

  • External content identified.

  • RAG sources identified.

  • Vector database identified.

  • Memory identified.

  • Cache identified.

  • Logs identified.

  • LLM identified.

  • System prompts identified.

  • Embedding model identified.

  • Model registry identified.

  • AI agents identified.

  • Agent identity identified.

  • Tools inventoried.

  • Tool parameters considered.

  • Tool chaining considered.

  • Models identified.

  • Frameworks identified.

  • Packages identified.

  • Datasets identified.

  • Model providers identified.

  • External APIs identified.

  • Internet data sources identified.

  • Attack surfaces registered.

  • High-risk surfaces identified.

  • Business impact considered.

  • Top five attack surfaces prioritized.

In this lab, you moved from:

What Components Exist?

to:

Where Could an Attack Begin?

You learned that enterprise AI attack surfaces may exist across:

User
+
Identity
+
Prompt
+
Application
+
LLM
+
RAG
+
Vector Database
+
Memory
+
Agent
+
Tools
+
Models
+
Datasets
+
External Providers

The key principle is:

Every place where untrusted input, sensitive data or privileged capability enters the AI architecture deserves security attention.

When you look at an AI architecture, develop the habit of asking:

Where Is the Input?
Who Controls It?
Where Does It Go?
What Can It Influence?
Which Data Can It Reach?
Which Identity Executes the Action?
What Is the Blast Radius?

Do not begin with vulnerability names.

Begin with the architecture.

Then identify:

Attack Surface
Threat
Security Boundary
Potential Impact

This approach scales across almost every AI technology.

You may be asked:

What is an AI attack surface?

A strong answer is:

An AI attack surface is any component or interface where an attacker or untrusted input can interact with, influence or compromise the AI system. This includes traditional surfaces such as APIs and authentication, as well as AI-specific surfaces such as prompts, RAG knowledge sources, vector databases, model registries, memory, agents, tool interfaces and training datasets.

Another question may be:

How do you identify attack surfaces in an AI application?

A strong answer is:

I start with the architecture and data flows. For each component, I determine who can interact with it, what input they control, what data it processes, what privilege it has and what other systems it can reach. I then document those interfaces and prioritize them based on exposure, sensitive data, privilege, autonomy and business impact.

Another question may be:

Which AI attack surfaces are typically the highest risk?

A strong answer is:

The highest-risk surfaces are usually those that combine untrusted input with sensitive data or privileged capabilities, such as RAG retrieval over confidential information, AI agents with powerful tools, agent identities with excessive permissions, multi-tenant vector stores and model or dataset supply-chain components.

Use this simple methodology:

Architecture
Interfaces
Inputs
Data
Privileges
Attack Surfaces
Prioritization

And remember:

An attack surface tells you where to look. Threat modeling tells you what could happen there. Security testing tells you whether the weakness can actually be exploited.

➡️ Lab 03 — Map AI Threats Using OWASP and MITRE ATLAS

You now know:

Where the AI System Can Be Attacked

The next step is identifying:

What Types of Attacks Could Occur
at Those Surfaces?

In the next lab, you will take your attack-surface map and correlate it with:

OWASP GenAI
+
MITRE ATLAS

You will map examples such as:

User Prompt
Prompt Injection
RAG Source
Data / Knowledge Poisoning
AI Agent
Excessive Agency
Model Supply Chain
Model Manipulation

You will move from:

Attack Surface Identification

to:

Structured AI Threat Mapping

➡️ Next: Lab 03 — Map AI Threats Using OWASP and MITRE ATLAS