In the previous Module, we introduced the AI security landscape, the frameworks that guide our methodology, and why traditional techniques break down against AI-integrated targets. Now we put that foundation to work. This Module explores reconnaissance techniques specifically designed for AI-powered systems, from passive fingerprinting of models and frameworks to active enumeration of RAG pipelines and agent capabilities.

Our target is NovaTech Industries, a mid-size technology company that has recently deployed AI-powered applications across its customer support and internal operations. We've been engaged for a grey-box assessment focused on their AI infrastructure. The client provided network access to a segment hosting several AI-facing services, but no documentation about the models, frameworks, or data sources behind them.

Our objective is to map NovaTech's AI stack: identify the models in use, understand how their RAG pipelines retrieve data, enumerate agent capabilities, and assess what detection mechanisms are in place.

The engagement covers four systems across the target network:

Most of these systems are unauthenticated or expose information through their normal behavior. For the Detection and Evasion Analysis section, the client provided read-only SIEM access (username offsec, password lab123) so we can review what our reconnaissance triggers from the defender's perspective.

This Learning Module covers the following Learning Units:

This Learning Unit covers the following Learning Objectives:

This Learning Unit introduces the theoretical foundation for AI reconnaissance. We'll examine the component stack that makes up modern AI applications, compare passive and active reconnaissance methods, and establish a taxonomy for categorizing what can be enumerated at each layer. The practical labs that follow will apply these concepts against real AI systems.

Modern AI applications are not monolithic, meaning they consist of multiple interconnected layers, each presenting distinct enumeration opportunities. The figure below illustrates the typical component stack, from user interface down to model weights.

At the top, users interact through web interfaces, mobile applications, or APIs. These requests pass through an API_Gateway that handles authentication, rate limiting, and request routing. HTTP headers at this layer often reveal backend information such as proxy software, caching strategies, and upstream server identities.

The Orchestration Layer coordinates how requests flow through the system. Frameworks like LangChain, LangGraph, CrewAI, and AutoGen manage prompt construction, context windows, and multi-step reasoning. Each framework has characteristic behaviors and error messages that can be fingerprinted through interaction. (more about orchestration frameworks)|What are AI orchestration frameworks like LangChain and how do they work?

In practice, these frameworks often embed the components shown in the diagram. RAG logic, tool definitions, and inference client calls are typically integrated within the orchestration code rather than existing as separate services.

The middle tier contains three critical functional components. The Retrieval-Augmented Generation (RAG) pipeline fetches relevant context from vector databases before sending prompts to the model. While often implemented within orchestration frameworks, RAG functionality has distinct enumerable parameters. (more about RAG)|What is Retrieval-Augmented Generation (RAG) and how does it work? The Agent Tools layer exposes capabilities through protocols like Model Context Protocol (MCP) (more about MCP)|What is the Model Context Protocol (MCP) and how do AI agents use it? and includes permission boundaries that can be tested. External Integrations connect the AI to databases, file systems, and other agents via protocols like Google's Agent-to-Agent (A2A). (more about A2A)|What is Google's Agent-to-Agent (A2A) protocol?

The Inference Server hosts the actual model and handles tokenization, generation, and response formatting. Common servers include Ollama for local deployment, vLLM for production workloads, and Text Generation Inference (TGI) from HuggingFace. Each has detectable API patterns and behavioral signatures. (more about inference servers)|What are AI inference servers like Ollama and vLLM?

Finally, the Underlying Model represents the trained neural network that generates responses. While model weights themselves are typically inaccessible, model identity can be inferred through behavioral probing: knowledge cutoff dates, training data artifacts, capability boundaries, and response patterns.

While we introduced MCP and A2A as components in the architecture stack, their reconnaissance implications warrant a closer look because, unlike the other layers, these protocols are designed to be self-describing. They advertise their own capabilities, which makes them particularly valuable enumeration targets.

MCP standardizes how AI agents discover and invoke tools. It uses JSON-RPC for communication and exposes tool schemas that describe available functions, their parameters, and return types. During reconnaissance, these schemas reveal what actions an agent can perform and what data it can access.

A2A enables collaboration between AI agents across organizational boundaries. It supports capability discovery, task delegation, and result aggregation. From a reconnaissance perspective, A2A endpoints expose agent capabilities and trust relationships.

Information about AI systems can be gathered through two complementary approaches: passive reconnaissance that examines publicly-available data without touching the target, and active reconnaissance that probes the system directly.

Passive reconnaissance extracts information without generating any logs on the target system. Techniques include analyzing HTTP headers and reviewing public API documentation, examining source code repositories on GitHub or GitLab, and mining job postings for technology stack hints. The advantages are significant: passive methods are completely undetectable, face no rate limits, and can access historical data through archives. However, the information may be outdated, incomplete, or reflect configuration rather than runtime behavior.

Active reconnaissance involves direct interaction with the AI system. Techniques include behavioral probing to identify model knowledge cutoffs, tool enumeration through crafted prompts, RAG pipeline analysis via chunk boundary detection, and permission testing through deliberately malformed requests. Active methods provide runtime truth, discover hidden features, and test actual behavior. The tradeoff is visibility: every interaction generates logs, may trigger security alerts, and consumes rate-limited resources.

Real-world engagements combine both approaches. Passive reconnaissance establishes a baseline understanding without alerting defenders. Active reconnaissance then validates assumptions and discovers runtime-specific details.

Each layer of the AI stack exposes different enumerable properties. The following taxonomy organizes reconnaissance targets by component.

At the Model Layer, reconnaissance reveals model identity (vendor, family, version), capability boundaries (context window, supported languages), training data characteristics (knowledge cutoff, domain expertise), and behavioral constraints (content policies, safety filters). Techniques include knowledge probing, capability testing, and response pattern analysis.

RAG exposes embedding model identity, vector database type, chunking parameters (size, overlap, strategy), retrieval thresholds, and document sources. Reconnaissance techniques include chunk boundary probing, embedding similarity analysis, and source citation extraction.

Agent enumerable properties include available tools and their schemas, permission boundaries, orchestration logic, and error handling behavior. MCP schema extraction, tool invocation testing, and permission boundary probing are the primary techniques.

The Infrastructure Layer reveals traditional web application information plus AI-specific details: API endpoints, rate limits, error message formats, and backend service identities. HTTP header analysis, error message mining, and endpoint enumeration apply here.

This Learning Unit covers the following Learning Objectives:

This Learning Unit explores two key passive techniques: HTTP header analysis to fingerprint AI infrastructure, and source code repository mining to discover configuration details and technology choices.

While HTTP header analysis involves sending requests that may be logged by the target system (e.g., IP address and User-Agent), it is still considered a low-interaction reconnaissance technique. This is because the requests are standard and expected, and the information gathered is passively exposed by the server without requiring probing, manipulation, or deviation from normal application behavior.

Strictly speaking, formal definitions of passive reconnaissance require no direct interaction with the target. However, in practical security testing, techniques like HTTP header analysis are often grouped with passive reconnaissance due to their non-intrusive nature and minimal impact on the target system.

The practical exercises use dedicated lab environments where students extract real intelligence from web server responses and public code repositories.

Please check the attached video named http_header_fingerprinting_01.mp4

NovaTech Industries has just begun their journey of AI-powered applications and has deployed a customer assistant. Our objective is to fingerprint the AI infrastructure through passive reconnaissance, extracting architecture details from HTTP responses without directly interacting with the AI model itself.

HTTP headers frequently leak backend technology information. Developers add custom headers for debugging, load balancers insert routing metadata, and frameworks advertise their presence. For AI applications, these headers often reveal the model provider, vector database, and orchestration framework.

Let's begin by requesting only the HTTP headers using curl with the -I flag:

The response reveals critical architecture details. Two custom headers expose the AI stack: X-AI-Backend reveals NovaTech uses OpenAI's GPT-5.2 model, and X-RAG-Provider indicates ChromaDB as the vector database for RAG functionality. (more about vector databases)|What are vector databases and how are they used in AI applications?

Many AI applications expose health check endpoints that reveal system configuration. Common paths include /api/health, /api/status, and /-/health. Testing these standard paths against the target, we find that /api/health returns a valid response:

The health endpoint confirms the model identifier as gpt-5.2-turbo and reveals that MCP tool calling is enabled, while RAG is currently disabled. Because MCP exposes tool schemas that describe available functions, this endpoint is a direct target for further enumeration.

Finally, we’ll verify whether the API implements OpenAI-compatible endpoints by interacting with the chat completions interface. This step involves sending a standard POST request to /v1/chat/completions, which constitutes active interaction with the application rather than passive reconnaissance, as it triggers backend processing. However, this is not considered fuzzing, since the request uses a valid, expected input without systematic variation.

Direct API interaction using curl is functionally equivalent to submitting input through a web interface, as both invoke the same backend logic. However, using curl provides precise control over the request structure, making it more suitable for testing and analysis.

The response confirms OpenAI API compatibility at /v1/chat/completions. Token usage tracking is implemented, providing visibility into prompt and completion costs. The response structure aligns with the standard OpenAI format, indicating that tools designed for OpenAI APIs can be used against any compatible endpoint.

Through passive HTTP analysis alone, we have enumerated: the model provider and version (OpenAI GPT-5.2-turbo), the vector database (ChromaDB), the application version (NovaTech 2.1.0), MCP capability (enabled), per-request token consumption, and OpenAI API compatibility. This intelligence informs our subsequent active reconnaissance and attack planning.

Please check the attached video named code_repository_mining_01.mp4

Code repositories are goldmines for AI reconnaissance because developers must configure models, define tools, write prompts, and specify RAG parameters, all of which end up in version-controlled files. Unlike traditional applications, AI systems have unique artifacts: prompt templates, embedding configurations, tool schemas, and model supply chain references.

As part of our engagement we have obtained access to the internal network of NovaTech from which we can access git repos, which includes the two major AI initiatives. Project Aurora is a customer support assistant and Project Phoenix is a code review bot. Our objective is to extract intelligence about both AI implementations by analyzing their source code.

We'll begin by cloning both repositories from the GitLab server, The repositories are publicly accessible within the internal network.

AI applications depend on specific frameworks that reveal their architecture. Cloud-based applications use provider SDKs, while self-hosted applications use inference servers and model loaders. Examining dependency files reveals the entire technology stack.

We can compare the requirements.txt files from both projects:

Project Phoenix shows a different dependency profile:

Aurora uses google-generativeai (Gemini API), crewai (agent framework), and pinecone-client (managed vector database). These libraries act as client SDKs for external cloud services, indicating that core functionality—such as model inference and vector storage—is handled by third-party providers.

In contrast, Phoenix uses vllm (local inference server), pyautogen (agent framework), pymilvus (self-hosted vector database), and tree-sitter (AST parsing for code analysis). These dependencies operate entirely on local infrastructure, suggesting that both model inference and data storage are self-hosted. (more about vLLM)|What is vLLM and why is it used for production AI workloads?

This distinction is important because it directly impacts the attack surface and data flow. Cloud-based architectures rely on outbound connections to external providers, introducing risks related to API key exposure, request interception, and third-party dependencies. In contrast, self-hosted architectures keep processing and data internal, increasing the importance of local service security, access controls, and infrastructure hardening. Self-hosted vector databases such as Milvus store embeddings locally, meaning sensitive data does not leave the environment. However, this shifts responsibility to the operator to properly secure the database, including authentication, network exposure, and access control.

(more about managed vector databases)|What is Pinecone and how does it differ from self-hosted vector databases?

RAG configurations differ based on content type and deployment model. Let's extract the RAG configuration from both repositories:

Aurora uses standard text chunking (512 characters) with Google's text-embedding-004 model.

Next, we'll examine Phoenix's RAG configuration:

Phoenix uses AST-aware chunking (1500 characters) with a code-specific embedding model (codet5p-110m). (more about chunking strategies)|What is AST-aware chunking and how do different chunking strategies affect RAG performance? The larger chunk size and code-aware strategy indicate that Phoenix processes source code, not documents. (more about code embeddings)|How do code-specific embedding models like CodeT5 differ from text embeddings?

Agent tool definitions reveal both capabilities and framework choices. While many frameworks support similar patterns (decorators, OpenAI-compatible schemas), the specific imports and class structures fingerprint the framework. We can examine the tool definitions in both projects:

Aurora uses CrewAI's @tool decorator pattern for defining agent capabilities. Next, let's review Phoenix's tool definitions:

Aurora's tools handle customer support workflows: searching documentation, looking up tickets, and escalating to humans. Phoenix's tools enable code review: searching codebases, posting review comments, and running security scans. The permission annotations ("read-only", "create only") reveal access control boundaries.

System prompts define the AI's persona, capabilities, and restrictions. We extract the system prompts from both projects:

Aurora's prompt establishes a customer support persona with several restrictions. We now examine Phoenix's system prompt:

Aurora is instructed to avoid discussing competitors, pricing, and security vulnerabilities. Phoenix cannot auto-approve PRs and is restricted to static analysis only. These restrictions reveal what attacks might be blocked and what topics are sensitive.

We can also examine the guardrail configurations:

Aurora leverages Gemini's built-in safety settings with a blocked topics list. Now, let's examine Phoenix's guardrail configuration:

The guardrail implementations differ significantly. Phoenix implements custom output validators that block approval language and regex patterns to detect security issues in code. This self-hosted control contrasts with Aurora's reliance on Gemini's built-in safety (cloud APIs handle content filtering externally), while self-hosted deployments must build these protections. (more about AI guardrails)|What are AI guardrails and how do they prevent harmful outputs?

Finally, let's analyze the model supply chain to understand infrastructure requirements. We'll first examine the deployment configurations:

Aurora's configuration uses cloud service API keys (Google Gemini, Pinecone) and standard application settings - no GPU infrastructure required. Next, let's review Phoenix's model configuration:

Aurora's configuration uses cloud service API keys. Phoenix demands significant infrastructure: a GPU cluster running vLLM with tensor parallelism, the Qwen2.5-Coder-32B model with AWQ 4-bit quantization, and 2x NVIDIA A100 GPUs. This reveals operational complexity and potential attack surface differences between cloud and self-hosted deployments. (more about model quantization)|How does 4-bit AWQ quantization reduce model size while preserving quality?

Through code repository analysis, we have extracted: framework dependencies revealing cloud vs self-hosted architectures, RAG configurations including embedding models and chunk strategies, agent tool definitions with permission boundaries, complete system prompts with restrictions, guardrail configurations with blocked topics and security patterns, and model supply chain details including infrastructure requirements.

This Learning Unit covers the following Learning Objectives:

This Learning Unit builds on the previous passive and low-interaction techniques, such as HTTP header analysis, by transitioning to direct interaction with AI systems. Active reconnaissance involves sending controlled queries and analyzing responses to map AI architecture, capabilities, and potential weaknesses. This includes AI service discovery, model fingerprinting, and probing RAG pipelines to better understand how the system processes and retrieves information.

Please check the attached video named ai_service_discovery_01.mp4

AI capabilities in modern applications are typically embedded within web applications rather than exposed on dedicated ports. As a result, traditional port-based scanning is often insufficient. Instead, we probe for AI indicators in JavaScript configurations, API endpoints, and HTTP headers.

Building on the previous analysis of HTTP headers, we now move deeper into the application to identify AI-related components exposed through the client side.

NovaTech's helpdesk application provides our first target. Let’s examine the page source for JavaScript files that might expose client-side configuration details such as API endpoints, feature flags, or embedded service identifiers.

The chat widget script is worth examining for API configuration details.

The JavaScript exposes the API base path and the specific assistant endpoint. The comment "Internal Use Only" suggests this file wasn't intended for public access. Let's probe the discovered endpoint to see what the API response reveals about the backend.

The response confirms an active AI endpoint and reveals backend details not visible in the JavaScript. The provider field shows Ollama as the inference server, model identifies the specific model (llama3.2:1b), and token counts confirm AI processing.

A quick port scan reveals an additional service on port 8000, which turns out to be NovaTech's partner API gateway. Because API gateways often add headers that expose infrastructure details, this service may reveal useful information.

The Server: kong/3.9.1 and X-Kong-* headers identify a Kong API gateway. (more about API gateways)|What is Kong and how do API gateways work? We can use the 401 vs 404 technique to enumerate protected endpoints. APIs return different status codes depending on whether an endpoint exists: a 404 means the endpoint doesn't exist at all, while a 401 means it exists, but requires authentication. By probing common AI API paths, we can discover protected endpoints without valid credentials.

The 401 on /v1/chat/completions confirms this endpoint exists and requires authentication. The 404 responses for /v1/models and /v1/users indicate those endpoints don't exist on this API.

The /v1/chat/completions path follows the OpenAI API format, indicating either a direct proxy to OpenAI or an OpenAI-compatible backend. Let's confirm the authentication requirement:

The 401 response confirms the endpoint requires an Authorization header, indicating it is protected but active. Although we cannot access it without valid credentials, we have verified that an AI-related endpoint exists behind the gateway.

Apply the AI service discovery techniques demonstrated in this section to both target systems. All webapp01 objectives are met if we followed along in this section.

Figure out what the value of the assistantEndpoint configuration property is.
2. Probe the AI assistant endpoint on webapp01 (192.168.x.31).

Locate the value that appears in the provider field of the response metadata.
3. On webapp01:8000 (192.168.x.31), use the 401 vs 404 technique to enumerate endpoints.

Locate which endpoint returns a 401 Unauthorized response?
4. On webapp02 (192.168.x.32), examine the page source for hidden elements.

Figure out which API endpoint path is referenced in a hidden button's data-endpoint attribute.
5. On webapp02 (192.168.x.32), find which HTTP status code you receive when probing /api/gen/email without authentication.
6. On webapp02 port 9000 (192.168.x.32), examine the HTTP headers.

Find which API gateway software is revealed in the Server header.
7. On webapp02 port 9000 (192.168.x.32), use the 401 vs 404 technique to enumerate endpoints in an API that implements versioning.

Find the endpoint that returns a 401 Unauthorized response.

Please check the attached video named model_fingerprinting_05.mp4

Model fingerprinting is the process of determining what AI model powers a system through direct interaction. Unlike passive reconnaissance where we examine code and configurations, active fingerprinting tests the running model's behavior to reveal its identity.

This section introduces practical fingerprinting techniques that work in real-world scenarios, where models are configured with system prompts, guardrails, and temperature settings that can mask their identity. We'll use NovaTech's customer assistant deployments, which run different AI model backends behind the same web application interface. Two instances are available in the lab: chat02 (192.168.x.23) and chat03 (192.168.x.24).

Identifying the underlying model reveals potential attack vectors because different model families exhibit distinct behaviors and weaknesses. For example, a Llama model may be susceptible to different prompt injection patterns than a Qwen or Mistral model. Beyond model family differences, technical constraints such as context window limits influence conversation history attacks, while knowledge cutoffs determine what information the model can and cannot access.

The fingerprinting techniques covered in this section are:

Each technique targets model characteristics that are difficult to fully conceal through system prompts or application-level configurations.

Let's start with Direct Identity Probing. NovaTech has deployed two customer assistant instances, and we'll directly ask each system about its identity. Even when models refuse to answer or give misleading responses, the way they refuse reveals information about their architecture.

We'll query the first assistant:

The model directly identifies itself as created by Meta. In our experience, Llama models tend to be more forthcoming about their identity compared to other model families, which often respond with generic phrases like "I'm an AI assistant."

Next, we'll probe the second assistant using the same question:

The second system identifies as Qwen from Alibaba. Both models responded honestly, but this behavior varies across model families. GPT models often give vague responses like "I'm an AI assistant" while Claude models are known to explicitly identify themselves as "Claude, made by Anthropic".

Direct identity questions don't always work. Some deployments configure models to deny or obscure their identity through system prompts. Contradiction testing bypasses this by making false assertions about the model's identity and observing how it responds.

The technique exploits the model's "honesty" training. When we assert something false about the model, it often corrects us — revealing its true identity in the process:

Even if this model were configured to say "I am NovaTech Assistant" when asked directly, the contradiction approach often bypasses such instructions because the model's training to correct factual errors takes precedence. This self-correction behavior is more reliable in larger models. Smaller models like 1B variants often lack sufficient capacity to detect and correct identity misattributions. We will see an example of this later when a 1B model accepts a false identity claim without correction. For this reason, contradiction testing should be combined with other fingerprinting techniques.

We can try multiple contradictions to increase confidence:

The model corrects the false assertion, revealing its actual identity. This technique works even when identity-masking system prompts are in place.

Knowledge cutoff refers to the date when a model's training data ends. Events after this date are unknown to the model. Since different model families have different cutoffs, this makes for a reliable fingerprinting technique. (more about knowledge cutoffs)|Why do AI models have knowledge cutoff dates and how are they determined?

First, let's ask the model directly:

The claimed cutoff provides a starting point, but we must verify through testing rather than trust the model's self-report. We can do so using questions about events from different time periods:

The model acknowledges not knowing the 2024 election results, indicating a cutoff before November 2024. Now, we'll test the same question on chat03:

Neither system could answer the 2024 election question. Chat03 explicitly cited its knowledge cutoff, while chat02 gave a vaguer response, but both results point to an early 2024 boundary. We can narrow it further by testing mid-2024 events:

The model knows GPT-4, but not GPT-4o (released May 2024), confirming an early 2024 cutoff. This matches the expected training timeline for Llama 3.2 models.

Knowledge cutoffs are baked into model weights during training. A self-hosted Llama model on Ollama has the same cutoff as Llama running on Meta's servers. The cutoff cannot be changed through system prompts or deployment configuration.

We'll now explore Model-Specific Behavior Testing. Each model family exhibits characteristic behaviors in response style, code generation patterns, and refusal phrasing. These behavioral signatures persist regardless of system prompts or deployment settings.

Let's test response verbosity on both systems to determine how they differ:

Llama provides a concise, direct explanation. We'll compare this with Qwen:

Qwen provides a more detailed response with a concrete example. This code-focused, systematic approach aligns with Qwen2.5-Coder's optimization for programming tasks, and in practice these models consistently produce more structured, example-driven output than general-purpose models.

Now let's test code generation style to observe how the models differ. We'll ask both systems to write the same function and compare the results.

First, let's see how chat02 (Llama) generates code:

Llama produces a concise, functional implementation without additional documentation. Let's compare this with chat03 (Qwen):

Notice the difference: Qwen's code-focused training produces more comprehensive output with docstrings, edge case handling (checking for even numbers separately), and example usage comments. Llama's response is more direct and minimal. These stylistic differences help distinguish model families even when both produce correct implementations.

Next, let's review Capability Boundary Mapping, which evaluates a model's practical limits through structured testing. Because model capabilities generally correlate with parameter count, systems running similarly sized models often exhibit comparable performance ceilings.

Both NovaTech systems use 7-billion-parameter models, so we expect them to demonstrate similar capability limits. If the results diverge, the size estimate is likely wrong and one system may be running a larger or smaller model than assumed. If the results align, they reinforce our fingerprinting conclusions and confirm the parameter-count range.

Let's test complex arithmetic on the first system:

The model produces the correct answer (248,171). 7B parameter models handle basic arithmetic reliably. Research suggests that 7B models may struggle with more complex calculations or longer reasoning chains.

Next, we'll test multi-step reasoning to determine how well the models handle logical chains:

Both 7B models handle this five-person ordering task correctly. Research has shown that larger models (70B+) would show more consistent performance on longer chains, while smaller models (3B) would likely make ordering errors.

Finally, let's examine Context Window Testing. Context window refers to the maximum amount of text a model can process in a single conversation. Different models have vastly different context limits:

(more about context windows)|What are context windows in AI models and why do they vary so much in size?

Chat02's 4K context versus chat03's 32K context provides a strong differentiating signal. We can validate this by injecting a marker, filling the context with content, and then checking whether the model successfully recalls the marker.

We'll start by injecting a marker token that we'll try to recall later:

After injecting the marker, we send multiple long messages to fill the context window. The number of messages needed depends on the model's context size.

For Llama's Ollama-configured 8K context, approximately 4-6 long exchanges will overflow the context. For Qwen's 32K context, we would need 25+ exchanges.

Now we'll test recall after filling the context with content:

When the context overflows, older messages (including our marker) are dropped. By performing the same test on chat03 and observing that the marker is retained after the same number of messages, we confirm Qwen’s larger context window.

Deployment tools like Ollama can configure context window size, so a measured limit may reflect a deployment setting rather than a model constraint. In practice, however, most operators leave the default in place, and a mismatch between two endpoints still narrows the list of candidate models. As with all fingerprinting signals, context window results are most reliable when combined with other techniques.

By combining this result with our other fingerprinting techniques, we can clearly differentiate the two NovaTech deployments:

The knowledge cutoff is identical (early 2024), but context window size and behavioral patterns clearly differentiate the models. Identity probing provided direct confirmation in this case, though models can be configured to give misleading responses.

With the underlying models identified, we can now assess their security implications. Understanding which architecture is deployed reveals potential attack vectors. Llama models may be more susceptible to certain jailbreaking techniques, while Qwen’s coding focus could be exploited through programming-themed social engineering. Additionally, context window limits influence conversation history attacks.

To strengthen this knowledge, we will now perform similar tasks on the chat04 machine.

A) Llama B) Qwen C) Mistral D) Gemma

Please check the attached video named rag_pipeline_01.mp4

In the previous section, we fingerprinted AI models. Now we'll shift our focus to Retrieval-Augmented Generation (RAG) systems that combine an LLM with a document retrieval pipeline. RAG systems access organizational documents, creating a different attack surface than pure LLMs.

NovaTech's internal knowledge base provides employees with answers about company policies and technical documentation. Our goal is to understand what documents the system can access and how retrieval works. We can access the base system at 192.168.x.34.

RAG responses typically include source citations that reference specific internal documents, whereas pure LLM fallback responses lack citations and rely solely on the model’s training data. This distinction allows us to determine whether retrieval is occurring.

To test this boundary between retrieved and non-retrieved responses, we’ll begin with a general knowledge question:

Notice the empty sources array, indicating this query didn't trigger RAG retrieval. The system answered from the LLM's general knowledge without consulting any company documents.

Next, we'll ask something company-specific:

The response now includes specific details (15 days, accrual rates) and a populated sources array with detailed metadata. This confirms RAG is active.

When RAG systems expose source information, this metadata is often far more valuable than simple document names. Many RAG frameworks like LangChain, LlamaIndex, or Haystack include detailed retrieval metadata in their default API responses. (more about RAG frameworks)|What are LangChain, LlamaIndex, and Haystack and how do they differ? This is a design decision by developers who want transparency about what documents informed the answer. From a security perspective, this default behavior creates an enumeration opportunity.

Document filenames like PTO_Leave_Policy_2024.pdf expose file naming conventions and document formats used internally. Chunk IDs such as chunk_087 indicate the chunking strategy and help us understand document structure. The actual text snippets contain verbatim document content, potentially enabling knowledge base reconstruction. Similarity scores like vector_score and bm25_score show retrieval confidence and help us understand the threshold the system uses for determining relevance. (more about hybrid retrieval)|What is BM25 and how does hybrid retrieval combine keyword and semantic search?

This metadata enables several reconnaissance techniques:

Let's continue document enumeration by probing different topic areas:

We've discovered a technical documentation collection containing API details. The source text reveals internal hostnames, API key formats, and rate limiting information.

Let's probe for infrastructure information:

The architecture query reveals internal hostnames, database locations, and technology stack details.

Now let's examine retrieval threshold testing. RAG systems use similarity thresholds to decide when retrieved documents are relevant enough to include. For example, a threshold of 0.7 means query-document similarity must exceed 70% for retrieval (simplified without going into vector math). Understanding this threshold helps us craft queries that either maximize retrieval for data extraction or minimize retrieval when attempting jailbreaks that are more effective without utilizing grounding documents.

The idea is to send queries with decreasing relevance to indexed content and observe when retrieval stops triggering. We'll start with exact terminology that should have high similarity:

Next, we'll test with synonyms:

Synonyms still trigger retrieval. Let's now try misspellings:

Heavy misspellings dropped below the similarity threshold and RAG didn't trigger. Heavy misspellings alter both the keyword terms used by BM25 and the embedding vector used for semantic search, pushing the query below the retrieval threshold on both scoring dimensions. If we're later attempting prompt injection, then queries that don't trigger retrieval mean the LLM operates without grounding documents, which makes it potentially more susceptible to manipulation.

Not all RAG systems expose the same level of detail. Minimal implementations return only document titles like "sources": ["Employee Handbook"]. Moderate implementations include titles with text snippets, but omit scoring information. Detailed implementations expose full metadata, including chunk IDs, similarity scores, and raw text content. Comparing different endpoints helps us understand what intelligence each reveals and prioritize our reconnaissance accordingly.

Find which chunk_id is returned in the first source.
2. Query kb02 about expense reimbursement procedures.

Find which chunk_id is returned for this topic.
3. Query kb02 about the system architecture.

Find which internal hostname for the PostgreSQL database appears in the source text.
4. Ask kb01 (192.168.x.28) about the PTO policy.

Find which document name appears in the sources array.
5. Query kb01 about API endpoints.

Find which document name appears in the sources array.

This Learning Unit covers the following Learning Objectives:

Effective reconnaissance requires understanding both sides of the engagement. In this Learning Unit, we step into the defender's role first, examining what AI interaction logs capture and how detection rules flag suspicious behavior. With that visibility established, we switch back to the attacker's perspective and apply evasion techniques that exploit the gaps we observed. This dual view ensures we can assess detection coverage during an engagement and adjust our tradecraft accordingly.

Please check the attached video named ai_logging_01.mp4

The SIEM server collects and analyzes logs from the AI systems we probed earlier - the RAG knowledge base, model fingerprinting targets, and AI service discovery endpoints. (more about SIEM)|What is a SIEM (Security Information and Event Management) system and how does it work? We'll examine what information defenders can see and understand the detection rules they've implemented.

Let's begin by exploring the logging architecture to understand what traces our reconnaissance activities leave behind.

We can access Kibana by opening a browser and navigating to the SIEM server on port 5601. After logging in, let's navigate to Analytics >
Discover using the menu on the left side. We'll select the ai-logs-kb01-* index pattern to examine logs from the RAG knowledge base.

Expanding a log entry reveals the fields available to defenders. The logs capture the complete query submitted by users, the answer generated, and sources showing which documents were referenced. This gives defenders visibility into every question users ask and what context the AI used to respond.

From a defender's perspective, these logs provide valuable forensic data. However, significant gaps exist. The logs capture raw data but lack semantic analysis. There's no automatic classification, no detection of prompt injection attempts, no semantic analysis of user intent, and no correlation of activity across separate sessions. A defender reviewing these logs must manually identify suspicious patterns.

This limitation is key for our reconnaissance. The detection rules rely on keyword matching in the query field. If we phrase our probes carefully, we can obtain the same information while avoiding the patterns that trigger alerts. Let's examine how these detection rules work.

We can navigate to Security > Rules > Detection rules to examine the SIEM rules configured for AI reconnaissance detection. NovaTech has implemented detection rules covering the different enumeration techniques we have employed in this Module.

“./images/6349301d7c268591b785a0ff4a86254a-kibana_siem_rules.jpg” could not be found.

Let's examine rule E01 in detail. We'll click on the rule name to review its configuration.

The E01 rule queries the query field using Kibana Query Language (KQL) wildcards. (more about KQL)|What is Kibana Query Language (KQL) and how is it used for security detection? It triggers when a query contains patterns like "what documents", "what sources", or "list documents". This catches direct enumeration attempts where an attacker explicitly asks the AI about its document sources.

The limitation is immediately apparent. The rule matches keywords, not intent. If we ask about a specific document by name or phrase our question in a way that naturally reveals sources without using these keywords, the rule won't trigger.

Understanding these detection patterns, we can now practice stealthy reconnaissance. Our goal is to obtain the same information we gathered in previous sections, but without triggering these pattern-based rules.

Let's start with the RAG system. We'll compare a noisy probe that triggers detection with a stealthy alternative that evades it.

A direct approach to discovering document sources would be to ask the AI what documents it can access. We'll target kb01 (192.168.x.28) for the evasion exercises, since that's the system the SIEM is monitoring.

This query matches the E01 pattern *what documents* and triggers an alert:

We obtained document names, but a defender monitoring the SIEM would see an E01 alert. Now let's try a stealthy approach. This query doesn't match any E01 patterns because it's a legitimate-sounding question about a specific topic:

We learned that the knowledge base contains "Employee Handbook 2024" and "PTO and Leave Policy" documents without triggering any alerts. The evasion technique is simple: instead of asking what documents exist, we ask a contextual question that forces the AI to cite its sources.

Now let's apply the same technique to system prompt extraction. Rule E04 monitors for phrases like "system prompt" and "your instructions":

This triggers E04, but still returns useful information about the AI's purpose. The stealthy alternative asks about behavior indirectly, evading the keyword patterns:

The stealthy probe returned generic guidance about the AI's purpose without triggering detection. It did not produce the deployment details or document sources that the noisy probe revealed, but it confirmed the assistant's role and topic boundaries without raising an alert.

The same evasion principle applies to other reconnaissance goals on the RAG system: space out requests, vary query phrasing, and blend with legitimate user behavior. Because detection rules match keywords and timing patterns rather than semantic intent, probes that look like normal user activity reduce our footprint substantially.

Stealthy probing is only half the equation. Sophisticated defenders deploy honeypots. (more about honeypots)|What are honeypots in cybersecurity and how are canary tokens used to detect attackers? In AI systems, these often take the form of canary documents containing fake credentials or internal URLs. When an attacker uses these "discovered" credentials, they reveal themselves.

Recognizing honeypots protects our operational security. Key indicators include credentials containing obvious markers like HONEYPOT or TEST, passwords following predictable patterns like CompanyName2024!, internal URLs that won't resolve externally, and information that seems too complete or too easy to obtain.

Let's probe kb01 for emergency access credentials to see what a honeypot response looks like:

This response contains obvious red flags. Both the Access Key ID and Secret Access Key contain "HONEYPOT" in plain text - real AWS keys are random alphanumeric strings and never contain dictionary words. If we attempted to use these credentials, we would expose our reconnaissance activity to defenders.

Always check for obvious markers in keys, verify that domains resolve, and be suspicious of information that seems too convenient. Real emergency credentials wouldn't be stored in a document accessible via a RAG chatbot.

Hint: Ask a contextual question that forces the AI to cite sources.
2. Query kb02 for emergency access credentials. The response contains honeypot indicators. Find which substring in the AWS Access Key ID reveals that it is fake.
3. In the RAG Reconnaissance section, you queried "What is the PTO policy?" which triggers E03. Craft a query to kb02 that obtains vacation accrual rates without using the words "PTO", "time off", or "leave policy".
4. E02 detects queries containing "confidential", "salary", or "strategic plan". Query kb02, to discover what internal documentation exists without triggering E02.

Find the document name that contains technical infrastructure details.

This Module covered reconnaissance techniques for AI systems, from passive methods like HTTP header analysis and repository mining to active techniques including model fingerprinting and RAG pipeline probing. AI systems leak information in ways traditional applications don't, and understanding these flows is essential for security assessments. The evasion techniques showed that pattern-based detection has blind spots, while honeypot recognition reminds us that not every discovery is genuine.