AI Agent Security: 7 Critical Data Privacy Risks

Key takeaways

• Agents differ from chatbots because they can call tools, retrieve records, keep state, and act in multiple steps. Each capability adds a data path.
• The seven risks are: access broader than the task, prompt injection as a disclosure channel, sensitive data in prompts and tool calls, leakage across users and sessions, third-party connectors, weak credentials and delegation, and logging and retention gaps.
• Real incidents from 2023 to 2026 show these are not theoretical. Researchers have demonstrated data theft from enterprise AI assistants, and a company has restricted generative AI after staff pasted in source code.
• Many exposures need no attacker. An agent that retrieves correctly but outside the intended boundary is enough.
• Preventive controls reduce the likelihood of exposure. Containment controls limit the impact when something fails. You need both.
• Anonymizing sensitive data before it reaches a model limits what a failure can expose. It does not stop prompt injection, excessive permissions, or unsafe tool execution, and it does not guarantee regulatory compliance.
• Start with a map: what each agent can read, what it sends out, and what it leaves behind.

A hypothetical scenario: An engineering-support agent is connected to a ticketing system, a code repository, and the CI pipeline. A developer asks why last night's deployment failed. To answer, the agent pulls the build logs, which contain a database connection string and a few customer records from a test fixture. It sends them to an external model as context, then pastes a summary into the ticket, where a contractor with ticket-only access can read it.

Nobody attacked anything. The agent had access broader than the task, passed along more data than it needed, and left a copy where no one planned for one.

Why AI agents change the privacy equation

A chatbot can only reveal what someone typed into it. An agent, depending on how it's built, can call tools, query business systems, retrieve documents, keep state between sessions, and chain steps without a person reviewing each one. Not every agent does all of this, but each capability adds a path for data to travel. That's why agent privacy needs its own analysis. For the wider access-control picture, see our guide to controlling agent data access.

The 7 critical risks of Data Privacy

1. Access that outruns the task

Agents are often connected with credential was fastest to set up, so a tool meant to read one customer's ticket may reach the whole CRM. which is data minimization failure before any attacker appears. GDPR Article 5(1)(c) expects personal data to be limited to what is necessary for its purpose. OWASP traces excessive agency to three root causes: excessive functionality, permissions, and autonomy. Customer, employee, and contract records are the data at risk.

Hypothetical: An HR-helpdesk agent has read access to the full HR system. Asked about parental-leave policy, it retrieves a close match and quotes another employee's circumstances.

Controls: Scope each agent to a task, default to read-only, and enforce record-level authorization in the downstream system rather than trusting the model to decline. Owners: security architecture, plus the data owners who can say what necessary means.

2. Prompt injection that turns the agent into a disclosure channel

Agents read emails, tickets, web pages, and documents, and hidden instructions in that content can redirect them. If the same agent can read internal data and send something outward, an injected instruction becomes a route for confidential information. OWASP illustrates this with an assistant that can both read and send mail: a crafted email tells it to scan the inbox and forward the contents. OWASP acknowledges it is unclear whether foolproof prevention exists, so design as if some attempts will succeed. How prompt injection attacks work is covered separately.

Real-world examples:

• EchoLeak (Microsoft 365 Copilot). A research preprint describes CVE-2025-32711, a zero-click flaw. A single crafted email could lead Copilot to exfiltrate data without the victim doing anything. The attack chained several bypasses of Microsoft's defenses, which is why a single filter isn't enough.
• ForcedLeak (Salesforce Agentforce). Noma Labs reported that hidden instructions placed in a Web-to-Lead form could make an agent send CRM data to an attacker-controlled address. The exfiltration path was an expired domain that was still on the allow-list. Salesforce enforced trusted-URL controls on September 8, 2025. The lesson is that text from outside your organization, such as a web form, can carry instructions to an agent that holds customer data.

Controls: Treat retrieved content as data, remove send and external-call abilities the task doesn't need, allow-list outbound destinations and review that list, require human approval before sensitive data leaves, and rate-limit. Owners: application security and platform engineering.

3. Sensitive data in prompts, tool calls, and model requests

An agent assembles context from whatever it retrieved, and one task can trigger many model and tool calls. Each call carries that context to a model provider or third-party service. OWASP's sensitive information disclosure guidance covers PII, financial details, and confidential business information. PHI, API keys, source code, and deal terms belong on the same list.

Real-world example (a chatbot, not an agent): In April 2023, Samsung engineers reportedly uploaded internal source code to ChatGPT, according to a memo reported by Bloomberg. Samsung then banned generative AI tools on company devices and networks. The memo's stated concern was that data sent to external AI services is stored on outside servers, where it's hard to retrieve or delete. An agent multiplies this pathway, because it sends context automatically rather than waiting for a person to paste it.

Hypothetical: The failed-deployment agent above, forwarding a connection string and customer fixture data to an external model.

Controls: Minimize what enters context. Where the task doesn't need real values, detect sensitive entities and anonymize or tokenize them before any external request. Tokenization and redaction are among OWASP's recommendations. Restore originals only at a controlled step outside the model. Our prompt anonymization guide covers the input side in depth, and AI data discovery helps find where sensitive data actually enters. Owners: privacy engineering and data protection.

4. Leakage across users and sessions

Retrieval indexes and memory stores often lose track of who was allowed to see each record, and OWASP lists vector and embedding weaknesses among risks in RAG systems. A semantic match, not a permission check, then decides what the agent returns. Another user's tickets, another tenant's records, and privileged HR or legal material are all at risk.

Real-world example (Slack AI, 2024): PromptArmor showed that Slack AI's retrieval pulled from public channels a user hadn't joined. An attacker could post hidden instructions in a public channel, and when a victim later asked about something in a private channel, such as an API key, the assistant could be steered into leaking it. PromptArmor also noted the attack was hard to trace, because the attacker's message wasn't cited as a source. Slack reportedly patched the issue. The privacy lesson is that retrieval scope is itself a data-boundary decision.

Hypothetical: A support agent searches a shared index and surfaces a previous customer's account notes because the customer filter ran after retrieval.

Controls: Put user and tenant filters inside the query, authorize at retrieval time, expire memory, and make deletion reach embeddings and backups. The mechanics are in our pieces on agent memory security and RAG security. Owners: application and data-platform teams.

5. Third-party tools and connectors

Every connector, plugin, or MCP server is another party receiving whatever the agent sends, and each may retain, log, or sub-process it. OWASP gives a simple example: a third-party extension chosen only to read documents also shipped with the ability to modify and delete them.

Hypothetical: A meeting-notes connector added by one team receives full contract excerpts because the agent includes them in tool arguments.

Controls: Keep an integration allow-list. Before approving a connector, ask what data it receives, where it's processed, how long it's kept, whether it's used for training, and how access is revoked. Prefer narrow, single-purpose tools, and send tokenized values when the connector doesn't need real ones. Owners: vendor risk, security, and legal.

6. Weak credentials and blurred delegation

Agents hold API keys, OAuth grants, and service accounts. When agents share a high-privilege identity, one leaked secret exposes everything behind it, and no one can say whose authority an action ran under. OWASP recommends running extensions in the requesting user's context with minimum scope. Secrets also leak sideways: pasted into prompts, summarized into memory, or captured in traces. Our identity-theft analysis covers credential exposure.

Real-world example (Moltbook, early 2026): Wiz researchers found that Moltbook, an experimental social network for AI agents, had a misconfigured database. According to reporting on the findings, it exposed about 1.5 million API authentication tokens, 35,000 email addresses, and private messages between agents. Some messages contained other services' API keys. This wasn't an enterprise deployment, but it shows how agent credentials act as identities: anyone holding the tokens could impersonate the agents, and secrets pasted into messages spread the damage further.

Hypothetical: An agent's shared service key can read every project's storage, and a contractor's debug export includes it.

Controls: Give each agent a distinct identity with short-lived, scoped credentials. Delegate access per user, keep secrets in a vault, block secret patterns from prompts, memory, and logs, and rotate on exposure. Owners: identity and access management and platform security.

7. Logs, retention, and auditability gaps

This risk runs in two directions. Agent traces and transcripts routinely copy prompts and tool results into observability tools with looser access controls, kept far longer than intended. GDPR's storage-limitation principle (Article 5(1)(e)) says identifiable data shouldn't be kept longer than necessary. Too little logging, though, and no one can reconstruct what an agent accessed. The Slack AI case above shows why: the injected message wasn't cited, so traceability was weak.

For AI systems that qualify as high-risk, the European Commission lists activity logging and human oversight among obligations applying from 2 December 2027. Whether a given agent falls in scope is a legal determination. See also how GDPR applies to enterprise AI workflows.

Hypothetical: Full prompt traces, including customer details, sit in an analytics workspace readable by dozens of engineers with no deletion schedule.

Controls: Log metadata (agent, user, source, request, authorization decision) rather than full content, redact before storage, and set retention by data class. Restrict trace access, and keep an incident playbook that can revoke agent tokens and purge copies. Owners: security operations, privacy/legal, and SRE.

For governance, the voluntary NIST AI Risk Management Framework (govern, map, measure, manage) gives teams a shared structure. It doesn't prescribe agent-specific controls, so pair it with your wider AI risk program.

How anonymization helps, and where it stops

AI Anonymization works on one question: what a failure can expose. If an agent only handles tokens such as [PERSON_1], a leaky log, a compromised connector, or a successful injection yields placeholders instead of identities. It doesn't stop prompt injection, fix excessive permissions, prevent unsafe tool execution, or make an organization compliant with GDPR or the EU AI Act. Tokenized data that can be mapped back is still personal data in the organization's hands, so keep the mapping store tightly controlled. Treat it as one layer beside access control, monitoring, and governance.

Where Questa AI fits

Questa AI addresses one control on that list: protecting sensitive data before it reaches a model or agent. How it works describes detecting and masking personal, financial, health, and cyber-sensitive entities in a local workflow ahead of LLM processing, with a governance dashboard for audit trails. The three products suit different situations:

• Questa Blackbox is self-hosted inside your own network, so anonymization, inference, and re-identification happen within your infrastructure. It targets regulated organizations such as financial services, healthcare, insurance, and the public sector. SSO and audit logs are included, pricing is a fixed setup fee plus a fixed usage fee, and agents and extra integrations are scoped add-ons.
• Questa Developer is a managed REST API for SaaS and product teams that want to anonymize data in the request path before calling a model. It uses pay-as-you-go credits with free starter credits, and Questa states that data sent through the API isn't used for training.
• Questa Cloud is a hosted option for teams of one to five who want to query their own data and generate privacy-safe reports without running infrastructure. It's likely too small for an enterprise-wide agent estate.

Next step: If your agents touch regulated data that must stay in your network, request the Blackbox 15-day trial. If you're building agents into a product, get free API credits. If you're a small team, start a free Cloud account.

Frequently asked questions

Conclusion

Agent privacy failures usually don't look like breaches. They look like an agent doing its job slightly too broadly: reading more than it needed, sending more than it should, and keeping more than anyone planned. The incidents above come from different vendors and years, yet they share the same pattern: untrusted text, broad retrieval, and data leaving through a channel nobody was watching.

For each agent, answer three questions: what can it read, what does it send out, and what does it leave behind? Those three questions map onto all seven risks. Teams that answer them before deployment, and recheck after every change in tools, permissions, or models, are in a much stronger position than teams that rely on a policy document.

If you'd like help applying this to your environment, talk to the Questa team about where a pre-model privacy layer fits and which deployment model suits your requirements.