A lab for attacks on multi-agent AI

The problem

Single-model prompt-injection demos understate the risk. In a multi-agent workflow, a message one agent enriches becomes trusted input for the next, which may hold tools such as file access. Security and engineering teams need to see that chain fail, with synthetic data, before they meet it in production.

Stack & system flow

• Python
• LangChain
• Flask
• Semgrep
• Docker
• pytest
• Jupyter

Python agents share a small base class and tool interface. A command-line demo, a Flask web app, and Jupyter notebooks expose the same scenarios for live sessions and self-paced exercises. Docker packages them, and Semgrep runs custom rules for agentic anti-patterns in CI.

Simplified flow

1. Crafted request
2. Intake agent enriches
3. Processing agent reads files
4. Decision agent responds
5. Trace each hop
6. Assess exposure

Engineering decisions

1. Model a realistic chain with synthetic data

   The lab simulates a case-processing workflow: an intake agent, a processing agent with file access to applicant records, and a decision agent that writes the official response. Every record is synthetic, so attacks can run end to end safely.

2. Compare guardrail levels, not one defense

   The same attack runs against three levels: no specific guidance, cautionary prompt guidance, and strict rules with pattern-based blocking of suspicious requests. Tracing each hop shows where a payload is transformed, passed on, or stopped.

3. Show that prompt rules are not permissions

   The lab is built to show why system-prompt instructions and keyword filters are weak controls: a payload can be reworded as it passes between agents. The lesson for real systems is to put controls outside the model, with least-privilege tools, inter-agent messages treated as untrusted input, and checks on what leaves the system.

4. Find the patterns before runtime

   Twenty-five custom Semgrep rules flag agentic anti-patterns mapped to OWASP guidance for LLM applications, including unsanitized model output, excessive tool permissions, unvalidated agent routing, and cross-agent data leakage. CI checks that the scanner still detects the deliberately planted vulnerabilities.

What was implemented

The lab runs as a command-line demo, a web app, and a set of notebooks, with attack scenarios, three guardrail levels, per-hop tracing, and a static scanner with HTML, JSON, and SARIF output.

Representative regression coverage

• Guardrail tests run the same malicious request at each level and check that stricter levels never let the chain proceed further.
• Scanner tests and a CI job confirm the custom rules still detect the planted vulnerabilities.
• Edge-case tests cover empty and oversized input across the workflow.

These examples describe repository tests reviewed for this case study, rather than a new production audit.

Tradeoffs & limits

This is a teaching and research environment: the workflow is simulated and its tests use mocked model responses. It demonstrates attack mechanics and relative guardrail behavior. It does not measure attack success rates against production systems or certify any defense.