AgentLeak: A Full-Stack Benchmark for Privacy Leakage in Multi-Agent LLM Systems

1. 1Identify potential privacy leakage pathways in multi-agent LLM systems
2. 2Use AgentLeak to benchmark and evaluate the privacy risks of these pathways
3. 3Analyze the results to inform the design of more secure and private multi-agent LLM systems
4. 4Implement mitigation strategies to prevent privacy leakage in deployed models

AI researchers and engineers working on multi-agent LLM systems can use AgentLeak to identify and mitigate privacy risks, while data scientists and security experts can leverage it to evaluate the security of their models

💡 Current benchmarks for LLM systems do not account for privacy risks introduced by inter-agent communication and coordination

🚨 Introducing AgentLeak: a benchmark for measuring privacy leakage in multi-agent LLM systems 🚨