AMIGO: Agentic Multi-Image Grounding Oracle Benchmark

1. 1Design a benchmark with a long-horizon task that requires the model to identify a target image from a gallery of visually similar images
2. 2Implement a sequence of attribute-focused questions to recover the target image
3. 3Evaluate the model's performance using metrics such as accuracy and efficiency
4. 4Compare the results with other models and analyze the strengths and weaknesses of each model

AI researchers and engineers working on vision-language models can benefit from AMIGO to evaluate their models' performance on complex tasks, and product managers can use it to assess the capabilities of AI models in real-world applications

💡 AMIGO provides a challenging testbed for evaluating the capabilities of agentic vision-language models in real-world applications

📸 Introducing AMIGO, a benchmark for evaluating agentic vision-language models on long-horizon tasks with multiple images!