GPA: Learning GUI Process Automation from Demonstrations

📰 ArXiv cs.AI

GPA learns GUI process automation from demonstrations using vision-based Robotic Process Automation

advanced Published 7 Apr 2026

Action Steps

Collect demonstration data of GUI interactions
Apply Sequential Monte Carlo-based localization for robust detection and handling of rescaling and uncertainty
Implement vision-based Robotic Process Automation for process replay
Refine and optimize GPA model for improved performance and accuracy

Who Needs to Know This

AI engineers and software engineers on a team can benefit from GPA as it enables fast and stable process replay with minimal demonstrations, improving automation efficiency and reducing errors

Key Insight

💡 GPA introduces robustness via Sequential Monte Carlo-based localization for vision-based GUI automation