GPA: Learning GUI Process Automation from Demonstrations

📰 ArXiv cs.AI

GPA learns GUI process automation from demonstrations using vision-based Robotic Process Automation

advanced Published 7 Apr 2026

Action Steps

Collect demonstration data of GUI interactions
Apply Sequential Monte Carlo-based localization for robust detection and handling of rescaling and uncertainty
Implement vision-based Robotic Process Automation for process replay
Refine and optimize GPA model for improved performance and accuracy

Who Needs to Know This

AI engineers and software engineers on a team can benefit from GPA as it enables fast and stable process replay with minimal demonstrations, improving automation efficiency and reducing errors

Key Insight

💡 GPA introduces robustness via Sequential Monte Carlo-based localization for vision-based GUI automation

Key Takeaways

GPA learns GUI process automation from demonstrations using vision-based Robotic Process Automation

Full Article

Title: GPA: Learning GUI Process Automation from Demonstrations

Abstract:
arXiv:2604.01676v2 Announce Type: replace-cross Abstract: GUI Process Automation (GPA) is a lightweight but general vision-based Robotic Process Automation (RPA), which enables fast and stable process replay with only a single demo. Addressing the fragility of traditional RPA and the non-deterministic risks of current vision language model-based GUI agents, GPA introduces three core benefits: (1) Robustness via Sequential Monte Carlo-based localization to handle rescaling and detection uncertain

Read full paper → ← Back to Reads