GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Grounding

1. 1Formulate GUI grounding as a text-based coordinate generation task
2. 2Use a context anchor to align intrinsic multimodal attention
3. 3Integrate the context anchor with the MLLM to improve coordinate generation
4. 4Evaluate the performance of GUI-AIMA on various GUI grounding tasks

AI engineers and researchers working on multimodal large language models (MLLMs) and computer-use agents can benefit from this approach, as it enhances the accuracy of GUI grounding

💡 Using a context anchor can enhance the accuracy of GUI grounding by providing a more intuitive and data-efficient approach

💡 GUI-AIMA improves GUI grounding by aligning intrinsic multimodal attention with a context anchor