PopPy: Opportunistically Exploiting Parallelism in Python Compound AI Applications

arXiv:2605.18697v1 Announce Type: cross Abstract: Compound AI applications, which compose calls to ML models using a general-purpose programming language like Python, are widely used for a variety of user-facing tasks, from software engineering to enterprise automation, making their end-to-end latency a critical bottleneck. In contrast to traditional applications, execution time is dominated by the external components, which cannot be handled by traditional language optimization systems, like op