AI Engineering at Jane Street - John Crepezzi

Programmers using mainstream languages enjoy a wealth of intelligent coding assistants and tools. At Jane Street, where we primarily use OCaml, we faced the challenge of building these tools for a powerful but low-resource functional programming language. This talk explores our journey in creating custom assistants and editor tooling for OCaml, tackling everything from data collection and model training to seamless editor integrations. We'll dive into our end-to-end process: gathering quality training data, developing meaningful evaluations for custom-trained models, building out underlying infrastructure, and creating tools that fit how we work. Recorded live at the Agent Engineering Session Day from the AI Engineer Summit 2025 in New York. Learn more at https://ai.engineer and purchase tickets to our next event, the AI Engineer World's Fair, in SF June 3 - 5 here: https://ti.to/software-3/ai-engineer-worlds-fair-2025 About John John Crepezzi is an engineer at Jane Street, where he works on building LLM-powered coding assistants and the infrastructure to enable others to create applications leveraging large language models. His work focuses on enhancing developer productivity, particularly in Jane Street's OCaml-centric environment. Before joining Jane Street, John was a Principal Software Engineer at GitHub, where he contributed to several impactful projects in the developer productivity space, including Codespaces, Merge Queues, and Contribution Graphs. With a career dedicated to improving how developers work, John is passionate about creating tools that empower engineers to solve complex problems more effectively