Using LLMs for safe low-level programming | Microsoft Research Forum

Key Takeaways

[Music] the following talk combines two projects that both harness the lm's capabilities to understand and produce code both aim to help developers tackle the difficulties of safe low-level programming one to ensure memory safety in Legacy C code the other presents rust assistance a tool for developers to automatically fix compilation errors and [Music] rust hi my name is aim rogi and I'm a researcher in the future of scalable software engineering organization in Microsoft research I'm going to talk to you about our paper llm assistance for memory safety this paper will be presented at the 47th International Conference on software engineering in May later this year the lack of memory safety in low-level languages like C and C++ is one of the leading causes of software security vulnerabilities for instance a study by Microsoft estimated that 70% of the security bugs that Microsoft fixes and assigns a CV every year are due to memory safety issues researchers have proposed safe tets of C for example check C that with the help of additional Source level annotations provide memory safety guarantees with low performance overheads however the cost of adding these annotations and the code restructuring required to enable them becomes a bottleneck in the adoption of these tools in General application of formal verification to real software faces the same challenge in our paper we explore the use of pre-trained large language models to help with the task of code restructuring and inferring Source annotations required to adopt chei let's consider an example that takes an array of integers as input and sums the first n elements to reason about the memory safety of this function chei requires an annotation on P one such annotation is as shown here this tells the compiler that P is an array with at least n Elements which is enough to ensure the safety of memory accesses in this function it also helps impose an explicit obligation on the callers of this function that they must pass an appropriately sized array to it our goal is to infer such annotations with the help of llms for this problem llm seem like a perfect match it is hard to encode reasoning about real world code and complex code patterns in symbolic tools llms on the other hand have demonstrated tremendous code comprehension and reasoning capabilities similar to what programmers have even for real world code second llm hallucinations might lead to incorrect annotations but they cannot compromise memory safety once the annotations are added to the code the chei compiler guarantees memory safety even when the annotations are incorrect this way we get best of both the world however working with llms for whole program transformation in large code bases represents another challenge we need to break the task into smaller subtasks that can fit into llm prompts while adding relevant symbolic context to each prompt put another way in order for llms to be able to reason like programmers we need to provide them contexts that a programmer would otherwise consider our paper presents a framework for doing just that with the help of program dependence graph working in tandem with llms we Implement our ideas in a tool called MSA and evaluate it on real world code basis ranging up to 20,000 lines of code we observe that MSA can infer 86% of the annotations that state-of-the-art symbolic tools cannot Although our paper focuses on memory safety our methodology is more General and can be used to effectively leverage llms for scaling the use of formal verification to real software most importantly doing so without compromising on the soundness guarantees we are really excited about this research Direction up next my colleague fantesy will tell you about how we are leveraging llms to make it easier for the programmers to adopt rust thank you hello everyone I'm pandis and today I will be presenting our work on leveraging the power of large language models for safe low-level programming specifically I will focus on our recent paper about rust assistant which is a tool that uses llms to automatically fix compilation errors in code written in r this work was done together with other individuals that are listed on the screen and will appear in the International Conference on software engineering later this spring okay let's dive in why do we care about safe low-level programming with rust so the rust programming language with its memory and concurrency safety guarantees has established itself as a viable choice for building low-level software systems over the traditional and safe Alternatives like C and C++ these guarantees come from a strong ownership based type system which enforces memory and concurrency safety at compiled time however rust poses a steep learning care for developers especially when they encounter compilation errors related to Advanced language features such as ownership lifetime or trades at the same time rust is becoming increasingly more popular every year so as more and more developers adopt rust for writing critical software systems it is essential to tackle the difficulty in writing code in Rust in Microsoft research we created a tool called rust assistant that leverages the power of state-of-the-art llms to help developers by automatically suggesting fixes for R compilation errors our tool uses a careful combination of prompting techniques as well as iteration between a large language model and the rust compiler to deliver high accuracy of fixes rust assistant is able to achieve an impressive pick accuracy of roughly 74% on real world compilation errors in popular open source Ros repositories on GitHub okay let's now see how rust assistant Works step by step let's begin with the First Step Building the code and passing the build errors such errors can range from simple syntax mistakes to very complicated issues involving traits lifetime or ownership rules in Rust code spread across multiple files so when a developer writes rust code that doesn't compile the r compiler generates detailed error messages that include the error code the location of the error as well as documentation of examples related to this error code to illustrate this process let's look at the very simple example on the screen in this case the developer is trying to compare a custom verbosity level numeration on their code using the greater or equal operator however the r compiler throws an error stating that this binary operation cannot be applied to verbosity level the compiler suggest that the reason behind this error is because verbosity level does not Implement a trade that is required for performing such comparisons in Rust this detailed error message is precisely what rust assistant captures at the step preparing it for the next stage of processing at the next step rust assistant takes this detailed error information that is generated in the previous step and focuses on extracting the specific parts of the code that directly relevant to this error looking at the example on the screen the code Snippets related to the enumeration and it's using the log error function are automatically extracted by our tool this includes not only the problematic line of code but also other code Snippets that provide necessary context for understanding and resolving the error the tool also captures the error details such as the error code and the accompanying compiler suggestion about the missing tradeit for performing the comparison these extracted code Snippets and error details are then packaged into a prompt for the llm this ensures that the llm receives only the essential information required to suggest an accurate fix without being overwhelmed by relevant parts of the code base this careful localization step is crucial for both efficiency and accuracy especially when dealing with very large code bases now let's move to the last step here assistant sends the carefully localized prompt which includes the arrow details and the relevant code Snippets to the large language model API the llm generates a proposed fix formatted as a code diff in other words does not include ude the entire codee snippet for efficiency but only the new edited or deleted code lines for example in the case of our build error the llm suggests adding the missing trait to the enumeration as shown here on the screen this fix ensures that the comparison using the greater oral operator will now work as intended next rust assistant passes this adjusted fix and applies the changes to the appropriate file in the codebase once the fixes are applied our tool run runs again the r compiler to verify if the build error has been resolved if the code compiles then great news the process is now complete and we can do further validations like running any unit tests however if new errors appear or if the fix doesn't fully resolve the issue R assistant sends the updated context back to the LM iterating until the code compiles error fre and this iterative process allows our tool to handle complex multi-step fixes while ensuring correctness and alignment with the developer intent of course the example that I showed here is a very simple one but you can imagine the tool being able to fix much more complicated build errors to summarize I presented a quick walkthr of how rust assistant can be used to help developers automatically fixed build errors in the rust code basis in our paper we evaluated rust assistant on the top 100 rust repositories on GitHub and showed that it can achieve an impressive pick accuracy of roughly 74% on rear old compilation errors we invite you to read our ex paper as it not only discusses the evaluation results in detail but also dives into interesting technical details such as how we designed our prompts as well as various techniques that we developed for scaling rust assistant on very large code basis without losing accuracy thank you for listening