How to elevate software development with AI-assisted coding

Key Takeaways

hello thank you for joining this technical session at Google IO 2024 my name is way and I'm a developer Advocate at Google today my colleague shista and I will be talking to you about the exciting topic of AI assisted coding generative AI is transforming many Industries today the software industry is no exception and the many practitioners are already benefiting from adopting AI for example in a typical software development life cycle AI could help you analyze user requirements and write prds in the planning phase while you're designing the architecture of your application you could brainstorm with AI and come up with new interesting Solutions when you start coding AI could help you code faster and better when you finally release your application to the world you could use AI to monitor usage and analyze user feedback for future developments let's zoom in on the coding phase AI could automate away many T tasks so that you have more time to work on the more creative parts of your workflow you may also use AI as a coding body to review or critique your code and it will improve your code quality sometimes AI could even bring more creativity to the table by suggesting alternative or even better implementations for you ultimately taking advantage of AI coding assistance could lead to higher productivity and more developer happiness which is why when stack Overflow released their annual developer survey last year they reported that 70% of the respondents were already using or planning to use AI in their development process and 77% of the responders were in favor of AI tools for developments that's a lot of developers in action this year the numbers maybe even higher so if you still haven't used AI for coding now it's a good time to get started AI coding assistance covers a wide range of use cases in your development process from code completion code generation and code chat to code explanation and error debugging and so many more this use cases are well covered by a number of Google products since we have long been working on AI and developer tools so let me demonstrate how to work first our coding workspaces or idees such as collab project idx worldex AI Android studio and chrome dep tools have all Incorporated coding AI for example here we're in Google collab which is a hosted jupyter notebook service that provides Computing resources including a usage tier available free of charge you can use the AI generated suggestions in COD laab for code completion based on your current coding context let's start by creating a simple function to compute a circles area as I type the function name the building AI is able to guess what I want and suggest implementation without me even asking how convenient this is I can simply hit tab to accept the suggestion or escape to ignore it what if I want a class instead of a function I can type the first few letters and AI is again able to figure out what I want and have the circle class defined I like this class but in the calculation uses only 3.14 which is lower Precision for pi let me change that I know the mass library has a pi value defined in higher Precision so I import that as I'm my typing AI predicts I want to swap out 3.1 for for Mastery and all I need to do is to hit tab next let's run a simple test on this new Cass I Define a list of circles based on the AI generated suggestions but I don't like the default values so I manually change them I started writing a for Loop and immediately AI predicts that I want to Brint out the area value for each circle and that's it I accept this code snipp it as well and I round the last two code blocks to verify the results and they look great as you can see here I have barely typed 30 ccors and AI is able to figure out my intentions and finish what I want to build code generation is another important use case instead of manually writing code I can simply specify the functionality I want in natural language and let AI build it for me this time I'm using Google's Project idx a webbased workspace that enables full stack multiplatform app development workflow on the card in idx I'm using command I on my Mac to invoke the inline Ai and the prompt it to write a function to find the non-common elements of two lists without using other data structures idx follows my instructions smoothly and generates the function I want I can then accept or discard the code to make sure my function here is solid we need to write some unit tests which sometimes feels like a tore to many developers with AI it doesn't have to be like that I simply ask AI to add three unit tests and voila it is done automatically now I think our function is more or less ready so let me put a Finishing Touch on it by adding documentation writing documentation sometimes is tedious but it is very important for readability and maintainability so we got to do it similar to unit tests all I need to do is to ask idx AI to add Doc streen and it automatically populates python documentation for a function without a hitch sometimes you inherit a code from somewhere else and you need to be able to understand the code here we see an example of code explanation on Gemini code assist Gemini code assist is available in multiple idees such as Visual Studio code jetb IDs Cloud workstations cloud share editor and supports more than 20 programming languages including go Java JavaScript Python and SQL and sometimes the code works but you want to make it more optimal in some way maybe more performance maybe more secure maybe more Ematic maybe just simpler in this case we see an example where the underline code was translated from java to coding using the building intelligent functionality however if you want to make the code more Ematic we ask for that transformation in natural language and we can see from the resulting code that it follows the cing style better event syntax stream templates smartcasts and so on have you ever had any experience when the console just throws up an error and you have no clue about it like here in Chrome Dev tools I have no idea what's causing the issue in my Fibonacci number implementation but I get to ask the beauty AI to explain this error and I immediately get insight to help me debug it seems my recursive calls are not properly terminated so with that hint I can easily fix it now I'll pass it to my colleague shista to explore even more use cases with you hi I'm sha basum Malik I'm a group product manager at Google working on generative AI now I'll talk to you about the next set of products the first product we'll talk about is SE generative experience also known as sge historically developers have used Google search extensively to ask their implementation questions to find out when there are errors in their code what to do or to find documentation and sometimes for more open-ended Pursuits such as researching an architecture decision or learning a new skill we have brought generative AI into Google search through sge you can ask questions now and get an answer with code Snippets natural language explanations and relevant code examples in certain cases the code is executed to show the results as you can see here the next product that we will be talking about is Gemini chat developers can also ask their code related questions here and get back tag code answers use this to refactor their code use this to explain code for them and generally chat about anything code related here in this example you can see that Gemini chat not only executes the code for you but also lets you get in there edit your code and rerun that code the next product we'll be talking about is AI onev developers.google.com which is Google's main developer documentation website which is visited by millions of developers we have added AI capabilities to simplify the developers Journey so that the answers they get is a combination of generative AI as well as authoritative documentation from subject matter experts here on developers.google.com you will have ai search which allow developers to quickly grasp key information from search results documentation chat where developers can interact with a conversational AI assistant for focused guidance regarding particular developer products and Pages an AI code explain which provides a deeper understanding of cod samples through AI explanations looking ahead we plan to expand these AI features to other Google developer properties the products mentioned so far are all built on Google's Cutting Edge Foundation models for developers who directly want to build applications on top of our code models we have also released code Gemma which is our latest open code model specifically targeted towards coding tasks code Gemma is a family of open code models based on Google deep Minds Gemma models continuing from the base Gemma models code Gemma models were further trained on up to 500 billion tokens of mostly code they use the same architecture as the Gemma model family these models achieve state-ofthe-art performance in both code completion and code generation tasks while maintaining strong understanding and reasoning skills CT Gemma comes in two different size variants two billion and 7 billion the smaller 2 billion model is the pre-trained model with lightning fast speed and it's perfect for the code completion use case the bigger 7 billion model has two flavors the base model and the instruction tuned model the 7 billion model requires more computational resources but you can still use the 7 billion pre-trained code variant for code completion if you have a powerful machine the instruction tune 7 billion model is better suited for the code chat use case here you see the example code to use caras to run code Gemma we first load the code Gemma instruct 7B model with the from preset function and then call the generate function to get the prediction this is similar to how you would use the base Gemma instruct model except that we are passing code in the prompt and we get a code Gemma prediction as inference if you plan to use code Gemma to build AI Core capabilities within your coding environment a simple inference may not be sufficient in certain cases you will need to know about techniques such as fill in the- middle or fim and retrieval augmented generation or rag fill in the middle is a technique that uses language model to impart infilling capabilities by packing both the left context and the right context separated by special tokens what we mean by this is when you're using Code completion inside of an IDE you sometimes have code above where you're doing the completion and Below where you're doing the completion the above is the left context here bow is the right context here and you pack both of those into the prompt feel free to check out the code Gemma technical report to Lear learn more about this retrieval augmented generation or rag is you is another technique which is used to make the llm answers either more accurate or more relevant uh it can also sometimes help when you have to deal with a use case where you need need a large context such as using a get trying to get an answer based on a large code base for retrieval augmentation what you do is you take the code base you break it into chunks of code sometimes you can convert these codes in code chunks of code into embeddings and you store it in a vector database then at inference time you re retrieve the relevant Snippets of code and you add it to your prompt and you you add it to your query and you pass that as prompt to the model this ensures that the answers you get back are more potentially more accurate or more relevant in summary Google is not only offering you Codi Innovation across a lot of surfaces we're also making our open models available to you to build your apps on top of them we have built all our code AI features to serve developers of all Stripes be they web developers mobile developers Enterprise developers or even developers of all experience levels such from beginners to experts a lot goes into developing these features we start with the right task sometimes a task may require building a model from scratch sometimes we may fine tune on top of our base models it all depends on the task for example for code completion you may want to use a smaller model because you need to have lightning fast speed on the other hand if you're doing code AI inside of collab you need to make sure that the response style aderes to the notebook coding format or if you're building Cod features in Android Studio then you may want to make sure languages that Android developers care about such as Java and cotlin are well represented when we start building these models the first thing we have to think about is the data we use highquality permissively licensed data and we make sure that we sanitize the data of all sensitive information the next step is you know pre-training and specifically if we are pre-training for a specific domain then we would do domain specific pre-training in this case we train models on more tokens of code the next step is instruction tuning instruction tuning is to make sure that the answer adheres to a specific style or format for example if you're using Code chat then you want your responses to be a little chatty the next one is offline evaluations here we do a combination of human evaluations as well as automatic evaluations and finally comes the integration step where we integrate the model with the other components of the llm building block such as serving configurations monitoring safety guard rails and of course the ux a lot of magic also happens in this postprocessing step for example when you're outputting a code completion response you need to know where to truncate the response you may do it based on what's known as a stop token a specific token that you decide in advance or you do it based on say how much of the response overlaps with the suffix or what we call the right context so a lot of decisions like this are made in the postprocessing ux you've already seen examples in Way section you sometimes can interact with the models through chat sometimes within your workflow inside of your coding environment and finally we need to talk about safety guard a lot of attention is paid to to safety we make sure first when we are coming up with the data we make sure the data is probably scrubbed and debiased uh we also take proper anonymization into consideration then when uh we have safety filters in the API to uh Safeguard against unsafe responses and we also show citations whenever we uh we have a code that requires uh showing the source we make sure to responsibly show that Source Enterprise users even can block out code that requires citations as we look into the future we are thinking about code agents here we show you an example of a data science agent that we are building the things we want to highlight about this data science agent is that it can build notebooks from scratch with detailed plans and complex reasoning it supports a bunch of data tasks such as data cleaning exploratory data analysis getting insights from data statistical analysis and predictive modeling in the spirit of experimenting with the user experience we will provide the user not just with the end result but also provide an insight into how the agent is doing reasoning and planning and this will be provided via a web app format as well as in the final cab that you see on the screen here this collab that you see is entirely generated by the agent we want to make code Innovations like this available early to users through a developer playground so that we can test experiments related to functionality related to user experience and other aspects of code development in this next phase of AI We Believe generative AI will become more mature and ubiquitous the future of software development We Believe will involve empowering developers in AI to collaborate on Creative problem solving and Innovation we are super excited about this future and can't wait to build it together with you with that thank you for watching this technical session at Google iio 2024 and happy coding with AI [Applause] oh