Can we make better software by using ML and AI techniques? With Chandra Maddila and Chetan Bansal

Key Takeaways

one of the biggest disconnects we used to have in box product world where we used to ship software as a as a standalone product and give it to customers is once customer takes the product it is in their environment we don't have any idea about how it is being used and what kind of issues people are facing unless they come back to microsoft support and say hey we are using this product and we get into these issues can you please help us but with advent of services one of the beautiful thing that happened is now we have the ability to collect telemetry about various issues that are happening in the service so this helps us proactively fix issues and help customers mitigate outages and also join the telemetry data from deployment side of the world all the way into coding phase which is the first phase of software development life cycle welcome to the microsoft research india podcast where we explore cutting edge research that's impacting technology and society [Music] i'm your host sridhar vedantha [Music] the process of software development is dramatically different today compared to even a few years ago the shift to cloud computing has meant that companies need to develop and deploy software in ever-shrinking time frames while maintaining high quality of code at the same time developers can now get access to large amounts of data and telemetry from users is it possible for companies to use machine learning and artificial intelligence techniques to shorten the software development life cycle while ensuring production of robust cloud scale software we talk about this and more with chandra madila and chaitan bansal who are research software development engineers at microsoft research india chandra and chetan welcome to the podcast and thank you for making the time for this thank you for hosting this thank you thanks for having us great now there's something that's interested me you know when i decided to do this podcast with you guys you're both research software development engineers and microsoft research is known for being this hardcore computer science research lab so what does it mean to be a software developer in a research org like msr and how is it different than being a software developer in say a product organization if there is a difference yeah that's a great question sridhar about the difference between the rst role which is research software development engineer at msr versus the product groups and microsoft so in my experience the rst role is sort of open-ended because oftentimes research streams work on open-ended research problems so the rsd engineers often work on things like prototypes and building uh products from the ground up which are deployed internally and which are the precursor for products which are shipped to our customers so there's a lot of flexibility and openness in terms of what the rst's work on and it can range from open-ended resource to actually building products which ship to our customers so there's a wide spectrum of things and roles which rsc plays chandra what is your take on that i think shaytan summarizes it pretty well so rsd in general is much more flexible compared to a typical software engineer role in product groups uh you can switch from areas to areas and products to products so i for example was working on nlp for some time then web applications and learning platforms for some time and then i switched to software engineering so we have this flexibility to move across different areas and also one thing we i think do as rsds is working on long-term problems problems from ground up which takes some time to incubate and productize whereas software engineers and product groups have well-defined scope and well-defined problems which are aligned to their products vision so that way um they have slightly more constrained uh in terms of what kind of problems they work on but at the same time one of the greatest advantages people in product groups have is the accessibility to customers so they are very close to customers and they really work on customer problems and ship things quite faster whereas rsds in msr don't have access to direct customers interesting so it sounds like it's kind of a play between customer access and freedom as far as rsds are concerned yeah as rsds in microsoft research we have a lot more flexibility and and provision to explore more interesting areas in research new and upcoming areas like probably quantum computing or blockchain or advances in aml etc and do more exploratory things uh just wanted to add another thing here so uh a lot of times people have misconception that in microsoft research or even other research organizations a doctorate or phd is required to get a job or to work for these organizations but there are roles uh such as rsts and product managers program managers or even designers uh which people can take on without need to have a phd or a doctorate and they can still contribute to the research happening in companies like microsoft great now you know we keep hearing nowadays that the process of software development has changed tremendously over the last few years so what's actually caused these changes i think to start with the two uh things i've which i in my opinion have caused this sort of revolution uh in the software development our software industry so one of them is this move to the services oriented world so we are no longer uh shipping boxed products in a cd or dvd but we are actually shipping services we are actually selling services which are used by our customers uh unlike before where you ship a software and then that's used by our customers for a couple of years and then they update it so i think that's one key change which has happened in the last decade and the other major uh paradigm shift which has happened is the move to cloud so even in terms of software deployment uh today it's being done on cloud instead of on-prem which is within the premise of a customer or a company uh so that has brought in a whole range of changes in terms of how software is developed and deployed and maintained within small and big companies like like even microsoft and today startups or any new company doesn't have to actually spend a lot of money in capex capital expenditure on buying buying servers or hiring people to maintain those servers but they can basically uh ship and operate out of cloud which saves a lot of money and time so in my in my opinion these are the two major paradigm shifts which has happened and which has positively impacted the software industry compared to 90s when we used to for instance shipbox products now everything is becoming a service that is also primarily driven by customer expectation so these days customers are expecting companies to actually ship services more faster make the new features available at much faster pace which is also accelerated by the development and you know growth in cloud computing technologies which makes uh software companies or software developers to scale the services uh really fast and serve more people and ship things much faster so you know i know for a fact that earlier there used to be these long ship cycles where you know somebody would develop some software uh there'd be a bunch of people testing it and then after which it would reach the customer whether it be the retail customer or an enterprise customer right i think a lot of these processes have either disappeared or been extremely compressed so what kind of challenges and opportunities do these changes provide you guys as software developers so this rapid development models where people are expected to ship really fast brought down the overall ship cycles the duration of the ship cycles down to even like days uh are in a single day you experience the entire software development lifecycle all the steps of the development lifecycle starting from coding to testing to deployment uh in a single day this definitely poses a lot of challenges because you have to make sure you are shipping fast but at the same time you are making sure your service is stable and customers are not experiencing any interruptions so you need to build tools and services that aid developers to achieve this so the tools and services has to be pretty robust and make sure they catch all the all the catastrophic bugs early on and eight developers to achieve this feat of shipping their services much faster so the duration between someone writing a code and that code hitting the customer has come down like significantly which is what we all need to make sure we we support i just want to add two more things uh two more changes which have helped evolve the software uh development life cycle and processes first is the possibility of collecting telemetry and data from our users so basically uh we are able to observe how our features or our code is behaving or being used in near real time which allows us to see if there's any regression or if there any changes or if there are any bugs which needs to be fixed this wasn't possible in in the past with in the box software world because we didn't have access to the telemetry the second aspect is having a set of users which are helping you test your features and services at the same time so now we can sort of do software development in parallel as we roll out our current set of features cool so uh it sounds like you guys are now able to get a large amount of data as well as telemetry from the users right uh how does this actually help in making the software development lifecycle more efficient or faster so i think there are two aspects like one of them which i just highlighted was now we are getting real real-time or near real-time telemetry in terms of how different uh aspects of our software or services are being used and the second is if there's any regressions or any anomalies which are happening we are able to detect that and then resolve that very quickly which wasn't possible before so i think these are the two aspects one of the biggest disconnects we used to have in box product world where we used to ship software as a as a standalone product and give it to customers is once customer takes the product it is in their environment we don't have any idea about how it is being used and what kind of issues people are facing unless they come back to microsoft support and say hey we are using this product and we get into these issues can you please help us but with advent of services one of the beautiful thing that happened is now we have the ability to collect telemetry about various issues that are happening in the service so this helps us proactively fix issues and help customers mitigate outages and also join the telemetry data from deployment side of the world all the way into coding phase which is the first phase of software development life cycle and give valuable insight to developers so that when the code itself they have an understanding of how this code is going to behave out there in the wild and be more cautious and cause less bugs or issues [Music] there have been a couple of terms which have become i think very predominant very prominent over the last few years there are two terms that come to mind immediately to me one is devops and the other is ai ops what exactly are these so devops is basically a commonly used term across the software development industry which refers to basically the set of practices and tools for uh developing software deploying software and shipping software so basically how different parts of our industry different companies are actually building software and what are the set of practices for example how do you do code reviews how do you check in code how do you deploy the code so different set of practices and also the tools and infrastructure which is involved so in my opinion that's sort of the definition of devops it's it's a it's a very abstract term which refers to different set of practices and tools for software development uh lastly ai ops so that's basically a recently uh introduced term in probably in the last few years because of this access to telemetry and data from our software and and users we are able to leverage data science and machine learning for optimizing a lot of key aspects of the devops life cycle for instance uh while doing code reviews can we use machine learning and data science for catching bugs that's a very simple example that but that gives you an idea that how ai ops or artificial intelligence can be used to help with different aspects of devops and that's branded as ai ops so devops actually is a combination of two words right development plus operations in box product world when companies are shipping software through cds or dvds as jethro mentioned we used to develop software and sell it to customers and all the operational aspects of the software that is deploying the software in their organizations and maintaining it and making sure that the software is running properly etc is in the hands of the customer who take the software from the vendors like microsoft but with the advent of services microsoft is also becoming a services provider like satya famously says microsoft is now a services company and we provide solutions to customers so we definitely got into this innate need of doing operations also inside microsoft itself which makes us do both the development and operations together devops inside microsoft itself so this basically combines different aspects of software development life cycle starting from coding testing and also deployment and customer support and filling the feedback loop back into development and iterating over all these phases again and again aiops is a term that has been coined in last couple of years uh specifically means using technologies like artificial intelligence and machine learning and leveraging that to solve problems and operational challenges in software development for instance you take a fancy algorithm and use it to solve root causing problem in in software services right that is a classic example of using a for solving a real problem in operations and we have a variety of different problems that occurs in the operation side of the software development now because of the scale at which software development is happening and using and applying aiml techniques to solve those problems put together can be called as aiops okay now i know you guys have been working for a few years on this very interesting research project called sankey and i think this has elements of using ai and machine learning in making the sdlc more effective talk a bit about that sankey is a project which we started at the end of 2016. one of the primary goals of sankey is to provide an ability to join various data that is being collected at different phases of software development life cycle and leverage techniques like aaml and do analysis on top of the data and provide valuable insights which can aid various stakeholders in each phase of this software development life cycle the whole motivation of behind thank you was to infuse aiops into the software development processes across microsoft and it has been a huge collaborative effort with several collaborators such as pierre ashok rahul kumar ranjita bhagwan sonu mehta jim klewin and not just these folks but also several research fellows across msr and other or other counterparts from across microsoft okay now i get the feeling that both of you have kind of oversimplified what sankey is actually i've sat through various talks in which there seems to be huge amount of work that goes in different components that feed into sankey which seems to be kind of like a platform uh why don't you guys talk a little more about that what sanki actually is and what the different constituent parts are so to speak so sankey is actually a platform that we have been building sankey basically have loaders that ingests data from various phases of software development lifecycle for instance from development phase it ingests data about pull requests commits various builds from testing phase it ingests data about test cases test executions what is the status of the tests and from deployment phase it ingests data about alerts exceptions and various other telemetry that is collected at the deployment phase and we we basically put all this data together in a single queryable data source that is very important because this data exists in various disparate data sources which are exposed at various levels and sankey basically gets all this data into a single relational data store which can be easily queried and joined against each other then we use this data we feed it into various a and ml tools to provide insights and recommendations in various phases of software development life cycle for example we mine all the commit data that is which files are changed together which files go into a pull request together etc to basically discover rules that explains the files that are always changed together and we use that knowledge to provide recommendations when developers are creating pull requests if they are missing any files to include in their pull requests we call it as related files analysis similarly we developed tools like orca online root cause analysis tool which is intended towards root causing service incidents and service disruptions as quickly as possible so in case of arca it's pretty interesting that it uses data from both left side of the software development life cycle and right side of the software development lifecycle that is data from commits and code that is written and the differences of code and the telemetry that is collected at the deployment site that is exceptions errors that are occurring in the service so arca basically takes all these exceptions errors that are happening and has an ability to point them towards the actual code change that introduced these problems in first place which is pretty fascinating because this greatly reduces the amount of time developers spend in root causing issues which typically takes probably a couple of days or sometimes even weeks depending on the complexity of the issue and sanki has close to eight such recommenders which which combines data from various different fields of the sdlc and leverages the aiml techniques the aiops processes and make the entire development lifecycle more optimal and efficient so to add to what chandra just said about sankey i just want to mention that in the beginning of the podcast we briefly discussed how the move to cloud and service oriented software development has posed some new and interesting challenges for software development but in this case we are actually able to use that to our advantage since in sankey we are basically building services which we can deploy and iterate on very fast uh based on the feedback from our users and also based on the telemetry we are getting from the services and lastly uh because of this cloud oriented architecture we are able to leverage our big data technologies and these service oriented architectures which allow us to leverage terabytes of data or telemetry which are being produced by a different user-facing services and then combining that with machine learning algorithms and providing insights which are very valuable to the end users of the sanki platform now is sankey available to the world outside of microsoft as part of sankey one of the key focus has been on making sure that all of our techniques and algorithms are published in major software and system conferences so we have published research papers and articles about the sankey platform architecture and even the eight different recommenders which chandra talked about okay so if it's all available in the public domain i think we'll make them available along with the transcript of this podcast okay so let's do a little bit of crystal ball gazing now uh where do you guys see software development engineering and devops evolving in the future i think that's a great question as mark anderson famously said software is eating the world so a lot of traditional companies are becoming more and more tech companies you can see that in every industry automobile pharmaceutical retail everywhere tech is penetrating a lot this actually makes software development more complex and we need to react to customer requests in a more faster ways which basically makes ai ops much more relevant using all the ai and ml technologies to make the entire software development lifecycle more efficient and deliver value to the customers and users who are subscribing to our services is going to become way more important to add to what chandra just said so i think there are two things which makes me excited about how we can evolve sankey and other similar projects to prepare for the next shift and software of our industry so i think first is the more and more usage of software and and machine learning and cyber physical systems for example in self-driving cars in agriculture and uh these are systems which are like safety critical time critical and impact humans in a big way so evolving sankey and others and similar tools and techniques and for those set of those vertical of uh software and services i think will be a key challenge and opportunity and the last one is uh the move from software industry has seen this software 1.0 2.0 and now this move to the edge right where a lot of times the cloud or the compute is available on the edge of the network so that is accessible it's located close to the to the user so how we can leverage sankey and other similar techniques for the edge focus cloud as another interesting aspect which we are excited about okay so chandra and chaitan this has been a fantastic conversation and fascinating thank you so much once again for your time thank you thank you for this insightful conversation thank you