Why I Switched from Python to Go Lang for AI Deployment

Key Takeaways

I deployed the same AI project with go and python this meant developing the same project twice once in each language why you ask because bugs will only scare you once and we like to keep our coding CER stories bilingual the project was a recommendation Engine with multi-arm Bandits that was responsible for personalizing the whole Discovery screen for hundreds of thousands of users the implementation needed to be fast scalable and efficient in resources so deploying in a language that can achieve that is key with though go is famous for its speed and efficiency it wasn't as straightforward as it seemed you see I had this recommendation engine working in python as a chrone job for some time now and it resulted in a noticeable increase in Revenue however we wanted to make this recommendation engine more Dynamic responsive and have them update every time the user refreshed their screens I went with using the fast API library to create my serving API in Python most of the code was written in pandas and numpy two highly efficient libraries written in C the last function was the only one without pandas and numpy using a for Loop to D duplicate elements rearrange items and handle promotional requests despite trying to make it faster it took up to 60% of the response time but that wasn't the big issue as the response was around 400 milliseconds locally the issue came and we tried to deploy on 5% of users the response time was not as expected and it kept increasing till it reached 5 Seconds we had to skate the number of pods in kubernetes up to 46 just for the small percentage of users this small percentage had a lot of concurrent requests per second but that meant that to fully deploy to all users we needed over 920 pods when I brought this up with the devops team they made me feel like this $1 100 billion the next day was the start of the weekend and I decided to take a break psych I didn't I knew needed to find a way to handle this fast as this project might be postponed till further notice because it's not impactful enough and I really wanted to dump my spaghetti code for this CR jop aside since Python's implementation had little room for improvement I decided to switch to goang I had seen a team successfully make this transition at my old company and our backend team already uses go daily my goal was to learn go as quickly as possible and then rework some parts of the recommendation engine this would allow me to report back on Monday with the performance difference between go and Python and get everyone on board using go on this project I started by getting a taste of what goang can do so I created a function to sum up the first n numbers in Python and go and found that go was a whopping 100 times faster this was reassuring so I started learning the basics of goang from a fast YouTube video as quickly as possible then I jumped straightforward to developing the recommendation engine in the process I relied on chat GPT to help me write some code translate code from python to go and even explain a couple of Concepts to me it wasn't always producing the correct code but it sure helped me move faster along the way I used trucks and go interfaces to structure my code and it came naturally since I used design by contract in a lot of my data science code I also found a numpy alternative called go num and a panda's alternative called go and go data frame I was happy to find all theun functions I needed were available in those Cod packages I used group bu merge and aggregate functions every function I searched for was supported I kept working on the project until our Monday meeting but I hadn't timed or tested the code yet during our meeting I suggested trying out go Lang but there were concerns about the time it would take to redo it in go and the availability of all python data science libraries in goang a senior backend engineer also mentioned that most of the Python code is already optimized in pandas and numpy and that we won't see much difference between the two languages he was partly right but more on that later in the end I convinced him to see the run times of the go code I already have and decide from there after the meeting I compared Python's runtime with go and found that go was 30 times slower wait what the backend engineer was right pandas was highly optimized and its go counter part was so much slower surprisingly what took Panda's 9 milliseconds to process to go data frame more than 330 milliseconds but what if I ditched data frames and merged and aggregated data by using stroks maps and Loops I mean the experimental summation function was 100 times faster than python so it might reach Panda's performance I restructured the data into a map of structs using the merging column s key I also implemented the merging and aggregation functions by hand but it was worth it at the end when I saw the code that I've written was three times faster than pandas and go overall runtime was 10 times faster than pythons we kicked off G version for 5% of users using the same 46 pods and it ran smoothly scaling up to 10% then 50% then eventually 100% of users on the same 46 pods showed it would handle the load faithlessly go not only showed Superior response times but also exhibited better control over resource management and concurrency crucial elements for the project scalability however it's essential to note that while go excelled in these aspects it doesn't Encompass all the data science libraries we might need and some libraries might exist in go like go data frame but are not efficient as Python's implementations moreover working with gold demanded on more code and in development I would move away from python only if it fails to handle the amount of traffic being thrown at it I hope you enjoyed this video and have a wonderful day