Kubernetes and High Performance Computing

Key Takeaways

hello and welcome to the latest edition of the new stack makers podcast this week recording live in live streaming from the show floor of cube con and cloud native con in Barcelona and today's topic we're going to delve into the fascinating overlap between HPC high performance computing and kubernetes both are forms of distributed community distributed computing and we're very interested in what one can learn from the other and how we could perhaps use kubernetes to do some of those HPC jobs and our panelists today we got we have start with Alfonzo Santiago and he's a researcher for Elam biotech which is a spinoff of the Barcelona computing center exactly so yeah it's trying to take to the market a tool that we have been developed in the Barcelona supercomputing center for the last 15 years all right yeah it's not not the best name we know that yes we have essentially built a way to search maps data in the world around us by time instead of distance you can say what could I do within half an hour by public transport driving walking or cycling or all those combined we now do that in 35 plus countries and because of that we have a big data issue of which HPC in kubernetes and then has been very helpful and also we have TJ funding who is a software engineering consulting member of that's right I've been working here at for just over four years working on building what we now call OCI Oracle cloud infrastructure where I function as an architect across our container related portfolio so our managed kubernetes our functions as a service our container registry etc nice alright fantastic fantastic and HPC space what kubernetes came around I thought oh well this is this is high-performance computing it's the same thing it's distributed computing to execute a larger job but actually the kind of worlds are different but that's no reason that one can't learn from the other so I'm very curious about both of you are HPC veterans and you're using Oracle cloud or using a cloud service of some sort and so what's the transition like what's the benefits what what's going on first let's start with perfect so as I told you the spin-off has been created more or less six months ago and it's actually a spin-off of the word so far solution supercomputing Center on the Barcelona supercomputing Center we have been building this model of the human heart so it's a fitting human heart instead of the computer the good thing about this model is that you can try and test different techniques and as far as those inside of the on this organ right so this is a great - a super powerful tool to be used by clinicians and by designers of your medical devices there are two there are two main problems with this these congressional simulations are really hard to configure right because you have like thousands of parameters that you happen to know and to the detail to make it work right and the second problem that we have is the compositional cost of these simulations because even though it's very efficient there comparation expensive so we need more or less about 200 to 500 course for in simulations and about two hours right and that's a single program as a single execution for yeah so the thing is that with this type of resources I mean everyone has these type of resources on their references right so what we're trying to do is trying to tackle this problem with Oracle cloud trying to democratize this tool and make it available for people that didn't have this type of tools we have before right so we have a front-end web application where you can configure the the executions that they are very clinician friendly with without all the numerical complex parameters just the physical parameters that you want and then we have a back-end where we schedule the different executions and run different executions an Oracle cloud so um these are software is yeah it was originally developed to run efficiently on supercomputers it is actually an academic code right so when you're trying to move it to more industrial framework it's you have to talk on some some issues so for example these code generally runs on supercomputers on computers that are for research and these are very these type of computers have very optimized compilers and libraries for this type of server so we need if we want to keep the efficiency on the cloud we need to port this type of optimizations to the cloud and we need to make it run efficiently also into the cloud to start to have the same times in one side on the other so these are mainly due to two big challenges that we have on the topic of so do you use kubernetes to orchestrate we are today we are using virtual machines to run the case so we're using slurm - there is another queue manager to execute the cases but in our future what we are trying to do is creating this spiritual population of hearts so you can try different devices on different hearts and for that we are planning to use containerization solutions that are being developed right now and in that moment we probably are gonna use Cornell is to manage those applications because if you need to run ten thousands of different infertile patients to to test the population you need an efficient smart workload manager and so slurm was a orchestration workload manager for academic and specifically in biomedical use cases where it's been mostly used right it's like it's a workload manager generally used for some computers it's like race no specifically for say and it also has some benefits around the topology like for instance using our DMA or any or things like that right okay okay if you have a workload that's in super for supercomputers you usually have a very fast interconnect among yeah so and also it's true that this type of workload managers can be simpler because generally supercomputers you have like more homogeneous classes and we have everything on site so you don't need you don't need a smart tool to distribute this type of jobs along and the genius classes so you can use this image in this class or it's much simpler to run these type of executions in a supercomputer compared with cloud environments yeah one thorny thing I remember from some of the HPC programs is that if one node goes down the whole job go back to the last checkpoint I guess exactly and actually theoretically well the idea is that it does because once you have your execution on a container it is independent from the others so if that nodes for some reason fails you should get the other the other hundreds of thousands of executions and Charlie you're doing some pretty heavy lifting when is computation indeed nothing as fascinating as reproducing the human heart so we essentially ingest a huge amount of transport data whoever that is the driving dates a walking data cycling and most importantly the public transport data that we take in nice and you would consider public transport data to be fairly static and the same but a change is actually much more regulated and you believe whether those services that are running one day as engineering works and when you multiply that not just by the city but to the country level and then in the 35 countries that we're in and the size of our organization there's about 30 people is a really hard challenge especially when you're then trying to actually have a product that people can can use this is real time so you know we we took our journey in terms of high performance compute really it was more of how do we get a product we can sell so it was you know really on a shoestring start up how do we take this this data how do we refresh it in such a way that we can put it back onto our runtime servers and have everything available and the code that we were having we were running was really quite bad in the sense that it had grown from the sense of you know we've got this country let's out this country we've got this data formats at this data format and it just got big that sort of Leviathan process that comes out of those ideas that when you start then you get someone paying for it and then you've got puke clients relying on it and suddenly you don't know which way you should go you know your technical debt versus your potential revenue invite all your new near-term revenue so when we start speaking to Oracle and then introduction to kubernetes we were actually started using the bare metal instances which are these really really high performance instances that we were using to take for example all of the public transport a tour in Germany Israel Spain or the whole of the United States for example and making sure that the data coming in is correct because most of this data has been put in by a human being at some point human beings are terribly unreliable including myself and afterwards they're making sure that it all makes sense but as we replenish that data that it also makes sense in a historical term so you can't now get from this side of the country to that side in 90% quicker than the day before for example and this problem just got more complex and more complex and more complex and we kept on adding in other layers and other countries to the point where you know to process our data was taking longer than it was taking for the data to be valid so the date was being replenished faster than we can replace our datasets now that was even more complicated when more and more users started using it through through our clients because people get very attached to travel times 35 minutes they may turn around and work I can do it in 33 so we always need to be very transparent and very clear around the data sets were using how we gather those data sets and how we process them and what kubernetes enabled us to do was to basically take not just the countries but down to the cities and even the modes of transport and those data sets and test them so we can push them out we can basically just take one country or one different data set whether that's in our data team to the data research team who don't have to be that proficient in the encoding or or our entire stack but they can launch they can test they can put that back and then that can go into the city data country data and so on and so forth until we have what we believe to be you know the best dataset we can get to deliver the product to our clients and they're actually so it actually took us down to to be around those 35 countries in about eight hours so we can just do that several times a day now instead of waiting for over two weeks and so that that actually enables yeah the business model exactly so for us you know we're again not as sexy as the the human heart if the human highs is sexy but in terms of actually you know for us we've we read in taking that much investment at all as a company so we've always been quite scrappy in how we've gotten got clients and how we've maintained them and for us it was just a state-of-the the desperation of trying to serve those customers and those near-term revenue versus the technical debt that we had built up and not just technical debt in code but just the process of how we're doing stuff one of the other things we're having is that we were able to just run servers all the time mm-hmm and part of the thing we're trying to do as a company is to get more people to use public transport and to limit with some of our new products to limit the amount of vehicles that are needed in an urban area to reduce pollution levels and it sort of seemed a little bit contradictory that we were then just powering up all these servers these are very hungry hungry hungry machines and we've actually then reduced our carbon footprint of our environmental footprint as a result as well and which one I get some analytics out of that as well to show that the whole the whole of what we're doing is really geared towards helping people reduce their emissions but also that we're doing that as well and that's sometimes the thing that gets missed and I'm not a massive you know I'm not here always talking about the environment and but you know we've seen recently especially in the UK big amounts of protests around this but there's lots and lots of servers out there running lots and lots of stuffs probably very inefficiently and kubernetes has helped us become a lot more efficient nice-nice it's a fascinating technology do it you were demonstrating this before but yeah it's a app travel-time taught me if you wanna have a go my marketing team now they can tell you well you can go this far in 30 minutes or 40 minutes yeah are they're giving you like five miles it can actually tell you all right yes the perfect use case for here is always and I need to be not just in Barcelona but at this conference center many conferences you know majority of hotels get booked up very very quickly and then everyone raises their pricing though you could stay in a hotel within 45 minutes from here that could be way cheaper other hotels and it's providing is removing that guesswork from those location-based searches so it's not seeing everything on a map and then copying and pasting that address into another application and finding the route it's having all of that stuff up front so that if you're trying to monitor locations online whether that's property recruitment hotels anything like that we can remove the guesswork and make sure that there's much more relevancy driven in the searches themselves it's also I would imagine very handy if you have to get to a birthday party and don't have a gift yet if I arrive late for a meeting [Laughter] yeah that's why I say all my watches closed about two hours early well uh TJ I'm hearing a lot of demócrata democratization of what was formerly I guess you'd go HPC workloads and I know what's like the you do the US laboratories you have to if you have a big job you want to study the human heart you have to put in a request to go to Lawrence Livermore and maybe you'll get the hours this sounds like this is kind of opening up a lot of potential work yeah I mean right like basically what's been locked up for a lot of people using supercomputers the idea of like I have an API and I can go run my job whenever I want to at the time that I need to do it or I can and that infrastructure doesn't always have to be you know a Mirta sitting around waiting for my job to run right like both of these scenarios are being talked about and that's really what the whole cloud premise is supposed to be there you get a billing model that you only have to pay for what you want you get some that you get greener experience for that because then everybody can start spinning down those things and from a supercomputer so you're not waiting in line for everybody else to complete their job when it's time for you to go run yours you can just do it on demand when you're ready to run that well yeah and also you don't need to have physically supercomputer in your office that's right I mean that's expensive and there's a lot of place and energy I mean you can run what we're doing here it's like trying to put the up this super computing power to the people's hand the clinicians fingertips right so this is the important the critical thing that the cloud is supporting yeah it's really just making everything as lean as possible absolutely in every single way the delivery with the physical location of it the energy driving it and the availability of it as well and you know we we have guys that use our data are in a sort of GIS and so there's sort of business analytical sense and they were running some of them very heavy-duty machine they were expensive machines they're just sitting there they may get a question that requires those answers maybe once a week but that system is always there we cannot deliver that over API straight in their application and they've only used it for that yeah and so like all of this is really important and you know it's if it if you think it's not as sexy to model the human heart talking about building infrastructure is really not it's not that sexy we didn't get it into it I mean it's nice to know that people are using it for either for improving my compute commute time yeah or for you know saving my saving my life but we didn't build it we didn't build the cloud specifically with those ideas in mine but we knew that we what we wanted to do was actually really go out there and solve for any kind of use case our particular use case that we thought about in the beginning was like the database like what can we do you know obviously more Oracle we have really yeah big business around database so we built things that we need a really good for us to do that it turns out that it works out really well for the rest of HPC workloads so we have you can run metal instances we also do you know our DMA so that you can actually have massive data sets and then address them across all of that with a little latency or at like hunter gig between that there's no other cloud that you can do that on today and it's not because those other clouds don't see the value in it it's just that we build it because we have these databases and that data is you usually at the heart of a lot of these workloads and so you know really we were optimizing around that and it's been really fun to work on you know solving these really complex use cases and then being able to attach it to you know modern state-of-the-art technologies when you're using your supercomputer you weren't really thinking about how could I ever use containers necessarily to use that that wasn't you had your tool chain you had all of that that life cycle and that's what you were going to use but now that you're in the cloud leveraging those things you get to think about you know how would I solve this problem today with micro server you know you got a front end you've got a back end that's doing this you can split up that it's the concerns and you can use whatever tools that you want to solve those problems be it you know whatever orchestration management that you have or kubernetes containers or VMs or bare metals if that's where your workloads can run so our job you know so we really get out of the customers way and give you as many of the building blocks that we can to solve those problems yeah and you the quality of the services that especially the speed and the performance in the late their low latency and the other stuff around that has been really helpful for us but it's also you guys do a great job because you are selling an infinite proposal in a finite world right unlimited computing power whenever you need it but we need to make sure we have those computers it's not as if we can just click fingers and suddenly they just multiply yeah there is actual physical infrastructure out there that has to be there but things like kubernetes can make sure that that is being used much more efficiently that's right yeah we think that's where it really really is helpful it allows you to what we use it ourselves to increase our own density which allows to reduce our you know our base footprint and then we hopefully give you the ability to leverage it for your own purposes so you can drive up density and use smaller instances if necessary or etcetera and then the really thing is having as much of a big pipe as possible to get the data into the region right like yeah their data is and that's the expense in many ways getting in the data there and then computing on it is secondary how do I get the data there and how do I know that it's always going to be available for me to run my simulations on or my analytics nice so what is the preferred configuration for HPC style jobs I'm hearing bare metal quite a bit and so bringing your own district distro containerization of course provider we have a variety of shapes that you can leverage from virtualmin small virtual machines to virtual machines with GPUs to bare metals with GPUs to even our HPC shapes which have our DMA 100 gig connect right so like you can pick any of those that you want after that I'd leave it to you yeah I mean for us it's understanding what is available in the balance you know how quickly do we want to do something versus the cost of that you know things like the availability of those instances coming at a cheaper instance at a certain time can we run the process then do we need it done in a I was 15 minutes as a I was 10 makes it you know for us it's not the certain about the runtime environment about accessing it straightaway but those batch processes that we can really shrink down and then make everything else more efficient on top so it's a different question I'll ask I mean you know there's lots of things that we could take advantage of GPUs but at the moment we dedicate everything on just the standard CPU and then high very high levels of memory as well and we course we do go right down to the bare metal which is that's what the idiot that's what the the service is called bare metal but then also virtual machines on top of that and we don't do any of the the service containers at the moment but just at the moment distribution or by docker it etc yeah yeah in our cases something similar we choose the solution strategy depending on the type of problem that we're trying to solve now we have this is some sort of hierarchical pyramid scheme of simulations to date yes that you can run a lot of very simple cheap simulations so for that containerization solution system is the best and on the other extreme of the range is when you have very expensive detailed simulations that has a lot of fittings inside and that they are very complex from the physical and computational standpoint so in those cases we want to run bare metal because we want to run us optimized and close to the hardware as possible right so we also target I mean it depends for each case what you have to use to get the most of it well where do you guys store your data do you use like nvme or to use object storage or we use a combination of both we use nvme for the the runtime environments like pulling out more of the detailed deep data so not just like the top number what we do is about returning a time value but then we do fair cost then we do route description and we're description is gonna have stuff like the agency name the room for that we have to have over here many stories too fast deliver then the object storage is great for the historical stuff so you know with we might be used on will you get used on so many everything so like trying to find a property trying to find a job we get using the UK to distinguish to make sure that betting shops aren't within a certain amount of time okay because there's a competition Commission around that now that's not something I ever dreamed that we would ever be useful but that gets used and in some cases that's a commercial well it is a very commercial responsible set of data and if there's a problem with it we need to go back and save that data but when data's moving sorry when the proper transport they are driving is changing all the time not only do we need to keep current but we also need to hold all that data behind us as well and that's what the object stores very valuable to us because we can just retrieve it so in our case it's mainly nvme because we the configuration of the case is really simple actually we actually have a problem yeah from that we are solving now that is with the results because sometimes we have like gigabytes of results right right and you need to develop that to the web client that may be on the other side of the world right so we are trying to optimize the result that we get so we can only provide and transmit to the other side versus the slices that the a clinician acquires right interesting it's do you guys do a lot of shared memory between among servers I remember that was a thing with HPC yeah yeah yeah again it depends depends on the case yeah but um we generally we generally need to you need to run more than one node at time because these are generally large cases again if it is a small case where you those type of cases that you run maybe 10 or 20 on a node you just use from process but for the larger case is more complex with more physics cases we do run MPA and we need to we need to use this route in memory to solve those cases yeah I mean so on the database side of things obviously and the the Oracle cluster is one of the reasons why we have like you have these massive data warehousing kinds of things and there's this compute jobs that need to run across that and so that's one of the main reasons why we built we included our DMA in our in the clouds to be able to do massive memory allocations but also these bare-metal instances already have a ton of memory in them yeah so like north of a half a terabyte of memory and so it's it's if your memory if your data set can fit is less than 500 gigs or whatever you can okay there's always a memory now is it you know cheaper I was actually discussing someone earlier on about ages ago there was a flood I think was in Thailand where they produced all the memory memory parties shut up computers just to steal the RAM because yeah just so and then now you can have a terabyte instance like that it's just it's so different but for us again it's another balance of do we make code more efficient than the data structures more efficient when we produce stuff that we need less memory or can we move faster and do more we know the code could be better all the time who wants to do that right do we need to do that and there are things that we would like to do the shared memory across multiple instances especially if we were to do to a multi-country lookup so you know if I'm a Heathrow Airport in London or Barcelona Airport where could I be within 12 hours of cross the globe that would be very interesting but use cases right now in terms of our where we're looking in clients is more sort of city or a country based and the surrounding countries not air travel or the globe yet nice not summer I'm very curious about is the effect that the cloud will have on the traditional HPC community and you know these are a lot of government universities and other academic labs they're buying hundreds and thousands of servers and they're trying to get it just a boot up basically I mean while there's my competition with the US and China who can have the biggest computer are there jobs that are inherently supercomputer bait that could never go on the cloud or we certainly have not found that limitation yes I think the answer comes like from other side National Laboratories for supercomputing are a way of fundings for computer resources for public research right right so on the cloud Hospital target has like industry and private business right so I think that national supercomputing centers will never will actually never fade away because you see will be doing HPC research sector but the cloud services will start to grow because the truth is that the part a part of the problem with HPC applications to the industry is that it's hard for an industry to access HPC resources on National Laboratory yeah right so that's what allows the cloud I also have the types of computing power to the industry so the I think that the National Laboratories will never never go away but we are gonna start seeing more and more every time have these cloud services and clarifications and what as I said before like supercomputing powers are the fingertips of the clinician Sun everyone you think it's possible that people would when you're building I'm not coming from a background of building these kinds of ecosystems would you prototype your the workloads and the jobs in the cloud and then when you have time when you can go actually run them on the supercomputer you would know that it's going to work I never had never having not had to debug yeah yeah yeah the truth is that the cloud environment gives you a lot of tools that makes the platform much more flexible than supercomputer right because you're very restricted and what you can do so that's something that having thought but it's a very good idea once the the tool on the cloud is pretty well developed you can run a thousand of simulations on container solutions and then when you want to run the largest case though you need to use 100,000 course or whatever you can computer that's a solution that I see do you think as well the the question of whether authorities sort of governmental authority doesn't have their own it's still going to be down to the security factor of not putting it on the cloud as well on that the fact that they is that is this sort of ownership you know we did this we ran it here it was inside this university this region this government authority and then also the security layer on top of that yeah well that depends a lot I think on the on the regulations of each institution right so in their case our subpoena has the exploitation regulations for the code that we developed right so we can we can do this we can run on the cloud and then run on the ass computer as we wish but I'm I suppose that it depends a lot of on each research center how the policies are there right yeah okay there's some working like intelligence or defense it's never you know if you job a problem that we will face in the future in some point is what we do with patient there yeah because that's very sensible right you cannot take the images of a patient from hospital and put it back on the on your own cloud I mean nowadays each health institution like meet a large evolution has its own like cluster where they store the patient that and generally spin-off that works with make all data what they do is like running their their machine learning algorithms let's say on the cluster of the hospital and they are taking the train that works outside right right but that's something they train on the supercomputer or in the cloud and then use the model locally yeah so that's a problem that we will have to tackle at some point so what you're saying is there's a world of edge trader edge model verification for clusters that we haven't figured out HPC HPC see folks would be would like being considered an edge well anyways they are yeah anyway awesome talk any closing thoughts any closing ideas about HPC and kubernetes in the cloud so we haven't discussed yet I mean I think I think there's a lot that you know the workloads and the practitioners that are going out there either leveraging kubernetes today or using you know other orchestration environments that were previously used on supercomputers there's a lot that can be learned in the kubernetes ecosystem and there's a lot the kubernetes can do to improve that and make it so that you know the reliability that you look for being able to scope jobs down if you need shared memory being able to you know describe your resources for those things in the declarative way it's still really compelling I mean the kubernetes is you know exists for a reason because it's a lot easier to describe your workloads for those things but and but it it's not meant to fill every gap yet and so there's a lot that it could do so as you guys you know start to identify those things it's it's you know really important that you make sure anything that you see is a gap missing from those things as you let you let people know so that it can take advantage so I think that there's a lot that it could solve for you today and I think that there's a lot that it can also learn and improve to make even better for you guys you know we've only really just started on the journey of using it and that just came down the fact that we had this big problem about processing loads of data and this was a solution from that you know we now run our web our web applications on kubernetes because it was just the next natural thing to do but there's lots of stuff in the runtime we could do you know if someone does a search in Japan and we need to deliver the data right there for that instance at that you know that's exactly what we can use it for and yeah we've got a lot more to do yeah in fact too much yeah of course I mean you need to use each solution for each problem I mean for each problem that you have there is a best fit solution for that problem right the truth is that cornetist has been like in the market for not so many years so this is a framework that the yessiree damn the supercomputing key monitors and the resources resources for public research it was what our having deep use undeveloped for the last maybe 50 years right so but is now this new to appear and it's much more flexible and you don't need to have your computer under your desktop so so it has a lot of future and it's I don't think that it's gonna replace all everything every other tool that is actually now but it's gonna give you a new it's gonna give us a new flexible tool to to attack some other problems that we couldn't attack or weren't efficient enough with the tools we had right now nice alright well thank you gentlemen thank you so much thank you and really appreciate it thank you and I thank the audience for tuning in and we'll see