What’s Platform Engineering? And How Does It Support DevOps?

Key Takeaways

[Music] you're watching the new stack makers a podcast for people who develop deploy and manage at scale software for more conversations and articles please visit the newstack.io now on with the show Humana Tech is the platform orchestrator at the core of your internal developer platform it lets platform teams remove bottlenecks by letting them build golden paths or developers developers self-serve the tech they need to deploy and operate their app driving productivity and velocity hello and welcome to another edition of the new stack makers podcast I'm your host Heather Joslin features editor of TNS and today we're going to talk about platform engineering what it is the problems that it solves and the connection it it has to uh devops platform engineering was a Hot Topic at kubecon cloudnativecon this fall in Detroit and we're going to dig in and get you up to speed over the next several minutes Our Guest to help us do that today is Casper Von grunberg founder and CEO of Humana Tech welcome Casper Heather thank you so much for having me um thank you for joining us uh Casper can you tell us a little bit about Ximena Tech and what it does Trump so humanitech is a platform orchestrator that's a component that you plug in at the end of your CI pipelines your Registries and it's basically streamlining all of the tech and tools that you have into a efficient digital assembly line if you want to talk to call it like this it is enabling Dynamic configuration management and that allows developers to express what they need and then the basically the response from the platform team will be automatically matched and then orchestrated and it's used by a good amount of Enterprises across the world to build what we refer to as Dynamic internal developer platforms okay thank you thank you and we'll be talking about internal developer platforms quite a bit in this conversation uh we'd also like to thank and acknowledge uh acknowledge and thank Humana tech for sponsoring this episode of makers uh there's a lot to talk about so let's just jump in as I mentioned uh platform engineering was a big Topic at kubecon uh this fall uh let's get everyone who's listening to our conversation on the same page how do you define platform engineering well platform engineering for me is the art of Designing and binding all of the different Tech and tools that you have inside of an organization into a golden path that enables self-service for developers and reduces cognitive load that is very important in my opinion for the individual contributor and thus if you look at the operation teams reduce their burden to do repetitive things and so platform engineers build and design internal developer platforms and help and serve a user so we're always big proponents of treating your pro platform as a product and so your users the users that platform engineering team serves other Developers it does seem like there's a lot of discussion in the field about um a realization that you know a golden path as you as you put it is um a solution that um kind of is replacing the the um do-it-yourself Assemble you know assemble your your your stack from your developer environment from all the all your choices that you have what are some of the problems that that platform Engineers intended to solve I think we're really like in that in that whole conversation especially around that little bit of tension that we had with uh the the question how does it relate to devops and um all of these things I think we need to look back at why do we need to do this why is this a problem we need to solve as an industry if you go back to 2006 there was this famous talk by Vanna Fogel the CTO of Amazon web services and he proclaimed you build it you run it that idea that you should actually have end-to-end ownership and that is conceptually a great idea but you need to see it in the context of the time 2006 was a different world it was a you know low micro service world we were faced with monoliths that we're running on data centers the the digitization was yeah it was it wasn't at the level that we have right now globally distributed systems massively complex um systems we have to serve millions of users so the scale that we're operating today is just totally different the applications are much more complex and so that idea of self you build it you run it in a cloud native world end to end the end to end you build your own end-to-end ownership is to a certain degree a noble dream but unfair towards the individual contributor we're asking developers to do so much at once and then we're always complaining that hey you know the output isn't there or not delivering fast enough but we're not making it easy for them to deliver that's the underlying problem the complexity has grown exponentially but our response to taming that Zoom has not actually adopted and platform engineering is an attempt to be respectful with the cognitive load of the individual contribute to the individual user and binding things into a golden path to say hey you should have the freedom to do whatever you want as long as it's compliant and secure but if you want to focus on what you're really good at you're an excellent front-end engineer you're great at QA you're a great backend you're great at infrastructure as code well then you should also be it should be okay for you to focus on these areas and just to fold to other things in the other areas that the platform engineering team actually yeah supplies for you um what you mentioned it reduces cognitive load on the developers um they have the are there other benefits as well I mean for example in terms of security in terms of just having standardization yeah so the we need to look at their their different people that benefit in different ways we have the individual contributor for them it's definitely uh cognitive load it's reducing waiting times in the Enterprise the huge problem is that you need a new environment you're waiting you need a database in that particular way security is checking it you're waiting you know very very draining so that's definitely it for the for the individual user and then for the organization as a whole for the operation teams it's standardization by Design if you have a lot of people doing things all over the place you know keeping track of things maintaining things becomes increasingly complex there are a hundred thousand ways to express the same state that your service should be in if you're using whatever like a Helm job but if you're able to trim that down slightly and drive standardizing Behavior if you want that is such a relief for the overall organization and that's what platform engineering is about as well for for Ops Engineers as well does it does it help in terms of over time making it easier and faster for them to solve incidents or solve problems because their their self the environment is the platform is the same for every developer and so they're they're learning you know how to solve the same the same problems yeah absolutely so um not only that it's also about um reducing pressure on Ops so if you want to know whether it's a good idea to look at platform engineering I recommend go to your service desk and look at the tickets that you're receiving and if you have things like hey can you debug me that deployment can you spin up and around all of these repetitive requests you know that that's probably like a like that's probably a good time to take a step back and ask yourself should the operations people actually spend time doing these menial things which puts them under so much pressure and it's so draining and frustrating for them or should you actually look at automating that and build these golden paths and build these internal developer platforms yeah um are there any new challenges that platform engineering introduces to it teams it can absolutely it can if you if you if you do it the wrong way um so I think what we've seen in in some of the Platforms in the 2010s is that they've been introducing too much abstraction that's probably the worst thing you can do right if if you're saying well everything is over all over the place we need to standardize completely and then you build a CLI that you think works great in 90 of cases but then it falls apart on 10 percent but then the developer runs into that 10 case that's incredibly frustrating and they they start mistrusting the platform and then that can lead to a lot of cultural fighting and very draining situations so I'm always advocating golden path over golden cages um I like to draw the parallel to natural language processing if you're using Amazon Alexa or Siri or any of those assistants and they work in nine out of ten cases for us as you as humans that's not enough we still distrust right we all know that feeling and that's the same with platform engineering and the resp like the the thing you need to do is layout abstractions the individual users should choose how much cognitive load they want to consume and then actually consume accordingly and they should be able to follow the golden path or circumvent that and if you want to get a more tangible example they could decide to say I'm staying on a on an abstract level I'm just saying my workload needs a database of type postgres but they should also be able to go down and say hey you know what I want to be the one tweaking the infrastructure as code file that then results into me getting that database so that's that idea of layout abstractions that is a good response to that General problem um we may have covered this a little bit already in turn talking about um the impact on on Ops teams but um how how is platform engineering related to um SRE and devops and what kind of impact will it have on if you have a devops team or if you have a you have an SRE team so the the term devops to a certain degree has been abused I would say um we've put that label on almost everything on operations and on sle you know like you know uh people that just help devops is to a certain degree of philosophy it's a way of communicating it's a way of aligning teams that doesn't go away that shouldn't go away you know devops and platform Engineers you know platform engineering and devops are actually you know friends um like like a hundred percent um platform engineering is the practice of Designing these paths that are respectful like it's actually like platform engineering is actually a product group that has a product owner I very much recommend that team to have a product owner that has a user the developer that builds a product the internal developer platform and that's not what neither SRE nor devops is supposed to do as there is supposed to be built reliable systems that are secure and that are you know up for the user and that you know default gracefully all of these things devops is a philosophy how do you design how do you design the relationship between teams how do you structure that ownership if you want and platform engineering is a practice on how to build internal developer platforms for a user okay um what are some of the most common uh fallacies about platform engineering I think the the thing that I see most often is that people try to start solving the most the thing that for them feels most obvious so fallacy number one is definitely um I called the prioritization fallacy they start thinking about the life cycle of an application that could be um when I'm starting an application now let's say I'm adding a new microservice then I need to clone a template or I want to clone a template and then I want to configure CI and I want to have that deployed somewhere very often you know that's so the the key thing that new platform engineering teams start off with or another example could be they're thinking about the life cycle of a developer at that particular organization and then they'll start at the beginning and say well at some point the developer joins and that experience of joining should be great and that's again you know an approach to take but if you think about and we're actually treating our platform as a product then if you think about how we would develop a software feature we wouldn't you know be sitting in a room and taking some assumptions and then building something what we would do is we would go out to the user and then actually interview them and say hey what's your problem like what's the most pressing problem and the prioritization fallacy means I'm not treating my platform as a product I'm just coming up with what I think is most obvious but that doesn't doesn't need to be true like how often are you adding a new service how often are you adding like in the lifetime of a developer is the onboarding really the critical thing or the fact that they need to change environment variables every 20 deployments because that adds up you know onboarding doesn't so there the recommendation go out interview your users make sure you don't fall for that prioritization um fallacy then another one is definitely the idea that you want to build that huge platform that works for everything and then scale that to hundreds of users in the first three months that's almost doomed to fail there is a reason why there's no vendor out there that gives you a heroku-like experience across virtual machines um kubernetes Lambda and all the databases in the world well because it's an almost unsolvable problem and if no vendor can do it it's unlikely that your platform engineering team no matter how large it is can actually pull that off like I've never seen that really work so I'm always advocating for start really small come up with what's the future most lowest common Tech denominator if you want you know is that containerization with eks perfect well and focus on that and make sure you actually build a great experience for that for a small group for a lighthouse app for a lighthouse team make them fans prioritize the right way and then show that to other teams and say hey you want to join in okay what's the next cool thing we could build so that's definitely the second and the third one is what I call the visualization fallacy there's so many teams that say well by just visualizing stuff and showing nice dashboards and incredible portals that have our logo on the upper left corner we are going to solve everything and developers will love us and what I find very often is that if you look at the users 12 months later developers don't actually look at yet another interface it needs to be deeply ingrained into the daily workflow it needs to really make a difference on the ground every day every time you go beyond the simple update of an image by definition this is actually when you need a platform to do things um and so just putting out like a fancy portal probably won't cut it I'm always saying you need to think Do you want to build the house first or the front door you know and having the front door is maybe not what's really moving the needle that that does bring me to a question about um how how you make we all know that you know the the saying like every tech technology challenges actually the issue is is actually a people issue how do you yeah um you talked a little bit about you know how do you get trying to get a sense of the cultural changes that need to happen to get developers to use the platform rather than them or to use to use the the IDP rather than you know oh it's yet another dashboard with a our logo in the upper left-hand card Corner yeah yeah exactly and I think the the answer to that is lies in the way you're approaching platforming um again if you're doing this big bang hey there is the portal and why don't you use the developers you'll not see usage like the the numbers will be catastrophic so that is the pull versus push if you push that to the user not much effect if you take a small Lighthouse application that's on your lowest Tech denominator it's already containerized already in kubernetes let's take that as an example and you work with a lighthouse team I always like to use the teams that are known to be open and receptive to Innovation there's always this one team that is the front runner you want to work with that front runner team um and on that Lighthouse application and then you want to do that one thing really good so that that Lighthouse team says hey this made a real difference and it's our in you know this was developed with our input and they really listened and it was such a like such a healthy collaboration between platform teams and that individual team if even if it's only like a tiny little thing like a CI that helps you spin up something or whatever like an optimized CI CD process where whatever it is that small thing done really really well then those Lighthouse teams will create a pull factor they'll go to the rest of the organization and say Hey you know we're one of you and we've been working with them they're they're remarkable they're listening to us and we are developing how you should develop in the 21st century and you can join in do you want to have this as well now that shows remarkable results much much faster and so so turn to those that Lighthouse team or internal evangelists they might call them or early adopters in starting small sounds like those are two of the two of the um takeaways there um what does humanitex approach to platform engineering so our approach is that the platform Engineers are the builders and we are you know supplying some components that help you build amazing platforms we believe in a slightly um in my interpretation more modern idea of configuration management we call Dynamic configuration management the idea is that you are basically describing your application in all possible environments in a almost like a baking recipe and then you let that run through your existing Version Control pipelines everything into the orchestrator and that's us and you can think of as like a baking machine so we're getting the baking recipe we're baking the the platform we are baking the application with every single deployment new and that allows the developers to now choose how deep they want to go in that design of layered abstractions basically do they just want to describe how the cake looks if we take that analogy um and say hey it's a you know um strawberry cake with clotted cream um and then they sent that against the platform orchestrated the platform orchestrator says well what's the context aha environment of type staging and this is what the how the platform team now wants me to add in the strawberries compose everything and then bake my cake all the developer can say hey I want to go down to the terraform level and tweak everything here and then deliver and that allows for this great standardization by Design low cognitive load developer self-service and it helps you to craft these platforms and we have a lot of different interfaces we have an API CLI UI fully git based version so they never have to leave the version control system or you can just wire us up with your own you know service catalog developer portal um we have click and plugins with backstage and other other portals out there and yeah allows you to develop reliable Dynamic internal developer platforms um if if you want to introduce platform engineering and an IDP into your organization um we know that and a platform team um we know that we're in we're headed into kind of tricky Economic Times here possibly people are we're hearing about companies being more cautious about about spending money about hiring how do you how do you make um the the case um to the stakeholders who who will decide whether to fund a platform team yeah so this is actually one of those cases that's really straightforward um and this is exactly the same thing or methodology that you would use to um figure out where should you start with your platform so the prioritization you can use that to actually build your return on an investment case for your management um I recommend take a wide piece of paper write down all the things that go beyond the simple update of an image I am going to like a developer would spin up a new environment a developer requests a database a developer needs to roll back a developer needs to send me an audit log all of these things and then ask yourself how often do you do this against 100 deployments how much time does that eat up from developers how much time does that eat up from operations and then you build that list you multiply this times you the number of total deployments and the number of or the average salary in your organization and you have a beautiful case right there especially in times of Crisis making sure we automate making sure we are getting the most of the colleagues that we have around us enabling them to really outmaneuver the crisis through great Innovation that's a proven recipe to help you leave the crisis stronger than before right the other approaches you know fire everybody reduce Investments try to look away that has never worked so platform engineering is actually the thing you want to do in situations like uh like now and you know look at Gartner top trend for 23 wherever you look it's the thing to focus on no absolutely I'm in it as we know like cutting Innovation during a crisis doesn't as you mentioned doesn't pay off the crisis is over that's all all I have right now unless there's anything else you'd like people to know about about um idps about about platform engineering well I encourage you to you know participate in the global community of platform Engineers it's a very strong Community very healthy fast growing there's platform engineering.org there are meetup groups probably in your city there's platform con so definitely something that you should you know be part of learn use that opportunity to become a platform engineer and introduce that practice at your organization I think that's a tremendous opportunity both for you as an individual as well as for your organization thank you Casper Von grundenberg for joining us today and we'd like to thank Humana Tech as well our sponsor for this episode and thank you to all of you listening for joining us we'll see you next time if you like this video please give us a thumbs up and if you'd like to see more videos like this you can always subscribe to our YouTube channel we're on all the major social media platforms you can always find us at the newstack.io we hope to see you soon foreign [Music]