A ChefConf Roundtable with The New Stack

Key Takeaways

have it on have it off Jim get on it Jimmy a necklace you're on oh okay I was just asking if if Judi has my necklace which she calls our the conference badges yes necklace necklaces yeah yeah yeah so this is the first time we've done this is we're calling it past the mic so our first try at this and the idea is just to see get a chance to talk to more people who may not necessarily have the chance to talk to during the course or a conference and so Stephen thanks for joining us appreciate it it's good to be here Stephen agile technology solutions correct that is correct so agile technology solutions is an organization within the University of Kansas so we work with a Research Institute there to develop software to help them with their their projects their grants and things our flagship being one that does testing for the students in Kansas cave k12 general assessments and such so so what is this software it's a general assessment software that's what it is it is an assessment yeah it's a kite assessment and the the kite is a collection of applications that allow people to develop content for for testing to deliver the that content to students and to pull in the results of that and report on it so so what brings you here we have been working with chef at this particular group agile technology solutions for six years and have had a lot of success with success we recently used our chef our chef made it very easy for us to move to AWS you did yes hi how do you define easy well yeah so so moving to vws have a lot of complexity to it like learning how to set up security and networks and things like that one thing we really didn't have to worry about and this was how do we configure the ec2 notes that we're going to create because we already knew how to take a blank ec2 node and make it our applications so each each of the different components of our applications we're being deployed to ec2 nodes and so we were able to focus instead of on lifting and shifting say our applications from our non cloud environment and instead focus on how we could best leverage the cloud so so tell me about that when did that migration start that migration started in October of last year we had testing that was coming up early early on the spring and so by the holidays we had to have an initial environment test environment in AWS and so within that first couple of months we had established most of what we would need in order to get all of our production into AWS we spent the next couple of months making sure that we had our production up there and transferred over to AWS and that all the pieces were in place to keep it secured and monitored so what did you establish that how would you establish what you need how did you go through that process so we knew what we were using at least four things because we'd already been in production for for several years so we knew that there would like to say be a component for load balancing we knew that there would be components for monitoring those applications so is mostly just a matter of learning what those pieces were in AWS and then modifying what we did just a little bit so that it would fit within that structure mm-hmm-hmm and then you then began the deployment after that and Tommy what was the what was the whop view was that you know the point where you were most concerned about in terms of deploying that you know over to AWS where there was it in the transfer the data what was what was it that was the come or what was what was it about it that made it challenging the well a lot of it was just because we we didn't have a chance to like really in advance learn as much as we would like about a WN most of our concerns were just around what we didn't know right we chose AWS though because of certain things that it would be able to provide us that we did know but didn't really want to worry about like being able to use RDS for our databases was a big win for us we aggregate our logs using and an Elk's daca elasticsearch and log stash in Cabana and so we were able to translate that over into tools that AWS just provides and manages and backs up for you one of those also was we didn't have to worry about a chef server because of opsworks chef automate weren't easy for us to deploy it was in a just basically say you want wine what size you want it to be and it builds it for you it makes it available so so did you so you started with using AWS tell us what the services you started with yeah first was just carving out the VP see the Neff yeah and establishing the network rules that was actually a good portion of what we were learning to begin with was just how do we build a secure Network how do we carve this up to provide some isolation for our production environments in our development and QA environments mm-hmm and so that this pretty much we're where we started in there then from there it was get a see two instances in place that provided our application get the database servers in place that would have our data and then trying to learn how to tie all of that together yeah how does scaling groups was one thing that we wanted to be able to do as our work load rises and falls right seasonally as well as during the day and so we wanted to be able to reduce those costs in the cloud so that meant setting up out of scaling groups which we didn't think we would be able to do originally but we're able to like work in overtime now you're in the management yeah well I'm somewhat in the manner just kind of it's kind of like the house that's done but you know the tile maybe need to be put in and yeah so we successfully went through our first testing season and in comparison to previous years we pretty much sailed through there was a lot of nervousness going into it because of just not not having a full understanding of how to predict how this load was gonna fall out you know how this was going to work for our users but as we started watching it we gained greater and greater confidence in what we had deployed and it just was that in the monitor and the monitoring done yes and in the monitoring of it we were monitoring the loads on systems to see how they were performing and how that compared to what we had previously we did some extensive load testing previous to that and we're able to actually push our loads higher than we could in our on-premise deployment so we were feeling pretty good after those came through but you know when your load testing you're not quite sure exactly how that's going to match to real-life experience we are using to do the load testing oh good question that I am spacing the name right now like whoa is it that you use we were using an open-source tool yeah it generates that those sorts of queries and loads on the system huh and just right now that name of escaping me so so what so what you find in that load testing did you find the works or some of the anomalies he started to service well it was more like what anomalies we did yeah what you didn't see yeah I guess in in previous years there was like a tipping point that we would hit in which case some of our nodes that served up and an API for our app an internal API would become saturated and fall over often due to slowdowns and talking to the database in this case we were able to size it correctly and didn't run into that we we never like finally we kept going like higher trying to see where we could actually make it fall over and when once we had like exceeded our expectations several times over we were like okay we don't know what exactly we were like burning up more CPU trying to push them over then we were actually burning up and answering the queries and so now you're yes oh now you're going to be moving into that management phase yeah so now now it's like what can we do to start using using more that that AWS provides to manage and monitor those systems and also looking to use more of what chef automate can provide especially in providing auditing of our systems like currently do periodical auditing and we would like to move that auditing up so that it's continually audited and we know that those systems are secure so so chef ah to me have you used it before just didn't use it up until the point that we started to move into AWS and even there my experience was with chef server and I just once deployed I chef automate I used it like it was chef server and so now now that we've made it through that first trial period or that first period of that production period actually and we've sailed through that now it's like okay now we want to like start looking at what we can do more of a chef automate so what is the difference for you between chef server and chef automate them currently there wasn't a lot of difference other than the chef server that was provided to us through ops work chef automate was automatically backed up it's automatically updated so there was less operational need to like fuss with that server to keep it up-to-date so what it will I believe look like is that chef automate will begin running in spec checks against our hotter so you be running inspect them too so that the inspector then will will tell us whether or not our systems are in compliance how about habitat then habitat is looking looking very good for us right now once we get to auto-scaling groups and we were rebuilding systems we had that that build process was still taking us like 15 minutes or so to bring up a new ec2 instance install shop on that instance converge that system have the application come up which it was kind of a fun challenge when my developers came to me and said this is taking too long and I'm like 15 minutes 19 minutes is taking too long or like yes it's we need it to go faster in the land so already we're looking at then how do we set up systems so they don't require quite so much configuration and docker is high on our list kubernetes possibly as well and providing some uniform way to do that that makes it easy for us to package and deliver all of our apps in the same way is a big win for us so habitat I think will help us to move more of that automation of the application itself to the development teams where they can have more control and power over that as opposed to automation of the configuration of the server which they don't always care about so much well Stephen sounds like you guys are well on your way what do you what do you want to learn over the next day or so here the next year or so here I am like digging into whether or not habitats going to be a win for us and so I'm looking at that I'm beefing up my knowledge about inspect uh-huh yeah so those are the main things plus I'll be talking a little bit about this tomorrow on stage during the keynote so great well Stephen what you see them your last name again Stephen Figgins Biggins yeah and at SFI GGI NS great so great well Stephen thank you so much for taking the time to talk with us for our year you're the first person we gotten past the mic so congratulations right now we need to find another person yeah we actually got someone to pass the mic to this is the honorary past the mic here come on come on down - pass the mic there you go thank you very much you're welcome first pass the mic anyone want to take a picture that GD can you get a picture of passing the mic oh we pass the mic do it again come on here we go this is it pass them cheese passed the mic yeah all right great all right how are you we met yesterday why did you tell everyone your name and who you're with my name is Pradeep Mayer I'm its relevance lab yeah what's relevance lab the relevance lab is a DevOps automation solutions and services company we help come help the organizations out there automate their workflows the software delivery workflows and everything that comes along with DevOps and automation which is helping companies move to the cloud embrace hybrid infrastructure adopt various DevOps technologies which are available market confusion so how did the company what's the company's versions worse but what did it start as we we started with automation in mind to help help the market with automation services through solutions solutions we come from a previous background of helping come is fees or high tech companies build products ranga and helping them go from licensing their software to SAS SAS efficient as they call it going SAS and in that process we learnt a lot about how you know how to put put the CI CD in development and deployment tool chains in place few of us came out of the PR Prius company and and decided to make a business out of this mm-hmm and at that time the DevOps term was just getting coined ah so you guys have been around for a while so here was a it's like oh eight oh nine it's just DevOps had just started getting spoken about and yeah let's start it off I remember those early days yeah and you guys DTM upstairs yes we do dev upstairs and which one do we which one where's your hometown it's San Jose Calif yeah yeah guys all right so how's the DevOps how's the DevOps community in the Silicon Valley is that you say it's just like teeming it's yeah its thriving and you know things come out of the valley yeah unfortunately you know things get invented there but you know for us solutions company there's not much of Bigham not a big market out there what is the solutions company anymore what it I mean with the cloud services and everything else you know what are what is the solutions company anywhere what companies and all enterprises out there need is a systems integrator so solutions is call it systems integrator or solutions company what we do is bring together bring together the different technologies with which are out there to help you automate the workflow and we as a company have built the expertise having done this for a number of number of customers of ours so we know how a build management system needs to be set up yeah what our build management system needs to be integrated with configuration management what are the confounder how does what are the different configurations which need to be automated how do you move from homegrown way of doing things to a more disciplined configuration management weighing things how do you move workloads onto cloud in a better way either you do lift and shift or lift transform and shift so which which are the different technologies to use do you use cloud native technologies do you use cloud independent technology strategy so bringing all these things together and and and creating the setup so what are the patterns showing you about user customer behaviors for the adoption of cloud native versus other technologies so what what we are seeing in the market is as people as companies get more aware cloud aware they definitely want to invest in technologies which are more cloud neutral you know cloud agnostic initially they might dabble with some of the offerings given by the cloud provider yeah for example so they want a cloud a cloud agnostic they wanted they wanted Florida because we were just hearing earlier that they think people think that the cloud agnostic stories a bit of a myth as far as possible they would like to be cloud agnostic because the multi cloud way of working is catching on yeah they don't want to be not many strictly large enterprises don't want it tied to one cloud how do you see catching on the multi cloud yeah just through adoption so just to adoption and also people are embracing the clouds which serve a certain purpose you know for example people go to Google Chrome Google cloud if they want to do machine learning and big data work people are looking at as your if they were are there a big sequel shop and want to put their sequel services there no EWS of course has got a head start and they the gordei are stable they've that's so that's how they are kind of distributing their workloads across the cloud so these sensors yes yeah so that's interesting because you have the major resource providers right and they're all kind of you're all kind of building these different services on those resources to fight to you know based upon their own market view yes right right so we you know so so in that then you know so in that then there's this kind of this story about how do you map out kind of your you know the control playing right and you were telling you about the software that you guys have recently developed he tell us about that and how that works and how it fits into this story sure sure so across our consulting engagement one one one one observation or one learning that we had as the Ops teams were struggling with there were a lot of these exciting technology there's chef puppet ansible but to develop and operate them was becoming a challenge so there was a need for to ease that usage of these tools yeah yeah how do you is there an easy way to deploy configurations is there an easy way to you know create an orchestration task which is a combination of different things in your DevOps tools here so using those use cases we built a visual tool which sits on top of the DevOps and what the tool chain there was tool chain out there and helps the end user you know drag and drop and do things much more seamlessly as such so that's our tool RL catalyst and we not only help we can connect to the cloud providers we help provision the cloud environments and then we integrate with configuration management tool like chef closely so we can pull in the chef's code and help them deploy the chef's code into the interest about the shaft tool chain that you're using to help these customers migrate so were you using chef automate the Shakur chef engine which is chef configuration management not chef server chef server using chef server and chef Oliver yes how you distinguish it so I just asked a partisan okay so how do you how you distinguish right now how the way it is selling is chef server come as part of Chef ornaments right like okay i sensing so you so don't get you don't you don't really buy chef server stand alone you know unless you're using hosted chef right which is offering so you buy the chef automate stack which comes with chef automate and and sir chef server and and and the various supporting technologies which help you create a pipeline you know rank for your chef course chef Quran as such right so that's that's chef automate and chef server are the primary tool tools that night are getting used inspect and chef compliance which is integrated which is another big you know product offering that that that we specialize in habitat as we are getting into it now we've done some as we're of close chef partner when habitat was introduced we did some initial work chef creating plants so we are fairly habitat capable as well and we are excited because I in the valley in there are there a lot of semiconductor and networking and storage companies out there and they want to distribute their software in different different packages and I see habitat playing an important you know it could be a very good to you see habitat yes really habitat is a very useful tool to achieve that you know why not just kubernetes why not you know I mean is this end of the day container orchestration tools right so when you produce a container yeah it is can run it right but you need an engine which can build your software package and then give you the software package in multiple flavors as a container as a VMDK package or as a VM or as a as a as a tar file or MSI built in different flavors you might want to distribute it so some end customer won't want a container you know and you so you ship the container when customer might want a VM image you can give that this is a technology I think which can help the ISV community you know very much ship software yeah yeah yeah oh I see so they can offer habitat yes to run with their software and they could use habitat as the control plane - and then kubernetes would serve what role kubernetes is the orchestration orchestration container Orchestra so like if you want to like building workers so if like you're getting started and you're trying to find a way to kind of think through how do you leverage the cloud services really these resources that are there you know so rich at compute storage networking you can build through you have a your pipeline habitat becomes part of that pipeline and it serves as that control plane to integrate with kubernetes which provides that container or support that container together right so I see I see habitat as an extension of something like Jenkins okay so is it considered like a continuous integration and development environment yes it would fall under continuous integration and development environment ecosystem okay and so it'll help you build your product or your application it'll help you package your product and then it will do the last mile of deploying your product mm-hmm that's how we see and fairly exciting and what in so your software then that fits into that is what then so we haven't yet you haven't really build build okay you know we we haven't really worked out how yeah our software is going to work with habitat yeah but I do see our software is the visual medium so we hope to fill in that void almost like a graph on almost like a graph all right very good analogy yeah is your tool open source yes it is okay yeah okay so we have the tool is open source we've you know we just charged for the support and services all right okay okay so once you've seen customers get up onto Amazon Web Services using you know chef automate for example and perhaps starting to use habitat what services on AWS do you see them starting to adopt as they become more mature and they're do you know and their deployments in their management yeah so every organization has to all different make a choice you know again going back to the same point about whether how much of cloud native resources do you want to use right so if you're okay with being tied to AWS you might want to use their did their data weight data warehousing to redshift you might want to use their container management tool container orchestration tool ECS service the storage services s3 if they're their messaging queuing services SQS you know so those are different services that companies start their application start using if you want to be if you want to be cloud neutral then then you go with alternate you know more open source technologies as such and we we come in and consult on those fronts as well okay but it all boils down to you know what's the best thing to do for your application and what's the strategy for the company so what are you looking for from this conference we are constantly you know we want to be lockstep with which chef all the time we are we are a consulting partner of theirs so we want to we want our workforce to be well equipped with the latest and greatest a chef yeah we want to talk to Marcus chef customers yes sir we would like to tell stories about what we have done with some in the in the field with chef and with some of the other tools which go which chef like I was mentioned I do like terraform you know from husha husha car which which glues very well with chef and and so I'm sure there come yeah habitat I'm there and bring a lot about terraform yes yeah so you want to tell those stories and you know hopefully people will sign up for our services that's yeah well great well thanks so much for taking some time to chat this has been great you've been the second gas on past the Mike I think everyone's downstairs at the gaming haven't you having fun playing games down is happy we should be passing the mic around happy instead this is the most quiet it's been up here all day but thanks so much for you we really appreciate your time thank you Alex thank you