Create Actionable Visibility for Enterprise Cloud Applications and Resources - AWS Virtual Workshop

Key Takeaways

[Music] hi there and welcome to the third episode of this series and today i'm gonna give you a quick overview of visibility aspects of systems manager as well as a demo in the end my name is felipe de bene and i invite you to join me into this journey as i mentioned previously the agenda for today we're going to start with a quick recap to assistance manager i know this is the third episode so by now you should be very familiar with the tool so i try to be as brief as possible there and also i'm gonna attach the four uh essential modules which allows us to to give us a better visibility into our environment as you guys know at this point systems manager is part of what we call aws management tools specifically focused on the operations and compliance management and at this point i hope that you guys are already convinced of its benefits but i'll i will try to bring them briefly so the idea behind the visibility is to short shorten the time to detect the problems to automate the risks increasing efficiency improve visibility and control over the infrastructure uh whether this infrastructure is on the cloud or on premises because it works on on hybrid environments and also uh gonna help you guys to maintain security and compliance because it integrates with our security and compliance tooling i have highlighted on the screen the modules that we're going to touch basically on our demo so we're gonna give a little show on uh systems manager explorer we're gonna get a deep dive into op center especially the ops items we're gonna see a short demonstration on application manager uh we're also gonna go into fleet manager we're gonna run command to finally fix the issue at the end and we're gonna connect to our fleet or our instances using session manager throughout the place as i mentioned before uh this is kind of a repetition so we're going to start by using quick setup it's the easiest way to set up systems manager and i hope that you guys have already had a lab specifically on this uh the most important thing i think about quick setup is that it enforces the best practices on your environment and it can support multiple accounts in multiple regions and and it's going to ensure that your our setup gets compliance all the time so it's a great tool we have great way to to to start with uh next as i said we're gonna get into explorer explorer is more of a dashboard so the idea is to give you some insights on operational issues across all your aws account and regions and also work as a consolidator for other tools as well such as compute optimizer and trusted advisor check so it's a single pane of glass for our operations and we're gonna do a little tour of this part once we get the demo running next this is the focus of this presentation we're gonna dive into up center option of center is designed to help you guys view investigate and resolve operational issues so the main concept that we have here is that uh you're gonna have other tools for in this example cloud cloudwatch that's go they are going to generate uh what we call upsighting or an operational issue over which we can prioritize and even respond using brand books or automation documents so that's the whole idea behind op center and that's what we're gonna explore in a minute uh the next module that we're gonna see as well is the application manager that serves just like a consolidator for our resource groups so we can auto discover their resources like the cloud formation stacks or the manual resource groups that you create the launch wizard and even ecs and aks clusters so you can get a holistic view of your resources and how they are behaving from the application perspective and not from an individual resource perspective and that's what you gain from this from this tool from this module we also want to deep dive we're going to spend a lot of time here actually on session manager so schedule manager is a way of connecting into the instances via centralized control we don't need to use ssh keys when we manage the the instances in session manager we eliminate the need of creating busting host of creating viral exceptions uh and also gonna enable people to directly connecting to instances just using iam control so it's a very powerful tool and as i said we're gonna use it a lot there is also fleet manager which is one of the latest uh tools that we're gonna explore today the purpose of fleet manager is to manage your fleet or your instances or group of instances at scale so you can do things like browsing windows registry change values on windows registry browse the file system tail file logs and and all of this without the need of actually rdp or ssh into the instances so it's also a very powerful tool for which we're gonna also explore in our demo slash lab of course there are other features to systems manager which we are not covering this episode episode but uh i i highly encourage you guys to be tuned and uh and check for episode four and five that are upcoming which will give you better insight into the other features of system managers uh i'll leave you guys with some resources with most up-to-date uh information regarding our tools uh most notably uh the the management and governance blog channel and uh and of course we are always open to hear your feedbacks right so without further ado i'm gonna get into the demo okay guys so the idea of this episode is to create visibility for for the cloud and its resources so we are gonna use uh these seven features on on on systems manager and we're gonna orchestrate those together to kind of simulate an alarm so we're going to create a cloud watch alarm that is going to monitor the cpu utilization on an ec2 instance this alarm will trigger an upside of cyto we're going to capture that up cyton and we're going to do some troubleshooting and eventually fix the issue uh using run command so that's basically the idea so uh and in the end i have a kind of an addendum where i'm going to show you guys some of the explorer and what else can you can you get out of it and also uh gonna be showing you application manager as well right so let's start with it uh for our initial setup there are two ways of doing this uh you guys could do using clone formation or you could do this using the manual instructions both of them work the same i'm going to start since this is a a learning session i'm going to start with the manual instructions just to make sure that you that you be able to capture every nuance or every detail that we have on this uh on this demonstration so when you guys are gonna do this on your accounts you can use the conformation just to save time but i think it's important at least once to see how it behaves in a manual fashion right so we're going to start by creating a situ instance a windows in c2 instance so just a regular launch instance something that you guys do every day i gonna get the vanilla windows base nothing special about it i'm going to choose a t family so t2 or 33 would work i'm going to use size median because since we're going to stress the cpu if you choose one cpu you're going to be 100 of cpu all the time and that's you're going to have a a very slow performance when doing the lap so medium is is more appropriate to to avoid this so i'll click click next most of the uh the default value here's are are acceptable the only thing we're going to change is that we're going to run a script to go ahead and install some software on the instance and make sure this uh software is gonna stress our cpu so we have something to work on so we are basically on this part of the guide where we put the uh the script so i'm gonna go ahead and put the script on my end so i'll go to storage i don't have to change anything here the default is fine i gonna add two texts to this instance first i'm gonna name it windows test instance and also i gonna put attack so i know this is from the ssm workshop that's basically it the value is true let's go to the security groups we are not connecting to this instance uh at least not directly so the only thing that i'm going to change here is i'm going to put the source as my ip address just in case i might need i need you to connect but uh that won't be necessary because we are going to be using system manager right so let's go to review and launch let's take a last look at the details and if i'm fine with it i'll just go launch as i said we're not connecting to the instance so it doesn't matter if it doesn't have a key pair so i will proceed with the other key pair and i acknowledge and there you go launch instance the instance should launch uh shortly there you go we have the instance running up and running so at this point we have advanced uh pretty much the first step the next step we need to do is to run quick setup as i mentioned so it configures session manager on my environment okay so the way to to create or run this the quick the quick setup is going to system manager you're going to see here on the left where it says quick setup click there it's going to list you all the configurations that you have in my case i don't have any so i'll go ahead and create a new one uh it's going to ask you which kind of configuration you want we for now we just want the host management to be active right so we're going to choose host management choose next most of these values are already uh acceptable so it's gonna create an association that will make my agent get update every two weeks it's gonna collect information from my instances every 30 minutes and it's going to scan my instance for for missing patches daily right so that's what i want to do i want to deploy it to my current region i wanna enable it on all my instances and that's it so pretty much default configuration now create click create so the tool we're gonna start working it's gonna launch a cloud formation and it's going to update me on the progress on the on the interface right so i just have basically to wait so if we go back to our instructions we are basically uh after creating this creating the host management we just have to wait until it is successful and once it is accessible we're gonna see all the associations it created based on the options that you chose right so once this is uh finished creating if we click the associations we're going to see all all the ones that are pending in the ones that are successful as of now i just have one pending overall is in success that means that if i go back to ec2 i'm going to open a new tab that means that my situ instance should have by now a security role iam role and this was created by the quick setup you can tell by the name right so these allow me to see in system manager this instance as part of my fleet okay so it has passed like two minutes or so and i can already see my instance on my fleet manager so that tells me it is configured for system manager and the quick setup was success so the next step on my guide will be create a kms key because some some aspects of system managers require you to give a customer a managed key on kms so we're gonna go through that part just to make sure i can manage every aspect of my instance if you leave the configuration just with the quick start there are some features that not gonna work right off the bat until uh especially session manager until you give it a kms key so let's go let's create the key so we're gonna go to kms i'm gonna open a new tab because i'm going to need to be changing tabs i have already two keys but i'll go ahead and create a new one just so you guys can see how the process is i actually gonna use the default value so it's a single cymetric key i'm gonna give it an alias a descriptive alias so i'll call it session manager key 0 3 because i have 0 1 and 0 2 performed before i'm gonna fill in the description click next and next it's going to ask me for the administrators for this key so this is basically my user if you guys created the iam user that come into the instruction as an optional you're going to have to show to choose that which i'm putting here or uh you should put your your regular user there so mine is aws reserved administrator access that's the one so i'll go ahead and click next uh who is going to use this uh this row this this key so in this case uh this is the row that got assigned to the instance so i'll go ahead and choose this row and perhaps my give me my own user as well if i wanna manage using that and that's pretty much it i can go default until the end okay so my key here and and it's created okay i'll just have now to give uh permissions to use this key so i have to go to english identity and access management yeah i'm gonna look for my role the quick start rule so i can click quick search here there you go i'm also ssm grow for quick instance setup in this case i already have a key polish for key number two which i'm going to go ahead and delete i don't need that i just create a new key so i'll go ahead and create an inline polish i'll go to jason to put the json and you guys are gonna find in your guide the required ian permission so i'll just go ahead cop that paste it i have to replace the key id so i'll erase it here and i lost the kms okay so let's get the key id again it's k03 copy the rn and paste it here and that's it review police all right so my key policy is here and i have the other two roles that uh that the quickstart create created for me so i should be good on this part so i created the kms created the permissions uh now i have to tell a session manager that my keys uh the keys that i wanted to use so i'll go back to systems manager go to session manager there you go here in preferences when you do edit you have to select i want to enable kms encryption and i want to use this key and that's it right all the rest and you can leave default save it says it's enabled okay so what can i do now i have enabled the uh the kms so let's confirm that we still see the the instance here actually you can confirm by using fleet manager and if i select the instance and go to instance actions start session i should be able to connect to my instance and as you can see it says that the session is encrypted using aws kms so if i got to this point means that my key is uh is configured correctly with the proper permissions so i'm able to to manage my instance from the aws console right and that's our end goal so we have system managers fully implemented okay so this is the uh the end of the manual part this is the thing the instructions if you guys were doing using clone formation which we're gonna skip so let's go to an next session right so as i mentioned uh now we're gonna dive into op center so if you get to this point we assume that you have already set up the environment the workshop environment which is basically a windows instance with uh with a special script that's gonna cause it to to consume cpu so basically what we're gonna do now we're gonna create a cloudwatch alarm uh to check the cpu credit utilization on this instance and see if that triggers well and that has to trigger an ops item for us which will allow us to react and investigate so the easiest way to create the uh to create these uh this alarm is using clown shell so that's the way we're gonna do it but first i need to copy this to a text editor which in my case i have already done okay so i have to fill in the instance id which i can get from here i get the instance id and fill in in my template there you go uh be aware also that you have to change the uh account values as well right so i'm gonna go ahead and save this file on my desktop so i have it right handy so i'll go ahead and launch my cloud shell if it's the first time you're using this it but it might take you some time to do the first initial setup there you go preparing your terminal and now we have a terminal here so first thing that i'm going to do i'm going to upload the file that i just added and it was start on my desktop put alarms metrics json that's the one that's my instance id so i know this is the right file and my account number so open and upload it's uploading the file okay once it's successful i'm going to go ahead on my guide and get the command to put the metric file alarm there you go so this is the easiest way of using the cli as you can see the uh the file is here so i'll just go ahead and issue the command so it imports the alarm for me i'm gonna show in a moment how that looks like in the console all right looks like it's done so let's go ahead and check cloudwatch to make sure it's created so since i just created the alarm it's in the state of insufficient data okay it's gonna it's gonna take some time to until it reports either an alarm or or okay depending on the value so the idea is that when the instance goes below five cpu credits it issues me uh up citing alert so i can capture that on the uh system manager as you can see the actions here is when in alarm create ops item with severity queue and category cost as you guys know uh the t family uh instance the instance type t they are burstable so it's always good to ensure that you're not consuming cpu crates or that you are bursting in other words right because this will reflect directly on your cost so that's the idea that we are working uh behind here as you can see the alarm has settled already because it has the data points it needs and it's showing me an alarm right so this in alarm must have created me an ops item here in op center there you go i have a couple of alarms from security hub which has nothing to do with this and we have my cloud watch alarm if i click down we're gonna see that the uh a minute ago that's the one so that's by my upside the alarm that was triggered due to the fact that this instance has low cpu credit so credit balance is low right i don't have any run book set up to be executed automatically but if i wanted i could go ahead and and fire up a run book if i if i so wanted right so it also gonna give you uh some insight on the alarm itself what caused the alarm operational data and this kind of stuff okay so i'll show also the related resource details the other up items the description okay so it gives you a little context of what's going on includes including the history right when it was configured where one was set up when the alarm was triggered etc so it's a very interesting to see these things in action right so if it's triggering a cpu issue or a cpu alarm the next net next thing i would like to do here is check to fleet manager or are my metrics for this instance in real time so if i go ahead and select my instance instance action vm performance counters so it shows me real time this is the equivalent of going into the instance directly into a a task manager right so we can see what what the sap utilization looks like and as you can see we are around 50 cp utilization this is due to the script that you guys launched within this instance when you first created created it so we have an alarm on our site and open for us so we have to investigate what is causing this right so this is the part where i would go ahead and connect to the instance and to make things a little more easy for you guys i have created a guide of the commands that you can use to troubleshoot the cpu issue right so the performance counter that we were seeing approximately 55 so we are we are doing good and the guide so we're gonna start a session and first we're gonna issue some commands to see what we what's consuming my cpu or what is causing the cpu to be more than 40 percent right so i'm gonna change that accuracy real quick there you go you have a powershell uh process which is consuming a lot of cpu for 14 minutes of obviously because it was the time that we uh first launched the instance right and if i repeat you're gonna see that this behavior is kind of consistent with the instance so we definitely have an issue here right so from here i could obviously kill the process but that's not what i want i want to go ahead and deep dive a little bit this is uh this is a powershell meaning that i can go ahead and attach a debugger to it and see if i can understand what part of the script is causing the cpu to spike so in this case i have to put the process id which is 4 000 there we go i connected to the process now my next command gonna show me how to get the run space of the powershell we have one available here front space number one so i'm gonna connect to that run space for debugging which coincidentally is also one and the guy there you go it says that the script is hung in a while condition right while a whatever a means is equal to one that it does a loop right so if i want my more context i can go ahead and do at least one it's going to show me all the script that i have so here it says don't kill this script that's a good thing i didn't it's very critical to the workload blah blah blah and instructs me to change a key in our registry from one to zero so these logs are not written therefore i won't have any cpu issue right so basically what i need to do is edit the registry right uh as i said previously you have the option of changing windows registry here or you can run a system manager which is that the route we're going to take or you can run assist a command run command that will change the value for you and make it easier right so i'm gonna go run command uh the command the name for this command is here in the guide so there you go aws flip manager okay that's the one set windows register value i want to make sure that i'm running the latest version of it ask me for the path so if you remember the path is here so just simply copy and paste it uh okay sample up the kind is d word and the value is zero and of course i'll put performing actions to yes and i think that's it i forgot the column there let me check h a name crazy logs sample app the word okay it looks good right so the next thing i have to check is the target so i always choose the instance manually because i want to do to this instance only but in a scenario where i would have a lot of instances we could do the same i will not change any other parameter i won't write the outputs to s3 should be a short output anyway so i'm gonna go ahead and run it and let's wait for the results okay it says it was successful right so if i click here i can see the output the registry value has been configured okay so if i go back to my instance uh conflict manager according to my documentation the cpu utilization is like four percent so it indeed uh fixed my issue right and that's what we wanted to do that's pretty much what i had for this lab so the next session is going to be take a look on the explorer and application manager basically okay so the explorer dashboard we do have since we use the quick start so we do have explorer already enabled so if i go here to explorer it's going to give me like a a overview of my data of what i have my ops items and stuff like that and if i go to settings i can customize my dashboard and tell it what i can see or what i want to see what i cannot see right so that's pretty much what you can do here uh [Music] explorer settings okay configure dashboard these are the sources or or where can i get the data to be centralized here so in my case i have everything enabled so in the guide tells you to enable at least up center so you can view the ops items that we just created and also gives you the option of exporting this data to an sd3 bucket so the idea this is a little out of scope for this presentation but the idea here is that when you have a multi multi-account uh multi-region scenario it's convenient to go ahead and consolidate your your status or offer your or for your explorer in a bucket and then from this bucket create your dashboards right so that's the idea so you can do the uh the data sync in a more scalable way okay so i think that those are two main characteristics that i like to highlight regarding explorer uh for the rest of it well it's a regular dashboard right so what's interesting here is to put more information since my environment was a demo environment that i just created we don't have much to see here right but the idea is that it updates as you can see it's telling me i have security issues they have an upside which is a cost of cycling which is the one that i just work it on gives me a findings by severity uh tells me my compliance in terms of aws config and the compliance summary and stuff like that so i highly encourage you guys to spend some times and see uh what else can you get from here right including the trusted advisor checks and and stuff like that next next session we're going to explore is the application manager so application manager can auto can give you context application context on your resources more than uh more than you can get from individual items so uh in this part basically what we gonna do we're gonna create a resource using resource groups i'm gonna do the most more manual way right so i'm the part if i go to the guide i mean this part here where we are exploring application manager actually here where i click create the custom application okay there you go explorer explorer dashboard create to create a resource group go to resource groups and create and filter by tag right so that's what i'm gonna do so uh this is my resource group the ssm workshop uh let me create one from the from scratch so the criteria is gonna be tag the resource type i'm not gonna change that i'm gonna leave everything the tag itself it's called ssm workshop let me just confirm i have the right one yeah there you go ssm workshop so and the value is gonna be true right so i'm gonna add that so every resource that gets this tag gonna have this uh gonna be part of this custom application workshop great group okay there you go once the group is created uh you can go ahead it's gonna filter all the resources that you have under this tag in my case it's just one instance that tests windows instance so it should be see something similar to that if i go into [Music] uh that basically what you what you have to do for resource groups so let's go into application manager to see how this fits together in the application manager right so if i go to application manager here let's go back to system manager go into application manager there you go uh you guys are going to see here an overview right with everything that it discovered and if i go to applications it's going to see my ssm workshop 2 which is the uh the group i just created it's gonna load all the information that it has regarding this application and from here right off the bat you can see that it got the cloud watch alarm recreated so it's telling me hey this application has an alarm tells me the resources associated with this application so i have the alarm the network interface the volume the instance itself the compliance that this issue has with the aws config so this is also interesting uh shows you the monitoring of part of these resources that is cloudwatch basically so you're going to saw the alarms by by services showing me the the alarm that we just work it on this is going to clear as soon as we start getting cpu credits and going gonna show me the ups items that i created in this case there's none gonna show me the logs the log groups associated with this and the run books if i so wish to choose to run run books and i had configured this right so this is basically the insights that application manager can can give you right and you can see on the overview as well that we support a lot of resources just like ecs aks clusters actually detected that i have ics cluster and has put both of those here for me okay guys so once we are done there are some uh tear down instructions which i would like to to go ahead and and and go with you guys so make sure you don't have any resource left from this workshop that we're gonna cause you to create uh we're gonna cause you to to cause to charge to your accounts right so we want to avoid it so first of all let's delete the quick setup host configuration so to do that just navigate back to quick setup here it's gonna list your configuration so in this case i just have one i'm just gonna go ahead and select it and actions delete configuration okay once it's done i can go ahead and do a delete configuration i'm gonna have to go ahead and confirm that by typing the lead and that should disappear forever and ever there you go no have no configurations yet because i just deleted so this i i deleted the first part my turn down for the uh quick setup next i have to delete the cloudwatch alarm so let's go to cloudwatch [Music] wait for each load there you go i'll go into my alarms here's the alarm we created so just go ahead and uh do a delete and that should delete the cloudwatch alarm as well definitely and that's why we want right so uh we'll tear down the uh the cloud watch alarm there we go let's check the cloud formation stack this is would be if we had done this using confirmation we have not so i'll skip that one let's terminate the ec2 instances that's the windows instance we've been playing with so if i go ahead to ec2 select my instance right click it then do a terminate instance and that should do the trick terminate there you go successfully terminated okay it's working uh and now delete the resource group as well right so let's go to my resource groups i'll select my resource group v on details and click delete and there you go my resource group is deleted and we have cleared everything that we did and that's it thank you so much guys and i hope to see you soon you