Modernize Your Applications Using RDS for PostgreSQL, Amazon MemoryDB, and Containers

Key Takeaways

hello everyone thank you so much for joining us today my name is vijay indian i'm senior solution architect with amazon elastic cache team hi everyone uh thanks so much for joining us today my name is krishna sarabhu i'm a senior database specialist solutions architect specialized in amazon audience postgresql and amazon audience arora thank you krishna so in my current role and also in my previous roles i spent a lot of time working with customers who are migrating their applications from monoliths to microservice architecture or in other words modernizing their applications using amazon services and i see services like eks amazon memory db and amazon rds for progressive pro postgres sql play a prominent role in this journey with our experience we collaborately bringing you today's topic which is modernizing your application using rds for progress sql amazon memory db and also containers and our goal for today's presentation is to provide you with enough information so that you can take informative decision while building modern building or modernizing your applications using aws services as for the agenda goes for the next 30 minutes or so we'll talk about characteristics of modern applications followed by a case for modernizing your application or in other words move to microservices from monolith architecture we go deep into some of the different architectures that we provide here at aws services will design a retail application built on the principles of reference architecture and will go over the components and feature set and followed by demo okay to begin with what are the characteristics of modern applications so what are the characteristics of modern modern applications so these applications can be in any industry vertical known or unknown for example this can be from uh online business store e-commerce media and online gaming it could be cryptocurrencies or space technologies things like that however when we are talking about modern applications there are some unique characteristics for these applications for example there could be hundreds and thousands and even millions of online active users using your website real time and from any type of devices and these applications themselves could be processing high high volume and also high velocity of data where it in turn process the data and provide the insights back to user in a really ultra fast response time space so that the users can take informative decisions so quickly and these applications um this application has to be scaled to a higher degree so that uh it supports the uh the change in the data growth and also uh change in the the popularity of the users of the application so that it attracts more users things like that so it has to be highly scalable applications and also these applica these applications uh or building its application rather support some kind of cloud native economics model for example you can start as slower or you can start a smaller application and then you can add the capacity as as an as and when you're growing again by data or request what second things like that and these applications should support um high agility which means they should support open source api so that if the customers are planning to create uh new features or things like that uh they have these are the the development of this application should support the fastest or fast phase of the development and testing cycle so that the go to market uh window will be smaller and then they reach as many customers as possible so these are some of the characteristics of modern applications let's look at how the how traditionally the applications were built and more specifically let's make a case for migrating from monolithic applications to microservice architecture okay to begin with back in 60s and 70s mainframes are the predominant way to build an applications so some uh i mean when looking back from from some 2000s people characterize these type of building applications as single tier applications and it lasts until like 80s and when client server architecture were introduced usually people prefer to refer that as two-tier architecture and from that point onwards that changes significantly how the applications were built and uh when internet arrives in 90s and as a result of that that three-tier application architecture emerged and which means your application will be divided into three different tiers predominantly we call them as ui layer application layer and also database layer so in three-tier architecture majority of for majority of the applications the client and application layer can be distributed although it's not necessarily all the applications were built that way however there is a flexibility for distribution distributing your your your client and application code and also scaling so both of them were introduced in three-tier architecture yet your database layer is predominantly a monolith in in nature and and this is pretty much the norm for building applications until until 2000s and on this presentation of course we are talking about modernizing applications which means we are essentially advising for moving away from three-tier architecture to microservices which are which also refer to as multi-tier architecture so let's understand what is the definition of monolith application in context to three-tier architecture and we talk about uh some of the some of the disadvantages in this model and talk about the advantages in microservices in next few slides first let's understand what is the what is um the definition of monolith architecture to begin with a monolith architecture in terms of software engineering it is defined as a single tier software layer in which the users the user interface the business logic the data access code all combined in a single program that that actually deploys in a single platform so this complete application with all the business logics across the all the development teams in a in a given company uh will be tested will be tested together and deployed in one program in one in in in production for example um if i take a retail a retail website there could be many different development teams developing different features in the retail website however once the development was was completed they actually tested together for all the services together and uh deployed together in a uh in a single platform right that's so that's what the most likely application means and also this application code base uses single um kind of uh application code to interact with the database layer as well i'm presenting in the screen the reference the sample reference architecture for three-tier architecture where let's say on left hand side if a user trying to access a retail website in this example from from there from their desktop uh the the the connection or the call usually lands in presentation layer where essentially your code base uh still use in some kind of ec2 machine following that the the request lands on business layer where your entire code base again deployed in uh mostly in a single ec2 machine or it could be multiple ec2 machines um where the call call goes to institute machines to a load balancer for high scalability aspects of it but the predominant the main point here is your code base is still mostly uh in a single um monolithical kind of architecture and that code base also has business logic to interact with your database where your database is also uh is is essentially a single database that where you will be used in majority of these cases here so so all in all this is how the uh this is how a monolithical application will be represented using three tier architecture and again as you see here because of this there are some limitations so yeah i i as as we talked about a few minutes ago uh because of the nature of this uh application where we are building the whole different services different components together and deploying uh all of the code based together in in one single platform there it's really hard to scale scale the application for example as you are building and deploying uh in one single platform you cannot isolate different tiers for example you cannot isolate um a database here per say uh to scale just that component you have to uh you have to scale all the components in many cases you have to scale all the components together you cannot isolate and scale differently and also um because different departments or different teams um code base is developed and test together it's really hard to isolate with the component failures fail happens right for example if your code base is interacting with different services or if there is a problem or some kind of error in one service we it's really hard to identify or narrow down that particular service which is causing problems majority of the cases if there is an issue issue uh in this kind of applications all the teams in an organization come together troubleshoot it together and narrow down the the where the issue is and it takes time and because of again by nature of this type of architecture the options to choose the right kind of database or right kind of application layer or the or the programming language is really really hard you have to pick and choose one service that is that is more more suited for all the services right so you you may not have you don't have many options to do that and lastly uh the the fastest you can go and deploy in production is based on the slowest dollar cycle for any given team so so it's not really uh well suited for innovation kind of thing where you have where you have to develop your application fast and deployment uh deploy the application first things like that so these are the limitations now let's look at what are the what is micro service uh how can we define micro service services and what are the advantages of it so a micro service is defined as a loosely coupled tightly scoped independently deployable and scalable applications which means again if i go back to my retail example um if if i pick one team uh in the organization the whole team uh i mean all of their code base and even the database connections everything can resides within the team which means they can create one or few of the services and then they they develop the business logic and they deploy the services independently of the other teams right and because of this the the it's easy to identify with the um what is the service that that was errored out because it is again we are working in the scope of one team and because of that the class radius is also um very small and um and and these are as as we can develop and deploy these services separately we have this option to choose the right database and write the application layer for for my service and i can even scale my application layer and database layer um as and when needed right so these are all different uh different advantages of this service again the whole concept is coming from um containerizing or or or tightly scoping your application and uh and deploying your application independently and that actually opens up the um the room for other um other innovations and also make uh things will be much faster in this way so let's look at the reference architecture how this application uh how microsoft uh microservice architecture will be built so what i'm presenting in the screen is the differential reference architecture um we're not talking much about the the user interface layer in in this in this in this diagram however typically customers use uh something like cloud front turn backed by s3 for providing or responding back to the website in a static static pages things like that but when it comes to the business layer this is where the the this is where the the concept of uh breaking down your monotheial application to micro service application come into picture for example uh here in the screen we're presenting instead of your deploying your code in a easy to based service services you can use a containerized applications for example there are you can use eks ecs and also maybe lambdas right so you can deploy your code in a containerized manner so that you can increase or decrease the capacity for that service things like that and because of that you can choose different type of database databases that are suitable that are suitable for your um for for your for your business logic right so what i'm presenting in the screen is um in database tier we have elastic cache service and also rds which which actually using i mean they're they're being they're being used in this example for different components or different containers thank you vijay um let's talk about building a modern retail website using the very principles we just discussed on microservices so for this what we are going to do today is we are going to build a modern data website uh which will be using it for rest of the presentation and of course for the demo so what we need here is a rich user experience with low latency and of course the scalability and high availability with our micro services so a user will access our new website should be able to register login and browse the products in various categories and should be able to add items to the cart and place the order so these micro services which we wanted to build should be able to hyperscale automatically whenever there is a increase in users days for example significantly and the number of requests they go up drastically on our new website so these microservices should be elastically uh scale and should be able to automatically restart when they whenever there is a hardware failure so as we discussed in previous slides each microservice should have unique responsibility and the cacd workflow for automated testing and deployment cycle that should make our micro services to be deployed very fast so we will be building micro services for for various responsibilities so one micro service we are going to use in the presentation layer it's called web app so which is a ui user management so where i can able to add the user delete the user or they extract the user information and similarly a product microservice which is responsible for product cataloging and management a cart microservice that's responsible for cart management for a particular user and order microservice should be responsible for order management so you should be able to place the order extract the order etc and for operational aspect we'll be using uh two microservices one for monitoring and alerting and another one for orders fraud detection and in the database layer um so let me move on to the next slide so we can talk about the database persistence so here at high level the architectural components for our retail research let's starts with um amazon cloud french so that will give us the low latency user experience we are looking for and the micro services for presentation and business logic and operational so that will be containerized and orchestrated through amazon elastic humanities service so the content the containerized user and card micro services will be using purpose built-in databases with amazon memory db for data persistence layer and the product and order micro services will use rds postcard sql engine for data persistency layer so we'll discuss in detail in next slides on why we picked each purpose multi database service for and amazon eks for container orchestration thank you krishna for next few slides let's look at aws services that we are using in this architecture to begin with let's talk about amazon eks which is amazon elastic container service for kubernetes and to understand that let's spend few minutes talking about humans itself so kubernetes is an open source container orchestration system for automating deploying scaling and managing application application application code so amazon elastic container services for kubernetes which shortly amazon eks is a managed service that make it easy for for you to run kubernetes on aws without needing to be an expert in or in open source kubernetes so essentially amazon eks orchestrate your business business layer using using this container service where you can def you can provide your definition for example you can start with saying how many containers of any given application you need and also in which which availability zones in a given region things like that you can create a definition of it and amazon eks make sure the definition is put in place for example if there are issues where one or two containers goes down for any reason uh amazonic is managed for you which means it actually replaces the uh the failed uh containers for our file containers and make sure your definition is put in place at any given point of time and uh and also it makes it easy for um of managing aspects of it which which means uh the upgrade to a little latest version and um scaling even your master and work on things like that all of that taken care will be taken care automatically for you and amazon eks also support classic global load balancers and application load balancers and network load balancers for scalability and it can integrate with other aws services for example it integrates with iam for security and ebs and efs for storage and so on so forth it does supports multi ac deployment for a high high availability and with by using federated deployments you can extend the service for different reasons as well let's talk about the second service in this architecture and this is mainly the database service which is memory db for for rs so so memory db is a fully durable in-memory database service it is purpose-built specifically for uh for modern applications which were built with microservice architecture uh let's dissect that a little bit so why we are saying as this is the purpose built service for micro service architecture there are a couple of reasons number one the memory db is a fully durable service and also it supports open source apis for redis which means um which means that it will provide agility because uh the because you can use open source products or apis and build your application and also it supports multiple data structures everything all are in memories data structures for example is suppose strings list sets shortest sets hyperlog logs geospatial and strains so so when you when you are building microservice architecture there is there will be a need that we are we we need to choose different type of data representation in other words different type of data structures that's that that are suitable for those those micro services so memory db does provide a wide variety of structure data structures which will which could be suitable for or which for different type of uh application needs and also memory tv does support multi-easy deployments as well let's talk about amazon rds for proskus sql so to begin with proskull sql is a popular open source databases for over 30 years and uh and the primary reasons for the popularity is it it provides a high performance database and also scalable database that matches commercial databases that supports commercial databases and it supports wide variety of indexing type partitioning native json b support and also parallel processing amazon rds for process sequel is a fully managed service that supports or that that's compatible with progressive open source process progress sql and also it supports the extensions like post gis hedge store pg logic and post gis etc and when it comes to performance and also performing high um high performance applications proskas amazon rds process equal is a very good fit because it supports up to 80 000 iops per second and also it can scale up to 64 terabytes in storage and for read scalability it supports um up to five read replicas in multi-use aces and when it come when it comes to security the service can integrate with other aws services for example it's it it can integrate with the iam and thus and the service itself is deployed securely in a dedicated vpc and also it supports features like kms which is encryption at rest and also encryption transit and also it supports kerberos scaling time thanks vijay so let's look into the scaling aspect for our micro service and other various services we picked so i'm gonna begin with aks scaling capability so kubernetes natively provides a rich set of capabilities that allows both traditional and modern workload types to scale dynamically based on consistent and the declarative deployment bar model so the kubernetes cluster auto scaler so it automatically adjusts the number of nodes in your cluster when the parts fail or are we shared you down to other nodes so the cluster auto scaler is typically installed as a deployment manifest in kubernetes cluster so it uses underneath it uses leader election mechanism to ensure high availability but scaling is done only by one replica at any given time so the cluster auto scaler it's a tool that automatically adjusts the size of our kubernetes cluster when either there are parts that fail to run the run in the cluster due to insufficient resources or there are nodes in the cluster that have been underutilized for an extended period of time and and their parts can be placed into other existing nodes the second one i would like to highlight is the kubernetes horizontal part auto scanner so which automatically scales the number of parts in a deployment manifest using replication controller or replicas edge based on that the resources cpu utilization this can help your application scale out to meet increased demand or scaling when the resources are not needed and the third point i would like to highlight is the kubernetes vertical power auto scaler so which automatically adjusts the cpu and memory reservations for your parts and to help right size your applications so in addition to that eks supports elastic load balancing which includes classic load balancer application load balancer or network load balancer so with that i'm going to move on to memory db for the scaling aspect memory db cluster can be scaled horizontally or vertically with zero downtime so amazon memory db can be scaled horizontally by adding or removing nodes or vertically by moving to a larger or smaller node types amazon memory db supports right scaling with sharding capability and read scaling by adding additional replicas amazon memory db cluster continues to stay online and support read and write operations during the resizing operations so you can create amazon memory db cluster with up to 500 nodes so this will give you the maximum memory storage capacity of 100 terabytes let's move on to uh scaling for from audience postcard see consoles point of view so you can scale compute memory and storage with push of a button or using rds api see for example you can able to scale compute capacity in dbr5 machine class family for 12x large so which will give you uh 48 vc virtual cpus and 768 gigafrom memory with the eba storage with 19 000 provisioned iops and and within 25 gigabits per second network bandwidth so in addition to that audience automatic uh supports storage auto scaling you can enable storage artist scaling for amazon rds dbm db instance if your workload is unpredictable so with the storage auto scaling enabled when amazon audio states that you are running out of free database space it automatically scales your scales up your storage when free available space drops below 10 percent or 5k whichever is greater and and the low storage condition stays at least for five minutes you can modify provision divers for rds instances and scale up to 80 000 ibops in 1k iops increments and the storage capacity up to 64 terabytes and you can add up to five read replicas for scaling out read intensive workload for horizontal scaling thank you krishna so when it comes to performance there are certain aspects in this in this architecture that we need very high performance for example if you remember the the difference architecture for this particular application that we're building the user and the cart are the two micro services that needs high performance uh for example uh thinking think of this scenario where if a customer is trying to um log into the website you're using username and password we want to log in or we need to provide the info we need to take the information from the customer and log in log the customer in as soon as possible and make it available for the customer for to search for the orders in a faster way and if the customers trying to purchase something he can add different items to the cards and then um he can we should have the flexibility for the customers to change and modify the cart in any different devices and things like that so the performance aspects of this application is very important that's the other reason we are backing this application with the memory db for for redis so so when it comes to performance for memory db because of the in memory uh nature of this database the response times are ultra fast response times for example memory db does provide single digit microsecond latencies for reads and single digit millisecond response time for rights this was tested out for different type of node types as well and also um memory db does support ultra fast uh graph down to base instance instances as well and when it comes to the scaling of number of reads which in turn comes constitutes to for for overall performance so the memory db can support thousands to millions of reads and write operations per second thank you vijay so let's look into agility aspect so with the eks the kubernetes provides the container orchestration we already discussed in the previous slide which is easier to automate and scale container based workloads so kubernetes framework consists of controllers so that track at least one community resource type so it's a control loop mechanism that reconciles the state of the resource against declarative state of the resource best example would be unlike a thermostat in a room so when you set the temperature that's telling the thermostat about your desired stage so the actual room temperature is the current state so the thermostat acts to bring the current stage closer to the desired stage by turning equipment on or off so capabilities embraces the declarative devops paradigm so it dynamically takes corrective actions uh for your applications based on resources resources desired stage defined in the manifest versus its actual state or the current state which makes your application resilient to component level failures naturally so this makes the managery case a natural choice for container orchestration for our new retail website with the ci cd integration into eks this would reduce the time to build test and the deployment of our micro services for our new retail website so it helps deploying our new website micro services more reliably faster less overhead and more importantly the blast radius so with the micro services if micro one micro service goes offline our website is not affected so the blast radius is going to be much much slower smaller comparing with monolith so without the further due let's jump on to the demo so before i go into the demo i just wanted to explain the how the architecture for our new retail website looks like using kubernetes and persistent data stores so now putting it all together for the demo to build our new retail website using microservices xk so we have discussed in previous slides about each micro service that should have a unique responsibility should be able to scale and should be highly available so we are using manager amazon eks for orchestrate uh for orchestrating our newly built containerized micro services so the amazon uk's cluster has cluster auto scaler which scales out worker nodes when there are parts that fail to run in the cluster due to insufficient resources and when it comes to the presentation layer we have built a micro service called web app so which is the prese front end deployed as a kubernetes deployment manifest with three replicas so a kubernetes deployment it provides declarative updates for pods and replicases so you describe the design you describe the desired state in a deployment and the deployment controller changes the actual state to the desired state in a controlled rate at a controlled rate so in addition we have defined horizontal part auto scaler for scaling out pods when the part cpu utilization goes over 550 percent so it automatically scaling when the load subsides so this is printed by a application load balancer in our case as you can see and in the business logic layer we have built user card product and order micro services and each micro service is deployed with two replicas and using the deployment manifest and with the horizontal part autoscaler to scale out these parts based on cpu utilization and they are printed by cluster ip load balancer and microservices user and cart will be using amazon memory db which is a purpose-built database for the data consistency layer and the order and product micro service is gonna be is using amazon audios process sequel in data persistency layer and the most importantly i would like to highlight before i move on to the demo is amazon audio proxy so in our architecture we are using rds proxy which is a fully managed highly available database proxy for amazon relational database services so that makes your application more scalable more resilient to database stages and more secure so many applications including those built on model and serverless architectures can help large number of open connections to your database server so and they are they may open and close database connections at very high rate exhausting your database memory and compute resources so amazon rds proxy allows applications to phone and share connections established with the database improving database efficiency and application scalability rds proxy it minimizes application disruption from outages affecting availability of our database by automatically connecting to a new database instance while preserving the existing application connections so when failover when failover occurs audios proxy routes the request directly to the new database instance so this reduces 12 over times for rds and aurora databases by up to 66 percent so going back into the visibility layer in the visibility we have using eft stack which is amazon open search service fluent bit and kibana so which consists of so fluid bit is an open source and multi-platform log processor and forwarder which allows you to collect data logs from various sources that including the sources from docker or the kubernetes environment and they send them to multiple destination in this case we'll be sending the content to amazon open search service so amazon open search service it's a fully managed service that makes it easy for you to deploy secure and run open sets cost effectively at scale with that we have to use we are using kibana it's an open source front-end application that sits on top of amazon open search service providing the search and data visualization capabilities for data indexed in open search so with that i'm going to jump on to the actual demo yep so let's go by before i start i just wanted to explain i just wanted to walk you through the setup how it looks like the setup consists of we talked about elastic humidity service out of scaling groups and horizontal part auto scale is for various deployments in kubernetes so let me start showing with what the on the clusters how the clusters are set up in our eks environment so ek cto is an open source utility which you can use to communicate with the kubernetes clusters so in this case you can see i have defined a cluster named ek clu in the region yes east one and the second one i wanted to show you is the node groups and the nodes that are part of the worker nodes that are part of main hdks so what you are seeing it is um then the node group which is using auto scaling group and and with a minimum size of four and and it can scale up to eight and with the instance types it's using tt large and if you can describe the auto scaling group or how it looks like on on what conditions it's gonna scale so what you're gonna look at is the minimum size for the auto scaling group you know you can see here is four the maximum it can scale up is 2h and the desired capacity is full and the default cooldown period is 300 so it's going to look it so whenever as we discussed in the previous slides that you know whenever there is a pod it couldn't be scheduled due to insufficient resources that's when auto scaling group is going to take an action and add worker nodes and let's look into our manifests which is the micro services we deployed part of our our retail web application so what you are looking at is uh various manifests that are different uh in a name space called oak tank which contains all the micro services that relate to operating our retail website so at high level we are looking at bunch of parts so caught a cart deployment which is coming from carter deployment manifest and it's been off two replica pod two replicas for the pod card parts and then similarly we are looking at order deployment product deployment user and the nd uh the front end presentation layer web app deployment and then we are also using bunch of services our web application is printed by application load balancer so which you are seeing here and other business logic micro services are printed by cluster ip endpoint and in addition to that let me um there is also we are using uh the vis we are using a free entity which is collecting logs from all our kubernetes parts and sending the logs to elastic search engine and this is our kibana dashboard we are using it for the visibility as you can see pretty much all the locks are being uploaded here so you can able to debug or build a dashboard in kibana for efficient troubleshooting you know in addition to that you can even uh do the integrations within a page of duty let's say suddenly there is um the traffic in your website goes up significantly like your tenfold 100 fold so you want to get notified so you can able to implement the integrations into that and uh so i just wanted to show uh the the database uh layer side so this is how um so the audio postgresql is for enter printed by rds proxy this is for high availability and connection pooling and multiplexing we talked about and similarly we have amazon rds postcards engine so that's for data persistence layer so this is um which is using dbm file launch and and the engine version we are using is 11 13 with the multi multi-easy sector pro and the storage layer we are using gp2 and as for memory db we are using um so this is our amazon memory db cluster so we are using a number of shards three and and we are using the node types of db r6g large and that's the cluster endpoint and engine version is 6.2 and and if you notice in both the cases we have enabled data minor version upgrade 2 to true so that means aws is going to manage automatic minor versions upgrade for you uh during the scheduling maintenance window and finally how what is the website we built so let me show you that screen so this is our e-commerce website this is how it looks like you remember we talked about the user whenever you register the user so automatically it should communicate with the user's microservice and should be able to and that in turn uses memory db for data persistency layer so let's start with that one right so i'm going to create a new register user so i'm going to show you some test 12 some arbitrary number so i mean i haven't put any validation here you know just to ah show the whole workflow so now that we registered test values so you can able to log in with the testosterone so once you login sorry about that so let me try the test level just a little yeah here we go so it looks like i did not register the user properly for test 12 so but i did uh create the test level register the tesla and use it just before the demo so once you log in the test level so basically the users see this part of user uh section it's interacting with user micro service which in turn interacting with the memory db for scalability and similarly we have the cart where you can able to add any products you want and so that should be communicating with um cart micro service so this is the one right you know this is this part goes into cart micro service so and this cart what this should make it is you know our micro services stateless what i mean by that is for any reason let's say i put some items in the cart and and our micro services dies for some reason and uh the deployment manifest automatically gonna restart these micro services in a new node or new uh other available uh worker nodes so when it restarts it still leads to the current state still needs to be maintained and obviously it does so if i open uh a incognito window and let me show you that but i can delete the parts you know what let's do that let me delete the parts and then we'll i'll show you how the state is maintained uh within within cart microservice using memory tv so what i am doing here is i am deleting the pods uh for the i'm deleting okay ctl delete pod and what we wanted to delete is in oak tank the label is service equals to cart so that's the service we wanted to delete both the parts so when you run that you can see here both the parts are deleted so what it's going to do is it's going to restart new parts automatically remember we talked about it the way deployment manifest works is underneath it's going to use the controller loop which is going to check the declarative state and check with the current state and if the current state in this case you know the parts are deleted it's going to automatically reconcile so that means it's going to uh restart the parts automatically so if you can see here so what i'm doing is i'm extracting the all the parts in oak tank namespace tangling space with the label service equals to cart you can see both both the parts are restarted so when i restart and if i go here and if i come back my card state should be maintained as you can see essentially we made the services our micro services stateless in this case right and the second demo i wanted to highlight is by inducing bunch of requests by scaling out the number of requests coming to our website and see how the parts are automatically uh able to scale so what i'm going to do in this case is i'm using a utility called arpeggio apache benchmark utility a b and and with that what i'm gonna do is i'm gonna use the number of clients 50 which are going to process 100 requests and up to 300 seconds and it's gonna file now it's going to execute this particular url so which is going after our website and looking into fashion category in the product section and once we run it so this is gonna run a huge load on our website so while it is running so what will to our kubernetes and we'll watch out uh for web.plan part how the cpu is gonna reciprocalization is gonna show up on our web deployment pods and you can see here so currently it's at eight percent cpu utilization and the target i said to 50 so what i mean by that is whenever um whenever the cpu utilization goes beyond 50 it should start automatically scaling our parts which is the horizontal part out of scaling we talked about so it's currently at the 41 percent now you can see it went up to 56 percent so now you should start seeing the number of parts are automatically scaling out for us so with the second attempt whenever it says above 50 percent so slowly it is going to increase by one part at a time so now you can see the number of parts went from two to three replicas here and similarly you can get it from here also okay pod remember we talked about minus n work tank and label service equals to webapp and you can see currently there are three parts running right so we only deployed actually two so now the cpu utilization is 15 so as i said right so when the load subsides it automatically scales down uh the number of parts back to the initial the desired uh set which is two in that case and i just wanted to show you how the horizontal part auto scaler uh setup is set up in our uh oak tank retail website so what you're looking at is this is the hpa we have set up and you can see pretty much you know all the hpas are using for the card order product user and web micro services which are set it targets 50 cp utilization when it goes beyond that obviously it's going to start scaling it so that's the point i want to prove and similar so this is the front-end layer right we talked about now let's talk about um the rest api layer you know basically the business logically here so same thing all right you know and what i'm what i wrote was a simple code which is uh in a python so all it does is uh so let me show so in this case all it does is it's going to spin off uh 10 worker trades and calling from order microprocessors for example let's say you're having a huge sales event and and the number of orders going up significantly for example so how our order management platform looks like how it's going to scale right how our database is going to keep up so basically that's the idea in this case what i did was a simple python program it's going to open a rest api thread to our internal microservice which is uh um order and with the ten threads so i'm gonna use seven different terminals so each terminal is gonna execute ten threads each so it's like you know so almost we are simulating of uh 700 concurrent connections to uh threads to our micro service and show at a minimum scale to show you know how the pod is going to automatically scale for order micro service so let me run this so while it is running so i'm going to do pretty much all seven videos and kick off the same so now we started running order work tank order test so i used all seven so now what i'm going to do is i'm going to use so what we are using is you know the work of 10k started which is going to spin off our threads and each thread again you know is going to use 50 uh concurrent threads within each port uh that makes connections to the database uh also microservice called order or deployment so so we are going to use seven so we are gonna uh almost you know seven times the workplace so you can get you know when it comes to the workload i'll show you that the number of requests on order surveys how it looks like from the visibility in kibana right so whenever we are kicking up that limit so our order management service should be able to automatically scale so this is to tackle you know let's say you're doing a major image and releasing a new product and you're expecting uh user cloud for example and suddenly we are going to look at you know significant spiking order placements or ordering movements so our service should be able to dynamically scale and the database should be able to handle without any outrage so this demo is going to pretty much you know talk about that so in this case i used almost seven terminals uh to execute the workload on order management service so while it is in progress as you can see so the pods when it started it was only using two percent safe utilization and had minimum number of ports too and the replicas of actual actually so the minute when the cpu utilization went to 73 percent consistently so immediately uh horizontal powder autoscaler spun off one additional part and the overall cpu utilization will drop to 50 so which is you know little below the threshold research so it is operating so tell me that you know if you see more connections coming in it's going to scale automatically up to 10 hours be different in our manifest and the next one i wanted to talk about is um so we talked about the stateless card management so the next one i wanted to talk about is usually that's how it looks like from the memory dvd you remember we created a user um cancel this and when you create so basically kubernetes exit so what i'm saying is you know basically i would like to connect to one of the pod in car deployment in namespace work tank and i would like to execute a call command so remember we talked about you know for all uh business logic micro services we are using cluster ip endpoint not a load balancer so which is the reason i need to connect to one of the parts to be able to query the cluster ipm point so what i'm doing is i'm using restrip and giving stp a call to a user micro service and and i'm asking you to give me the user with the email test attendant or that currency so it's going to give you the information what you need right and so this is the rest api account and similarly i wanted to do one final one with the database failover see what happens when the database fails forward so that's the one we wanted to test out so it should be transparent okay um uh what i'm going to do is i'm going to run a one of the workload test so what this is going to do is okay so what we are doing is we are executing again we are connecting to obtain test deployment uh deployment to one of the part in main space of tank and executing a rest api call to uh product test using five threads and 600 uh for 600 seconds of duration up to 10 minutes so this is the simple container we built for for testing our product to see how effectively we would scale part of the cacd integration and so while it does that while executing uh the workload what we're gonna do is we are gonna fail over the database right so now the workload started so what i'm going to do is not just one place let's execute the workload from multiple uh windows so that you can see i'm using almost three windows to execute the similar workload so it's going to use five threads from each it's almost 15 threads and it's switched for 10 minutes so while it is doing so let's go back and do the rds database failover so what i'm doing in this case is i'm going to use aws rds api to cli to reboot the instance with the failover of our rpg rds post sql instance so this is going to do the database failover so once it fails over so what we can do is uh you can look at the log files from the pods so with the cube city unlocks deployment so what we are doing is so while we are executing our threads against product microservice you can see how the requests look like you can see right you know all the requests are coming the majority of them with the status quo 200. so the idea we wanted to is whenever the database is failing over what happens typically with the audience proxy i talked about so with the proxy during the database transition road transition it's going to retain the existing connections and and automatically it will re-establish the connections as soon as the failover is done so for you so that means application doesn't have to worry about uh reconnecting to the database whenever there is a failover exchange so let's look at if our failover is completed successfully so i'm going to use rds a rdcli with the discrete events that will show us uh whether the failover request processed or not as you can see here the instance failover is completed for multi-ac and user requested for the failover of the db instance right so the failover is completed so let's look at kibana's to see if there are any failures during the film you remember we talked about in the kiwana dashboard we are sending all the logs so i have wrote a simple query uh let me go back to the discover and in the discover i'm gonna open so one of the query roach for product so if you look at the product request what it is doing is it is filling down by the namespace of tank and with the labels product so that means i'm only looking at the lots from product parts and then the log process response status exists so that means if there is any status that is not 200 so which is failed i am interested in it and the status is not 200 so as you can see there is absolutely none so that means no single request failed so what what i mean by that is if you look at the contents right if you look at the contents you can pretty much see pretty much everything is going right almost we got uh we are getting like 60 or hundred and thirty two hits so two thousand dollars for getting the rate of two thousand requests per second um that's the rate we are getting and similarly you can also build remember we talked about building the dashboards so this is the dashboard that talks about um whenever we ran that orders microservice using a bunch of applications you can see we reached almost 120 000 orders per second rate and our service was able to scale without any uh problem simply if you open let me open another one i'm gonna go to visualization layer and i created one more dashboard for our products also so you can see here our work tank product request by time so even in the products you can see it it went up to almost 80 000 requests or 8 000 requests per second and we filled out the database in the meantime right so our data so you can see our micro service can be able to uh seamlessly hand in the database available without any errors from application point of view and so pretty much come brings down to uh the conclusion to all our demos so in this one right you know the takeout from here is um the micro services whenever you are designing so keep the principles in mind what we discussed you know about the scalability availability and automatic phenomena and and each micro service should have unique responsibility so that you can able to utilize purpose-based databases for the corresponding microsoft and part of the demo we demonstrated the now the scalability of we looked into the scalability of workloads and we demonstrated horizontal or auto scaling for our web app