Turning Digital Performance into Competitive Advantage

Key Takeaways

[Music] good afternoon good morning or if you're from far away good evening thank you for joining us today for this webinar this webinar is brought to you by ssta and we're happy to have Lowe's Office Depot and intellic here with us today our webinar topic today is turning digital performance into competitive advantage and again it's sponsored by ssta we're here to today to bring you a lot of really great information about how you can improve your digital properties performance today you're going to learn some key Concepts behind what we call digital Performance Management and why it's so important to your organization you'll learn why it's really critical to your success as you move forward especially as we continue to be a more and more hyperconnected World you'll also learn about the benefits you'll see by adopting digital performance for your business and to discover DPM related best practices that you can leverage to really boost things when it comes to digital uh the digital Assets in your organization we're very happy today to be joined by Jason Bloomberg president of intell and a forms contributor Jason thank you for being here with us today yes it's my pleasure we also have Trey kler joining us he's the director of e-commerce it at Lowe's we'll be introducing Trey um a little later on in this broadcast and same with Mr Eric uh Leon he is the performance engineering team lead for Office Depot I'm Scott low I'm the partner at actual techch media and I will be your moderator for today's discussion before we continue on I want to make sure you're aware that in the good webinar control panel you're going to see one panel there called questions if you have any questions you'd like to ask of any of today's presenters throughout today's webinar please ask it in the questions control panel at the end of today's event We'll be asking our Audi sorry our presenters any of the questions that you've decided to ask in the questions control panel make very liberal use of that control panel it's a lot more interesting for the presenters and it's uh to be able to answer some questions and it's very useful for the audience to be able to see some of the challenges that some of their peers might be facing in the realm of digital Performance Management and without further Ado I'm going to turn it over to Jason he's going to talk to you in general terms about digital Performance Management Jason this floor is yours well thank you very much Scott see if I can get this slide to advance there we are o little too far okay okay so uh what is digital performance really about well it's it's really about money right that's really what business boils down to and the the relationship between time and money is really what performance is is all about so uh start with a couple examples Walmart right largest retailer on the planet uh had challenges with their website uh where where uh many seconds to load for for some of the users they realized the problem was too many page elements uh too many thirdparty scripts too many different hosts serving up these various uh elements uh so uh by getting a handle on their web performance they were able to uh improve their performance and realized that every one second of load time equals a 2% conversion rate increase and with for a company the size of Walmart that's that represents quite a bit of money another great example the Obama campaign uh right they they uh you know were were tasked with raising massive quantities of money uh and they realized that they could improve performance by serving Pages via uh a Content delivery network uh but they also realized that they could uh by improving performance they could uh increase their donation conversion rate by 49% raising over a billion dollars as a result so um so what what is what is digital Performance Management well let's let's start off that word digital so if somebody had asked me what word to use there I wouldn't have picked the word digital because of course the word digital is been with us for 70 years now right it means computers that use ones and zeros those are digital computers but we have this uh new uh sort of we've brushed off the old term and given it some new life and new meaning and this is really how intell seeds the word Digital customer preferences and behavior Drive Enterprise technology decisions so digital begins with a customer and the reason why we are talking about technology at all is because customers are driving organizations to improve and change the way they Implement technology to meet customer preferences and behavior so when we talk about digital performance what we mean is that the endtoend behavior of customer facing systems must meet these customer demands for Speed and functionality and the key term here is end to end right one of the key challenges with digital you don't want to fall into the Trap of thinking it is solely about the interface yes the interface the touch points whether it's mobile or web are critically important but everything has to work from from the front to the back in order to provide that that digital experience for the customer so then what is digital Performance Management well it's Management in a holistic and continuous way so by holistic we mean we talking about the big picture of it performance on the technology side customer experience in terms of what the end user is actually experiencing and the business outcomes right as we saw with the Walmart and Obama Campaign examples business outcomes translate customer experience into revenues or other business uh performance metrics in the context of e-commerce it's about conversions on that uh e-commerce website so uh on this slide we're basically talking about how uh digital Performance Management is a natural evolution of application Performance Management APM has been around for a few decades now focusing on application Level performance issues as opposed to network and underlying you know INF infrastructure issues and now we're taking that to the next level with web and mobile uh and all the various touch points and connecting application Performance Management to the actual enduser experience and that's the role that web and mobile testing play so uh actually connecting the dots between the end user and the underlying applications the third piece is business TR uh transaction monitoring which gets back to that uh the key uh factor in digital performance which is the the business uh metrics whether it's uh e-commerce conversions or whatever the uh key business uh goal is you want to be able to monitor that the transactions that you are uh putting into place uh achieve those um uh achieves those business goals so um so from the customer perspective uh we have a number of different things going on and this is a critically important part of the story right on the one hand uh users are interacting ING with some sort of touch point but there's really more to the digital story than just that point of interaction so one key element of this is the customer Journey right that that that it's not just an interaction when the customer comes to a website but it's all the interactions between a customer and a company over time so the customer may you know follow an ad to come to the website the first time and they may come back to the website or go to a mobile app the next time the third time may be the time they actually purchase something then they receive something and maybe receive a package in the in the in the mail that's a touch point they may have customer service issues they may come into the store either to buy something else or to return return something or uh for for some sort of support or they may call a call center and then there there's an ongoing upsell and resell as well and all of these are part of the customer Journey so you don't want to assume that that one interaction right they're coming to the website and buying something is the total sum of of your interactions with the customer so all of those interactions that are part of the customer experience so yes the web performance the mobile performance is critically important when that is how you're reaching out to your customers but call centers are also important your in-person uh interaction is important uh and that's one of the key challenges with the two U uh retailers we have on the call right Lowe's and Office Depot yes they have also well known for their uh retail brick and mortar efforts and those have to be properly coordinated so uh all of that is part of the story so what is the result well you want your customers to be happy of course we can all just take off our business hat and put on our consumer hat and we realize when we interact with a big company whether it's a large retailer or any other big company uh we want a delightful experience we want to have good products we want to have good service if there's a problem we want it taken care of we want the big company to take to understand us and understand our needs and all of this is part of that uh coherently customer Centric Journey it is so difficult for big companies to achieve but the critically important part of this digital story so um one of the key parts of the story now as well is the explosion of touch points so we're all pretty much familiar with the fact that people have laptops and desktop computers and now they've added smartphones and now we have tablets and various formats of tablets and tablets and so there's big and small and there's Android and iOS and there's uh you know native apps and mobile web but even that is not the whole story right we also have for example uh interactions when people are watching television so there's the set top box and that's becoming increasingly sophisticated in supporting e-commerce and other transactions uh we we have gaming uh platforms and those have you know in in in-game purchasing and that can be a part of the story and then when a customer comes into a a retail establishment they are interacting with salespeople who may have Point of Sales Systems a tradition sense or they may be carrying around mobile devices that act as uh customer touch points or the customer may bring in a a mobile phone into the store and maybe interacting say with uh barcodes or shelf tags with their phone that's becoming increasingly part of this mobile story so or the digital story so uh this is uh and this keeps going right whether it's digital signage or uh retail beacons and Retail that know when a customer walks by uh we've only just started this this journey journey toward uh explosion of customer touch points and all of this is part of the uh digital story so where does digital Performance Management fit into this broader scheme of things well this this um uh slide comes from uh intell's new poster the agile digital transformation roadmap poster which you can get at agile digital transformation.com what you see in the middle there is digital performance uh digital business Performance Management emphasizing that it's it's about that this interaction uh within the broader uh context right it's it supports devops efforts at uh in in the software development organizations it's big data related you notice the lower right machine learning feeds into this that's a critically important part of the story uh governance from the upper left is a part part of the story uh and it supports real-time Behavior Uh in in the context of you know real-time interactions with customers so there's this very broad uh interrelated tangled web of digital Technologies and digital business priorities that are all part of the digital Performance Management story and digital Performance Management as well supports this broader story right you really can't have all the other pieces working unless you support the performance goals of the organization so this is all part of the story so digital Performance Management then on the one hand we have uh customer experience what is the actual experience from that customer perspective when they're interacting with with the website with the mobile device or with any other touch Point uh for the organization it performance critically important part of the story that is end to end and covers many different participants uh because so many uh Enterprise websites have multiple uh tags and ads and widgets in them they're all served up by Third parties and then it's running over a Content delivery Network which is another third party and some of the elements maybe in a cloud which may be uh still more third parties all of this has to be performing and that's a part of the it performance story and then the point of the triangle business outcome all of this technology is for a purpose and that is to achieve the goals of the business you know to to drive drive purchasing decisions to drive revenues or uh better customer support happier customers it all ties down to the to the bottom line priorities organization and all of this is is a central part of the digital performance story so what are you actually measuring when you're doing digital Performance Management they're obviously the business metrics in the e-commerce context we have conversion rates card abandonment rates order value size uh but keep in mind the world is not just about e-commerce right this applies to other organizations that have different business priorities so it could be uh you know patient satisfaction in health care or it could be uh uh manufacturing metrics in the manufacturing organization in terms of uh you know reducing defects and and uh maintaining the schedule of the manufacturing uh assembly line so it could be any number of different business metrics we have customer experience metrics uh which are you focus on pages per visit visit paths click-through rates and other metrics that are important to understand just how good the customer experience actually is and then from the it perspective we have page load time site availability and other techical metrics including third-party compliance and that's a key part of the story since there are so many participants in this digital story it's very rarely just one company serving pages to its customers there are many different parties and managing service level agreements with those third parties is a key part of the digital story as well so the goal then is continuous real-time visibility and control that balances measurement where you can't manage what you can optimization over time you want to continue to improve your performance you shaving just fractions of seconds off of response time ties straight to the bottom line so optimization is a critical part of the story and then testing right you can't get the uh results you want unless you have the appropriate testing and I know uh We've we've already heard heard some of that you Trey already talked a bit about testing as well and how important that is for the digital story excellent so hello everyone my name is Trey kler I'm a director here a Solutions director here at Lowe's responsible for low.com m. l.com and our apis that drive our digital properties just to give you a little more about low get this slide move forward there we go so Low's is a fortune 7 for low is a Fortune 500 company we're currently ranked number 47 headquartered in morille North Carolina which is just outside of Charlotte North Carolina uh 2015 company sales totaled just short of 60 billion at 59.1 uh with a recent acquisition of Rona which is a large uh chain of home improvement stores in Canada we're now at more than 200 or 2355 stores in US Canada and Mexico we serve more than 17 million customers each and every week and the digital space and e-commerce are a a very top priority for this company which we'll talk about a little bit uh here in another slide so this uh this article or this uh this slide highlights an interview with our CEO Robert niblock in this interview he outlines Omni Channel retail as our company's core strategy uh digital is at the heart of that focus and this creates a greater need to understand our customer our customer interactions and drive performance of our digital properties so to drive the digital experience we are working to build a Lowe's digital performance center of excellence this team is helping us to drive excellent Omni Channel customer experience driven by real user monitoring so we're using real user interactions to drive experience on the website we're leveraging actional analytics to increase revenue and Agility so realtime forensic an analytics on what first and third party resources and Predictive Analytics to forecast what our customers need and want to see from from our digital experiences we're working to achieve higher quality higher quality testing and reduced risk through actual customer interactions so we're leveraging data that we collect around how our customers use our website to to drive what we test for before we move changes to production and we're focused on a single source of Truth soan real time visual correlated insights across business devops and monitoring that help analyze our our performance and our and our customer interactions and that that results in a a war room at all times to to address any issues and and any any any customer impacts that may show themselves so this next slide is a uh a SL a picture of one of our operations rooms uh this is one of the tools we use to monitor our platform and collect user interactions you can see a a quote here from uh my VP of digital uh our data changes every day that makes data science the key so he's talking about is is really how important knowing our customers and knowing our data and knowing the performance of our assets is to lows so data not only helps us find technical problems but also identify conversion impact when those problems arise real-time data is key to knowing our customers needs and understanding reactions to to the changes we make and how how those impact our customers so our approach to uh DPM so Lowe's really focuses on four areas metrics people process and technology in metrics we look for real user engagement and consistently monitor the performance of our assets on the people front we take our business objectives and align our it capacity to those to accomplishing those objectives in the process space we're continually testing continuously testing and measuring what we do and performance is always a requirement we don't go into an effort without considering performance considering the impact it may have on our site comp considering capacity additions to to our properties uh performances is basically a requirement of what we do and is required to be measured and analyzed before we move a change to production and in the technology space we leverage indn we leverage s's indn DPM platform and we aut wherever possible leveraging open source such as Jenkins um to to help Drive efficiencies across our development practices so we've seen some very good benefits out of this approach uh first and foremost is availability uh obviously that's that's key to to Our Success uh High availability is driven by executing the right tests at the right time every minute of downtime or unavail unavailability of our our properties impacts not only the bottom line but also our brand and we focus on understanding the correlation between availability response time and its impact on sales so that not only are we working on availability but we're also working on reducing response times of our website to tie back to the the slide that uh that Jason was talking about um huge focus on realtime results we're leveraging so's impulse tool set to to help Drive real-time data but this helps us know how our site is performing at all time as well as what's important to our customers and we use this data to make smart decisions we target a strong customer focus to help drive those decisions and in addition making performance part of our culture is key uh we can't performance cannot be an afterthought it can't come in at the end of your project so we we focus on better planning strong focus and and this timly least to reduce cost when dealing with performance issues so out of that the one thing to take away is really make performance part of your culture uh performance has to be first and foremost on on every thought on every project on every effort performance can't be an afterthought testing is very expensive test the right thing the first time so know your data know your customers know what what interactions they're going to have with your properties uh actual customer interaction should drive those decisions so so being able to quantify what is happening at any given time and predict what what may change with your change is key to to maintaining that performance every day is a Black Friday so treat every day as if it's your as if it's your Peak day so always test your Peak uh performance metrics and always know that you can maintain those those those stats those that performance and always make sure you have visibility into marketing promotions and campaigns as those can drive flash sales uh large events that could drive performance problems across your properties and with that I think I hand it over to Eric and uh thanks again thanks again for having me uh here today uh my name is Eric Leon I am the lead performance engineer uh for Office Depot North America and worldwide uh really an honor to to have these minutes with you today uh let me go to the first slide okay so we're 100 uh we're the hundred 94th in Fortune 500 we headquartered in bokeh Florida uh approximately 14 billion in annual sales we operate out of 1,800 retail stores in 59 countries award-winning e-commerce sites uh we employ about 49,000 plus Associates and we also operate under the office uh Office Max Grand and toy and Viking Brands all right so Office Depot's digital transformation so you guys can see the slide so let let me just tell you a little bit about our story um so back in our day our performance testing team was uh embedded deep in the QA organization our te our testing was basically just push button uh really the analog age for us uh we would schedule low test invite different parties to monitor their perspective areas uh we report had any issues but our main focus really was the frontend response times um the testing team didn't have much visibility so we relied heavily on other teams to tell us what was wrong uh for us this was costly uh not a efficient uh really because uh we didn't have access to their tools also uh in those days the testing tools that that we did have uh we could only really test from one geographic location uh Redwood City California and and really at that and really what all that was doing was was uh allowing us to crush our our CDN in that one area not really a load test per se or a performance test uh at that time we ran a PC with ssta and Cloud test which was already using the cloud to deploy uh node test generators so this was awesome for us uh not only did we not have uh uh to finance additional hardware for testing servers and um an infrastructure but we no longer had a limitation on an amount of users we were able to inject to our sites uh really opening the door for for True Performance testing uh that single that single item of user count opened up the opportunity for really simulate our inst um simulate for instance uh Black Friday testing or peak season preparation testing uh which we weren't really able to do in the past with the kind of volume that we had on our on our site it it was really cost prohibitive uh with that previous Tool uh today we're part of the e-commerce operations team uh with access to realtime metrics to front-end server backend server database server and all the other support systems to our website uh this is streamlined our testing cycle uh we're now the focal point for complete roundtrip response times so we tell them um we tell the other organizations what uh you know where they need to look for for performance degradations if there's any to make recommendations of how to solve those um our our performance testing team has really evolved in that in that respect uh now we're uh we're also moving to an automated low test creation uh by the use of the Sunburst reports that will model and create better real user test cases uh this will be molded right from our real user traffic uh and this moves us into the digital age so Carl uh our SVP our senior vice president uh had a quote here uh today's challenge we have ton of data and we don't know what it's telling us uh so to expand on Carl's quote uh we actually we do have a lot of data from realtime user metrics uh synthetic APM tools core metrics several others but until lately uh we really didn't have a central analysis tool that could really help us correlate all that data uh data science workbench has filled that hole for us uh really now we're able to take all that data and see how performance is working in real time um how it's affecting conversion how our campaigns are performing are there on the right landing pages what are those landing pages are those landing pages optimized um and what are those landing pages telling us U so it's all about taking the data and correlating it all to make better decisions to truly uh positively impact our bottom line so real-time campaign tracking um so one of the features we found very unique and relevant when talking about campaign management was understanding the difference between machine learning and baselining uh everyone's familiar with Baseline alerting it's basically used to monitor CPU threads um those kind of items but with machine learning uh what we have not been able to do is use our own historical data to build Mo uh to build a model that that we can uh action of so if a campaign is performing out of the norm we can reactively amend to inspire conversion so basically if it's tanking we can uh you know uh send out a you know an addition to that uh to that uh campaign so in order to drive up that that conversion uh at the same time if a campaign is doing too good uh we want to make sure that we don't have a $99 printer offered at 99 Cents uh and we alert on any of that we we want to alert on all of that because uh we we want to stay you know we want to perform better but we don't want to we don't want we want to make sure we catch those outliers and on this slide um how of is Dea leverage DPM I think on this Slide the most important takeaway that I can share with you uh is to have alignment between the different e-commerce related teams in your organization uh for example the business team uses tracking pixels and core metrics tags on our web pages um but do they really understand uh the cost of those resources on our performance do we know we know that performance is is directly related to conversion so while yes we're able to offer more personalized experience in customers uh with the use of tags and pixels are those same resources costing us on conversion meaning you know are they are the effect they're trying to have negatively impacted because they're they're weighing down the page speed um again data science workbench uh has helped us here along with impulse is able to really report daily on those on how those are performing and their impact to conversion and any breach in that SLA can be discussed with those proper parties or even third parties benefits gained and expected um so here when uh when Office Depot talks about third party performance uh we have to hold them accountable uh to the slas that are expected on our page loads uh some third party resources are synchronous which means they will wait for the previous resource to load uh While others are asynchronous and load s simultaneously with uh with others well we're take uh we're currently taking that data uh science and uh to really help us understand the impact of uh that impact of customers uh and response times we really believe that the asynchronous resource should also load within the page SLA uh I know that most will tell you uh you know not it does not affect customer um experience but uh but we're doing is we're going to we want to take that and um and see how it's see how it's really affecting the overall conversion and the one thing there's curlyy again um correlate web performance with Revenue align with business focus on revenue and get realtime insights and and the one big thing that we had to understand is not only about page performance it's all about Revenue performance um with monitoring realtime campaigns and our do and setting up the correlating alerts we expect to really make a splash on our financial performance this year and with that I turn it back to you thank you very much Eric for that and I also love curly so that was really great to see him uh actually make an appearance here in your deck and for the audience we want to remind you that the quest the Q&A control panel is available for you to um ask questions uh throughout this presentation and we're going to Tak in some questions right now now so a number of questions here around the audio issue we had earlier um so we're going to ignore those but for the panels panelists do you measure per perceived performance as well as actual performance and if you're on mute make sure you come on mute yeah so this is try yes we we measure both um use uh perceived as a as a higher metric in terms of how the the site is currently doing um for the people who may not be you know familiar with what that actually means can you kind of give a brief overview of the difference between perceived and actual performance and how that may manifest itself and so perceived performance for us is when the page is usable by the customer and the what is in front of you is rendered so there may be activities going on Lower in the page out of sight or there may be backend calls that are that are happening to continue rendering aspects of the of the site excellent what other now remember folks there's a we have some experts here they can talk to you about uh digital Performance Management overall um questions regarding uh website performance and things like that so please make sure you make liberal use of theel uh we have another question here what's different that ssta is bringing that you didn't have before um that was creating some challenges you know maybe Trey maybe you can um kind of weigh in on that a little bit sure uh so first it's it's really the real-time view um the data this collected by uh by impulse is is not it's not a compressed view of the data it is every interaction that that is collected off your your digital asset so that that's huge if you look at some of the other tools out there they compress on a time scale so every day every week every month there's more compression so the further back you look uh the less accurate your data is so that was huge um te tying that data back to the test cases that are actually running your performance testing has been huge um and really s to helped us uh not necessarily launch our performance practice but but really mature our performance practice uh several years ago when we when we adopted the tool set and uh Eric what's your answer to that question what's supposed to bring to the table that was really lacking before I I think for from our perspective and and we learned this time and time again pretty much on on a weekly basis that is that they continue to to move forward and and search ahead with with technology um anytime we bring any any new requirements to to SOA they're always eager to to find that solution and really deliver on that product uh I believe with u well I don't believe I experienced it with with previous uh with previous vendors uh we don't we didn't get that it was pretty much here's the tool and this is what it's this is what you have uh we really feel that that s has really become a a great partner of ours in that respect uh to continue to move forward with our technology um to better you know to better increase performance another question here um how are you testing mobile app performance uh for your various properties and let's send it over to to Eric this time um as far as as far as uh ask me that question one more time just to make sure I had it right sorry sure how are um how are you testing mobile app performance um mobile app performance we're we're testing it at same as we would our web our our our full render uh version of our website U you know we calculate the number of of of sessions that we create in a month or a week or a day and and simulate those load tests accordingly and uh what about Trey can what are your thoughts on that and so for measuring performance uh we're measuring during the API calls that come out of the application um and then we have some other some other Tech built into the app that that helps us monitor uh but specifically for testing the apps we use Touch test which is another Socia tool excellent um we have another one here and this is a really important question because we all want to see um whenever we do anything new in technology the first question that gets asked is what's it going to cost and what's going to be the return so the question and I'll start with you Trey is have you been able to quantify the Improvement and actual Revenue gained based on the implementation of um digital Performance Management tools from SOA so we do have some metrics around that but if my PR guys were sitting here they'd yell at me for answering that question so I'm going to have to defer on that one okay so the answer but the answer from a yes no perspective is yes you just can't provide the numbers yes okay what about um Eric yeah uh we can definitely see the benefits of um uh we can definitely that as a matter of fact uh we're able to really tell uh what a half second performance uh gain does for our page uh in conversion uh we're really able to um to set that standard actually and and really without giving a number but yes excent so um Eric I'm gonna come back to you um for just a second here from a mobile app performance perspective there's a clarification question which are you referring to the viice site or to the backend site he may not I'm not sure if he's asking that question appropriately I'm not if if he means am I testing the native app uh I'm testing the the m. off depot.com not the application got it um another question is uh for Jason how has the e-commerce industry really adopted this idea of digital performance what are they what I mean how are they how are they reacting to this and what are they doing well I think um uh you know the two uh case studies at the beginning really highlight what's going on that anybody who is selling things online is realizing how critically important even fractions of a second are to the conversion numbers so for the larger retailers the the money can be dramatic so it's really now becoming sort of a race to the fastest right who can build the fastest best experience for customers realizing that it's a competitive landscape and uh and you always have to think about what the competition is doing and you want to do it a little bit better and for many retailers the competition you know number one on the list is going to be Amazon so it's sort of everybody versus Amazon uh regardless of the category right because Amazon just sells in so many categories that uh retailers are looking at what Amazon is doing how fast Amazon is doing and then and making sure that they can be at least in the same ballpark if not faster and More performant in other ways you know better experience overall comparing themselves to Amazon so it'd be sort of interesting I don't know if if the two folks on on this call if they think of that am it's like them against Amazon or is it more of a sort of multiple competitor situation that's sort of an interesting question um on the screen for the audience you're going to see a paper that Jason has written um it's available for download the link you see on your screen um it is a free download so if you're interested in learning about how to steer your Enterprise right when it comes to digital Performance Management um please make sure you download that paper um also we're going to be putting some more links up on the screen here while we continue Q&A um I would highly recommend you take some screenshots here um so you can uh refer to these links later on um I'll leave this one up here for a few more seconds and I'm G to ask you Trey what before ssta what were you doing um with regard to to all this stuff I mean did you had you cob together tools I mean you don't have to name names but what was the what were some of the challenges that were leading you to to look for something I mean you described them a little bit but in a little bit deeper way like spec what are some specific things you were you were experiencing um so we actually had uh two providers before sta uh that we were using and uh really what Drew us to sosta was uh the competitive price of of the model and the distributed uh the distributed view of the traffic so the fact that you were more geographically dispersed uh with the the size and traffic that we throw our site was was very appealing and and that was ultimately why we made a switch Eric was that similar for you or were you facing other challenges oh I think I think he hit it right on the head really it was the distribution and uh the cost it was much more cost effective with an unlimited number of of users that we could actually transact on the website it gave us a stronger visibility and and and really when you really push your servers uh on in any direction um you really come out with a more clear picture of what is the weakest link and that's what we needed to do and and that's what we were able to do excellent so supp to helped you sort of understand um what that weakest link was in your on your various properties exactly excellent um we have another question from the audience um this is around wanting to be able to try and see what s can do for them so the question is can I get started with some kind of a test um my understanding is that there's a product they had called Cloud test on demand Jenny can you um unmute for just a second and sort of mention that to the answer that question for the audience yeah you bet so if you were to come to our website today uh check out our product section we do have a product called Cloud test on demand and this allows you to actually use our product for a one-time project and test it out and pressure test for your particular environment and customize it for your needs thank you very much and if you have other questions um this is your last chance to ask them for the few experts we have on the line right now um so we encourage you to do that and while we're waiting for that I'm going to bring up the next uh page here this is the one with lots of other resources that you can leverage to uh figure out ways you can improve your own digital Performance Management your organization um SOA does have a comprehensive blog that talks about performance they also have some future webinars and past recorded webinars um that if you have an interest in this topic you should definitely sign up for or watch if they're um previous webinars and they'll also do a free one-in-one website performance assessment so they'll B basically work with you and look at where things might be going wrong and where you might be leaving money on the table when it comes to website performance so make sure you take a screenshot of this screen um because it's pretty important stuff here uh particularly if you're responsible for this um kind of stuff in your organization we will include in the followup email to this webinar all of these links as well um so if you don't have screen Snipping Tool in front of you um you'll be able to just click the link um shortly and I believe that's all the questions that we have right now um we're going to do one last thing which is to give away a uh Asal Tech medas doing a $300 Amazon gift card for a lucky winner and today's lucky winner is Nick Noble uh we'll be reaching out to you after the event and you'll be able you'll be the uh we give you instructions on how to get your $30 gift card thank you to all of our presenters for being here today thank you ssta for sponsoring this webinar and thank you to our audience for participating everyone have a great day [Music]