Data-Driven Design [MozCon 2017] — Oli Gardner

Key Takeaways

[Music] foreign [Music] who are you what it's me who's me your inner voice this is not the time I'm trying to give a talk here that can wait ah this is marscon there's 1500 people here I'm actually trying to impress some people right here look at them half of them are still asleep maybe so but what do you want I just saw what you did to the home page what is that all about oh it's kind of amazing it's a we're going to be a complete brand refresh no no it's gonna be amazing it's gonna be about the day in the life of the marketer we're gonna have cinematic background video Parallax scrolling no no a giant thing at the top with all these promos coming back and forth it's going to be really special that sounds awful no no but we're also going to have an explainer video like it's going to be an epic explainer increase conversion to 20 percent guarantee I read a blog post okay but what if it doesn't win the test well well you know I want this on the homepage regardless we put a lot of time and money into this oh no you just became one of those marketers who doesn't care about performance shut up you shut up gosh it's like it's all day every day in here it just never stops talking how's the test going tests going losing by nine percent but that doesn't matter so it doesn't work we need to find a way to make it work data says it doesn't you know what [ __ ] data that's what today's talk is about not really I do like data I wasn't angry at data I wasn't even angry at the explainer video they can be great if they're done properly I was angry because the results didn't confirm our assumptions and our biases we all thought this was going to crush it how could it not how could it not win this this video on the home page but it lost by 20 and the team was crushed like everybody worked so hard on it and it was just an epic failure the problem was there wasn't a single piece of data anecdotal uh qualitative quantitative that suggested we should do this right I didn't give a single piece of data to the copywriter and the designer at the start of this project in fact in my research 67 of marketers say they don't have enough data to do their job properly and 51 said their analytics isn't even set up correctly that's terrible it's not good enough we didn't do it for the right reasons we did it because everyone else was doing it right it's just flooded with requests to do this kind of thing and that's not how we should be operating we have to make decisions based on insights and available information not just because it suggests that we should do it because this happens with every Trend there'll be 400 blog posts about why you should use an explainer video followed by 150 videos in the form of explainer videos explaining why you should use an explainer video it's too much nice no no you laugh and I promise I didn't do this on purpose this is one of our dogs and I thought I'd give them a mohawk but I used human Clippers which is not and it just got so messy I had to go all the way down the skin let's put sunscreen on him so he wouldn't fry in the sun just sorry if there's anyone from Peter here it didn't didn't mean it uh everything will be at this length there's a little landing page there that has all the slides everything I'm talking about I'll share that a couple of times throughout see what you mean Rand Okay so in my research over 700 people in marketing teams I I think marketing teams are dysfunctional I've seen it I've seen that frustration and I think I know how to fix it uh but you know so I interviewed marketers designers and copywriters primarily so we're gonna have a little look at how they how they feel about one another so we'll give them each three lives and then we'll hear what they have to say so marketer it's not a marker that's the design they have no understanding of customer Behavior okay he loses their life the designer's job is to make it look good like make it pretty he can you just make this pretty designers hate hearing that that's a life away from him writers don't understand the difference between people and search engines and they want to make the designer begin before giving them the content this is so common and it's a massive problem 81 of designers say they have to begin before they have any copies so no data no copy now they're just building kind of templates based on their imagination or copying someone or downloading a theme even worse that's a big problem marketers on my up and enjoy Naval gazing actually someone Google that I don't even know what enable gazing means I guess it's not good it's good for the mic volume but and they don't respect how design you know what design is and they think they know how to do it 98 of marketers like a surveyed have to give design feedback to designers and 87 of them think they're qualified to do that [ __ ] total [ __ ] I've been on both sides of this for 20 years that's way overconfident that's not actually true they don't understand and really trust or believe in what the other people are doing it's just a messy dysfunctional system and then you can't say that to a designer that's just the worst possible thing you could say right now quick hands up like put your hand if you're a designer wrong we're all designers designers are problem solvers we're all problem solvers that's the job of a designer it's the job of all of us this is my biggest hero because he can take a situation and fix it using anything available available to him we as marketers designers and copywriters need to start living our lives through the lens of an optimizer because when you do that when you find things that are broken or annoy you in your day-to-day if you imagine or actually try and fix them you'll become a better designer a better marketer a better business person as a result this is our bathroom the car will hate me for sharing this on a big screen but you can see there the the shower curtain if you're having a bath there she has a bath every day like it's just it's blocking the light it's annoying it's kind of you know easy fix bungee cord now it looks like an expensive hotel look how classy that is I mean I know it's a bungee cord but still you know it's all nicely wrapped it's beautiful this is what we should be doing all day every day it's my favorite pastime it's all about empathy uh understanding our visitors and customers obviously but also understanding one another on the team understanding what it means to be a designer or a writer and how they feel having empathy for how they would actually like to do their job it's not about just taking what you see other people doing and implementing it so there are a lot of design Trends out there that are all kind of coming together to cause this problem every year more and more of them come out some years way more than others and the problem is they typically don't go away they just build and build and I blame theme designers in part because they're not well they're sales people right they take every possible interaction model and design Trend and they put them all inside these themes so they're so bloated and slow and their usability nightmare enhance a conversion nightmare that's just they're trying to make sales and that's not fair on the people who buy these because they get them because they're non-technical and they don't know that it's messing with their performance so this slide this is from Google Trends the red line is Pinot Noir the blue line is a different kind of white I want to take you back to 2004. something profound happened Pinot you see was down the bottom and then changed Forever Until Now so uh I went to see a movie with a good friend of mine is a movie about wine and because we're adults and Incredibly childish we decided this is Jeremy our CMO and me we decided to sneak in a couple of balls of Pinot inside our jackets because that's what you do and so we're in the middle of the movie and we're taking turns it's my turn sound I'm like this trying to pour the wine nothing in the cup the cup and there's a wine tasting scene guy two rows in front of me goes holy crap this is amazing I can smell the running around the edge and it's going all over the floor it was red wine everywhere do not sabotage me then this scene happened Changed History forever do not sabotage me oh aye aye Captain you got it and if they want to drink Merlot we're drinking Merlot no if anybody orders Merlot I'm leaving I am not drinking any [ __ ] Merlot okay okay relax so Merlot is the blue line and that's when the movie came out I predicted it at the time and sure enough four years later a sideways effect was confirmed they're powerful I think this quote this came from a box of matches at one point but uh it only takes one tree to make a thousand matches one match to burn a thousand trees that's the kind of power that Trends have to influence us sometimes we're not even aware that they're doing it it's a true fact I love judging wine by its cover I'll pick one up like oh it looks amazing with Merlot and I just put it down again I can't help it okay so I've created this process data from design 3d I like alliteration I actually prefer data informed design but that doesn't have 3DS and it's a process to help marketing teams work better together understand one another better collaborate better and also deal with these influences that come at us from Trends and from requests from clients and that kind of thing so it's very simple the first thing you do is you consult the 3D Playbook which is a kind of lookup table so in this case I'm interested in working on it you know creating or optimizing the hero shot on my page so I should look at visual ID false bottom stock photos what this does this takes all the data that's out there which is overwhelming and it'll just say just pay attention to these few things just simplifying this is what it looks like it's a big Google sheet again it's at that link now what it does along the topper is all the data types and and sources and down the left are the objects that you're interested in and there's notes on all of these cells explaining what it is and why you should care about it for that particular object and there's a custom menu at the top when you click on that it has them all and when you click one of those it will isolate that in the table so you can just look at that alone speeds up the process then you collect the data ideally as a team this is a simple checklist you've seen which data you should collect from The Playbook you just put it in here so you can run through this have a note where you're getting it from you know what tool how much you need sample size that kind of information will be in the Playbook it just helps you get aligned with what you're supposed to be doing a third make observations this is the start of the really fun stuff and you need to do this as a team look at the data you've collected whether it's graphs charts spreadsheets heat Maps recordings of users and usability tests look at it together because that's when the empathy begins not just with the visitor but with the people in the room when they see and react to these things then you just jot them down print this out and do it by hand that's the best way it's a lot uh it's just a better process that way and you make your observations you put them in here and then you rate them by severity if it's a block or something it's stopping people from continuing that's an instant five otherwise just do that to help prioritize and then you go into the design phase now copying a bit from Google's Sprint model what's great to do here is have the team leave the room for like 10 minutes and go and sketch their own way of solving these problems how to MacGyver these Solutions when you come back you present in front of each other and you learn and you're it can all evolve collectively so you can either two ways of doing this you can either take each one and sketch how to solve that problem so maybe we need bigger around your pants maybe we need to put more toilet humor in our copy pandas are always good for conversions and we have to be unique so you can do it that way or you can combine all of it into one kind of idea that you would use to solve this problem depends how you think foreign right product idea someone grab a domain all right then the latest Trend I saw last year was something called conversational forms fascinating new thing came from an agency in Denmark in Copenhagen called space 10. they're really interested in human behavior and interaction now what it is it's a simple script that you put on your page and it will take a regular form and change it into a chat like this so it's like a seemingly more human style interaction so I wanted to validate this and they were super stoked because like we don't have the capacity or ability to up to like test this I'm like awesome I'll do it with your collaborate so first thing we do check the Playbook what should we be looking at well let's look at session recordings do some mobile QA usability tests scroll and click maps and Lead data they're the things that I want to focus on to understand and validate this new trend collecting the data going to use hot jar my phone Camtasia and unbounce for these things different things require different sample sizes scroll Maps one or two thousand a usability test maybe six people and so on okay so what did I observe from this now these are the six data points I want and these yellow bubbles this is basically the notes that appear in the Playbook so that's kind of what it's going to tell you in there to help you figure out what you should be doing so I put one on a form I launched it on a page launched it put hot gel on there to record the session this is five minutes after I launched lots of scrolling up and down then it starts to get a little wacky get a bit panicky all over the place then there's an error and you can't escape and it just gets more and more crazy and this goes on and on and then he refreshes the patient does it all over again I'm just like this is where the empathy comes from I'm like oh my God I'm ruining this guy's day this is awful so my observations lots of wild scrolling and there was a bug with the air handling so what do I do well first of all I freaked out then Nicole who's much smarter than me this is my fiance uh it was like but Bruce put his email address and you should reach out well brilliant so I sent him an email I'm so sorry Bruce I was testing something and I saw a recording it was terrible I apologize here's the content you were looking for again I'm really sorry two minutes later he replies in 25 years the best way of owning up to a mistake I've ever seen much more likely to engage with the unbounce the brand as a result so took a horrible experience and turned it into something special spotted the bugs fixed them within the first hour and pushed it back out there what was it like on a phone okay so this is what it looks like when you get to the Forum looks pretty good it's kind of hard to tell what it is but more importantly when you click where you have to type the keyboard comes up and you can't see the question anymore so you don't know what you're answering now this is when it's integrated in a system like you put on an unbounce page because it's not designed for that when they're Standalone it's actually beautiful interaction so this was just like oh it's not quite set up so I took it off mobile for the time being usability test now this is so easy and so little done people talk about it but I slacked people around the company I said hey do you have two minutes 10 people I was like yeah sure because people love being involved and this is again how you develop a culture of optimization I say the instructions they're standards like we are evaluating this page not you you can't do anything wrong if something doesn't work it's the Page's fault not your fault so I'll give you a task read it out loud this gets them in the kind of how they should be doing it read it out loud and then verbalize everything you do as you are going through this experience and uh so the task you're looking for a landing page course in Google you find one find a wave I recorded with Camtasia find a way to take the course now the reason I recorded it with that lots of tools is so I can see the screen but also the the person's face so you can see how they react to things which is really important especially with something so new quite a few observations although the three dots kind of represent like okay am I typing or is somebody else right because we're used to that from any chat devices where when the dots are animated it's like someone on the other end is typing so a this is like oh there's someone else there also it didn't stop because it's just an animation there's nothing there and that person's already typed so that she continued but it gave her a reason to pause and go not really sure what's going on I gave us a very two because it wasn't that big a deal but interesting now if you have a drop down list it converts into a series of buttons if it's a long list you have to filter by starting to type and it will give you the right answers it's really cool but you still have to click the button some people would type see the correct answer and go oh and hit enter what Matt did is it submitted the first or default value so now we're getting a bit of fake data because that's not what they intended to do so that's a bit of a problem give that a four information am I going to be asked for no so I'm like oh my God am I going to be having to do this for a long time and I don't know how long it's going to take me and this is great benefit for them but is it a great benefit for me because it's not a standard form with four form fields or whatever she had no idea how long this was going to take I think that's like how am I talking to someone is it a bot and what how how many questions are involved here so another great Insight all right click map now I had scroll map on here too and show you that because it didn't really show much and if you do a click map you're going to capture the scroll map too have a look in this case it didn't do much it's a very short page but on the click map I saw something really interesting I guess I could check it out okay I can't stick on it that's the first question that's not what you type I'm gonna take it down here she kept clicking on it uh 12 of all visitors did the same thing I don't want them to be doing that I want them to get straight to the entry field so we need to try and change Behavior there okay so because it's so different my biggest concern was that people would put fake email addresses in there because they're like I don't know who or what this is and the little you know sketched out so that was a big fear I had and sure enough two and a half times as many fake email addresses came through this all right I think it's because of that fear and not being familiar with this kind of interface oh rewinding a bit so spam talking about that so Trends over time some of them get better some don't but in 97 this was the first interrupted device on the web even before pop-ups came two years later sometimes funny but usually just a pain all right but then 10 years later a group got together to do recapture which is actually for good because when you type in here you're helping crowdsource digitize old books that aren't that machines can interpret yet so you're actually helping and then Google acquired them to use after that so I don't mind when I'm filling out this kind of thing because it's doing good now however I mean I know that's great for accessibility but there are some books that I don't believe should be continued in their life I think there's a couple of exceptions and I promise I was not reading that book at the time okay that was not my intention at all it's a thing come on God I mean unexpected gifts from the animals of Africa and if you're from Canada this is the best spam filter of all you put a Red Dot in your mailbox they don't put junk mail through your door anymore it's amazing it's like a little-known Canada Post hack sorry it doesn't work here but maybe someone will do that one day it's amazing and good for the environment okay so all these observations now is the time to try and solve these problems through some design exercises so this is what it looks like with my sketches on here I'll go through a couple of them show what I did so the first one the animated dots I went to space 10 the Creator their GitHub account there was a discussion and someone else had observed the same thing about these and had a line CSS to just hide it so I stuck that in unbounce hit it perfect that was dead easy now people clicking on the oh typing instead of clicking on these buttons I simply made them look more like buttons classic kind of RN CTA color because you know orange always converts better than any other color and I put instructions in the thing so do you work for an agency yes click yes or no below so I explicitly told them how they should interact with this so they don't really want to go to that length but I'm trying to validate if I can change on page Behavior which is the goal of optimization and then I put a subhead just four quick questions so people knew what to expect from this experience now from the which one was this from The Click map okay I didn't want people clicking on that first question so all I did was say instead of what's your Rook email I said which email address would you like us to send the course link to that's not what I'm talking about is it I can't remember no it's not uh ah it is not no I changed the entry field to look like a CTA to draw people's attention to it instead of that first question right because none of it's familiar but you know an orange button is familiar so that was my goal to bring them down there right away and then this one I just said yeah which email address trying to reduce the span because if you think well I have to give him a real email address to start the course you'd be more likely to do so that was my hypothesis at least so now you have all of these from all the team you've got all this uh data and research around it your decisions your observations and then how you solve that through design you did this as a team so you got all this stuff now the designer can go away with this and start doing meaningful work that is all informed by data and the collective opinion of the team and how they all think these things should be you know group uh team-led hypotheses which is a much smarter way so I can take the original one show how they changed it no annotate all of the things they change based on the numbers you put in the put on the worksheets and now you have something to present and to launch that's actually meaningful but the extra step if you're running a B test is not just about the conversion rate right everything we've learned there becomes a micrometric we're trying to change Behavior so all of those numbers get better everything we saw that was a problem all has to get better and it's a much smarter way of thinking about optimization instead of just going we want to move this one number no we want to change Behavior so the first one number uh percentage people click on the first question that went way down because of that design treatment number of fake emails way down again succeeding a lot in these micrometrics but then on the number of personal email addresses I want professional email addresses at mars.com unbanks.com I don't want Gmail or Hotmail because then when I'm doing my email marketing it's not going business hours to their business inbox that went way up in the wrong direction and I think because in my research in Prior years putting the word work or business in the email address field label gets a lot more Pro email addresses I took that out by mistake when I changed this question so I need to put that back in somehow good learning there and the conversion rate it's basically flat lining it's been running for four weeks it's low traffic page it probably needs about eight or nine weeks which is a really long time but I think it's just going to Flatline so the good thing is this didn't hurt conversions and actually and the previous test was against a regular form and that also didn't lose so we're starting to validate this design Trend instead of just implementing it blindly it looks like it's actually going to be a good thing and might be something we want to consider remember this guy at the start this is actually Corey one of our marketing directors at Unbound sad Quarry we call him and I asked them the other a couple it was for a video he did he had to look sad and uh I asked him I texted him late one night I'm like hey you were on the team that did the explainer video what do you remember about it well we made up a bunch of [ __ ] and we started the project because we thought we needed a video like completely wrong reasons and and it brought some of the pain back and he was like I really hated that video I'm just gonna give you a little sneak peek at this offensive piece of creative I'm often and we need them fast I could use some help from design but she's in the middle of a big project and I'll need a developer to actually code the page for me our developers are busy building a landing page is not at the top of their list that's how getting landing pages built used to be now I use unbounce with the unbounce landing page builder I can start with a template or I can build my landing page from scratch that's quick and it's easy unlike watching it the snack is on and on it was terrible we you know and he and we hate the way he spoke at the end in the Integrations part he read Salesforce and we're like don't say it like that but in the take where he didn't see that he couldn't say our name right it was unbalanced I'm Unbound somebody I can get fired okay this is a Venn diagram of uh frustration between these three disciplines and you can see most of it is between the marketer and the designer the rest are some there but it's not as bad 32 of marketers are frustrated 45 of designers and that's because of this dysfunctional relationship I believe this process fixes that problem so there was a lot of frustration at the beginning so I want to try and leave on a more positive note so I reached out again in my research and I said what's awesome about working with these people a few of the responses marker is talking about designers the word magic came up a lot right because they do special things that we don't know how to do [Music] superpowers this is great stuff how about the flip side designers talking about marketers they're enthusiastic and motivated yeah marketing full of ideas [Music] still got some work to do on that I don't know what designers talking about copywriters they're intelligent [Music] finally [Music] they're a symbiotic organism we die with eye of them okay that's some serious passion that is under the surface there's a lot of respect there but it doesn't often get to come out because we don't collaborate and follow this kind of process I think this will change it you know because when it's data driven data informed that leads to more empathy which leads to results thank you very much um that's the link again but I've wanted to speak at Mars for a long time you're probably going to hear this a lot you know Ran's a good friend and we've always unbounce we've always looked up to them so I want to just if you'll indulge me I want to take an old school selfie on this in the stacks thing uh so it's not wide angle this could be horrible can we get the lights up please on the good job all right uh okay so if you can all do like a actually I'm a designer like just go and then this side too I'm not gonna wait for it to develop don't worry all right come on come on recharge thank you very much I'll be around all week so come chat