TWed Talk: Minor Gordon on "Test early, test often, and keep your master branch stable" (4 Sep 2019)

Key Takeaways

we are we are live all right there go the projector let me hit the Go button so good evening everybody and welcome to 24 the fall 2019 season yay all right and just for those for whom this is new treads our Telus world educational sessions by curiously totally by coincidence are our logos almost famous TED Talks we so twit talks will occur this term approximately every couple weeks and there is a sign-up sheet which has been circulated on the telus world list these will be interspersed with approximately monthly our pirates the RPI users group meetings and occasionally a little bit less than in the previous term idea talks which are similar to 12 hour focus on my idea graduates data analytics schedules but this term there's going to be a lot of twit talks by idea staff layoffs by Maya Gordon are our new tennis world director of medical edition as opposed to Semantic Web as opposed to data as post work sizes a very creative but our director titles that's right but that's probably get right all right so uh so we're we're very pleased to give minded this opportunity to lead a lot a couple logistical things we are um we are actually not streaming live right now but we are recording on and it's going to be a screen share video recording and these will be posted to the Telus world YouTube site richly metadata eyes so we can so people can find it the way they've been finding our other stuff and that's both you've gotta size but you also free to take on the share of paper okay person good girl aw yeah and the only stipulation is Elvis has got to leave the room at 7:00 right all right yeah so yeah yeah go ahead onion switch all right so this is going to be a talk about great so there's going to be a talk about testing automation and unit testing techniques unit testing and integration testing into investing a little bit of about continuous integration these techniques can be used essentially in any kind of projects from one person projects to huge projects posters coming from my experience in industry I've just been here for two and half months now or that I was an industry for ten years so let's do this day in and day out spreading tests setting I build pipelines and things like that Sarah I want to put a diffuse to acknowledge and we help the practice in the lab so just throwing I'm going to do that why do we need some foundation people write tests they can run test my machine why do we need a lot of a play with this so what are we really talking about with this talk about the test pyramids different kinds of tests how they're related to each other how what tools do we use to develop these tests with methodologies Dewey's told us finally a win and we're so where are we running the tests when are we running access focus here will be on testing for correctness other the software is correct and not on performance and we won't be hearing things like load testing scalability concerns the general prerequisite is you've done some coding some kinds of testing it sometimes doesn't matter what language pretty much everything is going to be language agnostic its overview it's not going to delve into a lot of specific examples so getting into the why why do we do tests and if I do automate tests I think we can all agree that better testing leads to better software that's been known for a long time the more tests we have and the better in they are hired by our subgroup so but originally the the test increasing complexity in proportion to the complexity of the software and so such negative tests coverage we want ninety percent test coverage well if you have a hundred thousand lines okay a lot of tests at some point manual testing becomes impractical in terms of time and effort we need to automate the testing so we can't really automate the writing of the test school now you can get a security context and things like that so we're mostly going on automating the execution of the test once so what are we talking about when we talk about automated tests this is a version of a fairly famous diagram marking taller the pump works as well as this quite often this is the test pyramid and the idea here is this is what you should have you should have a large base of automated unit tests there should be so units we'll talk about that in the next section but it essentially every small bit of your kids so something like I'm knackered all your code should be covered by those things that's not it some people think that's enough oh I just begin testing everywhere that's fine it doesn't really test the interactions well there is going to be isolated so we need another layer of things sort of higher level components idea some notions depend on what the language here it I've been integrations between components so your components could also be things like like a database interface and then API tests one with those two and then finally end engines you notice it's structure to the pyramid in this data and I have prophets I am think I mean I've spent only the very basic as long as I look right go to if I result would look right the idea is that we have mazing here and very connected so we write a lot of unit tests from them all the time they are very quick every fun during coverage we do very little by the way this is what happens in the most part a trip this would be antipattern the test ice cream cone you can see that there but where we have a lot of manual I polling a regression testing so I made this change I'm going to poke around on the user interface a little bit and see if anything that's pretty common approaches especially for small projects and then we have some sort of smaller amounts of automated testing animal yeah yeah it's a little bit odd because particular you're like unit survivor I got here I've been a lot of people given the truth which I haven't run any tests I started out by writing new tests at all that's manual regression testing where do I start writing that I know I need to ask myself is too complicated now to get away with not having tests where do I start all the time people say well I'm going to do in to end tests I've sort of been to the bucket and there's some merit to that of a unit of situation there will always be many regression systems there's no getting away from I've always especially as you're initially developing them uh you just need to like we're talking on the grading area there's some sort of mental model you don't ever internally get away from that but it should be a very minimal part it should not be part of the main middle network for I envisage of merging the natural branches up here suggesting that this is an inverted priorities yes this was the antipattern way too much manual testing so that it becomes a practical so I'd like to discover these individual types of tests of a little bit of detail where the Union test this is a row the workhorse test but almost everyone who knows how to rights and has written have one eye or another you'll see there's in J units I was a sort of canonical testing framework system out of a tree burger meaning kept back and so on your testing you individual units of carried in isolation their unit is different from language to language Java something like a method Python also really a public method so sometimes this is distinguish between public and private communion tests I think people pretty much understand this so you have a adder or some time that's adding numbers and they added public methods in Java it's public because we're sort of assuming that so the test I'll access to it in some way and then you're going to do assertion on the results so a large code base wind up with generally one of these purpose I think I think my servers memories integration test a lot of lot of people don't do these and they'll either your unit tests in an event test or just in your tests integration tests as I do that you have a little bit larger unit of code so say your view rendering engine and you're going to immigrate it with the database you can do this in a couple dinner ways you can essentially have internal integration tests internal Indian test more or less where we integrate everything at once and then a tested with some internal machinery or you can compare life integrations which is generally the preferred approach the idea with testing is that we need to be able to isolate the source of a bug very quickly so there there Welman testing their abilities the whole reason the unit test exists is this test failed but I know exactly we started looking for problem it may not be in them evident maybe in some members of that Baseball's but but it gives me a starting place you might be buggy whereas more coarse-grained you get the harder is and when I get started Oh so if you step up another level thinking about web applications in particular where they have a separation between the front end of the back end the back and console sort of API many systems could be described as we're at the client-server systems we'd like to be able to test the API without without using the actual clients so because you have an HTTP API or a binary protocol I like the FPC or command line interface or something like that I'd like to feel good to simply stimulate that we are considered rightly that's what I need I just is what you end up testing here is what is the interfaces department so it's not a word that's used on currency there these days very much but this idea that interfaces defines by this contract which says which has invariants on inputs and outputs liquid it's not like you know I'm going to give you this khakis you're going to be BAPS I did by educators like that but ultimately what the gate guys are tests are so that interview might have a lot of people module that you're getting media has it and you simply pay you today do that great weight unit have sex with mine who's more girls than that I'm going to cover those included and tools one other was just over here kind of what these things are getting rid of but the news you don't combine the models so the sum of unit tests is not an average confess their their testing in two different days beginner test is the goal of isolation and the integration test is control integration so when unit test for example you don't want the database to fo of as a general rule because then you you're creasing the area of potential player sources okay not only my testing my carry on testing the database as well means if there's an issue I had to pen question at the Davis works their idea is you have this notion of isolation where I want to decrease the sort of radius of possible faults there's something you can't assume like the hardware it kind of make some assumption of what you're integrating race that's like Virginia and Java or whatever but of the thing you control you want to win it in Tripoli I really see us example be denied acknowledge it and then test the interaction especially intense migration integration is more module to module and module with external things there like with the database or a network service so on this a notion of integration is we have your application is running in a very large and Varmus and easier the module might be someone else's things that's ready to great go into index finally at the very top of experiment test the entire application from the perspective of end-user and web of occasions could be there that the browser talked about earlier we want actually simulate a user as they're actually going to be cooking for your a your application or there's utilities that's expects to do this with command-line options for example to do a tiny home alabama' we want to get as close as you can to actually have the users doing because there's assumptions embedded and doing tests and so on about how sort of things are going to be inputs that we can kind of do away with that here we're actually usually implemented testing of programmer because so how do we do this gives us some tools there are a lot of them I must live derived from effects to work on X units so there was the extreme burning book interpretation that's lose the notion that we're going to define this for the standard set of things that unit testing frameworks do I did it's grown a little bit I don't recall iti have really strong the trigger hand rose front locking facility but Pena is that we are able to define tests in some way and click the test into some sort of Suites of related functionality so for example assess could be the liberal methods and we would be all episode clubs and then has some support for various common testing patterns like fixtures and executing good before nap here and so on a test render index or anything and I thought we make assertions there that lets but is really important languages comes built in with assertions but they're generally Korean assertions and K Union pom provides rich assertions there's a Pancras library which was really worth evening Scala tests as I've a similar thing I'm China JavaScript have should based expectations the idea is then narrow down further by having fine grain of services really narrow down what the calls of the player is those new business note that this doesn't really talk about the granularity of tests right so these same tools can be used in our use for a lot of different kinds of testing you can do all the way at the end to end testing with unit tests there are some more specialized tools that are designed for engineering testing accessible you can do of those by the way to token chime in JavaScript s JavaScript if browsers is really hard lead you test-driving it's kinda urgent in there there I want them to rely on day and testing for browser things focus as much on because there's so much display logic Eric's not there's a logic market out there so you know testing tends to be a pyramid doesn't really capture that certain areas of a good are more amenable with circuits so something that you do pretty pretty when you're ready tests particular unit test but also your integration test is mocking this is basically a pretty simple idea is you want to replace part of the application part that you're not so sweet window click there it's going to be a little bit of code that pretends to be a database or pretend to be a network service or whatever the idea is the same as mentioned earlier we want to reduce the amount of credit we're testing where we can actually players let's load when mocking I don't want to be testing a remote network server in my test because I don't care if he fails at the live over here there must at I said they I give it you the network server and HTTP requests with you packed me back in ancient response and that way understood but I know that this program yes right here with the possible exception of if you have sort of triangular dependencies between things like a you know one piece of pages on both the network server Emma Davis may want to be paranoid integrations are no it says my application carried out the database so that I'm on the network so I'm controlling the scope of tests so increases test isolation also the speed of test execution we keep reaching on the network is slow so having this mark that is there's a bunch of in memory memory manipulation professor responsible trees these are also language bizarre tools there here a test family now comes with a mock framework as well so the parents want to tell those positive actually within Semantic Web we have a special advantage because a lot of the clauses the different libraries for estimating with access including the memory version are get fast that are functionally equivalent to accessing those opportunities it's hard to find a sequel world you can get things like sequel lives that are in every databases but even though nothing used code base because there's what's these equals big advantage to other American bases of things so integration testing allah azza mentioned earlier he typically will use the same set of tools like a unit to do integration testing mocking and so on because their test frameworks I've been not really given a unit is about the scope of the test not a house or something is really special about the nature of my race sir Toby reviews metaphor for integration testing as well just with warmup of course the tests are more coarse-grained so you're testing out a whole package in Java or a whole subset of functionality rated individual methods API testing this is come a long way as people are arguably more API oriented than they used to be together with all the web applications in home page web applications so they're not specialized helpers and services and tools to do this probably the biggest of them is called postman but there's various clones okay and I'm talking about working here today but you can all these inaccessible Spears so I can basically in Python unit tests or getting or whatever I can instantiate an HTTP client these mobile grids or whenever I dared to call my server a little bit wonky that's it could be done I wish for glass to be a lot of what frameworks now have support some circuits for doing this but that actually came up already when I think about some haricots spenders was really three things like integrating inspire or open a guy and really like any sure your definitions are covered and your schema I think there's absolute by using a specialized ones are getting something that you wouldn't say getting energy testing framework those doesn't mean tests that actually boasts up maybe I but just really the level of things that were something hasn't come to the best yet finally ended in testing so until relatively recently everyone useful William and William I've suffered from a lot of the failures of the underlying platform with namely asynchrony working with singulated web applications tests which are to be more or less deterministic is really hard because timing issues and people kind of grew to hate winning their other still lots of rappers and just associate out of sort of de facto way to test websites I grabbed um elements wait the miss to load input this text it's this painting manipulating a browser there's essentially a library at a splinting library which is communicating over a binary protocol with a web browser to automate it the basic notion of this been a few new ones recently one that I really like is called Cypress which is they read a lot of the assumptions that we're gay but when I read it get it on the moment of the plastic point in time either they're not there actually be some motive actually tried things like that which take some of the pain away from it not based on the non determinism out of this we've got a lot of nice tools of boards so you can see snapshots of the things you're doing you're going through like at every point a research and you made you can see a snapshot of what the screen applied snapshots Dom like that which is very useful Oh are these light Cypress is it acting as a client and so he's like is it tearing units I know it's him so to speak see all a model of what the construction would be within the browser but also yes that's automatic browser there you get into Sam up sort of situation where you have how determinism introduced by a user twist it like you need to store state as I chosen simply this screen in a river but there's running point of test complete us everywhere close they kind of like rich determiners then and I think about this is really a you can see here like it's showing all the requests going back and forth as the girls changing the colors insects are essentially talking here at a very high level would have happened in the browser which can be true with debugging accuracy is it import like a for example roaming call so you can say like I want this version works with any brother so it's working tandem with it so it's not like poking on the process provides all the brothers support a protocol which allows you to automate them they'll agree was based on swim they actually force all of the other ones and ground the night exploring salon sets for the automation protocol which just a Commandments response protocol at binary gig are their support okay more questions so big offers there Pisa yeah definitely yes so typically do and then is you can have multiple configurations for this so this thing can read its configuration introduced a little term isn't but you can basically say yeah I'm using this version of Chrome and I will turn my behavior based on test the main form of non-determinism you're trying to avoid is picking random things in the application takes good claps thank you you want that to be really tightly controlled but in terms of like not interpreting icons from browser variation things like that you can forget that you just specify your app doesn't work for export whatever that based moobot so yeah yeah yeah it's pretty nice in testing just to be extremely painful because it's the asynchrony and now something's done a lot easier those years there's just so many people would do front-end like complete front end went up occasion so much javascript on them but api testing wasn't really doing it sir but the front-end testing unit testing like the tools are hot rates they needed in doing testing so painful Oh it is but you can sew it certainly the way a lot of people did browser about testing so like user visible performance which is what most who were concerned about hey he's big did things like write scripts for selenium to automate browsers and they were clubbers a service company that said we're going to spawn 500 browsers a cloud and we're all going to run experience its greatest auto-hide the browser does make your be company with the ad what application us I haven't seen people do them in Cypress although I'm very happy that this is a to life Cypress cab so one of the things you want to test is for the beer whether the web app validates inputs going to text invalid inputs plus you like kids with this input from this added these like buzzers or there are separate tools to just using very type things yes this is really important pretty control base what is that I thought so security yes I wasn't so much thinking security I just meant to verify that these informations using validation yeah yeah funny presumably if you put in $1.00 inputs at some point you're going to get feedback accuser right and that's what your it detected here so I grabbed a hold of the dumb the in typed text box or whatever I type something and then I hit this button or just the languages this really looks like wait and then look for something so can you a detective share that message so much going to use every connected is it like sucking out in order to know no you have to do on yep I am yeah so what do you end up doing is like creating parallels between sort of internal structure or like multi controller or whatever and have notion of a page are there any pre can seem like an RBI into the world used to make all of our HTML except I wish are there there another nurse there are HTML accessibility checks like tools that are bacteria yeah they're not just check where oh yeah are good to be jerseys yeah okay they may actually but not pushes people well they will be pointing up if you haven't already but they did before yeah yeah I mean I experiences and it's the lightweight is simply it's really simple Felicity consider 80-85 such as all things you know I'm wrapping a high GI lock final hundreds of toys concerts officially contrasting so that's one level of accessible a shovel we're all done we tailed it on why don't know I know what I'm talking about the things that I've actually seriously know about life those are the relatively look like a but they're using those frameworks yeah you can use yourself how much I regret yeah a little bit of mr. noodles at the touch on this Oh like I'm trying to visualize about certain developments like why you should do this this is returning home in an industry I pretty much do this as I have a matter of course the basic idea is that you write tests book work it and then there's various mechanics Ravis these are the three was I was in most age a queen coder is but Martin it's a vertical called clean codes kind of a guru around some of these tougher engineering practices these days so easy see the call but most write a failing test before you write any production code so the idea is that the testing part should be the embodiment of the requirements this methods go back to the conflict this methods given this Olympics should be this sir I make the assertions that input and output basically before I ever read it you must not write more of the test insufficient to fail or fail with the final so couple educator councils as a test later this is the idea that we hop from one small stable state to the next okay so we don't want to like write a ton of tests and write a ton of to write one test a little bit up there one plus a little bit okay we call some we move one stable safe oh nice at one point you must not write more code insufficient to make the country so if your production code is supposed to be more and you need to add more text so the test should locks them you usually talk about this you'll be rendition people what about red greener factor red is one comes from like a test runners interconnect red and I've got a failing tests I fix two tests by coding something orca campus and then I forget it every number again so breaking it is adding more functionality for example or second book or whatever refactor is the change here's design you define item but unit test is a method is a public method generally in Java or Python every time you add a new public method but having here when you join a project or nice beat you send like the I'm personally oh yeah or you just I just started a bit product where you're at turns oh yeah so I not that he is that reason yeah I never seen so in the general rule I'd seen is that you write tests for all new code and then you write tests for any type of you touch their wondering that affecting your budget and I write a test and prove that the buddies yes and I think the bug there over time for desk for you okay okay hungers if you refactoring egl feedback tricked out in order to extend its capability use the same thing where you have actually I'm doing this with lies right now so if there's a piece of code that I'm looking at it like I don't really know if this is working quite right I need to make sure that it's going to keyboard in this way right a test I confirm that it's going to keep working that way they write City to be factored so that confident that you have a change in the semantics the test we're going to hear security around the vision for business blog you I wanted to talk about one of my favorite things even a little bit the let's comment behavior driven development so this is the idea that a combination of testers and developments I would just talk about San Domingue urban design and I mean urban design is [Music] time to hash in terms of the way it's often expressed but basically the idea that we should talk with the main users and domain language we should have a common language with the main users and this just extends the idea to say well we're going to write tests and oh I use the domain TCS understand so this obviously only applies to fairly high level things right and ends up having too many users don't care about methods as a general rule although you can kind of shoehorn it that way that typically this is talking about whole systems there we server the user story remember that but there's the idea that users giving you something that they want your application to do at a certain time the user I want this feature from your code so that I could get this and patched to that I have some way of saying did I actually get this to come to do like definitely have done so this isn't stepping down I'm actually going to implement this feature say it's a shopping cart or whatever how do I know that it's good for users I can find you acceptance criteria which are what are we going to be determined by the music oh I want to click on this and so on listen dialog Gator of course but even then I'm on this page when I click on this then this should happen when I click on the cards given that I'm already added items to my carts when I click on the cards of a nice because I okay there's a really good blog post from civil webs about how to write the user stories of users about them over here that I just want to choose more in action so this is with us someplace and breakfast when you companies are they want to employee banking example there's an account holder they want to withdraw cash so you've got this scenario which is open way to finding a little bit of work cloaca application right really you know the whole thing's a couple of screens I'm going we'll do our patch because the scenario soak up some sort of setup and use an English language description of things and then be given Wednesday really using about this is we defined the test in terms of the domain language accounts and cache and things like that we're actually going to use get other natural language processing see your part stats and translate that into actions on your application so usually just regular expressions of some kinds given this my personal that and then I translate that to for example commands on Cypress your go dessert page Emerson data set up the whole thing and then when I do some action okay I kind of like that too and from a browser I did it over and then I type to see that I'm at the side of associates the great thing about this is then you can really provide a basis of negotiation with the clients on designer cases when you say you write just write every demo naturally is probably don't even doing more there's there's more a complexify sometimes that's alright you want it's similar to English right away you know I think you say I'm only going to words wind blows and I know look for account holder and this curve and I'm going to have an idea I'm not good for something I thought but you're not like writing oh thank you like as a human looking at this and I'm doing what piece of code are you actually bringing the client rights to be for the client in combination with usually like a business analyst price piece and then the developer is the one you write the red X translate reference to actual like other image oh yeah no yeah that'd be amazing if you could do that well first writing through psychic Oh give you that kind of maybe easier them just partners so I couldn't change them out right and I just reuse that page that there's no set of commands from the browser to do that the goods that how about the page by me when and where so most of elders when they're starting out run tests in their development environments the president is quite convenient of course the IDS all support this you know chestnut panel on it extremely sufficient of a red gradient back to work so I've broken the code and I can pick it on my local slips on that I can worry about showing it to other people given that we're making these really small changes we don't want to be committed so how does this is you're not gonna run all the tests this way as you get it really large system accessible just explore in slower there's more than to run their generation tests which are testing like databases and things like that which is flowing you down so that makes it less likely at all the tests will be run which is a really bad run all tests are right and whenever we change make significant changes in the system it also was isolated your your system is set up for developments it's not a production system production system is state away because the only has a Giri not the JVM things like that say we really want to emulate the production system it's only one environment so you got one ticket version of your OS and so on and it also may not reflect away if there's so the idea is that we want the tests to run somewhere else and we want them to run a lot as fast as bones so that's where they've been use integration okay once we make some significant changes we're going to push the changes somewhere else and some server is going to pick them up recreate a production like environments and run a little test I tell you what what failed I was all into difficult for these we use those top two I think it is when you some host the others are sufferers of service early circle CIA why as a lot of these are now are using a docker is a graduation things like a been humble criticisms really cheaper offering because many variations so I summary automate testing about the order header software cluster and its years that there many different kinds of software testing and educating them all and fortunately care of it even for small projects in sports parties on these custom hopefuls people early IBM damn intestine to us if you're building out a designer business those in continuous integration there even even a very personal projects that I do I've continued efficient services well worth the effort set up questions yes as a rule of thumb Ford has been titled Kimberly or protection for that um they're generally Schnapp the exceptions because if if you have public methods it's one per customer I'll say 'amen yeah also no merit hey you can ask mobile people when they would give you different answers but typically a licensed public is you're not going to get you can't test the private it's really hard to test if you don't want to do indirect testing with Utica so I don't want to like sort of testing your private method by going to a public method that's going to work say the testing in Java for example is typically package prizes or your public potestas don't think I can just serve you know that does look right nothing's ever anyway one in 100 private methods I read people since there's a if you think is a classic contract that the private method is if you're not really have the public look at the public eye that actually does would I can doesn't matter how the plan some prior hours work step other questions be I can help in DC updation majority of the time you have to multiple yes and you get into things like passive use um where you can basically replace put the model and controller yeah there's special patterns because in your business no necessary blighted generic algorithm occurred so they're going to be on to get white testing patterns that's before we get rich you mentioned yeah yeah so it's a common problem in software development that's the developers just don't know when to stop for whatever reason and so you have to agree what that's going to be that's the definition of done you can do that in with the levels you can say well acceptance criteria I thought so be Steffes criteria but I haven't documented so you're some sort of like supposed to be a wizard in some of these I don't just want functionality I want all my other justice you interview passenger games documentation and I'm scared to do things like that but in the end a lot of this technology so here's a weird question okay a million years ago I would say consequently developing and I started outlining a career path before the creation of different self just work work now so my question is is there some sort of equivalent oh yeah yeah sure obsessing or extracting some financial while self-testing is possibly components yeah that was this whole idea Thomas Piketty way back in the day is was like of self-healing systems the systems would watch each other and reckon around areas and you see them built into there's a really kind of philosophically subtle problem of this notion of like were goals right so you can really tell you what the real stays in the air because you can essentially got built into languages so like airline Kirk levels like the health of telephone switches as national watchdogs book into the languages watching this process watch me networks are possibilities all right created by comparison yeah sure I'm sorry there's a implemented in Prolog buzzer yes I'll be aware you're getting sort of touched when it's coming in during that's a huge mess yeah this is you know like isolated component to something that keeps recurring himself or engineering and better I never quite get it right part of the it was just so tempting to break layering boundaries so another sort of related question similar question is ideally we're not using we're not creating our entire system getting software components from other pieces library packages are there what sort of testing facilities are preferred have provided for you for in modules appendages elsewhere sofas for higher levels so really yeah so you can't really count on other people to do their own thing plug them to get up you'll see these badges and seen this what do you say it's absolute testament you can get it so use one of these public like Travis was the main one that one people use for a long time where they is when Devin tree problems worse projects on we would like to dive into do I you don't run someone else's text those are gonna roll your ears right yeah you just have to trust people we're coming in okay so why don't you know like cocconi standard that would be here baby so probably yes I don't we um I want to shut them self aside Castle let's thank minor week actually Candace idea talk and if it's on blockchain and smart contracts see an announcement for it i watch the announcement it will probably be at this same time hope it will be you