Building AI Products with Hilary Mason - #11

Key Takeaways

[Music] hello and welcome to another episode of twiml talk the podcast where I interview interesting people doing interesting things in machine learning and artificial intelligence I'm your host Sam charington I want to start out by wishing everyone a very happy and very belated New Year I'm finding it really hard to believe just how quickly the last few weeks of last year and the first few weeks of this year flew by needless to say I'm super pumped to bring you this new episode of the show before we get going I've got a bit of a holiday gift for some of you that's right over the last few weeks I've received a few requests from listeners who've wanted to listen to the podcast on their favorite home assistance well it's taken a bit of doing but I'm happy to report that the podcast is now available on both Amazon Alexa and Google home check this out Alexa play the podcast this week in machine learning you'd like to play the program called this week in machine learning right yes this week in machine learning and AI getting the latest episode here it is from TuneIn hey Google play the podcast this week in machine learning learning an AI podcast twiml Talk number 10 Francisco Weber statistics versus semantics for natural language processing note that for whatever reason Alexa doesn't like when you ask for the podcast using its full name this week in machine learning and AI but this weekend machine learning works fine on Google either works if you have any problems just repeat the commands that I used in the demo now I like to think that at least some of you are listening at home on your phone speakers and I've just commanded your device to play the podcast if that's the case enjoy it nice all right moving along to our program this time around our guest is Hillary Mason who I interviewed last year at the O'Reilly Ai and strata conference in New York City I don't know that she'd refer to herself this way but Hillary was really one of the first quote unquote famous data scientists I remember the first opportunity I had to hear her speak was back in 2011 at the strange Loop conference in St Louis at the time she was Chief scientist for bit.ly the company that popularized short links on the web nowadays she's running fastforward Labs which helps organizations accelerate their data science and machine intelligence capabilities through a variety of research and Consulting offerings I tracked Hillary down at at the AI conference after hearing from an attendee that her talk on practical AI product development was their absolute favorite session Hillary and I had a wonderful although somewhat brief chat that I'm sure you're going to enjoy and learn a lot from of course you can find this week's show notes at twiml ai.com talk11 and now on to the show all right hey everyone I'm here with Hillary Mason of fast forward labs and we're at day two of the O'Reilly AI conference uh the first actual O'Reilly AI conference as we were just discussing that's right and uh Hillary gave a talk yesterday that I didn't get a chance to see but I heard great things about it so why don't we start by having you introduce yourself and uh then you can tell us what your talk was about sure so I'm Hillary Mason I'm the founder of a independent machine intelligence research company called fastforward labs and we look into approaches and algorithms that are emerging in the machine learning AI space but that are not yet widely understood and we do our own independent research to make them useful to people um so we write reports that are a survey of the techniques from a technical perspective at a conceptual level talking about where we think it's going to go any ethical issues that might come up um do a survey of the commercial landscape um so what vendors are out there what we think the interesting application opportunities are we also build build workking prototypes of these things and finally we act as technical advisers to our clients like their nerd friend um and help them actually build their AI machine learning products more effectively um so yeah that's what we do uh everyone needs a nerd friend yeah I mean we all have that friend even you know if you are a nerd you have your nerd friend on on your music nerd friend and your friend who's most likely sitting in front of a computer at 9:00 p.m. you know we all have those people right right so your talk what was the title of your talk so my talk was practical AI product development um and what I was trying to accomplish with this talk was that coming into this AI conference there's a lot of hype and a lot of um lack of clarity around what it means to actually build an AI product um what an AI product is so what I was trying to talk through are some of the challenges we see in going from the idea and the algorithm um going from the press release if you want to say it that way to the product so being able to say uh we have a data set we have a business problem we understand and we have some you know we're willing to invest in trying to make something how do we actually do that and how does it differ from um data analytics and how does it differ from software product development because a lot of people today are trying to take uh a machine learning product and sort of put it into the software development framework and they tend to run into a few common friction points when they do that okay and so I suppose the those friction points were the body of your talk yes so um things like how agile software development is really optimized for building a product with commodity technology um but that isn't how you build a data product um because you have to understand that maybe even if it's a good idea sometimes the algorithm you've chosen won't work um you have to make sure that the accuracy of the system is within sufficient bounds um there's a lot of work to do around how you productional eyesee and operationalize these things how you monitor not just that the server is up but that the model continues to return high quality results over time as the context and the data changes um and so all of those details are something that you really have to learn right now by doing it and we have yet to really standardize on a common accepted practice and so in my talk I was sharing what we've learned and what we do and then hoping to have conversations with people around what they do and what they're trying to do um and so yes that's what the talk was it seems like agile would be perfect for this environment where you know things like uh working closely with your customer the end user of your uh your product um you know failing fast kind of at least the things we commonly think about agile there's also a whole software development life cycle thing which may be what you're referring to so so on the surface it's absolutely a compatible philosophy the idea which is why everyone falls into the track exactly um because when you go to implement the details is when you run into the problem when you have to say you know how long do I how many points will do I think it's going to take me to find an algorithm that can produce a useful result and it doesn't take into account the machine learning process of of uh developing really experimenting and saying that you know might try to solve problem a but it turns out problem a is really a lot harder than I thought it would be but I can solve problem B that's also useful in this product context and it also doesn't deal with the um once you have something that sort of works doing the simplification and scalability work which is just as hard as the initial algorithmic work um but often gets overlooked in a you know AI con conference where everyone's excited about what's shining and new okay so to me that says that it's not that agile methodologies are fundamentally you know ill-placed in these types of problems it's that our sensibilities for estimating and you know understanding the development process that kind of feed into an agile methodology are off like we don't have have philosop right they agree but the the mechanisms are second class citizens so you can allocate a Spike time to go figure out an algorithm but that's a hack and there's no first class mechanism for this sort of experimentation and iteration okay okay and so how did you um with that being kind of the premise for your talk what were some of the things that you that you dove into uh so I love to tell stories so I talked about a couple of projects we've attempted that didn't work out so one was using a deep learning image classifier to let you take a picture of your plate and get a calorie estimate okay which okay that's sounds maybe that sounds like a good idea my team we thought it was a good idea at least worth trying we found a lot of data there's a lot of food photography out there um and there's also a lot of data on you know a cheeseburger has this many calories it did not work because that data um a cheeseburger can have anywhere from 300 to 2400 calories and these data sets just simply don't agree uh and we did you know first we're like okay we want the actual calorie count from the plate and then we decided on a more modest problem which was can you tell us if it's very healthy healthy or not at all healthy and eventually we uh decided that it was no longer worth the time and investment um because the the quality of result we could get was not actually useful and of course this is a fun story to tell because a couple months after we did this whole process Google announced that they had in fact solved this problem uhuh um and uh you know to me that sort of validates that it was a good idea but we didn't have the resources uh to make it work so I talked about that story I also went into depth on uh brief which is a extractive summarization prototype we built using neural networks for articles so being able to take an article and pull sentences out of that article that are an effective summary of the entire content of the article and that's something where there's a product design piece and there's an algorithm design piece and they have to work together well in order to make a usable useful fun prototype and so so I went through that whole example in the talk uh can you talk about that one in a little bit more detail like how did you go about that and what was the process like yeah so um so the work we do is always framed around an application and so as much fun as as is it might be to say like okay we want to spend four months using deep learning to analyze text which is really what we did um we decided to focus first on summarization and then under summarization there sort of two major schools of system one is extractive so pulling words and sentences out of the body of text um and there's abstractive which is constructing a summary that may contain language that does not appear in the underlying text um that's new language uh we focused on extractive because again in the product context we could actually build something with a high enough quality result to be useful whereas on the abstractive side we're still as a community very early right um and so the results are kind of variable so again there was that focus and then within that you know we looked at a couple formulations of the problem so one is can I take any article um and extract those sentences and that's uh the system we ended up building it's trained on about 18,000 human authored summaries with quotations of news articles okay um and it works very effectively on those we also did a second formulation of the problem around multi-document summarization so if you have 5,000 documents on the same topic can you cluster them effectively and then Summarize each cluster and for that we used LDA for that first step um and actually my colleague Mike Williams will be at strata tomorrow talking about all of the technical fun stuff underneath it um yeah if you're interested in that okay and so for that uh for that example the data set that you use was that a public data set or yes it's from a website called the browser which is a terrible website name um because of the ambiguity there but um yeah so it's a public data set um and one that turned out to be quite effective oh interesting and LDA latent duray allocation absolutely and how does that I've heard that come up a few times I don't really know how it works what's the 30,000 foot on that okay so the quick conceptual overview is that um it's a non-supervised or unsupervised algorithm meaning you take the stream of text and it is able to infer related clusters in the text fairly effectively one of the limit ations is that you have to tell it how many clusters to look for which you may or may not have an intuition for going into an analysis um which again means that practically the way people handle that On Any Given body of text is to sort of try 10 clusters 100 clusters and then narrow their way in uh intuitively and by clusters are we talking like engrs or are we talking conceptual clusters we're talking groups of documents in this particular case so we applied it to Amazon product reviews and we found particularly great uh results in the pet product review category because this is a a section where people are quite passionate about the the items surprise yeah it's it's no surprise I guess um but we were you know a couple of uh examples we ran into were things like a dog Choy that um you know 90% of the reviews were five star and 10% were one star and so when you look at the clusters of those reviews you see that you know most of them are things like this is cheap I can buy it at Amazon it's great this is really good for my dog's emotional well-being and yes people are very concerned with their dogs emotional well-being and then the the 10% were sort of like yeah my dog ate part of this and had to have a $4,000 surgery oh wow um and so that's the kind of structure you're able to pull out with LDA um and uh the utility there I think is fairly obvious um or or rather one of the things I I mentioned in the talk is that we tend to see these algorithms applied to making things we already do more efficient so if you can make that 20page article down to two pages that's making me more efficient but if you can make me able to read 5,000 documents which I could not possibly I could not possibly ever stand to read 5,000 reviews of the same dog toy I can tell you that um but now I can get a similar amount of value um that's sort of a a really useful AI product um and when you say a similar amount of value what was your what was your optimization function what were you how did you measure whether the value was similar um so that's a really good question and in the case of our brief prototype we had um you know some human curated test data uh but to be honest a lot of this is really uh intuition which I know is a dirty word in this context the world of AI but I I really do believe in the value of um user testing feedback loops and human intuition and guiding the product aspects of these this sort of work MH so what were the uh did you have a kind of an enumerated list of takeaways for the from the talk was it prescriptive or was it so it was more laying out a shared vocabulary um and then sharing some experiences but I'm not going to presume to tell you how it's done right um because I think that where we are are in the development of the practice of of AI product building is still very early and this is um you know I've been a data scientist since the very beginning and it's very similar to what happened with the evolution of the profession of data science where a lot of people are doing a lot of different interesting things that are all related um but there's no one vocabulary and no one process that everyone has agreed on yet and so I shared my point of view I got to talk to people afterwards for an hour and a half out here um hearing other people's point of view and it's it's just we're at one of those really exciting moments I think yeah yeah um have you done have you uh sat in on anything else at the event like what else do you think is is cool and interesting um kind of in this realm so at this particular conference one thing I'm really impressed by is the different perspectives in the room so most of the conferences I've been to are either technical or sort of business or sort of a product design um here we have everyone in the same room uh which is great you know VCS business folks startup people big company people um you know software developers machine learning professors all here so that's really cool um I've heard a couple of um you know I always love the opening Keynotes they were pretty great um and then there have been talks on everything from from you know tensorflow for mobile poets which was Pete warden's talk and he has a great blog post if you haven't seen it um all the way over to the future of natural language Generation Um from the folks that automated insights so that's you know just a few of the things I've been enjoying yeah nice nice so how long have you been doing fast forward Labs so fast forward Labs is uh going to be 2 and a half years old soon okay um and we are eight people plus two interns based in Brooklyn oh nice where in Brooklyn we are actually moving our office this week over to Atlantic G barklay Center oh cool so yeah you are any of your audience should uh let us know and come stop by if you're in the neighborhood nice nice awesome well I appreciate you taking the time I know you've got a meeting to run off to um thank you so much it's great to get an overview of your talk oh yeah it's great to have this conversation thank you all right thanks all right everyone that's it for today's show a quick note for you guys tomorrow I'm off to reworks deep learning Summit in San Francisco if any twiml listeners are attending or will be in the area please reach out to me I would love love love to connect up with you also please do leave a comment on the show notes page at twim a.com talk11 or tweet to me at Sam cherington or twiml AI to discuss this show and let me know how you liked it thanks so much for listening catch you next time