Chatbots

Key Takeaways

[Music] welcome to data skeptic data skeptic brings you discussions about how data is changing our world our interviews from conversations with thought leaders in topics like data science machine learning and artificial intelligence I give a few talks last year and wrote a little bit on the data skeptic comm blog about my experience researching and building a chatbot you know surveying the landscape of options and speculating a bit about what the future is going to be like I don't usually like to speculate too much on the show but here goes anyway chat bots are the user interface of the future to some of you that might not sound all that revolutionary but I might mean that more broadly than you're thinking I think eventually pretty much every website will have a chat bot why well for starters ask my bank who insists upon changing their user interface every six weeks or so such that when I need to do something simple the way the I did it previously no longer works and I haven't spent 20 minutes figuring out why it's different if they had had a chat bot I would have tried to bark a few quick commands at it to see if it could do what I wanted actually that's a lie I know the current state of chat pots not to mention the lack of technical expertise at my particular bank so I wouldn't even expect the chat bot to work well the chat bot landscape is littered with clickbait hype and chat BOTS that don't really need to be chat BOTS but just past a hype wave are a few interesting applications we'll talk about some examples in today's show what's interesting to me about chat BOTS and why I think they'll eventually be ubiquitous is how many degrees of freedom natural language gives us I can throw down some mean Linux commands when I need to I love the command line for its versatility but I don't think too many of the next generation are going to learn about grep set and OGG describe exactly what you want in whatever terms you want and get a response what could be better Google is a bit like this for certain things my Google assistant can add a new appointment to my calendar almost perfectly well but it can't readjust my calendar when an important meeting is move causing a cascade of other updates to need to happen updates that bear with them consequences in people I you know are more or less likely to want to offend by changing I need an assistant for that who is trained in all the unspoken constraints of my time and preferences or eventually maybe I'll have an agent who can learn those constraints if the technology can get there there's no better way to handle ad-hoc abstract requests of your machine and the technology is getting progressively closer my name is Vince and I'm affiliated with IV AI can you tell me a little bit about what IV day I does sure yeah we are a company that focuses strictly on machine learning and we build ml for other companies we tend to get brought in on things that have to do with language we've especially to you when it comes to language understanding we can go from something quite high touch that is kind of very consultative to being something that's just people using licensing or software well we're hearing more from Vince today about a chatbot IV dot a I built for Sony Pictures called the Red Queen as part of their promotional efforts for Resident Evil the final chapter before we get into that application let me introduce our other guest for today Heather Shapiro so I work primarily on BOTS and cognitive services machine learning work with the community developers startup students trying to get them onto Microsoft platform that's the Microsoft bot framework specifically which is one of many ways you can build your own chat bot we're about to hear about a chatbot application Heather worked on but before we do should I give a definition of a chat bot is that really necessary a chat bot is just a console interface that you can type into and get text back or maybe pictures or video you get content back could be on SMS or Skype or anything chat BOTS may or may not have speech to text and text to speech involved like Siri and Google assistant do anyway onto the application one that I talked about yesterday I've been working with Cornell Medical School they're instituted precision medicines they have thousands of genes that they're looking at for cancer detection and genes and variants and interpretations and it's really time critical their doctors and clinicians have to go through this website too that they have a knowledge base and it takes a lot of time for them so we spent a week hacking with them a few of our developers and a few of their developers and we created a but that they're using now in Skype and in teams so essentially the clinician can just ask find EGFR it's a gene so it shows all the interpretations for that specific gene and all of the variants that are related to that so it saves the researchers some time there and they can get back to their patients which is so many pictures came to us for Resident Evil which was the final film in the chapter so we went away and came up with this concept around taking this character from the film that was called the Red Queen in the film she was this like fictional AI character and then we wanted to turn her into real AI so that the fans of the film could have engagement with her to have some kind of like a dialogue with her to have a feel kind of like they were getting closer to this character that they loved and make it really fun and playful so the first version of that was to kind of have this natural feeling conversation where the fans could communicate with her and it feels really interesting and then there were a few different game dynamics that we applied to that to make it feel like it could be something that would be would be really sticky for the users as well so what we ended up doing to kind of facilitate the AI part that we could actually say we were you know turning this character into AI is we took the scripts from the film there were like six films and then trained a model on generating new scripts where the character was just like a Markov model and then through that output looked at the outputs that made sense through the training and obviously finding other trinidade online and trying to get enough robust set to do something interesting then took the output of the model to give you to the studio studio goes yes approved this is good this is bad this is good this is bad they took these AI generated scripts and then the ones that they approve were worked into a conversational flow and then the fans of the film ended up playing a game against his character the game was very structured and merry decision tree based but the lines they were engaging with from the character were actually generated by this Markov model that was kind of like a workaround where anytime you have the word AI or at a time you have machine learning and there people kind of get freaked out they're worried about how it's going to move through in this case it was part of the promotional mechanic that they were trying to deploy for their project but it was a way for us to achieve that without actually raising too many alarm bells do you have any guidelines on what's a useful bot what are some of the problems we can solve more effectively as a bot and through conversation than a web form let's say one scenario that I've worked on like a little over a year ago that I was working on at the White House we were doing a foster care initiative hackathon our project was for pregnant and parenting teenagers who were dealing with substance abuse for these women most of them are very afraid they're often alone they don't have constant internet connection they might not have someone they can speak to so we created a chat bot that was a counselor pot and it would provide them resources and it could be something that yes they could go on and search in search engines but it gave them the feeling as if they were talking to a counselor or gave them like as if they had a confidant that they could be like oh where can I find the Narcotics Anonymous group or that a a group one's next meeting what where is it the goal for the bout is to be proactive and say hey did you go to that meeting did you do this here's some nutrition guidelines for like pregnancy and all these different things interesting so this application is different from the first two it's presumably managed over SMS and sort of asynchronous or at least a long-form conversation unlike the game of the Red Queen or maybe the doctor looking at the information or even the chat bot that we've been working on that's now available at data skeptic comm it can help you find episodes answer some very basic customer service questions we're still working on that it can tell you a little bit about the staff of data skeptic and it's got a few fun Easter eggs as well those of you on our mailing list should already have known this because the blog post describing how we built the bot was the feature of the week this Monday if you're interested in our feature that we get on the mailing list at data skeptic comm you can also check out our chat bot there let's get into how you build these sorts of things what are some of the tools available yes we've ended up building a conversational kind of a performing modular Enterprise framework it allows for multi dimensional trees allows for many developers to work across the same project and focus on different pieces of business logic for their apps it's not tied to any like particular service so it can work across whatever output we want or whatever channel we look with a tweaking any of the it's a core framework we have a proprietary ml stack that is really good at making sense of language and we also have this like conversational framework which we're going to talk about today which enables us to have lots of conversations of scale with lots of people in that environment and be able to keep track of the state of all the users as they speak to us and how we manage all that work for companies like Vince's have built their own frameworks in order to facilitate the specific needs in advanced types of things they want to integrate into what they're building there are a lot of other generalized platforms available as well such as the Microsoft bot framework mentioned previously yes the bot framework it's an SDK actually in c-sharp or node and there's also a bot service in the azure portal that allows you to it accelerates the process of making a bot it will give you different templates for creating about in any either of these SDKs essentially it just allows you to create different dialogues using the SDK to go back and forth and have that communication with a user when the landscape kind of took off I think that you know everybody and their sister and their brother when I would made chat bot platforms and pretty much every country had kind of 40 or 50 chat bot platforms where everybody went and raised kind of like you know one to ten million dollars one out built a platform and then like great here you go we have this framework and you can build a chat bot and then everybody went and built pretty rubbish chat BOTS and then obviously that was a very good experience for the consumer for anyone that was looking to engage with a business or brand or our celebrity or whatever and I think that was kind of classic example of a hike cycle kind of carrying through this momentum of this thing that pushes a technological hype machine that doesn't equate to something the users want or need and so most those companies have now either fallen by the wayside or doing something else and the reason that that was the case is because if the experience you're creating for someone isn't as good or as useful as just going and doing a Google search or finding the information out another way then why would they go and sit through some kind of decision tree in order to get there it doesn't make sense logically I think one of the ways that the market is settled based on that piece of learning is kind of what is the utility factor this being added by using chat and whether that's a chat pod or that's chat home page or there's someone communicating through SMS how do you use that engagement to really super serve whoever it is that is looking for information and and why should they be doing that through chat versus doing that through another channel it's important to have to make sure you have like a very strict not strict but like focused goal because a lot of people try to create a whole a I saw something like Cortana something like Siri or Alexa that's a great goal but it's extremely complicated you can easily add these intents you just go in and say oh I want to have one for shipping or I have one for receiving a package something like that and it will be the exact same as if you were to click add a pre-built domain but I always caution people make sure you actually need these intents or make sure you're not like setting too far of a lofty goal make sure not to have too open-ended of a question for the bot because it can't answer everything yeah that's a lesson I learned very early on when I started tinkering with creating a bot we actually launched something quietly on the site last year and it was pretty ambitious and ultimately I rolled that back and took a much simpler approach the concept of intense that Heather mentioned was very helpful this comes up a lot in bot design an intent is an abstract idea of something a user would want to do a classic example might be to book there's many questions necessary to do that one of which is something like you know what time should we have it now as you know computers want things in a very specific format but human beings we speak in a lot of variety on time I might say now in five minutes yesterday the day before my wife's birthday maybe the bot wouldn't get the last one but intense nonetheless if I can identify the user's intent I can often just route them to a very specific subsection of the code or even an external service which handles specifically in exactly just that yes Louis is one of our cognitive services cognitive services has I think 30 API is right now so there are different machine learning api's from speech to text vision all different ranges and so Louis is language understanding and creates machine learning models for natural language processing and essentially with Louis is you go in and you add different uh pterence --is so in other ins will be what's the weather in New York today somebody couldn't say that as what's the weather in New York today or they could also say what's the weather here and since we're in New York the bots should know it means New York so there's different ways that people will say things the intent there would be what's the weather or and find weather so Louis when you put in all those intents and all those utterances it will create a model that will automatically be able to determine that that's what you're trying to get the bot or the computer to recognize you don't have to put in every single phrase that someone might say so people can now say different things and it will be able to figure that out there's a whole set of pre-built domains and they're constantly trying to add more so even ones like medical industry they're looking to try to build a corpus for that so one of the problems that we had with Cornell was especially for speech-to-text things like CT scan when you put it through speech to text it automatically thinks it's C T like icy tea that I drink let's talk a little bit more about the development process do you spend a lot of time in ideation and then get down to coding or is it iterative development where you try different things and see how they work out for us because it's part of our core tech we tend to go and do the development once it's really well I'm isolated as far as what the scope is the project and what it looks like for us it's kind of everything is organized from a creative point of view and then goes into development well some people like Vince can be very well organized and plan ahead I've never been the type so when I set out to do my chatbot I just started hacking away and I had the experience that many first-time bot builders do I want to give a quick shout out to Dave Mathias from my mana lytx he put a bot together to help support some of the great conferences they put on and as I recall I think the first person who went talk to it asked a question that the bot just wasn't equipped for despite quite a bit of preparation on their part making sure it answered a bunch of clever things cutting it closer to home shoutout to Adam see who's one of the first people to check out the data skeptic comm chat bot his utterance of I want to buy a shirt was responded to with try one of these choices episode recommendations listener survey store or profiles Adam replies can you take me to the store bought replies I didn't understand you please try again or say exit clearly we're a ways off from the Turing test like we've all experienced this before where you you're calling up and you're angry and you're like you know some kind of customer service file on your phone and you just want to speak to someone you like operator operator and they're like so nice to have you here it's lovely today I hope you're having a nice day we'll be with you shortly please listen to this music while I talk over it and then we'll see what happens in a second after I finish speaking I'm going to finish yeah I hope you're ok - talk to you soon like really overwritten like like you know you tell it's been it's been written by someone who is thinking about making sure that their being is polite and nice as possible but the signal that they're receiving is operator operator operator speak to a human speak to a human yeah I think it can be much more frustrating when you are trying to cram in this false politeness in hoping that this somehow satiates the anger for the person of the opposite either give them three tries kind of like oh maybe you meant this or try again I wanted the number so I'm inclined to call this user interface design but I'm not totally sure is that the right way to label it definitely yeah in the same way that you know you can see it on on a social channel or someone has that you know really well curated social channel where the way the kini cake feels natural feels like you get their character and then you have corporate social channels where it just feels really dry and where's that fine line and what can you get away with and then how can you best you know serve the customer in this case we're gonna take a quick break from this episode and Linda's gonna tell us a bit about our sponsor for this week Warby Parker the first thing I noticed at the Warby Parker eyeglass doors was how stylish they were and their low prices starting at $95 for prescription eyeglasses a third of the price of standard prescription eyewear and frames basically Warby Parker sells glasses directly to consumers which cut costs and save me money I admit it I first started buying Warby Parker's because I was cheap but then something happened this was five years ago and since then they have released more stylish frames especially ones that are lighter and fit my small face and plenty of people have complimented me on my Warby Parker Sony's I ordered both my prescription sunglasses and eyeglasses exclusively for more V Parker and I love them the most unique feature is the risk-free home Tryon experience go online pick five pairs to try on they ship it to your door this is all free and then you get to try them on if you're me you ask Kyle's opinion there's no salesperson pressure looking over your shoulder there's no overly flattering lighting skewing your perspective and you get to make your decision hassle-free at home and mail them back and that's free and there's no risk of buying a pair that don't make you look good visit Warby Parker comm slash data that's Warby Parker comm slash da ta to start your free home try on experience today so in building the data skeptic bot the design philosophy I've really kind of learned from talking to these two is keep it simple and try and control the conversation unless I want to build my own artificial general intelligence which you know maybe we'll get to that eventually if I want a fluid seamless experience for users I should work on making the bot focused but naturally focused try and lead the conversation essentially let the bot always be asserting something and let the user kind of respond in kind I don't know that I can anticipate everything a user might type but we'll keep logs of that and we'll make it err ative improvements with time in fact you'll hear about some of those improvements on the show is we make tweaks to the bot the cool part is since I respond in text the sky's the limit on what theoretically the bots might be able to help you do there's also cards there's carousel cards that also have buttons so if you want to display images or visuals the bots can display them that way you know the Internet has trained us and trained our instincts to be more button based I guess maybe it started before that with magazines some kind of newspapers where we're thinking about things in relation to their physical form and then we're in taking that input of this physical thing we're seeing to inform our decision making process as the internet evolved those button based ways of approaching understanding or asking questions or buying something or engaging with external sources from ourselves the Train is to get really good at that so I don't know what that swath will look like what that device looks like if it tends to become something so seamless that it just becomes conversational and we say exactly what's important to us and get it or if the Internet has started has done so much work of shaping the way we think about how we communicate that buttons and kind of like multiple-choice functionality now so ingrained in who we are and how we'll think about our decision making that it becomes second nature I interesting yeah so a bot isn't just about chat necessarily it can also share multimedia type ideas maybe even control things for you I wanted to bring some of those concepts into our bot I put in something similar to what heather was talking about the cards that are available on the Microsoft bot framework asked about to tell you about the profiles of the host of the show myself Linda our bird Yoshi some of the writers for the site the bots will respond with an image and a little bio again a gimmick but a little demonstration why our chat BOTS becoming a major topic today and not a decade ago I think we had a lot of a lot of success which and I think that's driving a lot of the cultural understanding across different messaging platforms tech companies that are doing really well chat are clearly wanting to make chat more relevant for every decision that you're about as a consumer because then their platforms are worth more and more valuable to the overall thread also I think that we're just spending more time inside of chat generally it's our user behaviors becoming more and more chat focused we're spending more time on chat speaking to our friends and we're spending on social media now that trend I think that change happened it's over a year and change ago where we're that that's which happened where it's actually a majority of our time is sitting in chat which means that in order to modify it's on channels like that we need to figure out ways of communicating seamlessly with the customer and being able to do that at scale and the the only way we're able to do that is if we actually are investing in that infrastructure I think the fact that everyone's already using these different platforms and channels is that makes it such a great time to be able to use like the bot framework because so people are already on facebook Messenger people are already texting so the fact that you can write this code once and then deploy it anywhere it makes it super easy to reach Millennials to reach different generations and markets people are already using these technologies and are texting non-stop or on their phones and social media non-stop so why not target that community that already knows how to how to do that so we put together the bots you could find it data sceptic calm with the help of Zhang Fei Zhang who worked on that for a few months with me also one of my developers Gleb really made a big impact on getting this out onto our site but I was wondering what does it take for a more significant bot than what we have so far to be put together yeah it kind of depends on the project we usually have five engineers working on each project and they kind of vary from like front-end back-end to an ml - as someone who's like more full stack you mention from a product point of view we are based in London Los Angeles and Toronto yeah we are actively hiring and always looking for people that are great ml engineers with a focus on N LTE or full second engineers who are excited about I can get further into it spend more time on it right from juniors up to more senior people I think go to IV dot AI or they could come and find us on Twitter it's at the AI agency let's get some final thoughts on where this whole chat bot thing might be going I think the biggest change will be the speech component we're starting to see that already with like Alexa skills and Cortana skills how they will really contribute to our everyday lives like if you ask Cortana or looks like what can i what skills can i use they're like oh we can play jeopardy or something like that and you're like alright that's great but like how is that really gonna help what I need to think about on the day-to-day so that I can do more of like my job or like enjoy my life I think that's really what we're gonna see of either more in tune with your calendar your commute you see it in the home like it was like nest or there's a bot that will you can turn on your oven or your coffee maker as a certain time well call me old-fashioned but sometimes I just want to press a actual physical button that I can feel rather than have to unlock my phone go into an app and turn my stove on that way huh although you know asking the bot hey did I remember to turn the stove off that's not bad maybe hey I'm running late can you preheat it for me all right I talked myself into it now is that user interface or is that a chat bot I guess the bot part comes into play when there's a agent of some kind when rather than just barking commands at the machine and expecting a pretty prescribed response we start asking the machines to figure things out for us in order for a machine to do that it really does requires some level of intelligence in fact holding a conversation seems to be amongst the most difficult of problems we might ask a machine to work on as you'll learn very quickly talking to the data sceptic bot as it's a day in March of 2018 it's not that much of a conversation partner it's got a few fun tricks and a couple of Easter eggs but we're gonna need to invest a lot of time into it next week in the podcast we're gonna talk to some people who already have invested quite a bit of time in building con agents in fact there's a whole prize given out to the best conversational agents join us again next week to hear more about that data sceptic is a listener-supported program to support the show visit dana skeptic comm and click on the membership tab you