Stanford Seminar - Alphy and Alphy Reflect: creating a reflective mirror to advance women

okay 12380 uh this is uh November 2nd and uh our speaker is Ted Zucker who's going to talk to us about the new company he is putting together called healthy yeah hi everybody I'm uh the CTO of Alfie but more importantly I've always been interested in how people communicate using Ai and user interface for those purposes and I've run Labs at MIT and CMU on considerate systems as I call them and today I'm going to talk as much as about considered systems and my research as I am probably going to be about Alfie um you see on the left of this screen some of the people that are involved with this company and our goal is to advance women um I will pay about you know just a couple seconds revolutionary app designed urgency women it's about career again deeper connections and belonging and better communication so the better communication is mostly uh what this Alfie reflect thing that I'm so excited about is about and we use Alfie we use AI throughout the system to recommend things and to help people improve their understanding of who they are and what what they um are learning and what they can do you see here on the right hand side kind of a uh an image of a kind of a report card that it gives you it's not my favorite but it's one of the things that we uh do within Alfie to help people understand uh how they are how they are pronouncing themselves um I'm going to start with um this this talk has these uh five parts uh and I'm gonna start with AI can help human uh interaction so AI uh can help humans communicate um if you look at what ai's been helping over the last uh while it has been successful in various regards although for a long time people believed it wasn't going to be reliable um or predictable at interacting with people in the 80s it helped with investment opportunities and made people very rich people like d e Shaw who is a PhD student here um online searches in the 1990s became a huge uh use of of AI e-commerce in the 2000s and car safety began using it in many ways everything from from representation uh and modeling of people's braking uh in in a ABS to to other things as well in fact ten percent of car fatalities come from driver technology overload uh so um this is this is you know there's books written about it and and in fact my student Raul Rajan at Carnegie Mellon wrote a paper with me where we actually were able to use a very we tried 13 different Biometrics and found one that could discover uh using it's a camera based one uh whether you're stressed fast enough that we're able to get people to stop making um errors and in fact avoid crashes in our simulator or driving simulators so AI can help in that way uh if you notice the difference between that and uh the way that the autonomous vehicle Direction's gone my stuff tends to be about helping Advance people as opposed to replacing them uh you know if you you know give a person a fish stay fish tonight if you teach a person how to fish they fish for their life and I think in in general there's an awful lot of um issues there's a lot of value in simplifying our life by doing things for us but there's also some value in learning um and here we are at Stanford so um technology and disruption and frustrations have been part of many many meetings um and uh one of my systems Roger that that we'll hear a little bit more about later um that if it uh if it if it says Ted selker has entered the meeting it actually is a disruption for everybody so uh we've done some things to think about uh ways of improving um how how communication happens to people when they're interacting and if you think about a meeting it's actually human human communication interaction that the computer now is starting to mediate so that's kind of interesting is how we are moving from human computer interaction to human human interaction in our nursing in our goal goals for for a lot of a lot of the systems we make so I I I stand here to tell you that automated feedback needs considerate systems what is a considerate system well a context aware system is one that um is aware of what you're doing and where you're doing it using sensors and AI modeling um and AI modeling uh includes so many things we can use I mean today we use Transformers we used to use neural Nets we still do random Forest support Vector machines hidden Markov off mobs Common Sense rules rules are still a big deal knowledge representation has been part of um uh you know expert systems and and a lot of time people say that stuff isn't cool I was I will I have a lot of examples where uh simple uh simple methods of reasoning um can be can be quite helpful but in any case my view my my my my position is that um systems are only helpful to people when they respond in socially appropriate ways we we have been living in a world where computers are egocentric and they expect us to understand everything they understand we never do that with another person we we start off by trying to make them feel at ease that we're actually not going to be um a pain uh and so um this this idea of adding this element of empathy to to the computer is something that I've been really focused on for for for decades um if you look at social contexts in which online tools help people did they have a long and an illustrious uh and and also uh bad uh background our email uh we're faster than letters but flaming became a problem right off the bat at the very beginning flaming is when you uh are un filtered and make dramatic statements um Stanford AI lab a few steps away from here I built a system there where uh we used um a common frame buffer where everybody could collaborate and it supported that entire community of researchers working together with you know keystrokes intertwino I think that we now do online is is edit together happened all the time there but but uh backing off to you know what what happened with this communication again bulletin borders uh without monitors go dramatic just to give you an example I was in the room when the CEO of IBM a guy named Lou grishner was being shown for the first time a bulletin board and he typed in hi I'm Lou and what came back was hey Lou want to screw and uh he said oh yeah I knew they would be like that uh which was you know sad because they aren't only like that but people want to be noticed they want to make a presence of themselves and they go dramatic it isn't just uh the social media platforms although they have been accused of bias towards and making dramatic dramatic silos um you know and they use all sorts of f now Facebook spends a lot a lot a lot of energy trying to to counter this in fact my I heard through the grapevine uh maybe it's not true that they have 15 000 people trying to monitor uh Facebook right now that's pretty crazy of course they also have ai Bots doing it too but uh you know and if you go and you say well where where has the green always represents in this in this in these uh this uh places where things went well so slumberger long ago started representing everything about a person whether they'd taken a course whether they'd dug a dug a dug a uh oil well whether they you know so you know whatever they've done at slumberset and they used this in an automated way to build teams um to to to uh go off and do some some Consulting to build a you know to make a oil well or something and it was so important to them that they they wouldn't share the software with anybody so you know it's people's experiences helped them it seems quite not natural right now but when I first ran into this it was it was a big fancy thing travel role is a is a uh um a prototype uh uh shy ride sharing system that I designed and actually published about where it kept track of um why it would expect that you know I and you Dennis would be okay in the car together you know a little bit around when Uber was being formed and then what was my need and your need that was coincidence so you know I want to I want to study French three times a week it would notice right and so I also want to go to the ball game and I live close to you so maybe say hey why don't you guys go to the Ball Game find yourself speaking some French um so uh coach was um a more complicated thing which which actually is still believe it or not sold in what used to be os2 and what what is it is it or does is it watches your demonstrated performance and experience and decides how to help you based on that what helped to give you um in a hierarchy of um of of of abilities going from novice to to expert and uh people using it in my experiments uh produced five times as much code as people that didn't use it including the same exact um help it was choosing this is this came you know this was done my PhD it was done at a time when people believed that any AI in the user interface was going to be brittle and difficult in fact it competed beautifully uh so how did I do that well we can talk a little bit more about that but uh empathy buddy is a system that uh weighs heavily on this talk and I probably shouldn't be going through this in this detail but it it um it's it's all about making people aware of how they're how they believed as a as a as a communicator and uh happy sad surprised angry emotions that the psychologists always talk about it kind of put those up on the screen in pictorial form and people uh did uh reflected on the emails they were sending and actually changed them so as I go go on uh I will just say that Roger that is another communication support system considered conference calling system is one alpha recommend is and these these last uh you know five or six are all things I've built but Alpha reflect is most pre Uh current one and it um uh also recommend and Alfie reflect um it previews how people will uh how other people might interpret your contribution and makes you a little bit aware of that so uh my I I would start my AI can help interactions Story by saying that technology is inevitably used for social purposes everything people do is about other people um and and I want to move on to we're always social actors so um AI can be considerate the question is when am I doing something uh remember talking doing something or thinking uh timing uh for requests and response is kind of part of being considered isn't it uh you don't want to be talking over another person how social rules of engagement you know uh include that you know maybe there should be five affirmations for one one uh criticism you know that's something people talked about anyway what does listening mean anyway um what peripheral you know I'm aware that my my PowerPoint might go down the door might break the projector might fall from the ceiling but that's not what I'm thinking about I'm thinking about these PowerPoint slides in front of me and that peripheral stuff I'm able to keep out of my mind unfortunately with egocentric Computing they they tend to as this one did pop something up that I have to pay attention to which isn't typically the way people interact with each other unless they absolutely are falling down or something uh why appropriate responses are fundamental to social success so the there we are so so in fact the first thing I wanted to talk about in Social actors was that when we were communicating with a person we use you know non-verbal behavioral cues like their posture and their height and their gaze and their vocal behavior and their forward posture and their interpersonal distance and Destroy and what they're saying so they start with an intentional stance they have social actions and then we recognize the activity with in this case text and then we have uh intentionally recognition and finally we have a rational agent that does something so that kind of that kind of circle is is kind of what happens in a social Dynamic there's a couple of papers uh that are really nice about that um that stuff um and and in fact when we want to be considerate um we want to uh inform uh and and negotiate a relationship Stephen Pinker talks about that uh considerate systems are have this appropriate social uh feedback and my case I want to say what using AI models of social context and user and the point is to reduce uh effective load you aren't going to be uh as worried about your emotions control responses and reactions to each other because the information is pertinent and valuable thoughtful and caring receptive and reciprocal not distracting and disruptive and condescending and hostile and manipulative right so okay so somehow it had jumped to these two slides before but this is a huge number of examples that I've made over the years to demonstrate some of these ideas and to explore them and often what I've found in running experiments is that um what I started off as a as what as a vision turned out to be a question so for example with empathy buddy up at the corner left um that funny ugly face up there was the result of it not are people not responding at all very much when it was text that said surprised angry happy sad but the Picture People responded very quickly so the interface and the interaction in that interface is always critical to to um to part of the to unders to going from the recognition um and the interpretation to the action as I was showing in that Circle earlier um so as an example um of of uh reactions uh expressing intention here's Andrea Lockard she's now a full professor at um uh at University of uh Texas in Austin and Floyd Mueller is an uh Professor these are two of my students um in Australia um and what they did is they went out with a couple of uh sensors on them um and one was a GSR that's galvanic's skin response when you're upset you sweat um also you sweat when a car honks a horn or when the cloud goes overhead so that uh you see uh the galvanic skin response going up and down and up and down and up and down while they are filming there's a video camera and she's filming people around around um uh Harvard Square actually uh right at the right at that uh the red line in uh entrance to the subway and what we see is that she giggles these four times and those gigglings correspond to somebody videotaping her videotaping them somebody playing uh drums on a um you know on in the street Etc the point is that those are actual places where she was delighted by what she was videotaping and so even though it seems like a very blunt uh um piece of information her voice was quite strong at indicating what she was interested in what she was doing much more much better than the biometric that that people have been playing with at that at that stage and uh so that that's that was kind of exciting to me um in trying to um understand social mistakes just um I'll give you an example of this this uh study up here we had um somebody that was dominant they were this this represents that while there were two people were trying to work out um a chess a a chess situation where there were three Three Steps From A Checkmate and they had to work together to solve this problem um this green person was doing all the communicating until we said and that statement of turn taking made a precipitous drop in their dominance and the person that wasn't speaking at all down down here we had them we said to them and they immediately started collaborating at the same level as the person that had been dominant and this is five minutes notice that yeah the dominant person gets a little more dominant but not much what's shocking to me is how that little intervention verbal intervention made a huge change in the in the in the relationship between these two people um just to give you an example of um of of that Ted has entered the room here here's here's a two alternatives to Ted has entered the the uh the meeting John so that is a door opening and saying John notice that the error rate is very similar to Ted has joined the enter the meeting above let's take a look at an alternative this is me leaving the meeting John so we literally by saying John as we leave the meeting and John as we enter the meeting we're able to get half the errors in Communications between people not not in people knowing whether he was there but literally the mistakes people made in communicating with each other so I've done a bunch of this and some of it is in um these papers uh and more of it has crept into a system I've made called c3.chat you're welcome to go online and and try out this video conferencing system I've more or less uh waiting uh for for uh um to have the funding that it needs to go forward but what it does in the meantime is it does make people aware of who's speaking and improves their ability to communicate um uh you know gen uh over over um not having interventions like that and it uses AI to do that it has this modeling um another another example of something I built that was um so in this case what you see is a background is there was a desktop and it had a window open with a spreadsheet and a window open with your email a window open text messaging a window open with with with an order form and your job was to sit there for an hour and a half filling out orders and you'd sometimes get a text message from somebody um saying hey we've changed we you know we're out of inventory or we've changed the pricing or or by the way you know you should take a break now so someone who is social some of us work related and um anyway this this uh up here this box is kind of modeling the the the behavior I mean detection and what we did is we would literally take text messages that came in and reorder them by what whether they were about the same topic whether they're about the topic you were you were you're operating with in your email or text or whatever window you're in whether they were um uh um um uh more important than the one uh in front of them and we would delay them zero to two minutes okay 25 percent uh performance Improvement like like the state and we actually anyway with it's there's a lot of work that can be done this area it's a very exciting area just imagine that if you don't disrupt people when they're doing something important if it would matter to them it would okay and and uh here is my the empathy buddy that I was talking about quite proud of the fact that um in 2019 it got a lasting impact award 20 years after it was uh published but um this is another thing which is this weird Ransom note and this Ransom note email system that we built I say Ransom note because it got this orange and different sized fonts All These Bars all over it and all this stuff um it it actually um when people had um this feedback about whether this was somebody important for you to respond to or this was uh some something that was urgent or whether this was uh somebody new people uh even though they looked at the same number of emails as a normal email system they responded more appropriately and what was especially important from my point of view about this is our this was this was a you know a little while ago our machine learning and our modeling wasn't so good it was a graduate student working on it whatever um some of our some of our machine learning was only 60 to 75 percent correct so that means that the indicators in these awful colors wasn't even correct and people still did that not only that but you did that you know maybe an email you'd want to respond to is green and when you know you shouldn't would be orange oops we made the opposite direction we said the things that are orange are urgent people still did better with it and they still liked it better so very interesting lots of opportunity with user interface to take AI results even when they are not perfect and design fail soft interfaces that improve people's success okay so um I've done lots of different things with uh thinking about uh feedback um and it's really really I guess I will just um I don't know I guess I just want to point out that um okay I'm gonna there's a couple things that that well track point you guys might know is the pointy device in IBM's notebooks there are lenovos that I designed and what was interesting about that is that by matching it has a cognitive model of of eye movement and handshake and by matching the abilities of humans we were able to get a 30 performance improvement over anybody's joysticks before so that was pretty impressive um but you know when we tried to make it learn the most interesting part about it was that when it was learning to see how fast you could move your finger the the hardest thing was not to have it learning about you and you learn you learning about it so if it changes and you change those if they get out of phase you end up with something that's that's terrible and one thing that we found out in the in the uh 1980s was before the 1980s all phones had what's called side tone side tone is that when you're talking it puts some of your voice in the earpiece so you can hear that you're talking well for a while cell phones took that away and people get stressed when they don't have side tone I believe people get stressed when they don't have side tone and other in other regards too I'm relying on you looking at me intently saying oh my God I'm not I'm not completely off base in my in my way of communicating I'm not wearing a frog on my head or whatever but we do find ourselves looking at ourselves in zoom and staring at our face why because we want to we want to know we're we're actually succeeding at especially with zoom who knows you might disappear um at communicating with others um the other thing I want to say is that constructive feedback is a very dangerous thing and and um I guess I will leave you with um when we were making a um a car that was trying to improve your driving by giving you feedback we had two two knobs on the on the dashboard one for affirmations and one for criticisms and when people uh when we gave people any criticisms at all they made more errors driving any criticisms and it was funny as they didn't recognize it yes I just was wondering Ted I you know I don't want to interrupt too much but I see here for for both emacs and Gmail text completion spell correction in some cases yes which is in many which is in many cases for the person typing of course yeah that's right so so what's interesting here is a guy named George heidorn in graduate school came up with this whole legibility Improvement for writing and he turned it loose on some you know you know Young Writers not not you you're a very professional writer a lot of professional writers hate these these things but in fact they would have in the center of a huge industry grammarly and so on um and and so you know now we have even things like Gmail and um and an emacs that do text completion um thank you for the invitation right it's going to add those those words after it um I I uh another another system that that we all watched was something that would dance on the screen would have thought you needed help uh I didn't uh people didn't like clippy yeah exactly and the thing with Bob or it could be as it became known later was in my mind it popped up one of the worst things for an eye is something popping up over there because it makes your eye drag over there it's a physiological response second thing is that it would talk about why it wouldn't talk about how and so it would tell you what you know this this is used for this purpose because it didn't know quite how to do that and the third thing about it is that it would take your eye away from your from whatever text that you were editing which again was a distraction disruption so I I you know I kind of was bragging at the time the same time coach came out in what was called smart guides unfortunately the uh clippy got lots more attention because well they sell they sold you know maybe you know orders of magnitude more Microsoft uh product than than we sold uh os2 but when when what I was another thing that I did with um coach I talked about a little earlier which was recognizing your level of expertise and experience is you see this this cover and you see through and you see this this magline glass that is where coaches literally um thinking about graphical syntax on a graphical user interface and helping you step by step through it with uh novice intermediate professional or expert um script and you can go off the script and it changes what it thinks you're going to do whatever it has to say how you see how barely there um maybe I'll make it bigger um it's going to put very close to where you're working so that that was kind of um what what it what it did and I'm still proud of it now if you reflect kind of takes takes uh human human communication uh to another level uh it uses these faces and icon iconic symbols um and short texts to to make you aware of how maybe somebody else will see you so if you said something like this is great it's going to have this smile and it's going to say positive okay so that's that's just reacting to your to your speaking and um so here we are where so much of this industry has gone about legibility I'm thinking about tone sentiment and semantics and we had that one saying this is great we have this is a perfect way to think about this we give a uh um a deferent that's that's uh you know we call that deferent it's a it's a thank you for this um this is a Hot Topic um is is a more neutral response so we're constantly actually on deciding whether and how to give you feedback and when we started building the system it was kind of thinking how do we get Beyond shame blame and complaint we started off by saying you know when somebody would say something like women are not inspiring it have this this Grimace it's a terrible thing to say and so then you know maybe it's too edgy um very very you know negative women are not intelligent again uh women are not are not haters it would be happy that it's very uh are complicated to understand through 12 rounds of design of thinking hard about what would be a reasonable feedback we came up with well first we came up with terrible ideas like we had this uh making women and and Men equal we thought we'd put them on a scale now we have their heads on a platter it's a very bad idea um and we've come to things that are much simpler so um men are Dumber is of course a sexist statement so we have this this image of women and men together being equal and the word inclusive uh you were too old for that we really loved the idea of of the sad part about being a kid is they don't give you the good tools tell you're 32 and then you're 53 they take them away again uh so there's there's a lot of ageism that goes on in this world and uh so um we kind of have this young woman and this old older person um uh holding hands so um you know are you too old for this job chooseability not age because these are a little commentary and they're pictures of the of the man the young the young woman and the old woman holding holding hands um am I making sense well you know um that's kind of a roundabout way maybe we should be um maybe we should be uh straighter with our text and just say is this clear um so that's that's those are some some examples of um of of of how how it responds and you know you can go um to alfieco.com and you can uh find out more about it you can also go to your um to um to the play play a store or the Apple Store and download it for your phone and play with it there um and you know what where this Alpha reflect comes to play is when you are posting or commenting on some uh piece of content or um sending in a text uh to somebody else in the app so um the last thing I'm going to talk about is how we did this with machine learning and ai's trajectory uh used to be knowledge based engineering um and we used to talk about knowledge being power but um you know it it it did lots of things in the 80s you know it drove factories made big complex things work you know configured uh our our two our three our our two configured computers um it did it had linguistic models and medical models and logistical models an example is um there was a system at SRI um that wooden figure um where to put things like porta potties and and you know temps and things in in a bunch of airplanes that are going to fly fly them into some remote area and um they had a simulation that took four or five hours on a big supercomputer to to configure these 22 airplanes I guess they call it I don't know Battalion or something and and um no one used it and people would just Shuffle paper around on a table as a group to decide where they're going to put things in the airplane until some friends of ours um made this rule system that said hey you know you put the heavy things under the under the wings and the and you put the things that are going to slide around where you can tie them down and and and you know if you're gonna if you're gonna let them come out of the airplane on parachutes you load them a little differently than if you're gonna have them come off on forklifts and and and guess what in one minute it's a long time ago uh it would configure this whole airplane all these airplanes and then then you would interact with it and move some things they didn't like where they were and then it would run another another iteration and immediately um most of the people doing that kind of work converted to using it so the user interface with the reasoning system made a system that worked with people and computers working together using AI reasoning and representation the trouble with that approach we ran into a psych psych was this is a system that's been going for I don't know 40 years at this point it they have ma a really great model of the Suez Canal in the Middle East um and they have lots and lots of uh other other things that they've modeled and the problem is that um it's done by hand and there's a problem with consistency and in fact even for small models there are trade-offs and when something is true and it gets very complicated to have something that's reliable and consistent and girdle's Theorem says that you can't really do it all so it it runs out of power it gets very complex and you know 40 years down the road Psych is not the dominant AI system we're using um so did multi-layers Network save us absolutely you know before we had multi-layer neural networks uh people thought that maybe we couldn't do a good enough job at learning with neural networks I actually did my undergraduate thesis and Master's thesis to show that I thought it would work but that's a long time ago so the thing that's interesting is what do you do to to decide how do you just throw a bunch of modeling you know this this machine learning at a problem you've got a big data set today that's what we all have and the fact is John lamping who worked with Greg Hinton and a bunch of other people making the AI systems at Google says you gotta eyeball it first right you have to have an intuition you have to understand the topic area you have to understand reasoning about it or you will not be able to design an AI system and people say oh no no no he does it really well well most the people that you have work on image understanding with machine learning did it at another company first did it at another University first so the the we really are at a point where we don't quite understand but we do know that representation matters it might not be encoded explicitly in our machine language machine learning things but um anyway that's my little rant about that but uh in Alpha reflect we've got you know some things to just sing with it surprise business so you know we we started off thinking oh sorry whenever a person says sorry we start off with a very simple model sorry it was was an apology and apologies are are no good so um sorry to bother you versus sorry for your loss so an apology is not as is not what that is that is empathy so we model it incorrectly for a while we we got confused between uh sex and sexist we uh you'd say woman and all all said with liquid was sexist uh we thought you know you'd say black and all of a sudden I wouldn't think it would it would be racist not racial so this becomes very interesting and if you look at this huge um array here forgive me for not changing views I just don't want my system to blow up again um what you see here is US thinking about how so many different ratings of these sentences that we put together as our training sets uh there were you know thousands of them now um uh related to these different um different aspects that we're trying to look at we're looking at racist and sexist and Aegis and and uh confident and and happy and sad and anyway there's a bunch of them and um and you know when you when you when you do stuff like that you start with training data you do supervised learning and then you can you know you can use adversarial networks too but um you know you gotta invalue evaluate the interpretations and the performance and rethink the calcification let's say we have built the AI system for Alfie four times at least that I that I that I can name literally changing and upgrading our technique each time we started with you know the the one shots and we'll get into it a little bit more but anyway after you're reassembling the results when you have this huge bunch of classifiers all over the floor then what do you use you use in the bad old days we used to use blackboards very proud of blackboards use them in lots of my AI systems that is that there's you know there's these uh these results and they're all vying for for which One Believes more that it's the answer and that that's a really great way to go except that it's kind of a a single uh it's making a single decision um and what we find is very complex backwards with different different parts to them and then we end up with Bayesian Nets where we can where that's where all of these all of these expert knowledge experts that come back with with machine learning results can can fight over which which state they are in a network and then what what the result should be well that's all fine and good but in fact um we can we can actually do do other things um and I'll talk a little bit about that so we started off with some direct matching sorry sorry I bothered you um you know was we found ourselves saying looking for the word sorry very not you know it worked for a great demo um and then we tried to use other people's uh models hug and face Spacey and so on um but they are they're trained for different reasons and different kinds of porpoises and different re you know points of view about what hate means and what all various things that they've trained for me um and so we found ourselves stumbling a little bit and then we tried to say okay we can use these beautiful little little pithy uh training data sets with one shot learning and it's hard to get Nuance what we found is multi-label learning is a much deeper way of looking at the relationships between the training sets as well as as uh reducing the performance problems that you get by using by Outsourcing to other people's systems so we found ourselves owning much more and more and more of the of the um of the um of the of the machine learning until now it's all all done in-house um and yeah so so the in the end uh you know we have to decide when we're going to respond to a person with considered systems do we know enough to comment um you know are we going to say something affirming ambiguous or negative as I said negatives a really dangerous thing to do it's very easy to want to do it in fact all of our our responses at the beginning were you know we're thinking we're very easily kind of you know giving people a when you say something sexist why not let a person know a sexist don't say it well actually I as you saw we turned it around and we try to say be inclusive right um it's timely and short and simple so when I made uh I taught a class I taught the AI class here once a long time ago and and I had a bunch of people make these help systems and it was great we had 60 of them and the biggest um value that I learned from that was it didn't matter how much AI was in it it mattered if it could respond fast enough for a person to use the information while they were thinking about it if they had to wait for it the system worked worse and so timely is important short is important uh and simple is important those three things lessons I keep learning over and over again uh I try to think about things not being distracting uh either in visual or auditory ways um and you know I think that a lot of uh these systems we build try to put words in people's mouths you know when it completes things for me it's nice saves me time but it's also not very polite for me to put words in your mouth and I don't know whether it's going to come back and bite us but it's something that I'm thinking about and really the thing that I like to talk about is assistive versus advisory agents so an advisory agent is one that that teaches you um and and gives you a suggestion an assistive agent does it for you self-driving car versus a car coach that teaches you to drive better um and um so you know um yeah so if you lose subtlety uh you know you can stop people's thinking if you push them too hard in any direction I can be just you know pushing people too hard or or or telling them something they should say can be disturbing and it makes them feel like they're losing agency we all spend a lot of Our Lives hoping that we have enough well personal power to to be respected by ourselves and others um and then there's this whole idea of side town so that is realizing that I am actually succeeding at communicating that those those are some of the reflective principles that I've come up with and anyway um it's 5 30 and I just wanted to end exactly on time and say that um uh that we've been building this uh social network uh learning um Learning Network to support women and it probably will care about other people but besides women at some point but uh also this idea of communication reflection I think is a broader topic than just uh what we're doing in the app and um anybody that's interested uh we I'd be delighted to talk to you about about ways that this stuff can go forward and be useful um as a technology uh so thank you very much um and I leave you with the four traits of of Alfie are optimism ambition bravery and kindness and uh that was another thing we thought of is that instead of just having um a simple uh yes no we wanted to give people a direction a vector about where what what it what what would it mean to to to give people feedback that was that was um not not just bad or good but took you in a different direction so thank you very much for attending and I opened any questions yes we do and if you hope question and you're on the net you just unmute yourself and ask the question identify who we want ah yes I'm just going to wait till after two questions any other questions out there I think nothing yet so so I have two one's a fairly short yes or no and the other one is slightly more detailed because so no no seriously I mean I looked right into Seattle talking about this um when you when you mention women here are you including trans women I'm included but we're all about inclusiveness okay now I mean there's women who do not include trans women as women so and there's and yeah I I I just want to I've had lots of people work with me that that are all every every shade of the of the gender yeah it's not a line it's it's a spectrum well my second one in terms of most of the examples I think you showed uh have a particular component of what we would call interest that a lot of people would have and when I started talking to other people a couple decades back about when Sergeant bomb is pushing expert systems yes he was uh fifth generation okay yes no no yeah too but I mean actually at the last open computer forum and stood up and he apologized that's very interesting but there's probably a question here yeah yeah you know some some degree of empathy yes for for the system of both machine and people for somebody who might be helped and I thought that the exam a good example that might need to be painted would be something like what we would call social worker so we had the EDD up in Sacramento and they have to make the decision how do they know how does a person know whether or not somebody is actually trying to gain the welfare system or not that you you hear this story and you know I get emails all the time I you know we're in fact you know I get the word sorry as an example no medical problem and um you know you're a patient or you're really part of EDD no not well I'm just talking about the emails you're getting right right EDD has this person or this problem that they have but they only have so much Personnel yet they have this flood of people especially during the pandemic and they have to make and you know that's at the state level what does EDD stand for Employment Development you know and of course at the national level at the beginning of the pandemic you know there was this dispersal of money and it's a fair number of people not not the majority certainly uh you know did multi-million dollar scams on on the Federal disimbursement of funds but at the same time at the at the lower level there were smaller people in in build real need and determining what what the availability the amount of funds to disperse how can you I mean do you have some things to think yeah there's all sorts of questions there so number one they're sensing you have to understand uh what what you know whether a person's being truthful or not and whether they are making a good argument or not and whether they are they actually their argument actually represents um a need that is real so those are three different things that we have to we have to measure um and uh we don't have the Personnel because we think we can do it a better simpler way that's fine so I don't think that that's actually I don't I don't actually it doesn't bother me that it takes a lot of people to figure out if people have needs I think it's worth worth people have needs but but in fact yes there are things we can do to automate things like that I think that the part you know so there's there's the parts about measuring it and sensing it and everything like that but the part that I'm really caring about mostly is by in my career is about the part that people aren't doing which is how do you how do you even communicate with a computer how do you communicate with another person using a computer to support you in every social situation we are in now there is also a phone often there's a phone that's listening for a question even uh do you know do you want to find out when the baseball game starts I mean you know then you just you know it just and then all of a sudden it'll pop up in the middle of your conversation and say you know you know when it's going to start um the question is whether that's appropriate or not and I think that we have spent way way way too little we've spent a lot of time thinking oh can we do a better job of sensing and detecting and being and finding people that are cheating and this and that and that yeah that's that's the field that has you know you know 10 million people working on it the the one about trying to make people feel more comfortable work together better uh that's not very big at all and so that's kind of what I'm standing up here saying I want computers to be um eugenic I want them to I mean to work better with people uh uh and and and still I will say you know with regard to your question which I see why you deserve an answer anyway um yeah I mean people you know the computers uh modeling of these of these conversations getting better and better and yet you see Facebook who who has to do this for a living or die um figure out whether people truth or not they they have you know an army of people and an army of bots and I think that this collaboration between computers and machines a people is the most is is the most fun thing we have we always have been playing with machines and collaborating with with each other using you know uh whether it's a saw that has two handles or whatever um we're collaborating with another person and and that will continue and I want to celebrate that I want to celebrate the integration of using computers for what they're good for and have them communicate with people in a way that makes people um succeed better and feel better about themselves and each other augmentation yeah no I'll call it learning okay so the augmentation you know is is Marvin Minsky one of my dearest dearest friends until he passed um um um you know was hoping for you know augmentation replacement I'm just hoping to have a good life and it's getting hard because we got super computers on our phone in our in our pockets that are distracting us enough that we are falling off the Taj Mahal when the when the phone knows exactly where we are there's no reason that phone couldn't have stopped us from falling off a dodgeball dinner any questions anybody online foreign basically a you know best selling author Julian got three started this company and she wrote a book called Alpha girls and it was about women that did really well and she when she goes around giving talks about it she started recognizing that there was an awful lot of people that didn't they were getting ahead because they had glass ceilings of various sorts and so she thought you know wouldn't it be cool to make an app that would support women advancing and what she did because it's her wheelhouse is she got some of the best writers in the world to write articles make video tape videos about like you know what's it mean to Pivot that's changing kind of job you're in what inspirational women you know that have done things gone to space gone down to the bottom of the sea whatever and she has articles about them articles by them um videos about them there's talks that happen several times a week that are that are supposed to be inspiring there's a game you can play that that kind of makes you think about a person that's inspiring there's there's uh social network you can create between people to talk about um let's talk about whatever you want to to you know and relative to advancing you're feeling better about about your your chances and working towards it there's even things to help you think about how to get to get better jobs so there's an awful lot it's it's you know thousands of of pieces of of of content in it and there's you know little little lessons um we might call them uh you know hacks um so it's it's quite it's quite a robust platform and I you know I welcome everybody to try it um I was just talking about one little corner of it that's got a lot of attention this is this way of of watching a person communicate and improving their communication and I'm talking about it because you know it's it's it's a very direct um experiential thing that is that works with whatever content you're you're working with so that's different from the typical stuff which we we have a lot of in in Alfie that is stuff that's you know appropriate for for the person that wants to learn about how how women can succeed and that's stuff is is and especially interesting to me is that I think that there's very little um well-written stuff about ageism and that's that's something we focused on that I think is is is interesting to to the world but it's also uh been been under under people don't know what to do with it um and so we're exploring that as well as uh other things about about um about about about these these issues so no thank you for that question uh uh and yes please you suddenly worked on this experience expertise recognition system that takes your experiences in terms you how expertise yeah and you know I give the app a bunch of my experiences is it going to like give me a plan give me an idea on how to fix it sure I mean it could give a per chart and it could give you again you know to show you how to do this and then that and the other thing no what it does is it uses what it what it watches you do by watching you it learns your expertise and experience yes you can fill out a profile but mostly our idea is that we watch what you're interested in how you interact with it what comments you make and then we suggest other content and there's a for me page which is a list of all the things we hope we've thought through well enough and reasoned well enough to know would be useful to you and so that that's that's the idea yes our activity while using this app okay so it's not it's not just collecting it's not just swiping up everything about you in the world you know it's not scraping scraping your Facebook I mean maybe that'd be interesting but it'd probably be intrusive so what we're doing is we're watching how you how do you interact with our app you know yeah is there any more okay any nobody's raised their hand out there they're already busy eating dinner or something but you know it's always amazing what you get yourself into who you or me but this is this is this interesting AI talk I've heard in a long time oh thank you so much the reason for it is that it's simple and practical and you know have a fairly high impact well thank you I think that it's practical and I actually believe there's some theoretical uh underpinnings that you know will seem easy to us afterwards but have been coming 50 or 60 or 80 years and and that that's the interesting thing is that this modeling the the people's response is is a worthy task in my view yeah and but I think I think that it points out something which I've become to believe that I'm not really willing to tell anybody I do and that is I'm not certain that all of our concern about privacy and information is in the long run a practical Choice ye