How to Use Azure NLP and Graph Databases for Intelligent Knowledge Mining

Key Takeaways

hello everyone uh Welcome to our guitar session those four who are new to this session let me give give you a brief introduction uh data session is a one hour dedicated session where you uh where you learn about the data data science and its related topics and Technologies so for this session we will be uh for this session our topic is intelligent knowledge mining with azure cognitive search graph database so uh for our session today's our speaker is uh Priyanka Shah and she'll be taking up the topic she is currently working as a group manager in avinade and with the experience of 10 plus years in analysis hope I didn't mispronounce it uh in design and developmental client server web-based entire applications she is an influential speaker of at Microsoft and other technical events for Microsoft Technology and Tech and ml.net conversational AI bot framework Azure cognitive services like speech to text space recognition and custom visuals so Priyanka I'll give this dish over to you and you can take over all right um hi guys hope you can see me and uh I'm just uh uh let me know if my screen sharing is okay [Music] are you able to see my screen all right so okay so let's start so hi hey everybody good evening thanks for joining and uh let us uh walk through uh you know an interesting topic here a topic here which is intelligent Enterprise content uh mining okay with the help of azure cognitive search and knowledge graphs all right uh so uh this is the agenda today we will we will look at why there is a need for enterprise-based Content intelligence content mining we will see what is knowledge mining we will look at various demos we will see how typically a knowledge uh mining architecture is implemented and how graph databases and uh you know uh intelligent indexing document indexing comes into play here and uh feel free to you know put up your question answers anytime you want to all right okay so uh just just trying to see okay I'm just trying to see how to sort of minimize this screen okay but let's continue right so um so uh sorry yeah so you know like today we have lot of knowledge around us right so I mean not knowledge we have a lot of data around us right so data in the form of uh your social media data in the form of your social media interactions data in the form of uh you know your Enterprise related stuff data in the form of your uh video audio interactions on your web pages right so that is just data right unless we mine that data we harness that data and convert it into some in insights the data is not really useful so there's a lot of data lying around there's a lot of data which has been so if you typically visit websites everywhere you have this accept cookie sort of thing right and then you have to accept all cookies which is trying to actually create a digital footprint and all of our activities are tracked so I mean have you given the thought why all this is tracked and how it helps uh typically media houses and social media or websites to build a profile for you and uh you know personalize the feed for you right so all of this data which is collected how do we harness that data right and not only data which is available publicly but data within your organization so here's a very realistic use case right so let's say like you know you wanted to uh check in your organization that what was a certain onboarding process correct so probably you know what is the way to reclaim back your uh employer Provident fund once you design from a company and then you might be directed to some person to the HR then HR department will guide you to the claims department the claims department might guide you to some other uh Department there's a lot of paperwork to be done so which is why uh organizations in probably a ticket ago or so had a concept of you know content management so there's a lot of different content which is siled in different departments right so they wanted to bring that con the content into a central uh uh Marketplace sort of a thing right so central location where everyone can search and then uh content would be made available to everyone okay so so far so good so then there were content Management Systems where their SharePoint came in uh right and after content management there came a problem of your um uh tracking the you know versions of that particular document like upgraded version uh modified version sort of thing so that's why we had a lot of good systems there for Knowledge Management kind of systems right but now the problem is we are above that so we are just above searching information uh via just you know pure keywords key phrases so now we are in an era or we are in uh probably uh sort of you know modern situation where users are expecting a lot from systems right so we are not just happy with with searching pure keywords key phrases and happy with the hit in the search result so let's say I'm searching for electric cars and then you know there are documents in my Enterprise talking about electric cars or or you know electric related stuff the keywords electric and cars and they give me those documents as my search results I'm no longer happy with that I want to know more I want to know that you know if electric and car that particular word was not present in my document but probably it was like you know energy friendly cars right or renewable energy car sort of thing or green cars environmentally environment friendly cards so I also want to dig out search results which are semantically similar in in in meaning not only just pure keyword key phrase based uh searches right again I also want to be reminded of recommended documents so you know people who search this also search this what you typically find in your Amazon eBay sort of uh settings right where you are purchasing a product and then you're immediately recommended that people who who purchase this also purchase this correct so this this sort of a thing if you want to do where it has word semantics where it has searched based on you know uh semantic similarity contextual similarity between different documents between the question which you are ask and now we have gone over and about that and I want my system to be able to understand the question which I am asking and if the answer is hidden somewhere in the documents which I have fed to my system it expects me to give that answer back from the document verbatim without me having to actually open the search results and see which you know document actually is the most fit to my query all right so which is why we are in the middle of a paradigm ship from just a search driven uh sort of a system to an intelligent Enterprise okay what I mean by intelligent is all these things I no more want just keyword key phrase driven searches I want semantic searches I want recommendations I got personalized searches okay so next time I hit the website last time what I searched and probably you know what I like to search those sort of uh the search results should be oriented uh based on my personality my choices okay my user profile I also want to have feedback based learning which means in the sense that you know if a particular Church result was consistently down voted a lot of times next time around the search results would be should be re-ranked right so this sort of a feedback May uh based mechanism is also necessary right so this is what we term as a paradigm shift so users are typically expecting a lot from your end systems and that is how Ai and knowledge graphs and machine learning data science all coming in together to give us this holistic intelligent Enterprise experience all right so this is how you know typically some of the uh the statistics like data explosion right so what is data explosion meaning just let me move this guy a bit okay all right so a data explosion meaning you know we are typically spending more time searching right or as even in an organization if I want to uh particularly particularly your search for some term which is related to some document so imagine in an oil and natural gas setting right I want to probably search for a particular equipment a part number or probably you know some oil field location the drilling conditions of an oil field geological conditions and stuff like that so if if we were to go with the previous you know keyword key phrase based search then a lot of time I have to just guess that okay the search term which I'm typing may not make a lot of sense while searching the documents right so I have to keep on guessing the correct search term and then trying to do a trial and error on the search result to get the most relevant search result correct so because of the amount of data now which we have which we typically have I cannot really afford to lose so so much time in doing this hit and trial method where I'm just you know punching in keywords and trying to see which is the result closest to my search right and I think all of us have been in in this scenario when you're trying to search in Google some technical error which you've got and then you know you're probably modifying the word or two here to uh sort of get more search results or to get more relevant search results you will try to put some a particular word into uh inverted commas as well so that you know that term is probably zero down or given more importance in your search so these kind of things but we can't really afford that in current scenario where the data explosion is exponential and then we typically don't have a lot of time to waste doing this sort of uh hit and miss search right so what are the current Enterprises Enterprise challenges with data so how do I enable Empower my workers to be a part of my intelligent Enterprise so one thing is to to build an intelligent Enterprise but then another thing is to actually Empower my people to sort of get used to it to derive the most potential out of it and uh to you know proliferate its usage across the organization right so and then I I want to have improved collaboration between workers as I said I want to have a personalized content and search right I want to have a consolidation of all the content platforms because in an organization Enterprise your data will be scattered like you will have videos audios or you will have images you will have unstructured data you will have structured data so bringing all that data into one platform linking it linking that data and making it searchable right and then of course having a beautiful UI to go with it in tutu as well right so UI cannot be cryptic it has to be simple and easy to use in YouTube and of course Empire your users to use it okay make it a a part of their daily routine all right so which is where you know knowledge finder comes into picture so knowledge finder is you take the data which you have you organize it you make it searchable and then you extract the knowledge out of it you extract insights out of it so that is not just you know down searching but the data which you the results which you find right they will give you actually some sort of insights some sort of uh knowledge which is why we call it as Enterprise knowledge mining intelligent knowledge mining right so what does a typical knowledge finder look like right so it has AI assisted search right it has summarization and preview so I'll show you what a summarization and preview mean right okay so summarization and preview typically without you know diving into the demo what it means is like you know probably when I search let's say let's go back to our uh you know uh experience of our search term of electric cars so when I'm searching let's say you know electric cars or let's let's say lithium batteries okay so lithium battery then probably it will give me a search results of documents which which has that term lithium battery or something semantically similar to lithium battery let's say ionic batteries now I don't have the time and the patience to actually go through each and every every search result link open it look at the document content and see you know if it is relevant to me or uh is this what I'm looking for so which in turn gives us a summarization and preview feature meaning the AI on its own like on on the go dynamically summarizes the document content on the Fly and then gives you a summarized view of the content right so even without opening the document going into the text and I can immediately have a preview a summary of what is there and then move on if it doesn't uh match my search query all right and again you know so not only just AI assistant search and summarization feedback and analytics most of the time you know you have these option of down voting certain things on you know if you see on certain Microsoft websites MSD and Ms learn uh so if you're searching for something and then you know some search page pops up and then they ask you that did it solve your question or or did you find what we're looking for and you can either upload it down vote it or have a Star based rating system right two star three star so this is called human in Loop right so you also typically systems have a human in Loop which will improve uh the um cognition of the AI and then you know in the long run after it has garnered sufficient feedback it will be able to re-rank the result or can give you better relevant results not only that not only Tech search it can also do video and image search okay it can also personalize the search so personalize the widgets personalize the UI based on your searches recommend you uh you know stuff articles or images or videos based on what you've viewed in previously or you know content which is related to your searches all right so this is uh you know also what typically are social media websites do so what you see on your YouTube YouTube videos your Instagram feed feed or your Facebook feed do you think that it is you know by a stroke of luck that you are seeing what you want to see no right I mean as soon as on Instagram you interact with some video some feed probably it could be let's say you know I interact a lot with the art and craft videos for my little one so immediately the next time I log into Instagram I am being showed shown similar videos right so how does it do that it depends on what sort of interaction you have with that particular field if it is a positive interaction then because you clicked on that link you opened it right you generated some sort of an interest by clicking on that link and which was recorded as a positive reinforcement and that is awarded like as a uh you know this is a reward given to the algorithm and then that is how it plays my profile of my likes and dislikes and uh personalizes the content so this is also what you know uh intelligent Enterprise content search would do it would organize the search results based on how my uh user profile based on you know semantically similar search results and stuff like that all right uh I mean we will we will also explore the AI assisted search a bit okay so typically what happens is you know imagine you have this whole Corpus of unstructured data line everywhere okay it is in the form of images it is in the form of uh audio it is in the form of video it is in the form of structured data relational databases unstructured data PDF files document files Excel csvs and you know think of all of the unstructured data you can think of right so you need to extract the content from all of that so imagine a PDF from a PDF you need to actually do optical character recognition so there are services available in the Azure uh AI service cognitive Services Suite which will enable you to extract the content out of the PDF so typically something like Azure form recognizer right which will which will enable you to extract your PDF content make sense of it right so whether it's a table whether you know uh it is uh um a dictionary key value pair of certain pre-defined things like for example invoices supplier name dates various date fields and stuff like that so you in to extract that content right you uh index it index it index it meaning so there's an interesting concept here right how do you make a document searchable so take a normal word doc right so take a normal word doc how do you make it searchable searchable meaning like you know let's say you have five to seven uh Word documents probably just a page each and I want to search a word uh probably somewhere from from those seven documents right so for a human it is like how do you do is like probably you will open each of the document you will do Ctrl F and you will search is there any word called as summary or summarization and your you know word search will probably match the word partially or fully okay okay and then show you the relevant matches all right how does it work in a uh AI system so an AI won't really open your documents and search the term right Search the word right so there is something called as indexing okay so indexing meaning um it is it it will take the content from the document the text content from the document ingest it into an uh you know inverted index engine okay so something like a leucine index so it will ingest that content itself into a sort of you know elastic engine okay it's it's not a database we will call it as probably a you know and a mechanism to make your content searchable so it will index into an elastic engine and then from there you can search the content so it will give you a lot of natural language processing things like uh partial uh Search terms okay or uh you know uh your group searches so for example if I'm searching for playing then it will match played play okay and uh so and so on and so forth but if I'm searching for something like uh uh Gone okay so it will also go vent these things also it will match so based on the NLP root stemming limitization Concepts right so these are all the advanced NLP Concepts we will surely not uh dive deep into those uh but yeah just to put things into perspective right just to put uh to drive the point hole that this is how AI makes a content searchable by indexing it into a engine and from there you can search it right so this is where it comes this this Concepts comes AI assisted search you are ingesting the content you are enriching it and you are exploring it okay so let me uh you know uh give you let me give you some interesting examples of how text analytics works right so let us look at some of the interesting you know concepts of what is enriching a data what is you know exploring the data and What are keywords and key phrases before we move on to some of the advanced concepts and we look at some interesting demos all right okay so uh let me let me quickly walk you through what are some of the text analytic facilities which are afforded To Us by modern Cloud systems all right okay so let us check uh something which is I'm so sorry it uh closes off so let us do um yeah let us look at the cognitive language Studio okay let me sign in and we will look at some of the interesting language uh text analytics functionalities okay Advanced NLP so from that you will come to know a bit about what are keywords key phrases and so on and so forth okay I need to sign in with my account just give me a second okay okay while it is opening oh it is open so um uh you know language studio is something where uh you know you can look at the language offerings which are so NLP is one of the backbones of making your Enterprise search engine right okay so let us look at you know some of the uh NLP facilities okay so what is extract key phrases right so I'm going to show you how AI actually extracts meaningful uh you know key keywords and key phrases from your text without even being trained to do so okay so these are out of the box NLP stuff available okay so let us for example look at one of the sentence here so I'm taking some of the existing content here right so which is saying Mateo Gomez a 28 year old man support a car accident driving near his home on Hollywood Boulevard and so on and so forth and was admitted to some Hospital in Los Angeles the patient showed signs of chest trauma indicating possible refractor and so on and so forth okay so these are some of the sentence which is detailing about some accident which took please send some of the medical reports of that patient okay so what were the procedures performed and patient was in ICU and uh the patient was discharged under the supervision of his caretaker now what I'm going to do is I am going to run a service to extract keywords and key phrases all right so I'm just going to run this and then I as you will see it is able to directly you know without me writing a single piece of code the AI the NLP inbuilt out of the box in Azure language studio is able to extract certain meaningful keywords and key phrases from the document itself all right so for example in this above paragraph as a human user right without without AI as a human user what my brain immediately registers is the name of the person his age his uh you know the date on which is of an accident and so on and so forth correct so you look at the AI generated keywords key phrases it is you know all almost doing the same thing it has registered Mateo Gomez okay it has registered that he's a 28 year old man then it has given that he was in a car accident it has identified that as a key phrase and Hollywood and so on and so forth okay why keywords and key phrases are important keywords and key phrases become important because they will enable you to organize your content together okay so probably if I have a lot of keywords and key phrases in the document talking about all these medical things right like there was a chest x-ray taken accident related stuff so keywords key phrases enable content organization by organizing or classifying uh similar content or related content together all right okay so this is also one of the important uh aspects when I'm searching a document or searching of Text corpus all right okay another important content uh concept when I'm searching uh Text corpus is entities now what are entities okay so entities meaning like for example from a text letter CL example so from a text here let me let let us you know take probably some yeah simple text uh this is complex okay anyways these are an indentities okay let us do that okay uh let us take the same example which we had a person suffering an accident all right now let me run the named entities thing and from this text out of the box out of the box meaning using the inbuilt AI from the Azure AI Services what is being extracted is person okay so it has identified what material Gomez is a person age is a quantity then uh car accident is an event okay this this particular thing Hollywood Boulevard is an address uh this one August 17 is a date then some Hospital name is an organization uh this one Los Angeles is a location date and time percent type so these are entities all right these are entities which is which has been you know defined by the NLP of azure you can have your own entities defined for example if I am indexing or if I am you know making uh uh open source music demo in which I am making music file searchable then I can have my own entities like artists for example the music genre which is rock or pop or metallic rock and so on and so forth and the albums of the artist so these will be my entities so next time when I'm training my own AI algorithm for a music data set which which is one of the demos we will see shortly for uh uh content mining content search and recognition so let's say you know if I am searching for uh let's say Van Halen then Van Halen it will recognize as an artist okay Van Halen recognized as an art artist uh The Beatles it will recognize as a band okay uh then you know uh metallic rock or uh you know uh pop rocks or sorry metallic rock or hard rock it will uh it will identify as a ntp which is uh genre music genre okay so this is also the way in which you know it will extract and enable you to group your search results together so you remember like on your search Pages you have the left hand side menu so let say you go on your hotel booking websites or your flight booking websites right so on the left hand side you have a faceted search visited search meaning let's say I'm searching for uh some holiday resorts in in Philippines all right and then on the left hand side I'll I'll get an option of four star five star and within four star five star resorts I will have you know Garden facing a swimming pool with all the gym facilities one room two room residential Suite Executives meet Villa so on and so forth right so this left hand side which you see are the entities this is how your uh results are organized all right okay so a lot of powerful stuff which is afforded by you know some of the text analytics another interesting thing I wanted to show you was Dynamic summarization okay so um this yeah summarize documents all right okay so summarize documents as I said right on the fly it will generate a summary of the document okay so let us let us uh look into this document so as you can see it's a huge humongous text right and as a person who's typically busy we don't have a lot of time to you know look into all the details and uh capture the most important things essence of the text which is where the summarization service comes in handy so I just run and when I'm running this I will get a quick you know bird's eye view of what my text is trying to convey in probably three to four sentences right so that this executive summary is given and then this original text is given and what sentence makes the most uh relevance and sort of you know gives me a context of that entire paragraph is a year mark and that is given to me as the summary so as you can see most of that from all of the text there are three to four sentences which are given to me that will help me put this a content into perspective all right so this is one of the features of dynamic summarization how AI summarizes so comprehends your text and then gives you result back all right okay uh so with this you know few features being shared let us move back to our content search okay so when I'm talking of ingest and reach and explore is what I just showed you so I'm ingesting the content from the document I'm enriching it meaning I am extracting keywords key phrases entities relationships from those documents okay and then I am making it available for searching exploring okay and how we do that is by using natural language processing flows and then by using you know uh some of the AI algorithms out there based on lgbt3 birth and stuff okay so what typically happens like when you have this sort of an intelligent knowledge finder in your Enterprise uh what happens right so it uh helps you achieve 15 percent increase in the employee experience so quick searches semantic searches right and uh again you know a very reduced result set like when I'm searching something it typically doesn't give me hundreds of results so imagine this sort of a very powerful search engine how it would serve our uh you know Talent sources so Talent acquisition teams across ever various Enterprises if they could just put in few keywords and then the absolute perfect fit candidate resumes would pop out and they don't have to sift through to a barrage of applications then that would make their life so simple right so this is the uh you know upcoming areas in which content finder is being uh being implemented uh to give the user a seamless and intelligent experience right so of course if you get a very relevant and a result set then you would lose less time in sifting through the results and further refining them and then you are sure that you know whatever is popped out in the first top 10 are actually the good candidates in case of our resumes or sort of a engine right okay so let us look at a few of the demos and then you know we'll come back to a bit of technical details I don't want to bombard you with a lot of intricate the technicalities with Advanced NLP and stuff so let us look at some of the fun Parts before we go into the last uh you know to technically uh boring slides into the nitty gritties of how actually the system works so let's do the fun part first and then probably we keep a bit of the somber stuff and the last okay so uh I'm going to show you a demo of music as I said right so this is a demo which we have made for searching music right and then what uh let me go to semantic question and answer let me show you the difference between what is just search and what is antique search Okay so let's say you know I want to uh so let me uh just tell you the what we have done is we have ingested around 2000 Wikipedia music documents here open uh Source music documents which are talking about various artists their albums collaborations bands and stuff like that and from all these 2000 odd documents which we have extracted from Wikipedia we have defined our own entities like we have defined three entities one is artist one is genre one is an album and uh then you know uh we have we have put these documents into Azure cognitive search engine for uh searching all this and now imagine that I have 2 000 odd documents talking about possibly uh many artists and music and their uh you know albums and now I want to search something so these are all you know popular artists and music right so no names not something like which is obscure or unknown so let me search so for example who are the many verse of The Beatles okay right so now when I'm asking a question like this to probably you know Google right so what will Google give you which will give you a list of uh you know search results directly like uh uh document links and if there is an answer somewhere which is available it will give it to you much of the same way the search algorithm the Azure cognitive search with its AI engine what it is doing is it is going through all those two thousand note documents which we have put in the search engine and if there is an answer to the question which we have posed here it will give that answer back to us directly without me having to go into a list of search results and then it will give me this confidence with which you know I can I I can trust the search it has given okay so I I don't know like you know this answer was taken from which of the 2000 or documents and as a human probably I would have to go and look into each of those document but see with the AI it is so simple that you know and this is called a semantic search semantic search meaning whatever you are searching the question if it has a direct answer in the text which is in your search engine it will give you that search engine okay and it will give you the score compare that with your typical search so let me just go to normal search and normal search much oh come on normal search when you do clicked on the wrong link so normal search when you do here let us go back to the search so don't be search when I'm doing it will give me search results so let's say for example I am searching for when Helen okay and when I am searching for Van Halen it will give me all documents which has Van helden listed in it right so as you can see Van Halen all the exact keyword kit of match right okay so you can see that hidden panel and these are the entities okay so as I said right I had three and it is defined artist album and genre so I'm searching for Van Halen Van Halen I can see that there are so many genres available for me to uh refine my search so I want to say van hilden in grunge I can click on these uh you know left hand side filters okay I can also click on Artist Artist meaning all the documents which I have search results which I have given here what are the other artists listed apart from Van Halen Right so for example uh Alice Cooper Jimi Hendrix okay and if you look at album then it will be all the you know albums which are related to the search results which will show me Van Halen and as you can see you have you know probably uh you know you can have more than uh one page of results and then there will be paging available here another interesting thing to note here is that there is entity map now what does ntp map do NTT map will enable me to understand the graph relationships between my search results all right so let me uh here for example limit my search result only to artists okay artists meaning what are the artists that Van Halen had collaborated with all right so you can see here now from graph relationships I will be able to see all the first order second order and tertiary relationships I can control the levels here okay so let me put the level at only one so which means you know artists who are directly in a collaboration with Van Halen on one of his albums right so for example Van Halen and Elvis Presley Van Helen and Led Zeppelin so the direct relationship tells me that they were very closely related and you know probably collaborated in some way or the other in some type of music or the other if I drill down further you know like I want to check now what are the tertiary relationships which means if you can see Van Hill and Elvis Presley Elvis Presley John Lennon so there was you know at some point of time uh John Lennon had also collaborated with Elvis Presley on probably something different okay but I'm able to also see the tertiary relationships right so for example even here you can see uh Van Halen and then uh yeah uh David Bowie and uh probably fire and something like that okay right then I can increase the complexity max level three and then I will be able to see the tertiary relationships also on and so forth so that will help me understand you know how my results are connected and that was where graph learning comes into picture so so graph uh search okay so nowadays uh apart from just your normal typical Enterprise searches graph searches are the next big thing right so graph searches it will enable me to search and derive meaning out of my search results so probably I will be only interested in search results which are directly talking about Van Halen's immediate relationships with certain artists or genres or or or um uh albums okay so now I'll probably move on to genre I can I can select the facets for the graph search here right so now I am saying that I want to see all the facets yeah I mean sorry I want to see all the genres in which Van Halen was directly related so okay I just came to Queen let us go back to Van Halen and yeah okay so now I want to search uh what are the uh the type of music which is directly Associated to Van Halen and I'll be able to see immediately from my 2000 note documents what are the type of you know uh genres which van helden was immediately uh associated with right so progressive rock emo Christian rock country music hard rock Blue Rock so that enables me to understand more about my own data these sort of graph relationships so the things which you are seeing here in circles are called nodes which are entities and then these things are called as relationships okay so something like Van Halen performs in a genre okay Van Halen belongs to a particular album or is a part of an album okay so these These are relationships and these are notes and this is how graph databases also work in pandem with your typical knowledge search to enable you uh to be able to you know make sense of your search results okay so we talked about semantic question answer we talked about graph so what about something which I had asked you at the very first right so if I'm searching for electric cars but my documents don't have electric cars okay so probably you know just to put it into perspective what I'm uh telling you let us go back to your search results all right and let us try and search I'm just saying okay let us try and search uh I don't know maybe some Tom like uh okay uh Metro project just just just you know throwing something out of the blue here right and then if there is no document uh with you know this particular full phrase Metro project so it is it is it is matching certain keywords Metro and all but then if I wanted to actually see the uh you know Metro project sort of a thing I don't have anything in mind uh search results because I'm actually looking only for perfect matches perfect keywords key phrases all right but what if I wanted to have synonym search like I said you know electric cars but there is no electric car mentioned anywhere in my document but there is something called as um uh you know for example or probably I am searching for robotic cars and there is no word called as robotic cars in all my text documents so there is no keyword hit but there is a synonym it might say driverless cars okay it might say uh you know uh your smart cars okay something like that those are still robotic cars right or it could it could even link me to documents which are talking about uh you know Tesla or grab because they are the ones who are uh innovate uh you know revolutional revolutionizing the um driverless car space right smart cars and driverless cars robotic cars so what about synonym searches all right so as I said right not only keywords key phrases but also semantic uh word searches synonyms is something which is uh also you know big thing for intelligent um uh Enterprise searches okay so let me show you something else here so let me tell you about if I'm searching for Railway projects in Australia okay so now my documents don't have Australia made mentioned anywhere so this is you know probably a search based on one of one of uh you know another mock data for one of a client of arts and I'm searching for some you know Railway based projects which they have carried out in Australia now I am searching and just look at you know how smart these content search engines are so Railway search engine Railway projects in Australia and it has searched so there is nothing called as Railway directly you know keyword mentioned in any of the documents but Railway is synonymous to Metro right Metro or Subway or MRT in in Singapore we call it as mass rapid transport MRT in some places you call it Metro in some places you call it a Subway right so then it is able to understand this sort of a word linkage synonym search or word proximity search so Railway and Metro then Australia it is also you know synonymous to all of the Australian cities like you know Sydney or Gold Coast Melbourne or Brisbane or Queensland it is able to understand that when I'm saying rail projects in Australia I'm actually you know trying to search for Metro projects which are based in and around Australia and then it is able to search these kind of projects for me all right and then I can always you know uh rate my search so here I'm giving it a five star because yes that's what I was looking for and I can slice and dice the search and refine the search further so now I can search for uh you know for example project value say project value more than 10 million time frame is something less than last seven eight years and it will progressively you know refine my search result and highlight those uh aspects as well and this is what I call as feedback based learning so you can rate the search result okay so as soon as you know over a period of time as more and more users array to this search result for this type of a search query higher up or with more stars the engine learns right the engine learns to uh to uh re-rank and and show this result higher up consequently from consistently rating it lower next time around when I search for Railway projects in Australia this won't be the project or the document which will be shown to me higher up with my search results but uh probably you know or down uh lower it down in the uh search results all right okay and let's stop also of keywords key phrases right so when I'm searching for uh keywords meaning from All My Text corpus as you can see it has extracted keywords and uh which are you know which are Salient features of my text okay so for example spatial design CCTV project location 270 meter cable State Bridge Okay uh right even going back to the music demo if you click on one of the search results you will be able to see the metadata and on the left hand side you will be able to see the keywords key phrases extracted and you can search those keywords key phrases within your text right so here in the Rock postmark right okay so yeah and then uh you can have a lot of interesting stuff done here like you can download the document and you can you know uh have Maps like for example if there are uh entity relationships mapping your content like for example you know mapping your content uh uh as uh something like for example this is uh punk rock is a sort of entity or you know uh someone is playing punk rock that sort of a map view it will show to you in case you have that functionality enabled okay all right so a lot of you know Access Control based mechanisms lot of searching Advance searching semantic search lot of interesting stuff possible with with the intelligent content search and graphs okay and uh increasingly you know uh semantic search and graph relations and graph searches are are you know being uh taking the course front are in the Forefront for the entire content mining knowledge mining space right now okay so they both go hand in hand like graph machine learning or graph based searches knowledge graphs and uh you know your index like document search engines go hand in hand I neither can survive without each other because one gives you the text search like full text search and other gives you a narrowed down relationship between your uh documents between your uh you know um uh between between the knowledge that you have ingested all right okay uh so another you know full demo which I wanted to show was for again if you go to the language Studio itself I go back to my language Studio I wanted to show you custom named entity recognition Okay so uh here like the previous music demo I have indexed documents which are movie plots all right okay so which are movie plots here so following example you know I just showed you the music demo and I told you like uh for example how how does it know that Brit pop is a genre okay or that it knows that you know uh someone like a Led Zeppelin or Alice Cooper is an artist or something else is an album how does the search engine know this or how does the AI know this so This is how we do it so this is something called as you know uh how you define your entities so whatever text you have you need to annotate it okay so that we have a lot of data annotation tools so what I have done here is I have this text okay so these are movie plots which are taken again from Wikipedia all open source data there is nothing confidential here right so and from that I am going to so in a movie plot what are the possible entities right I will have a protagonist the central character an antagonist okay uh who is like the villain uh I will have some supporting actor I will have a movie setting I will have the type of movie it is like it is a drama comedy tragedy Thriller okay um uh horror and then I'll have the period period meaning you know the year in which that movie was set and stuff like that like Renaissance period or it was a World War period or it was you know uh the term Millennium period and so on and so forth right so I have I can if as you can see here I can add entities here so I'm let's say adding an entity and I am adding an entity say probably uh movie uh I don't know yeah director probably who is the director of the movie all right something like that okay and now if I'm adding this entity I can tag the text here and uh indicate What entity it is okay so for example Danny O'Neill is madly in love with this teacher so on and so forth so I can actually tag Danny O'Neill and say what a type of entity it is so I've tagged it as protagonist I have tagged uh his uh the other characters as protagonist I have tagged someone else as antagonist and so on and so forth right so you can you can uh tag here something like 30 year old man and you can indicate that is the age of the villain and so on and so forth okay and I can um you know so you can you probably I can go on to another uh file and show you so adhuri a man finds herself the target of federal agent so she self-helper it's okay so I am going to tag this person and say she is a protagonist okay and the strangers I'm going to tag this as the antagonist okay this include United States war on terror tsunami blah blah blah okay so uh yeah and then I can I can you know various contemporary events meaning you know this is in the millennial period okay so in the 2000s and stuff so I can you know you can I can put this tag here and then you I will say the it is a period okay this is the period of the movie setting correct and then this is how I tag text and then I am able to train okay so I click on training and then I uh it will you know whatever text I have tagged it will train on that and automatically it is trained now to predict or to uh to infer the entities in unseen text so this is how we are actually getting entities and you know recognized from the text so that is how by tagging an annotation now my AI knows that brick pop is a type of genre or you know uh Led Zeppelin and Bob Dylan are artists and and it makes the searches easier so these entities are then populated into the knowledge graph as well right so these are the entities which you see here the knowledge graph is actually showing you nothing but the entity so the entities which we are tagging here are durian and safe and so on and so forth are the things which end up getting showcased to you in your knowledge graph okay so uh now coming back to our uh content search so how it works in the background so background I told you right we are ingesting documents from a variety of sources uh we are enriching it enriching it meaning we are extracting these entities we are tagging the data we are extracting keywords key phrases we are enabling Dynamic summarization and finally we are exploring it okay so exploring it meaning how we have searched it on all of the uh demos which which we have seen now okay so this is a typical technical architecture how it will look like right so these are the different data sources video media uh document PPT PDF content databases that you are going to put it like pull it into Azure and then you are applying cognitive skills okay so NLP uh which is entity extraction OCR uh image recognition and then you will be indexing into the cognitive search and from there you will have a web interface or a chatbot interface to be able to mind the knowledge which you have harnessed from your data so this is data here it is converted into knowledge okay with the help of all these background infrastructure AI algorithms and stuff and then this is how the knowledge itself becomes searchable so this is your data converted by some processing into knowledge and this is mining the knowledge onto the UI all right okay so what are the use cases for knowledge finder okay so knowledge mining so you have infographic correct so visualization of contextualized information like we saw in the graph databases bundle content so that grouping your content together similar documents or similar categories together solving NLP questions and answers okay so for example if I'm trying to like I showed you right I am posing a question and it is able to find the answer in that entire Text corpus collaboration meaning you know people like-minded people uh tagging together another interesting thing for NLP right you see on stack Overflow whenever we are asking questions uh there are you know answer tags related to that question or forget your stack overview that is like a very technical example your Instagram post right or your Instagram posts your um Facebook feeds and all you have a lot of hashtags Twitter feeds right so your popular instagrammers and bloggers they always have associate their post with hashtags like hashtag Yolo or hashtag women in Ai and so on and so forth so normally when I tweet my technical posts I use uh MVP because I'm a Microsoft MVP for AI so I use a lot of MVP related Microsoft Azure AI related hashtags right so what does this hashtag do the hashtags are trying to identify so there are you know people who are not people there are accounts are watching that hashtags and you know uh uh screening uh content which is uh tanked with similar hashtags right so this is how a tweet was viral too it goes viral meaning you know probably uh it it is attending a lot and then some of the hashtags get uh attraction and then you know it is called their trended and so on and so forth right so social collaboration meaning with all of these uh tagging of your data and you know semantic similarity searches and all your collaboration increases all right and then best possible information within a blink of an eye right so as you can see I just typed a question and it was able to search a plethora of text documents next best action like what you have on your uh um content websites right or your marketplaces right people who bought this also bought this Auto crawling which is you know getting similar information so if I train my algorithm to probably get me weather information now similarly I can also train it for getting the sports or scores live sports scores right and then people send thick meaning uh I I can also find the best people related to a topic so let's say in my Enterprise I'm searching for uh the best you know um software Engineers for a particular project then getting and extracting out the correct people for the correct match for that particular such term is also something which you know possible with the help of uh knowledge finding okay knowledge Mining and then these are these are the solution companies like we are using AI Based Services form recognizer Azure functions cognitive skills and uh enriched uh synonyms like for example as you saw right Metro and Rail and Subway and Australia and all the Australian cities together so uh uh recognizing synonyms doing semantic word searches these kind of things or green energy is equivalent to redeemable energy robotic Parts is equivalent to driverless cars these kind of uh word semantics cosine word similarity Vector surges so so much so much more is possible and you know this is just the tip of the iceberg right whatever I Define to you we are going beyond that uh so beyond even you know just searching uh uh your static text now we can also search videos okay we can tag videos we can identify speakers in the videos you can do an image search so if I'm searching for let's say you know Van Helen with a guitar in my music demo it will all it will show me documents which are which are having Van Helen with the guitar so I'm not saying that we have implemented it in my demo here but just to you know uh put things into context for what knowledge Mining and all of these AI systems are able to achieve so this is one of the things like it will be able to also do the image searches like how you are typically doing your Google image search right you are searching for show me or daffodil how does a daffodil look like and then in the images you get the image of a daffodil similarly if I'm you know uh checking here for Taylor Swift with the guitar then it will give me documents which actually have the image of Taylor Swift playing a guitar right and which is I mean the power of you know modern day AI systems algorithms and cognitive search engines and also you know the power of advanced um language analytics with the help of open AI API gpt3 Bird right so a lot of lot of humongous interesting possibilities uh available for us to explore today and yeah so with that I come to the end of my sessions I'll be happy to take any questions before uh we proceed to answer your question I would like to uh request each one of you to please fill the feedback form it really help us a lot I'm launching that then cover uh answer only question uh it's enough q a section I see a lot of interesting questions here so um uh let me know you know uh analytics with Dell let me know when I'm okay to answer the questions because I see a lot of nice questions yes I'm yoga answer all right okay okay so I see a question from Suman tupal similar to transform Transformer attention based model absolutely so if you see right Transformer attention based model these are all what your birth and gpt3 and all Implement anyways so absolutely like you know attention to detail what is Transformer Transformer attention it is attention to detail in an image in a text and exactly these are the building blocks of your NLP correct so these are one of the models I will say which are used in your Enterprise systems but there are a lot more right because as I said the search engine is capable of searching images videos and a lot of other uh content okay so yeah one of the aspect is Transformer attention based model you're absolutely on spot uh I see a question from madhu can these past keywords transfer to a table spreadsheet I mean can it be classified properly absolutely is it a paid Service uh depends right so if you uh you know in Azure if you using this see if you are using out of the box service for keywords key phrases you need to have an Azure subscription right so I have an Enterprise subscription so I don't have to pay out of my pocket I have my own MVP subscription so yes these AI as a service which are given to you by Cloud platforms are paid services but otherwise there's nothing stopping you from building your own you know so there are a lot of libraries python libraries out there which are free of costs and which enable you to do these keyword key phrase extraction okay just that we try to reinvent the wheel right why to write a wordy Python program to do the same simple NLP thing which is available uh uh with a few you know bugs of paid cloud services and which are very accurate right I mean you don't have to keep on uh using all the pre-plane models out there and uh apply your python code on top of that so yeah if you want to use out of the box service of course Google and the AWS and uh Azure it is paid but I mean if you look at the charges they are ministerial right I mean it doesn't really if you want to do a POC or your some sort of your own learning I would say that it was worth the investment it's not much okay I the content language is different so again from madhu if the content language is different than English how this service will help in classifying the language basis or headers which are in English which can be no problem right so as I said there are for the cognitive search itself gives you also the option for detecting the language okay so if your language is uh is Dutch no problem so if you have a language detection service not service language detection feature ticked there in your Azure cognitive service it will enable you to detect the content so let's say it is probably French and then you could also give a translation so I also want to translate it uh uh um uh text translated into the destination language so your source is French I will choose that I also want language detection and then language translation so from French to English and I'm going to show the English translated text only in the search result possible this is done by using your detection and translation services okay so in current times do we already have some products that give our give out an Enterprise Knowledge Graph or it is mostly written as a custom project no you have Enterprise graph products which is neo4j for example the most popular graph database right what to insert into that graph depends on uh you know the content which you're harvesting become content which are ingesting mining right but there are Enterprise ready products if you want to have Knowledge Graph for example a health domain okay so I'm going to probably you know put the link into the chat window for you to explore more so I'm going to uh another uh website here okay so let us look at Azure websites assured yeah here Azure covet demo okay and let me okay let me search for ebook profane okay and craft relationships okay so for a medical domain uh information site like this it will have all the knowledge graph entities and relationships predefined okay so this sort of thing you can get uh out of the box as a graph right so they have I don't know you know what uh graph not the graph database they have used either it is Cosmos DB with Gremlin API or it is uh uh you know uh neo4j but nevertheless for predefined domains like these Health domains you might get a Enterprise ready graph already but otherwise uh um yeah otherwise you might able to yeah otherwise you might be able to use some of the content databases graph databases like neo4j okay yeah so as I said right the Enterprise uh databases you can note down for neo4j or then you can search these uh health related uh predefined ml models on which you have predefined entities and relationships and uh you know um data sets from which you will be able to get this sort of a graph information I see from we know do we need to tag manually all the time or there is some automation involved um yeah see when we want to train on custom entities right it goes without saying that you have to tag the data that you know after uh you know probably after a lot of uh times your model is trained then you might be able to do some sort of automated prediction so if I see a file my algorithm is smart enough to automatically build the ontology okay but otherwise yes if you want your algorithm to to be able to recognize custom entities you have to tag but then the data tagging is probably if I show you the language studio right you have to tag only 15 documents so if I have a laundry list of thousand documents I don't have to tag all the Thousand Ones I need to tag probably 15 or 20 enough for the machine learning algorithm to get a sense of the you know training data and then it will be able to do the predictions on the rest of the files on its own but it goes without saying for all the ml algorithms right beat image recognition object detection text analytics if you want to have some custom detection to be done you have to tag there has to be image labeling there has to be data labeling data tagging uh there's just no way out of that right which is why uh you have domain experts you have human in Loop mechanisms to refurbish your search results so yeah absolutely yeah I will share it later on uh because I need to remove certain you know I'm uh confidential references and uh I mean organization references from that monali Roy I see what is NLP based search engine optimization work on behalf of enterprise software development um I don't really understand the question I try I'll try to uh attempt to answer it right so uh okay so when you say that NLP based search engine optimization all of the things which you are doing with the Enterprise uh um things like you know the okay so so software development for Enterprise all right I can I can show you one thing but I don't think it is going to work you can look at Visual Studio GitHub co-pilot okay so what GitHub co-pilot does is as soon as you start writing your code it will give you the next sentence like when you are writing your Gmail or your messages right you are on team stack messages on teams your prompted auto replies Auto completes right same way when you are writing code your GitHub autopilot based on certain syntaxes and the way you are structuring your code it is able to generate code Snippets based on text comments and it is also able to like say probably you know uh when I'm writing a code and I have assignments right like uh probably uh um o dot employee name equals to x y z o dot salary equals to so and so and then o dot it will automatically complete that entire uh uh object population so your intellisense your co-pilot these are some of the enterprise software development uh mechanisms enabled by NLP search NLP engines okay uh how to create a new 4G graph okay so Indio 4G is a product right so if you go on the website of neo4j you will be able to see that you are they have their own language called as Cipher so using Cipher language you can Define nodes which are the central you know these These are the nodes here the circles which you see and you will be able to define the relationships edges vertices and you will be able to query that with the cipher language so just just go to Google check out neo4j and it is very easy to get started right so there are free versions and there are rapid Enterprise the licensed version so you can very well start with the free version anything that is in progress towards standardizing the tags say in a domain yes absolutely absolutely so for example I can give you a very nice Financial base example right so a lot of the other domains right your uh normal generic domain Your Food domain your health domain they have a lot of standardized text okay a standardized mechanism to do sentiment analysis to do uh you know keyword key phrases extraction but Financial domain is very tricky okay so in financial domain you will uh probably see that if I'm trying to uh do financial analysis based on the sentence alone it can be very misleading let's say for example um I have a sentence something like you know the Australian uh sorry the US Dollars skyrocketed as compared to the Indian rupee okay so which means the Indian rupee was probably uh let's say one US dollar was um 80 Indian rupees and now it's skyrocketed to probably 180 Indian rupees so which means it's skyrocketed so as a person who is monitoring this news as a part of India it was actually a bad sentiment it's a negative sentiment but normally what happens is sentiment analysis is based on certain keywords like you know for example increase skyrocketed and you know or or the losses increased so actually it's a bad thing in a finance domain right or or uh you know uh the uh something that this uh loss ratio tripled so only based on these things like ratio prepared and you know losses increased only based on interesting to try to do the sentiment analysis you will fail in a financial domain so which is you have certain you know standardized models to be used or so let's say finbot Okay fin but in that case uh for standardizing your tags in a financial domain and how to do the sentiment analysis in a financial domain you have different ontologies predefined for certain domains like ISO standards are their ISO standards so for an oil and natural gas domain you will have certain set of ontology predefined uh for its pipes and you know relationships between different equipments and nozzles and how they are to be organized uh the uh engineering diagrams and all of that so there are ISO standards one four two two four if I'm not wrong which has a set of the standardized Stacks to be used in case of certain government so yes obviously Health domain has its own Finance domain has its own utilities energy domain has its own so there are a lot of predefined ontologies taxonomies uh if if you really want to you know go that direction So currently we are working for a knowledge search Enterprise knowledge based uh search for an oil and gas and this is where you know we are coming across these kind of standardizing uh or taxonomy standardizing ontology for tableau are time series data how to implement semantic search what is your definition of semantic search omendo so semantic search normally comes in for unstructured data for tabular data okay or time series data it is more or less a structured data right why would you want to implement a semantic search for that semantic search is mostly in a text context okay so it is mostly mostly for unstructured data I will not limit it to unstructured data but mostly for unstructured data for a tabular data I might have a graph search of course I might have you know uh um uh what you can say relative relationship search within a graph search but not semantic search definitely not semantics but semantic search which is your question answer sort of a setting can be implemented only in a full text sort of a scenario right so only in your uh search engines like your cognitive search or elastic search so if you want to implement some sort of uh um what you can say similarity between different nodes you know how similar two nodes are to each other you can still do that in a graph setting a graph database setting all right so I don't see any more questions here if at all you can always reach out to me on my Twitter handle or on my LinkedIn and you can always drop me questions there glad to answer and yeah I think no more questions so yeah so I just see last question tabular date of product description and finding yeah yeah which is correct right so product description you can index the product description into cognitive search the tabular data the tabular data itself will be indexed into uh your graph database like okay so probably your nodes will be there in the graph database and the description will be there into your cognitive search and then you can still do the exact same thing for uh recommending you know a similar product so you need to have a link between your cognitive search and your graph database and you will be able to do this uh recommending similar products and your semantic search no problem all right okay uh I think uh all the questions are over time for wrap up yeah okay then thanks a lot on behalf of financial I would like to thank you for your time and for delivering such a wonderful session no problem okay and I am sure our audience found it very insightful and hopefully we can conduct more such sessions with you absolutely absolutely you anytime one more request to the attendees please fill the feedback form as it will be helpful to conduct more such sessions