LlamaIndex Webinar: Community Project Showcase (07/07/2023)

Key Takeaways

we actually record this okay wait a few seconds I'll give an intro um cool awesome um hello everybody uh welcome to another session of the Llama index webinar series uh today we're really excited to feature some awesome projects built by members of the Llama index Community uh whether using llama index uh within their application or integrating with lava index and today you'll find a range of applications uh ranging from kind of the question answering chat Bots all the way to agents and all the way to text generation for a very specific use case and so we're very excited to feature uh car tech today we'll be talking about Albus Eduardo we'll be talking about express.ai and a talk we'll be talking about a new hackathon project that they built uh called immigrant first.ai and uh we'll first start with partec so um we'll probably do around 10 to 15 minutes for each presentation we'll leave some time at the end if there's any questions um and then you know if we finish early then then that's awesome but we'll try try to time box this within around like 50 to 55 minutes so without further Ado uh car tech take it away yeah thank you so much for that intro um just when you quickly share my screen can you all see my screen not good perfect okay so uh this is me Karthik I'm demoing Albus which is an EIT made for workplace search conversational bot right now inside slack and Chrome extension very soon Microsoft teams web Facebook workplace you know wherever work happens this is me uh you know I did my masters from Carnegie Merlin in machine learning back in 2012 2014 I run this company known as spring works we are remote bootstrap profitable 200 plus people we we basically build category to filing you know culture products across when you have clients across India Us and other countries and I've been doing this for a while so we are you know powered by openai and lgbt4 uh no surprises there we started on GPD 3 then moved to 3.5 and now on you know four and we use llama index that's the backbone of you know the entire application so the problem we are solving is especially in today's kind of workplace you have multiple tools you have a slack you have a notion you have your cheetah you have your Confluence you have your Google Docs then you have your CRM then you have your support software and stuff for that so there's a lot of information uh and very hard to kind of collaborate across all of these tools they don't really talk to each other there's no single you know Universal search uh you don't really have a knowledge based companies move very very quickly everyone starts up you know with some type of handbook then very you know hard to kind of keep it updated kind of it's becomes a living document and almost everyone has to spend a significant amount of time to update it most questions get asked and slack they don't get translated to your notion knowledge base or a Google Docs knowledge base so information is always you know it's kind of scattered across all of these places so what we have kind of done with Albus is uh pull in all these different you know information from all of these Integrations that's where llama Hub you know comes in you know we were looking at other you know options there are there's companies like merge and unified and AP API deck and whatnot but Lama Hub was this open source package you know which kind of did you know helped us kind of get to Market very quickly I think now it's around 40 50 you know connectors so we have everything which Lama Hub supports and plus you know some more so Google Drive and notion and you know jira and Wikipedia and website and whatnot so that's the first then you have your you know multilinguals obviously with gpt4 that is a big thing 3 3.5 did not really work hard for non-english languages especially for Chinese and Japanese and German and whatnot four works very well uh so we had we have lots of users from Asia then we've also you know obviously as the entire chain AI concept we have a chatbot which works inside slack in a Chrome extension as of now this is how it looks now help me with a few courses on udemy LinkedIn it can do typical AI stuff but can also you know get from your company making when we save you know HR time so I charge don't have to kind of spend time on questioning answering uh ticketing and there is little tools like gr service desk and all build for this but with generative AI you are actually able to not just taken a ticket but also answer and perform actions on it right and a bit of analytics and all of those things kind of you know come later uh just multiple teams across the company you know right now we're seeing mostly sales and marketing and customer support use it very well in terms of you know their handbooks everyone has a knowledge base like your customer facing knowledge base and sometimes the internal teams also need to be trained on that so that's where we see more use cases but over time I think uh I'm excited to stay around design when you have your multimodal uh you know image and things like that which come up at some point then code pretty limited use case for us people end up using copilot and you know other extensions on their you know Ides as of now all the typical Integrations uh you have your slang Google Drive notion all that stuff and how do we use it you know just you know a quick thing uh like there's a Code example you know this is your company Wiki example what is LSA it's a policy in our company so here you can say from your company making let's say stands for lifestyle spending account all employees are entitled to 2000 rupees per month uh it covers expenses as Internet expenses mobile you know prepaid post-paid food gym you know what not right things like that so you know you can kind of see it is coming in from your you know either AI or your company Wiki you can kind of toggle between the two to use the function routing which is this newest you know feature of 3.5 and 4 where you know we say you know categorize this type of query right is this your uh support question for Albus is it a company Wiki question or is it an AI question and then we kind of let it decide which one uh to choose with and then we answer right now obviously sometimes it gets it wrong and that's why we have a switch you know to AI or switch to you know Company Wiki this is the text tank at a very high level right Vector database we use Minecon we use elasticsearch or type sense then open AI Lam index that's really the backbone of everything right from your creating embeddings to chunking to you know creating the indexes to query you know do kind of even evaluating and stuff like that right we use re-ranking of cohere user API at the end but just before answer sending it to open AI to answer and all the passes connectors right so that's a very very high level uh you know kind of tech stack and I'll get into the demo let me just change my screen share to Slack can you see my Slack perfect so yeah so this is how it looks so this is a channel which we have set up you can obviously DM Albus as well you know any any types of questions but I will just show you in the channel s yeah so here you know Vegeta is asking what is my location budget uh now for this year so you can see it's coming up from your company Wiki it's giving you the answer as per policy and it's actually giving you the source where it's coming from right so this is a Google doc in this case right so in this case it's coming from two different sources so both of them are linked so sometimes if you want to get into more information and you have to uh you want to kind of uh understand or read more about the policy you can actually click on it and hyperlinks to that section as well right and for PDFs we give you the page numbers right so that's also alarm index in a feature uh so what does LSA that's an example I showed before um things like you know what if I get sick do I how do I apply for a sick leave you know without notice do I need to give a reason all of these things right so it kind of gives you the answer so you can see here yeah actually this links to slack you know or slack Channel which is our you know main General Channel and if I click on it I deep link to that exact winner message uh and it tells you you know the more information about the answer there so that's really powerful because sometimes you want to go and understand that we also have like a four downward so this becomes like a feedback loop so if I you know up for it it's kind of says you know I I was helpful if I downward um it will kind of ask me I could escalate to the admin I can even generate an AI answer so in we've seen you know Albus kind of answer seven out of 10 times as of now based on the data so uh in those three cases people actually end up you know escalating or creating a ticket for the admin to go answer and then what we do is we take that answer we index it and that becomes a you know way of training uh Albus to kind of answer that next time same thing you know we are you know we're doing an offsite in Jaipur in India so this you know I asked this question it said the location for the alcohol you know Rings last area up this is where it comes from right so exactly from that channel the exact location so this is the chat this is the post with you know the photos and all of that uh and you can kind of see sometimes it gets it wrong right so I said how do I reschedule my flights so here are the routing itself you know went to AI uh which is you know for some reason did not pick up from this channel even though this channel had that information so this is a case where you know Albus failed in in a way to you know give the right answer because I was looking for a you know answer from the company Wiki and not no from the internet right so this is just saying go to the headlines website you know go uh go to your manage booking stuff for that which is not really helpful because what I wanted is go to this Channel and somewhere it says how do I reschedule the flights uh because we have a travel agent to to do that right so that's that's on that's the case where Albus you know gets it wrong so in this case the routing itself was wrong but in some cases the answer does not come up so that's the reason why we've kind of expanded we started with the embeddings First with Pinecone uh like a vector DB search then we started doing um text search with uh type sense elasticsearch and then um you know this latest release from you know llama around Knowledge Graph is what we are working on now that's in our Dev so hopefully we will go to production soon so I think these three combined and then send it to go here for re-ranking I think our search results should be very good then things like radio upload the bill for the food reimbursement this is how you do it you know this is the handbook and you can actually click on it and stuff for that so you know that's how it works and we imagine some more you know features where you could you know now just do if you want to force you know sometimes skip the routing uh and you know that you want to get it from the wiki you can do this as well right what is the GST number which is like a unique number you know companies in India have or even you know let me see what is store address um for our USA office yeah so it got the answer no it's told you the source you can now then kind of Knob or downward right same thing here let's see sometimes it takes a minute uh or two yeah so did not get the answer even though so I'll show you this document but it's interesting I was not able to bring it up um it was not exactly listed the address but it wasn't on let me just show you so there are cases you know like here if you could see the address is there and it says USA but it does not say explicitly this is the USA office so somewhere the embedding or the re-ranking you know or open AI you know failed or the evaluation failed as well um something there's multiple layers of failure at this point uh so we just need to you know I think as we added more steps in the process of query uh we added more steps for failure as well so that's where I think you know there's some a lot of scope for improvement you know for us and obviously overall some of the AI space as well then other you know cool things is uh you know you can just some very long thread yeah you can kind of go in you can summarize this as well so you will see summarize it so all that you know all that typical you know stuff uh you know which kind of send it to open AI it will give you the answer uh as well here in some point then uh you know we have we have a whole you know dashboard around it on how do you kind of answer stuff like that one second let me just shift to my Chrome again now so you sign in with slack as of now over time will when I will have other logins as well this is how it looks so you know you can add in any classroom documents you can do Google Drive you know picks it up from your Google Drive all that stuff so I'll you know connected then you have your internet you have your gr you have your slack you have a Confluence notion and all that stuff then this is where you can see you know questions which have you know been escalated you could you know you can kind of actually see the answer and you should be able to go fix the answer as well yes that's pretty much from my side sweet um perfect thanks so much for the presentation um as there if there's any questions from the audience feel free to drop them in the chat uh in the meantime I I do have one question I think this will actually help clarify uh thanks for the audience as well um uh you mentioned you're using both backer search as well as like text search uh through stuff like typesense so could you maybe describe like the pros and cons of each and and how maybe you could use them together to actually help improve retrieval sure so I think Vector search works very well uh when you want to do um similar uh words because you know embeddings like uh embeddings book in a way where similar words have similar embeddings right so I think Vector search works very well for those things uh and you know speed is also fast uh in terms of you know querying from we use spine and corn there are other options as well but you know in cases sometimes you want the exact keyword as well like uh there was an example where someone wanted to search for the word figma you know in in the design right or in this case the word USA for the you know office address and um that's what I think Tech search generally you know sometimes does better because you don't want the you don't want similar words you want the exact word and we kind of you know do both now and then we do a combination with a re-rank with go years API and I think there are other options but go here is pretty good at this point to find so we do a top key on both say and then kind of send it uh Center on 30 35 then you ask it to give us you know three based on the you know chunk size then you send it to open AI over time you want to build a Knowledge Graph uh as in not overtime actually next week you know we'll do a Knowledge Graph as well where um like say the USA and India will be connected because they're both offices right in this case uh or your figma or like two projects will be connected you know in some way and we could use the knowledge graph query as well and then all three get combined with The Courier re-rank and then you know that gives us the final top three top four based on the context size so we are using GPT 4 8K you know so that's the limitation from a context size we tried 16 okay as well but not actually gave worse results even though it was higher contact size great thank you and then the last question I think from the audience uh Rohit asks can you please explain how Dr use cases integrate with uh wanna index yes so um if I remember I think um lamab has it but I'll have to double check but yeah so we built a jira connector so you can have you know do an auth connect to your jira instance and so things like what's the status on this ticket uh how many tickets were deployed last week how uh you know can you give me the acceptance criteria when on this ticket for Alba's knowledge graphs you know uh Improvement right start for that what's the latest on this ticket how many you know is there a pull request attached to so all those you know type of things as possible so this is just workplace search as of now and we are setting up the you know groundwork to you know go to actions right which is uh setting up Google Calendar meetings updating a ticket from new to done you know updating uh doing a PR review and put request review updating uh you know stage a deal in CRM to closed you know or lost right things like that so that's the next step right the first step is getting all the information in second step is now with you know open AI is function capability kind of letting us know what functions to call when and doing actually doing the actions right so it's no longer a just workplace search but also becoming your personal you know EA um across you know these different tools sweet great uh I think we're out of time there uh thanks uh for the awesome presentation and we'll transition over to edwarda for express.lia okay thanks um so that was a cool presentation so let me um get set up here real quick we need to share my screen and uh I'll do a little overview of um kind of uh Express Ai and and what we do and then move on to mainly a demo of uh what you can do with the tools so we do that so I only have uh an investor presentation so ignore that um but uh yeah well it'll give you the gist of it um so yeah so let me tell you guys a little bit about Express AI so um AI agents are an interesting uh thing and making demos are kind of really easy but doing them at scale is uh pretty hard and there's a lot of reasons for this and the AI agents that I'm talking about here um aren't always necessarily just conversational agents there can be many other things that you want to do but for example you want to eval things once you have the agent how do you evaluate it good I kind of learned recently that llama index has has some tools for this so I'm gonna spend some more time integrating those things we also have fine tuning so that's a kind of big question right now like how do you fine-tune your agents to their specific customers right the customers that are using that agent how do you fine-tune them that's going to be very interesting to look at in the future also rate limits it's kind of possible that you might run into certain customers that just use up all your tokens very quickly and that can cause blackouts for other customers if you're doing things um in a way where you know like right now people get around this by just asking everybody to give them their open API key right um and then there's also security policies right uh not every company is okay with sending their data to open AI particularly if they have like patents and things like that or um or if it's like a hospital and there's HIPAA compliance issues that they have to worry about um and then there's also Integrations and llama index can do a lot of Integrations on the data ingestion side but you also have uh action-side uh things with AI agents um and then finally how do you deploy it like a lot of people are kind of doing these demos on replit but replica isn't going to be like the end um kind of uh deployment that you have for your app and so what we're doing at Express AI is to enable developers to make amazing AI agents right we want to give them solutions to all of these problems in one place um and uh yeah we've been working on this for a while and I can show you um more or less what it looks like um but in general the the idea is that we have large language models and we want to give them to you in a way that is abstractable you can kind of choose which one you want to use based off of your needs right and by you I'm talking about like the end user customer um so um if that customer is a large Enterprise but they have a you know Azure subscription that they might not have some issues with using openai for example um but others might and they will you know want to use Google um or even for the the most stringent security policies that want to use like an open source model that can run locally like rwkv which my favorite or or recuna and then you have Vector databases so uh you know you can use pine cone you can use chroma DB or vecto if you need something that's local vecto just happens to be a vector database that Express AI did before and is available and then also agent Frameworks like Lang chain baby AGI and hugging face agents right these are kind of Frameworks for building agents and for me I don't think of these as the end goal right from from a useful agent perspective these are the building blocks that you use to create a real useful agent and then finally Integrations right so um you know you don't want to only kind of access slack messages or Gmail messages you want you might want to send Gmail messages or create a Google Document and these are the kind of Integrations that um that I'm talking about here um and so espresso has that platform so let me show you you know like a kind of demo of what it looks like so right now if you go to our platform there's so many things to be done but you can kind of create a project which will be like a space for your agent and in that project you'll have a Jupiter lab session that you can use to define tools and workflows in a drag and drop Banner so I'm going to open the session here and use the easy easy agent uh kind of toolkit that we we are currently working on so the drag and drop tool that we have here you can see has a lot of kind of Integrations already and what you can do is you take these um components and you can drag them onto the canvas and hook them up together and the way this uh is kind of working is in the style of Unreal Engine blueprints so if you're familiar with Unreal Engine this will look very familiar but if you're not uh Unreal Engine blueprints is a way that game developers and like game designers people who aren't really used to coding um add logic into their games and we use the same kind of style so there's triangles and that controls the flow of execution and then you hook things together it makes it really easy for kind of like end users to make custom pipelines that they want so for example here I'm at I you know added it to my Discord bot and now my Discord bot has the ability to use you know hugging face agents to scan business cards and put the information onto Google Drive um like like so uh and I think that it is a very interesting kind of future of uh AI agents not ones where you can only just talk to it but ones where it can actually do work so you can ask it to make a big report for you or scour the internet uh for things and I'll show that in a little bit um but yeah so Express AI itself we're focused on that privacy aspect right A lot of people um want to use openai right now but they can't because it's like you know Bard or banned in their uh company uh particularly in Japan this is a thing that happens a lot and then having the flexibility to deploy to cloud or local so what I'm going to show you is actually all running on my laptop with the exception of open AI which I'm using um and then also that learning part so this is the future uh but we're actively looking at ways we can uh gather the experience from the agent based off of whether it was successful or or not and fine-tuned the llm so that it makes less mistakes in the future and I think for customers like from an agent developer's perspective once your agent has learned from experience that customer is going to have a really hard time migrating away from that agent to another because they're going to lose all that experience um so yeah the rest is more like a kind of pitch kind of thing so I'll skip this and we'll maybe get to the demo but we've been getting a lot of um uh stars on GitHub recently for the xai GPT agent toolkit and if you are on GitHub and this looks cool please you know head over to there and give it a star let me know what you want to do um cool so anyway so we won't need to go into the the market although I do think it's kind of interesting to think of this uh in a interesting way that right now there's like a company like Fiverr where you hand off tasks to other people and but there's also zapier where you can automate tests but you have to like program it yourself and um what I'm trying to accomplish at Express AI is agents that kind of can do things automatically but you don't have to program them you just kind of describe what it is you want to do and so let me go over to our platform over here to show you uh what it's like and I can also show the um integration with llama index so to create let me create a new thing really here so we can just see it kind of working so uh as you can see this is essentially Jupiter lab right so anything that you could do in jupyter lab you can do here right so you can open up and edit python code you can also use notebooks but in circuits you have now with the circuits extension you now have the ability to like drag and drop things you can look at it here or you can just drag from a triangle into empty space and then you can search for a component so if I do like uh print I can print some sort of string just drag away from message and it'll ask me what kind of variable I want so if I do like you know quote hello world and submit that I'll get this hello world node and then if I connect this to the Finish uh I can now save and run this thing and then it's running over here and this output has the same um it's running as a Jupiter lab kernel so you have the ability to use um kind of the Jupiter lab kind of visualizations or tooling uh in here as well so now it's kind of generated this code and it's run it the interesting thing from my perspective there's two things one the compiling of this program so this one's called Untitled three I think you'll notice it's actually a python file right so you can take this python file and and you know the virtual environment and the XI components folder you move it to another server and you can run it so that allows you to kind of easily distribute this into another um kind of environment and that's kind of important for um the future of like the deployments and things like that and also let's say I don't like the way Print Works well I can right click on it and go to open script and I can edit the code for print right here so I can you know for example add these to make it a little bit more exciting right because this is a demo and then if I run it again you'll now see it says hello world with the exclamation points so when you're using circuits you're never limited by the library here right if you need something new uh you can you know right click on a node and edit it or um you know just add another component because the component if we open the script really quick is a class right that has a set of in-args and out cards and these in args and out args become nodes in the graph that you can connect to and then you just have an execute method that kind of does whatever it is you want to do to the out args from the in args Assumption so that's um you know like basic stuff uh what about more interesting things so one of the interesting things that we've been uh looking at recently is incorporating llama indexing so for example uh in here I have a interesting llama index uh kind of uh integration that I'm using which is using the Gmail reader so if I take the Gmail reader and then I look for something so I can say I don't know what I may look for Jerry you um I think that might work uh although I might out some other Jerry news I don't think I know any others um but anyway so I'll do that and then for each of those items I'm going to get the metadata and I'm going to print it right so it's a very simple um thing that I can just save and run here um and yeah then it's going to run it's gonna search my Gmail for that and load the data and you know you can see the Snippets of things right so we were recently looking at this webinar uh and we were kind of talking about the webinar so you can see that that actually works so um the interesting thing that uh I think this works is or what interesting thing about what you can do with this uh comes when you start adding the GPT agent toolkits utilizing actors and things like that so even though I do have the easy agent toolkit that I don't have installed here um I also have a you know GPT agent toolkit itself unlike the easy agent toolkit is a low level one where you can actually use it to create uh like actors so for example if we look into the GPT agent toolkit um there's an example of baby AGI and so this is uh essentially baby AGI coded completely in a graphical Manner and essentially instead of using Pinecone it uses a numpy memory that I added so that way you don't need a server for it and then you can see there's this cast executor agent I've added the ability to run tools so here has like a python executor and then the result of the executor agent goes to the Creator agent and then the Creator agent the task Creator agency that creates tasks of things to do and then the prioritizer agent kind of re-ranks those based off of um uh you know the priority of what it needs to do next we sleep for five seconds and then we you know loop again right so this is all just like a loop um that just happens over and over so um based off of this we can actually create anything so I have in my Discord uh over here a uh agent so let's go to I guess segment test so zybo is a Discord bot that I have on on my website um it starts with next IBO is like a sidekick in Japanese but anyway you can ask it to do things and the logic for zybo is like a stress test that I have for um uh for circuits so all the logic is contained in this circuits file and it kind of has some interesting capabilities like using llama index if it needs to so I have this email Search tool that I created so I can search my emails if I ask it to search my emails then I also have another tool I think up here called adro to spreadsheet so if it needs to add rows to specific spreadsheet it can do that um and uh yeah then I have the image generator that uses open AI instead of the default so that shows how to override the things uh yeah and then after that I'm you know I have the logic to kind of create the hugging face agent and Trigger um work on Discord messages so when people join or if I ask it to do something they'll do things uh so um it does take a little bit to run so I won't run any here but for example uh you know one of the things I like to use it for is to generate images but there's also ones like this where I can say you know in this image which I have to call document for for reasons um and I say like what is the name of the what is the name in this business card and it you know tells me the name so that's pretty cool I can also ask it you know extract the amount in Yen from this receipt and put it into the spreadsheet and it does it and it's pretty cool although my own logic is why this is currently status to cells but you know that's like simple things that can be fixed um uh with stuff um so yeah very cool uh stuff and that's Express AI uh happy to answer any questions if people have any um awesome thanks Edward uh this is a great presentation and and um the fact that I I think I personally found it super cool that you basically just uh created all like baby AGI or like an entire uh basically functioning agent uh within a graphical interface and so I think I think that's awesome and we have time for about one question um I think Shinju asks is some part of this similar to laneflow which is another uh UI based project for creating you know agents and and chains yeah I think um there's you know like there's uh when I started this there has always been like um kind of these drag and drop uis for uh doing uh AI um uh like like Azure ml studio is like the first one that I used in particular um so it's not a unique concept by any means um but what I think is unique about us is that one the flow of execution is explicitly defined by these triangles um and I think that was very successful in Unreal Engine because it means that there's no ambiguity on what order things are going to be executed in which turns out to be pretty important when you need it to do something right if you're just like cooking everything together to make a chat bot it really doesn't matter you know in the end what order everything was run in but in here the focus is more on doing things right so I don't really care about having an agent that I can use to look into like the HR documents and answer questions like in the previous demo that's a totally cool use case I don't see people using Express AI to do that when I see people using Express AI for is to you know whenever there's a customer support email for example uh if it's something that um I don't know let's say some sort of I.T thing where somebody's asking to create a new user then this agent would figure out how to do that and create the user that way you don't need to um you know uh you don't need to do that by hand for example right um so it's more like I want it to be something that's used for work for automations uh in a in a new way in this world of Agents um and then the other thing is how customizable it is because I can always right click and open the script and change anything I want I'm never limited by what is included out of the box so I think that's pretty unique and this is all open source so if you want to you know try it out just go to circuits.io and then you can you know pip install it and then have it you know Jupiter lab environment um the company itself is going to be mainly focused on you know making money from hosting these things rather than than the actual driving instructor super cool all right thanks Eduardo for your time uh and next we'll pass it to uh at all uh we'll be talking about uh immigrant first IAI and just for some context both Eduardo and Adolf were hacks on winners of uh hackathon that happened a few weeks ago uh the ABC hackathon by Norwest Venture Partners uh that llama index was also proud to sponsor um and so it's all take it away foreign thank you Jerry and uh beautiful presentations before I don't think I would be able to do as they did because we are just like more of a hackathon idea and in the early stages but today what I would like to show is uh what we presented in hackathon in terms of what things we uh why we started with immigrant first and also show you kind of like a demo of what already exists so that you have idea and then we can take questions um I'm hoping a lot of people on this call are probably immigrants as well and uh being an immigrant um is kind of tricky when you are away from your country and trying to build a life and then dealing with all the laws and I know Karthik is still in Bangalore so maybe he wants to come here and then we can help him immediate first is more of a mindset of looking at the world from a perspective of immigrant as the first priority rather than the third priority in the world especially when we are looking towards a global world and immigrants are and we are having technology going in every country so we what we are doing with immigrant first so that people can also actually uh navigate the world in easier and easier way so it is to help immigrants live their dreams uh they started as a you know as we shared the hackathon ideas jelly mentioned uh we had a small team uh it's just like a bunch of friends so I am a immigrant I have been in the U.S I live in San Francisco uh working product and uh I graduated with masters from UCSB and IIT kharagpur moved from India five plus years ago uh then my friend Arafat he is he worked for Shopify and uh built scalable products there and he's also from IIT kharagpur then Bhaskar who was there with me um on the hackathon day that day he worked he studied at triple it Delhi um computer science and he's also an immigrant and utkarsh who has experience of working with American Express and also from triple it Delhi so what we thought is like let's just uh let's just see how we can solve our problem and if we are able to solve our problem uh how we would be able to scale it so let's go down to what the problem is if you understand how immigration or any needs of humans work it goes into this mascular hierarchy of needs and the psychological needs kind of come as a basic need before you go to this realizing your true potential and when you are in a new country and if you are trying to create more and more value and live up to your potential if you are struggling with your basic need of would I be able to stay in this country or not would I need to move would I be able to do what I want to do you would always be challenged at the basic place and you won't be able to live up to your true potential and that's where I was realizing that by taking care of immigrants needs of being able to live uh safely is becoming more and more important so I'm calling this as an immediate disaster uh what So currently there is a worker visa H1B visa and if you lose your job in on H1B then you have just 60 days to find a job and a lot of tech employees are struggling to find job in 60 days because it requires a lot of processes for employer to go through as well and this is just such a short period of time and there is a lot of huge cost associated in doing an H-1B transfer as well the second is if you are a student in us and you wanna you wanna like stay here you have 12 to 6 36 months to actually figure out your next Visa otherwise you would have to leave the country and these all these students come on huge loans and o1 Visa which is more about how to help create for people to create companies um this is very hard to understand given the complex complexion of the law and how subjective this is so what is the situation here is employers don't want to hire immigrants and immigrants are having a hard time building their own companies here as well these are couple of tweets which we saw recently coming from Paul G and Sam Altman about one of the easiest policy wins I could imagine for the US is to reform High skill immigration um people want to be work in America this is a gift and embracing them is the key and uh Paul you also mentioned that American can pick any smart people anywhere because people want to come here now these tweets also tell you that how less control even these people have on the policy including us like we don't have control on policy they don't have control on policy but what we want to do is with the limitations that we know how can we help immigrants so that's where we were like what are the how generative way I can be helpful and what are the problems people are facing so eb1a is more of a green card Visa and o1 is more of an exceptional Visa uh these processes currently are very expensive so it costs around fifteen thousand dollars to get eb1 or one application file through a lawyer uh it takes a while to get the application filed so people spend need to spend a lot of time with the lawyer to actually write the whole documentation together and the third problem is people do not know that they can do some of these things self-filing like they can file on their own and USCIS has a has a good history about approving self-filing applications as well so as I was as I'm an Actron B and I was like struggling with this problem myself in San Francisco living in San Francisco I was like let me start solving it for myself and as I was solving it for myself I was slowly realizing how much value generative AI can actually create in this area and that's where we started considering like there is a Persona of a San Francisco techie who is currently on H1B and they're having a hard time being able to be at peace knowing that they might have to be leave the country in 60 days so how can we solve the problem for this thing so we divided our product into four categories first is uh the evaluation So based on your resume and Linkedin profile we can actually understand your probability of success for these visas using generative AI and then we will provide people with Direction and guidance on how they can actually improve their profile based on the resources available so that they can actually build a very strong argument and then we can generate an application based on carefully created prompts based on USCIS guidelines guidelines and whatever applications we already know from our database and then fine-tuning models based on users their own history and uh based on their experience and everything and then have the lawyer review the application so it is very important to know that we want to have lawyer actually review the application before it goes to USCIS because it's a pretty sophisticated issue so all the work if let's say 100 of the work was the total work that the lawyers and their team was doing right now we can automate up to 80 of the work which the paralegal was supporting while 20 of the review work can still be done with the lawyer uh so that's kind of main thing how we what we present in hackathon not talking about how we have been using llama index so what we shared is uh we can index the user's data document so that's what we are doing using the vector stored index and we are currently using Pinecone and also having the data in our local local systems so that's where lamb index can also be very useful um so we found height query transform much better than semantic search results so that's also we have been kind of exploring further these days and some of the new things such as what are the connectors available with the help of llama index like appify actors and all so we have been exploring uh so this is like more about how we have been trying to learn about we found a lot of features from uh lava index very useful for us already but we still in the early stages and trying to figure out what other features can be useful and presentations before this were also very useful for us to to learn what can be used now just to give you a demo of how it's working so this is this is kind of our current website this is just a simple website find your freedom and learn how to put it first immigrant first.ai and any immigrant can go here and kind of like this is like website and can get an eb1 as EB assessment from us for free so you upload your LinkedIn profile and email address and your resume and then we will have your profile getting evaluated in the back end so right now we are offering kind of two things here one is for immigrant attorneys who can be the one side which is like a SAS model so immediate attorneys will have access to our product and they would be able to 10x their business using our product uh basically the process is going to be they don't need to go on calls with candidates um they don't need to worry about long email conversations uh they will start with a pre-filled application and then they can take it to the finish line so this is like a SAS product for immigrant lawyers uh and the other product is uh just more of helping immigrants directly so they can do self-filing and other things so just working on getting their information and getting them to the next stage so just to give you uh some this is like how the product stays at this point so someone enters their email address LinkedIn profile and resume here and click on get get analysis button and then this is the initial evaluation for ev1a where it will categorize your resume or whatever information you provided into different categories and share about where you have some information and where you don't have information so like let's say evidence that you have been asked to judge the work of others either individually or on a panel so we retrieve that information about from your resume on what work that you did in the past that is actually going to be relevant in this category and where you can build a very strong argument with the help of documentation and all um so this is like more of USCIS categories and this is like something which lawyers will look at and understand if they would be able to position yourself you as an exceptional individual and all this information can be very useful to build arguments so as you see it is kind of like dividing everything into different categories what category which which part of my which part of my resume actually falls in uh so this is one part of what we provided as an evaluation the second part is letter of recommendation uh generation so USCIS needs a lot of letters from uh very high quality people who will say that you are exceptional and this requires a ton of uh analysis so what we did is uh based on the work that you submitted you would be able to generate uh recommendation letters here and also if the lawyer can edit these letters quickly here so we are working on some of those features as you can see here it's talking about a specific letter about my work as a placement coordinator at IIT kharagpur and some of the other work and this is little more from a professor's perspective at IIT then there could be something about my social initiatives here which letter can be written and it can be edited here um so this is all using generative Ai and we're using open AI gpt4 uh for this so I would say this is like the initial part of what we have been doing um and kind of like just talking to a lot of people about how this is going to help people so just focusing on helping immigrants and lawyers able to generate high quality petition for for their candidates and show that themselves as an exceptional individual uh that's pretty much I had uh for the presentation thank you sweet thanks at all um and a question from the audience is uh and this kind of reflects that question I was going to ask you is uh so does this application help to reduce the attorney's work in evaluating a candidate for o1 and other Visa types or more broadly speaking uh who is the end user of immigrant first IAI so we are defining it into two different businesses here and this is like still on the early stage on how it's the product is gonna emerge based on the market demand uh but the way I'm looking at right now is lawyers are getting on calls with candidates for 30 minutes they charge around 300 to 500 dollars and then they provide like okay you do have a profile or you don't have a profile can we file for you or not and that's a lot of money and only very few immigrants are able to even get that evaluation done so now on the back end we will have a ton of data about which petitions have been approved and what were kind of the exception qualities and how the art it can be crafted as story and lawyer can actually just look at our quick review and say that yeah it looks like I would be able to build a strong argument for this candidate so they don't have to do all the pre-evaluation and all and we are working with a few lawyers here and one of the lawyer for whom we did a few evaluations he suggested that it used to take her like four four and a half hours to actually do this process and with our product it only took her 30 minutes to do it and that is just based on what we did initially so now I think we we would be able to help her with over the whole four hours uh of the effort she did oh so you already tried this out with with that's not one I already trial of your product yeah yeah so yeah a lot of lawyers since I was working on my prediction I was connected with a lot of lawyers and a lot of immigrants to being an immigrant so I have been like trying to help people in real just to actually understand if our product is helping or not and one of the lawyer um she has given great feedback and have been able to use it awesome um last question uh and I think that uh we'll wrap up the session is what what tool was used to generate this document uh maybe the the letter uh based on for instance like the resident and also how do you avoid hallucination so a great question um I think it's very important to put a lot of guard rails on the prompts in GPT to actually ensure that it is only pulling information from what is already available in your in your documentation like initially as I shared this is just a resume some people are their resume it does not even have information about hackathons they have judged or the awards they have gone because this is like a more of a job resume so in those cases what we are building the chat feature to be able to collect more and more information from that uh from that applicant to understand what are the other activities they have done in their life so that we can have a wider context around each of these things if you see in my evaluation it clearly mentioned in a few categories you do not have any achievement in this area which is what we want we don't want to charge GPT to be just like saying that no no you can put something random here and make stuff so that is like how we are ensuring with the help of a lot of prompt engineering and ensuring there is enough context and also tell tell directly no if there is nothing you know so the uh yeah that's how we are ensuring sweet makes a lot of sense all right well to wrap up thank you it's all for your time and thank you card Tech and Eduardo for the earlier presentations um this is a blast and uh as mentioned we'll make this recording available on YouTube uh for your watch later if you missed uh any part of the original recording all right thanks everyone and have a great Friday