GenAI with TensorRT LLM inference Engine: Simplifying Documentation Review on Atlassian Confluence

The application is powered by Nvidia's TensorRT-LLM inference engine, 13B llama-2 (Weight only quantization), llama index, LangChain, and Streamlit and runs locally on a Windows device with an RTX 4090 GPU. Motivation: Confluence by Atlassian improved project documentation, addressing the issue of timeline discrepancies and scope creep. However, the challenge of documentation overload remains, leaving new hires to navigate through extensive documentation, and risking burnout. Chloe is a new hire and has been invited to a project on Atlassian Confluence. She feels nearly overwhelmed by the lengthy documentation review required to contribute meaningfully to the project. Enter KaraamAI! KaraamAI will not only help Chloe to quickly get an in-depth gist of the project but will also facilitate new content creation by leveraging key ideas from the existing documentation in the team’s space. The content can be a long form article or a short PowerPoint presentation. Today, she is tasked with creating a blog and a short presentation on some new APIs. Luckily for Chloe, this API is well documented in the existing space. Chloe quickly launches the KaraamAI application and logs in with the required credentials. On successful login, all the articles in the team’s space are downloaded and indexed in a FAISS vector store. Next, she navigates to the content generation interface, enters the title of the article, and clicks submit. Once generation completes, she can edit, save the changes, and publish the refined version to the Confluence space. Similar steps can be followed for the short powerpoint presentation. Once the presentation is generated, she can download as powerpoint to her device and carryout modifications if necessary. Lastly, she can use the ‘Chat with Documentation’ functionality for interactive QA. Github: https://github.com/MustaphaU/karaamAI #nvidia #rtx4090 #llama2 #tensorrt-llm #genai #GTC24 #GenAIonRTX #DevContest