Why Most PMs Fail AI Vibe Coding Interviews (TRUST Framework Explained)

Key Takeaways

Good morning. If you just joined, agenda today is um we last time, if you attended the session, we build a back end or a full-fledged app in N8N where you can send a contract, it can pass the contract, and send you information back. The idea was that you are a compliance officer in a back office company, and you have to report all your leases, what is the data in those leases, to your IRFC reporting, which is a compliance requirement in United States, Canada, and other countries. Uh so, how can you build apps with back end and front end during live coding if somebody throws that at a live coding interview or gives you as an exercise, then you can see that uh or you can build that. So, that's that. Then, I think all of us can build the first version of these apps. Now, live coding is no more like the thing. Uh it was a really cool thing like last year this time. Now, I think my son coded an app. So, my son is like 9 year old. Uh when we told him that he's going to be 10 next year, he told me that uh uh oh, my childhood is already over. Uh it's still that memory stuck with me. So, his childhood is still there, and he's building live coding apps. So, I talked to most of people like in my community, which is at this point 10,000 people, and people reach out to me during interviews. And uh I've seen some patterns why people are failing in these live coding interviews or in these case studies. So, today my focus would be to build on top of that and show you how to succeed in these interviews or in your real products. And I kind of build as this framework so that you can remember these things. It's called trust framework. And uh I will talk about what trust framework is, and how can you use it to build application that actually stick with customers and also helps you differentiate yourself in interviews or in getting more money or getting more business if you follow the framework. Okay? And then I will give you some exercises or some things to go do. That's pretty much the agenda. So, let's get started. Last time I was here, I built this app. If you want to follow along, there is a video for it now and here is a lab link. The lab is pretty straightforward, simple. The idea was that, hey, how can you use Claude code to write the front end and Anit and to write the back end. And what we built was this we solved this problem of lease compliance reporting which is the requirement by international foreign reporting standards. And the number is 16 where they say that if you take anything on lease you have to report it with all the details. And I built this PRD live with all of you using Claude code and we gave you the prompts and this lab. End of that lab, what we got was that we had this whole multi-agent application working on Anit and and we could some webhook. We put it to production. We published it so anybody can call this if they know this URL or this address on this webhook. Then any app which knows this address can call this this back end function. I also talked about what is front end back end in that video. So, I would highly recommend if you miss last week or last Friday session, watch it. But for the time being today, imagine somebody gave you this problem to solve and they said, "Hey, can you build a compliance agent which allows me to upload a lease and get a reporting ready PDF that I can send to IRS IRS or whatever your reporting agency is. That's the problem. And we solved it and the app looked like this. This is the app I built with all of you. So, what the app does is you can go in and you can upload a a lease and it extract all the terms that that needs to be reported for this compliance use case. So, I upload my lease and I can see my lease. I have some terms and then I can extract key terms and I hit analyze contract. This request goes to my NA10. You can see in a second something will come up here. You see there is a job running right now and it will respond to it and when it is done you will see an output like this where you will get all the terms and then you will be able to get a PDF out by clicking this button. Same page. Somebody gave you a request. You put some prompts. You created this app front end. You can't create the back end during the interview, but if you'd got a whole study, you are showing all your strength like you can build whole production scale apps now. But what's the problems if we build this as is and try to score points or try to secure a job or try to put marks like this and try to get to customers. I talked to most like 40 50 people last week or last uh week just to review or review their stuff and if they didn't make it or didn't go through this interview and didn't made it. And the feedback was like and I asked them I nobody knows like why they failed. It's hard to know why you failed in an interview because nobody tells you. But I think when I spoke to them or when I spoke talked to people there are three main patterns that people forget especially during live coding or under pressure. One is they forget to clarify with the interviewer or with their customers or with their teams that what is the input output of which we try to solve already. But make sure you have what is the input and what is the output. And what's the data model? What is a data model? That means what we going to store from the user and what we going to store from product perspective. Spend 5 minutes if you got any an interview or if you got in a problem, spend 5 minutes understanding this. In our case, input is a lease. And output is a report structure or report template ready compliance for leases. Good? And data structure is hey, we're going to store the name of that particular lease, all the key terms we need in the database. And product wise, we want to see a lot of things that we will land in a second. Anything else that I need to land? What are the main problems with AI? So, let's say I upload now I upload a contract and I'm getting these key terms back. Key terms are what is the lease duration, what is the price I'm paying for this lease, what is the interest on lease I'm taking. All these terms you are getting back. Good? I showed you on the screen. What are the three problems that AI uh specifically to AI have when they're trying to adopt AI tools or AI agents. Anyone? One problem at least. Trust. Great. It's very hard to build trust. And here the trust manifest in the way that I give you a lead, you were telling me a price. How can I know that this price is the right price? And if it is wrong, who is liable? The person who is using the agent? You are not liable as a company. You will put a star star saying AI agents can make mistakes. Please check my work before reporting. Nobody will sign up, right? No company signs up today. ChatGPT doesn't sign up. Claude doesn't sign up. Co-work doesn't sign up and you won't sign up either if you build this agent. So, somebody put this star star. That means that this can be wrong. And if you but click the button which is export to PDF and report it to this agency, and if you are audited, whatever the consequences are, they are your company, yours. And if you are the product manager who accepted or the leader who took this tool or trusted this tool, now you have a big problem to solve. So, one problem is people can't trust AI agents. Second problem is Okay, one problem I got, trust. Anything else? Anybody just unblock You can unmute yourself and say it also. It's just cool to say things here. The transparency. Transparency. So, another thing is to build trans trust, you need transparency and by default they are not. They are input output machines. So, you have to build it in the experience, which is a way to get to trust. Okay. Anything else? Consistency. Consistency. Somebody said consistency. I will put it because I need to create a framework today, right? Which it has to have some t r u in it. So, let's >> [laughter] >> Let's call it uncertainty. The problem is that, hey, I can have this result today. If I go upload the same lease again, I might get very different results. Or I might get a very different outcome. How I handle the uncertainty in the output. Also, the quality is not guaranteed every time or for every lease. So, I have to handle that somehow or at least talk about it when I build products. What's the last problem? Security, data loss. I mean, those problems were always there, right? I'm looking for AI specific problems. I can say that I can agree to that like that problem exists. So, yes, but AI specific problems are I have AI is hard to trust because it's not rule-based. It's inconsistent. Speed is another problem. I will put speed as like a small problem. The main problem is that The grounded >> They need user data. They need user data to improve. And in your design or in your mocks or in your product, if you don't design the product with feedback, then you have a problem. Right? So, it needs consistent feedback to improve itself. Which earlier things did not need. It uh does have It's going to be uncertain and it's going to be very hard to trust if you leave it as is like you build apps like you think it's Salesforce that I go and do something and everything is going to be perfect. If there is a report, I can trust it. It doesn't. The report might be uncertain or different reports at different time if you generate them. And it expects you to work on it. Right? Consistently. And in your product design or in your mocks, if you have not solved for this, then you have a problem. Okay? So, how can you solve that? So, let's solve the first problem. So, what I did is I introduced this framework, and the framework talks about five things that you need to have, and then I will give you a cheat sheet in the end on how can you succeed in live codes because it's just a prompt. Uh but, let's just understand what's in the prompt first. So, when you try to solve these problems, you run into more problems. That's life. When you so want to solve one problem, you get into other problems. So, the first problem is Okay, I wanted to trust. And the second problem, once you start Okay, trust comes from transparency. Okay, great. Uh I got it. Then, if you introduce trust or if you want to target First, you want to find target or target the moment where people are going to lose trust in your app. So, you would need to have a discussion on that, or you need to have a prompt for that. Hey, this is my app. I want to know where I will lose trust. So, we need to target the moment of trust in our products or in our product design, or at least show that you have this design talent. But, okay, you got it. But, then can you show everything? That's results into second problem. So, this is one bucket. This is product sense, and this is execution. And people forget to land these important signals in interviews. You can build something. Building is easy because obviously you are not building the tools are building, and the tools have improved a lot. So, you give a prompt, and they give you something, and you feel like you solved the problem. But, you need to show me these two skills. If you want to show me trust, that can come by transparency. That can show be built by showing me show your work. But then what would be the second problem if you start doing all this then? Information overload. Great. Then what you have to do at the same point is reduce the cognitive overload of doing it. Right? So we have to reduce the cognitive overload. So I want you to trust while reducing the cognitive overload. Okay? So for that, what I did is I said, "Hey, you know what? Let's just show people." So let me show you what I did. For this, it was a simple prompt. I said, "Hey, find the moment of trust in the app and give me a few ideas where I can build trust while not overwhelming the customer." And obviously AI is good at it. Or this whole thing is good at it, but I gave some clues also. And then when generating report if you click on it, I introduce these 90%, 95%, 89% and all that. Will that work? And then I introduce this 65% which is where I'm not sure. And now the user can come in and do things with it with it. Will that work? Will that solve the first problem? No, yes, maybe. Maybe. Up to some extent. Yeah, only >> What would be the best? If we know the reason where I got it from here. I can also see like where I'm getting this information. So I am transparent, but I am putting it here and if you click only then I see it. Right. Reducing the load, cognitive load. Great. But there is one problem with this which AI improved for me. Can you guys guess what that was? Color code is perfect. Uh what happened is then I go went ahead and when I did it uh this is course error, so let's look at the real one here. So I took one more step and I said, "Hey, there is still a lot of cognitive overload and help me improve that." So then one thing and there may be five things you can do. I'm just trying to show you that if it allows me to move it here, just give me 1 second. Okay, you can't see this screen and I can't move this here, so there has to be some easy way out here. Okay, let me share the other screen and you will see it. That's the easy way out. So this is my cloud screen. So what I did is after this I said, "Okay, how can we improve one more thing which none of you are able to guess right now?" But this 98% doesn't make any sense, right? What is this 98% which I was showing you earlier? Instead of that, I just put the confidence as verified needs review. And you can review it. Good? Same page? Yeah. And you can put more, right? So now I have clause, I have verified, and my output is looking like this. So I am building trust on each side and I'm explaining what this is, what does clause means, what is AI assumed means, where calculated means. So these are calculated values, so don't worry about it. I'm not taking it from the clause, but verify these and if it is verified, don't worry about it. If it needs review, then let's talk about it. Good? People were happy. People said great Mahesh, now you're talking like a product manager and you can build stuff and you have a lot of good taste or product sense. Now I have to address the second problem, which is the execution piece of this in my app which is I have to handle the problems of uncertainty and feedback. Right? So I got I got I targeted the moment of trust. I want you to remember that that you need to remember to target the moment of trust. Plus, I need to reduce the cognitive overload. And then I have three execution things I need to do in my app. One is uncertainty or reduce uncertainty as a UX feature. So I have to figure out if it is going to be uncertain, then can I cache this? So I put a storage inside my app and now my storage figures out if I have got these terms once, the next time I'm going to go, I'm not going to go ask the AI or this whole entity, I will just use my cache. Good? So now I got the uncertainty. Then I have to handle the feedback. How can I get the feedback from the user? There's one problem that I still have that I am kind of dictating that here it is, you take it, you leave it, but I don't know when they are not going to leave it and they're going to write different things in their report and they are doing it by hand today, right? If you saw it, how can I improve that? The last bit. You can have a manual input or like uh user response there to tell what happened what went wrong and how would they actually solve this? Great. So, now if you look at this, there are more things in the app that I introduced, which is I can come and edit it. And every time I edit it, it goes and says it's edited. So, let's say it's July 2023, but actually it's July 2025. It didn't got it right. Even if it said verified, it was wrong. And when I say wrong, it goes like edited and it stores in my database. I use this data to train my model to see what went wrong, to evaluate it every day. And now I have a signal that I get it here. Also, if you look at the app, the app has more signals to it, which you need to put me which you need to make sure that they are there when you go and build these or even during live coding, you have to show these signals to your hiring managers or anybody who's going to do this for you is thumbs up, thumbs down signals that are there so that you can give me feedback when you upload when you upload and when I give you results, I show you that, "Hey, here is thumbs up, thumbs down." that I can also track. And then on each point when I talk to you during these interviews, you have to show me your build trust plus show me a good higher quality or signal continuous quality. So, make sure your product is signaling quality continuously. How are we doing that? First, we are extracting value, then we are showing debugging quality. We We a debug box, which I didn't show you, but I will show you in a second if you want to see it. Then I have thumbs up, thumbs down. Then when you go to my report page I show you edit. I show you it's verified. I show you it's from clause. But all these are hidden from you. If you click on it, I'm giving you color schemes and all. And it's all hidden from you. If you don't want it, you can just take it. Then you can edit it. That I'm using as a feedback to improve or to know whether it's working in production or not. And with all that I can build apps with this framework called trust. I gave you a prompt in the lab where you can actually take the prompt and say, "Hey, first just build the app." So, what are the three takeaways for all of you? First, make sure what is the input, output, and data model. Give that Give that in your first prompt as we did in the lab. And you all have the lab now, so all of you can try it. Make sure your input and output are clear. And your data model, so you can get some result out. Then the second prompt can be just simple. You can take my framework and you can just say, "Hey, here is a framework to build AI apps because AI app has these three problems. And tell me five things I should improve on this." And the AI will tell you five things. You pick out of those five three and then say, "Go implement it." And then go implement two more. And that's your interview or that's your first iteration of your mocks if you're building these products. And now you have covered for it, and you can continuously build this quality signals as you go through these things. So, remember there is a framework called trust now, and you have to build or target the moment of trust. You have to reduce cognitive overload. You need to make sure that uncertainty becomes a UX feature and not uh onboarding or trust barrier for your customers. You have to build feedback in your apps and you have to give me continuous signal qualities. That's the framework. That's all I had for today. I have few more things and then we can get into Q&A. But I'm trying to see how can I land things faster or give you crisp things as I get into May 2026. So, this is the trust framework that I talked about. So, we talked about quickly why it is not working. Main challenges are people don't understand. They straight to coding. Execute execution signals and focused to they don't add product sense. And then I introduced this framework where you can just copy-paste this in your prompt. That's what I did by the way on the app I was showing you on Cloud Code. I copy-pasted this whole table and said, "Hey, use this table and give me what are the moment of trust? How can I improve it without adding cognitive overload? What are the uncertainty and how can I fix it with the app? And then what is the feedback points and where should I take feedback? Store my feedback in Supabase. And then I did quality signals. So, I revised quickly what I did last time which is building an end-to-end app and putting it in production using N8N and Cloud Code. Today I landed few things that you can't or you can't miss during interviews or when I talk to you as a AI product manager. You have to talk about trust. You have to make sure that there is no uncertainty in your application because user is buying a product from you. They You can explain anything about AI and say that AI has these limitations, but it's your job to cover those limitations and offer them as features. And here are few things that you can go build or do. The lab links are already there. If you want to go and build these and are really serious about these, we are starting our cohort next cohort tomorrow. There are only few seats left. If you join in next couple of hours, we will continue to offer this promo code for our community, people who have joined this. There I talk about how to actually identify problems and put them in execution. We build your muscle for evaluations. You will get all these signals, but how to evaluate them and continuously improve your product. GTM and pricing. And also we build a lot of stuff with mentors like Pooja who is in this call. And uh the idea is that you will get like six to eight weeks of rigorous practice and applying what I just talked about. This is just one framework. We have five or six of these. And then actually do it with your projects, with your real-life examples. That actually builds the memory and the capability. And I figured out that that's the bare minimum is needed to succeed. So, if you are looking to succeed in the IPM roles or dominate this field by next year, I would highly recommend you to get started. And cohort nine will start tomorrow. If you want to stay connected or want to be invited to these sessions for future, uh just sign up on our Substack and we will continuously invite you to these sessions. That is all I had. Hopefully it was fun.