Model CI/CD Course: Webhooks

Key Takeaways

hello in this next section we'll be talking about web hooks but we're going to go over the fundamentals of web Hooks and what they are and build one from scratch to give you intuition about how they work and what they are so web hooks are very fundamental in software engineering they're everywhere if you Google web hook you'll see a lot of different references to web hooks um particularly you'll see web hooks used in many different Dev tools so you'll see things about GitHub integrating web hooks on GitHub zapier twio stripe srid so on and so forth web hooks are very common in a lot of developer tools and in a lot of infrastructure in other words web hooks are really common mechanisms that different infrastructure providers use to communicate with each other and it's like a flexible way for you to um package of communication between one tool like weights and biases and another tool so what is a web hook well if you Google it and try to read about it it can be kind of confusing so for example the Wikipedia page web hooks uh defines web hook as a web development it's a method of web development um basically where you have a web application with custom callbacks if you don't know what a callback is that's fine a callback is just a piece a function or some code that is going to be executed at a later time now even that I think is a little bit more complicated all of this terminology is more complicated than it needs to be for you to understand what a web Hook is and I think the best way to understand what web Hook is is to is to basically build one and we can do that with python and so that's what I'm going to we're going to do right now is let me open some code so we're going to so there's two components to web hook one is a web server that will receive the web hook and then the client that's going to send the web hook and so I have the code in this repo called WB modal web hook and let me go ahead and open uh the the code that I want to go over so the first thing we want to make is the web hook server and we're going to use fast API for this fast API is a very popular python library for creating web servers and basically what this is is um we do the usual fast API things is we set up the the server and this class is actually the data this is a pantic model is basically like a data class if you if you're not familiar with pantic um it's it's it's like a data class that just specifies what kind of inform information the the web server is going to receive from the client so it's going to receive some kind of uh package that has all of these fields an event type an event author an alias artifact version so on and so forth and I made um this particular kind of data for a very specific reason this is actually a mockup of what weights and biases sends to a web server we don't have to get into that now it doesn't really matter what this is to be honest with you this is just the data that this particular web server it expects and it's going to validate that it's in this in this schema this uh underscore this Dunder Str this is just customizing exactly how this information is printed out so if you don't know about Dunder Str Str this is just a special method you can override in Python that lets you control how something is printed out and in this cas we're just printing out all of these different variables in in a in a very particular way finally we have the web server the web server has different endpoints so this is the root endpoint here and this root endpoint is going to get two different is going to get the event which is this kind of this Json that has this schema and then it's going to get a token it's going to uh the token is is going to be pulled from the header and I'll show you that in a moment that's going to just the token is going to it's going to check that the token is the secret random token this is not secure by the way this is just me showing you um a very basic workflow and very basic web hook and then this is just a check to say hey if the token isn't correct just raise an exception and this is just you don't have to you know this is just a a little bit of a verbose exception but just want to set the stage for how you would do it properly and if the token is correct the web server is going to return this message that's all it's going to do and um so what what do we mean by call back so call back is just any function that you would execute um when receiving this event in this case we're just returning this string but you can imagine there could be another function like let's say pull model or something like that and then you would ex you could even you could execute that here that would be an example of a call back where this web server receives some information upon validating the information it decides to execute some code we don't have to get into we don't have to get into that cuz I don't want to confuse you I want to just show you the very basic of a web hook like what does it mean what does it what does it mean and so let's start this web application so I'm going to go ahead and open the terminal and I'm going to zoom in so you can see my screen this is a little bit of a spit split screen so hopefully you can see it let's just zoom in all the way make it really big let's make it really really big here okay let's make this side really really big as well well okay so the first thing we want to do is start that web server so I'm going to type in this command um let me just get back to it oops there you go so uorn is just the thing that runs web server and this is the name of this file is fast API server.py so this is fastapi server.py and then this is this is the app the name of the application is app that's that's why you have uicorn fast API server app and this starts the web server and you can see here on this side this is kind of serving this is the logs of the web server and what we want to do is we want to send a payload to the web server like some kind of request and so this file curl. sh has a minimal request basically what we can do is say uh we have this curl command let me just zoom in a little bit more so this curl command again this is en curl. sh um has the following headers headers has the authorization header you have to make sure that you put this be word in here otherwise it's not going to work um and then you want to have the content type is Json and we're going to send exactly that schema that we saw here in the EV the base model we have the event type the event author the Alias artifact version so on and so forth uh you can see that this is that's exactly the information that we're sending it that's that's what it expects and here is where it's going to check that this token is going to match the secret random token is actually is is going to match that's what it's checking okay and then um you don't have to mine this stuff up here this is just saying uh use Local Host unless I say otherwise and so if we go back here um we can just run that shell script and when I run that shell script you see message the event is processed successfully and you see these logs printed to these logs have been printed and these logs being printed in this way is just the result of this specific uh Dunder Str Str definition because this this this gets printed see where it says print event right here on this side um this print event is actually what's responsible for the event being printed like this and we'll come back to this this is how when we deploy this web server onto a more production grade application you're going to be looking for logs like this and this is actually uh this information is kind of the same things the same type of information that weights and biases will send you it doesn't make sense now like what this information means like what is it event type what's an alias what does that mean that will make sense I just wanted to ground you on like what a web server is and what you should do is like play with this you should say okay let me kind of uh what happens if I mess this up up like what if I take this out and then you know try to send a request like what happens and you get an error see like this unprocessable entity and it says hey you have a field required that you didn't send um something is wrong and we can we can put that back and try again and then you see now the now it's successful and now we have those logs printed again so this is key this is the basic kind of idea of what web hooks are just one application so on this side this is just me using curl sending some data to another application which in this case was this fast API server and what's going on is in production uh what you'll do is instead of using Curl we're going to be you we're going to use weights and biases to send this information to some application that could be GitHub that could be another machine learning uh uh infrastructure application we we're actually going to use something called modal just because modal is very lightweight and is accessible to everybody just to show you how you can wire up weights and biases to modal and kind of understand what is happening but this like a minimal example is really helpful for you to sort of understand what's going to what's happening when I show you web Hooks and weights and biases thank you