6 Easy Tips to Design an AWESOME REST API

Key Takeaways

today I'll share six easy tips for designing a great rest API even big companies Overlook these things making their apis really hard to use I have noticed that most content online focuses purely on the technical aspects of developing a rest API how to use fast API how to host an API in the cloud those kind of things we've also done videos about that but what does it actually take to design an API that's a pleasure to use and that integrates well with other system systems it's not that hard but what makes me sad is that even big companies with apis used by millions of users don't do the simple things that I'll talk about today make sure you stick around for the last tip it's a really nice thing that I wish more apis would do let's dive in the first step is to follow the standards one of the most common standards that you'll find is open API this is their website so this is basically an API description standard that you can then also use to generate documentation a website where you can test the API things like that this is also what we did for our platform learn till here's an example of what such a spec document looks like this is written in typescript because we use next GF for the front end of the platform but as you can see there's various information here like What's the title of the API what's the version description what the base server URL is where you can find the documentation the various paths that are part of the API such as/ quizzes to get the quizzes from the platform platform etc etc we have written this manually because currently nexs doesn't generate this automatically but some tools like Fast API can actually generate this spec for you automatically from endpoint names descriptions arguments and so on so I highly recommend you do this another standard that you should incorporate in your apis is a standard for authentication such as o off for example if you look at the stripe API you see that they use an API key which is a standard way of authenticating with an API that comes from off on top of this there are standards for API such as the restful standard which is one of the most common type of apis you will encounter rest is an acronym for representational State transfer in essence this means that a representation of a resource state is sent to an HTTP or htps endpoint for example if you want to create or update some documents in a database you send the new desired state to an HTTP endpoint and then you will typically also get back a representation of the updated or created object as a result now there are naming conventions for restful API you should follow see restful api. net this is what that website looks like and if you click on naming rest resources you can see what is sort of a standard way of approaching this for example if you have some sort of collection resource like users or products or orders or customers or you name it then you should name this as a plural so slash customers and not/ customer withs so take a look at this site because it's going to help you write more consistent apis also depending on what you want to do you should use a different HTTP verb to fetch a resource you use get if you want to create a resource you use post to delete a resource you use delete verb and finally when you want to update a resource you should use either put or patch if you want to completely replace a resource you use put if you just want to make a partial update you should use patch now to be honest most apis actually don't follow these guidelines precisely in particularly the ones for updating resources for example stripe uses post instead of put or patch for updating resources here's another example of an API this is from moneybird which is an accounting platform so this does update incorrectly so it has a patch but there is no put option which well honestly that's reasonable I don't think there are many apis that have both put and Patch to be honest here's yet another API which is the convertkit API so this is a tool for email marketing so this uses put for partial updates and it has no patch in other words make sure you read the documentation second tip is to be consistent make sure you use the same formats everywhere for things like dates ranges query parameters and so on make sure pation works the same everywhere now the stripe API that I mentioned before does a really good job at this there's an intro section to the API that explains basically all the standard common stuff that the API offers including for example pagination and how that works and they have various options for that if you're using PayPal instead of strip well they also have a getting started section with some common things like how to post request and what kind of responses you get but there's definitely room for improvement in the PayPal API for example here is the endpoint to get an order by ID and actually this is not very consistent because if you look what you get back is that you have payer which has a name and it has a payer ID but if you look at the endpoint for getting an invoice there's also a person that's supposed to pay the invoice and there there is no ID whatsoever and it's called not a payer but a recipient another example of inconsistency that I want to quickly show you is the transactions endpoints in the PayPal API so there is a transactions endpoint that allows to list transactions if you want a transaction with a particular ID then you have to pass transaction ID as a Qui query parameter versus if you look at an order for example well there we simply have a get request where we Supply the ID as part of the URI which is the standard way of doing this making sure your API is consistent is an ongoing job but it's definitely worth the effort to spend some time here because it makes your API so much easier to use other examples where you can make sure things are consistent is in naming conventions or error handling or versioning Etc the the third tip is to keep things simple for your end user because this also makes your API much easier to use an example is to make sure you have sensible defaults for Arguments for example if I go back to the moneybird API so here is the endpoint for updating a sales invoice so there you see that we have various default values for example there is a document style for this particular invoice if we don't provide it it uses a default there's also a workflow ID that's basically the payment flow that's associated with the invoice and there also if you don't provide a value you will get the workflow that's saved in the contact and if there is no workflow saved in context then it's going to use the administration's default workflow so the API makes an effort to make sure that default values actually make sense again let's take a look at the transaction search endpoint that I mentioned before so in order to search for transactions we have to supply a start date and an end date both are are required so that means if you want the transactions for the last 24 hours which probably is pretty common you also have to specify the current date and time which is completely unnecessary it would be much easier if the end date was simply by default the current date and time another way that you can make things simpler is by making sure that you're using clear and simple date and time formats if you look at the formats for end date and start date for example you see that we have to supply Supply the seconds why do we need to supply the seconds why can't seconds by default be zero it will be much easier and also it uses the internet date and time format and if I click on this then it goes to some really complicated ISO standard that's going to take me like half an hour to figure out how to actually represent a date and a time and use that in payal's API and if you go back to the PayPal API here you see that there is actually one example of how you're supposed to use this so you can sort of figure out the dates from here but as you can see you now even have to think about time zones and maybe there is a mismatch between the time zone on PayPal server and whatnot so this is not really easy to use keeping things simple by the way is really important in all aspects of software development also in terms of software design if you want to learn more about that I have a free guide for you you can get this at iron. gold/ design guides this contains the seven steps I take when I design new software hopefully it helps you avoid some of the mistakes that I made in the past iron. c/d designu the link is also in the description of this video the fourth tip is to write clear documentation make sure there are examples of API request ideally you do that for curl and for different programming languages with common HP libraries so PayPal has examples like this like I mentioned before stripe also does a pretty good job at this so there is a curl example you have various examples including for example how to do it in python which is pretty neat one thing that's really important in documentation is that you also explain what the various Fields actually mean so what stripe does is that per resource they have a page called the resource object that contains for each field a description of what it is and what it looks like including an example which is really great but definitely not all apis do this for example here is again the Mone bird API and this is API endpoint for uh purchase invoice that you can upload to the accounting system well it's completely unclear what some of these fields mean for example here's the documentation for creating a new purchase invoice well when I look at this I have no idea what underscore destroy is supposed to do or what exactly is the revenue invoice Boolean it's not explained at all in documentation what this actually is another example is this trading API from interactive brokers so this is very incomplete there's no explanation whatsoever what all of the field mean and that's makes the API pretty hard to use for example here we can query currency balances but then when you look at what you actually get as like a ton of fields here under let's say the US dollar currency but for a lot of things I have no idea what that means maybe I'm not a Trader so if you are like a professional Trader you will know all of these things but for example second key what does that actually mean it's no idea is that always going to be the same as the currency or is that sometimes different there's there's no explanation whatsoever another thing here is that it doesn't help that all of these field names are actually just words mashed together make sure you use snake case because this is going to make your field names much easier to read the fifth tip is to make sure your API is easy to navigate what do I mean by that for example if you have an order object from that object you should be able to get the customer if you have a customer you should be able to get the orders if you have an invoice you should be able to get to the payments if has been paid and so on and so on now if you look at stripe they do a pretty good job at this basically each object type in stripe has a reference to the things that are related to it for example here is a payment intent object that's a payment flow in stripe so you can see that there is for example an Associated customer or uh there is the latest charge so charges to a card are also directly accessible via this payment intent object and the other way around this is also the okay so if you look at the charge object for example you can get to the transaction that's associated with it or the invoice that this charge is associated with or the payment Intel so that makes the API really easy to navigate you can get from any place in the API if you have some object to anywhere else as counter example again here's PayPal so if you have a transaction which is what you can see here as an example then it's unclear what this transaction actually belongs to we see there is transaction info like the ID when it was last updated the amount etc etc you can see who paid it but there is no invoice ID there is no order ID so if you have a transaction ID then well you basically have to guess what invoice belongs with it and another issue with this API by the way is that if you have some sort of ID you have to make sure it's Unique for example here a transaction ID is not unique and that just makes it harder to use also there is a technical limitation that you may get a transaction ID for example from web hook or something but it may take up to 3 hours for a transaction to actually appear in the transaction search since you can't retrieve a transaction immediately by the ID so you may get an invoice and that could be marked as paid but then you'd have to wait for 3 hours until you can actually process it in your own accounting system because then you get the transaction ID and before that it's going to lead to some 404 not found page the six tip I have is to make sure integration is easy and one of the key ways that you can do this is by supporting storing extra data as part of the objects in your API now why do you want this it's nowadays quite common that you're using a variety of different apis for example we use money Birds we use stripe we use pipe drive we use convertkit we use kajabi for the online courses all these various apis are linked because a customer may buy something from us using r as a payment provider that gets booked in the accounting system they get automatic confirmation email so we have to add them to an email system they get added to the course platform so customers are in various places in the systems and we need to be able to connect them so if we have a stripe customer I want to be able to find back the contact in my accounting system and if I have an idea of a subscriber in my email marketing tool I want to make sure I can find them back in the course platform if they bought a course so we need to be able to link these various apis and a really nice way to achieve that is that we can store these IDs as custom data in our various API resources and this is not only useful for storing IDs to connect things there's also other information that you might want to store for example maybe you want to store how much time it took for your system to process an order so you can do analytics later on or store the payment processing fee with an invoice that has been paid so you can more easily book it later API solve this problem in different ways or they don't solve it at all moneybird has something called custom Fields so you have to predefine these custom Fields you can do that via the website and each custom field is going to have an ID it will be associated with a particular type of resource like a contact or an invoice etc etc and then later on when you create those things like for example here in the contact API then we have if we scroll down we have a custom field attribute object here so we can provide an ID and a value and then we can supply a value for the custom field then that's going to be stored inside the contact another way to solve this is by using a metadata object that's what stripe does and I actually kind of like that so each object each resource in the stripe API has a metadata dictionary this is a set of key value pairs that you can attach to an object and you can simply store any information there that you like you don't need to predefine custom Fields or anything like that you can just store stuff as Json data in the metadata dictionary it's really easy what's particularly interesting about the metadata dictionary is that there is some magic going on in the background so one of those things is that if you supply a Json object it doesn't just overwrite the previous metadata that was there it simply adds the key value pairs to existing metadata so it's a merge operation so if your customer has metadata I don't know money bird ID for example and then you do a uh post request in the case of stripe to update the customer with metadata order ID equals something else then the new metadata will contain both money bird ID and Order ID so it will keep the old data doesn't merge and similar magic happens when you want to delete a key value pair so in that case you just Supply the key but you set the value null as part of the Json data and then it's going to unset that value so it's really neat the way they solve this so what I did I wanted to build this in fast API and see if I could get this working and so I created this very simple example API this in the code examples get repository by the way the link is in the description of the video so I have a very basic API here built with fast API that just has one endpoint for users there is a get users there is post there is put or actually if I'm really precise this should probably be patch and there is also delete for deleting users and what I did here is that a user is very basic it has an ID it has a name it has a full name where you can add more Fields if you wanted to but I also added a custom data dictionary and what I did I created a separate F dbpi which contains database stuff so I'm using SQL Alchemy for this as an omm and that in turn relies on pantic so what I did is I created a base class that inherits from declarative base and I added custom data magic right here so that whenever I use this base and I inherit from it and uh that happens for example here so I have a user that's stored in the database that inherits from base so that gets this Behavior so one thing that it gets is that it has custom data which I store in the database as a string but then what I do is I have some helpful method methods here so I have a method for validating custom data so this will be run by pantic when you set the custom data if there is no custom data we simply return an empty object and if it's a dictionary we stringify the Json data and then that's what we return so that's the validation part and then I have two helpful methods for updating this particular resource so I have an update method that gets a dictionary containing key value pairs of the things that you want to update and then basically for each key value pair I call update item and pass the key in the value and this is also a custom method that I added to this base so in this case if there is no value we simply don't set it right I ignore it if the key is custom data which is our custom data object then I call update custom data which I'll talk about in a minute and if not I simply set the attribute value in the regular way then finally update custom data so update custom data first it does is that it takes a look at the current custom data that's stored in the object in the resource and then it creates a Json format of that so using json. loads so this parses the string that's internally stored to contain the custom data into Json object then I go through all the key value pairs in the new custom data that's being passed if a value is none we unset the value from the custom data else I overwrite the particular value so I keep the old values in the custom data and finally I convert the custom data Json object back to a string and store it in custom data so this is the stripe style magic that happens but built in fast API and then I also added another method dict that uh helps us to automatically parse Json data that's in custom data so that it's part of the dictionary so you can access it directly so that's how built this so DB user because that's a subass of Bas now has all of this behavior and gets custom data for free I can basically add this really easily now to all my resources in the API now I have a question for you I'm not entirely convinced that this is the easiest way to do this I did take a look at SQL Alchemy documentation and they do have some sort of custom type which to me sounds like something that could work in this way if custom data could be a custom type that has this Behavior of unsetting values that are null or merging Json data instead of overriding everything so I think that would be ideal but I found the documentation on how to actually do this pretty hard to grasp so I didn't get this to work so perhaps you know how to do this if you do let me know in the comments can we add a custom type that handles this I'd be very curious to know so those are the six tips I just want to give you a few other quick shots very quick tips one is use versioning put that in the URL for backwards compatibility api. whatever.com slv1 endpoint second make sure your error messages are clear use standard HTTP status code provide a response body that has additional information then protect your API make sure you have some sort of rate limiter make sure all your API requests are encrypted with htps validate inputs to prevent common attacks like SQL injection and xss if if using fast API with pantic you can easily add validation logic to the pantic object similar to what I showed you before with the custom data another tip is to add support for web hooks so applications can tie into when an event occurs in your system stripe also has a really nice solution for this where you can basically Supply an event ID and attach web hook to that so that it's called automatically when that event occurs and finally if you have the resources also provide an SDK next to the restful API that basically means it's an easy implementation for accessing your API in a particular programming language like python for example it's going to make it much easier if there is a dedicated package that handles things like authentication has some useful type annotations objects functions to make your life easier if you want to use the API Maya we haven't done all of these things for learn sales API we're a small team it's a work in progress I know it's a bad excuse learn Sil is a brand new platform that we're working on that helps you train your software development skills recently we added a ton of quizzes to the platform covering all sorts of topics ranging from python to data science machine learning software design architecture you name it and if you can't find a quiz on a topic you like learn ta can generate it for you on the Fly I'm really excited about it and we're continuously working on it and adding more cool stuff I'd love to hear what you think about it we have a really nice free plan as well check it out at learn.com so to design a great restful API follow standards be consistent keep it simple write clear documentation make it easy to navigate and support custom data for easier integration now I'd like to hear from you what do you think is the most important aspect of a great API share your thoughts in the comments what I haven't talked about in this video is how to write code that is nice and clean to learn more about that watch this video next thanks for watching and see you next time