Running Replicate models using Cloudflare Workers

Key Takeaways

hey party people I am Zeke from replicate and today I want to show you how to run replicate models using cloudflare workers so let's start with the demo here I've got a browser window open to a template repository I've got my terminal window open and I've got my timer open with one minute on it and my goal here is to go from nothing to having a working cloudflare workers app on my local machine and deployed to the cloud running replicate models so let's see if I can pull it off start the timer open up this read me grab this snippet paste it in my terminal run it go grab a replicate API token copy my token yes I want to use git for Version Control yes I want to deploy my application all right so here we go the cloud flare CLI is doing all this magic work for me of setting up this application deploying it setting up DNS come on DNS don't fail me now let's go into the demo directory run this Command put in the replicate API key and generate an image boom under one minute okay that worked so let's dig in and see what's going on here um first of all I didn't even explain what um what cloud flare is but now we can kind of get into that so um Cloud flare is a web platform serverless platform for building web app so Cloud flare does all kinds of things you've probably heard of cloud flare as maybe a DNS provider or um a service that helps you avoid um you know being uh dsed um to keep your website up and running but cloudflare actually has a whole Suite of um developer Services um which are akin to um some of the uh AWS services that we are all familiar with like ec2 and S3 um but cloudflare has sort of a more a developer friendly spin on a lot of these products so workers is basically just a serverless platform that lets you run server like functions using a node esque runtime or a python runtime um and then you can also just use it to host things like static web pages or cron jobs um so this this app also uses hono which is a uh web framework that's maintained by the cloudflare team that makes it really easy and sort of ergonomic to build applications using cloudflare workers that have both serverless components as well as uh front-end code as well um and then replicate if you're here you probably know what replicate is already but replicate is a platform for building and running machine learning models in the cloud um and flux Schell is the particular image generation model that we're using here in this demo so let's jump into the code a little bit and see what's going on so uh this is a very simple application there's only a few pieces of code that are are really um worth digging into so in the source directory we have this one typescript file and this is the ENT pry of our worker code so we initiate the hono uh Web Service Tool we create a hono app and we set up one route which is the serverless route for um accepting the prompt and send making an API request to replicate and then returning the the Json data that we got back from replicate so that the browser can render it so this is the the back end if you will the front end is just a regular old HTML file that uses a CDN hosted version of react and then the react app is also pretty straightforward it basically just renders a component with a form and when you fill out the form it calls that backend function and adds images to the the page so you can type in anything here like cats um smoking cigars and there you go you can see how fast it is a fuell is one of the fastest image generation models on replicate um you get subc response times here so there you have it quick demo how to uh run cloudflare workers to you to build um to build apps that use replicate one more thing I wanted to plug here we have an extended doc here on the website that shows you how to build an image cache using Cloud store workers so the idea here is um when you run a replicate model and it generates some kind of image response um if you're using the API those images only survive as a hosted URL for an hour and so if you want to keep them and use them in your application you got to store them somewhere so using cloudflare workers to sort of intercept those requests combined with a service like cloudflare R2 or Cloud flare images is a great option for um storing those uh model outputs from replicate so that they um are persistent and adorable so hopefully this is useful again here's the repo and thanks for watching have fun using Cloud flare and replicate bye