Transforming Images Using AI Without Leaving Your React App - React Summit 2023 - Luis Ball

Key Takeaways

[Music] all right so my name is uh Luis ball I'm a senior software engineer with imagix and today we're going to be talking about transforming images with AI without leaving a react app uh going to go because there's a lot of content so excuse me I'll try to slow down uh so first off little story time uh as developers we love the ship product we're laser focused on features uh customer experience and satisfaction as we should be that's our job but that makes it easy to forget about the parts that we don't code about things like images so you might find yourself in this situation where you shipped a viral product people love it but they're also dming you saying hey it's fantastic but you're shipping 100 megabytes of images uh let's not do that and you got to run run and fix it um so this is why it's important not to forget about images they are the connective tissue to your sites um images are the way the customers experience your product for the first time and they matter more than we tend to realize so what should we do about it we should adopt image best practices there's a bunch of them right you got souret sizes aspect ratio layout shifts lazy loading Etc and ensuring you hit all of them takes time so it requires you to do things like generate variants for all of your images uh how do we make this easier so we've gravitated to doing things like image transformation as a build step um but it's not great transforming at build time can take forever the feature set is more limited in terms of Transformations you can achieve um and you have to do some careful maintenance of your tooling to do it right uh you know long term the DX in short just isn't great so what else can we do we can do things like image transformation on the edge uh this is fast easy to integrate into any application it's always up to date you know you can spin up a new replicate m point pretty easily and it stays up to date for you but it comes with some overhead right you got to spin up new functions and every time you need a new transformation and you have to do things like maintaining and inv invalidating caches syncing the generated asset back to your bucket where your assets live um and so it can get a little complicated which is why people then gravitate to image cdns right with an image CDN you can take care of all that heavy lifting and then you can progressively adapt it using something like uh unpick image which will take care of hitting all these best practices for you it's fantastic it's it's zero config with most of the big cdns out there you just drop your image URL that lives in the CDN it knows what to do you just give it like a width and a height if you need it uh and that's pretty much it long story short you don't need to reinvent the wheel to achieve best practices so let's take a look at how you could actually use this in your applications we're going to have a contemple nexs app it's going to use an image optimization Library it'll use an image CDN to remove all that overhead and we'll L the CDN AI parameters to do something fun Transformations so first off we're going to remove backgrounds using AI we want to do some consistency uh enforcement on our site make sure that things match our designs so here we have our contal uh nextjs application it's got a hero image we want to move the remove the background all we need to do using n CDN is use its background removal parameter save and by appending that to the URL we'll have a removed background that was quick easy painless uh thanks to using unic image and a CDN let's go ahead and uh keep going uh we got uh some a facial cropping we want to do we want to make sure that our images are always focused on what's actually highly relevant and centered um so let's take a look how we can do that we've got an image here and we have somebody walking down the hallway but we can't really see this person's face and we'd like to so why don't we go ahead and crop it around this person's face the way we do that is We'll add some parameters to the URL we'll set fre crop We'll add some width and height dimensions and once we've done that now we also want to make sure that we use some facial recognition to automatically Center the subject space in the image easy and now uh last thing we want to do is do some upscaling this is my personal favorite it's like a magic trick you're going to add some definition to the pixel pixelated images you're going to increase the pixel density and you're going to do some things like you know enhance lightning and contrast so if we're thinking about our contal nextjs application we could have a image component right that all our images use uh but maybe we have a lot of user generated content it gets um a little hairy right not everybody has the latest and greatest stuff they has some more content we want to upscale it what we have to do in this case is add the upscale parameter save that and so now with the upscale equals true and our image URL our image gets automatically upscaled and that Carousel looks great um so just recapping what did we do we talked a lot about a lot of stuff uh we did a contemple xjs application we use an image optimization Library called unpick image and then we use an image CDN to remove all that nasty overhead of having to invalidate caches or store images in the cloud um and finally we leverage that CN to use some AI parameters um like facial recognition cropping like background removal like upscaling uh and make sure our images always look their best when they were being displayed um that's transforming images using AI without leaving your react up uh and I am sufficiently out of breath thank you all so much uh feel free to ask [Music] questions