Ibotta Personalized Rewards Innovation with Databricks and AWS

Key Takeaways

welcome to the datab bricks experience at the Grand Lux Cafe in the Venetian we're here at AWS reinvent 2024 and I'm joined by Jaylen stay principal data engineer at ibata welcome Jaylen thank you for having me so Jaylen by way of introduction can you tell us a little bit more about ibata and your role yeah uh so I'm a principal data engineer and I am uh the engineering lead for two data teams we have at ibata the first one is a data infrastructure and compliance team so that team is focused on uh the actual infrastructure underlying our data platform as well as the governance and compliance controls that we have in place uh the other team is the data enablement engineering team so uh they are focused on building tools that help our analytics and Engineering stakeholders interact with the DAT platform effectively we're here at AWS reinvent could you give me a sense of what data brecks and AWS mean to ibata everything that we do that has to do with data all goes through data brecks data brecks is our One-Stop shop for everything that has to do with analytics and business intelligence that kind of thing um the rest of our platform ibata like the service layer is also hosted exclusively in AWS so both of those platforms are extremely important to us as a company so Jaylen what would you say to other principal data Engineers or platform owners that they should think about to really take advantage of the data and AI landscape and the opportunities for their companies take some time to play around in the feature set um so I for me personally the the thing that I get most excited about with data bricks is that you get everything you need all in one place um so it's this huge Suite of cool and interesting and valuable tools um is it's a really broad portfolio and if you're coming from a less sophisticated platform um you may your inclination may be to do things the hard way um especially if you're migrating you you might uh at least my inclination is is to do things the hard way so with those investments in AWS and data breaks give me a sense of the scale of iata from maybe some of the technical capabilities but also just the services that you offer to the market um so we have around 50 million users total yeah so we've grown a lot over the last several years since I've been there and um we work with about 2700 Brands so ibata works with uh some of the biggest retailers that you've heard of here in the United States so that includes Walmart includes Dollar General um and a number of other uh partners that we have in our portfolio for the ibata performance Network um which is how we serve Loyalty Rewards to uh customers of those retailers um so some very big names in our portfolio and that means that we have a lot of Big Data coming into our platform as well everything that we do is driven by decisions that we derive from the analytics platforms um so what that means is H the scale of our data platform is pretty big so we have in and around 400 pipelines that run every day so thinking about those projects and the business challenges you get to solve what are some examples of maybe a workload that you have in production that's able to allow you to address those problems yeah absolutely um so some of our uh machine learning workloads that have are are really prominent in our portfolio are personalization capabilities so the ability to deliver the right offers to the Right audience at the right time um so this makes for a much better experience for our users and that they get to see something that is custom for them um and feels really Snappy and feels really relevant to them and the Machine learning models and other models that we produce Drive decision not only internally um about the direction of the company and and kind of the decisions that we make internally but also uh help enable our customers and our clients to make their own informed decisions right so whether that's uh we can see that this campaign is performing really well so we're going to add budget um or we can see that you know maybe we need to make adjustments to the targeting so that we can get this to the right people and and so give and give our uh users the best experience that's something something that is informed by the machine learning models and kind of Advanced Data products that we have available awesome what about on either maybe some of the governance side of the house or some of the more traditional analytics side how have you seen data breaks providing business impact uh Unity catalog is really unique um in that it's a One-Stop shop for not only managing the tables but also managing uh the governance and observing changes in uh the data platform across the entire thing in one cohesive place this is really really important for us as a team or set of teams that's trying to manage a multi-petabyte data Lake um there's a lot of things there's a lot there's a lot of moving Parts there's a lot of things to manage there's a lot of people involved and so being able to see all of those changes in a nice organized way uh that we can manage through an API or or infrastructures code um that makes life a lot better for us and I don't think that we would be able to do that without the capabilities data bricks provides for us sounds helpful and you've got thousands and thousands of pipelines yes so looking ahead 2025 is around the corner what's in iata's plans what are you excited about or what are you testing now to be ready for next year uh Honestly by favorite thing is just getting to work on really challenging problems with really smart people um so ibata is innovating really quickly and we are have been working on developing some really cutting edge features and functional ities to serve our customers better but also to serve our clients better um and we're doing that using the power of llms and Ai and machine learning all those kind of like Advanced Data handling capabilities we've invested really heavily in that and as somebody who specializes and has built my career around really cool data stuff this is very exciting for me um I get to participate in this like really exciting and Innovative projects that mean I get to have a really tangible and outsized impact on the direction of the company and the success of of what we're trying to build so that's pretty cool one of our biggest initiatives is we are trying to introduce to the industry a new way of measuring the impact and performance of an ad campaign um it's just really difficult to measure what kind of impact you're actually getting um so we think that we have a solution for that and it's data driven and it involves a lot of AI in Predictive Analytics and and all these really cool and interesting things that we're doing at data bricks well Jaylen thank you so much for sitting down and chatting with me today thank you [Music]