TensorFlow Recommenders Addons

Key Takeaways

hi there welcome back to our video series of building recommendation systems with sensorflow my name is way and I'm a developer Advocate at Google in this video we are going to talk about our community projects tfra tensorflow recommenders add-ons tensorflow recommenders add-ons is a collection of projects related to large-scale recommendation systems for example with model size exceeding one terabytes either leverages Dynamic embedding Technologies to make it easier to train evaluate and serve large-scale models for recommendation systems tfra has been successfully deployed into production in multiple companies with significant positive outcome for example vip.com an e-commerce company deployed the tfra in production and was able to see a 10 updates in customer converter tfra is fully maintained by the secret recommenders add-ons Community with support from the tensorflow team the special interest group currently has more than 500 members and you are welcome to join if you are looking to build large-scale recommendation models one of the biggest benefits of tfra is dynamic embedding if you recall from our last video we mentioned using the hashing trick to accommodate new user IDs or item IDs in your model that's one way to do it but tfra gives you another way to handle this tfra allows you to dynamically increase or decrease the embedding table size which often achieves a better recommendation results than the static embedding in Native tensorflow this is particularly useful for the online learning scenario in which the data arrives in a sequential Manner and is used to update the model continuously instead of in a batched mode tfra is a fully attention 402 compliant and supports all the tensorflow native components such as optimizers initializers save the model and so on there are two sub-modules inside tfra Dynamic embedding and the embedding variables we are going to focus only on Dynamic embedding today at a high level tfra Dynamic embedding expresses sparse weights which is very common in recommendation models in two layers via an efficient key value stored under the hood this enables you to train large and dynamically changing embedding tables the key value store has different implementations for example cuckoo hash redis and rocksdb and the community is working to add even more options you should definitely experiment and pick the one that suits your needs given all this context now you may be wondering how to use dynamic embeddings it's actually pretty straightforward since tfra tries to comply with existing tensorflow API interfaces and preserve compatibility by hiding all the implementation details under the hood if you look at this code snippet you can see that you can directly use our original tensorflow initializers to initialize your variables the main apis for dynamic embedding are very similar to the native tensorflow apis in terms of naming semantic and functionality you can also reuse all Native optimizers to train your model all you need to do is to wrap one with Dynamic embedding optimizer now let's walk through a concrete example this example uses the Amazon review data set for digital video game and it is available in the tfra repository this data set has a number of customer reviews along with contextual information such as product title and review body the label is verified purchase which is a binary our goal is to predict the purchase based on the review plus the customer and the product features now let's build a neural network with Dynamic embedding to tackle those from the dynamic embedding module we use the field wise embedding layer which encodes the feature IDs into fields a field means the category of a feature ID intuitively you can think of a field as a bucket every feature ID belongs to a specific field slot and feature IDs in the same field will be mapped into an embedding vector next we Define a flattened layer and three dense layers in the core function we stack them together and this becomes our neural network with our model in place we can instantiate it and wrap the native tensorflow atom Optimizer with Dynamic embedding optimizer in the training Loop we encode the features to fit into the our model and run the training with a familiar fit method what's new here is that we can restrict the size of our embedding table in the restricted method we are telling Dynamic embedding that when the embedding table reaches Max size shrinking it down to 80 percent of Max size and go on from there again now we run the code with a maximating size of 50 000 and Max steps of 500. in the top graph you can see that our embedding table keeps growing until its size reaches 50 000 then size shrinks down to 40 000 which is eighty percent of the maximating size we specified after 250 steps the embedding size bounces between forty thousand and fifty thousand which is exactly what we specified in the command line and the restricted method so to summarize today we introduced the tensorflow recommenders add-ons to you tfra is particularly useful when you want to train large-scale recommendation models or when you want to dynamically resize the embedding tables with that thank you for watching this video and see you next time [Music] [Music]