Generate text with KerasNLP

Key Takeaways

hi there welcome back to our last episode of apply the machine learning with caros TV and caros NP video series my name is way and I'm a developer advocate in Google ml team today we're going to cover a very exciting topic text generation large langage models are autoag these days and the core of this large Lang models is predicting the next word or token in a sentence which is often referred to as C LM pre trining large D models are complex to build and expensive to train from scratch luckily there are pre-trained LS available for use right away Caris NLP provides a large number of pre-end checkpoints that allows you to experiment with soda models without needing to train them yourself for example you can load the gpt2 model by calling the gbt2 co LM from preset method other than G gpt2 there are many other pre- Trend models available such as OPD arbert Roberta and so on you can find the complete model list in the link below now you can call the generat method to generate text where passing in a string my truth to emity was as a prompt and the gp2 model will try to write coherent text based on this prompt note that it takes about 18 seconds for the first round let's try another prompt this time it takes only 1.7 seconds which is about 10 times faster this is because the computational graph is compiled by X in the first round and reused in the second round behind the scene the quality of the generated text looks okay but we can improve it without fine tuning but before performance fine tuning let's spend a minute to talk about the architecture similar to the birth classifier we discussed last time the gbt2 Cal model also has a pre-processor tokenizer and backbone under the hood all of which can be easily loaded VI a simple front P set method for fine tuning we're going to use the Reddit tfue data set so that the output will follow reddits writing style here's an example of the training data since we're performing next word prediction in a language model we only need the document feature here next we Define a custom learning rate and kick off fine tuning using the fit method it takes quite a little bit of time and GPU memory but after it is finished the generated text will be closer to Reddit writing style and the generated lens will will be closer to our preset lens in the training set another thing you can do is to round the model on mobile devices you can convert the model to Testo light and run it on Android devices we want to go into the details of implementing this but you can find a code lab and a video below that teach you exactly how to do this one of the parameters that gives you final control over the generation process is a sampling method caring P offers several samply methods for example greedy search topk and beam search you can easily set the sampler VI stream when compiling the model by default the gp2 model uses topk sampling which basically means only sample from the topk candidate tokens or you can pass in a sampler instance like what we are doing here you can find out all the Samplers in the link below so that's it for text generation with caros NRP we have shown you how to generate text with a pre-trend gp2 model and find you the model for certain writing style and you can even build an Android app based on this now we have come to the end of our entire video series of Applied machine learning with caros CV and caros NP there are of course a lot more you can do with car CV and carp than we can cover in this video series I hope you find our videos useful to get started and I highly recommend you check out our comprehensive tutorials and documentation to learn more more importantly I look forward to seeing what you build with Caris CV and carp to solve your own practical machine learning problems and deliver real world business impact thank you very [Music] much [Music]