Advanced machine learning for web developers: Autoencoders, GANs, RNNs and more

Key Takeaways

foreign s you made it to the very final lecture in this course you've been on quite the journey so let's recap what you've covered so far you started by learning What mo is and how it differs from Ai and deep learning before moving on to tensorflow.js and the advantages of Performing ml in the browser or even node.js on the server side next you dive deeper into using off-the-shelf models written by someone else to perform common tasks that are often reusable you also learned how to work with the raw model.json saved model format directly setting tenses into and getting them out of the model to view results you then saw how you could train your own models from a completely blank canvas to solve problems for which an existing pre-made model did not exist now in this section you started with a simple single neuron that could take single or multiple inputs to solve linear regression problems and then moved on to nonlinear regressions and eventually classification using multi-layer perceptrons and convolutional neural networks it was at this point you also saw how much time it can take to train a model from a blank canvas and how transfer learning can help speed this process up especially for convolutional neural networks that can take a long time to train otherwise which then naturally LED you to discover how you can make use of cutting-edge research coming out in Python and convert it to running the browser either by direct conversion on a save model or retraining the python model in a co-lab first to save the resulting model and then converting it to the format you need and then finally you put all your knowledge into practice to create a full end-to-end web app that gives a tech spam using a natural language model understanding how to convert the text into numerical representation to then pass through the model to get results finally at the start of chapter 7 you saw how people from a wider Community are using tensorflow.js in the real world so now it's time to give you a glimpse into the wider world of machine learning to wrap up this introductory course each one of the following areas can have a whole course dedicated to on its own and based on your interests and experience you may choose to dive deeper into one or more of these areas now that you know the foundations so let's get to that first up autoencoders build upon your knowledge of multi-layer perceptrons they can be used for many things such as transforming a grainy input image into one that's smoother as shown by the example images on this slide or even generate new images that never existed before now the key thing here is that there are actually two subnetworks the first half of a network essentially acts as an encoder and the second half is a decoder that's joined by the middle layer in the network by forcing the inputs to go through a middle layer that has far fewer neurons than the original input size it makes your network use those neurons to encode meaningful features about the imagery it's trained on so we can then decode those features to recreate something meaningful again at the output you can think of this as a form of compression if you will essentially the network takes an input and tries to recreate an output similar to that input now this might seem odd at first but once trained you can then drop a network in the middle as shown on this slide and generate new output images just by changing the inputs to the three neurons that are in the middle of the original full Network as shown often machine learning Engineers will refer to as compressed representation as the latent space which you can then step through with different numbers and see what comes out at the outputs and here you can see that in Action a member of the community made its own autoencoder in tensorflow.js that's able to generate new faces in real time live in the browser just by tweaking two values represented by the circle moving in the Square at the top right of this animation that just changes to input numbers the active change of these input numbers generates new outputs from the model but you can then visualize and explore the latent space is length also on the generative side of things but ml architecture is known as generative adversarial networks or Gans for short bees consist of a generator and a discriminator whereby the generator is trying to produce outputs such that the discriminator cannot tell if they're the real thing or not essentially you're training a regular classification model for the discriminator such as a CNN that you already know about and then a second network is also created to generate outputs to try and fool the CNN classifier known as the generator this generator Network takes some random input values and uses deconvolution to then create new images which is the opposite of convolution that you learned about when understanding cnns this essentially allows image data to be produced from beaches instead of finding features from images once trained however meaning that the discriminator cannot reliably tell the difference between generated versus real inputs Gans can be used in many creative ways you can create new sound video or even generate images from text descriptions in more advanced gun based networks and here you can see again's outputs in Action a member of the community used again to generate rotational variations of a face in this famous painting and then use tensorflags to track their body's real position so it makes it look like the painting is watching you as you move around in this demo the image frames generated by the Gan are pre-computed in advance to achieve a real-time performance in this demo and by many variations of Gans like the ones shown here on the left researchers have been able to turn sketches into photorealistic outputs based on the knowledge began as length and on the right you can see how you can turn text into images to produce results that are really rather remarkable maybe in the future when you need a stock photo you can just generate one that never existed before and it fits your exact requirements for a placeholder on your website design next up are recurrent neural networks or rnns for short these are like multi-led perceptrons but with a Time aspect here you can see a regular multi-layer perceptron as you've seen before and on this slide is a simplified version of that same drawing so it looks a little bit less busy this is still a multi-layer perceptron at this stage now modifying the flow of it slightly you can now see the structure of an RNN whereby the hidden layer's outputs are actually allowed to loop back to become their inputs as well you may also hear of lstms which stands for long short-term memory and are a special kind of RNN in this case connections between neurons can form a sequence over many time steps allowing them to capture temporal relationships better than other networks often you'll see this sort of network being used to solve things like time series data speech recognition or even music generation which is time based to name just a few examples and one other popular architecture right now is the Transformer Network these can be faster than RNN described in the previous slide and can handle sequential tasks pretty well such as time series prediction or even Tech summarization and translation in fact one of their most well-known use cases is for natural language processing the key difference here is that it uses a breakthrough in research known as attention to understand at each given step through a given sequence what other parts of the sequence are most important to focus on when learning about that data now each one of those areas mentioned could be a course in its own right machine learning research is moving very fast but as you've seen often built upon prior concepts with your new understanding you can now choose a path that you might want to specialize further in and go deeper and let's not forget that this course focused on supervised learning techniques which at time of writing are by far the most popular form of machine learning across many different Industries however do also check out techniques in unsupervised learning and of course reinforcement learning which are evolving very fast too and again each of these could have a whole course dedicated to just those techniques there is still a lot more to explore if you wish to do so now as you go deeper into any one of these areas you may find that examples only exist in python or maybe even some other language like C plus which means there's great opportunity to bring such examples to JavaScript users too as you learn this will in turn mean even more models can be used across Industries and use cases than ever before in fact now is a very exciting time to be part of the webml space as it continues to grow very rapidly and on that note I'd like to point out some resources that you can use to connect or collaborate with other members of the tensorflow.js community which is a great way to share knowledge and go further first up check out our growing collection of codelabs at codelabs.developers.google.com you can simply search tentative.js on this site to find a whole bunch of useful step-by-step guides as they come out from a tensorflow.js team here at Google every single year and it's a great thing to share with your team or friends who might want to learn something specific around tensorflow.js next up the tensorflags team have also formed a special interest group to work on key areas of research in collaboration with all of you if you're interested in helping create the future of a tensorflow.js library itself then do come join us at the monthly tensorflow.js Sig meetings all of the areas shown of this slide are actively being worked on by companies and people just like you so do get involved or suggest a new area of research that is yet to be ported to the library if there's enough interest and you're willing to lead it check for URL on the slide if you want to learn more or maybe you want to help spread the love for tensorflow.js if you're feeling inspired after this course well we've also started a working group that meets in a separate monthly meeting to help scale the Outreach education and internationalization of content around tensorflow.js through presentation opportunities blogs videos and more get in touch with me directly if you'd like to Champion tensorflow.js in your country or region so to bring it all together here are some key links you might want to bookmark now first up you can find our official website for more details on the library itself and of course for full API on the link at the top of your slide or check out even more of our pre-made models as they come out using the second link you can view our open source code on GitHub and even make a contribution if you're up for the challenge we are fully open source after all next check the official tensorflow Forum which is a great place to ask more technical questions and it's monitored by the tensorflow.js team just remember to tag your post with tfjs so we find it and we've also got code examples that you can fork in minutes over on glitch.com and codepen.io that provide working boilerplate code to get started really fast as you've seen throughout this course naturally these examples will continue to grow so do check them out from time to time now if you're looking for inspiration head on over to our short 10 minute interviews on YouTube via our major tensorflow GS series here I interview people who have made great things using machine learning in JavaScript on the official tensorflow YouTube channel which might just give you some ideas for your next projects or find some interesting folks to connect with from the community either way it's a great way to fill a coffee break with these bite-sized interviews or share to others who might be interested to learn more about what's possible with machine learning in JavaScript and finally for reading I'd recommend checking out the two books shown here the first book by Manning was written by folk on potensified team here at Google it goes deeper into advanced topics mentioned in this chapter like Gans rnns and lstms in the second half of the book and for that reason it's a great book to read after taking this introductory online course to refine your existing knowledge further the other book listed here by O'Reilly is a good easy to follow introduction to machine learning in JavaScript similar to complexity in this online course you just took so if you're looking for a book to recommend to others on your team who prefer to read a book end to end instead of taking a weekly online course then this is a great introductory resources to show others as well now the only thing left to say at this point is to come join our really fast growing Community there's far too many examples coming out every single month to fit in a single presentation so do check out the major of tfjs hashtag on social media such as Twitter and Linkedin to see what people are making around the world and if you make something using tensorflow.gs be sure to use the tag for a chance to be featured at our future events and I really look forward to seeing what you all create with the knowledge gained on this course and with that thank you for listening you've just completed an incredible journey transitioning your knowledge from a pure web engineering background to the world of web engineering with machine learning a huge congratulations on that now as mentioned before this is a new chapter in the world of JavaScript so go forth and be creative use your skills and while your customers are friends with the knowledge that you now have finally feel free to connect with me and stay in touch too I'm Jason underscore maze on Twitter Creative Tech on LinkedIn or if you prefer I've also got a Discord server listed on this slide here I post all the latest finds around machine learning in JavaScript Innovation webtech and more so see you there and I look forward to seeing how you'll use tensorflow.js in your industry to create the next generation of web applications [Music] [Music] thank you