Deep Learning Frameworks Compared

Key Takeaways

hello world it's SJ in this video we're going to compare the most popular deep learning Frameworks out there right now to see what works best the Deep learning space is exploding with Frameworks right now it's like every single week some major tech company decides to open source their own deep learning library and that's not including the dozens of deep learning Frameworks being released every single week on GitHub by Cowboy developers how many layers you get let's start off with scikit learn scikit was made to provide an easytouse interface for developers to use off the shelf general purpose machine learning algorithms for both supervised and unsupervised learning psyit provides functions that let you apply classic machine learning algorithms like support Vector machines logistic regressions and kous neighbor very easily but the one type of machine learning algorithm it doesn't let you implement is a neural network it doesn't provide GPU support either which is what helps neural networks scale since like 2 months ago pretty much every single general purpose algorithm that pyit learn implemented has since been implemented intenser flow pyit you just got learned there's also Cafe which was basically the first mainstream production grade deep learning library started in 2013 but Cafe isn't very flexible think of a neural network as a computational graph in Cafe each node is considered a layer so if you want new layer types you have to define the full forward backward and gradient updates these layers are building blocks that are unnecessarily big there's an endless list of them that you can pick from in tensorflow each node is considered a tensor operation like Matrix add or Matrix multiply or convolution and a layer can be defined as a composition of those operations so tensorflow's building blocks are smaller which allows for more modularity Cafe also requires a lot of unnecessary verbosity if you want to support both the CPU and the GPU you need to implement extra functions for each and you have to Define your model using a plain text editor that is just ghetto model should be defined programmatically because it's better for modularity between different components also Cafe's main architect now works on the tensorflow team we're all out of Cafe but speaking of modularity let's talk about Kos Kos has been the go-to source to get started with deep learning for a while because it provides a very high level API to build deep learning models Cara sits on top of the other deep learning libraries like theano and tensorflow it uses an objectoriented design so everything is considered an object be that layers models optimizers and all the parameters of a model can be accessed as object properties like model. layers 3 . output will give you the output tensor for the third layer in the model and model. layers 3. weights is a list of symbolic weight tensors this is a cleaner interface as opposed to the functional approach of making layers functions that create weights when being called great documentation it's all Gucci yes I'm bringing that back but because it's so general purpose it lacks on the side of performance Kos has been known to have performance issues when used with a tensorflow backend since it's not really optimized for it but it does work pretty well with the theano backend the two Frameworks that are neck and neck right now and the race to be the best library for both research and Industry are tensorflow and theano theano currently outperforms tensorflow on a single GPU but tensorflow outperforms theano for parallel execution across multiple gpus Theo's got more documentation because it's been around for a while and it's got native Windows support which tensorflow doesn't yet damn it windows in terms of syntax let's just take a look at some code to see some differences we're going to compare two scripts in tensorflow and Theo both do the same thing initialize some phony data and then learn the line of best fit for that data so it can predict future data points let's look at the first step in both tensorflow and Theo we're generating the data pretty much the same way using numpy arrays so there's not really a difference there let's look at the model initialization Parts this is the basic yal MX plus b slope formula in tensorflow it doesn't require any special treatment of the X and Y variables they're just there natively but in theano we have to specifically say that the variables are symbolic inputs to the function the tensorflow syntax of defining the B and variables is cleaner then we Implement our gradient descent function which is what helps us learn we're trying to minimize the mean squared error over time which is what makes our model more accurate as we train the Syntax for defining what we're minimizing is pretty similar then when we look at the actual Optimizer which helps us do that we'll notice a difference in syntax again tensorflow just gives you access to a bunch of optimizers right out of the box things like radiant descent or atom theano makes you do this from scratch then we have our training function which is again more verbose see the trend here theano so far is making us Implement more code than tensorflow so it seems to give us more fine grain control but at the cost of readability finally we'll get to the actual training part itself they look pretty identical but tensor flow's methodology of encapsulating the computational graph feels conceptually cleaner than theano's tensorflow is just growing so fast that it seems inevitable that whatever feature it lacks right now because of how new it is it will gain very rapidly I mean just look at the amount of activity happening in the tensorflow repo versus the Theo repo on GitHub right now and while Kos serves as an easyuse wrapper around different liaries it's not optimized for tensorflow a better alternative if you want to learn and get started easily with deep learning is TF learn which is basically Kos but optimized for tensorflow so to sum things up the best library to use for research is tensorflow the world-class researchers at both open Ai and Deep Mind are now using it for production the best library to use is still tensorflow since it scales better across multiple gpus than its closest competitor theano lastly for learning the best library to use is tflearn which is a highle rapper around tensorflow that lets you get started really easily also shout out to Rahul Dio for being able to generate an upbeat midi file badass of the week please subscribe for more programming videos for now I've got to go worship tensorflow some more so thanks for watching