PyTorch or TensorFlow?

Key Takeaways

five years ago if you ask me this same question which deep learning framework should i use i'll be telling you about six deep learning frameworks mxnet cntk chainer keras tiano and cafe fast forward to 2020 uh they are pretty much all dead and the only two frameworks that matter are tensorflow and pytorch unless if you're developing some uh exotic jvm deep learning applications you'll be using dl4j okay so let's briefly go through the history of how both frameworks both by torch and tensorflow came to be so using google trends here we can see that the initial release of tensorflow happened in november 2015 and we had a huge spike here as it was probably uh like heavily hyped as the whole deep learning field currently is and then we had some busts here so fast forward a year later uh pytorch uh was initially released in i think september 2016 and by then tensorflow already gained a lot of traction as you can see here and it took a while but fast forward to 2020 they pretty much converged here let's briefly go into the worldwide view here and you can see the tensorflow is still more popular than than pytorch also if we take a look at github repos uh tensorflow's got uh 148k stars and uh whereas pi torch has around 50k uh stars now looking at this data from google trends and github you may say well okay like uh pytorch kind of caught up but tensorflow is still more popular but is it so now in this video i'm gonna give you uh an overview of this two uh frameworks along uh two dimensions so the first one being at the general ease of development how quickly can you prototype something how quickly can you do research and the second dimension is can you deploy it how easy is to deploy your models once you train them and push them to production now what this video is not it's not me telling you go use tensorflow or go use pi torch because neither google nor facebook is paying me to do this video so what this video is it's my review uh based on my research i've done uh on this topic after this video you'll know which framework makes more sense depending on your particular context so unfortunately tensorflow's got this nasty history with static graphs and that was the tensorflow version 1.0 where you basically had to define a steady graph of your neural network before you start using it and it was very hard to debug it was not pythonic and they are paying the price now because of that in the meanwhile uh tensorflow uh released uh tensorflow 2.0 where they basically pretty much copied the uh paradigm pi torque is using and that's dynamic graphs where you basically create the graph of your neural network like you'd write a simple python program so i said nasty because now they have problems with legacy docs is this 1.0 or 2.0 if you go and search your question on stack overflow you'll sometimes get answer from version 1.0 which totally does not make sense for tensorflow 2.0 so 2.0 is also referred to as eager execution and although it did brought dynamic graphs with it it's way slower than pi torch is and that's the second bad thing looking at the api itself now in 2020 they pretty much have the same apis they've converged even uh the one of the co-authors of pytorch said himself in this tweet that it doesn't make any sense to compare them anymore because they've converged so much in that sense fire torch on the other hand was pythonic from the very start it's super easy to learn it's got awesome documentation it's got awesome community and there is no ambiguity between which version is it can i use this one can i use this one it's simple so let's look at some curves uh i don't want this to be me ranting about fire terps is much better in research let's back this up with some numbers okay so there is this awesome website which i'll link in the description uh which is showing us the like relative popularity between tensorflow and pytorch and if we look at the first graph here you can see everything uh above fifty percent means that uh pytorch is uh better so looking at the most famous conferences on computer vision like cvpr or a natural language processing like em nlp or some more classic deep learning machine learning conferences like nips and i see lr pi torch is pretty much beating tensorflow if we look at some autographs like the percentage of papers written in certain framework we can see that cvpr like we 30 of the papers were written in in pythort whereas uh only 7.7 uh of the papers were written in tensorflow and we can see that the number is going down whereas the python trend is not slowing down at all uh finally the last plot just shows us the like the sheer number of papers written in certain framework and cvpr again uh had 418 papers written in pie torch whereas only 113 papers were written in tensorflow and again the trends priority is going up tensorflow is going down it's pretty obvious that pi torque is killing it in the research community now let's take a look at the second dimension uh deploy the dam model to production dimension now tensorflow is much more mature along this dimension so you have so first it's backed up by google secondly simply had more time to mature uh one year to be precise so it's got tensorflow serving so it's a broad term for let's say you you have a web app you have your model served there and basically what it allows you to do is to seamlessly just kind of update the model on the fly without the users ever noticing that happened so it's got a really strong support for that it also has a strong support for uh like deploying your models to mobile devices and also to different kinds of iot devices embedded devices that's known as tensorflow lite again and uh it also has this thing called tensorflow.js which enables you to deploy your models to to the browser pytorch on the other hand has got its own equivalents like pytorch serve and pi torch mobile which do pretty much the same things they are just much less mature than tensorflow uh versions they both came maybe less than a year ago and they still have a long time to just become as mature as tensorflow is in that aspect on the good note many companies like openai are embracing pytorch as their official framework of choice which is really reassuring uh also tesla is using pytorch heavily and andrei karpathy is being promoting pytorch all over the place microsoft officially became the pi torch maintainer for windows platform even big companies are starting to use pi torque which is a cool thing because they obviously have to deploy their models uh and that means they are betting that pi torch will eventually uh mature in that sense as a general conclusion if you're a startup a business and you wanna a cheaper product that has some machine learning deep learning components in it it's probably a safer bet to go with tensorflow as it's got a really mature ecosystem of uh deploying your models on the other hand if you are willing to bet that pi torch will get there eventually and you just want to have the ease of development and to be able to use the the best research out there i'd go with pytorch so for any one of you who doesn't have its own business you just want to learn deep learning i strongly suggest you start with pytorch as a final note it's worth mentioning that uh fast ai is teaching its deep learning courses in pytorch and stanford also started uh teaching its courses in pytorch which will in my opinion buy us new graduates and new phd students to love the framework and to start developing their own startups using pytorch uh which will actually put the inertia movement on pythorg site which was uh till now on tensorflow's site so hope you liked this video and found it useful i'd love to know what's your favorite framework and why and just comment down in the comment section and i'd love to hear your opinions on this one also subscribe to this channel and gently click that bell icon so that you get notified when i upload a new video until next time keep learning you