Build a Neural Network (LIVE)

Siraj Raval · Advanced ·🧬 Deep Learning ·9y ago

Key Takeaways

Builds and trains an LSTM neural network to generate city names using Tensorflow

Full Transcript

hello can anyone hear me who can hear me I am live I know I'm late but can you hear me is the question I am talking I am talking I'm not sure if anybody oh my God holy [ __ ] that was loud yep yeah you guys can hear me okay okay here we go hi everybody hello world it's SJ and we're today we're going to build a neural network okay it's going to be a special type of neural network okay awesome it's a special type of neural network high that is called an lstm neural network it's like probably the most advanced neural network I could think of but to do it an easy way so this is going to be a challenge but it's gon to be super fun we're going to generate um city names using this in Python we're going to generate city names just random city names so we're going to train it on a list of every US city in every single city in the US and then it's going to generate new city names from scratch yes a long short-term memory uh M Matias a neural network is uh an algorithm that replicates neurons in the brain and you can train it to learn things it's a learning algorithm okay so we're just going to get right into well I want to do five minutes Q&A and then we can just get started okay hes the different types of neural networks okay so so there's the regular type of neural network which is just feed forward in a feed forward neural network data just flows one way through the through from the beginning from the input layer all the way to the output layer that's a feed forward neural network and those things are good for uh predicting the next number in a sequence of numbers I'm going to build this using T TF learn yes and tensorflow it's a it's a library on top of tensorflow you guys G to you're you're gonna love this a recurrent neural network is a type of neural network where the data flows not just through it but then back again so it Rec so it's recurrent it goes back and forth through that and that's good for uh memory so anytime you need to uh remember maybe a sequence of music or a sequence of uh text or poetry if you want to generate something some new type of text that's what recurrent networks are good for uh I can build a neural network from scratch uh it's possible using numpy the scientific Computing Library um and then one more type of neural network and the one we're going to do in this video is called an lstm neural network which is a type of recurrent Network that's it's called long short-term memory so it can remember things from way back in the sequence of data so it's like all it remembers everything which is what we're going to need for the city because there's so many different cities for this uh City generation let me answer a few more questions I've got about three more minutes I want to get right into it this time how did you learn to build neural networks by just taking time and going through the machine learning sub going through GitHub and just downloading a bunch of neural networks uh from GitHub repositories just playing around with them seeing how they work comparing them comparing different Library Implement implementations of it and then building my own from scratch using nump pod where can you get tensor flow uh Google tensor just Google tensorflow it's the first link uh numpy is a pain um build it from scratch I'm I'm going to um build an lstm from close to scratch is C Sharp good for neural networks um there's the Microsoft cntk Library which I don't really like that much because I don't like Microsoft I mean no you know you know anyway I'm not gonna apologize all right let all right we're gonna get to the tutorial [Music] um uh yes LMS are definitely more powerful than hidden marov models I mean hidden marov models were kind of the precursor to deep learning and this whole craze going on with deep learning right now deep learning kind of makes hidden markup models not as not as relevant anymore um how to design and build a data set you just have to um how do you design and build a data set I mean you can just create an Excel spreadsheet and then for the X column you know like put like number of times that I've you know been laid and then yes no yes no for every day or something there you go there's your data set CSV and python can read CSV files very easily right now I'm in San Francisco um the goal of this neural network is to generate uh city names from scratch like new city names we're g going to train it on city names it's going to generate new city names okay so I'm just going to get right into it here we go let's see screen share I would recommend my videos to get started start from learning for hackers number one screen share so I'm going to go to the desktop okay and holy [ __ ] that's a lot of [ __ ] all right so I'm gonna move this and it's going to show me and we're gonna make this bigger so I'm G to be in the corner and you guys your text I'm gonna put right up here so I can look at it your chats while I code this thing okay um uh okay apart from tensorflow in Python yes there are dependencies I'm going to be showing you guys which dependencies we're going to use uh let me make sure that everything is working well here okay let me make sure everything's working well make sure my text editor is big nice and big text editor right that's what we want oh [ __ ] where the did that go oh [ __ ] all right there we go all right let me make sure everything's nice and big nice and big text is that um is that good and big for you guys is that readable right here this this level um this is good right all right we're just gonna get started and if anybody says anything we're gonna this is g to be uh probably 30 minutes I'll make it bigger okay okay how big that that's big right all right here we go a bit bigger okay that's as big as we're going to go guys okay let's get started oh [ __ ] yeah okay here we go okay so first things first we're gonna import our dependencies okay let me um bring up my notes for a second so I make sure that this is the right [ __ ] okay so and we're gonna build this in Python and it's going to be man I have so many [ __ ] windows open uh OSX for Ubuntu for deep learning I would say Ubuntu I know I'm on OSX but like okay here we go so we're going to start off by importing the first module uh this is going to be called uh future so the future module is for uh it's the link between Python 2 and three it's a way for us to use Python 2 or three without there being any kind of version errors between what we're what we're using so I'm gonna say from future import uh absolute import I'm going to have a recording of this session for sure Division and the print function those are that's and let me turn on python because right now it's in plain text so python boom hold on there we go there we go so hold on that shit's big so those are our those are our uh dependencies okay so we have one we have another one okay so the next one is uh OS so we're going to import OS that that allows us to have operating system dependent functionality we're going to use it to read in file paths which is where our data set's going to be we're going to read in our data set using this which we're going to download in the script okay so that's OS um what else we need to import six and we're gonna import something from six called uh moves so let me just write this at the top right [ __ ] LS sorry for the language I don't know I don't care no no no apologies ever lstm to generate city names okay that's what this is this whole thing okay so from six import moves okay and so what that's going to do uh six is another library for like commonalities between different python versions uh and moves is uh we're going to use that for the URL live module which is how we're gonna pull data from the internet okay in a second so what else we're going to import um SSL which is how we're going to long shortterm memory network uh long short long shortterm memory NN to generate City generate cities let's just say generate cities okay so import SSL that's going to let us connect to the internet um and then of course we're going to import TF learn um tflearn is going to be our machine learning library that's how we're going to do TF learn uh we're going to work with some operations for our data so we're going to say from TF learn 30 minutes exactly thank you from ke. dat utils work in full screen um work in full screen uh no okay here we go dot dot so we're gonna import everything from the data utilities okay um so okay that's those are all of our dependencies uh TF learn uh that's those are all of our dependencies that's it now we're going to start coding okay so the first thing we want to do is get our data so the step one retrieve the data okay that's this is the data okay we're gonna we're gonna get our data okay so so first things first we're going to create the path for our data set um and we're going to call it us cities. text okay that's that's what we're going to call our our data set so if we don't have that file already okay if we don't have it we want to tell our machine that so that's where the OS module is going to come in we're gonna say if not os. path uh is file and then the path as the argument TF learn is a is a wrapper on top of tensorflow that makes it a little easier to use so it's great for learning um okay so it's it's not really an add-on it's just like if I were to write an entire neural network maybe like a hundred lines of neural network in tensorflow and tflearn I could access that neural network and maybe two lines of code and it just says like access neural network so it's like an it's layer of abstraction on top of that complexity of tensor flow so the trade-off is you don't get that you know fine grain you know control over everything but the good thing is it's easier to understand it's more readable Kos is similar to uh tflearn in that it is also it's a layer of abstraction over tensorflow an theano and and a few other libraries okay so retrieving the data so if uh so if we don't have that inter file path we want to create it right so we're going to say we're going to create a context caps say Okay context that says um uh create unverified Bridget I'm too I'm too soon into this can't make it from scratch right now but next live stream I will okay but just follow along you're gonna you're gonna you're gonna like this uh create an unverified context what does that do that just initialize a bunch of key and Trust managers because we're going to get data from a server in a second so that's just some initialization stuff okay let me um let me make this a little smaller okay that's all just a little small okay so now we're going to say uh now we're going to retrieve the actual we're going to retrieve the actual uh data so we're gonna say we're going to use a URL live module of moves to retrieve our data and uh let's see URL live module we're going to retrieve our data and we're going to say request um URL retrieve make computers from scratch nice okay uh one day I actually have before it's not a big deal anybody can do it okay so um I will upload it um I'm recording it so six package is good for um being a kind of Gateway between Python 2 and three so you can write python two code and it works on Python 3 and vice versa okay and so here is what we're going to retrieve so this is the link to the data set that we're going to retrieve I have this P I have this written down somewhere so I'm gonna paste this in this is the data set okay it's a huge link it's a huge link okay boom this whole thing and that's our us City's data set okay so I'm just gonna say get data set boom just like that make it a little smaller okay so that's our data set all right we've done that and we've retrieved it and uh I want to save it to my path and with my context equal to the context I just created okay so boom that's it that's that's my data set so now okay it's time to do some machine learning on this stuff okay so um it's gonna be on GitHub it's gonna be on GitHub don't worry man it is such a trip to read comments and code and do it all live at the same time this [ __ ] is crazy I love it though I love it I love you guys you guys are awesome okay so max length equals path typo who who spotted that path typo Javier path typo oh thank you you thank you badass okay so max length so this is the max length we want cities to be city name max length we don't want them to the cities that we generate to be um break is a good idea thank you thank you for that uh so we don't want our city names to be longer than 20 characters so we're going to set this max length all right I checked the typo okay so mive max length uh and so now we're going to vectorize our text file okay so we're gonna say vectorize the text file what does that mean when we have a bunch of import typos hi ignot hi thanks for coming okay so vectorizing a text file when we have a data set we want to vectorize it when we have a data set of words we want to vectorize it so what does that what does that mean I'm G to get that typo in a second what does that mean it's it means take all these words all these thousands of words and find those commonalities between those words and make those abstractions of those commonalities so if I have a set of words like um uh raccoon squirrel pigeon and dog the vector that would be created from those eventually would be animal it wouldn't it wouldn't be the specific word animal but it would be a representation you like a numerical representation that that is all of those things okay so it so that's what vectors are they help us abstract huge serieses series of data into something that we can manipulate later and we can use these abstractions to generate new data so we could use that animal abstraction to then to then generate a new animal like a wolf okay that's the great thing about abstractions thank you wemerson okay so let's let's make our vectors so our Vector is going to spit back um our uh three things we're going to call it XY and the Char dictionary so it's going to let me just write let me just write this out okay so it's going to be called um text file to semi redundant sequences okay that's the meth that's the function that TF learn provides uh and it's it's going to create vectors for us and it's going to take arguments the arguments it's going to take is the path so where we are hi nithan where we are uh uh where's the data set that we're making vectors from so that's the path and then the sequence the max length which we defined earlier uh the Mac I will say I will post the repo at the end of the video so where we are posting uh what is the max length we already defined it right and then finally redundant step uh equals three okay so that means uh how many times do we want to do this so we're just going to say three times okay and I'll make a line break here okay so that's to vectorize and so that so what what x y and uh it's GNA be probably 20 20 more minutes uh we we'll see 21 minutes okay so uh what is this doing this is uh it is creating uh vectors from our words and it Returns the inputs the targets and the dictionary the inputs that's the words the targets those are vectors and then the dictionary of those things okay I am not on Aderall but I will admit I am on coffee yo you need coffee to do this this this stuff that I do full time you know what I mean I will admit it I love I love coffee couldn't do it without coffee okay um anyway but I I have never tried ader all I've never tried ater all I have friends who have I've never tried it okay all right thanks Amy TSH uh just 20 minutes oh you want more I mean I could go more you know what say okay the neural network is going to generate city names we have a data set of cities and it's going to generate new city names okay this [ __ ] okay Cur man shrooms I've done shrooms before in fact uh the reason I started the YouTube channel is because I had a very uh good shroom trip oh my God he just admitted that he just admitted that because he doesn't give a [ __ ] what anyone thinks okay here we go um here we go create an lstm uh yes T redundant step equals three means we're we're we're just going to uh go through three sequences to to make that happen Okay okay here we go here we go here we go so now we're going to create our LCM create create lstm uh really Jorge holy [ __ ] I love it I love it uh JavaScript maybe someday yeah I mean eventually hopefully Jorge that you quit your job to do something that you liked because of Sho it's not like you you just became crazy anyway okay uh create lstm let's do this so we're gonna create our lstm and we're gonna call it g okay we're gonna call it g uh on Udacity I am working with Udacity on the self-driving car course oh my God I just admitted that but it's going to be awesome yeah you know that self-driving C Nano degree I'm working on that I can't wait I'm going to show you guys next week okay next week I'm GNA I'm going down to Mountain View to meet with the team and stuff it's gonna be crazy they just saw my self-driving car in five minute video and then they were like oh you should work with us and I'm like oh my God Sebastian thrun it's you the guy who like you know made Google X and stuff anyway that's gonna be exciting okay here we go so TF learn. so we're gonna create our lstm and to create our LM we're going to create layers okay we're gonna just layer by layer that's what TM fling is great for you can Define entire layers in one line of code so it's like layer layer layer layer layer layer okay um I don't care what the government thinks come at me if the government is watching and they think they're gonna do something because I did some drugs come at me okay come at me you can't you can't stop me okay here we go so TF learn. input data okay and we're gonna say the shape and so this is our input layer we're this is our input layer okay this is our input layer this is where we're going to put the data in so we're going to say the shape equals um none max length I'm going to I'm going to I'm going to I'm going to talk about what what I'm doing in a second so Char idx okay Char idx okay so what is this doing this is saying make our input layer and this is the size of our input layer it's going to be up to 20 characters um and I wrote input wrong didn't I where input data no I didn't okay so that's our input layer and now we're going to create our next layer and our next layer is an lstm layer okay so TF learn. lstm G 512 that's going to be the size of our Matrix thank you Ace Ace I will I will get better at um this stuff to making it more understandable okay so return sequence equals true okay so what is this this is the lstm layer so M layer is basically you've got you've got 512 nodes in this layer that's that 512 number a layer okay you have 512 neurons or nodes in this layer that's a lot that's that's deep learning that's the Deep part okay you've got 512 of them um thank you Omar I got this okay 512 layers and uh the lstm is basically long short-term memory inside of each of those ner you've got uh little gateways basically an lstm was created recently and basically anything that we we put an we throw any kind of data that we throw in lstm at usually lstms are what works best to learn we're not sure why they're they're the most they're the most complicated complicated types of neural networks out there but they're also uh they also provides the best most performant results Okay so we've got four more layers to create okay why 512 I mean it's just it's one of those things where it's it's just kind of becomes standard like you know one guy did it and it worked and he got great results so let's just all do 512 I mean I can do 513 too you're right um Peter you're right so it's Peter Peter is right so it's not it's not just what's in the layer it's it's not just how many noes are in the layer it's how many layers but I guess in terms of wide and deep learning it it is both LSM means long shortterm memory Network lsms are great for box why are both variable names G because we're going to keep we're going to keep using G we're going to keep using G as we build our layers okay okay uh we're gonna keep using G because we're building our lstm okay so now we we built that uh and now we're GNA create Dropout okay so TF learn. Dropout okay and we're gonna say G again we're gonna keep using G okay so what does that mean what is Dropout okay so how does it relate through a current neural networks an lstm is a type of recurrent neural network so it is a subset of a recurrent neural network I'm not a part of the alpha go team yet but okay so um it's a type of recurr so what is Dropout when we are training our neural network when we are training our neural network okay data is flowing through the neural network right data is let me D this side hold on data is flowing through our neural network and when we train it sometimes we have something called overfitting whe that we have something called overfitting and overfitting is when I'm running it only on a CPU not a GPU overfitting is when the data when the learn model is fit too well to only the data set that we trained and we can't generalize it to new things so to prevent overfitting there's a technique called dropout dropout and what Dropout does is it randomly turns off some nodes it randomly turns off nodes while training so it doesn't go through the same grooves you know like these grooves that we create and then every time you know neural pathways are kind of like when you think a certain way and you get older uh you have these grooves right and so it's hard to break out of those grooves that's kind of a rough analogy but like Dropout basically turns off nodes so it has to find new grooves every time and what that does is it makes it easier to generalize okay so that's that's as good as my explanation is gonna get for right now so that's Dropout we're gonna create another lstm layer okay that's GNA fry my CPU that's G to fry my CPU exactly uh okay so um TF learn. Dropout so that was our Dropout layer we're going to create one more lstm layer and it's GNA be 512 again and then we're going to add more Dropout because we want Dropout at every layer okay so another Dropout G 512 okay uh and then oh no sorry drop out 0.5 oh by the way 0.5 when we create Dropout is just a coefficient it's just a a measure of how how random do we want to get we could say 04 we could say point6 lard I'm gonna get to that in a second okay a be Shake I'll spend Dropout one more time so Dropout is a way to randomly turn off nodes in your neural network as it trains and what this does is it prevents overfitting it prevents overfitting and overfitting means your your trained model won't work for new data sets it's only going to work for the data set that you train to make predictions but you want to make predictions of data that it hasn't seen before and that's why we use Dropout and it's and and uh Jeff Hinton the guy who's like The Godfather neural Nets made it and everyone's been batshit crazy about it since okay why did I write the last two lines twice because we have multiple layers to this lstm it's not just one layer okay uh uh M48 one uh that's that's good intuition but I it's I I don't think it's that I think it's like a it's a it's a measure of of it's it's just a it's just a coefficient and I think like later on it's like it's it's changed in a different way anyway I got to keep going here okay so so that's our Dropout layer and then what else is left what else is left we have our last layer we're gon to create our last layer our fully connected layer our last layer okay so g equals TF uh TF learn. um fully connected Okay g length the length of the fully connected layer is going to be the size of our dictionary and then we're going to create our activation function which is called Soft Max soft Max is a type of it's a type of logistic regression it's a type of logistic regression that is good for classification which is what we're doing we're classifying big sequences of text and then we're going to generate new sequences okay I'm gonna draw this you know what I'm gonna show you guys a visualization in a second let let me just uh uh the I'm gonna post you also of GitHub and you're gonna be able to watch this later so just follow along right now be interactive I'm gonna watch what you're saying and so okay so here we go so that's it we built our neural network that's all okay we're done now we're done with that now we're going to generate our actual sequences okay so generate cities okay we're at that part we're at that generate cities part generate cities okay so here we go we're GNA generate we're gonna call this generated file M um right we're gonna call the generat file M we're gonna say TF learn. sequence generator tflearn has a sequence generator for us it's made to generate the sequences for us sucks big fat talk okay well okay good to know um the chat is not frozen uh is it a regression problem or a classification problem uh this is a this is a regression problem that uses classification early on the end result isn't classification it's using classification early on to to classify to it's classifying words as vectors first okay so sequence generator why is G um wait is important uh I'm going to apply n neural network to teex summarization uh why is G interpreted as five separate layers when it's one variable being overwritten uh uh because it's uh no well it we're we're adding layers to what already exists is there a list of commands in TF learn yes uh it's on the GitHub okay so okay so G dictionary equals char idx and then sequence okay and then uh sequence max length or SL max max length no not over in Attitude it's exactly um clip gradients equals 5.0 I'm gonna explain this in a second let me just write this out uh not a lot of space in the brain for a lot okay there we go okay gon to generate new sequences I am uh Ahmed yes how big is your Source data it's about 20 megabytes it's just a list of every US city um and uh let's see um okay so so what this is doing it's going to say generate those sequences put them in our dictionary for the max length of 20 uh clip the gradients at 5.0 that's just a standard number we're just going to keep there it doesn't matter that much and then the checkpoint path model us ities so this is going to save checkpoints as we train which means like if something happens to my computer like you know I download some porn and it destroys everything it's GNA have those checkpoints saved so the model next time I start training it's going to go from there build it from scratch and Rico that would be so long but I I will do that later okay so here's our last part the training part okay that's it we have five more lines guys we have five more lines okay stick with me here so for I in range 40 okay so now is a trend part we're going to say create a seed I'm going to explain what a seed is in a second seed random sequence from text file um half Max one okay a seed what does a seed do a seed helps us start from the same point every time when we generate new things so when we're we're about to generate a bunch of city names right and so if we start from the same point that's what a seed does it lets us start from the same point so that we have a little bit more um we have a little bit more names are more similar than they would be than they would otherwise be okay let me just read this more could you please do a new line of T FL and fully connected yes good call thank you okay okay so that's our seed and now we're going to fit our model our generated cities model this is the this is this is the learning part okay so so I'm going to take those three inputs I'm going to say x is our input data Y is what we cre with our vectors our validation set is uh called 0.1 batch size is 128 so now I'm defining how much we're going to train I'm just gonna say 128 for batch size um how many epochs do I want an Epoch is like an episode how many episodes run ID is uh what we're going to call this thing we're gonna call it US cities okay this is that's the Train the fit is the train okay and so now so now we're going to print uh the testing we're going to say testing because we're going to print this terminal and we're going to say um print what else we're gonna say print test print so now we're going to print the generated we're going to print the generated cities generate 30 temperature equals 1.2 um sequence seed and then the we created um okay so that's it and maybe we can do this a few times so then with different temperatures what does temperatures mean I'll explain that in a second okay so this time it's with a temperature of 1.0 by 40 um you could do more than 40 I I just don't have like a huge GPU so I'm just like 40 is good enough for right now I don't feel like training it for longer okay so there that that's it okay that's it let's see um okay the m. fit line okay demo save that [ __ ] okay M do fit line um so m. fit basically is uh taking our what we've generated from sequence from our sequence generator that model okay that model and it is it is it it's taking the training data and it's adding it to our lstm so that's that's the actual training bit do you make enough money out of YouTube no the answer is no but I won't be bought out unless you're deep mind so it's all good um how did you choose the batch size it is in relation to the dimensions or no of samples of data uh it's not related I just randomly chose 128 shouldn't it be g. fit what's m so that's a good question so m is m is the the um model but uh so there's there there's a little bit of magic happening here as you can see like those two things aren't connected but TF learn knows that they're are connected um and yeah so let's just let's just train this um okay let's see what happens when we let me print out command line uh okay where are we where are we at python let me let me make this bigger okay make it big as hell oh my God have some errors okay here we go import boom okay oh my God okay hold on um Sho okay so let me answer some more questions uh how does I do and I if you don't have the file and you don't have the library how do you do uh you have to download the file and then what is the loss function so the loss function helps us um the loss function will help us it's something we need to minimize as we train the loss function is something we need to minimize as we train and it's basically a a way for us to measure how good uh our our learning algorithm is doing okay so I'm about to compile this thing um but I'm running out of time so I'm going to see what is this era let's see um there's going to be a lot of [ __ ] because the problem is that this text was so big that I didn't I should have been compiling and checking for errors as I was going um but it's okay it's okay I'm just double checking any does anyone have a link I'm gonna have a link to this video when it is done um and right now I am going to print out what's Happening Here let me go ahead and remove and see no I don't want to purchase that [ __ ] okay so python demo boom okay training where is that happening line 33 don't need that boom no module named tf1 are you [ __ ] kidding me okay I know I don't I don't register uh yes I Indian and American both uh born in the US parents from India uh no module named okay [ __ ] [ __ ] okay hold on give me a second here um hold [Music] onon okay so now it's now it's uh it's working okay so it's training on that data what's happening it's training on that data um and you can see that there's a bunch of like uh text here that's printing out like vectorizing text text total length distinct chars total sequences that we didn't code like we never printed this stuff we never did that okay that's because tflearn uh is doing this itself there there's a lot of magic happening with TF like these these functions like sequence generator right sequence generator and then random sequence from text file these are very high level functions okay bigger fun okay right so this is going to take Pro so let me this is probably going to take like uh probably uh 20 minutes to fully train which we don't have time for so I went ahead and I'm gonna pause that and so here's here's an example I I compiled this beforehand okay so you can see like with the temperatures that we wrote what's happening these are new cities that that have been generated they don't exist they've been generated from the cities that we already have like more W and you know C so rock okay so stuff like that and if you want to see the data set that we use for this let me just take that thing I'm just gonna copy that data set yes I'm gonna ask me anything yeah for sure give me a second let me just take this data set and then print it and I'm going to put it into um my browser so you guys can see what it looks like I'm gonna paste it in and I'm going to put it right here okay so take a look at this this is the data set it's huge it's every single US city okay so it trained on these words okay so that's about 20 megabytes all right so now that's that's that that's what we have time for today I want to keep doing these live streams I'm gonna do a five minute AMA before ending so let me just um stop screen sharing and go to full screen for a second okay I'm gonna put this down here so see what's going on all right and I'm going to answer five minute question questions okay any questions okay okay here we go five minutes go um I um I want a dual boot Ubuntu with Windows I'm scared of losing data use parallels then parallels is great for that parallels download parall parallels um thank you what exactly is an Epoch an Epoch is like an episode of training so you have like 20 Epoch that means like 20 episodes of training uh how many hidden nodes there were 512 in this one drop the uh neural network great idea um let me do that in a second did did I study at University yeah I went to Columbia University I studied computer science there where to start learning machine learning my channel start on machine learning for hackers one go through every single video up till now how should I start learning just my video um okay so for movie lens data for recommendation engine I would recommend using tensorflow uh a deep you would need a deep uh neural network there's a type of recommendation system called a collaborative fil filtering system if you go to GitHub and you type in tf- m m it's a great library for that um an image recognition application see U my video called create an image classifier in tensorflow in five minutes I did that um with TF learn you can you can specify the CPU or GPU using one of the parameters there's a method for that if you search CPU or GPU tensorflow on Google it's going to be the second link on stack Overflow I think any book you can recommend for a python newbie it's called Learn Python hard way it's called Learn Python the Hard Way it's the best python book you could read okay um what area to write your thesis on tensorflow I would say uh what's hot right now probably uh one shot learning with tensor flow oneshot learning and I have a video on that see search my channel oneshot learning um let's see Implement data compression and noise reduction you do that using an autoencoder I have a video on that called build an auto encoder in five minutes can you do video classification model Carlos that is a great idea and I I don't think I've done that yet so that that would be cool how cool would it be to generate video like random video so you train it on a movie and then they generates new scenes like that didn't exist so like real life scenes but didn't ex how cool would that that would be amazing right do I do YouTube full-time yes I'm a full-time YouTuber okay um you're really interested in theoretical CS um I'm not sure if I get in mathematics or computer science go for computer science that way you learn the Applied Mathematics for computer science hi Omar love you thank you uh how would you do it with Gans yes ywan yes I'm Indian how do you do um okay um anyone can be a data scientist that's what my news series is about and because you guys are here live I'm going to tell you what my next video is about I'm still writing the script but it's coming out in two days it's called uh it's using Twitter for sentiment analysis so we're gonna be mining Twitter data and we're going to do sentiment analysis that I know my hair is less crazy today right it's less crazy okay why are Indians good at CS I you know what it is I think it's just like your parents just growing up are just like study study study education education and and make money and that's all like computer science really um uh you're amazing George should I learn python if I already know Java and C++ Mr Bane is absolutely yeah um thank you Gian uh Udacity do the machine learning Nan degree not the data analyst one exactly it's easier to learn if you know can we make a peer-to-peer neural network yo I never even thought about that the answer to life that's a what would that even look like that's that's that's a good question um anyway America people are lazy well we're we're not lazy I mean we're going to Mars in 10 years we're gonna have a million person colony in 40 years and it's gonna be the whole world so we're all gonna do it okay we'll build a new we'll build a new country and on Mars it's gonna be crazy anyway I'm going off topic what are the behind you I'm in a sound room okay I'm in a sound room and this is good for blocking sound I'm working out of a co-working space in San Francisco thanks Kieran I love Toronto okay so that's all for today I'm gonna go I'm gonna stop the broadcast um I'll answer one more question did you participate in kaggle I have participated in kaggle kaggle is a great source to become better at data science look at past projects look at what they did and try to try to um replicate the results me and Elon yeah I'm actually I might be meeting Elon uh this week because Greg Brockman the CTO at open AI said he wants me to come in to talk and so like Elon comes in once a week I know that so like there's a chance I could be like yo yo Greg you wanna you wanna intro me to Elon or something you know what I mean so we'll see anyway thank you so much guys okay all right uh I'll do another one soon uh I don't I don't know when but thank you so much for watching um for now I've got to go drink some coffee so thanks for watching

Original Description

In this video, I'll be building and training an LSTM Neural Network on a dataset of city names. Then it'll be able to generate new city names from scratch. Code for this video: https://github.com/llSourcell/build_a_neural_net_live/blob/master/README.md I created a Slack channel for us, sign up here: https://wizards.herokuapp.com/ Read up more on TFLearn: https://github.com/tflearn/tflearn Incredible article on LSTMs: http://colah.github.io/posts/2015-08-Understanding-LSTMs/ If you liked this stream, support me on Patreon! I do this full-time currently. https://www.patreon.com/user?u=3191693 Follow me: Twitter: https://twitter.com/sirajraval Facebook: https://www.facebook.com/sirajology Instagram: https://www.instagram.com/sirajraval/ Instagram: https://www.instagram.com/sirajraval/ Signup for my newsletter for exciting updates in the field of AI: https://goo.gl/FZzJ5w Hit the Join button above to sign up to become a member of my channel for access to exclusive content! Join my AI community: http://chatgptschool.io/ Sign up for my AI Sports betting Bot, WagerGPT! (500 spots available): https://www.wagergpt.co
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from Siraj Raval · Siraj Raval · 40 of 60

1 What is Bitcoin?
What is Bitcoin?
Siraj Raval
2 5 Ways to Use Bitcoin
5 Ways to Use Bitcoin
Siraj Raval
3 BTC Fever - Siraj [Music Video]
BTC Fever - Siraj [Music Video]
Siraj Raval
4 5 Reasons to Build Decentralized Apps
5 Reasons to Build Decentralized Apps
Siraj Raval
5 The Interplanetary File System
The Interplanetary File System
Siraj Raval
6 How to Build a Dapp in 3 min
How to Build a Dapp in 3 min
Siraj Raval
7 Life Before Smartphones
Life Before Smartphones
Siraj Raval
8 4 Ways to Use Smart Contracts
4 Ways to Use Smart Contracts
Siraj Raval
9 3 Dapps You HAVE to See
3 Dapps You HAVE to See
Siraj Raval
10 Char's Life as a BitTorrent Engineer
Char's Life as a BitTorrent Engineer
Siraj Raval
11 4 Reasons AlphaGo is a Huge Deal
4 Reasons AlphaGo is a Huge Deal
Siraj Raval
12 Build a Neural Net in 4 Minutes
Build a Neural Net in 4 Minutes
Siraj Raval
13 Sentiment Analysis in 4 Minutes
Sentiment Analysis in 4 Minutes
Siraj Raval
14 The Hackathon Life
The Hackathon Life
Siraj Raval
15 Your First ML App - Machine Learning for Hackers #1
Your First ML App - Machine Learning for Hackers #1
Siraj Raval
16 Build an AI Composer - Machine Learning for Hackers #2
Build an AI Composer - Machine Learning for Hackers #2
Siraj Raval
17 Build a Game AI - Machine Learning for Hackers #3
Build a Game AI - Machine Learning for Hackers #3
Siraj Raval
18 Build a Movie Recommender - Machine Learning for Hackers #4
Build a Movie Recommender - Machine Learning for Hackers #4
Siraj Raval
19 Build an AI Artist - Machine Learning for Hackers #5
Build an AI Artist - Machine Learning for Hackers #5
Siraj Raval
20 Build a Chatbot - ML for Hackers #6
Build a Chatbot - ML for Hackers #6
Siraj Raval
21 Build an AI Reader - Machine Learning for Hackers #7
Build an AI Reader - Machine Learning for Hackers #7
Siraj Raval
22 Build an AI Writer - Machine Learning for Hackers #8
Build an AI Writer - Machine Learning for Hackers #8
Siraj Raval
23 Build a Chatbot w/ an API - ML for Hackers #9
Build a Chatbot w/ an API - ML for Hackers #9
Siraj Raval
24 One-Shot Learning - Fresh Machine Learning #1
One-Shot Learning - Fresh Machine Learning #1
Siraj Raval
25 Generative Adversarial Nets - Fresh Machine Learning #2
Generative Adversarial Nets - Fresh Machine Learning #2
Siraj Raval
26 Tone Analysis - Fresh Machine Learning #3
Tone Analysis - Fresh Machine Learning #3
Siraj Raval
27 Generate Rap Lyrics - Fresh Machine Learning #4
Generate Rap Lyrics - Fresh Machine Learning #4
Siraj Raval
28 Build an Autoencoder in 5 Min - Fresh Machine Learning #5
Build an Autoencoder in 5 Min - Fresh Machine Learning #5
Siraj Raval
29 Build a Self Driving Car in 5 Min - Fresh Machine Learning #6
Build a Self Driving Car in 5 Min - Fresh Machine Learning #6
Siraj Raval
30 Build an Antivirus in 5 Min - Fresh Machine Learning #7
Build an Antivirus in 5 Min - Fresh Machine Learning #7
Siraj Raval
31 TensorFlow in 5 Minutes (tutorial)
TensorFlow in 5 Minutes (tutorial)
Siraj Raval
32 Build a Recurrent Neural Net in 5 Min
Build a Recurrent Neural Net in 5 Min
Siraj Raval
33 Build a Simulation in 5 Min
Build a Simulation in 5 Min
Siraj Raval
34 Build a TensorFlow Image Classifier in 5 Min
Build a TensorFlow Image Classifier in 5 Min
Siraj Raval
35 Tensorboard Explained in 5 Min
Tensorboard Explained in 5 Min
Siraj Raval
36 Generate Music in TensorFlow
Generate Music in TensorFlow
Siraj Raval
37 Build a Game Bot (LIVE)
Build a Game Bot (LIVE)
Siraj Raval
38 Deep Learning Frameworks Compared
Deep Learning Frameworks Compared
Siraj Raval
39 Introduction - Learn Python for Data Science #1
Introduction - Learn Python for Data Science #1
Siraj Raval
Build a Neural Network (LIVE)
Build a Neural Network (LIVE)
Siraj Raval
41 Twitter Sentiment Analysis - Learn Python for Data Science #2
Twitter Sentiment Analysis - Learn Python for Data Science #2
Siraj Raval
42 Recommendation Systems - Learn Python for Data Science #3
Recommendation Systems - Learn Python for Data Science #3
Siraj Raval
43 Predicting Stock Prices - Learn Python for Data Science #4
Predicting Stock Prices - Learn Python for Data Science #4
Siraj Raval
44 Pong Neural Network (LIVE)
Pong Neural Network (LIVE)
Siraj Raval
45 Deep Dream in TensorFlow - Learn Python for Data Science #5
Deep Dream in TensorFlow - Learn Python for Data Science #5
Siraj Raval
46 Visualizing Data with D3.js (LIVE)
Visualizing Data with D3.js (LIVE)
Siraj Raval
47 Genetic Algorithms - Learn Python for Data Science #6
Genetic Algorithms - Learn Python for Data Science #6
Siraj Raval
48 Enter Siraj [Music Video]
Enter Siraj [Music Video]
Siraj Raval
49 Build a Web Scraper (LIVE)
Build a Web Scraper (LIVE)
Siraj Raval
50 Why is P vs NP Important?
Why is P vs NP Important?
Siraj Raval
51 How to Make a Neural Network (LIVE)
How to Make a Neural Network (LIVE)
Siraj Raval
52 How to Make an Amazing Tensorflow Chatbot Easily
How to Make an Amazing Tensorflow Chatbot Easily
Siraj Raval
53 How to Make an Amazing Video Game Bot Easily
How to Make an Amazing Video Game Bot Easily
Siraj Raval
54 How to Make a Tensorflow Neural Network (LIVE)
How to Make a Tensorflow Neural Network (LIVE)
Siraj Raval
55 How to Make a Simple Tensorflow Speech Recognizer
How to Make a Simple Tensorflow Speech Recognizer
Siraj Raval
56 Joel Shor - Really Quick Questions with an Awesome Google Engineer
Joel Shor - Really Quick Questions with an Awesome Google Engineer
Siraj Raval
57 How to Make a Path Planning Algorithm Easily (LIVE)
How to Make a Path Planning Algorithm Easily (LIVE)
Siraj Raval
58 The Best Way to Prepare a Dataset Easily
The Best Way to Prepare a Dataset Easily
Siraj Raval
59 Catherine Olsson - Really Quick Questions with an OpenAI Engineer
Catherine Olsson - Really Quick Questions with an OpenAI Engineer
Siraj Raval
60 How to Make a Tic Tac Toe Neural Network Easily (LIVE)
How to Make a Tic Tac Toe Neural Network Easily (LIVE)
Siraj Raval

Related AI Lessons

Want to get started with deep learning
Get started with deep learning by leveraging resources like Andrew Karpathy's playlist and frameworks such as TensorFlow or PyTorch
Reddit r/deeplearning
Building a Deepfake Detector From Scratch — What Nobody Tells You
Learn to build a deepfake detector from scratch and understand the challenges involved in detecting AI-generated fake media
Medium · Deep Learning
Unfolding the Meandering Path: High-Dimensional Invariance and the Flat 2D Plane of Neural…
Learn about high-dimensional invariance and its relation to the flat 2D plane of neural networks, and how to apply these concepts to improve model performance
Medium · Deep Learning
Implementing Neural Style Transfer from Scratch: The Project That Started It All
Learn to implement Neural Style Transfer from scratch and understand its significance in deep learning
Medium · Deep Learning
Up next
Image Classification with ml5.js
The Coding Train
Watch →