TensorFlow Full Course 2026 | TensorFlow Tutorial for Beginners | TensorFlow Course| Simplilearn

Key Takeaways

Hey there, welcome to TensorFlow full course by SimplyLearn. You're about to learn something really cool. So, what's TensorFlow? Think of it as a super helpful tool that Google created to make building smart computer programs way easier. You know how you can teach a child to recognize cats in pictures by showing them lots of cats photos? TensorFlow helps you teach computers to do the same thing. Recognize pattern, make predictions, and even make smart decisions based on what they have learned. Here's what we are going to do together. First, we'll make sure that you're comfortable with Python. Then, we'll help you get TensorFlow up running in your computer. From there, we'll explore how computers actually learn, covering some important math concepts like probability and statistic. but we'll keep it simple and practical. You'll also get to build something called convolutional neural networks. We will compare different tools like PyTorch, TensorFlow, and KAS so you know which one to use when. Plus, you'll get hands-on experience with object detection, teaching computers to spot things in photos and videos. And the best part, we'll wrap up with five real Python projects that you can actually build and show off. Ready to dive in and see what AI buzz is about? Let's get >> what is deep learning? Deep learning is a subset of machine learning which itself is a branch of artificial intelligence. Unlike traditional machine learning models which require manual feature extraction, deep learning models automatically discovers representation from raw data. So this is made possible through neural networks particularly deep neural networks which consist of multiple layers of interconnected nodes. So these neural network are inspired by the structure and the function of human brain. Each layer in the network transform the input data into more abstract and composite representation. For instance, in image recognition, the initial layer might detect simple features like edges and textures while the deeper layer recognizes more complex structure like shapes and objects. So one of the key advantage of deep learning is its ability to handle large amount of unstructured data such as images, audios and text making it extremely powerful for various application. So stay tuned as we delve deeper into how these neural networks are trained, the types of deep learning models and some exciting application that are shaping our future. Types of deep learning. Deep learning AI can be applied supervised, unsupervised and reinforcemental machine learning using various methods for each. The first one supervised machine learning. In supervised learning, the neural network learns to make prediction or classify that data using label data sets. Both input features and target variables are provided and the network learns by minimizing the error between its prediction and the actual targets. A process called back propagation. CNN and RNN are the common deep learning algorithms used for tasks like image classification, sentiment analysis and language translation. The second one, unsupervised machine learning. In unsupervised machine learning, the neural network discovers patterns or cluster in unlabelled data sets without target variables. It identifies hidden pattern or relationship within the data. Algorithms like autoenccoders and generative models are used for tasks such as clustering, dimensionality reduction and anomaly detection. The third one, reinforcement machine learning. In this, an agent learns to make decision in an environment to maximize a reward signal. The agent takes action, observes the records and learns policies to maximize cumulative rewards over time. Deep reinforcement learning algorithms like deep networks and deep deterministic polygradient are used for tasks such as robotics and game play. Moving forward, let's see what are the artificial neural networks. Artificial neural networks inspired by the structure and the function of human neurons consist of interconnected layers of artificial neurals or units. The input layer receives data from the external resources and it passes to one or more hidden layers. Each neuron in these layers computes a weighted sum of inputs and transfer the result to the next layer. During training, the weight of these connection are adjusted to optimize the network's performance. A fully connected artificial neural network includes an input layer or more hidden layers and an output layer. Each neuron in a hidden layer receives input from the previous layer and sends its output to the next layer. So this process continues until the final output layer produce the network response. So moving forward let's see types of neural networks. So deep learning models can automatically learn feature from data making them ideal to task like image recognition, speech recognition and natural language processing. So the most common architecture in deep learnings are the first one feed forward neural network FN. So these are the simplest type of neural network where information flows linearly from the input to the output. They are widely used for tasks such as image classification, speech recognition and natural language processing NLP. The second one convolutional neural network designed specifically for image and video recognition. CNN's automatically learn feature from images making them ideal for image classification, object detection and image segmentation. The third one recurrent neural networks RNN are specialized for processing sequential data time series and natural language. They maintain an internal state to capture information from previous input making them suitable for task such as speech recognition, NLP and language transition. So now let's move forward and see some deep learning application. The first one is autonomous vehicle. Deep learning is changing the development of self-driving car. Algorithms like CNN's process data from sensors and cameras to detect object, recognize traffic signs and make driving decision in real time, enhancing safety and efficiency on the road. The second one is healthcare diagnostic. Deep learning models are being used to analyze medical images such as X-rays, MRIs and CT scans with high accuracy. They help in early detection and diagnosis of diseases like cancer, improving treatment outcomes and saving lives. The third one is NLP. Recent advancement in NLP powered by deep learning models like transformers, chat GPT have led to more sophisticated and humanlike text generation, translation and sentiment analysis. So application include virtual assistant, chat bots and automated customer service. The fourth one defect technology. So deep learning techniques are used to create highly realistic synthetic media known as defects. While this technology has entertainment and creative application, it also raises ethical concern regarding misinformation and digital manipulation. The fifth one, predictive maintenance in industries like manufacturing and aviation. Deep learning models predict equipment failures before they occur by analyzing sensor data. The proactive approach reduces downtime, lowers maintenance cost, and improves operational efficiency. So now let's move forward and see some advantages and disadvantages of deep learning. So first one is high computational requirements. So deep learning requires significant data and computational resources for training. Whereas advantage is high accuracy achieves a state-of-the-art performance in tasks like image recognition and natural language processing. Whereas deep learning needs large label data sets often require extensive label data set for training which can be costly and time consuming together. So second advantage of deep learning is automated feature engineering automatically discovers and learn relevant features from data without manual intervention. The third disadvantage is overfitting. So deep learning can overfit to training data leading to poor performance on new unseen data. Whereas the third deep learning advantage is scalability. So deep learning can handle large complex data set and learn from massive amount of data. So in conclusion, deep learning is a transformative leap in AI mimicking human neural networks. It has changed healthcare, finance, autonomous vehicles and NLP. >> What is Python? Python is a high-level object-oriented programming language developed by Guido Van Rosum in 1989 and was first released in 1991. Python is often called a batteries included language due to its comprehensive standard library. A fun fact about Python is that the name Python was actually taken from the popular BBC comedy show of that time Montipython's Flying Circus. Now let's look at the top features of Python first. So Python has a simple structure and a clearly defined syntax. This allows the learners to pick up the language quickly. So it is easy to learn and use. Python can run on different operating systems such as Windows, Linux and Mac, making it a portable language. It enables programmers to develop the software for several competing platforms by writing a program only once. Third, Python is freely available at the official website. Since it is open source, this means that source code is also available to the public. Now, Python uses an object-oriented approach that encapsulates code within objects. Python provides a collection of libraries for various tasks such as machine learning, web development, and data analysis. And finally, in Python, you don't need to assign the data type of the variable. when you assign some value to the variable, it automatically allocates the memory to the variable at runtime. Now with that, let's move on to the uses of Python programming. So, Python programming language is used to develop desktop applications and build web applications too. It is popularly used in the field of data science, machine learning and artificial intelligence to analyze data, build predictive models and make business decisions. Python is also widely used in game development. Now let's see some of the popular Python frameworks and libraries. Python can be used for web development using frameworks like Zango, Flask, Pyramid and Churi. Now you can build graphical user interfaces using libraries and frameworks such as Tkinter or JustKE. You can also use PI GTK, PIQT or PYJS or Python JavaScript. Now, Python is also used to perform machine learning tasks using libraries such as TensorFlow, PyTorch, Scikitlearn, Mattplot Lib, and Scypi. You can also perform mathematical computations using numpy and pandas. Now, let's look at the best ids that you can use to write programs in Python and perform specific tasks. So, we have Jupyter notebook, which is part of the Anaconda distribution that is widely used these days. Even for our demo in this video, we'll be using Jupyter Notebook. I'll show you in a while. Then we have the visual code editor from Microsoft. This is also one of the preferred IDEs by learners and companies. Then we also have the popular text editor called Sublime Text Editor. Then we also have PyCharm followed by Python Ideally and Spider as our top ides. Now let's look at the top companies that are using Python in our day-to-day work. So we have Google, Kora, Facebook, even Netflix, Spotify, and Instagram. Now there are other top product based service- based and startups that also use Python programming. Although deep learning is uh been around for a while, it is just in its infant stages of development as far as exploding on the market. I mean it is right now they're building robots with it. Deep learning is used to train robots to perform human tasks. Music composition. Deep neural nets can be used to produce music by making computers learn the patterns involved in composing music. Image colorization. Neural network recognizes objects and uses information from the images to color them. Machine translation. Given a word, phrase, or a sentence in one language, neural networks automatically translate them into another language. Google Translate is one such popular machine translator. you may have come across. And you'll notice in here we didn't show any examples of straight numbers like uh projective sales in a business tracking your favorite stock. You can certainly do those with machine languages, but this is the next level. Uh save that for your regression models, your linear regression where you're actually processing and crunching just straight numbers. With machine learning and deep learning, we're going to a whole new level as far as what we can figure out on the computer. What's in it for you? We're going to cover what is deep learning. We're going to take a look at the biological versus artificial intelligence. What is neural network activation function in your neural network and the cost function and how do neural networks work. How do neural networks learn? So there's a little you'll see a switch right there. We just went from how are they working in the math in the background to exactly how are they learning. We'll be implementing the neural network. We'll do gradient descent deep learning platforms and we'll give an introduction to TensorFlow and implementation in TensorFlow. That's Google's platform that they open sourced recently and it's probably one of the most cutting edges in deep learning and even it is still in the infant stage which is one of the reasons they released it to open source. What is deep learning? Deep learning is a sub field of machine learning that deals with algorithms inspired by the structure and function of the brain. And you can see we have a nice picture here. We have artificial intelligence which is kind of the big bubble that encompasses all these different things we're talking about. This is ability of machine to imitate intelligent human behavior. And in there we have machine learning application of AI that allows a system to automatically learn and improve from experience. And if you looked at any of our other videos, you'll know that machine learning covers a lot. So deep learning is a subcategory of that. But don't forget machine learning has all kinds of other tools that people use to do very basic uh descriptive and predictive and postcriptive uh analytics. And then you have deep learning application of machine learning that uses complex algorithms and deep neural nets to train a model. Let's take a look at the biological neuron versus the artificial neuron. Now remember in the human brain and and this is true for most animals. There are a lot of different neurons going on. So this is the very basic one. I mean there's hundreds of different cells involved. So when we talk about neural networks, this is why I say it's in a very infant stage. They're really basing it on uh just the most basic thing that we're able to figure out going on in the neural networks. And you can see right here we have dendrites fetch information from an adjacent neurons and pass them on as inputs. So you have your data coming in and your data going out. Any computer model should be looking at that what's coming in what's going out. The data is fed as an input to the neuron. So we look at the artificial neuron. You can see we have our inputs. They come in each one is specially weighted into the neuron and then the neuron has an output. The cell nucleus processes the information received from the dendrites and the neuron processes the information provided as inputs. Axons are the cables over which the information is transmitted and the information is transferred over weighted channels. So you can look at that uh I mentioned weights briefly but you alter the data coming in. So those weights are what causes different information coming in to be weighted differently and processed differently. And the synapses receive the information from the axons and transmit it to the adjacent neurons that's in your biological model. And then when we look at the artificial neuron, the output is a final value predicted by the artificial neuron. So as we dig deeper into looking at the theory behind the neural network and we kind of flip back and forth between these because there's two huge aspects of it. One is from the outside. What are you seeing and what's going on from the inside so you can find to do what you need to do and give the best results you can. And we start off with what do we feed? We feed an unlabeled image to a machine which identifies it without any human intervention. And so you can see here we have a circle that comes in at 784 pixels and it comes in by 28x 28. And you can see how it colors in the um the circle on there. And we put a triangle in. And the triangle in also comes in as 28x 28 and it has 784 pixels. So you'll see between these two both of them are 784 pixels. This machine is intelligent enough to differentiate between the various shapes. So that's what we want to use our neural network to do is to say hey this is a circle. This is a triangle. That's more of a categorical. You can also do a regression model where you're actually putting out float value or a numerical value. We'll be looking at the true false or the categorical model mostly because that's where you usually start at the different there is no real difference when you as far as the way the internal functioning goes when you start flipping between them other than well we'll talk about that in just a minute. So you can actually go between the two quite easily and the neural network provides this capability. So we're going to use this capability to look between those two. One of the things I want you to note in here is that we're looking at 784 pixels. We're looking at 784 inputs. That's very different than stock with a high low or last year's sales based on date where we're looking at just a couple of numbers and they're very clear. They're numbers. They're very clear what they are, which is something you'd put into a machine learning linear regression model. This is a step up from that in that we're looking at complex patterns and how do you figure those complex patterns out. So, a neural network is a system modeled on the human brain and we looked at that comparing the two. Let's go ahead and look deeper into the neural network itself. We have our inputs coming in. So the inputs are fed to a neuron that processes a data and gives us an output. Input and output. This is the most basic structure of a neural network known as a perceptron. So if you see the term perceptron, that's what we're talking about. We're talking about this single node that has inputs and an output. However, neural networks are usually much more complex. Let's start with visualizing a neural network as a black box. And I always love that symbol. It's a black box. It's kind of magical. We have our inputs coming in and we want certain outputs. The box takes inputs, processes them and gives an output. Let's have a look at what happens within this box. And you can see me there in my uh secret agent get up and I got my hidden hood and everything. I guess I'm part of the uh black skull or something like that group. Uh so let's take a look at what happens within this magic box. And remember, we're skipping back and forth between the theory of what's going on in the box, which you have to know how to fine-tune and how to build, versus looking at it from the outside. We're programming this box, and we have an input and an output to the box as a whole. Within the box exists a network that is a core of deep learning. And you can see here we're showing one layer and we have our grid coming in. The network consists of layers of neurons. Each neuron is associated with a number called the bias. And you can think of the bias uh if you overly simplify this and we're doing a linear regression model. This is your y intercept in your uklidian geometry. You have to have something that offsets it. And so you always have a bias in these cells. Neurons of each layer transmit information to neurons of the next layer over channels. And so you can see each of our layers going through from left to right. These channels are associated with numbers called weights. These weights along with the biases determine the information that is passed over from the neuron to neuron. So just like the bias is your y intercept in uklitian geometry. You could look at the an one weight. Remember this is very complicated. So we're not looking at just one weight. You could look at the weight as your slope of the line. Or if you're doing x= uh my + c, it would be the m value. Neurons of each layer transmit information to neurons of the next layer. And you can see here as they light up going across into the final layer. and then to the output. And in this case, the output is going to be either uh a square in this one, or it might light up the other one, which is a circle. The output layer emits a predicted output. So in this case, we're looking at a classification. Uh true, false. Is it a circle? Is it a triangle? Is it a square? Let's now go deeper. What happens within the neuron? So we're going to dig deeper and start getting a little bit closer to some of the math. Don't worry, you don't have to be a calculus expert and know your differential equations. Even though this is one giant differential equation, you don't need to understand those to understand what's going on. Within each neuron, the following operations are performed. The product of each input and the weight of the channel it's passed over is found. This is simply addition. We're going to sum up the weight times the output from the previous channel and plus a bias. Sum of the weighted products is computed. This is called the weighted sum. Bias unique to the neuron is added to the weighted sum. The final sum is then subjected to the particular function and we'll discuss those that particular function. That part is really important because those functions uh have a huge impact on how well your model performs under different conditions. The final sum is then subject to a particular function. This is the activation function. So if you ever hear the term activation function, that's what we're talking about. What activates this cell and what doesn't. As we dig deeper into activation function, an activation function takes the weighted sum of the input as its input, adds a bias, and provides an output. And a lot of times you'll actually see one formula for the sum of the weight, the weighted sum and the bias. You'll just see that as a single line of everything added together. And here we've broken it apart because it makes it clear that this bias is not computed the same as the weighted sums. Here are the most popular types of activation function. And I always find these interesting because at one point I was sitting at a table with a gentleman who was finishing his PhD. He was in his last year and he said he went through all this stuff and he ended up just trying the four different activation functions on this particular problem he was working on. So knowing the math behind it doesn't necessarily mean you're going to know it right away. Uh so even somebody who might have a PhD and be doing the calculations on this comes back out of it and ends up just trying the different uh um activation functions to see what's going to make a difference. And a lot of times that's a final step. That's the kind of thing where you built your whole model. You've come back and you're like wait a minute can I do a better deal with a sigmoid function or the threshold or the rectifier? Knowing what they're doing is important so you can explain it to somebody else. And again you probably do this on a small set of data. If you're working with big data, uh you don't want to take down the full server farm just to test out your three different series. You take a small portion of that data, test it, and then you put it through to the big data. So let's take a look at this. We have the sigmoid function, and it's used for models where we have to predict the probability as an output. It exists between zero and one. And you'll see that's true of all of our activation functions we're working with. Either the cells on or off, it's true or false. And there might be a little variation in there which as an output could be used to compute uncertainty in your solution. So if you're getting a 7 with this activation function, it might be well I'm not sure if that's really a square or I'm not sure that's really a triangle. And that might be a flag for it to be looked at by a human observer at least in today's models where we're at right now. And you can see here where we have the formula is simply equals 1 over 1 + e the minus x where x is your value coming in. and it's going to give you a result that looks very similar to the graph on there which is somewhere between zero and one. Um, and right in the middle you can see that there's a huge uh kind of you can go through all the different values and uncertainties involved. So the sigmoid function is probably the default on most of them. Uh the next one is the threshold function. It is a thresholdbased activation function. If x value is greater than a certain value, the function is activated and fired. Else not. Pretty straightforward. Yes, no, true, false. um I don't want to test for improbabilities. I just want a straight answer. I don't want to know if there's a partial value on there. It either is true or it's false. And the rectifier function, it is the most widely used activation function. I would debate that. Um rectifier is pretty common one although I see that the sigmoid function is used to be the basic one, but it's up there. The rectifier function is very commonly used. It gives the output of X if X is positive and zero otherwise. And you can see here again just like um uh it's either you kind of get a value going up there. So max of x of zero. So it's it's again it's like the threshold function. Yes, no, true, false. Uh it's either zero or it's uh some kind of progressive value. And then we have the rectifier function. I would argue with this because the sigmoid function used to be the most common one. But with the rectifier function, uh, it now says it is the most commonly used or widely used activation function and gives an output of X if X is positive and zero otherwise. This is kind of nice because it now says absolutely not or it gives you a value of probability. Now, when I say a value of probability, be very careful there. I'm not saying that it's going to tell you this is 75% chance of being a circle. I'm going to tell you that it says, hey, if this says 0.1, it probably needs to be looked at or 2 or 3. It's going to depend on your data as to what that value means. In general, that just means it's flagging it that if it's not a one, then chances are it needs to be looked at by a person and re-evaluated. And there's a hyperbolic tangent function. This function is similar to sigmoid function is bound to a range of minus1 to 1. So you can see there's our 1 - eus 2x and 1 plus over 1 + eus 2x. Again, it's very similar to the sigmoid function. The bonus of the hyperbolic function is you have that variable coming through the middle. So again, you can look at it and you have a little bit more weight as far as you can process that down the line. That's a little bit more advanced than than what we're looking at right now. And a lot of times it's not even necessary in a lot of our different uh uses for these activation functions. Now, we looked at activation functions and I kind of said those are a little bit like a black box because even if you know all the math, a lot of times you end up just playing with them to find out what works. And it also depends on what model you're working with, whether you need a flat yes, no, true, false, or you need to have something in the middle that says, hey, this isn't quite a one. You might need to process this with the human intervention. And you can look at that. Uh, one example would be self-driving cars. You don't want a car to be yes, no, I'm going to go through the the light. You want it to be like, okay, if it's uh almost yes, maybe we stop and have human intervention so we don't get an accident. Cost function is something you can really see and measure and is very important. The cost value is the difference between the neural net's predicted output and the actual output from a set of labeled training data. So we have our group of data that's a square circle and since we're looking at geometrical shapes, we've had somebody already labeled that data. They've already said this is a triangle, this is a square. And so if this is coming up and it's giving us and it's saying a square is a triangle and it's saying a triangle is a circle, the output is wrong. And so that output can then be measured in the versus the actual output and that's the cost. Uh you might also hear this as error because that's the error value being returned. How far off is it? And what we're looking for is the least cost or the least error value. And it's obtained by making adjustments to the weights and biases iteratively throughout the training process. And this this is called back propagation. And we're going to look in that a little deeper as we look into an example. It's really hard to see when you're just looking at arrows without actual numbers and where that flow is coming from. But you can look at this is here's our inputs. They put out a prediction. The prediction comes out and says, "Hey, we've already labeled this data cuz we're in training mode and the training data is off. This is the cost. Can we send that error or that cost back and adjust those weights?" And we do it in very small increments across large amounts of data so that those weights minimize that cost or that error. But what happens within these neurons? So let's look at a little example of this. Kind of helps if you have some kind of visual. Let's build a neural network that predict bike prices based on a few of its features. And we'll see here we have our CC, our mileage, and our ABS. And these are our three input layers. And then we have the bike price and the output layer. Now, it doesn't do us very good to just uh pump it in from the beginning and pump it out. And to be honest, I would use a machine learning linear regression model on this since these are just straight numbers. But because we want a simple example, we're going to put this through and show you as a neural network what that looks like. And we got to put a hidden layer in there. The hidden layer helps in improving the output accuracy. And you could look at this as a bunch of ores. So it might say, hey, when we compare these three values on the first hidden layer neuron, we're looking at one set of features. And then we might weight them in the second one. So these are a bunch of different ores kind of how the math comes out in behind the scenes. And then they go out of course to the bike or the output layer. And each of the connections have a weight assigned with it. And you'll see here we have a mileage CC with the weight one and weight two going into our first neuron. And you'd also have your ABS going in there. And so X1 * weight 1 + X2 * weight 2 plus the bias of one. And step two is our activation. The activation function coming in there. When does this fire? And the neuron takes a subset of the inputs and processes it. And then we go through and we do that with the um second hidden layer neuron and the third one and so on. So you process each layer in order going forward. Now when I told you this is in its infant stage, they now have neurons that fire into the same layer or back a layer so that you now have a time series. And there's all kinds of wild things that they're experimenting with on these layers. This basic setup has been around since the mid90s. It's only now because of our technology that it's open to almost everybody to play with it. And that's why I say this is in an infant stage in development is this basic math is here, but what we can do with it is amazing. And what they're actually doing with all these different things is amazing. And so we're just at the beginning of how to use all these different tools and our deep learning and our neural networks. Uh and so once we have our hidden layer computed, the information reaching the neurons in the hidden layer is subjected to the respective activation function. And so each one of these fires an activation output uh and then those are each weighted to the final output layer. So the processed information is now sent to the output layer once again over weighted channels. And you could look at this as each one of these is um I always look at this as like a group of people. They're all looking at the bulletin board and the first person says this is what I project sales for the company and the second person and the third and so on. And then their perspectives are weighted based on their expertise. So your accountant might have a very high weight where the um maybe your janitor has a very low weight because their expertise is not in accounting and then that goes into the output layer and once in the output layer it goes uh the output which is the predicted value is compared against the original value. So now we have our output layer and since we have like already a list of uh bikes with their the different setups and what their value is we can now generate an error from this. The cost function determines the error in prediction and reports it back to the neural network. So this is the cost. This is how far off it is. This is your error coming back. And as you can see, this is back propagation going on. So now our error is going in reverse because we know we're not completely correct on this particular channel. The weights are adjusted in order to reduce the error. So each time we go back, we are changing those weights to reduce that error. and we change them in small increments. You don't want to fit one input. Remember, you might have a data pool with a terabyte of data. You don't want to solve for the first set of data that comes in and that be the main solution because everything else will be off. This is going to confuse you. That's also called a bias. So, we have the bias in the cell where we're adding a value, the kind of like the y intercept, and we have a bias of the whole neural network, which means that it's weighted towards one set of answers. So we want to make small changes in these weights so we don't create a bias and the weights are adjusted in order to reduce the error or the cost. The network is now trained using the new weights. Once again the cost is determined and back propagation is continued until the cost cannot be reduced any further. So let's go ahead and plug in values and see how our neural network works. So here we come in here and initially our channels are assigned with random weights. This is important because if you assign them all with the same weight, you might be able to reproduce it. But it turns out that if I put all my weights as one or all my weights as zero, it takes longer to train where if you have random weights, they already have like a little bit of adjustment and ores built in and that will give us a better answer and train faster. Our first neuron takes a value of mileage and CC as inputs. So here comes our computation, whatever those inputs are. And we do that again with the second neuron with those values coming in. You can see here we have weight three and so on and then our third neuron coming down and of course our fourth neuron. So we're adding all these different values coming in here in our hidden layer. The process value from each neuron is sent to the output layer over weighted channels. So again here's our weights coming in and we have N1, N2, N3 and N4. Once again the values are subjected to the activation function and a single value is emitted as the output. On comparing the predicted value to the actual value, we clearly see that our network requires training. So, here we have it that our bike price uh we put out, we thought it was worth 2,000 on our random weights and the bike actually was $4,000 on there. I'm guessing that's not US dollars because that'd be a very expensive bike, but maybe it is. There's some $2,000 $4,000 bikes out there. The cost function is calculated and back propagation takes place. And this is pretty simple. You can look at that as our um we're subtracting one value from the other. We square it and then we take half of that and that is propagated back up. And each layer generates its own errors. Let's go back one because you have your predicted Y and your actual Y. That goes back to the first layer. And then based on the value of the cost function, certain weights are changed. So when we look at the next layer, that error is not the original 4,000 - 2,000 squar / 2. This error is based on the error of each cell generated. How far off is that cell as far as its weights. We're not going to show you. It's actually a very complicated differential equation. And you can probably write it out if you wanted to. You just write out each formula that goes into the next level and you add them all together and you can write it out all the way through. Computers make it so you don't have to. And our neural network is considered trained when the value for the cost function is minimum. So when we get our error way down as low as we can, that's when our neural network is trained. And there I mean just recently they've come up with all kinds of different means for measuring that particular value. a little bit beyond the scope of today's neural network, but you can actually you can actually see it. You know, how far do you do this until the neural network doesn't need to be trained anymore and you can overtrain a neural network. Now, the tools that we're looking at automatically let you know when to stop, which is really nice. And that is just like I said, we're at the beginning stages in neural networks and it's just really cool what they can do now and how much of it's automated and how much of it is experimental. Right now, let's take a look at gradient descent. But what approach do we take to minimize the cost function? So here we have nice error thing coming in. This is our cost or our error. Uh let's start with plotting the cost function against the predicted value. And so you can see they fed in multiple y's and these are the errors coming in and the cost of each of these inputs and changes going on. Note we start at a random point on the curve. So usually you put in you know you pick up your data and you randomly pick where to start in your data. A lot of times you just run it from the beginning because you're going through so much data it's not that big of a deal. But you start with one point going in. So your forward propagation goes through. You're going to go ahead and find your cost or your error. It points that on the curve. And you can see how we're plotting it right here. Since the gradient at this point is positive, we may move right. So we're going to move a little bit to the right on here. And this time the gradient is negative. We move a little bit to the left. Eventually we try out the point where the gradient is zero. This is a least value of cost function. You have to be a little careful with this because this particular I mean they make it look nice and simple in this graph. Sometimes these curves look like stair steps and so there is global minimums and then there is local there might be a local point where the gradient is zero but it's not the global one. Uh so it might be way off to the left where it just happens to step down a little bit and you think you're in the right gradient. And with that we have all the right weights and we can say our network is trained. So here we have um just some major these are some of the big names out there right now in development for deep learning platforms. TensorFlow which we'll actually do an example in in a minute. Deep learning for J which is in the Java platform. Uh so if you're a Java programmer uh by the way as TensorFlow is accessed most people are using Python to access it but it is a system that's kind of separate from a lot of the programming languages which makes it a lot more um flexible as far as use. Deep learning forj is Java based and then cross is just exploding right now. And this is interesting. Cross is uh working with TensorFlow. It actually can sit on top of TensorFlow and it can also do its own thing. Uh so if you're studying deep learning or getting into it, you want to know the basics of TensorFlow, but you also are going to want to know the upper level of KAS sitting on top of TensorFlow. We're just looking at TensorFlow today though in our example. And there's also Torch on there. There's a bunch more that we didn't list on here. Um even sklearn or the uh side package in Python has a neural network you can program a very basic one and it is the same basic one that you could do in TensorFlow if you stripped everything out of it and then TensorFlow has a lot of tools they've added in and so has KAS but we're going to be looking specifically at TensorFlow in our example and TensorFlow is an open- source tool used to define and run computations on what they call tensors very common language now so more and more we see the term Tensor as being a standard in the uh deep learning language and this was originally developed by Google. So let's dig a little bit big in there. What are tensors? Tensors are just another name for arrays. So a tensor of dimension five. You can see here we have ab kmq whatever. So it's an array coming in. And the tensor of dimension 54 more like a picture. Very common to see that in a picture. You can also see a tensor even more detailed than a picture as we go to the next one. Tensor of dimension 333. This is 3D space. You might have a picture that also has colors. That might be the third dimension. You might have four dimensions because you have both your grid and your different color channels and your zplot. You can see where you can now process a very highlevel set of data coming in whether as an image or features. They could be features that have nothing to do with images. So there's a lot of stuff you can do now with the tensors coming in. Thus is where the term tensorflow comes from. So we have um right now the TensorFlow is the most popular library in deep learning and I did mention KAS now works with TensorFlow. So there's a lot of stuff you can do between the two. Uh it's an open-source software library developed by Google. Uh so they hit a roadblock and they realized hey this is an infant stage technology. You know we thought it was going to be the next greatest thing and we were going to have a hold on it but it's really infant as far as how it's applied and what we can do with it. Let's open source it so everybody can work on it. uh let's take it to the next level. And that's really what open source does to a lot of these uh packages when they release them. And you can run on either a CPU or a GPU. So when we look at the details, if you have your graphic processing units, um what's nice about those is they run a lot faster. The downside is you have to play with them a little bit to get them up and running and it's a hardware upgrade. When we run it, I'll be running it in the CPU mode. I have played with it in my GPU on my personal computer. you know, it does increase the processing. Uh, but I did run into some version problems with my Python and stuff like that. And when I did finally work it out, I went back to the CPU because it didn't increase my speed enough for what I was working on. But in a larger group, you might be able to put that on. If you're working with a larger stack of computers, you might want to run it in the GPU. You can create a data flow graphs that have nodes and edges. So there's our edges coming in. We didn't talk about edges, but that's very up and cominging way of looking at your analytical data is how do different nodes connect? What do those edges look like in between them? And it's used for machine learning applications such as neural networks. It is mostly a neural network, but they have all kinds of tools which sit on top of our basic neural network. They have new stuff evolving into the TensorFlow library. So, it's very much uh just exploding. great time to jump into TensorFlow because there's all kinds of cool things we're doing with it and all kinds of cool applications you can now use TensorFlow for. So let's take a look at implementation in TensorFlow and we're going to build a neural network to identify handwritten digits using the uh MNEST database or the MNIST database and that stands for modified National Institute of Standards and Technology database. It is a collection of 70,000 handwritten digits and the digit labels identify each of the digits from 0 to 9. This is a cool example because it's simple enough that you could actually run this through some basic machine learning categorizing algorithms and train them and you'll get about the same answer because again it's it's simple grid. The digits on the grid don't have a huge amount of variation like you would say an automated driving car looking at the environment. So you could still do this with a lot of your um different linear models and stuff like that. You can solve this and you'll get about the same answer. When I ran a comparison between TensorFlow and between some basic uh regression models or category models uh in machine learning, they came up pretty even as far as their output. Uh so this is kind of where we start to see the complexity of something coming in uh in this case a tensor you know or a grid of uh information where the deep learning model does as good as the regular models. And when you get past this kind of complexity and features suddenly the neural networks come up with better answers better solutions and a better build. And that's why there's such a move into neural networks is we live in a complicated world and it's just really cool we can do with this. So the handwritten digits from the um NIST database, they come in, the data set is used to train the machine, a new image of a digit is fed and the digit is identified. Um and if you've looked at any of our other machine learning tools where we're doing training uh where we train our uh model to fit and then you test it out, this should look pretty familiar. Uh and there is some tools out there for say untrained categorizing uh where it's just looking for features that fit together. So there are tools that don't need that training. But this is where uh we talk about neural networks, we do need to train them. And this is what we're looking at. So for this I'm going to use the Anaconda Navigator just because it's a very nice visual tool. You might be in PyCharm or one of your other IDEs for editing Python because we are looking at Python TensorFlow. And under Anaconda, we have the notebook which is something we use pretty regularly. And they have the Jupyter Lab. The Jupyter Lab is the Jupyter notebook, but with tabs and a few new features. So, we'll be using the Jupyter Lab today. And under the environment, you'll want to go ahead and and uh if you haven't yet, uh you'll see that I have a number of different setups in here. Right now I have the Python version 36 and the TensorFlow. In this case I have TensorFlow 1.12. If we scroll down you can see that uh here we go. TensorFlow and it's version 1.12. And in here if you haven't yet you'll need to install those. Go in and just open our terminal and u if you've never used the Anaconda or if you're in your other thing you might have something simple like pip. is what I use for my install. And you can simply do install TensorFlow. That should bring in the most current version. Now, when I installed this a few months ago, Python version, I'm not going to run this because I already have it installed on here. Python version 3.7, the newest one out, still had a couple glitches with the TensorFlow. I believe they've fixed it as of writing of this, but um I'm going to stick with 36 just so I don't get any surprises on there. So, this is Python version 3.6 6 with TensorFlow 1.12 on here. And if you haven't installed it yet, you also want to install Numpy for this example. That's Numbers Python or uh NUMP py. You can just simply run an install on there. Keep in mind if you're in Anaconda uh and you've created one of these environments specific to this, keep withd. If you're going to use pip, keep with pip. Don't install one package with pip and one under cond because that's how they track those version numbers and how they fit together and you can end up with a problem. They don't pip doesn't see and vice versa. Uh so just keep that in mind when you're running your installs. We'll go ahead and open up Jupyter Lab and we're going to launch that. So here's my Jupyter Lab. One of the really cool features of Jupyter Lab is you have tabs now. So you can open up multiple uh notebooks. This is nice cuz I have my notes I'm working on and then our actual window we're looking in. And we'll go ahead and zoom in a little bit here. There we go. So you have a nice u hopefully easy to see fonts. And then we'll go ahead and do a simple or get our imports out of the way. Um and so we're going to import our TensorFlow as TF. Uh that's pretty much a standard for TensorFlow. numpy our numbers python as py and we'll import our mattplot library as plt again these are very common so if you see tf or py or plt this is a standard that most people use do you have to no you could just do import numpy instead of doing as py and then from tensorflow examples this is always nice because they actually include data set we're going to play with so we're going